Re: [NMusers] 3Rs: Reproducibility, Replicability and Repeatability

From: Devin Pastoor <devin.pastoor_at_gmail.com>
Date: Mon, 01 Jun 2015 15:11:18 +0000

Dear Gavin,

My two cents are, in the simplest sense:

Reproducibility focuses on the ability to arrive to the same final
conclusions, even in the presence of some change, as the 'spirit' of the
definition should be about arriving to the same conclusions. (eg use of
monolix vs phoenix vs nonmem, or running a second trial)

Replicability: ability to recreate the "all" steps and conditions to get to
the same results (eg recreate figures/tables from a script, or same in
change in objective function value, etc)

Repeatability: Whether the same design can be used/achieved again.

I think the crux of the issue, is that intra-discipline these can have more
distinct definitions, however these are rarely synonymous.

One example of these types of semantic clashes is a computer science
professor I have spoken with always hated when people used 'real time' when
discussing things updating on the web (eg receiving streams of stock market
data), because he had worked for years designing nuclear reactors, where
real time was measured in fempto seconds, whereas you or I may consider
real time within a second or so. I believe the same semantic differences
present themselves be said whether discussing assay development, clinical
trial design, or modeling & simulation.

There could be long debates over small differences in opinion, but I think
the most reasonable goal is to land on what the 'spirit' of each focuses
on, and choose which term most aligns with your goals. (eg if data qual is
the issue, then replicability is important, but if whether a drug works
reproducing the outcome should be much more important than simply
replicating the same design.)

Devin

On Mon, Jun 1, 2015 at 12:39 PM Gavin Jarvis <gej1000_at_cam.ac.uk> wrote:

> Dear NONMEM Users
>
>
>
> Does anyone have a clear view on the semantic distinctions between the
> terms ‘reproducibility’, ‘replicability’,=
 and ‘repeatability’? There are,
> surely, different sources of data variation in experiments that might, on
> the face of it, be identical. However, I have the impression that these
> terms are being used more often than before, and in a manner that suggest=
s
> their meanings are transparent and unambiguous. Is there really a consens=
us
> out there about what specifically these terms refer to, and is that
> consensus the same in different scientific disciplines?
>
>
>
> Gavin
>
>
>
> __________________________________________________
>
> *Dr Gavin E Jarvis MA**(Cantab)** MA PhD VetMB MRCVS*
>
> University Lecturer in Veterinary Anatomy
>
> Department of Physiology, Development & Neuroscience
>
> Physiological Laboratory
>
> Downing Street
>
> Cambridge
>
> CB2 3EG
>
> Tel: +44 (0) 1223 333745
>
>
>
> Fellow and College Lecturer in Pharmacology
>
> Tutor for Graduate Students
>
> Selwyn College
>
> Cambridge
>
> CB3 9DQ
>
> Tel: +44 (0) 1223 761303
>
>
>
> Email: gej1000_at_cam.ac.uk
>
> Web: www.pdn.cam.ac.uk/staff/jarvis
>
> Twitter: _at_GavinEJarvis
>
>
>
>
>

Received on Mon Jun 01 2015 - 11:11:18 EDT

This archive was generated by hypermail 2.3.0 : Fri Sep 27 2019 - 16:44:28 EDT