RE: [NMusers] VPCs confidence intervals?

From: Ken Kowalski <kgkowalski58_at_gmail.com>
Date: Thu, 14 Mar 2019 14:01:23 -0400

Hi All,

I know what Bill is trying to say but it is not quite accurate the way he s=
tates it.

A prediction interval makes inference on a statistic based on a future samp=
le such as a sample mean of a future set of data. In contrast, a confidenc=
e interval makes inference on a parameter such as the population mean which=
is a fixed number. A prediction interval takes into account both the unce=
rtainty in the existing data used to estimate the population parameter as w=
ell as the sampling variation to make inference on a sample statistic (e.g.=
, sample mean for a future trial). A confidence interval only takes into =
account the uncertainty in the existing data used to estimate the parameter=
. Based on the Law of Large Numbers, the population mean can be thought =
of as taking the sample mean of an infinite sample size (i.e., sampling the=
entire population). For this reason, a prediction interval with an infini=
te sample size will collapse to a confidence interval.

An interval based on VPCs is more akin to a prediction interval since it ta=
kes into account the sampling variation based on a finite sample size, howe=
ver, one cannot assign a valid coverage probability (confidence level) to t=
his interval unless it also takes into account the parameter uncertainty. =
With VPCs applied to existing data (i.e, an internal VPC) it is customary t=
o not take into account this parameter uncertainty so many refer to such pr=
ediction intervals as degenerate as they place 100% certainty on the model =
parameter estimates used to obtain the VPC predictions. One could potent=
ially call these intervals â€˜degenerate prediction intervalsâ€=
™ but I tend to just call them â€˜VPC intervalsâ€™ (e.g., a 9=
0% VPC interval) so as to avoid misperception that these prediction interva=
ls have a statistically valid coverage probability. However, when VPCs are=
applied to an independent dataset not used in the development of the model=
, it is often advised to take into account the parameter uncertainty when p=
erforming the VPCs to essentially reflect the trial-to-trial uncertainty of=
the independent data not used in the estimation of model (i.e., refitting =
the same model to a new set of trial data will not give the same set of est=
imates and hence reflects trial-to-trial variation). In this setting, wher=
e the VPCs take into account both the parameter uncertainty and sampling va=
riation to predict on an independent (e.g., future) dataset, then one is on=
more solid ground to refer to these VPC intervals as prediction intervals =
with valid coverage probabilities.

Kind regards,

Ken

Kenneth G. Kowalski

Kowalski PMetrics Consulting, LLC

Email: <mailto:kgkowalski58_at_gmail.com> kgkowalski58_at_gmail.com

Cell: 248-207-5082

From: owner-nmusers_at_globomaxnm.com [mailto:owner-nmusers_at_globomaxnm.com] On=
Behalf Of Bill Denney
Sent: Thursday, March 14, 2019 1:10 PM
To: Soto, Elena <elena.soto_at_pfizer.com>; nmusers_at_globomaxnm.com
Subject: RE: [NMusers] VPCs confidence intervals?

Hi Elena,

VPCs are accurately called prediction intervals not confidence intervals. =
The difference is that a prediction interval shows what you would expect fo=
r the next individual in a study while a confidence interval shows what you=
would expect for the result of a statistic (often confidence intervals of =
a mean are shown). With many VPCs, the confidence interval of the median a=
nd the confidence interval of the 5th and 95th percentiles are shown.

Also, when the lines indicate the median, 5th, and 95th percentiles of the =
simulations, that is the 90% prediction interval since it is the middle 90%=
of the data (not the 95% confidence interval).

Thanks,

Bill

From: owner-nmusers_at_globomaxnm.com <mailto:owner-nmusers_at_globomaxnm.com> <=
owner-nmusers_at_globomaxnm.com <mailto:owner-nmusers_at_globomaxnm.com> > On Beh=
alf Of Soto, Elena
Sent: Thursday, March 14, 2019 12:49 PM
To: nmusers_at_globomaxnm.com
Subject: [NMusers] VPCs confidence intervals?

Dear all,

I have a question regarding visual predictive checks (VPCs).

Most of VPCs used now, include a line representing the median and 5th and 9=
5th percentiles of the data values and an area around the same percentiles =
that is commonly define as the 95% confidence interval (of the simulations)=
.

But is it correct, from the statistical point of view, to call confidence i=
nterval to this area? And if this is not the case how should we define them=
?

Thanks,

Elena Soto

Elena Soto, PhD

Pharmacometrician

Pharmacometrics, Global Clinical Pharmacology

Global Product Development

Pfizer R&D UK Limited, IPC 096

CT13 9NJ, Sandwich, UK

Phone : +44 1304 644883

_____

Unless expressly stated otherwise, this message is confidential and may be =
privileged. It is intended for the addressee(s) only. Access to this e-mai=
l by anyone else is unauthorised. If you are not an addressee, any disclosu=
re or copying of the contents of this e-mail or any action taken (or not ta=
ken) in reliance on it is unauthorised and may be unlawful. If you are not =
an addressee, please inform the sender immediately.

Pfizer R&D UK Limited is registered in England under No. 11439437 with its =
registered office at Ramsgate Road, Sandwich, Kent CT13 9NJ

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Received on Thu Mar 14 2019 - 14:01:23 EDT

This archive was generated by hypermail 2.3.0 : Fri Sep 27 2019 - 17:03:42 EDT