From: Ribbing, Jakob <*Jakob.Ribbing*>

Date: Mon, 11 Jul 2011 13:23:34 +0100

Resending and apologizing for any duplicate messages!

-----Original Message-----

From: Ribbing, Jakob

Sent: 11 July 2011 10:13

To: nmusers

Subject: RE: [NMusers] Confidence intervals of PsN bootstrap output

All,

This first part is more to clarify and I do not believe this is in

disagreement with what has been said before. The last paragraph is a

question.

The two examples I mentioned regarding boundary conditions are regarding

variance parameters. The second of these, however, is with regards to a

boundary at eta-correlation of one, which is a must rather than just an

irritating NONMEM feature.

I used these examples because they were less controversial and it is

difficult to come up with general statements that apply to all cases.

However, as a third example for a fixed-effects parameter: Imagine a

covariate acting in a linear fashion on a structural parameter that is

bound to be non-negative (e.g. a rate constant, volume, clearance, ED50,

etc). Imagine boundaries on the covariate parameter have been set to

avoid negative values for the structural-model parameter (on the

individual level). For this scenario if a substantial fraction of the

bootstrapped covariate-parameter values end up at one of the boundaries,

one may have to consider two options:

a) Decide that a linear covariate model is inappropriate (at least for

the goal of extrapolating to the whole population with more extreme

covariate values) and change the model into using a different functional

form

b) Dismiss this as random chance, due to small sample/limited

information and a (covariate) slope which "truly" is not far from one of

the boundaries. If this is the case, deleting the bootstrap estimates at

boundary would bias the distribution in an undesirable manner. For that

case the boundary condition is not due to local minimum and we would not

want to discard bootstrap samples at boundary). (Nick's example is of a

different kind, where it is either a local minimum or else not reaching

a minimum at all)

A related question - I am thinking more in terms of simulations with

parameter uncertainty; not just obtaining CI, which was originally what

this thread was about:

There are sometimes situations where a limited set of (clinical-) trial

data gives reasonable point estimates but with huge parameter

uncertainty (regardless nonmem covmaxtrix or bootstrap with appropriate

stratification). The distribution and CI on these parameters may include

unreasonable values, even though there is no obvious physiological

boundary (unreasonable based on prior knowledge that has not been

incorporated into the analysis, e.g. for a certain mechanism and patient

population Typical-Emax beyond 400% or 10 units - depending on if Emax

is parameterised as relative or absolute change). In these situations, a

simplistic option could be to trim one or both ends with regards to the

Emax distribution and discard these bootstrap samples, especially if

only a few values are unreasonable. Alternatively, before running the

bootstrap, one may set the boundary in the control stream (a boundary

that everyone can agree is unreasonable). One would then keep bootstrap

samples that ends up at this boundary for bootstrap distribution, which

is in a way truncated, but so that bootstrap samples indicating linear

concentration/dose-response maintains almost reasonable Emax and

ED50/EC50 values (but as a spike in the distribution at upper Emax).

Notice that re-parameterising the Emax model would not solve the

underlying issue with unreasonable estimates and reducing to a linear

model may be unsuitable, both based on the original dataset and also for

mechanistic reasons). Could you suggest alternative ways of dealing with

this, for these rather general examples (other than the obvious of

applying an informative prior on Emax)? I would be interested in your

solutions both in terms of the non-parametric bootstrap as well as the

parametric bootstrap (based on the nonmem covmatrix).

Much appreciated

Jakob

-----Original Message-----

From: owner-nmusers

On Behalf Of Nick Holford

Sent: 11 July 2011 06:37

To: nmusers

Subject: Re: [NMusers] Confidence intervals of PsN bootstrap output

Received on Mon Jul 11 2011 - 08:23:34 EDT

Date: Mon, 11 Jul 2011 13:23:34 +0100

Resending and apologizing for any duplicate messages!

-----Original Message-----

From: Ribbing, Jakob

Sent: 11 July 2011 10:13

To: nmusers

Subject: RE: [NMusers] Confidence intervals of PsN bootstrap output

All,

This first part is more to clarify and I do not believe this is in

disagreement with what has been said before. The last paragraph is a

question.

The two examples I mentioned regarding boundary conditions are regarding

variance parameters. The second of these, however, is with regards to a

boundary at eta-correlation of one, which is a must rather than just an

irritating NONMEM feature.

I used these examples because they were less controversial and it is

difficult to come up with general statements that apply to all cases.

However, as a third example for a fixed-effects parameter: Imagine a

covariate acting in a linear fashion on a structural parameter that is

bound to be non-negative (e.g. a rate constant, volume, clearance, ED50,

etc). Imagine boundaries on the covariate parameter have been set to

avoid negative values for the structural-model parameter (on the

individual level). For this scenario if a substantial fraction of the

bootstrapped covariate-parameter values end up at one of the boundaries,

one may have to consider two options:

a) Decide that a linear covariate model is inappropriate (at least for

the goal of extrapolating to the whole population with more extreme

covariate values) and change the model into using a different functional

form

b) Dismiss this as random chance, due to small sample/limited

information and a (covariate) slope which "truly" is not far from one of

the boundaries. If this is the case, deleting the bootstrap estimates at

boundary would bias the distribution in an undesirable manner. For that

case the boundary condition is not due to local minimum and we would not

want to discard bootstrap samples at boundary). (Nick's example is of a

different kind, where it is either a local minimum or else not reaching

a minimum at all)

A related question - I am thinking more in terms of simulations with

parameter uncertainty; not just obtaining CI, which was originally what

this thread was about:

There are sometimes situations where a limited set of (clinical-) trial

data gives reasonable point estimates but with huge parameter

uncertainty (regardless nonmem covmaxtrix or bootstrap with appropriate

stratification). The distribution and CI on these parameters may include

unreasonable values, even though there is no obvious physiological

boundary (unreasonable based on prior knowledge that has not been

incorporated into the analysis, e.g. for a certain mechanism and patient

population Typical-Emax beyond 400% or 10 units - depending on if Emax

is parameterised as relative or absolute change). In these situations, a

simplistic option could be to trim one or both ends with regards to the

Emax distribution and discard these bootstrap samples, especially if

only a few values are unreasonable. Alternatively, before running the

bootstrap, one may set the boundary in the control stream (a boundary

that everyone can agree is unreasonable). One would then keep bootstrap

samples that ends up at this boundary for bootstrap distribution, which

is in a way truncated, but so that bootstrap samples indicating linear

concentration/dose-response maintains almost reasonable Emax and

ED50/EC50 values (but as a spike in the distribution at upper Emax).

Notice that re-parameterising the Emax model would not solve the

underlying issue with unreasonable estimates and reducing to a linear

model may be unsuitable, both based on the original dataset and also for

mechanistic reasons). Could you suggest alternative ways of dealing with

this, for these rather general examples (other than the obvious of

applying an informative prior on Emax)? I would be interested in your

solutions both in terms of the non-parametric bootstrap as well as the

parametric bootstrap (based on the nonmem covmatrix).

Much appreciated

Jakob

-----Original Message-----

From: owner-nmusers

On Behalf Of Nick Holford

Sent: 11 July 2011 06:37

To: nmusers

Subject: Re: [NMusers] Confidence intervals of PsN bootstrap output

Received on Mon Jul 11 2011 - 08:23:34 EDT