From: Bob Leary <*Bob.Leary*>

Date: Thu, 15 May 2014 14:22:48 -0400

Hi Emmanuel,

While I am a strong advocate of using quasi-random rather than pseudo- rand=

om sequences for importance sampling in EM methods like IMP, there is a the=

oretical (and very real) problem with their use in the context you suggest=

ed in your message, namely with a multivariate t distribution as the import=

ance sampling distribution. The 3S2 option implies you are using a Sobol q=

uasi-random sequence, while

the DF=7 implies the use of a multivariate T-distribution with 7 degrees =

of freedom. The standard way of generating

a p-dimensional multivariate t -random variable with DF degrees of freedo=

m is to generate a p-dimensional multivariate normal and then divide by an =

additional independent random variable which is basically the square root =

of a 1-d chi square random variable with DF degrees of freedom. Thus to g=

enerate a p-dimensional importance sample, you actually need to use p+1 in=

dependent random variables. If you simply use a p+1 dimensional Sobol vec=

tor as the base quasi-random draw, the nonlinear mapping from p+1 dimension=

s to the final p dimensional result destroys the low discrepancy property =

of the final sequence in the p-dimensional space and in fact introduces a =

significant amount of bias in the final result. The problem arises directl=

y from the p+1 vs p dimensional mismatch.

There is no problem if the final p-dimensional result can be generated from=

a p-dimensional quasi-random sequence, which is the case for multivariate =

normal

Importance samples. So quasi random sequences should really only be used=

for the DF=0 multivariate normal importance sampling distribution case, =

not the multivariate DF>0 multivariate t case.

I ran across this effect in testing the Sobol-based importance sampling EM =

algorithm QRPEM in Phoenix NLME. It is very real and the net effect is to =

introduce a significant bias. There is a partial fix that works but gives=

up some of the benefit of using low-discrepancy sequences - namely use a =

p-dimensional quasi-random vector to generate the p-dimensional multivariat=

e normal, but

then use a 1-d pseudo-random sequence to generate the chi-square random var=

iable.

From: owner-nmusers

Behalf Of Emmanuel Chigutsa

Sent: Thursday, May 15, 2014 1:03 PM

To: Pavel Belo; nmusers

Subject: Re: [NMusers] SAEM and IMP

Hi Pavel

I have experienced a similar problem. In my case, the following code for IM=

P after SAEM (using NM7.3) greatly reduced the Monte Carlo OFV noise from v=

ariations of about +/- 60 points to variations of +/- 6 points (though stil=

l not good enough for covariate testing):

$EST METHOD=IMP LAPLACE INTER NITER=15 ISAMPLE=3000 EONLY=1 DF=7 =

IACCEPT=0.3

ISAMPEND=10000 STDOBJ=2 MAPITER=0 PRINT=1 SEED=123456 RANMETHOD=

=3S2

The settings are explained in the NM7.3 guide. If you are using NM7.3, you =

can also try IACCEPT=0.0 whereupon "NONMEM will determine the most approp=

riate IACCEPT level for each subject". Of course the settings for DF and IA=

CCEPT in the above code will depend on the type of data you have. Which bri=

ngs me to my own question. If I have both continous and categorical DVs in =

the dataset (which would mean different optimal settings) and I am using F_=

FLAG accordingly, what would the 'right' values of DF and IACCEPT be? I hav=

e noticed that the DF automatically chosen by NONMEM for individuals in the=

dataset can vary from 0-8 and this appears to be random. NOTICE: T=

he information contained in this electronic mail message is intended only f=

or the personal and confidential use of the designated recipient(s) na=

med above. This message may be an attorney-client communication, may be pro=

tected by the work product doctrine, and may be subject to a protectiv=

e order. As such, this message is privileged and confidential. If the =

reader of this message is not the intended recipient or an agent responsibl=

e for delivering it to the intended recipient, you are hereby notified=

that you have received this message in error and that any review, dis=

semination, distribution, or copying of this message is strictly prohibited=

. If you have received this communication in error, please notify us i=

mmediately by telephone and e-mail and destroy any and all copies of this=

message in your possession (whether hard copies or electronically sto=

red copies). Thank you. buSp9xeMeKEbrUze

Received on Thu May 15 2014 - 14:22:48 EDT

Date: Thu, 15 May 2014 14:22:48 -0400

Hi Emmanuel,

While I am a strong advocate of using quasi-random rather than pseudo- rand=

om sequences for importance sampling in EM methods like IMP, there is a the=

oretical (and very real) problem with their use in the context you suggest=

ed in your message, namely with a multivariate t distribution as the import=

ance sampling distribution. The 3S2 option implies you are using a Sobol q=

uasi-random sequence, while

the DF=7 implies the use of a multivariate T-distribution with 7 degrees =

of freedom. The standard way of generating

a p-dimensional multivariate t -random variable with DF degrees of freedo=

m is to generate a p-dimensional multivariate normal and then divide by an =

additional independent random variable which is basically the square root =

of a 1-d chi square random variable with DF degrees of freedom. Thus to g=

enerate a p-dimensional importance sample, you actually need to use p+1 in=

dependent random variables. If you simply use a p+1 dimensional Sobol vec=

tor as the base quasi-random draw, the nonlinear mapping from p+1 dimension=

s to the final p dimensional result destroys the low discrepancy property =

of the final sequence in the p-dimensional space and in fact introduces a =

significant amount of bias in the final result. The problem arises directl=

y from the p+1 vs p dimensional mismatch.

There is no problem if the final p-dimensional result can be generated from=

a p-dimensional quasi-random sequence, which is the case for multivariate =

normal

Importance samples. So quasi random sequences should really only be used=

for the DF=0 multivariate normal importance sampling distribution case, =

not the multivariate DF>0 multivariate t case.

I ran across this effect in testing the Sobol-based importance sampling EM =

algorithm QRPEM in Phoenix NLME. It is very real and the net effect is to =

introduce a significant bias. There is a partial fix that works but gives=

up some of the benefit of using low-discrepancy sequences - namely use a =

p-dimensional quasi-random vector to generate the p-dimensional multivariat=

e normal, but

then use a 1-d pseudo-random sequence to generate the chi-square random var=

iable.

From: owner-nmusers

Behalf Of Emmanuel Chigutsa

Sent: Thursday, May 15, 2014 1:03 PM

To: Pavel Belo; nmusers

Subject: Re: [NMusers] SAEM and IMP

Hi Pavel

I have experienced a similar problem. In my case, the following code for IM=

P after SAEM (using NM7.3) greatly reduced the Monte Carlo OFV noise from v=

ariations of about +/- 60 points to variations of +/- 6 points (though stil=

l not good enough for covariate testing):

$EST METHOD=IMP LAPLACE INTER NITER=15 ISAMPLE=3000 EONLY=1 DF=7 =

IACCEPT=0.3

ISAMPEND=10000 STDOBJ=2 MAPITER=0 PRINT=1 SEED=123456 RANMETHOD=

=3S2

The settings are explained in the NM7.3 guide. If you are using NM7.3, you =

can also try IACCEPT=0.0 whereupon "NONMEM will determine the most approp=

riate IACCEPT level for each subject". Of course the settings for DF and IA=

CCEPT in the above code will depend on the type of data you have. Which bri=

ngs me to my own question. If I have both continous and categorical DVs in =

the dataset (which would mean different optimal settings) and I am using F_=

FLAG accordingly, what would the 'right' values of DF and IACCEPT be? I hav=

e noticed that the DF automatically chosen by NONMEM for individuals in the=

dataset can vary from 0-8 and this appears to be random. NOTICE: T=

he information contained in this electronic mail message is intended only f=

or the personal and confidential use of the designated recipient(s) na=

med above. This message may be an attorney-client communication, may be pro=

tected by the work product doctrine, and may be subject to a protectiv=

e order. As such, this message is privileged and confidential. If the =

reader of this message is not the intended recipient or an agent responsibl=

e for delivering it to the intended recipient, you are hereby notified=

that you have received this message in error and that any review, dis=

semination, distribution, or copying of this message is strictly prohibited=

. If you have received this communication in error, please notify us i=

mmediately by telephone and e-mail and destroy any and all copies of this=

message in your possession (whether hard copies or electronically sto=

red copies). Thank you. buSp9xeMeKEbrUze

Received on Thu May 15 2014 - 14:22:48 EDT