Re: [NMusers] Cross-validation script in NM

From: Alison Boeckmann <alisonboeckmann_at_fastmail.fm>
Date: Sat, 06 Dec 2014 07:24:11 -0800


Dear Pieter, I think there is a mistake in the control stream. (Thank
you for including it.) The two problems use two different data sets:
RawdataCFP_cov_ext.csv RawdataCFP_xval_ext.csv

Nothing is wrong with that. However, the $INPUT statements are
not the same.

For the first problem: $INPUT ID TIME DV CMT AMT RATE EVID MDV UVOL
EXTRA IND_DIA OCC DIALYSIS ANALYSIS BV MISSING AGE WGT HGT BMI BSA SOFA
M1F2 GFR XVAL For the second problem: $INPUT ID TIME DV CP CMT AMT RATE
EVID MDV UVOL EXTRA IND_DIA OCC DIALYSIS ANALYSIS BV MISSING AGE WGT HGT
BMI BSA SOFA M1F2 GFR TDM XVAL

Notice CP and TDM are listed in the second problem but not the first.

When the problems are part of the same NONMEM run, the $PK abbreviated
code is specified only once. For the first problem, the variables of
interest are obtained this way in FSUBS:

EVID=EVTREC(NVNT,007)

UVOL=EVTREC(NVNT,009)

IND_DIA=EVTREC(NVNT,011)

DIALYSIS=EVTREC(NVNT,013)

CMT=EVTREC(NVNT,004)

If I run the second problem by itself, the variables of interest are:

EVID=EVTREC(NVNT,008)

UVOL=EVTREC(NVNT,010)

IND_DIA=EVTREC(NVNT,012)

DIALYSIS=EVTREC(NVNT,014)

CMT=EVTREC(NVNT,005)

The presence of CP causes them to be shifted over. If you have two
separate runs, then the code in FSUBS can be different, and IND_DIA
and DIALYSIS etc. will be in the right places. But when there is only
one run, the variables of interest have the wrong values in the
second problem.

The use of $MSFO/$MSFI has nothing to do with it.

I suggest that you change the second $INPUT record to specify CP=DROP.
You probably don't need TDM=DROP in the second problem; NM-TRAN will
find XVAL during the data pre-processor step, and will IGNORE/ACCEPT the
appropriate records.

As general approach to complicated models and trouble shooting: there is
no point running $ESTIM till you are sure you've got the model right!
Instead, *each* problem should display items of interest $TABLE ID TIME
IND_DIA DIALYSIS TDM etc. Now you can make sure you are getting the
correct data items from the correct records.

Alison Boeckmann


On Fri, Dec 5, 2014, at 02:03 AM, Pieter Colin wrote:
> Dear Kajsa and Dennis

>

> Thank you for your thoughts on this.

> I know of (and have used several times in the past) the mentioned
> functionalities in PsN and PLT-tools.

> However, due to the specific nature of my problem, I’m afraid the=
se
> will not work for me.

>

> Allow me to further clarify my problem. (For clarity, I’ve includ=
ed a
> piece of my control stream at the bottom of this message.)

> As Dennis pointed out, I’m fitting a training group and use the f=
inal
> parameter estimates in a subsequent run to predict the
> plasmaconcentrations of the validation group.

> I failed to clarify this in my previous message, but I’m predicti=
ng
> the plasmaconcentrations for the validation group according to a TDM
> setting.

> This means that for the validation group MAXEVAL=0 and only the first
> through sample per ID is included in the dataset as an observation
> event(EVID=0 and MDV=0).

> It goes without saying that the objective is to accurately predict the
> other plasmaconcentrations (EVID=2 and MDV=1) for the IDs in the
> validation group.

>

> Now to get to the problem. I tried this approach with two separate
> control streams and it works.

> I.e. plasmaconcentrations are predicted for the validation group based
> on the post-hoc corrected final parameter estimates of the
> training group.

>

> However, when I combine these in a single control stream (as shown
> below) the time-varying covariates are not taken into account for the
> validation group.

> More specifically, the following statement (under $PK) is not
> evaluated for the IDs in the validation group (statement used to
> switch on/off an additional CL due to hemodialysis).

>

> CL_DIA = 0

> IF(DIALYSIS.EQ.1) CL_DIA = THETA(6)

>

> IND=0

> IF(IND_DIA.EQ.1) IND=1

>

> This causes the hemodialysis moments to be ignored by NM in the
> validation group when using the control stream as shown below.

> Since it worked for me using separate control streams, it seems that
> the problem is associated with the use of MSFO=… and $MSFI in t=
he
> training and validation set, respectively.

> Do any of you have a specific solution for this problem or could shed
> some light on specific behavior of the $MSFI option in NM which might
> be causing this?

>

> Kind regards,

>

> Pieter Colin

>

>

> $PROBLEM No covariates

> ;; 1. Based on:

> ;; COMMENT:

>

> ;------------------------------------------------------------------------=
---------

> ;----------------------- FIT XVAL
> --------------------------------------------

> ;------------------------------------------------------------------------=
---------

>

> $INPUT ID TIME DV CMT AMT RATE EVID MDV UVOL EXTRA IND_DIA OCC

> DIALYSIS ANALYSIS BV MISSING AGE WGT HGT BMI BSA SOFA M1F2 GFR XVAL

>

> $DATA RawdataCFP_cov_ext.csv

> IGNORE=_at_ IGNORE(MISSING.EQ.1) ;Exclude missing values

> IGNORE(CMT.GT.3) ;Exclude CSF sample

> IGNORE(XVAL.EQ.1) REWIND

>

> $SUBROUTINE ADVAN13 TOL=12

>

> $MODEL COMP(CENTRAL,DEFOBS,DEFDOSE) COMP(PERIPH)

> COMP(URINE,INITIALOFF)

>

> $PK

> ;------------- Calculation of Time After Dose ------------

>

> IF (EVID.EQ.1.OR.EVID.EQ.4) THEN

> TDOS=TIME

> TAD=0.0

> ENDIF

> IF (EVID.NE.1.AND.EVID.NE.4) TAD=TIME-TDOS

>

> TVCLOTHER =THETA(1)

> CLOTHER =TVCLOTHER*EXP(ETA(4))

>

> TVCL = THETA(2)

> CL = TVCL*EXP(ETA(1))

>

> TVV1 = THETA(3)

> V1 =TVV1*EXP(ETA(2))

>

> TVV2 =THETA(4)

> V2 =TVV2*EXP(ETA(3))

>

> TVQ =THETA(5)

> Q =TVQ

>

> ;------------- Dialysis submodel ------------------------

> CL_DIA = 0

> IF(DIALYSIS.EQ.1) CL_DIA = THETA(6)

>

> IND=0

> IF(IND_DIA.EQ.1) IND=1

>

> S1=V1

> S3=UVOL

>

> K10=CLOTHER/V1

> K12=Q/V1

> K21=Q/V2

> K13=CL/V1

> K11=CL_DIA/V1

>

> $DES

> DADT(1)=-K12*A(1)+K21*A(2)-K10*A(1)-K13*A(1)-K11*A(1)*IND

> DADT(2)=K12*A(1)-K21*A(2)

> DADT(3)=K13*A(1)

>

> $ERROR

> IPRED = 1E-3

> IF(F.GT.0) IPRED=F

>

> Y = IPRED*(1+EPS(1))

> IRES = DV-IPRED

> IWRES = IRES/(IPRED*SQRT(SIGMA(1,1)))

>

> IF(CMT.EQ.3) THEN

> Y = IPRED*(1+EPS(2))

> IRES = DV-IPRED

> IWRES = IRES/SQRT(IPRED*IPRED*SIGMA(2,2))

> ENDIF

>

> $THETA

> (1E-9,1.097450) ; CLOTHER; L/h

> (1E-9,2.124530) ; CL; L/h

> (1E-9,8.640870) ; V1; L

> (1E-9,18.58180) ; V2; L

> (1E-9,34.13580) ; Q; L/h

> (1E-9,4.046690) ; CL_DIA; L/h

>

> $OMEGA

> 1.265890 ; IIV_CL

> .387112 ; IIV_V1

> .186287 ; IIV_V2

> .371892 ; IIV_CLOTHER

>

> $SIGMA

> 0.090199 ; Proportional plasma

> .106711 ; Proportional urine

>

> $ESTIMATION SIG=2 MAX=9999 METHOD=1 SORT INTERACTION POSTHOC PRINT=
=1

> MSFO=run61.msf

>

> ;------------------------------------------------------------------------=
---------

> ;----------------------- POST HOC
> --------------------------------------------

> ;------------------------------------------------------------------------=
---------

>

> $PROBLEM PREDICT XVAL1

>

> $INPUT ID TIME DV CP CMT AMT RATE EVID MDV UVOL EXTRA IND_DIA OCC

> DIALYSIS ANALYSIS BV MISSING AGE WGT HGT BMI BSA SOFA M1F2 GFR
> TDM XVAL

>

> $DATA RawdataCFP_xval_ext.csv

> IGNORE=_at_

> IGNORE(MISSING.EQ.1) ;Exclude missing values

> IGNORE(CMT.GT.3) ;Exclude CSF sample

> IGNORE(XVAL.NE.1) REWIND

>

> $MSFI run61.msf

>

> $ESTIMATION SIG=2 MAX=0 METHOD=1 SORT INTERACTION POSTHOC PRINT=1

>

> …

>
--
  Alison Boeckmann
  alisonboeckmann_at_fastmail.fm



Received on Sat Dec 06 2014 - 10:24:11 EST

This archive was generated by hypermail 2.3.0 : Fri Sep 27 2019 - 16:42:25 EDT