NONMEM Users Network Archive

Hosted by Cognigen

Re: [External] Re: M3 method - WRES, and CWRES

From: Bach, Thanh H Y <thanh-bach>
Date: Sat, 5 Sep 2020 16:45:23 +0000

I have a follow-up question on CWRES with M4 method.

I was able to run my model with M3 method, I got NPDE and CWRES (with MDVRE=
S=1) calculated just fine. Then I changed to M4 method by adding YLO to t=
he non-BLQ data, the model stilled converged but in the output table, some =
of the subjects had CWRES = 0 and NPDE was a constant (around 3). The pro=
blem with CWRES and NPDE is not specific to subjects with BLQ observations,=
 but rather, it followed a pattern like this: CWRES and NPDE was calculated=
 for subjects number 1, 3, 5, 7, etc. and not calculated for subjects 2, 4,=
 6, 8, etc.

I suspected that YLO option was the cause, so I ran the model with M2 metho=
d. Indeed, all CWRES was 0.

This might not be a new problem, but I searched through the NMusers archive=
s and most discussions focused on M3 only. Is there any reason why there is=
 less focus on M2 and M4?

Thank you

From: owner-nmusers
 of Matthew Fidler <matthew.fidler
Sent: Saturday, September 5, 2020 10:17 AM
To: Bauer, Robert <Robert.Bauer
Cc: nmusers
Subject: [External] Re: [NMusers] M3 method - WRES, and CWRES

Thank you Bob,

The NPDE 2.0 manual discusses the methods that NPDE uses to handle BLQ, inc=
luding replacing values with pred, ipred, or lloq, or simulating from a uni=
form random value while calculating the NPDE (cdf method). The NONMEM manu=
al doesn't mention the method used. My guess is the cdf method.

I realize that no one has answered Mu'taz's question.

As far as if the CWRES is appropriate for BLQ data, the CWRES method uses t=
he FOCEi approximation to calculate residuals. However with M3/M4 and othe=
r methods the likelihood for these points is not the FOCEi objective functi=
on but the M3/M4 likelihood so anything you do here with CWRES doesn't foll=
ow or add to the likelihood observed during minimization. Therefore in my =
opinion, there will be bias of some sort here.

Best Regards,


On Thu, Sep 3, 2020 at 3:06 PM Bauer, Robert <Robert.Bauer


The NPDE and NPD systems in NONMEM are described in the nm744.pdf manual ( = ), pages 70-75, and follow along the w=
ork of Comet, Brendel, Ngyuen, Mentre, etc. The NPDE R package is not used=
 within NONMEM.

Robert J. Bauer, Ph.D.

Senior Director

Pharmacometrics R&D

ICON Early Phase

820 W. Diamond Avenue

Suite 100

Gaithersburg, MD 20878

Office: (215) 616-6428

Mobile: (925) 286-0769


From: owner-nmusers
Of Matthew Fidler
Sent: Thursday, September 3, 2020 6:08 AM
To: Jeroen Elassaiss-Schaap (PD-value B.V.) <jeroen
Cc: Bill Denney <wdenney>>; Mu'taz Jaber <jaber038
Subject: Re: [NMusers] M3 method - WRES, and CWRES

Hi everyone,

As an aside, nlmixr's upcoming release (that supports censoring) simulates =
a value using a truncated normal based on the ipred, variance at that point=
 and the censoring column to produce an observation. This observation is u=
sed to calculate RES, WRES, CWRES. It is flagged so you can see which valu=
es use this approach. In theory, since this is simulated from the IPRED/tr=
uncated the CWRES would be likely follow the distribution closer.

I'm unsure if the new NONMEM uses this approach.

Another question from my end is the NPDE: There are many methods to handle=
 BLQ values with NPDE R package, does anyone know which NONMEM uses? Or do=
 you need to use the NPDE package to get these values from NONMEM?


On Wed, Sep 2, 2020 at 2:09 AM Jeroen Elassaiss-Schaap (PD-value B.V.) <jer=

Hi Mutaz, Bill,

It might be useful to use NPDEs, as discussed in
musers/2019-February/7376.html; the whole thread is worthwhile reading. NP=
DEs can be calculated also for BQL values.

Bill -thanks for pointing to excellent post of Matt! I would take as most i=
mportant point that CWRES for non-BQL values, calculated with a model with =
influential BQL, are biased because the influence of the BQL values is not =
accounted for. (if a certain prediction for a measurable concentration is c=
hanged by 10% because of the M3 method, that will turn up as a similar bias=
 in CWRES). The NPDEs as referenced to in the above discussion (Nguyen2012 =
JPKPD 0.1007/s10928-012-9264-2) do not suffer from that drawback as one can=
 see the complete profile (cf Fig 8 of Nguyen2012).

Hope this helps,




+31 6 23118438

-- More value out of your data!

On 2/9/20 2:32 am, Bill Denney wrote:

Hi Mutaz,

Matt Hutmacher described it well here:

A very brief summary of his excellent post is that subjects with a combinat=
ion of censored (BLQ) an uncensored (above the LLOQ and below the ULOQ) wil=
l be biased in their reporting of CWRES because you cannot calculate CWRES =
for BLQ values. (I say this before looking up what MDVRES does.)

My guess that Bob or someone else can confirm is that the bias is anticipat=
ed to be relatively small compared to the value of being able to compare CW=
RES values the other observations for a subject. It does not definitively =
mean that the results are unbiased (see Matt’s Tmax example), but general=
ly, the CWRES values previously omitted are more useful than excluding them=
 from calculation.



From: owner-nmusers
Of Mu'taz Jaber
Sent: Tuesday, September 1, 2020 7:25 PM
To: nmusers
Subject: [NMusers] M3 method - WRES, and CWRES


Back in April 2010, Sebastian Bihorel and Martin Bergstrand initiated a dis=
cussion regarding using the M3 and M4 methods for handling BQL data and how=
 it seemed to be a bug that NONMEM wouldn't compute WRES for the entire set=
 of subject data records whenever a BQL was included (https://www.cognigen.=
com/nmusers/2010-April/2445.html). Tom Ludden responded with the following=
 post (

This issue was discussed with Stuart Beal. He believed that weighted

residuals would be incorrect for an individual that had both continuous

dependent variables and a likelihood in the calculation of their

contribution to the objective function value, as is the case with his M3

or M4 BQL methods The code for both RES and WRES are intentionally

bypassed in these cases.

Since then, we now have easy functionality with the F_FLAG=1 condition of=
 the M3/M4 code in $ERROR to tack on MDVRES=1 that allows the calculation=
 of WRES and CWRES to be available in output tables.

My questions are: Is Stuart Beal's original concern still valid? Do these =
NONMEM updates give us appropriate WRES and CWRES for plotting purposes for=
 individuals whose records contain BQL data?

Thank you,

Mutaz Jaber

PhD student

University of Minnesota


Mutaz M. Jaber, PharmD.

PhD student, Pharmacometrics

Experimental and Clinical Pharmacology

University of Minnesota

717 Delaware St SE; Room 468

Minneapolis, MN 55414

Email: jaber038

Phone: +1 651-706-5202

~ Stay curious

ICON plc made the following annotations.
This e-mail transmission may contain confidential or legally privileged inf=
ormation that is intended only for the individual or entity named in the e-=
mail address. If you are not the intended recipient, you are hereby notifie=
d that any disclosure, copying, distribution, or reliance upon the contents=
 of this e-mail is strictly prohibited. If you have received this e-mail tr=
ansmission in error, please reply to the sender, so that ICON plc can arran=
ge for proper delivery, and then please delete the message.

Thank You,

ICON plc
South County Business Park
Dublin 18
Registered number: 145835

Received on Sat Sep 05 2020 - 12:45:23 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: