From: Immanuel Freedman <*drfreedman*>

Date: Thu, 24 Sep 2020 01:52:18 -0400 (EDT)

Date: Thu, 24 Sep 2020 01:52:18 -0400 (EDT)

Colleagues,

In principle, the log likelihood can be used for such comparisons, however an L2 or peak Signal to Noise ratio or cross entropy is widely used in the signal processing literature for such comparisons.

In NONNEM, the problem relates to numerical stability and accuracy of the approximations by which the OFV is estimated. The closed form exact derivative often results in far less numerical noise. This is also true for ADVAN6 when analytic derivatives (not numerical) are utilized.

The signal processing and machine learning fields have evolved methods to handle this and these questions of comparing structures and covariance matrix have good practical solutions.

It would be interesting to make the same comparisons in e.g., torsten, nlmixr or Pumas.

Regards,

Immanuel

Received on Thu Sep 24 2020 - 01:52:18 EDTOn September 23, 2020 8:49 PM Steven L Shafer <steven.shafer

Dear Dennis:

Gosh, that is super interesting. I would guess it was the differences in the first derivative between the methods. ADVAN4 will be closed form, and ADVAN6 will be (I believe) numerically calculated.

Steve

From:owner-nmusers

Mark

I posed a variant of this question to Stu Beal (Sheiner's statistician) > 20 years ago. He answered that one cannot compare ADVAN's, e.g., between ADVAN4 and ADVAN6 for the identical model.

I verified this today when I ran a model with those two ADVAN's -- both converged, they yielded quite similar parameter estimates, but the OF differed by 60 units.

I will be interested to hear Bob Bauer's reply to this issue.

Dennis

Dennis Fisher MD

P < (The "P Less Than" Company)

Phone / Fax: 1-866-PLessThan (1-866-753-7784)

www.PLessThan.com

On Sep 23, 2020, at 5:09 PM, Mark Tepeck <

Hi NMusers,

I believe below is a very common question, but I could not find a

clear answer in literature.

Sometimes, we want to find out which algorithm offers the better model

fitting for a given dataset.

Is it possible to use the objective function value (OFV) to compare

the model fitting computed by various algorithms (e.g. FOCE, IMP, and

SAEM) ? Put it into another way, the same input dataset with the same

fitted model/estimates will lead to similar OFVs among FOCE, IMP, and

SAEM?

Thanks,

Mark

- This message: [ Message body ]
- Next message: Rik Schoemaker: "Updating xpose4 stratification headers in VPCs"
- Previous message: Luann Phillips: "RE: OFV from different algorithms"
- Maybe in reply to: Steven L Shafer: "RE: OFV from different algorithms"
- Next in thread: Matthew Fidler: "Re: OFV from different algorithms"
- Reply: Matthew Fidler: "Re: OFV from different algorithms"