From: Jeroen Elassaiss-Schaap <*jeroen*>

Date: Fri, 6 Nov 2015 17:34:29 +0100

Hi Pavel,

For starters, it is simple to calculate using R:

mymat<-abs(matrix(rnorm(25^2),ncol=25))

mymat <- mymat /max(mymat)

#replace mymat with your nonmem $cov matrix

eigenval<-eigen(mymat,symm=T)$values# should be similar to nonmem

reported

cn<-max(eigenval)/min(eigenval)

eigenval<-eigen(mymat[1:10,1:10],symm=T)$values

cn1<-max(eigenval)/min(eigenval) # could be compared to the "PK"

parameters ratio from monolix

Assuming a 25x25 covariance matrix, and theta in 1:10. You will need to

do some rearrangement of the cells to isolate the off-diagonal elements

of $OMEGA, but with this approach you can compare apples by apples.

Until you have done that you will not know whether the platforms provide

different results or similar wrt the condition number.

The difference in behavior with respect to objective function impact is

puzzling, assuming you refer to SAEM estimation in Nonmem. My advice

here would be to focus on (visual) predictive checks, and compare how

well the two platforms perform on that aspect.

Hope this helps,

Jeroen

--

http://pd-value.com

jeroen

+31 6 23118438

-- More value out of your data!

Op 06-11-15 om 17:05 schreef Pavel Belo:

*> NONMEM demonstrated very large differences in objective function when
*

*> variability or correlations were added or removed. Monolix
*

*> demonstrated close-to-insignificant differences. When differences in
*

*> software start to affect important conclusions it becomes
*

*> interesting. It feels like we need to make sure we report the most
*

*> meaningful results.
*

*> NONMEM runs as if the covariance matrix is more a byproduct than an
*

*> essential part of the optimization. Monolix runs as if the covariance
*

*> matrix an essential part of the optimization. NONMEM teachers
*

*> recommend to try a full covariance matrix. Monolix teachers recommend
*

*> to be careful and try a diagonal matrix first.
*

*> Thanks,
*

*> Pavel
*

*> On Fri, Nov 06, 2015 at 08:42 AM, Pavel Belo wrote:
*

*>
*

*> Hello Jeroen,
*

*> Thank you for your response. It was a practical question. I
*

*> understand the theory. What is the reason different packages show
*

*> such different results and present eigenvalues differently? What
*

*> is the best way?
*

*> NONMEM demonstrated much larger max/min values but did not give
*

*> warning messages about non-positive defined matrix. The runs were
*

*> stable. Runs became unstable only when simulated annealing was
*

*> used; instability kicked in at the moment when NONMEM stopped
*

*> simulated annealing; so I had to remove simulated
*

*> annealing. Monolix sometimes gave non-positive defined matrix
*

*> stopping optimization in the middle; sometime it became unstable
*

*> in the middle with or without simulated annealing.
*

*> I do not take sides. I just try to understand it. As max/min is
*

*> frequently reported in BLAs, it is nice to understand what we
*

*> report and why it can be so different across different packages.
*

*> Thanks,
*

*> Pavel
*

*> On Thu, Nov 05, 2015 at 05:14 PM, Jeroen Elassaiss-Schaap
*

*> (PD-value B.V.) wrote:
*

*>
*

*> Hi Pavel,
*

*>
*

*> Principal component analysis can be validly performed on any
*

*> matrix, and it is just a matter of convention that the
*

*> eigenvalue ratios of min/max of the total covariance matrix of
*

*> estimation are reported as the condition number for a given
*

*> model. This as a metric of how easily the dimensionality of
*

*> estimators could be reduced.
*

*>
*

*> The idea behind the separation of eigenvalues, as you show
*

*> here for your model in Monolix, is actually attractive,
*

*> because the off-diagonal elements do reduce the freedom of the
*

*> described variance rather than increasing it. Furthermore they
*

*> are the byproduct of sampling methods like SAEM, not so much
*

*> the result of separate estimation. Two reasons to separate them.
*

*>
*

*> The separation of diagonal variance components and PK
*

*> parameters as you note is less obvious to me, although I am
*

*> pretty sure there will be a good rationale for that in the
*

*> realm of sampling approaches (tighter linkage?).
*

*>
*

*> Even though the off-diagonal elements are associated with a
*

*> decent condition number, it is still larger than the "PK"
*

*> block, assuming the blocks are of comparable size. In other to
*

*> better compare the results my suggestion would be to break up
*

*> the nonmem covariance matrix (as was done for Monolix) in
*

*> blocks of structural, diagonal and off-diagonal elements
*

*> (throwing away a large remainder), and calculate the condition
*

*> number on each matrix. Than you are comparing apples to
*

*> apples, enabling a more straightforward discussion of the
*

*> differences.
*

*>
*

*> Hope this helps,
*

*> Jeroen
*

*>
*

*>
*

*>
*

*>
*

*> http://pd-value.com
*

*> jeroen *

*> *

*> +31 6 23118438
*

*> -- More value out of your data!
*

*>
*

*>
*

*> On 11/04/2015 05:55 PM, Pavel Belo wrote:
*

*> Hello NONMEM Users,
*

*> I try to make sense of the results and one of the ways to do
*

*> it is to compare the same or similar models across software
*

*> packages. 5x5 full omega matrix is used because it was
*

*> prohibitive to remove some insignificant correlations from the
*

*> matrix without removing significant correlations (All
*

*> recommended ways to do it were tested. Diagonal omega was also
*

*> tested, of course). Adding correlations has little effect on
*

*> PK parameters, but it has some effect on simulations.
*

*> NONMEM provides all eigenvalues in one pocket. Here is an
*

*> example.
*

*> ************************************************************************************************************************
*

*> ******************** ********************
*

*> ******************** STOCHASTIC APPROXIMATION
*

*> EXPECTATION-MAXIMIZATION ********************
*

*> ******************** EIGENVALUES OF COR
*

*> MATRIX OF ESTIMATE (S) ********************
*

*> ******************** ********************
*

*> ************************************************************************************************************************
*

*> 1 2 3 4 5
*

*> 6 7 8 9 10 11 12
*

*> 13 14 15 16 17
*

*> 18 19 20 21 22 23
*

*>
*

*> 3.36E-05 5.69E-03 3.40E-02 6.32E-02 9.19E-02
*

*> 1.24E-01 1.53E-01 2.79E-01 3.20E-01 4.32E-01 5.74E-01
*

*> 6.45E-01
*

*> 7.25E-01 7.67E-01 9.73E-01 1.08E+00 1.42E+00
*

*> 1.63E+00 1.86E+00 2.14E+00 2.31E+00 3.12E+00 4.26E+00
*

*> Monolix provides them in 3 pockets:
*

*> PK parameters: Eigenvalues (min, max, max/min): 0.22 2 9.2
*

*> OMEGA (diagonal) and SIGMA: Eigenvalues (min, max, max/min):
*

*> 0.66 1.5 2.2
*

*> OMEGA (correlations): Eigenvalues (min, max, max/min): 0.097
*

*> 2.5 25
*

*> Even though the results look similar, eigenvalues look
*

*> different. Taking into account that max/min ratio is
*

*> frequently reported, it is important to understand the
*

*> difference. It almost look like different sets of parameters
*

*> are estimated separately in the Monolix example, which most
*

*> likely is not the case. Even if we combine all eigenvalues in
*

*> one pocket, max/min looks good. It is impressive
*

*> that max/min ratio for OMEGA correlations may look OK even
*

*> though there are small correlations such as -0.0921, SE=0.064,
*

*> RSE=70%.
*

*> What is the best way to report estimate and report max/min ratios?
*

*> Take care,
*

*> Pavel
*

*>
*

Received on Fri Nov 06 2015 - 11:34:29 EST

Date: Fri, 6 Nov 2015 17:34:29 +0100

Hi Pavel,

For starters, it is simple to calculate using R:

mymat<-abs(matrix(rnorm(25^2),ncol=25))

mymat <- mymat /max(mymat)

#replace mymat with your nonmem $cov matrix

eigenval<-eigen(mymat,symm=T)$values# should be similar to nonmem

reported

cn<-max(eigenval)/min(eigenval)

eigenval<-eigen(mymat[1:10,1:10],symm=T)$values

cn1<-max(eigenval)/min(eigenval) # could be compared to the "PK"

parameters ratio from monolix

Assuming a 25x25 covariance matrix, and theta in 1:10. You will need to

do some rearrangement of the cells to isolate the off-diagonal elements

of $OMEGA, but with this approach you can compare apples by apples.

Until you have done that you will not know whether the platforms provide

different results or similar wrt the condition number.

The difference in behavior with respect to objective function impact is

puzzling, assuming you refer to SAEM estimation in Nonmem. My advice

here would be to focus on (visual) predictive checks, and compare how

well the two platforms perform on that aspect.

Hope this helps,

Jeroen

--

http://pd-value.com

jeroen

+31 6 23118438

-- More value out of your data!

Op 06-11-15 om 17:05 schreef Pavel Belo:

Received on Fri Nov 06 2015 - 11:34:29 EST