From: Pavel Belo <*nonmem*>

Date: Mon, 16 Nov 2015 11:49:07 -0500 (EST)

Hello Jeroen,

Thank you for the advice

for starters.Â It advances some of us to an intermediate or even highe=

r

level.

The question can be

deeper.Â Here we mostly do not refer to populationÂ PK models as

hierarchical models.Â In Monolix books statements like "we

take advantage of the hierarchical structure of the model" are

everywhere. Â It makes little sense to estimate theta, diagonal of omeg=

a

andÂ sigma, and omega correlations all together as correlated parameter=

s

and

then ignore correlations between some groups of parameters when

eigenvalues are

calculated (for example, correlations of theta and omega).Â It makes=

more

sense to do it when there is a theory, which suggests that under certain

conditions the groups of parametersÂ are not correlated.

It is interesting to

understand a rationale for a potential assumption (or a result of

another

assumption) that estimates of certain groups of parameters are not

correlated.Â

Take care,

Pavel

Â

Â

Â

On Fri, Nov 06, 2015 at 11:34 AM, Jeroen Elassaiss-Schaap wrote:

Â

Â

Hi Pavel,

For starters, it is simple to calculate using R:

Â mymat<-abs(matrix(rnorm(25^2),ncol=25))

Â mymat <- mymat /max(mymat)

Â

Â #replace mymat with your nonmem $cov matrix

Â eigenval<-eigen(mymat,symm=T)$values #

should be similar to nonmem reported

Â cn<-max(eigenval)/min(eigenval)

Â eigenval<-eigen(mymat[1:10,1:10],symm=T)$values

Â cn1<-max(eigenval)/min(eigenval)Â # could be

compared to the "PK" parameters ratio from monolix

Assuming a 25x25 covariance matrix, and theta in 1:10. You will

need

to do some rearrangement of the cells to isolate the off-diagonal

elements of $OMEGA, but with this approach you can compare apples

by

apples. Until you have done that you will not know whether the

platforms provide different results or similar wrt the condition

number.

The difference in behavior with respect to objective function

impact

is puzzling, assuming you refer to SAEM estimation in Nonmem. My

advice here would be to focus on (visual) predictive checks, and

compare how well the two platforms perform on that aspect.

Hope this helps,

Jeroen

--

http://pd-value.com <http://pd-value.com>

jeroen

+31 6 23118438

-- More value out of your data!

Op 06-11-15 om 17:05 schreef Pavel

Belo:

NONMEM demonstrated very large differences in

objective function when variability or correlations were added or

removed.Â Monolix demonstrated close-to-insignificant

differences.Â When differences in software start to affect

important conclusions it becomes interesting.Â It feels like we

need to make sure we report the most meaningful results.Â

Â

NONMEM runs as if the covariance matrix is more a byproduct

than an essential part of the optimization.Â Monolix runs as =

if

the covariance matrix an essential part of the optimization.Â

NONMEM teachers recommend to try a full covariance matrix.Â

Monolix teachers recommend to be careful and try a diagonal

matrix first.Â

Â

Thanks,

PavelÂ

Â

On Fri, Nov 06, 2015 at 08:42 AM, Pavel Belo wrote:

Â

Â

Â

Â Hello Jeroen,

Â

Thank you for your response.Â It was a practical

question.Â I understand the theory.Â What is the r=

eason

different packages show such different results and

present

eigenvaluesÂ differently?Â What is the best way?=

Â

Â

NONMEM demonstrated much larger max/min values but did

not give warning messages about non-positive defined

matrix.Â The runs were stable.Â Runs became unstab=

le only

when simulated annealing was used;Â instability kicked =

in

at the moment when NONMEMÂ stopped simulated annealing; =

so

I had to remove simulated annealing.Â Â Monolix some=

times

gave non-positive defined matrix stopping optimization in

the middle;Â sometimeÂ it became unstable in the mi=

ddle

with or without simulated annealing.Â Â

Â

I do not take sides.Â I just try to understand it.Â As

max/min is frequently reported in BLAs, it is nice to

understand what we report and why it can be so different

across different packages.Â

Â

Thanks,

Pavel

Â

On Thu, Nov 05, 2015 at 05:14 PM, Jeroen Elassaiss-Schaap

(PD-value B.V.) wrote:

Â

Â

Hi

Pavel,

Principal component analysis can be validly performed on

any matrix, and it is just a matter of convention that

the

eigenvalue ratios of min/max of the total covariance

matrix of estimation are reported as the condition number

for a given model. This as a metric of how easily the

dimensionality of estimators could be reduced.

The idea behind the separation of eigenvalues, as you

show

here for your model in Monolix, is actually attractive,

because the off-diagonal elements do reduce the freedom

of

the described variance rather than increasing it.

Furthermore they are the byproduct of sampling methods

like SAEM, not so much the result of separate estimation.

Two reasons to separate them.

The separation of diagonal variance components and PK

parameters as you note is less obvious to me, although I

am pretty sure there will be a good rationale for that in

the realm of sampling approaches (tighter linkage?).

Even though the off-diagonal elements are associated with

a decent condition number, it is still larger than the

"PK" block, assuming the blocks are of comparable size.

In

other to better compare the results my suggestion would

be

to break up the nonmem covariance matrix (as was done for

Monolix) in blocks of structural, diagonal and

off-diagonal elements (throwing away a large remainder),

and calculate the condition number on each matrix. Than

you are comparing apples to apples, enabling a more

straightforward discussion of the differences.

Hope this helps,

Jeroen

http://pd-value.com <http://pd-value.com>

jeroen

+31 6 23118438

-- More value out of your data!

On 11/04/2015 05:55 PM, Pavel

Belo wrote:

Hello NONMEM Users,

Â

I try to make sense of the results and one of the

ways to do it is to compare the same or similar models

across software packages.Â 5x5 full omega matrix is=

used

because itÂ was prohibitiveÂ to remove some insign=

ificant

correlations from the matrix without removing

significant correlations (All recommended ways to do it

were tested. Diagonal omega was also tested, of

course).Â Adding correlations has little effect on PK

parameters, but it has some effect on simulations.Â

Â

NONMEM provides all eigenvalues in one pocket.Â Here

is an example.Â

Â *********************************************************************=

***************************************************

Â ********************Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â

********************

Â ********************Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â STOCHASTIC

APPROXIMATION EXPECTATION-MAXIMIZATIONÂ Â Â =

Â Â Â Â Â Â Â Â Â Â Â

********************

Â ********************Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â EIGENVALUES OF

COR MATRIX OF ESTIMATE (S)Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

********************

Â ********************Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â

********************

Â *********************************************************************=

***************************************************

Â

Â Â Â Â Â Â Â Â Â Â Â Â 1=

Â Â Â Â Â Â Â Â 2Â Â Â Â =

Â Â Â Â 3Â Â Â Â Â Â Â Â =

4Â Â Â Â Â Â Â Â

5Â Â Â Â Â Â Â Â 6Â =

Â Â Â Â Â Â Â 7Â Â Â Â Â =

Â Â Â 8Â Â Â Â Â Â Â Â 9Â=

Â Â Â Â Â Â

10Â Â Â Â Â Â Â 11Â Â=

Â Â Â Â Â 12

Â Â Â Â Â Â Â Â Â Â=

Â Â 13Â Â Â Â Â Â Â 14Â Â =

Â Â Â Â Â 15Â Â Â Â Â Â Â =

16Â Â Â Â Â Â Â

17Â Â Â Â Â Â Â 18Â Â=

Â Â Â Â Â 19Â Â Â Â Â Â Â=

20Â Â Â Â Â Â Â 21Â Â Â Â =

Â Â Â

22Â Â Â Â Â Â Â 23

Â

Â Â Â Â Â Â Â Â 3.36E-05=

Â 5.69E-03Â 3.40E-02Â 6.32E-02Â

9.19E-02Â 1.24E-01Â 1.53E-01Â 2.79E-01Â=

3.20E-01Â

4.32E-01Â 5.74E-01Â 6.45E-01

Â Â Â Â Â Â Â Â Â 7.2=

5E-01Â 7.67E-01Â 9.73E-01Â 1.08E+00Â

1.42E+00Â 1.63E+00Â 1.86E+00Â 2.14E+00Â=

2.31E+00Â

3.12E+00Â 4.26E+00

Â

Monolix provides them in 3 pockets:

Â

PK parameters: Eigenvalues (min, max, max/min): 0.22Â

2Â 9.2

OMEGA (diagonal) and SIGMA: Eigenvalues (min, max,

max/min): 0.66Â 1.5Â 2.2

OMEGA (correlations):Â Eigenvalues (min, max,

max/min): 0.097Â 2.5Â 25

Â

Even though the results look similar, eigenvalues

look different.Â Taking into account that max/min rat=

io

is frequently reported, it is important to understand

the difference.Â It almost look like different sets o=

f

parameters are estimated separately in the Monolix

example, which most likely is not the case.Â Even if =

we

combine all eigenvalues in one pocket, max/min looks

good.Â Â Â It is impressive thatÂ max/min =

ratio forÂ OMEGA

correlationsÂ may look OK even though there are small

correlations such as -0.0921, SE=0.064, RSE=70%.

Â

What is the best way to report estimate and report

max/min ratios?

Â

Take care,

Pavel

Received on Mon Nov 16 2015 - 11:49:07 EST

Date: Mon, 16 Nov 2015 11:49:07 -0500 (EST)

Hello Jeroen,

Thank you for the advice

for starters.Â It advances some of us to an intermediate or even highe=

r

level.

The question can be

deeper.Â Here we mostly do not refer to populationÂ PK models as

hierarchical models.Â In Monolix books statements like "we

take advantage of the hierarchical structure of the model" are

everywhere. Â It makes little sense to estimate theta, diagonal of omeg=

a

andÂ sigma, and omega correlations all together as correlated parameter=

s

and

then ignore correlations between some groups of parameters when

eigenvalues are

calculated (for example, correlations of theta and omega).Â It makes=

more

sense to do it when there is a theory, which suggests that under certain

conditions the groups of parametersÂ are not correlated.

It is interesting to

understand a rationale for a potential assumption (or a result of

another

assumption) that estimates of certain groups of parameters are not

correlated.Â

Take care,

Pavel

Â

Â

Â

On Fri, Nov 06, 2015 at 11:34 AM, Jeroen Elassaiss-Schaap wrote:

Â

Â

Hi Pavel,

For starters, it is simple to calculate using R:

Â mymat<-abs(matrix(rnorm(25^2),ncol=25))

Â mymat <- mymat /max(mymat)

Â

Â #replace mymat with your nonmem $cov matrix

Â eigenval<-eigen(mymat,symm=T)$values #

should be similar to nonmem reported

Â cn<-max(eigenval)/min(eigenval)

Â eigenval<-eigen(mymat[1:10,1:10],symm=T)$values

Â cn1<-max(eigenval)/min(eigenval)Â # could be

compared to the "PK" parameters ratio from monolix

Assuming a 25x25 covariance matrix, and theta in 1:10. You will

need

to do some rearrangement of the cells to isolate the off-diagonal

elements of $OMEGA, but with this approach you can compare apples

by

apples. Until you have done that you will not know whether the

platforms provide different results or similar wrt the condition

number.

The difference in behavior with respect to objective function

impact

is puzzling, assuming you refer to SAEM estimation in Nonmem. My

advice here would be to focus on (visual) predictive checks, and

compare how well the two platforms perform on that aspect.

Hope this helps,

Jeroen

--

http://pd-value.com <http://pd-value.com>

jeroen

+31 6 23118438

-- More value out of your data!

Op 06-11-15 om 17:05 schreef Pavel

Belo:

NONMEM demonstrated very large differences in

objective function when variability or correlations were added or

removed.Â Monolix demonstrated close-to-insignificant

differences.Â When differences in software start to affect

important conclusions it becomes interesting.Â It feels like we

need to make sure we report the most meaningful results.Â

Â

NONMEM runs as if the covariance matrix is more a byproduct

than an essential part of the optimization.Â Monolix runs as =

if

the covariance matrix an essential part of the optimization.Â

NONMEM teachers recommend to try a full covariance matrix.Â

Monolix teachers recommend to be careful and try a diagonal

matrix first.Â

Â

Thanks,

PavelÂ

Â

On Fri, Nov 06, 2015 at 08:42 AM, Pavel Belo wrote:

Â

Â

Â

Â Hello Jeroen,

Â

Thank you for your response.Â It was a practical

question.Â I understand the theory.Â What is the r=

eason

different packages show such different results and

present

eigenvaluesÂ differently?Â What is the best way?=

Â

Â

NONMEM demonstrated much larger max/min values but did

not give warning messages about non-positive defined

matrix.Â The runs were stable.Â Runs became unstab=

le only

when simulated annealing was used;Â instability kicked =

in

at the moment when NONMEMÂ stopped simulated annealing; =

so

I had to remove simulated annealing.Â Â Monolix some=

times

gave non-positive defined matrix stopping optimization in

the middle;Â sometimeÂ it became unstable in the mi=

ddle

with or without simulated annealing.Â Â

Â

I do not take sides.Â I just try to understand it.Â As

max/min is frequently reported in BLAs, it is nice to

understand what we report and why it can be so different

across different packages.Â

Â

Thanks,

Pavel

Â

On Thu, Nov 05, 2015 at 05:14 PM, Jeroen Elassaiss-Schaap

(PD-value B.V.) wrote:

Â

Â

Hi

Pavel,

Principal component analysis can be validly performed on

any matrix, and it is just a matter of convention that

the

eigenvalue ratios of min/max of the total covariance

matrix of estimation are reported as the condition number

for a given model. This as a metric of how easily the

dimensionality of estimators could be reduced.

The idea behind the separation of eigenvalues, as you

show

here for your model in Monolix, is actually attractive,

because the off-diagonal elements do reduce the freedom

of

the described variance rather than increasing it.

Furthermore they are the byproduct of sampling methods

like SAEM, not so much the result of separate estimation.

Two reasons to separate them.

The separation of diagonal variance components and PK

parameters as you note is less obvious to me, although I

am pretty sure there will be a good rationale for that in

the realm of sampling approaches (tighter linkage?).

Even though the off-diagonal elements are associated with

a decent condition number, it is still larger than the

"PK" block, assuming the blocks are of comparable size.

In

other to better compare the results my suggestion would

be

to break up the nonmem covariance matrix (as was done for

Monolix) in blocks of structural, diagonal and

off-diagonal elements (throwing away a large remainder),

and calculate the condition number on each matrix. Than

you are comparing apples to apples, enabling a more

straightforward discussion of the differences.

Hope this helps,

Jeroen

http://pd-value.com <http://pd-value.com>

jeroen

+31 6 23118438

-- More value out of your data!

On 11/04/2015 05:55 PM, Pavel

Belo wrote:

Hello NONMEM Users,

Â

I try to make sense of the results and one of the

ways to do it is to compare the same or similar models

across software packages.Â 5x5 full omega matrix is=

used

because itÂ was prohibitiveÂ to remove some insign=

ificant

correlations from the matrix without removing

significant correlations (All recommended ways to do it

were tested. Diagonal omega was also tested, of

course).Â Adding correlations has little effect on PK

parameters, but it has some effect on simulations.Â

Â

NONMEM provides all eigenvalues in one pocket.Â Here

is an example.Â

Â *********************************************************************=

***************************************************

Â ********************Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â

********************

Â ********************Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â STOCHASTIC

APPROXIMATION EXPECTATION-MAXIMIZATIONÂ Â Â =

Â Â Â Â Â Â Â Â Â Â Â

********************

Â ********************Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â EIGENVALUES OF

COR MATRIX OF ESTIMATE (S)Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

********************

Â ********************Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â Â Â Â Â =

Â Â Â Â Â Â Â Â Â Â Â Â Â=

Â Â Â Â Â Â Â Â

********************

Â *********************************************************************=

***************************************************

Â

Â Â Â Â Â Â Â Â Â Â Â Â 1=

Â Â Â Â Â Â Â Â 2Â Â Â Â =

Â Â Â Â 3Â Â Â Â Â Â Â Â =

4Â Â Â Â Â Â Â Â

5Â Â Â Â Â Â Â Â 6Â =

Â Â Â Â Â Â Â 7Â Â Â Â Â =

Â Â Â 8Â Â Â Â Â Â Â Â 9Â=

Â Â Â Â Â Â

10Â Â Â Â Â Â Â 11Â Â=

Â Â Â Â Â 12

Â Â Â Â Â Â Â Â Â Â=

Â Â 13Â Â Â Â Â Â Â 14Â Â =

Â Â Â Â Â 15Â Â Â Â Â Â Â =

16Â Â Â Â Â Â Â

17Â Â Â Â Â Â Â 18Â Â=

Â Â Â Â Â 19Â Â Â Â Â Â Â=

20Â Â Â Â Â Â Â 21Â Â Â Â =

Â Â Â

22Â Â Â Â Â Â Â 23

Â

Â Â Â Â Â Â Â Â 3.36E-05=

Â 5.69E-03Â 3.40E-02Â 6.32E-02Â

9.19E-02Â 1.24E-01Â 1.53E-01Â 2.79E-01Â=

3.20E-01Â

4.32E-01Â 5.74E-01Â 6.45E-01

Â Â Â Â Â Â Â Â Â 7.2=

5E-01Â 7.67E-01Â 9.73E-01Â 1.08E+00Â

1.42E+00Â 1.63E+00Â 1.86E+00Â 2.14E+00Â=

2.31E+00Â

3.12E+00Â 4.26E+00

Â

Monolix provides them in 3 pockets:

Â

PK parameters: Eigenvalues (min, max, max/min): 0.22Â

2Â 9.2

OMEGA (diagonal) and SIGMA: Eigenvalues (min, max,

max/min): 0.66Â 1.5Â 2.2

OMEGA (correlations):Â Eigenvalues (min, max,

max/min): 0.097Â 2.5Â 25

Â

Even though the results look similar, eigenvalues

look different.Â Taking into account that max/min rat=

io

is frequently reported, it is important to understand

the difference.Â It almost look like different sets o=

f

parameters are estimated separately in the Monolix

example, which most likely is not the case.Â Even if =

we

combine all eigenvalues in one pocket, max/min looks

good.Â Â Â It is impressive thatÂ max/min =

ratio forÂ OMEGA

correlationsÂ may look OK even though there are small

correlations such as -0.0921, SE=0.064, RSE=70%.

Â

What is the best way to report estimate and report

max/min ratios?

Â

Take care,

Pavel

Received on Mon Nov 16 2015 - 11:49:07 EST