From: Steimer, Jean-Louis <*jean-louis.steimer*>

Date: Mon, 23 Mar 2015 12:54:04 +0000

Dear Nele, Dear all,

Below in red in Nele's e-mail, you will find the input of Bjoern Bornkamp, =

a statistician from the Novartis Stats/Methods group. I forwarded your mail=

to him. Bjoern was involved in the qualification discussion with EMA toget=

her with Jose Pinheiro and Frank Bretz. He is one of the implementers of th=

e MCP-Mod methodology within Novartis, and applies it routinely in Phase 2 =

studies.

I am sure that Bjoern' answers will help.

Bye

Jean-Louis Steimer

+++++++++++++++++++++++

Dear all,

I am writing to you as we are currently discussing the implementation of th=

e MCP-MOD approach for dose finding based on Phase 2B results and would lik=

e to hear your opinion on this approach.

Original MCP-Mod is not intended to be used in Ph III, special adaptations =

are necessary (closed testing).

It would be good to get feedback from both statisticians and classical mode=

lers.

I have thought about the approach, and have a few problems about seeing the=

advantage of the approach over complete population-PK/PD modeling.

These are two different approaches that complement each other. MCP-Mod is n=

ot intended to replace population-PK/PD modeling (the idea is to replace AN=

OVA-type models).

I can see benefits to do both a simple cross-sectional dose-response analys=

is (like MCP-Mod) and a complete dose-exposure-response characterization.

If results are consistent between both approaches one would have more confi=

dence overall in the analysis results than from either analysis alone.

If results are not consistent one needs to dig a bit "deeper", but this is =

also useful information.

From what I understood, I can see the following issues:

MCP-MOD

· Only uses trial endpoints, i.e. it ignores the time course of t=

he treatment effect. I have a problem with this because there might be nois=

e in the endpoint (e.g. if the effect has reached a plateau), which might p=

otentially lead to the selection of the wrong model structure. Including th=

e time-course like in PKPD modeling approaches would detect that the deviat=

ion is just noise, and thus probably be able to identify the right model st=

ructure despite this.

MCP-Mod can handle longitudinal data, see Pinheiro et al. (2014), Stat Med.=

33,1646-61 for one example, which is also available in the DoseFinding R p=

ackage.

· Uses dose-response models instead of exposure-response models

Correct. Again, MCP-Mod is not intended to replace population-PK/PD modelin=

g. We have started thinking how to extend the key ideas of MCP-Mod to expos=

ure-response models and encourage the community to look into this.

· Pre-specifies the model structure. While I understand that for =

pivotal trials prespecification is crucial, I would assume that Phase 2 is =

performed to allow exploration of the data to come up with the best model g=

iven the data we have. What happens if the true model is not part of the te=

sted ones? What if we have new physiological insights that tell us about th=

e model structure after we have seen the data? Do we then ignore what we kn=

ow and fit all bad models, and if none gives a good description we do model=

averaging of bad models?

Excellent questions. Candidate models for MCP-Mod should always be selected=

based on entire teams input and operating characteristics should be evalua=

ted upfront. More specifically, our experience shows that MCP-Mod is relati=

vely robust if the true model is not part of the tested ones, see for examp=

le Pinheiro et al. (2006), J. Biopharm. Statist. 16,639-656. This is also s=

omething that can be evaluated to some extend upfront (at the design stage)=

by simulations.

Among other things one advantage of pre-specification is that it makes the =

modelling more transparent/credible for externals (e.g. health authorities)=

, if one specifies before seeing the data what will be done. But of course =

there is a trade-off: Not sure if it is possible to pre-specify a full popu=

lation PK/PD analysis.

· If we include a model with many parameters in the prespecificat=

ion and only have a few dose strength, wouldn't the model with more paramet=

ers be more likely to give a good fit (e.g. when comparing Emax to logistic=

), with the consequence that a wrong dose might be selected?

Not sure whether I fully understand this question. Of course the model-sele=

ction/averaging step of MCP-Mod would take into account the model complexit=

y by using AIC/BIC (not only looking at model fit). Again, operating charac=

teristics need to be evaluated in advance, which include precision of targe=

t dose estimation and also possible convergence problems if the number of p=

arameters is to larger.

Colleagues from statistics recommend to cover all potential models with dif=

ferent shapes in the candidate set to avoid potential bias in dose selectio=

n, but they argue that post-hoc model fitting leads to data-dredging and ov=

er-fitting, does not account for model uncertainty and gives overly-optimis=

tic results. I am wondering however what the difference in the approach is =

if anyway ALL potential models are considered (which can lead to overfittin=

g as well)?

There is a penalty for using many models in MCP-Mod: In the MCP step the mu=

ltiplicity adjustment would get higher if there are more models included (i=

n particular if they are very different).

In the Mod step the variance of the dose-response curve would increase with=

an increased number of models, so there one faces the usual variance/bias =

trade-off.

Might a good solution be to combine PKPD modeling with MCP-Mod?

Yes, see above

Your opinion will be highly appreciated, and I am looking forward to receiv=

ing comments both in favour and against the approach :-)

Best

Nele

From: owner-nmusers

Behalf Of Mueller-Plock, Nele

Sent: Friday, March 20, 2015 1:02 PM

To: nmusers

Subject: [NMusers] Using MCP-MOD in dose finding for Phase 3

Dear all,

I am writing to you as we are currently discussing the implementation of th=

e MCP-MOD approach for dose finding based on Phase 2B results and would lik=

e to hear your opinion on this approach. It would be good to get feedback f=

rom both statisticians and classical modelers.

I have thought about the approach, and have a few problems about seeing the=

advantage of the approach over complete population-PK/PD modeling. From wh=

at I understood, I can see the following issues:

MCP-MOD

· Only uses trial endpoints, i.e. it ignores the time course of t=

he treatment effect. I have a problem with this because there might be nois=

e in the endpoint (e.g. if the effect has reached a plateau), which might p=

otentially lead to the selection of the wrong model structure. Including th=

e time-course like in PKPD modeling approaches would detect that the deviat=

ion is just noise, and thus probably be able to identify the right model st=

ructure despite this.

· Uses dose-response models instead of exposure-response models

· Pre-specifies the model structure. While I understand that for =

pivotal trials prespecification is crucial, I would assume that Phase 2 is =

performed to allow exploration of the data to come up with the best model g=

iven the data we have. What happens if the true model is not part of the te=

sted ones? What if we have new physiological insights that tell us about th=

e model structure after we have seen the data? Do we then ignore what we kn=

ow and fit all bad models, and if none gives a good description we do model=

averaging of bad models?

· If we include a model with many parameters in the prespecificat=

ion and only have a few dose strength, wouldn't the model with more paramet=

ers be more likely to give a good fit (e.g. when comparing Emax to logistic=

), with the consequence that a wrong dose might be selected?

Colleagues from statistics recommend to cover all potential models with dif=

ferent shapes in the candidate set to avoid potential bias in dose selectio=

n, but they argue that post-hoc model fitting leads to data-dredging and ov=

er-fitting, does not account for model uncertainty and gives overly-optimis=

tic results. I am wondering however what the difference in the approach is =

if anyway ALL potential models are considered (which can lead to overfittin=

g as well)?

Might a good solution be to combine PKPD modeling with MCP-Mod?

Your opinion will be highly appreciated, and I am looking forward to receiv=

ing comments both in favour and against the approach :-)

Best

Nele

______________________________________________________________

Dr. Nele Mueller-Plock, CAPM

Associate Scientific Director Pharmacometrics

Global Pharmacometrics

Translational Medicine

Takeda Pharmaceuticals International GmbH

Thurgauerstrasse 130

8152 Glattpark-Opfikon (Zürich)

Switzerland

Visitor address:

Alpenstrasse 3

8152 Glattpark-Opfikon (Zürich)

Switzerland

Phone: (+41) 44 / 55 51 404

Mobile: (+41) 79 / 654 33 99

mailto: nele.mueller-plock

http://www.takeda.com<http://www.takeda.com/>

--------------------------------------------------------------------

The content of this email and of any files transmitted may contain confiden=

tial, proprietary or legally privileged information and is intended solely =

for the use of the person/s or entity/ies to whom it is addressed. If you h=

ave received this email in error you have no permission whatsoever to use, =

copy, disclose or forward all or any of its contents. Please immediately no=

tify the sender and thereafter delete this email and any attachments.

--------------------------------------------------------------------

Received on Mon Mar 23 2015 - 08:54:04 EDT

Date: Mon, 23 Mar 2015 12:54:04 +0000

Dear Nele, Dear all,

Below in red in Nele's e-mail, you will find the input of Bjoern Bornkamp, =

a statistician from the Novartis Stats/Methods group. I forwarded your mail=

to him. Bjoern was involved in the qualification discussion with EMA toget=

her with Jose Pinheiro and Frank Bretz. He is one of the implementers of th=

e MCP-Mod methodology within Novartis, and applies it routinely in Phase 2 =

studies.

I am sure that Bjoern' answers will help.

Bye

Jean-Louis Steimer

+++++++++++++++++++++++

Dear all,

I am writing to you as we are currently discussing the implementation of th=

e MCP-MOD approach for dose finding based on Phase 2B results and would lik=

e to hear your opinion on this approach.

Original MCP-Mod is not intended to be used in Ph III, special adaptations =

are necessary (closed testing).

It would be good to get feedback from both statisticians and classical mode=

lers.

I have thought about the approach, and have a few problems about seeing the=

advantage of the approach over complete population-PK/PD modeling.

These are two different approaches that complement each other. MCP-Mod is n=

ot intended to replace population-PK/PD modeling (the idea is to replace AN=

OVA-type models).

I can see benefits to do both a simple cross-sectional dose-response analys=

is (like MCP-Mod) and a complete dose-exposure-response characterization.

If results are consistent between both approaches one would have more confi=

dence overall in the analysis results than from either analysis alone.

If results are not consistent one needs to dig a bit "deeper", but this is =

also useful information.

From what I understood, I can see the following issues:

MCP-MOD

· Only uses trial endpoints, i.e. it ignores the time course of t=

he treatment effect. I have a problem with this because there might be nois=

e in the endpoint (e.g. if the effect has reached a plateau), which might p=

otentially lead to the selection of the wrong model structure. Including th=

e time-course like in PKPD modeling approaches would detect that the deviat=

ion is just noise, and thus probably be able to identify the right model st=

ructure despite this.

MCP-Mod can handle longitudinal data, see Pinheiro et al. (2014), Stat Med.=

33,1646-61 for one example, which is also available in the DoseFinding R p=

ackage.

· Uses dose-response models instead of exposure-response models

Correct. Again, MCP-Mod is not intended to replace population-PK/PD modelin=

g. We have started thinking how to extend the key ideas of MCP-Mod to expos=

ure-response models and encourage the community to look into this.

· Pre-specifies the model structure. While I understand that for =

pivotal trials prespecification is crucial, I would assume that Phase 2 is =

performed to allow exploration of the data to come up with the best model g=

iven the data we have. What happens if the true model is not part of the te=

sted ones? What if we have new physiological insights that tell us about th=

e model structure after we have seen the data? Do we then ignore what we kn=

ow and fit all bad models, and if none gives a good description we do model=

averaging of bad models?

Excellent questions. Candidate models for MCP-Mod should always be selected=

based on entire teams input and operating characteristics should be evalua=

ted upfront. More specifically, our experience shows that MCP-Mod is relati=

vely robust if the true model is not part of the tested ones, see for examp=

le Pinheiro et al. (2006), J. Biopharm. Statist. 16,639-656. This is also s=

omething that can be evaluated to some extend upfront (at the design stage)=

by simulations.

Among other things one advantage of pre-specification is that it makes the =

modelling more transparent/credible for externals (e.g. health authorities)=

, if one specifies before seeing the data what will be done. But of course =

there is a trade-off: Not sure if it is possible to pre-specify a full popu=

lation PK/PD analysis.

· If we include a model with many parameters in the prespecificat=

ion and only have a few dose strength, wouldn't the model with more paramet=

ers be more likely to give a good fit (e.g. when comparing Emax to logistic=

), with the consequence that a wrong dose might be selected?

Not sure whether I fully understand this question. Of course the model-sele=

ction/averaging step of MCP-Mod would take into account the model complexit=

y by using AIC/BIC (not only looking at model fit). Again, operating charac=

teristics need to be evaluated in advance, which include precision of targe=

t dose estimation and also possible convergence problems if the number of p=

arameters is to larger.

Colleagues from statistics recommend to cover all potential models with dif=

ferent shapes in the candidate set to avoid potential bias in dose selectio=

n, but they argue that post-hoc model fitting leads to data-dredging and ov=

er-fitting, does not account for model uncertainty and gives overly-optimis=

tic results. I am wondering however what the difference in the approach is =

if anyway ALL potential models are considered (which can lead to overfittin=

g as well)?

There is a penalty for using many models in MCP-Mod: In the MCP step the mu=

ltiplicity adjustment would get higher if there are more models included (i=

n particular if they are very different).

In the Mod step the variance of the dose-response curve would increase with=

an increased number of models, so there one faces the usual variance/bias =

trade-off.

Might a good solution be to combine PKPD modeling with MCP-Mod?

Yes, see above

Your opinion will be highly appreciated, and I am looking forward to receiv=

ing comments both in favour and against the approach :-)

Best

Nele

From: owner-nmusers

Behalf Of Mueller-Plock, Nele

Sent: Friday, March 20, 2015 1:02 PM

To: nmusers

Subject: [NMusers] Using MCP-MOD in dose finding for Phase 3

Dear all,

I am writing to you as we are currently discussing the implementation of th=

e MCP-MOD approach for dose finding based on Phase 2B results and would lik=

e to hear your opinion on this approach. It would be good to get feedback f=

rom both statisticians and classical modelers.

I have thought about the approach, and have a few problems about seeing the=

advantage of the approach over complete population-PK/PD modeling. From wh=

at I understood, I can see the following issues:

MCP-MOD

· Only uses trial endpoints, i.e. it ignores the time course of t=

he treatment effect. I have a problem with this because there might be nois=

e in the endpoint (e.g. if the effect has reached a plateau), which might p=

otentially lead to the selection of the wrong model structure. Including th=

e time-course like in PKPD modeling approaches would detect that the deviat=

ion is just noise, and thus probably be able to identify the right model st=

ructure despite this.

· Uses dose-response models instead of exposure-response models

· Pre-specifies the model structure. While I understand that for =

pivotal trials prespecification is crucial, I would assume that Phase 2 is =

performed to allow exploration of the data to come up with the best model g=

iven the data we have. What happens if the true model is not part of the te=

sted ones? What if we have new physiological insights that tell us about th=

e model structure after we have seen the data? Do we then ignore what we kn=

ow and fit all bad models, and if none gives a good description we do model=

averaging of bad models?

· If we include a model with many parameters in the prespecificat=

ion and only have a few dose strength, wouldn't the model with more paramet=

ers be more likely to give a good fit (e.g. when comparing Emax to logistic=

), with the consequence that a wrong dose might be selected?

Colleagues from statistics recommend to cover all potential models with dif=

ferent shapes in the candidate set to avoid potential bias in dose selectio=

n, but they argue that post-hoc model fitting leads to data-dredging and ov=

er-fitting, does not account for model uncertainty and gives overly-optimis=

tic results. I am wondering however what the difference in the approach is =

if anyway ALL potential models are considered (which can lead to overfittin=

g as well)?

Might a good solution be to combine PKPD modeling with MCP-Mod?

Your opinion will be highly appreciated, and I am looking forward to receiv=

ing comments both in favour and against the approach :-)

Best

Nele

______________________________________________________________

Dr. Nele Mueller-Plock, CAPM

Associate Scientific Director Pharmacometrics

Global Pharmacometrics

Translational Medicine

Takeda Pharmaceuticals International GmbH

Thurgauerstrasse 130

8152 Glattpark-Opfikon (Zürich)

Switzerland

Visitor address:

Alpenstrasse 3

8152 Glattpark-Opfikon (Zürich)

Switzerland

Phone: (+41) 44 / 55 51 404

Mobile: (+41) 79 / 654 33 99

mailto: nele.mueller-plock

http://www.takeda.com<http://www.takeda.com/>

--------------------------------------------------------------------

The content of this email and of any files transmitted may contain confiden=

tial, proprietary or legally privileged information and is intended solely =

for the use of the person/s or entity/ies to whom it is addressed. If you h=

ave received this email in error you have no permission whatsoever to use, =

copy, disclose or forward all or any of its contents. Please immediately no=

tify the sender and thereafter delete this email and any attachments.

--------------------------------------------------------------------

Received on Mon Mar 23 2015 - 08:54:04 EDT