Re: [NMusers] Covariate modelling question

From: Fiona Vanobberghen <fiona.vanobberghen_at_unibas.ch>
Date: Thu, 26 Feb 2015 11:14:51 +0000

Hi Kajsa

Many thanks for this quick and helpful feedback. I have looked at the
txt files you suggested, and they both have retries=1. Does that mean it
has one attempt (and no real "retries"), or rather that it has a first
attempt and max 1 retry? If the latter then it seems that this might be
the reason, based on your explanation below. And if so then is there a
way of setting the retries using the wizard (ie retries=0) so that I can
double check that in that case I get the results I'm expecting?

Many thanks
Fiona


On 26/02/2015 10:55, Kajsa Harling wrote:
> I have looked at the output now, and am writing the answer to the
> mailing list in case more people are interested.
>
> In the two different scm runs you get slightly different ofv values,
> and the small differences lead to different models being selected in
> the two runs. So why are the ofv:s different? One possible reason is
> that you used the retries option in PsN (automatic perturbation of
> initial estimates when a run is not successful) which will give
> different initial estimates depending on the random seed. You can
> check the settings of all PsN options, including the ones set by
> default, in the version_and_option_info.txt file in the top level of
> the scm run directory.
>
> Best regards,
> Kajsa
>
> On 02/26/2015 11:28 AM, Kajsa Harling wrote:
>> Dear Fiona,
>>
>> This sounds like a PsN question, but it is impossible to answer
>> without the example output. The mailing list does not accept long
>> messages or attachments, so I suggest you send the original email
>> with all output directly to me (kajsa.harling_at_farmbio.uu.se) so that
>> I can have a look.
>>
>> Best regards,
>> Kajsa
>>
>>
>> On 02/26/2015 11:00 AM, Fiona Vanobberghen wrote:
>>> I posted this message a few days ago but it doesn't seem to have
>>> been sent to the list - so I'm resending without the example output.
>>> Best wishes
>>> Fiona
>>>
>>> --
>>> Dear all
>>>
>>> I am attempting to do some covariate modelling, using the scm wizard
>>> in Pirana. I have seen some results which I wasn't expecting and
>>> would be grateful if anyone could shed any light on it for me.
>>>
>>> Initially, I used a forward inclusion p value of 0.1 and a backward
>>> elimination p value of 0.05. This resulted in quite a complex
>>> (implausible) model (we do have a reasonably large dataset), and I
>>> decided to be more stringent, using p<0.05 for inclusion (and the
>>> same p>0.05 for elimination at the last step). As a shortcut, I
>>> could see from the output from the first attempt (with p<0.1) what I
>>> expected the final model to look like if I were to run it again with
>>> p<0.05, ie where the process would truncate. Just to double check
>>> (and verify that nothing would be eliminated at the last step), I
>>> re-ran the scm wizard with the more stringent p<0.05. And the
>>> results are not what I expected... Below I have pasted the output
>>> for the first few forward steps from each attempt. The results are
>>> essentially the same up until the third step, although we see some
>>> small differences in the OFV creeping in from the second step.
>>> However, at the fourth step, the results are completely different.
>>> This isn't what I was expecting, based on my understanding of the
>>> model selection process. Is this a known behaviour? Has anyone
>>> experienced this problem and/or know why these differences might
>>> occur? I'd be grateful for any advice.
>>>
>>> Many thanks in advance for your help.
>>>
>>> Best wishes
>>> Fiona
>>>
>>>
>>> --
>>> *Fiona Vanobberghen (née Ewings), PhD*
>>> Swiss Tropical and Public Health Institute
>>> Socinstrasse 57, 4051, Basel, Switzerland
>>> Tel: +41 61 284 87 41
>>>
>>>
>>
>> --
>> -----------------------------------------------------------------
>> Kajsa Harling, PhD
>> System Developer
>> Department of Pharmaceutical Biosciences
>> Uppsala University
>>
>> Kajsa.Harling_at_farmbio.uu.se
>> +46-(0)18-471 4308
>>
>> http://www.farmbio.uu.se/research/researchgroups/pharmacometrics/
>> -----------------------------------------------------------------
>
> --
> -----------------------------------------------------------------
> Kajsa Harling, PhD
> System Developer
> Department of Pharmaceutical Biosciences
> Uppsala University
>
> Kajsa.Harling_at_farmbio.uu.se
> +46-(0)18-471 4308
>
> http://www.farmbio.uu.se/research/researchgroups/pharmacometrics/
> -----------------------------------------------------------------


Received on Thu Feb 26 2015 - 06:14:51 EST

This archive was generated by hypermail 2.3.0 : Fri Sep 27 2019 - 16:43:21 EDT