RE: [NMusers] Humongous log file with parallel NONMEM

From: <E.Olofsen_at_lumc.nl>
Date: Mon, 22 Jun 2015 18:23:52 +0000

Dear Bill and Ron,

I was thinking about:

$ ln -s /dev/null <control_stream>.log

(After deleting the log if one already exists)

Data written to the log file is actually written to the null device, which =
discards the data - two tests seem to indicate that it could work.

Best regards,

Erik
________________________________
From: owner-nmusers_at_globomaxnm.com [owner-nmusers_at_globomaxnm.com] on behalf=
 of Ron Keizer [ron_at_pirana-software.com]
Sent: Monday, June 22, 2015 7:32 PM
To: Bill Gillespie
Cc: nmusers_at_globomaxnm.com
Subject: Re: [NMusers] Humongous log file with parallel NONMEM

hi Bill,

a simple hack is to delete the file continually, by running the following c=
ommand before you start NONMEM:

watch -n 60 'find . -name *.log -delete' &>/dev/null &

Notes:
- will delete all log files every minute
- I'm using 'find ...' instead of just 'rm *.log' here to ensure that log-f=
iles in subfolders will also be deleted, e.g. useful when using via PsN
- run the command only once, the watch process will stay active
- only works on linux

best regards,
Ron


----------------------------------------------
Ron Keizer, PharmD PhD
Pirana Software & Consulting BV
California / the Netherlands
www.pirana-software.com<http://www.pirana-software.com/>
----------------------------------------------

On Mon, Jun 22, 2015 at 6:22 AM, Bill Gillespie <bgillespiester_at_gmail.com<m=
ailto:bgillespiester_at_gmail.com>> wrote:
Hi all,

I'm running NONMEM (METHOD = BAYES) in parallel on 32 cores and it genera=
tes a humongous log file with repeated entries like the following:

 ITERATION -577
 STARTING SUBJECTS 1 TO 4 ON MANAGER: OK
 STARTING SUBJECTS 5 TO 8 ON WORKER1: OK
 STARTING SUBJECTS 9 TO 11 ON WORKER2: OK
 STARTING SUBJECTS 12 TO 15 ON WORKER3: OK
 STARTING SUBJECTS 16 TO 18 ON WORKER4: OK
 STARTING SUBJECTS 19 TO 20 ON WORKER5: OK
 STARTING SUBJECTS 21 TO 24 ON WORKER6: OK
 STARTING SUBJECTS 25 TO 27 ON WORKER7: OK
 STARTING SUBJECTS 28 TO 29 ON WORKER8: OK
 STARTING SUBJECTS 30 TO 32 ON WORKER9: OK
 STARTING SUBJECTS 33 TO 35 ON WORKER10: OK
 STARTING SUBJECTS 36 TO 39 ON WORKER11: OK
 STARTING SUBJECTS 40 TO 42 ON WORKER12: OK
 STARTING SUBJECTS 43 TO 46 ON WORKER13: OK
 STARTING SUBJECTS 47 TO 50 ON WORKER14: OK
 STARTING SUBJECTS 51 TO 53 ON WORKER15: OK
 STARTING SUBJECTS 54 TO 58 ON WORKER16: OK
 STARTING SUBJECTS 59 TO 62 ON WORKER17: OK
 STARTING SUBJECTS 63 TO 66 ON WORKER18: OK
 STARTING SUBJECTS 67 TO 70 ON WORKER19: OK
 STARTING SUBJECTS 71 TO 71 ON WORKER20: OK
 STARTING SUBJECTS 72 TO 74 ON WORKER21: OK
 STARTING SUBJECTS 75 TO 77 ON WORKER22: OK
 STARTING SUBJECTS 78 TO 80 ON WORKER23: OK
 STARTING SUBJECTS 81 TO 84 ON WORKER24: OK
 STARTING SUBJECTS 85 TO 86 ON WORKER25: OK
 STARTING SUBJECTS 87 TO 88 ON WORKER26: OK
 STARTING SUBJECTS 89 TO 90 ON WORKER27: OK
 STARTING SUBJECTS 91 TO 93 ON WORKER28: OK
 STARTING SUBJECTS 94 TO 96 ON WORKER29: OK
 STARTING SUBJECTS 97 TO 99 ON WORKER30: OK
 STARTING SUBJECTS 100 TO 103 ON WORKER31: OK
 COLLECTING SUBJECTS 1 TO 4 ON MANAGER
 COLLECTING SUBJECTS 5 TO 8 ON WORKER1
 COLLECTING SUBJECTS 9 TO 11 ON WORKER2
 COLLECTING SUBJECTS 12 TO 15 ON WORKER3
 COLLECTING SUBJECTS 16 TO 18 ON WORKER4
 COLLECTING SUBJECTS 19 TO 20 ON WORKER5
 COLLECTING SUBJECTS 21 TO 24 ON WORKER6
 COLLECTING SUBJECTS 25 TO 27 ON WORKER7
 COLLECTING SUBJECTS 28 TO 29 ON WORKER8
 COLLECTING SUBJECTS 30 TO 32 ON WORKER9
 COLLECTING SUBJECTS 33 TO 35 ON WORKER10
 COLLECTING SUBJECTS 36 TO 39 ON WORKER11
 COLLECTING SUBJECTS 40 TO 42 ON WORKER12
 COLLECTING SUBJECTS 43 TO 46 ON WORKER13
 COLLECTING SUBJECTS 47 TO 50 ON WORKER14
 COLLECTING SUBJECTS 51 TO 53 ON WORKER15
 COLLECTING SUBJECTS 54 TO 58 ON WORKER16
 COLLECTING SUBJECTS 59 TO 62 ON WORKER17
 COLLECTING SUBJECTS 63 TO 66 ON WORKER18
 COLLECTING SUBJECTS 67 TO 70 ON WORKER19
 COLLECTING SUBJECTS 71 TO 71 ON WORKER20
 COLLECTING SUBJECTS 72 TO 74 ON WORKER21
 COLLECTING SUBJECTS 75 TO 77 ON WORKER22
 COLLECTING SUBJECTS 78 TO 80 ON WORKER23
 COLLECTING SUBJECTS 81 TO 84 ON WORKER24
 COLLECTING SUBJECTS 85 TO 86 ON WORKER25
 COLLECTING SUBJECTS 87 TO 88 ON WORKER26
 COLLECTING SUBJECTS 89 TO 90 ON WORKER27
 COLLECTING SUBJECTS 91 TO 93 ON WORKER28
 COLLECTING SUBJECTS 94 TO 96 ON WORKER29
 COLLECTING SUBJECTS 97 TO 99 ON WORKER30
 COLLECTING SUBJECTS 100 TO 103 ON WORKER31

The result is a lot of disk I/O and a file in the GB+ range. It dwarfs the =
file containing the MCMC samples. Is there some way to suppress that file o=
r reduce what gets written to it?

Thanks,
Bill

William R Gillespie, VP Strategic Modeling & Simulation
Metrum Research Group LLC
2 Tunxis Road, Tariffville, CT 06081
Direct & FAX: 919-371-2786<tel:919-371-2786>, Main: 860-735-7043<tel:860-73=
5-7043>
billg_at_metrumrg.com<mailto:billg_at_metrumrg.com>
www.metrumrg.com<http://www.metrumrg.com/>


Received on Mon Jun 22 2015 - 14:23:52 EDT

This archive was generated by hypermail 2.3.0 : Fri Sep 27 2019 - 16:44:52 EDT