NONMEM Users Network Archive

Hosted by Cognigen

Re: AMD vs Intel

From: Rikard Nordgren <rikard.nordgren>
Date: Tue, 19 Nov 2019 10:32:52 +0100

Hi Leonid,

When upgrading from gfortran 4.4.7 to 5.1.1 we ran around 20 models with bo=
th compilers and turning off the -ffast-math. The runs where on the same ha=
rdware. The differences in the parameter estimates and OFV were in general =
small. One big difference we could see was that the success of the covarian=
ce step was seemingly random. It could succeed on one compiler version, but=
 not the other and it could also start failing when the option was turned o=
ff. I have kept the runs, so let me know if you would be interested. I also=
 started some experiments using machine dependent compiler flags, but as ou=
r cluster is heterogeneous I abandoned this testing.

I think that getting identical results could be possible, but that it would=
 be quite a challenge. There are many components that affect the results. T=
he compiler, the compiler flags, the libc implementation, the hardware and =
sometimes the operating system. To see for example where the standard libra=
ries comes into play you can do nm nonmem on the nonmem executable (in linu=
x) to list all symbols compiled in. Some are function from external librari=
es, for example my exponential function is from libc: exp
rings could introduce rounding errors since the text representation is deci=
mal and the internal floating point number is binary.

Best regards,
Rikard Nordgren

Rikard Nordgren
Systems developer

Dept of Pharmaceutical Biosciences
Faculty of Pharmacy
Uppsala University
Box 591
75124 Uppsala

Phone: +46 18 4714308<http://www.farmb=>

On 2019-11-18 23:54, Leonid Gibiansky wrote:
Hi Jeroin,

Thanks for your input, very interesting. As far as the goal is concerned, I=
 am mostly interested to find options that would give identical results on =
two platform rather than in speed. So far no luck: 4 combinations of gfortr=
an / Intel compilers on Xeon / AMD processors give 4 sets of results that a=
re close but not identical.

Related question to the group: have anybody experimented with gfortran opti=
ons (rather than using default provided by Nonmem distribution)? Any recomm=
endations? Same goal: maximum reproducibility across different OSs, paralle=
lization options, and processor types.


On 11/18/2019 5:28 PM, Jeroen Elassaiss-Schaap (PD-value B.V.) wrote:
Hi Leonid,

"A while" back we compared model development trajectories and results betwe=
en two computational platforms, Itanium and Xeon, see The results roughly were: 1/3 equal, 1/3 rounding=
 differences and 1/3 real different results. From discussions with the tech=
nical knowledgeable people I worked with at the time, I recall that there a=
re three levels/sources for those differences:

1) computational (hardware) platform

2) compilers (+ optimization settings)

3) libraries (floating point handling does matter)

Assuming you would like to compare the speed of the platforms wrt NONMEM, m=
y advice would be to test a large series of different models, from simple A=
DVAN1 or 2 to complex ODE, ranging from FO to LAPLACIAN INT NUMERICAL, whil=
e keeping compilers and libraries the same. Also small and large datasets, =
as in some instances you might be testing only the L1/L2/L3 cache strategie=
s and Turbo settings. And with and without parallelization - as that might =
determine runtime bottlenecks in practice.

Just having a peek at Epyc - seems interesting (noticed results w gcc7.4 co=
mpilation). As long as you are able to hold the computation in cache, a big=
 if for the 64-core, there might be an advantage.

All in all I am not sure that it is worth the trouble. For any given PK-PD =
model there is a lot you can tune to gain speed, but the optimal settings m=
ight be very different for the next and overrule any platform differences.

Hope this helps,

+31 6 23118438
-- More value out of your data!

On 18/11/19 6:34 pm, Leonid Gibiansky wrote:
Thanks Bob and Peter!

The model is quite stable, but this is LAPLACIAN, so requires second deriva=
tives. At iteration 0, gradients differ by about 50 to 100% between Intel =
and AMD. This leads to differences in minimization path, and slightly diffe=
rent results. Not that different to change the recommended dose, but suffic=
iently different to notice (OF difference of 6 points; 50% more model evalu=
ations to get to convergence).

On 11/18/2019 12:15 PM, Bonate, Peter wrote:
Leonid - when you say different. What do you mean? Fixed effect and rando=
m effects? Different OFV?

We did a poster at AAPS a decade or so ago comparing results across differe=
nt platforms using the same data and model. We got different results on th=
e standard errors (which related to matrix inversion and how those are done=
 using software-hardware configurations). And with overparameterized model=
s we got different error messages - some platforms converged with no proble=
m while some did not converge and gave R matrix singularity.

Did your problems go beyond this?


Peter Bonate, PhD
Executive Director
Pharmacokinetics, Modeling, and Simulation
1 Astellas Way, N3.158
Northbrook, IL 60062
(224) 205-5855

Details are irrelevant in terms of decision making - Joe Biden.

-----Original Message-----
From: owner-nmusers
Of Leonid Gibiansky
Sent: Monday, November 18, 2019 11:05 AM
To: nmusers <nmusers
Subject: [NMusers] AMD vs Intel

Dear All,

I am testing the new Epyc processors from AMD (comparing with Intel Xeon), =
and getting different results. Just wondering whether anybody faced the pro=
blem of differences between AMD and Intel processors and knows how to solve=
 it. I am using Intel compiler but ready to switch to gfortran or anything =
else if this would help to get identical results.
There were reports of Intel slowing the AMD execution in the past, but in m=
y tests, speed is comparable but the results differ.


När du har kontakt med oss på Uppsala universitet med e-post s=
innebär det att vi behandlar dina personuppgifter. För att l=
äsa mer om hur vi gör det kan du läsa här: http://www.u=

E-mailing Uppsala University means that we will process your personal data.=
 For more information on how this is performed, please read here: http://ww=

Received on Tue Nov 19 2019 - 04:32:52 EST

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: