NONMEM Users Network Archive

Hosted by Cognigen

RE: [NMusers] Context-free lexer for NM-TRAN

From: Bill Denney <wdenney_at_humanpredictions.com>
Date: Thu, 14 Jun 2018 09:17:56 -0400

Hi Ruben,



I’m also interested in a lexer-parser for NONMEM. The regexp-based=
 ones
that I’ve used have typically had issues (I’ve tried about =
4 different ones
including one that I wrote), and they are working for many but not all
models. I’m unaware of a reasonably complete lexer-parser for NONM=
EM
(though I know of at least one non-public effort; I’ve contacted th=
at
author to see if he is interested in joining this conversation).



I’ve wanted to build the abstract syntax tree for NONMEM to help wi=
th
computational model-building, and I’ve been looking into ANTLR as w=
ell.
Three questions: Are you interested in collaborating on the parser (can
you create a GitHub project for it)? Why ANTLRv3 instead of v4? Do you
have a way to get an ANTLR parse tree into R?



Thanks



Bill



*From:* owner-nmusers_at_globomaxnm.com <owner-nmusers_at_globomaxnm.com> *On
Behalf Of *Ruben Faelens
*Sent:* Thursday, June 14, 2018 8:55 AM
*To:* Tim Bergsma <Tim.Bergsma_at_certara.com>
*Cc:* nmusers_at_globomaxnm.com
*Subject:* Re: [NMusers] Context-free lexer for NM-TRAN



Hi Tim,



Thanks for pointing to that.

Unfortunately, nonmemica uses regular expressions to simply split the
character stream into subsections.

This is not the way to go. As an example, nonmemica would get confused by
the following input:

$PROBLEM This is a problem with special $PK section

$PK ;Refer to $ERROR for more information

CL=THETA(1)

$ERROR

Y = W*F



Probably a contextual lexer is the way to go; fortunately ANTLRv3 has
functionality for this.



Kind regards,

Ruben



On Thu, Jun 14, 2018 at 12:42 PM Tim Bergsma <Tim.Bergsma_at_certara.com>
wrote:



Hi Ruben.



Related: the CRAN package “nonmemica” has a function as.mod=
el() that parses
NONMEM control streams. Type “?nonmemica” at the R prompt a=
fter loading.
See also https://github.com/MikeKSmith/rspeaksnonmem . Happy to discuss
further.



Kind regards,



Tim



*Tim Bergsma, PhD*

Associate Director

Certara Strategic Consulting

[image: image001.png]

m. 860.930.9931 <(860)%20930-9931>

tim.bergsma_at_certara.com



*From:* owner-nmusers_at_globomaxnm.com <owner-nmusers_at_globomaxnm.com> *On
Behalf Of *Ruben Faelens
*Sent:* Thursday, June 14, 2018 4:33 AM
*To:* nmusers_at_globomaxnm.com
*Subject:* [NMusers] Context-free lexer for NM-TRAN



Hi all,



Calling all computer scientists and computer language experts.

In my spare time, I am working on a lexer and parser for NM-Tran. Primarly
to teach myself about grammars and DSL, but perhaps something useful will
come out of this (e.g. a context-sensitive editor with code completion).



When lexing, I am having a hard time describing the keywords used by
nm-tran.

Let us take '.EQ.' as an example.

1) It seems that *.EQ. *is a keyword used to describe a comparison.

2) However, a filename could also be 'foo.eq.bar'

The same thing applies for keywords on the '$ESTIMATION' record. These
keywords could also be used as variable names.



Am I right in saying that NM-TRAN cannot be tokenized with a context-free
lexer? And that I should focus my efforts on building a lexer-less parser?
(Or building my own lexer-parser, see
https://en.wikipedia.org/wiki/The_lexer_hack )

I assume building a parser for NM-TRAN was already done in the DDMoRe
project, but I failed to find the source code...



Kind regards,

Ruben Faelens



*NOTICE: *The information contained in this electronic mail message is
intended only for the personal and confidential use of the designated
recipient(s) named above. This message may be an attorney-client
communication, may be protected by the work product doctrine, and may be
subject to a protective order. As such, this message is privileged and
confidential. If the reader of this message is not the intended recipient
or an agent responsible for delivering it to the intended recipient, you
are hereby notified that you have received this message in error and that
any review, dissemination, distribution, or copying of this message is
strictly prohibited. If you have received this communication in error,
please notify us immediately by telephone and e-mail and destroy any and
all copies of this message in your possession (whether hard copies or
electronically stored copies). Thank you.

Personal data may be transferred to the United States of America and, if
this occurs, it is possible that US governmental authorities may access
such personal data.


buSp9xeMeKEbrUze



image001.png
(image/png attachment: image001.png)

Received on Thu Jun 14 2018 - 09:17:56 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request_at_iconplc.com. Once subscribed, you may contribute to the discussion by emailing: nmusers_at_globomaxnm.com.