• Home
• Users
• Members
• Committers
• Downloads
• Resources
• Projects
• About Us

• A A

Useful Information about LPG

This page contains various sorts of useful information about LPG, the LALR parser generator included with IMP.  (LPG was formerly known as JikesPG and you may still find references to JikesPG in the IMP documentation and code.)  LPG also has its own SourceForge project at http://sourceforge.net/projects/lpg/.

IMP does not require any particular parser-related technology, but it does require support for parsing and AST representation.  We like LPG and have adapted it to support the parser- and AST-related interfaces on which IMP depends, and we have made it easy to use LPG through IMP.  For instance, there are IMP wizards for using LPG to generate a parser and AST, and we have provided an IMP-based IDE to facilitate the editing of LPG grammar files.

This section addresses the following topics:
LPG Quickstart for IMP Users
Running LPG as an External Tool
LPG Documentation
Troubleshooting

LPG Quickstart for IMP Users

The IMP installation includes a version of LPG for generating a lexer, parser and related components in support of the instantiation of a IMP IDE for a new programming language.  The lexer and parser are generated from "grammar" files that specify the lexer and parser grammars for the language.  IMP uses LPG with three grammar files:  a "KeywordLexer.gi" file (".gi" stands for "grammar include"), which specifies the keywords of the language, a "Lexer.gi" file, which specifies the general lexical grammar (given the keyword lexer), and a "Parser.g" file (".g" stands for "grammar), which specifies the parser garmmar (given the general lexer).

When the New Programming Language wizard is run on a IMP project, it installs a LPG builder in the project, creates a parser package in the source folder of the project, and copies in three default grammar files.  The builder then generates Java classes for the lexer, parser, and related components, using templates provided with LPG and IMP.

A brief overview of the sections of the LPG grammar files that IMP users may encounter (not all sections will occur in all files):

• The "Options" section includes options for LPG.
• The "$Define" section is used to define macro symbols that are used by LPG itself.  The macros may be defined to include other symbols or source texts that will be substituted for subsequent occurrences of the symbols.  If a symbol is defined more than once, an earlier definition is overridden by a later one.
• The "$Globals" section is intended for material that should appear at the head of a generated file, such as Java "import" directives.
• The"$Headers" and "$Trailers" sections include members that are to be incorporated (respectively) at the beginning or end of the generated parser class  (more specifically, before or after the expansion of the action blocks).
  
• An "$Include" section causes material to be interpolated into the input file at the point at which the $Include occurs.
  
• The "$Export" section defines the token types recognized by the lexer or parser (in effect, the output elements)
• The "$Terminals" section defines the terminal symbols recognized by the lexer or parser (in effect, the input elements)
• The "$Start" section gives the root symbol for the processing of grammar rules.  In the parser grammar it is typically something like "program"
  
• The "$Rules" section defines the grammatical rules that map the terminals in to the exported tokens

What do these mean for the IMP user?

• The $Export, $Terminals, and $Rules sections contain the representation of the grammar.  These will be the focus of user customizations.  The rules may include arbitrary Java code (default delimiters "/." and "./").  The inclusion of such code may entail additions of supporting material to the $Globals and $Headers sections.
• LPG is confgurable with many different options.  The generated grammar-file skeletons are given a basic set of options that are sufficient for generating a IMP-ready parser and these options should not generally need to be edited.  (Once you have attained some mastery of LPG by other means, then you may wish to edit the generated options if it suits your purposes.)
• IMP makes some use of the $Globals section to add import statements for interfaces that the generated classes will implement.  IMP does not by default make any use of the $Headers or $Trailers sections.  However, if special-processing code is included in the grammar rules, then the $Globals, $Headers, and $Trailers sections can be used to add  imports, fields, methods, and so on in support of that code.
• IMP uses the $Define section to define macros that are used elsewhere in the grammar files to more completely parameterizes the templates.  It is not generally necessary for the IMP user to add definitions (although exerienced LPG users may wish to do so).
• IMP makes some use of $Include sections to incorporate portions of grammar specifications that have been put into separate files.  There is not any necessary reason for IMP users to customize the $Include sections, unless this is needed to support special-purpose language processing.
  

The LPG grammar templates used by IMP, along with the Java classes generated from these, usually give some examples and brief instructions about how the files should be completed for a new language.

Running LPG As an External Tool

LPG can be run manually in Eclipse as an external tool.  To be able to do this, you must have the project lpg.runtime.java imported into your workspace.  This project contains both the LPG executable (in the folder "lpgexe") and the Eclipse launch configuration (in the folder ".launchConfigurations).  When this project is in your workspace, the launch configuration should be recognized automatically and should be available through the External Tools menu.   (To open the External Tools dialog, navigate "Run" -> "External Tools" -> "External Tools ..."; you should then find LPG under "Programs")

Important note on system compatibility:  As of 18 Sep 2007 only the Windows version of LPG is being maintained and only the Windows version of the LPG executable is up-to-date.  You may be able to create an executable for other systems by downloading the source code from the LPG project on SourceForge (http://sourceforge.net/projects/lpg/) and building it locally.  Alternatively, if your non-Windows system has a Windows emulator, you may be able to run the Windows version of LPG under that (see Troubleshooting below).

To run LPG as an external tool on a specific grammar specification (".g" or ".gi" file), select the grammar file and then invoke LPG from the External Tools dialog.  If you run LPG on a ".g" file, LPG will generate a complete set of tools (parsre, lexer, and keyword filter).  If you run LPG on a ".gi" file, you can generate just the lexer or keyword filter (depending on the specific file).

The default parameters of the launch configuration for use with IMP are as follows:

• Location:  this is the location of the LPG executable:
  

• Working directory:  this is the location in which the input source will be found and the output specifications generated.  It is set relative to the selected grammar file.
  

• Arguments:  this tells LPG where to find the templates to use in generating the parser, lexer, and keyword filter:
  

(note that the second second argument refers to an IMP project that contains IMP-specific templates).

If your workspace is configured differently than what the default presumes, or if you have renamed the relevant projects, then you may need to adjust these parameters for your particular situation.

LPG Documentation

There is documentation for LPG on the LPG website at http://sourceforge.net/projects/lpg/.

Troubleshooting:

LPG build on Linux doesn't run correctly:  One LPG user tried building the LPG executable from source on a Linux system and got the same errors when he tried to use it as when he ran the Linux version of the executable that is provided in the LPG release from SourceForge (see the message thread "Problems with LPG parser", dated 9/17/2007, in the eclipse.technology.imp newsgroup on news.eclipse.org, started by Oleg Murk, olegm@gmail.com).  He was able to get the Windows version of LPG running on his system under the Windows emulator Wine: