• Home
• Downloads
• Users
• Members
• Committers
• Resources
• Projects
• About Us

• A A

Useful Information about LPG

This page contains various sorts of useful information about LPG, the LALR parser generator included with IMP. (LPG was formerly known as JikesPG.) LPG has its own SourceForge project.

IMP does not require any particular parser-related technology, but it does need a way of producing a model (AST) for the source text. We've adapted LPG to support the parser- and AST-related interfaces on which IMP depends, and we have made it easy to use LPG through IMP. For instance, there are IMP wizards for using LPG to generate a parser and AST, and we have provided an IMP-based IDE to facilitate the editing of LPG grammar files.

This page addresses the following topics:

• LPG Quickstart for IMP Users
  
• Running LPG as an External Tool
  
• LPG Documentation
  
• Troubleshooting
  

LPG Quickstart for IMP Users

The IMP installation includes a version of LPG for generating a lexer, parser and related components in support of the instantiation of an IMP IDE for a new programming language. The lexer and parser are generated from "grammar" files that specify the lexer and parser grammars for the language. IMP uses LPG with three grammar files: a "KeywordLexer.gi" file (".gi" stands for "grammar include"), which specifies the keywords of the language, a "Lexer.gi" file, which specifies the general lexical grammar (given the keyword lexer), and a "Parser.g" file (".g" stands for "grammar), which specifies the parser garmmar (given the general lexer).

When the "New LPG Parser" wizard is run on a IMP project, it installs an LPG builder in the project, creates a parser package in the source folder of the project, and copies in three default grammar files. The builder then generates Java classes for the lexer, parser, and related components, using templates provided with LPG and IMP.

A brief overview of the sections of the LPG grammar files that IMP users may encounter (though note that not all sections will occur in all files):

• The "%Options" section includes options for LPG.
• The "%Define" section is used to define macro symbols that are used by LPG itself. The macros may be defined to include other symbols or source texts that will be substituted for subsequent occurrences of the symbols. If a symbol is defined more than once, an earlier definition is overridden by a later one.
• The "%Globals" section is intended for material that should appear at the head of a generated file, such as Java "import" directives.
• The"%Headers" and "%Trailers" sections include members that are to be incorporated (respectively) at the beginning or end of the generated parser class (more specifically, before or after the expansion of the action blocks).
• An "%Include" section causes material to be interpolated into the input file at the point at which the %Include occurs.
• The "%Export" section defines the token types recognized by the lexer or parser (in effect, the output elements)
• The "%Terminals" section defines the terminal symbols recognized by the lexer or parser (in effect, the input elements)
• The "%Start" section gives the root symbol for the processing of grammar rules. In the parser grammar it is typically something like "program"
  
• The "%Rules" section defines the grammatical rules that map the terminals in to the exported tokens

What do these mean for the IMP user?

• The %Export, %Terminals, and %Rules sections contain the representation of the grammar. These will be the focus of user customizations. The rules may include arbitrary Java code (default delimiters "/." and "./"). The inclusion of such code may entail additions of supporting material to the %Globals and %Headers sections.
• LPG is confgurable with many different options. The generated grammar-file skeletons are given a basic set of options that are sufficient for generating a IMP-ready parser and these options should not generally need to be edited. (Once you have attained some mastery of LPG by other means, then you may wish to edit the generated options if it suits your purposes.)
• IMP makes some use of the %Globals section to add import statements for interfaces that the generated classes will implement. IMP does not by default make any use of the %Headers or %Trailers sections. However, if special-processing code is included in the grammar rules, then the %Globals, %Headers, and %Trailers sections can be used to add imports, fields, methods, and so on in support of that code.
• IMP uses the %Define section to define macros that are used elsewhere in the grammar files to more completely parameterizes the templates. It is not generally necessary for the IMP user to add definitions (although exerienced LPG users may wish to do so).
• IMP makes some use of %Include sections to incorporate portions of grammar specifications that have been put into separate files. There is not any necessary reason for IMP users to customize the %Include sections, unless this is needed to support special-purpose language processing.
  

Running LPG As an External Tool

LPG can be run manually in Eclipse as an external tool. To be able to do this, you must have the plugin project lpg.generator and the appropriate platform-specific plugin fragment (e.g., lpg.generator.macosx86) imported into your workspace. These projects contain the LPG executable and the platform-independent template/include files, as well as the Eclipse launch configuration (in the folder ".launchConfigurations). When this project is in your workspace, the launch configuration should be recognized automatically and should be available through the External Tools menu. (To open the External Tools dialog, navigate "Run" -> "External Tools" -> "External Tools ..."; you should then find LPG under "Programs")

Important note on system compatibility: As of March 2010, the only platforms for which IMP distributes LPG are Windows, Mac OS X (32- and 64-bit) and Linux (32- and 64-bit). However, you should be able to create an executable for any system that has an ANSI C compiler and POSIX-compliant 'make' by downloading the source code from the LPG project on SourceForge and building it using make. Alternatively, if your system supports the execution of another O.S. environment within a virtual machine, you may be able to run one of the provided executables under that (see Troubleshooting below).

You can also run LPG manually as an external tool on a grammar specification file (".g" or ".gi" file). To do so, select the grammar file and then invoke LPG from the External Tools dialog. If you run LPG on a ".g" file, LPG will generate a complete set of tools (parsre, lexer, and keyword filter). If you run LPG on a ".gi" file, you can generate just the lexer or keyword filter (depending on the specific file).

The default parameters of the launch configuration for use with IMP are as follows:

• Location: This is the location of the LPG executable:

  ${workspace_loc}/lpg.generator.${target.os}_${target.arch}/lpgexe/lpg-${target.os}_${target.arch}

  Note that this will use the executable located in the appropriate platform-specific plugin fragment.

• Working directory: This is the location in which the input source will be found and the output specifications generated. It is set relative to the selected grammar file.

  ${container_loc}

• Arguments: This tells LPG where to find the templates to use in generating the parser, lexer, and keyword filter:

  -include-directory=${workspace_loc:/lpg.generator/templates/java};
  ${workspace_loc:/lpg.generator/include/java} ${resource_loc}
  

If your workspace is configured differently from what the default presumes, or if you have renamed the relevant projects, then you will need to adjust these parameters accordingly.

LPG Documentation

There is documentation for LPG on the LPG website.

Troubleshooting:

        #/bin/bash
        `echo wine $0 $* | sed "s/lpg-linux_x86/lpg-win32_x86.exe/" | sed
        "s/\"-include/-include/" | sed "s/'\"/'/"`
        

        $ chmod u+x $ECLIPSE_HOME/plugins/lpg.runtime_2.0.6/lpgexe/lpg-linux_x86