| Re: [jdt-dev] parser generation automation? |
|
Hey Stephan, Thanks for sharing the script, that'll save me a ton of time. (FWIW, I translated it to English, with no further mods, attached.) I've downloaded jikes-1.22, will try firing it up on the current
ejc grammar, and will let you know how it goes. (Did any of the
bug fixes in your private copy address any functional defects e.g.
in code-generation, hence I might need them, or just stability?) One last issue: I've been scouring the web for any specification of jike's metagrammar, and haven't found any. Got a link? I *can* reverse-engineer it from the sources and the ejc and OT grammars, but not being a masochist, would really prefer to work from documentation 8-) . Thanks again! -rjs
On 10/16/2019 11:52 AM, Stephan
Herrmann wrote:
I personally use a variant of what we have in the Object Teams project at: |
#! / bin / bash
################################################# ################################################## ################################################# ######
# This file is part of "Object Teams Development Tooling" software
#
# Copyright 2003, 2006 Fraunhofer Gesellschaft, Munich, Germany,
# for its Fraunhofer Institute and Computer Architecture and Software
# Technology (FIRST), Berlin, Germany and Technical University Berlin,
# Germany.
#
# All rights reserved. This program and the accompanying materials
# are made available under the terms of the Eclipse Public License v1.0
# which accompanies this distribution, and is available at
# http://www.eclipse.org/legal/epl-v10.html
#
#
# Contributors:
# Fraunhofer FIRST - Initial API and implementation
# Technical University Berlin - Initial API and implementation
#
# This script extracts all the automatically generated files from the java.g file
# and copy these into the correct positions within the current source
# The script needs the "OTDTCore" folder
# The script requires the file "objectteams.g" which can be extracted from the Eclipse source code.
# At the beginning of the past, a copy of all files changed in the sources will be created
################################################# ################################################# ################################################# ######
JIKESPG_HOME = $ {JIKESPG_HOME: - / usr / bin}
export JIKESPG_HOME
JAVAC = $ {JAVAC: -java}
if test $ # -ne 2
then
echo "Attention: JIKESPG_HOME needs to be set correctly!"
echo "Call: generateOTParser.sh <grammar> <eclipse-project-OTDTCore directory>"
echo "Bsp: generateOTParser.sh objectteams.g OTDTCore"
echo "(look up the directory's directory)."
exit
fi
GRAMMAR = `readlink -f" $ 1 "`
SOURCE = `readlink -f" $ 2 "`
if test -f $ JIKESPG_HOME / jikespg
then
echo "jikespg be found"
else
echo "JIKESPG_HOME / jikespg could not be found"
exit
fi
if test -f $ GRAMMAR
then
echo "Use Grammar:" $ GRAMMAR
else
echo "Error: Grammar" $ GRAMMAR "could not be found!"
echo "Call: generateOTParser.sh <grammar> <eclipse-project-OTDTCore directory>"
exit
fi
if test -d $ SOURCE
then
echo "Using Eclipse Sources:" $ SOURCE
else
echo "Error: Eclipse resources" $ SOURCE "could not be found!"
echo "Call: generateOTParser.sh <grammar> <eclipse-project-OTDTCore directory>"
exit
fi
scriptDir = `dirname" $ 0 "`
pluginDir = `readlink -f $ {scriptDir} / ..`
echo "Initialum tidum"
if test -d output
then
rm -r output
fi
if test -d backup
then
rm -r backup
fi
echo "Empty Learning Directories"
mkdir output
mkdir backup
echo "Retrieve original files prior to customization"
if test -f $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / Parser.java
then
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / Parser.java output
else
exit
fi
if test -f $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / TerminalTokens.java
then
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / TerminalTokens.java output
else
exit
fi
if test -f $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / ParserBasicInformation.java
then
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / ParserBasicInformation.java output
else
exit
fi
echo "A copy of the original files that were changed in the backup folder"
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / Parser.java backup
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / TerminalTokens.java backup
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / ParserBasicInformation.java backup
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / parser? .rsc backup
cp $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser / readableNames.props backup
cp $ GRAMMAR output
################################################# ################################################# ################################################# ######
cd output
echo "Start parser generator"
islalr = `$ JIKESPG_HOME / jikespg $ GRAMMAR | tee /tmp/jikes.log | grasp grammar`
if ["$ islalr" = "This grammar is LALR (1)." ]
then
echo "Grammar is LALR (1)."
else
echo "Grammar is not correct!"
exit
fi
chmod 644 *
#echo "Rename constants of javasym.java"
cat javasym.java | sed -e s / TokenName \ $ eof / TokenNameEOF / g -e s / TokenName \ $ error / TokenNameERROR / g> javasym_tmp.java
rm javasym.java
etc. javasym_tmp.java javasym.java
cd ..
################################################# ################################################# ################################################# ######
cd output
echo "Cut away all unnecessary: javadef.java"
cat javadef.java | awk '{
if ($ 0 ~ / ^ [\ t] * {[\ t] * $ /) {
while ((getline nextline) == 1 && nextline! ~ / ^ [\ t] *}. * $ /) {
print nextline;
}
print "}";
}
}
'> javadef_tmp.java
echo "Cut away all unnecessary: ParserBasicInformation.java"
cat ParserBasicInformation.java | awk '{
if ($ 0 ~ /^.* {[\ t] * $ /) {
print $ 0;
while ((getline nextline) == 1 && nextline! ~ / ^ [\ t] *} [\ t] * $ /) {
nextline = "";
}
while ((getline nextline) == 1) {
print nextline;
}
} else {
print $ 0
}
}
'> ParserBasicInformation_tmp.java
echo "Copy the contents of javadef.java to ParserBasicInformation.java"
cat ParserBasicInformation_tmp.java javadef_tmp.java> ParserBasicInformation.java
cd ..
################################################# ################################################# ################################################# ######
cd output
echo "Cut away all unnecessary: JavaAction.java"
cat JavaAction.java | awk '{\
if ($ 0! ~ / ^ [\ t] * \ / \ / This method is part of an automatic generation: do NOT edit-modify [\ t] * $ /) {\
print $ 0 \
} \
} \
'> JavaAction_tmp.java
echo "Replace the ConsumeRule method against the Token TMP_CUT_OUT"
cat Parser.java | awk '{
if ($ 0 ~ / ^ [\ t] * protected void consumeRule \ (int act \) \ {[\ t] * $ /) {
while ((getline nextline) == 1 && nextline! ~ / ^ [\ t] *} [\ t] * $ /) {
nextline = "";
}
while ((getline nextline) == 1 && nextline! ~ / ^ [\ t] *} [\ t] * $ /) {
nextline = "";
}
print "TMP_CUT_OUT";
while ((getline nextline) == 1) {
print nextline;
}
} else {
print $ 0
}
}
'> Parser_tmp.java
cat Parser_tmp.java | awk '{
if ($ 0 ~ / ^ TMP_CUT_OUT $ /) {
while ((getline nextline) == 1) {
nextline = "";
}
} else {
print $ 0
}
}
'> Parser_1_tmp.java
cat Parser_tmp.java | awk '{
if ($ 0 ~ / ^ TMP_CUT_OUT $ /) {
while ((getline nextline) == 1) {
print nextline;
}
}
}
'> Parser_2_tmp.java
echo "Copy the contents of JavaAction.java to Parser.java"
cat Parser_1_tmp.java JavaAction_tmp.java Parser_2_tmp.java> Parser.java
cd ..
################################################# ################################################# ################################################## ######
cd output
echo "Cut away all unnecessary: TerminalTokens.java"
cat TerminalTokens.java | awk '{
if ($ 0 ~ / ^ [\ t] * public interface TerminalTokens \ {[\ t] * $ /) {
print $ 0
while ((getline nextline) == 1 && nextline! ~ / ^ [\ t] * int TokenNameNotAToken = 0, $ /) {
print nextline;
}
print nextline;
while ((getline nextline) == 1 && nextline! ~ / ^ [\ t] * int TokenNameIdentifier = 22, $ /) {
print nextline;
}
print nextline;
while ((getline nextline) == 1) {
nextline = "";
}
} else {
print $ 0
}
}
'> TerminalTokens_tmp.java
echo "Cut away all unnecessary: javasym.java"
cat javasym.java | awk '{
if ($ 0 ~ / ^ [\ t] * public final static int [\ t] * $ /) {
getline nextline; # skip TokenNameIdentifier = 22,
while ((getline nextline) == 1 && nextline! ~ / ^ \} $ /) {
gsub (//, "", nextline);
gsub (/ = /, "=", nextline);
print "\ t \ t \ t \ t \ t \ t \ t" nextline;
}
print "}";
}
}
'> javasym_tmp.java
echo "Copy the contents of javasym.java to TerminalTokens.java"
cat TerminalTokens_tmp.java javasym_tmp.java> TerminalTokens.java
cd ..
################################################# ################################################# ################################################# ######
cd output
cp ../UpdateParserFiles.java.
echo "Compile UpdateParserFiles"
$ JAVAC -classpath $ {pluginDir} / bin UpdateParserFiles.java
echo "Extracting resource files (* .rsc)"
java -classpath $ {pluginDir} / bin:. UpdateParserFiles javadcl.java javahdr.java
cd ..
################################################# ################################################# ################################################# ######
echo "Remove all updated class files (Parser / TerminalTokens / ParserBasicInformation)"
if test -f $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser / Parser.class
then
rm $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser / Parser.class
fi
if test -f $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser / TerminalTokens.class
then
rm $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser / TerminalTokens.class
fi
if test -f $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser / ParserBasicInformation.class
then
rm $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser / ParserBasicInformation.class
fi
################################################# ################################################# #############
echo "The original files were replaced in the sources by the new ones"
cp output / Parser.java $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser
cp output / TerminalTokens.java $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser
cp output / ParserBasicInformation.java $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser
cp output / parser * .rsc $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser
cp output / parser * .rsc $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser
cp output / readableNames.props $ SOURCE / compiler / org / eclipse / jdt / internal / compiler / parser
cp output / readableNames.props $ SOURCE / bin / org / eclipse / jdt / internal / compiler / parser
################################################# ################################################# ################################################# ######
#echo "Run Eclipse Build with options build -os linux -ws motif"
#cd $ SOURCE
#build -os linux -ws motif
#cd ..
################################################# ################################################# ################################################# ######
#echo "Remove all unnecessary or temporary files"
#rm -rf output
################################################# ################################################# ################################################# ######
#echo "Launch Eclipse"
#cd $ SOURCE
#eclipse
################################################# ################################################# ################################################# ######
echo "Finished"
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!! "
echo "Please run a 'Refresh' in Eclipse Package Explorer"
echo "Since Eclipse does not update the dependencies on the changed interfaces at all"