Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-patch] Automated JUnit test

I have a couple questions/concerns about this test. First, why are you not including the test files (.c/.cpp) in the cvs tree? It seems like for test repeatability they should all be included in revision control, and available in the same place as the test cases. This is the way most of the current CDT tests work, and how the core JDT seems to do it. In general they will create a .resource directory that contains any source/projects that are needed for the tests. That would also make it possible for other people to easily drop in new problematic source files that would then just be part of the normal testing run, which should become an automatic thing at some point.

Second, why the separate failure log file? Why not generate a full error message String, and then fail the test with that? Once again, once we have regular test runs, it will be much easier to deal with various failures if we only need to deal with a single results file.

And the third and final question, why does the test have a global C or C++ flag? I am guessing for your testing you just have a directory of C files, and a directory of C++ files that you switch between, but if all the source does end up in cvs, it would be nice if you could do a run over both the C and the C++ parsing stuff all at the same time (one again, especially for the automated tests run..). I am guessing it would be fairly easy to distinguish c files from c++ files by the file names. Header files may cause a problem, but since we would control the files we test against, we could leave c headers as .h and make all the c++ headers .hpp/.hxx or something like that.

Thanks,
-Peter

Niefer, Andrew wrote:

Attached are the patches for the automated test framework.  Note, that
like
John's line number test, this automated test requires the PDE JUnit
plugin
and should be run as a "JUnit Plugin Test".

The automated framework will take a directory and for each c/cpp/h file
in
it and in its subdirectories, the framework will add a JUnit test to its
testSuite which will parse that file.

The JUnit test will fail if the parse fails or throws an exception.

Filename and line number for each failing test is output to a specified
file.

There is a properties file (AutomatedTest.properties) in which you
specify
the following options, which should be customized before running the
test:
cppNature = true, means the files will be cpp files, any other value
means c
files.
outputFile = Z:\CDT\report.txt, the file to output the results
testDir = Z:\CDT\test, the directory to look for source files in.

-Andrew


------------------------------------------------------------------------

Index: ChangeLog
===================================================================
RCS file: /home/tools/org.eclipse.cdt.ui.tests/ChangeLog,v
retrieving revision 1.30
diff -u -r1.30 ChangeLog
--- ChangeLog	16 Apr 2003 13:52:08 -0000	1.30
+++ ChangeLog	16 Apr 2003 19:38:35 -0000
@@ -1,3 +1,6 @@
+2003-04-16
+	Added AutomatedTest
+	
2003-04-15 John Camelon
	Added ScannerTestCase::testBug36434().
Added ScannerTestCase::testMultipleLines(). Index: parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties
===================================================================
RCS file: parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties
diff -N parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties	16 Apr 2003 19:38:36 -0000
@@ -0,0 +1,3 @@
+cppNature = true
+outputFile = Z:\\CDT\\report.txt
+testDir = Z:\\CDT\\test
Index: parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java
===================================================================
RCS file: parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java
diff -N parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java	16 Apr 2003 19:38:36 -0000
@@ -0,0 +1,205 @@
+/*******************************************************************************
+ * Copyright (c) 2001 IBM Corporation and others.
+ * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Common Public License v0.5 + * which accompanies this distribution, and is available at
+ * http://www.eclipse.org/legal/cpl-v05.html
+ * + * Contributors:
+ *     IBM Corp. - Rational Software - initial implementation
+ ******************************************************************************/
+
+package org.eclipse.cdt.core.parser.tests;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.FilenameFilter;
+import java.util.LinkedList;
+import java.util.Properties;
+
+import org.eclipse.core.runtime.Path;
+
+import org.eclipse.cdt.internal.core.parser.IParserCallback;
+import org.eclipse.cdt.internal.core.parser.NullParserCallback;
+import org.eclipse.cdt.internal.core.parser.Parser;
+
+import junit.framework.AssertionFailedError;
+import junit.framework.Test;
+import junit.framework.TestCase;
+import junit.framework.TestSuite;
+
+
+
+/**
+ * @author aniefer
+ *
+ * To change the template for this generated type comment go to
+ * Window>Preferences>Java>Code Generation>Code and Comments
+ */
+public class AutomatedTest extends TestCase {
+	public AutomatedTest(String name){
+		super(name);
+	}
+	
+	public void doFile() throws Throwable {
+		assertNotNull( fileList );
+		
+		File file = null;
+		Parser parser = null;
+		
+		try{
+			file = (File)fileList.removeFirst();
+			FileInputStream stream = new FileInputStream( file );
+
+			parser = new Parser( stream, nullCallback, true);
+			parser.setCppNature( cppNature );
+			
+			assertTrue( parser.parse() );
+ } + catch( Throwable e )
+		{
+			if( e instanceof AssertionFailedError )
+			{
+				String output = file.getCanonicalPath() + ": Parse failed on line ";
+				output += parser.getLastLineNumber() + "\n";
+	
+				if( report != null ){
+					report.write( output.getBytes() );
+				}
+	
+				fail( output );			
+			} else {
+				if( report != null ){
+					StackTraceElement frames[] = e.getStackTrace();
+					String output = file.getCanonicalPath() + ": " + e.getClass().toString();
+					output += " on line " + parser.getLastLineNumber() + "\n";
+					output += "\t" + "at " + frames[0].getClassName() + "." + frames[0].getMethodName() + "\n";
+					output += "\t" + "at " + frames[1].getClassName() + "." + frames[1].getMethodName() + "\n";
+					report.write( output.getBytes() );
+				}
+				throw e;
+			}
+		}
+	}
+	
+	public void reportFailed(){
+		fail( "Unable to open " + outputFile + "for output of results." );
+	}
+	
+	public void propertiesFailed(){
+		fail( "Unable to load properties file." );
+	}
+	
+	public static Test suite()
+	{
+		TestSuite suite = new TestSuite();
+		
+		try{
+			loadProperties();
+		} catch( Exception e ){
+			suite.addTest( new AutomatedTest( "propertiesFailed") );
+		}
+		
+		if( outputFile != null && !outputFile.equals("") ){
+			try{
+				
+				File output = new File( outputFile );
+				
+				if( output.exists() ){
+					output.delete();
+				}
+				
+				output.createNewFile();
+			
+				report = new FileOutputStream( output );
+			
+			} catch( Exception e ) {
+				suite.addTest( new AutomatedTest( "reportFailed" ) );
+			}
+		}
+		
+		File file = new File( testDir );
+		if( file.exists() )		
+			fillSuite( suite, file );
+		
+		index = 0;	
+
+		return suite;
+	}
+	private static void fillSuite( TestSuite suite, File path ){
+		File files[] = null;
+		if( path.isFile() ){
+			files = new File[ 1 ];
+			files[0] = path;
+		}
+		else
+			files = path.listFiles();
+
+		File file = null;
+		int i = 0;
+		try{
+			file = files[ i++ ];
+			while( file != null )
+			{
+				if( file.isDirectory() )
+					fillSuite( suite, file );
+				else if( file.isFile() && nameFilter.accept( file.getParentFile(), file.getName() ) ){
+					fileList.add( file );
+					suite.addTest( new AutomatedTest( "doFile" ) );
+				}
+				
+				file = files[ i++ ];
+			}
+		} catch( ArrayIndexOutOfBoundsException e ){
+			//done
+		}
+	}
+	
+	protected void tearDown () throws Exception	{
+		if( fileList != null && fileList.size() == 0 ){
+			report.flush();
+			report.close();
+		}
+	}
+	
+	static private void loadProperties() throws Exception{
+		String fileName = org.eclipse.core.runtime.Platform.getPlugin("org.eclipse.cdt.ui.tests").find(new Path("/")).getFile();
+		fileName += "/parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties";
+		FileInputStream propertiesIn = new FileInputStream(fileName);
+		
+		properties.load( propertiesIn );
+		
+		String val = null;
+		
+		cppNature = ( properties.getProperty( "cppNature", "true" ).equals("true") );
+		outputFile = properties.getProperty( "outputFile", "" );
+		testDir = properties.getProperty( "testDir", "" );
+	}
+	
+	private static LinkedList fileList = new LinkedList();
+	private static int index = 0;
+	private static FilenameFilter nameFilter = new Filter();
+	private static FileOutputStream report = null;
+	private static IParserCallback nullCallback = new NullParserCallback();
+	private static Properties properties = new Properties();
+	
+	private static boolean cppNature = true;
+	private static String outputFile = null;
+	private static String testDir = null;
+	
+	static private class Filter implements FilenameFilter
+	{
+		public boolean accept(File dir, String name) {
+ if( name.endsWith(".cpp") || + name.endsWith(".c") || + name.endsWith(".cc") || + name.endsWith(".h") )
+			{
+				return true;
+			}
+			else
+				return false;
+		}
+	}
+}
------------------------------------------------------------------------

Index: parser/org/eclipse/cdt/internal/core/parser/Parser.java
===================================================================
RCS file: /home/tools/org.eclipse.cdt.core/parser/org/eclipse/cdt/internal/core/parser/Parser.java,v
retrieving revision 1.31
diff -u -r1.31 Parser.java
--- parser/org/eclipse/cdt/internal/core/parser/Parser.java	16 Apr 2003 12:30:46 -0000	1.31
+++ parser/org/eclipse/cdt/internal/core/parser/Parser.java	16 Apr 2003 19:19:59 -0000
@@ -106,13 +106,17 @@
				// Mark as failure and try to reach a recovery point
failParse();
-				if (lastBacktrack != null && lastBacktrack == LA(1)) {
-					// we haven't progressed from the last backtrack
-					// try and find tne next definition
-					consumeToNextSemicolon();
-				} else {
-					// start again from here
-					lastBacktrack = LA(1);
+				try {
+					if (lastBacktrack != null && lastBacktrack == LA(1)) {
+						// we haven't progressed from the last backtrack
+						// try and find tne next definition
+						consumeToNextSemicolon();
+					} else {
+						// start again from here
+						lastBacktrack = LA(1);
+					}
+				} catch (EndOfFile e){
+					break;
				}
			}
			catch( Exception e )
@@ -2269,5 +2273,12 @@
	public int getLineNumberForOffset(int offset)
	{
		return scanner.getLineNumberForOffset(offset);
+	}
+	
+	public int getLastLineNumber(){
+		if( lastToken != null ){
+			return scanner.getLineNumberForOffset( lastToken.offset );
+		}
+		return -1;
	}
}




Back to the top