Re: [cdt-dev] unit test fail on windows 10 with mingw

[Jonah Graham <jonah@xxxxxxxxxxxxxxxx>]

The Linux builds should be 0 failures (out of 9000 tests) apart from a few flaky tests that I and others try to monitor and fix the worst of them. 

On Windows there are some issues with stability, but there are more issues with which toolchain(s) to test against. Some versions of tools add quotes, some escape, some are posix like (cygwin/msys2), some are Windows like. 

There is a slow simmering project to have at least a baseline running on Eclipse windows build machine so that we know one complete set of tools is correct. The current baseline for Windows is best obtained from the windows build job: https://ci.eclipse.org/cdt/view/Main/job/cdt-master-windows/

I can review the individual failures you are asking about when I am at a computer (writing this on my phone), but you can also just push your change to gerrit and it will run all the tests there (in less than 20 minutes). If your patch isn't ready for review, that's fine, just make a note of that to save a reviewer some time.

As for which tests to run locally, you should run the ones in the same area. I.e., run the dsf tests if you are doing dsf changes, core/ui tests for indexer or editor, etc. (IIRC there is more than one 

AutomatedIntegrationSuite.) But if you want to run as the build machine does (I almost never do a full test run locally) then use maven (mvn verify) to build and test.

HTH, 

Jonah 

On 29 May 2017 19:04, "Jan Baeyens" <jan@xxxxxxxxxx> wrote:

Hi

Given my experiences of previous deliveries I tried to run the unit test suite AutomatedIntegrationSuite on the master branch of CDT.
I don't know if this is the correct suite but I really would like to be able to run the "test suit used by the build" to run without problems on my system.

7 out of 214 test fail.
I'm running on a windows 10 64 bit OS and Did the eclipse install with oomph.

I added mingw 5.3.0 to the path before starting D:\eclipse\cdt-master\eclipse

I also had to set D:\eclipse\cdt-master\ to have write rights before the test would succeed (strangely  enough windows forced me to do this on all children as well)

I added echo.cmd echo.bat and echo.exe (rename from size.exe) to my path.

What I found out about the problems.


The 1th problem is test30MultiResConfig failing because it can't find junit.framework.AssertionFailedError: File D:/eclipse/cdt-master/junit-workspace/multiResConfig/MBS30ConfigName.Dbg/main.d could not be read: D:\eclipse\cdt-master\junit-workspace\multiResConfig\MBS30ConfigName.Dbg\main.d (Het systeem kan het opgegeven bestand niet vinden)
The file is on my system as

D:\eclipse\cdt-master\junit-workspace\multiResConfig\MBS30ConfigName.Dbg\'main.d'

Yes with single quotes


The 2th problem test30CopyandDeploy

Fails in console with

'cp' is not recognized as an internal or external command,

Seems logical as it is windows.


The 3th problem test1DepCalc2
contains a wrong include path statement
Sources/sub\ sources
instead of
Sources/sub sources
After fixing this the linker starts making the same mistake
gcc  -o "test1DepCalc2"  ./Sources/sub\ sources/func\ 3.o
Using \space instead of quoting the string

The 4th problem is more puzzling test1DepCalc3
It starts with the same \ problem in /Sources/sub\ source but after fixing this the build still fails with
'Invoking: GCC C Compiler'
gcc -DFUN3 -I../Headers -I"../Sources/sub sources" -O0 -g3 -Wall -c -fmessage-length=0 -o "Sources/sub sources/func 3.o" "../Sources/sub sources/func 3.c" && \
echo -n 'Sources/sub sources/func 3.d' Sources/ sources/ ./ > 'Sources/sub sources/func 3.d' && \
gcc -MM -MG -P -w -DFUN3 -I../Headers -I"../Sources/sub sources" -O0 -g3 -Wall -c -fmessage-length=0   "../Sources/sub sources/func 3.c" >> 'Sources/sub sources/func 3.d'
Het systeem kan het opgegeven pad niet vinden.
Sources/sub sources/subdir.mk:18: recipe for target 'Sources/sub sources/func 3.o' failed
make: *** [Sources/sub sources/func 3.o] Error 1

It looks like make is trying to execute the 3 commands in one go which may be related to the &&\ at the end of the lines.

The 5th problem test1DepCalcPreBuild
exactly the same as test1DepCalc2


The last problem testDesRebuildState
I have no clue what this is about. So I simply copy the error trace
junit.framework.AssertionFailedError
    at junit.framework.Assert.fail(Assert.java:55)
    at junit.framework.Assert.assertTrue(Assert.java:22)
    at junit.framework.Assert.assertFalse(Assert.java:39)
    at junit.framework.Assert.assertFalse(Assert.java:47)
    at junit.framework.TestCase.assertFalse(TestCase.java:219)
    at org.eclipse.cdt.managedbuilder.core.tests.BuildDescriptionModelTests.testDesRebuildState(BuildDescriptionModelTests.java:2150)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at junit.framework.TestCase.runTest(TestCase.java:176)
    at junit.framework.TestCase.runBare(TestCase.java:141)
    at junit.framework.TestResult$1.protect(TestResult.java:122)
    at junit.framework.TestResult.runProtected(TestResult.java:142)
    at junit.framework.TestResult.run(TestResult.java:125)
    at junit.framework.TestCase.run(TestCase.java:129)
    at junit.framework.TestSuite.runTest(TestSuite.java:252)
    at junit.framework.TestSuite.run(TestSuite.java:247)
    at junit.framework.TestSuite.runTest(TestSuite.java:252)
    at junit.framework.TestSuite.run(TestSuite.java:247)
    at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:66)
    at org.eclipse.pde.internal.junit.runtime.PlatformUITestHarness.lambda$0(PlatformUITestHarness.java:43)
    at java.lang.Thread.run(Unknown Source)


I would be more confident pushing changes if these test would succeed :-) Or some could tell me this is not a problem.

Best regards
Jantje
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev