Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » RCP Testing Tool » Different results between Jenkins and test cases run from RCPTT IDE(#Jenkins #RCPTT_IDE #different_results #frustrating)
icon5.gif  Different results between Jenkins and test cases run from RCPTT IDE [message #1735576] Tue, 21 June 2016 09:49 Go to next message
Angelo Luciani is currently offline Angelo LucianiFriend
Messages: 129
Registered: September 2015
Location: Milan
Senior Member

Hi Folks,
When I run a test from RCPTT_IDE it always PASS.
The same not happened in JENKINS because it FAILS .
The Database and AUT are the same.
How to check it to have the same behavior ?
Please let me know. It is really frustrating.
Thanks in advance.
Angelo Luciani



JENKINS LOG
[INFO] Started at Tue Jun 21 11:00:35 CEST 2016
[INFO] RCPTT Runner version: 2.0.0
[INFO] Initializing target platform...
[INFO] Target platform is valid.
[INFO] Looking for tests...
[INFO] Existing projects in workspace:
[INFO]     <none>
[INFO] Importing projects to workspace:
[INFO] Refreshing projects:
[INFO]     C:\rcptt_tests_cases\Optimization\target\projects\productTests... OK
[INFO]     C:\rcptt_tests_cases\Compliance\compliance... OK
[INFO] Refreshing projects:
[INFO]     Optimization... OK
[INFO]     compliance... OK
[INFO] Searching for tests in projects:
[INFO]     Optimization... 
[INFO] -- Testcase is skiped by tag "GC_1.1.1_create_a_constraint_folder" -- GC_1.1.1_create_a_constraint_folder.test
[INFO] -- Testcase is skiped by tag "GC_1.1.1_create_a_constraint_folder" -- GC_1.1.2_edit_a_constraint_folder.test
[INFO] -- Testcase is skiped by tag "GC_1.1.1_create_a_constraint_folder" -- GC_1.1.3_create_a_constraint_subfolder.test
[INFO] -- Testcase is skiped by tag "GC_1.1.1_create_a_constraint_folder" -- GC_1.1.4_delete_a_constraint_folder.test
[INFO] -- Testcase is skiped by tag "GC_1.2.1_add_roles_to_folder_and_subfolder" -- GC_1.2.1_add_roles_to_folder_and_subfolder.test
[INFO] -- Testcase is skiped by tag "GC_1.2.2_add_roles_to_parent_folder" -- GC_1.2.2_add_roles_to_parent_folder.test
[INFO] -- Testcase is skiped by tag "GC_1.2.3_remove_roles_from_parent_folder" -- GC_1.2.3_remove_roles_from_parent_folder.test
[INFO] -- Testcase is skiped by tag "GC_1.3.2_delete_a_global_constraint" -- GC_1.3.2_delete_a_global_constraint.test
[INFO] -- Testcase is skiped by tag "GC_1.4.1_search_in library" -- GC_1.4.1_search_in library.test
[INFO] -- Testcase is skiped by tag "GC_1.5.1_edit_a_global_constraint" -- GC_1.5.1_edit_a_global_constraint.test
[INFO] -- Testcase is skiped by tag "GC_1.5.2_clone_a_global_constraint" -- GC_1.5.2_clone_a_global_constraint.test
[INFO] -- Testcase is skiped by tag "GC_3.1.1_constraint_editor_autocompletion_variables" -- GC_3.1.1_constraint_editor_autocompletion_variables.test
[INFO] -- Testcase is skiped by tag "GC_3.1.2_constraint_editor_autocompletion_constants" -- GC_3.1.2_constraint_editor_autocompletion_constants.test
[INFO] -- Testcase is skiped by tag "GC_3.2.1_constraint_editor_check_syntax" -- GC_3.2.1_constraint_editor_check_syntax.test
[INFO] -- Testcase is skiped by tag "GC_3.2.2_constraint_editor_check_syntax" -- GC_3.2.2_constraint_editor_check_syntax.test
[INFO] -- Testcase is skiped by tag "GC_3.2.3_constraint_editor_check_syntax" -- GC_3.2.3_constraint_editor_check_syntax.test
[INFO] Complete OK
[INFO]     compliance... 
[INFO] No tests found
[INFO] Testcase Artifacts:1
[INFO] AUT-0:Launching
[INFO] AUT-0:Product: xdm.client.rcp.xdm
[INFO] AUT-0:Application: null
[INFO] AUT-0:Architecture: x86_64
[INFO] 64bit arch is selected because
[INFO] - JVM is 64bit
[INFO] - AUT contains both launcher plugins
[INFO] 	-Equinox launcher
[INFO] 	- C:plugins\org.eclipse.equinox.launcher.win32.win32.x86_1.1.100-20120516\
[INFO] 	-Equinox launcher
[INFO] 	- C:plugins\org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.100-20120516\
[INFO] C:\rcptt_tests_cases\Optimization\target\aut-ws-0: AUT arguments: "-os ${target.os}" "-ws ${target.ws}" "-arch x86_64" -consoleLog "-console -debug" -user=xdmadmin -pwd=xdm -url=rmi://localhost:12999 -os ${target.os} -arch ${target.arch}
[INFO] C:\rcptt_tests_cases\Optimization\target\aut-ws-0: AUT VM arguments: -Xmx1024m -Xverify:none -XX:MaxPermSize=256M -XX:+UseParallelGC -XX:PermSize=20M -XX:MaxNewSize=32M -XX:NewSize=32M -Duser.language=en -Duser.country=EN -Dosgi.instance.area=@user.home/xdm/workspace -Dosgi.user.area=@user.home/xdm/user -Dosgi.configuration.area=@user.home/xdm/configuration -Dclient.update.memsize=-Xmx128m -DmultiInstance=true -Dorg.eclipse.swt.browser.DefaultType=ie,mozilla -Dorg.eclipse.swt.browser.IEVersion=9999 -Xmx1024m -Xverify:none -XX:MaxPermSize=256M -XX:+UseParallelGC -XX:PermSize=20M -XX:MaxNewSize=32M -XX:NewSize=32M -Duser.language=en -Duser.country=EN -Dosgi.instance.area=@user.home/xdm/workspace -Dosgi.user.area=@user.home/xdm/user -Dosgi.configuration.area=@user.home/xdm/configuration -Dclient.update.memsize=-Xmx128m -DmultiInstance=true -Dorg.eclipse.swt.browser.DefaultType=ie,mozilla -Dorg.eclipse.swt.browser.IEVersion=9999
[INFO] Fail 1 (1) processed. 1 failed. spent: 1:06, 0:00 mins remaining. GC_1.3.1_add_a_global_constraint. time: 30362ms  Cause: The Editor "Library : test_folder_pet_costraint" could not be found.
[INFO] Failed Tests:
[INFO] GC_1.3.1_add_a_global_constraint
[INFO] Process terminated. Shut down AUTs
[INFO] Runner exit code is: 56
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:47 min
[INFO] Finished at: 2016-06-21T11:03:05+02:00
[INFO] Final Memory: 9M/244M
[INFO] -------------------------------------



POM.XML
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <!-- Set artifact id according to a RCPTT project name  -->
  <artifactId>productTests</artifactId>
  <!-- Set group id based on application under test name -->
  <groupId>com.company.product</groupId>
  <version>2.0.0-SNAPSHOT</version>
  <!-- RCPTT Maven Plugin provides this packaging type -->
  <packaging>rcpttTest</packaging>

  <!-- RCPTT Maven Plugin and RCPTT Runner are hosted in this repository -->
  <pluginRepositories>
    <pluginRepository>
      <id>rcptt-releases</id>
      <name>RCPTT Maven repository</name>
      <url>https://repo.eclipse.org/content/repositories/rcptt-releases/</url>
    </pluginRepository>
    <pluginRepository>
      <id>rcptt-snapshots</id>
      <name>RCPTT Maven Snapshots repository</name>
      <snapshots>
        <updatePolicy>never</updatePolicy>
      </snapshots>
      <url>https://repo.eclipse.org/content/repositories/rcptt-snapshots/</url>
    </pluginRepository>
  </pluginRepositories>

  <!-- 
       If RCPTT tests are planned to be run on an update site, 
       RCPTT Maven Plugin can download Eclipse SDK of required version
       for current platform from repository below (to see 
       an up-to-date list of available versions visit this link:
       http://maven.xored.com/nexus/content/repositories/ci4rcptt-releases/org/eclipse/sdk/
  -->
  <properties>
    <rcptt-maven-version>2.0.0</rcptt-maven-version>
  </properties> 


  <!-- The main part of RCPTT Maven plugin -->
  <build>
    <plugins>
      <plugin>
        <groupId>org.eclipse.rcptt</groupId>
        <artifactId>rcptt-maven-plugin</artifactId>
        <version>${rcptt-maven-version}</version>
        <extensions>true</extensions>
        <configuration>

          <!-- This element describes where to get an AUT to run tests -->
          <aut>
            <!-- There are several ways to specify AUT location,
                 Uncomment an element corresponding to a most suitable way
            -->

            <!-- Use AUT from current build results. [classifier] will be
            automatically replaced according to the current platform -->
            <!--
            <explicit>${project.basedir}/../product/target/products/org.eclipse.ui.examples.rcp.browser.product-[classifier].zip</explicit>
            -->

            <!-- Or specify a path to AUT folder -->

           <explicit>${path-client}</explicit>
           <args>
		   <arg>-os ${target.os}</arg>
		   <arg>-ws ${target.ws}</arg>
		   <arg>-arch x86_64 </arg>
		   <arg> -consoleLog </arg>
		   <arg> -console -debug </arg>
	       <arg>${user}</arg>
	       <arg>${password}</arg>
		   <arg>${link}</arg>
           </args>
		   
            <injections>
              <!-- features are optional - when omitted, all features from given site will be installed -->
              <!--
              <injection>
                <site></site>
                <features>
                  <feature>com.comanyname.featureid</feature>
                </features>
              </injection>
              -->
            </injections>

        
		   <vmArgs>
			 <vmArg>-Xmx1024m</vmArg>
			 <vmArg>-Xverify:none</vmArg>
			 <vmArg>-XX:MaxPermSize=256M</vmArg>
			 <vmArg>-XX:+UseParallelGC</vmArg>
			 <vmArg>-XX:PermSize=20M </vmArg>
			 <vmArg>-XX:MaxNewSize=32M</vmArg>
			 <vmArg>-XX:NewSize=32M</vmArg>
			 <vmArg>-Duser.language=en</vmArg>
			 <vmArg>-Duser.country=EN</vmArg>
			 <vmArg>-Dosgi.instance.area=@user.home/xdm/workspace </vmArg>
			  <vmArg>-Dosgi.user.area=@user.home/xdm/user </vmArg>
			 <vmArg>-Dosgi.configuration.area=@user.home/xdm/configuration</vmArg>
			 <vmArg>-Dclient.update.memsize=-Xmx128m </vmArg>
			 <vmArg>-DmultiInstance=true</vmArg>
			 <vmArg>-Dorg.eclipse.swt.browser.DefaultType=ie,mozilla</vmArg>
			 <vmArg>-Dorg.eclipse.swt.browser.IEVersion=9999</vmArg>
		   </vmArgs>
        
          </aut>

   <runner>
            <!-- RCPTT Runner location can be set using the same methods 
                 as AUT location:
            -->
            <!--
            <explicit>/path/to/rcptt/runner</explicit>
            -->
          <explicit>C:\Users\angelo.luciani\Downloads\rcptt.runner-2.0.0\eclipse</explicit>    
            <!-- but the most convenient way is to just set its version,
                 RCPTT maven plugin will automatically set right groupId and
                 artifact ID and will download RCPTT Runner from Xored Maven Repo -->
            <version>2.0.0</version>

          	 <vmArgs>
			 <vmArg>-Xmx1024m</vmArg>
			 <vmArg>-Xverify:none</vmArg>
			 <vmArg>-XX:MaxPermSize=256M</vmArg>
			 <vmArg>-XX:+UseParallelGC</vmArg>
			 <vmArg>-XX:PermSize=20M </vmArg>
			 <vmArg>-XX:MaxNewSize=32M</vmArg>
			 <vmArg>-XX:NewSize=32M</vmArg>
			 <vmArg>-Duser.language=en</vmArg>
			 <vmArg>-Duser.country=EN</vmArg>
			 <vmArg>-Dosgi.instance.area=@user.home/xdm/workspace </vmArg>
			 <vmArg>-Dosgi.user.area=@user.home/xdm/user</vmArg>
			 <vmArg>-Dosgi.configuration.area=@user.home/xdm/configuration</vmArg>
			 <vmArg>-Dclient.update.memsize=-Xmx128m </vmArg>
			 <vmArg>-DmultiInstance=true</vmArg>
			 <vmArg>-Dorg.eclipse.swt.browser.DefaultType=ie,mozilla</vmArg>
			 <vmArg>-Dorg.eclipse.swt.browser.IEVersion=9999</vmArg>
		   </vmArgs>
          </runner>
          <!-- Test options for RCPTT Runner, most popular options listed here.
               Full list of options is available at:
               https://ci.xored.com/doc/runner/
          -->
          <testOptions>
            <!-- Timeout for all tests, in seconds -->
            <execTimeout>3600</execTimeout>
            <!-- Timeout for a single test case, in seconds -->
            <testExecTimeout>1800</testExecTimeout>

            <!-- When set to true, in case of test failure
                 AUT will be restarted. This significantly
                 slows down execution, but may be useful 
                 for some test suites -->
            
            <restartAUTOnFailure>false</restartAUTOnFailure>
          </testOptions>

          <!-- By default RCPTT Runner runs tests from a project directory,
               but in some cases it might be required to import additional 
               projects into runner's workspace -->
          <!--
          <projects>
            <project>${project.basedir}/../project</project>
          </projects>
          -->
		 <projects>
            <project>C:\rcptt_tests_cases\Compliance\compliance</project>
          </projects>	
          <!-- By default RCPTT Runner runs all tests from workspace,
               but it is possible to pass test suite names, so it
               will execute only test suites from given project. -->

          
		  <suites>
			<suite>optimization</suite>
		  </suites>
          

          <!-- Sometimes it might be useful to skip a test case
               (for instance because of some unresolved bug). RCPTT
               can skip tests based on its tags. By default RCPTT skips
               tests with tag 'skipExecution' (this value has been 
               chosen because on one hand it is descriptive enough,
               on another hand it is unlikely that this tag name
               will collide with some user's tag)
               -->
          
          <skipTags>
		  <skipTag>GC_1.1.1_create_a_constraint_folder</skipTag>
		  <skipTag>GC_1.1.2_edit_a_constraint_folder</skipTag>
		  <skipTag>GC_1.1.3_create_a_constraint_subfolder</skipTag>
		  <skipTag>GC_1.1.4_delete_a_constraint_folder</skipTag>
		  <skipTag>GC_1.2.1_add_roles_to_folder_and_subfolder</skipTag>
		  <skipTag>GC_1.2.2_add_roles_to_parent_folder</skipTag>
		  <skipTag>GC_1.2.3_remove_roles_from_parent_folder</skipTag>
		  <skipTag>GC_1.3.2_delete_a_global_constraint</skipTag>
		  <skipTag>GC_1.4.1_search_in library</skipTag>
		  <skipTag>GC_1.5.1_edit_a_global_constraint</skipTag>
		  <skipTag>GC_1.5.2_clone_a_global_constraint</skipTag>
		  <skipTag>GC_3.1.1_constraint_editor_autocompletion_variables</skipTag>
		  <skipTag>GC_3.1.2_constraint_editor_autocompletion_constants</skipTag>
		  <skipTag>GC_3.2.1_constraint_editor_check_syntax</skipTag>
		  <skipTag>GC_3.2.2_constraint_editor_check_syntax</skipTag>
		  <skipTag>GC_3.2.3_constraint_editor_check_syntax</skipTag>
          </skipTags>

          <!-- By default RCPTT generates a single HTML report file with
               all tests, but it is possible to generate one file per
               test -->
          <!--
          <splitHtmlReport>true</splitHtmlReport>
          -->
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>


"Ce sont les petits désirs qui rendent un jeune homme hardi."
Giovanni Giacomo Casanova

[Updated on: Tue, 21 June 2016 10:13]

Report message to a moderator

Re: Different results between Jenkins and test cases run from RCPTT IDE [message #1737013 is a reply to message #1735576] Tue, 05 July 2016 06:47 Go to previous messageGo to next message
Andrey Sobolev is currently offline Andrey SobolevFriend
Messages: 75
Registered: February 2015
Member
Hi,

You need to look into report to understand why it failed in CI environment. Rcptt .report file is generated via runner for deep investigations, you could open it with RCPTT and look in details.
Also you could look into junit or html reports generated by runner to check a SWT widgets dump in text format to understand why test is failed.

Best regards,
Andrey.
Re: Different results between Jenkins and test cases run from RCPTT IDE [message #1806498 is a reply to message #1737013] Thu, 09 May 2019 23:04 Go to previous messageGo to next message
Ming Hu is currently offline Ming HuFriend
Messages: 3
Registered: June 2016
Junior Member
Hi Angelo,

I'm experiencing the same issue here, I wonder if you have find a solution for this yet?

Thanks,
Ming
Re: Different results between Jenkins and test cases run from RCPTT IDE [message #1806681 is a reply to message #1735576] Mon, 13 May 2019 18:56 Go to previous message
Ming Hu is currently offline Ming HuFriend
Messages: 3
Registered: June 2016
Junior Member
Hi Angelo,

Have you got this resolved? Can you share how you've resolved it?

Thanks,
Ming


Previous Topic:Feature Injections not working in 2.4.2 nightly builds
Next Topic:Script pass from rcptt but not from hudson
Goto Forum:
  


Current Time: Fri Apr 26 02:47:23 GMT 2024

Powered by FUDForum. Page generated in 0.03081 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top