Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Archived » SeMantic Information Logistics Architecture (SMILA) » Build failed with fresh installation (on OpenSuse Linux)(The build - started within eclipse and from shell - fails due some failed tests for org.eclipse.smila.importing.test.)
Build failed with fresh installation (on OpenSuse Linux) [message #876991] Fri, 25 May 2012 16:38 Go to next message
A Developer is currently offline A DeveloperFriend
Messages: 3
Registered: May 2012
Junior Member
Hi,

I'm new to SMILA. For my first steps, I decided to just set up a new development environment and try to build a SMILA application.

Therefore I followed the instructions in the wiki-pages. Unfortunately the build always ends up in "BUILD FAILED" due some failed tests. For my tests I did the following steps:


  • downloaded eclipse-SDK-3.7.1-linux-gtk-x86_64 (Classic) - as recommended
  • extracted the zip archive two times - one for developing and one for building the application (target)
  • installed the eclipse-delta-pack to the target
  • installed some plugins in eclipse (Subclipse, BPEL Designer, SMIAL BPEL Designer Ext.)
  • downloaded the SMILA-1.0-core-source.zip, extracted it and imported the projects into the workspace
  • set the target platform using the separate eclipse installation
  • added additional libraries to the workspace (ant-contrib, checkstyle, emma, pmd)
  • changed the configuration files build.properties as well as make.sh


The OS is OpenSuse Linux 11.4 64bit with Java 1.6.0_24 (OpenJDK 64 bit).

If I start building the SMILA from within eclipse or from the command line, the build ends always with "BUILD FAILED". Here are the last lines from the log:

  [concat] ******************** Testsuite org.eclipse.smila.importing.test FAILED!
   [concat] * Testcase name:testCrawlMultipleLevels
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestFileCrawling
   [concat] * Message:Waited too long for job to complete
   [concat] * Type:junit.framework.AssertionFailedError
   [concat] * Testcase name:testNoUpdatesCrawl
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestFileCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException
   [concat] * Testcase name:testCrawlAdditionalFiles
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestFileCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException
   [concat] * Testcase name:testCrawlUpdates
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestFileCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException
   [concat] * Testcase name:testWorkers
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestWebCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException
   [concat] * Testcase name:testWebServerAvailable
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestWebCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException
   [concat] * Testcase name:testCrawlSimple
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestWebCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException
   [concat] * Testcase name:testCrawlWithCycles
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestWebCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException
   [concat] * Testcase name:testCrawlWithNonHtml
   [concat] * Testcase class:org.eclipse.smila.importing.test.TestWebCrawling
   [concat] * Message:Could not start job 'buildBulks': Job 'buildBulks' is in state 'FINISHING'.
   [concat] * Type:org.eclipse.smila.jobmanager.exceptions.JobManagerException

BUILD FAILED
/home/developer/workspace371/SMILA.builder/make.xml:499: At least one unit test failed.

Total time: 52 minutes 58 seconds



I have also tried the latest SMILA sources from the repository but this didn't worked too. Besides this, it produces some more failed tests. So I thought, it would be save and better to use the released source code.


The question is, why isn't it working? Does anybody have an idea?


Many thanks in advance!

A.Developer

Report message to a moderator

Re: Build failed with fresh installation (on OpenSuse Linux) [message #878388 is a reply to message #876991] Tue, 29 May 2012 07:51 Go to previous messageGo to next message
Juergen Schumacher is currently offline Juergen SchumacherFriend
Messages: 35
Registered: July 2009
Member
Hi,

Am 25.05.2012, 18:38 Uhr, schrieb A Developer <forums-noreply@xxxxxxxx>:
> Hi,
>
> I'm new to SMILA. For my first steps, I decided to just set up a new
> development environment and try to build a SMILA application.
>
>
> If I start building the SMILA from within eclipse or from the command
> line, the build ends always with "BUILD FAILED". Here are the last lines
> from the log:
>
>
> [concat] ******************** Testsuite
> org.eclipse.smila.importing.test FAILED!
> [concat] * Testcase name:testCrawlMultipleLevels
> [concat] * Testcase
> class:org.eclipse.smila.importing.test.TestFileCrawling
> [concat] * Message:Waited too long for job to complete
>
> [...]

> BUILD FAILED
> /home/developer/workspace371/SMILA.builder/make.xml:499: At least one
> unit test failed.
>
> Total time: 52 minutes 58 seconds
> ...
> The question is, why isn't it working? Does anybody have an idea?

I have never seen this in our tests. It seems to me that you are doing
this on a quite slow machine,
a complete build with all tests takes about 30 minutes on our computers.
So it may be that one test
is not completed in the expected time and causes the next test to fail,
too, because some cleanup
is missing (yes: that should not happen, of course). For a try, you could
check line 99 in the test
class org.eclipse.smila.importing.test.TestFileCrawling, it should be

waitForJobRunCompleted(JOBNAME_CRAWLFILES, crawlJobId, 15000);

increase the number to at least 30000 (30 seconds), and try to run the
test again?
We will change the test in the trunk to be more robust against such
problems, thanks for the hint.

Regards,
Juergen.

Report message to a moderator

Re: Build failed with fresh installation (on OpenSuse Linux) [message #879448 is a reply to message #878388] Thu, 31 May 2012 08:42 Go to previous messageGo to next message
A Developer is currently offline A DeveloperFriend
Messages: 3
Registered: May 2012
Junior Member
Hi Juergen,

thank you for your answer. I did several tests and started the build from inside and outside of eclipse with different values. I found out, that the build works outside of eclipse with a value of 60000 (60 seconds) but it still fails when it's started in eclipse.

You're probably right that my machine is to slow. For the build I used my 5-years old desktop PC which is an AMD Athlon 64 dual core 4200+ (2,2 GHz) with 8 GB RAM. I didn't expect that it is too slow Sad

So, by the way. What are the recommended configurations for:

  • a developer machine,
  • a build machine and
  • a productive runtime environment
for SMILA applications?

Regards,
A.Developer

Report message to a moderator

Re: Build failed with fresh installation (on OpenSuse Linux) [message #879503 is a reply to message #879448] Thu, 31 May 2012 10:20 Go to previous messageGo to next message
Juergen Schumacher is currently offline Juergen SchumacherFriend
Messages: 35
Registered: July 2009
Member
Hi,

Am 31.05.2012, 10:42 Uhr, schrieb A Developer <forums-noreply@xxxxxxxx>:
> Hi Juergen,
>
> thank you for your answer. I did several tests and started the build
> from inside and outside of eclipse with different values. I found out,
> that the build works outside of eclipse with a value of 60000 (60
> seconds) but it still fails when it's started in eclipse.
>
> You're probably right that my machine is to slow. For the build I used
> my 5-years old desktop PC which is an AMD Athlon 64 dual core 4200+ (2,2
> GHz) with 8 GB RAM. I didn't expect that it is too slow :(

No, that should be OK, at least for these tests. I'm working on a
quad-core with 8 GB RAM. Sorry, my suspicion was a bit hasty (;

Another thing that has a great impact on performance is the location of the
data directories used by the Zookeeper server embedded into SMILA.
Zookeeper
performance tends to be bad if it has to write its logs and snapshots to
the
same hard disk used by the rest of the system, and the crawler tests are
quite
task-intensive. So we strongly recommend to put the ZK data on a RAM disk
which
is quite easy to set up on Linux (at least if you have sudo rights ;-). See

http://wiki.eclipse.org/SMILA/Documentation/Bundle_org.eclipse.smila.zookeeper#Configuration_of_data_directories

for how to set them up and additional details. To have the tests using the
RAM disk,
you can then just set and export an environment variable SMILA_ZK_DATADIR
to point
to a directory inside your RAM disk (not the RAM disk directory itself)
before
starting Eclipse. That should really improve things.

Using a RAM disk is OK even for production setups as long as you don't
need currently
running tasks to survive even a power failure on this machine. In this
case we (and the
ZK people themselves) recommend to put the ZK data directories on an own
hard disk that
is not used by anything else in the system.

How much CPU power, memory and storage is needed by a productive SMILA
system depends
largely on the components integrated into SMILA and how much throughput
you need to
reach, so we cannot give any general recommendations.

Regards,
Juergen

Report message to a moderator

Re: Build failed with fresh installation (on OpenSuse Linux) [message #880145 is a reply to message #879503] Fri, 01 June 2012 14:18 Go to previous message
A Developer is currently offline A DeveloperFriend
Messages: 3
Registered: May 2012
Junior Member
Hi Juergen,

thank you for the tip with the RAM disk. I created one as described and run the build process several times inside and outside of eclipse. In combination with a wait time of 30 sec in the class "org.eclipse.smila.importing.test.TestFileCrawling" all the builds were successful.

Many thanks.

Regards,
A.Developer

Report message to a moderator

Previous Topic:Missing HTTP Proxy setting causing web crawler to fail
Next Topic:1.1 M1 is out!
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Thu Apr 25 12:11:54 GMT 2024

Powered by FUDForum. Page generated in 0.03121 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly