Eclipse Community Forums: SeMantic Information Logistics Architecture (SMILA)

Help

Home

Home » Eclipse Projects » SeMantic Information Logistics Architecture (SMILA) » Flushing records to late

Show: Today's Messages :: Show Polls :: Message Navigator

Flushing records to late [message #660190]

Thu, 17 March 2011 05:50

SMILANewBee
Messages: 42
Registered: August 2010

Member

Hello,

the web crawlers crawls some page and the CrawlThread collects them into a record buffer. If the record buffer is full it will be flushed after a new record is processed. If the crawler waits for new pages a long time the buffer will not be flushed within the wait time.

This can be handled easily to move the flush statement within the "processDataReference" to the top.

Have we understand this problem correctly?

Report message to a moderator

[Message index]

		Flushing records to late By: SMILANewBee on Thu, 17 March 2011 05:50
		Re: Flushing records to late By: Daniel Stucky on Thu, 17 March 2011 11:17
		Re: Flushing records to late By: SMILANewBee on Fri, 18 March 2011 01:34

Previous Topic:	Announcing changes in SMILA
Next Topic:	Nightly builds are now also available for Mac OS X!

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Mon May 20 12:21:09 EDT 2013