Eclipse Community Forums: Blinki » using it as a crawler

Help

Home

Home » Eclipse Projects » Blinki » using it as a crawler

Show: Today's Messages :: Show Polls :: Message Navigator

using it as a crawler [message #553054]

Mon, 16 August 2010 05:33

LiLi Mising name
Messages: 15
Registered: February 2010

Junior Member

I want to use it as a web crawler. It can navigate some url, submit a form and extract something useful. I need to find element by xpath(this is may be done by execute javascript) and interact with end user who click on an element to tell me he need this element extracted.
So I need get information such as the total DOM tree of the web page(the rendered information such as element's width, height and position), auto fill form(this can be done by execute javascript) and get the element click/selected by user. Can webkit for swt help me with this goal? thanks.

Report message to a moderator

Re: using it as a crawler [message #553056 is a reply to message #553054]

Mon, 16 August 2010 05:45

Christian Sell
Messages: 27
Registered: August 2010

Junior Member

I suspect you are out of luck on that one, because people here are still dealing with more fundamental issues than providing programmatical access to the browser internals.

I suggest you take a closer look at the SWT mozilla engine. It offers a mozilla-specific (XPCOM) java API with which you can do almost anything that is also possible in javascript, directly from within java. You can access the DOM easily; doing the xpath stuff may require additional work

regards,
chris

Report message to a moderator

Previous Topic:	using it as a crawler
Next Topic:	Re: using it as a crawler

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Mon Oct 07 12:39:28 EDT 2013