Skip to main content

Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Archived » Blinki » using it as a crawler
using it as a crawler [message #553054] Mon, 16 August 2010 09:33 Go to next message
LiLi Mising name is currently offline LiLi Mising nameFriend
Messages: 15
Registered: February 2010
Junior Member
I want to use it as a web crawler. It can navigate some url, submit a form and extract something useful. I need to find element by xpath(this is may be done by execute javascript) and interact with end user who click on an element to tell me he need this element extracted.
So I need get information such as the total DOM tree of the web page(the rendered information such as element's width, height and position), auto fill form(this can be done by execute javascript) and get the element click/selected by user. Can webkit for swt help me with this goal? thanks.
Re: using it as a crawler [message #553056 is a reply to message #553054] Mon, 16 August 2010 09:45 Go to previous message
Christian Sell is currently offline Christian SellFriend
Messages: 51
Registered: August 2010
I suspect you are out of luck on that one, because people here are still dealing with more fundamental issues than providing programmatical access to the browser internals.

I suggest you take a closer look at the SWT mozilla engine. It offers a mozilla-specific (XPCOM) java API with which you can do almost anything that is also possible in javascript, directly from within java. You can access the DOM easily; doing the xpath stuff may require additional work

Previous Topic:using it as a crawler
Next Topic:Re: using it as a crawler
Goto Forum:

Current Time: Sat Jan 28 08:17:44 GMT 2023

Powered by FUDForum. Page generated in 0.03757 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top