Skip to main content



      Home
Home » Archived » Blinki » using it as a crawler
using it as a crawler [message #553054] Mon, 16 August 2010 05:33 Go to next message
Eclipse UserFriend
I want to use it as a web crawler. It can navigate some url, submit a form and extract something useful. I need to find element by xpath(this is may be done by execute javascript) and interact with end user who click on an element to tell me he need this element extracted.
So I need get information such as the total DOM tree of the web page(the rendered information such as element's width, height and position), auto fill form(this can be done by execute javascript) and get the element click/selected by user. Can webkit for swt help me with this goal? thanks.
Re: using it as a crawler [message #553056 is a reply to message #553054] Mon, 16 August 2010 05:45 Go to previous message
Eclipse UserFriend
I suspect you are out of luck on that one, because people here are still dealing with more fundamental issues than providing programmatical access to the browser internals.

I suggest you take a closer look at the SWT mozilla engine. It offers a mozilla-specific (XPCOM) java API with which you can do almost anything that is also possible in javascript, directly from within java. You can access the DOM easily; doing the xpath stuff may require additional work

regards,
chris
Previous Topic:using it as a crawler
Next Topic:Re: using it as a crawler
Goto Forum:
  


Current Time: Tue Jul 08 03:12:49 EDT 2025

Powered by FUDForum. Page generated in 0.03342 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top