Eclipse Community Forums: Blinki » using it as a crawler

Home » Archived » Blinki » using it as a crawler

Mon, 16 August 2010 05:33

Eclipse User

I want to use it as a web crawler. It can navigate some url, submit a form and extract something useful. I need to find element by xpath(this is may be done by execute javascript) and interact with end user who click on an element to tell me he need this element extracted.
So I need get information such as the total DOM tree of the web page(the rendered information such as element's width, height and position), auto fill form(this can be done by execute javascript) and get the element click/selected by user. Can webkit for swt help me with this goal? thanks.

Re: using it as a crawler [message #553056 is a reply to message #553054]

Mon, 16 August 2010 05:45

Eclipse User

I suspect you are out of luck on that one, because people here are still dealing with more fundamental issues than providing programmatical access to the browser internals.

I suggest you take a closer look at the SWT mozilla engine. It offers a mozilla-specific (XPCOM) java API with which you can do almost anything that is also possible in javascript, directly from within java. You can access the DOM easily; doing the xpath stuff may require additional work

regards,
chris

Previous Topic:	using it as a crawler
Next Topic:	Re: using it as a crawler

Goto Forum:

-=] Back to Top [=-

Current Time: Tue Jul 08 03:12:49 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter