Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[wtp-dev] Screen scraping and other changes coming to releng

After having recovered from the embarrassment of bug 172361
I've been making some improvements to our build summary reporting.

It's not displayed too well yet (and will get worse before it gets better),
but the principles are completely changing, and
the strategy will be to slowly change things until complete.

I've moved us off of _some_ of the platform's releng tools, so they
will stop breaking us :) [as they change their own results pages] and also
so we can do exactly what we want .. such as capture certain totals
that the platform does not capture ... such as total number of compile
logs. Our wtp releng tools will be in '' .

Most important, we need to practice what we preach and separate
the web data from the web presentation. I'll be converting our scripts
to produce only xml files, that "self document" the build results and summaries,
and then use some XSL transforms, etc., to display these in pleasant and
meaningful ways. That might even also include the Phoenix L&F :)
In theory, the download servers themselves should not be serving up PHP,
but instead the normal web servers should present the pages, and those
pages use the XML files to point to the proper data on the download servers.

Why should you care?

1) if anyone is (still) scraping the HTML pages
to figure out what to automatically download, then I will be breaking you.
The good news is, once it's in XML form, you'll have true data to use, and
not have to scrape any longer.

2) if anyone has any ideas .... that they would like to implement! :) .... then feel
free to contribute.

3) If anyone has any questions or comments, don't hesitate to ask.

I've opened bug 175972 to use to document activities in this area.

Back to the top