[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] Download stats and p2

Wayne,

without re-iterating everything that has been said on this topic, I think it would be great if you could let us know why it is such a concern for the board to have accurate download numbers?

Peter

On 12.06.2009, at 20:51, Wayne Beaton wrote:

Greetings all. We have a small problem. Actually, I guess that the problem is as big as you choose to decide it is...

The Eclipse Foundation tracks downloads that go through the download.php script:

http://www.eclipse.org/downloads/download.php?file=[...]

This includes things like the packages and direct downloads provided by projects (assuming that everybody is using the script in their download links).

Downloads that occur through p2 do not go through this script. They go directly to our download server and to our mirrors. The mirrors do not (and arguably cannot reasonably) provide us with download stats.

So... if somebody, for example, downloads the "Eclipse IDE for PHP Developers" we will know that we have one more download of PDT. If they instead download the "Eclipse IDE for Java Developers" and then use p2 to add PDT to their configuration, we currently do not have any way of tracking that download of PDT.

Inability to accurately track downloads is a huge concern for the Eclipse Board.

We have explored several mechanisms for tracking this download. Unfortunately, we've not been holding these conversations as publicly as I'd like, so I'll summarize them briefly below...

1. Get mirrors to give us their download stats. We could ask. But most will not give them to us. Besides, their logs probably contain information about everything they mirror, which will be way more information than we need. And it'll be a heck of a lot of information for our webmasters to weed through.

2. Add a plug-in that gathers information from p2 post install and send that information to eclipse.org. Effectively, this is a call- home mechanism that will require some additional UI elements and considerable effort awfully late in our development cycle. Ultimately, it will require some kind of opt-in from the user; many of whom will refuse leaving us with incomplete data. FWIW, we could use the UDC for this, but it has the same problem.

3. All p2 downloads go through eclipse.org. Denis is concerned that the download.php script and--to some degree--the rest of our infrastructure will not be able to scale to handle the value that can potentially come from p2 downloads. FWIW, we're not increasing our bandwidth for Galileo; instead, we're depending very heavily on mirrors.

Bug 239668 [1] has been open for some time to discuss this issue.

We've decided that the best approach is something that we've been calling the "Single File Hack". In this hack, we configure the p2 metadata (artifacts.xml) to send requests for some small subset of the files to eclipse.org. Ideally, we send requests for one plug-in or feature for each thing that we need to track. The number of files needs to be kept relatively small.

There are problems with this hack. For one, eclipse.org becomes a single point of failure for all downloads. Further, we will have to let organizations that mirror our downloads for internal consumption know how to turn it off.

What we're going to need from each project is the names of the files that we need to be tracking.

I'd love to hear your thoughts on this topic.

Wayne

[1]https://bugs.eclipse.org/bugs/show_bug.cgi?id=239668
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev