[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [cross-project-issues-dev] Download stats and p2

Hi Wayne et al,

I'd like to ask back regarding option (1) from your E-Mail,
direct download stats from the web and ftp servers' access
logs on Eclipse.org (and those mirrors who happen to give them 
to us).

I'm assuming that for download.eclipse.org such logs already
exist, and recalling Denis' excited "shooting for 1 Mio
downloads now" blog or similar in previous years, I'm further 
assuming that at least for Eclipse.org the analysis is not 
that bad.

Going for the server logs gives the most accurate data at
zero impact for the release itself. I'm not a web guy, but
I do assume that tools exist for analyzing those access logs.
Why not just go and ask some of the mirrors and see who is 
willing to collaborate?

But perhaps that is happening already, the stats are being
prepared but details are confidential for strategic members 
only [some small reward for strategic membership]... while
some aggregate numbers are shared with the Community...

Cheers,
--
Martin Oberhuber, Senior Member of Technical Staff, Wind River
Target Management Project Lead, DSDP PMC Member
http://www.eclipse.org/dsdp/tm
 
 

> -----Original Message-----
> From: cross-project-issues-dev-bounces@xxxxxxxxxxx 
> [mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx] On 
> Behalf Of Wayne Beaton
> Sent: Freitag, 12. Juni 2009 20:51
> To: cross project issues
> Subject: [cross-project-issues-dev] Download stats and p2
> 
> Greetings all. We have a small problem. Actually, I guess that the 
> problem is as big as you choose to decide it is...
> 
> The Eclipse Foundation tracks downloads that go through the 
> download.php 
> script:
> 
> http://www.eclipse.org/downloads/download.php?file=[...]
> 
> This includes things like the packages and direct downloads 
> provided by 
> projects (assuming that everybody is using the script in 
> their download 
> links).
> 
> Downloads that occur through p2 do not go through this 
> script. They go 
> directly to our download server and to our mirrors. The 
> mirrors do not 
> (and arguably cannot reasonably) provide us with download stats.
> 
> So... if somebody, for example, downloads the "Eclipse IDE for PHP 
> Developers" we will know that we have one more download of 
> PDT. If they 
> instead download the "Eclipse IDE for Java Developers" and 
> then use p2 
> to add PDT to their configuration, we currently do not have 
> any way of 
> tracking that download of PDT.
> 
> Inability to accurately track downloads is a huge concern for the 
> Eclipse Board.
> 
> We have explored several mechanisms for tracking this download. 
> Unfortunately, we've not been holding these conversations as 
> publicly as 
> I'd like, so I'll summarize them briefly below...
> 
> 1. Get mirrors to give us their download stats. We could ask. 
> But most 
> will not give them to us. Besides, their logs probably contain 
> information about everything they mirror, which will be way more 
> information than we need. And it'll be a heck of a lot of information 
> for our webmasters to weed through.
> 
> 2. Add a plug-in that gathers information from p2 post 
> install and send 
> that information to eclipse.org. Effectively, this is a call-home 
> mechanism that will require some additional UI elements and 
> considerable 
> effort awfully late in our development cycle. Ultimately, it will 
> require some kind of opt-in from the user; many of whom will refuse 
> leaving us with incomplete data. FWIW, we could use the UDC for this, 
> but it has the same problem.
> 
> 3. All p2 downloads go through eclipse.org. Denis is 
> concerned that the 
> download.php script and--to some degree--the rest of our 
> infrastructure 
> will not be able to scale to handle the value that can 
> potentially come 
> from p2 downloads. FWIW, we're not increasing our bandwidth 
> for Galileo; 
> instead, we're depending very heavily on mirrors.
> 
> Bug 239668 [1] has been open for some time to discuss this issue.
> 
> We've decided that the best approach is something that we've been 
> calling the "Single File Hack". In this hack, we configure the p2 
> metadata (artifacts.xml) to send requests for some small 
> subset of the 
> files to eclipse.org. Ideally, we send requests for one plug-in or 
> feature for each thing that we need to track. The number of 
> files needs 
> to be kept relatively small.
> 
> There are problems with this hack. For one, eclipse.org 
> becomes a single 
> point of failure for all downloads. Further, we will have to let 
> organizations that mirror our downloads for internal consumption know 
> how to turn it off.
> 
> What we're going to need from each project is the names of the files 
> that we need to be tracking.
> 
> I'd love to hear your thoughts on this topic.
> 
> Wayne
> 
> [1]https://bugs.eclipse.org/bugs/show_bug.cgi?id=239668
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>