Re: [cross-project-issues-dev] List of files on Eclipse Mirror servers

[Denis Roy <denis.roy@xxxxxxxxxxx>]

On 06/19/2012 03:40 PM, Konstantin Komissarchik wrote:

Currently, only the mirror selection script is able to delegate to archive.eclipse.org, so auto-archiving would break those repositories.

That gives me an idea.

We currently use a small PHP script to handle 404's on download.eclipse.org.  If the request comes from Eclipse or a robot, we send a small 404 response.  If it's from a browser, we send the stunningly beautiful page you see here:  http://downlaod.eclipse.org/khasjkas

We could alter the script to return a 301redirect if the 404 file in question is indeed found on archive.eclipse.org.  This means in theory, you _could_ move files and p2 repos to the archives without any breakage at all.

Or even better, the PHP script could just send the file's contents down the wire with a 200 response, no redirects.

I would be concerned about the increased CPU load, since download.e.o currently handles about 6 million 404's a day.  But it would likely be a very elegant solution.

Denis

 

If there are resources available to work on this, it would be much better if download.eclipse.org served as central gateway to both the primary download server and the archive. Then one can implement all sort of archiving policies without involving projects or affecting end users referencing these URLs… You can auto-archive based on age, you can auto-archive based on demand, heck you can even pick a goal for how much to mirror and have a heuristic that shuffles between primary and archive to meet that goal.

 

- Konstantin

 

 

From: cross-project-issues-dev-bounces@xxxxxxxxxxx [mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of Wayne Beaton
Sent: Tuesday, June 19, 2012 12:15 PM
To: cross-project-issues-dev@xxxxxxxxxxx
Subject: Re: [cross-project-issues-dev] List of files on Eclipse Mirror servers

 

Does it make sense to consider moving files that are old--say three or more years old--to the archive server automatically?

Wayne

On 06/19/2012 10:33 AM, Denis Roy wrote:

To help you clean up older builds and files from the Eclipse download server, I've compiled a list of files that are actually sync'ed to our mirrors.

http://download.eclipse.org/technology/phoenix/download.eclipse.org-filelist.txt.gz

I each project to look at the file list, and either remove files that are no longer needed, or move older releases[1] to the archives.

Thanks,

Denis


[1] http://wiki.eclipse.org/IT_Infrastructure_Doc#Downloads