|[cross-project-issues-dev] Release engineers/Builders : infrastructure changes coming soon|
We're planning a substantial disk capacity increase this quarter by adding a new disk array for the download.eclipse.org/archive.eclipse.org files, and to store the build.eclipse.org/CommonBuild working directories.
With this upgrade, our ability to freeze the download.eclipse.org content will no longer be. We currently use rsync to mirror the download files from one server to another, but with our large downloads tree, this technique is not only inefficient, but it has a performance impact on everything else.
For major releases such as Galileo, we will need to substitute our usual content freeze by simply not publishing links to newly uploaded files for some time. For most projects, this is simply a matter of delaying a page commit for 24 hours, letting the new files sit silently while no links are published. However, some projects use PHP to dynamically read the directory structure and show links for files the minute they appear. This is problematic, and this is the reason we started doing freezes in the first place.
If your project's download page reads the directory structure, you will need to discontinue this approach. Here are two proposed alternatives, in the order of my preference:
1. Keep your current 'dynamic' PHP page, but use it to generate a static HTML version of the download page every X hours via a cronjob. The static HTML version of your page becomes your main download page. The added bonus is each hit to your (static) downloads page no longer generates multiple disk accesses for reading directory structures.
2. You alter your current PHP page to examine the date/time of the files, and only display files older than X hours. The problem here is that your page is still wastefully inspecting directory contents for each hit. Furthermore, if the date/time of a file is mangled somehow (for instance, you upload with rsync -t to preserve times), the files on download.eclipse.org may appear to be older than they are.
Option 1 is not only superior, but it's also likely the one that will require less work. It could be as easy as a) rename index.php to something.php b) add 'wget' cronjob to dump the HTML output of something.php to index.php c) carry on normal activities.
We don't yet have a definitive date for the upgrade, but we're expecting it to happen on a weekend before the end of March. As always, if your download pages need to be fixed, we're here to help you find a solution to make this work. Feel free to open a bug and CC webmaster@xxxxxxxxxxx.
Back to the top