|Re: [cross-project-issues-dev] What's up with download.eclipse.org?|
How is the download content mirrored to Amazon S3?
E.g. For our cloudfront offerings I am using s3sync for doing this with settings like this:
s3sync.rb --recursive --public-read --progress --make-dirs ... bucket-name:...
To me it looks like there are too many "access denied" and maybe the public read access bit is not set. Maybe this is the root cause for the other failures?
Regards, Markus2009/9/29 Eric Gwin <eric.gwin@xxxxxxxxxx>Denis,
I don't know if the main proxy is just at capacity or if there is a bigger issue. I also don't know if this will help any (but I'm hoping so).
I'm seeing similar results when just trying to download from:
using the Main Download Site (Canada):
which remaps to:
<?xml version="1.0" encoding="UTF-8" ?>
Denis Roy wrote:
On 09/29/2009 11:14 AM, Thomas Hallgren wrote:Sorry, but you are stating the obvious. I would not deliberately open the taps if I didn't think we were ready. Really. We did mirror tests. But then we had one of those unforeseen problems (see bug 289408 comment 37 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=289408#c37>). The problem is that there are many moving parts to these releases, and we seem unable to foresee those unforeseen problems.
If this list is sorted, I think the top 40 files or so (down to the first zip) fit well under 50Mb so uploading them should be easy. Perhaps P2 meta-data should be prioritized for upload to cloudfront given the request frequency for those files?
p2 metadata seems to be scattered across our directory tree, making it difficult to strategically redirect content. It's also easy to draw up a plan 5 days after the release. I mean, would you have expected /tools/mylyn/update/e3.4/content.jar to be a top-3 downloaded file?
But all in all, as I said earlier, I think the process when we release needs to be improved. We can't open the taps until we are absolutely sure that the infrastructure is ready to take the hit.
I know this is frustrating. I've spent the last five years trying to make these releases as painless as possible, but as long as our servers and your code keep evolving, there are always going to be some unexpected results.
### phone: +49 721 664 733 0 (GMT +2)
### fax: +49 721 664 733 29
### web: www.eclipsesource.com
Innoopract Informationssysteme GmbH
Stephanienstrasse 20, 76133 Karlsruhe Germany
General Manager: Jochen Krause
Registered Office: Karlsruhe, Commercial Register Mannheim HRB 107883
cross-project-issues-dev mailing list
Back to the top