[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cbi-dev] [eclipse.org-committers] Fwd: [eclipse-mirrors] Eclipse mirror size ~1.4 TB

Sam,

Adding the cbi-dev mailing list, as your question is a good one.

On 2019-04-15 6:13 p.m., Sam Davis wrote:

Given that there is no longer shell access to build.eclipse.org, what is the recommended way to do this cleanup? Create Jenkins build jobs that list directory contents in the console, decide what can be deleted, and then create another job to do the deletion?


We really need a file browsing mechanism, but in the meanwhile, your suggestion of using Jenkins jobs is how I recommend you do it. It's clunky, but it's all we have right now.

Denis






Sam

--
Sam Davis
Senior Software Engineer, Tasktop
Committer, Eclipse Mylyn
http://tasktop.com


On Mon, Apr 15, 2019 at 7:41 AM Denis Roy <denis.roy@xxxxxxxxxxxxxxxxxxxxxx> wrote:

Folks,

The disk footprint of mirrored content on download.eclipse.org is up to 1.4TB. We need to do a better job (much better) of keeping it small. I've filed https://bugs.eclipse.org/bugs/show_bug.cgi?id=546434 to track this.


download.eclipse.org should only be used to store current release builds. Archived builds should be moved to archive.eclipse.org, and nightly and integration builds should be deleted once stale.

See the docs for more info:
https://wiki.eclipse.org/IT_Infrastructure_Doc#Downloads

Below is a table of projects that seem to be consuming more disk space than I'd expect.

_ALL PROJECTS SHOULD CLEAN THEIR AREA, NOT JUST THOSE LISTED BELOW_

14G	4diac	
6.6G	acceleo	
15G	app4mc	
57G	birt	
11G	dirigible	
20G	e4	e4/sdk/drops 2011 -> move to archive.eclipse.org
72G	eclipsescada	
12G	ecp	
9.7G	efxclipse	
9.5G	epsilon	
16G	gemoc	
16G	ice	
25G	jdtls	
9.8M	keyple	
13G	kura	
6.5G	mat	
4.8G	mmt	
160G	modeling	
37G	emf	
36G	gmp	
72G	mdt	
8.0G	tmf	
14G	n4js	
41G	orion	
98G	rcptt	
76G	eclipselink	
38G	sirius	
18G	staging	
398G	epp	
71G	cdt	
110G	orbit	
41G	ptp	
36G	tracecompass	
40G	webtools	Lots of old stuff that can be moved


-------- Forwarded Message --------
Subject: [eclipse-mirrors] Eclipse mirror size ~1.4 TB
Date: Sat, 13 Apr 2019 19:29:29 +0100
From: GÃkdeniz KaradaÄ <gokdenizk@xxxxxxxxx>
Reply-To: Eclipse download mirror administrators discussion and announcements <eclipse-mirrors@xxxxxxxxxxx>
To: eclipse-mirrors@xxxxxxxxxxx


Hi Eclipse Mirror team,

Currently eclipse mirror is ~1.4 TB, surpassing both debian and ubuntu in size.

Is it possible to create a more compact rsync module containing most requested files?

For example OpenSuse has "hotstuff" modules for mirrors who want to provide value while conserving space and downstream sync bandwidth. (i.e. I don't like to download 375 versions of a file -over the life time of the mirror- for that single person over there using that project, they can go to the main mirror for that file)

--
Gokdeniz Karadag
_______________________________________________
eclipse.org-committers mailing list
eclipse.org-committers@xxxxxxxxxxx
https://www.eclipse.org/mailman/listinfo/eclipse.org-committers

IMPORTANT: Membership in this list is generated by processes internal to the Eclipse Foundation. To be permanently removed from this list, you must contact emo@xxxxxxxxxxx to request removal.