Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[] Infra update: fixing servers, more disk space soon


After last weekend's chaotic upgrade, I thought I'd inform you on what is going on behind the scenes:

- Both of our database/file servers have been updated and rebooted, and the file locking issues are resolved.  We're still running only one file server, however [1].

- Our second database server is back into production, so Bugzilla should no longer lag while long queries run.

- Two new storage servers are standing by, with 5 Terabytes of redundant[2] disk space.  The downloads and archives areas will be moved to a 4TB space, thus freeing 600+GB of space for CVS, SVN, Bugzilla[3] and everything else while improving disk performance for all our services. 

- The build server will inherit a 1TB working area (/shared).

The new storage servers will be placed into production next weekend, Saturday Feb. 21 at 10:00am ET.  At that time, many file systems will be read-only (for real this time) for about one hour so I can get exclusive access for the migration.  No file paths or locations will change.

Thanks for reading.  If you have any questions or concerns, please feel free to contact me directly at webmaster@xxxxxxxxxxx.


[1] I haven't put our secondary file server back in service because I'm using it to sync the 600+ GB of data to the new storage servers (see [2])

[2] For those who like technical details, the two servers each have 5TB of storage (8 x 750GB disks in a RAID-5 configuration) which are connected to our file server via a dedicated SAN.  Both servers are configured in a RAID-1 mirror.  This nested RAID setup can tolerate the failure of multiple disks, or even an entire server, without any downtime or serious performance degradation, while giving fantastic file access performance.

[3] Did you know our Bugzilla database is now 13G in size?

Back to the top