Greetings,
After last weekend's chaotic upgrade, I thought I'd inform you on what
is going on behind the scenes:
- Both of our database/file servers have been updated and rebooted, and
the file locking issues are resolved. We're still running only one
file server, however [1].
- Our second database server is back into production, so Bugzilla
should no longer lag while long queries run.
- Two new storage servers are standing by, with 5 Terabytes of
redundant[2] disk space. The downloads and archives areas will be
moved to a 4TB space, thus freeing 600+GB of space for CVS, SVN,
Bugzilla[3] and everything else while improving disk performance for
all our services.
- The build server will inherit a 1TB working area (/shared).
The new storage servers will be placed into production next weekend,
Saturday Feb. 21 at 10:00am ET. At that time, many file systems will
be read-only (for real this time) for about one hour so I can get
exclusive access for the migration. No file paths or locations will
change.
Thanks for reading. If you have any questions or concerns, please feel
free to contact me directly at webmaster@xxxxxxxxxxx.
Denis
[1] I haven't put our secondary file server back in service because I'm
using it to sync the 600+ GB of data to the new storage servers (see
[2])
[2] For those who like technical details, the two servers each have 5TB
of storage (8 x 750GB disks in a RAID-5 configuration) which are
connected to our file server via a dedicated SAN. Both servers are
configured in a RAID-1 mirror. This nested RAID setup can tolerate the
failure of multiple disks, or even an entire server, without any
downtime or serious performance degradation, while giving fantastic
file access performance.
[3] Did you know our Bugzilla database is now 13G in size?
|