[Techtalk] keeping packages in sync

[John.Sturdy]

Ours is probably an odd case, as it's a beowulf cluster rather than a normal user network, but we use systemimager.  Here follow notes from our internal beowulf maintenance wiki (`wulffile' is the fileserver within the cluster):

  Image Updates

  Changing security updates and obsolete software mean that the beowulf will regularly have to be updated. This is normally done by upgrading a single rack machine (e.g rack1) using “apt-get upgrade”. This    operation will fetch and install any software that needs to be upgraded. Sometimes this may require that the machine be rebooted before the upgrade can take effect. It is important to supervise this process (it only takes a few minutes). Even if a reboot is not demanded, it is best to reboot anyway, just to be safe (from painful experience). Once this is done, prepare the SystemImager processes on the machine by running “si_prepareclient -server wulffile” Answer Yes to all questions. Now as root on wulffile, call the command “/root/bin/refreshImage.sh rack1”. This invokes a shell script which calls the SystemImager image update program. Answer yes to all questions except when it asks about updating the autoinstall script. The autoinstallation script had a bug in it and was replaced by a manually fixed script. Now rollout the image to all rack machines by calling the command “/root/bin/rolloutImage.sh”. This will connect to each rack machine and update the system based on the software in the rack image. Reboot the entire grid by calling the command “/root/bin/broadcast.sh reboot”. This should be ok.

  When upgrading wulffile, simply use “apt-get upgrade” and reboot. Do not try to reboot the rack and wulffile together after an upgrade, as the rack machines will come back online much faster than wulffile. They will then fail their DHCP init routines because wulffile is still booting up.

__John