Today I did brain surgery on a running compute cluster. I swapped out the head node (the scheduler, or the interactive machine depending on when you learned this stuff) for a new one. I did this with thousands of jobs sitting in the queue and some of them still running. I did this without dropping any jobs and without the users really noticing.
In my own little world, this counted as pretty badass.
Leave a Reply