| Excerpt |
|---|
This list includes things that require the cluster to be turned off.page includes a list of to-do's to consider during maintenance periods (when the cluster is turned off). |
| Table of Contents |
|---|
Purpose
This checklist can be used for:
1) A list of things that require the cluster to be turned off. Or are tasks that are more safely done when the cluster is turned off, even if not strictly required.
2) A list of things to consider doing when the a clusters is being turned off / rebooted for other reasons.
Examples of activities taken in past maintenance windows:
During this down-time, we intend to:
- Test new GPU software capabilities
- Update the OS of the storage system.
- Update the BIOS of the 4 GPU clusters.
- Update the UPS software to address current software's limitations.
- Confirm backups and review other system software configurations.
Maintenance Checklist
Activity | Item/ topic | Who decidesknows if action possible or needed? (Presumed Lulu, unless marked otherwise) | Notes or comments | |||||
|---|---|---|---|---|---|---|---|---|
| P/T | Network / switches | Project lead or ticket owner | ||||||
| P/T | Disk changes / swaps / repairs | Project lead or ticket owner | ||||||
| P/T | Add / remove nodes | Project lead or ticket owner | ||||||
| D | Motherboard: BIOS / firmware updates (and why it's done...) | Michael | D | Scheraga only: Synology updates | Michael | (how get reminded/ asked?)
| Risk-management consideration. Vs. expected benefit. Q: Just head node? Q: CAC: What do they do? (ex. Ananth) Any other "storage system"? | |
| D | Firmware updates, including drives, cards, etc. | Michael (how get reminded/ asked?) | ||||||
| D | Scheraga only: Synology updates | Michael (how get reminded/ asked?) | Any other "storage system"? | |||||
D | Kernel updates
| |||||||
| D | OS Updates
| |||||||
| D | UPS / check / updates / settings: Review and confirm settings
| Confirm thresholds make sense, especially as battery gets older. |
Key:
P/T: Default answer is "no action". This is here in case a Project or Ticket has an impact on this topic, which is to be done during the maintenance work.
...