...

Summary information for Hoffmann

...

Recap:

During our recent test period of Sol in November, researchers confirmed that the new system's hardware and base configuration fully enables the proper functioning of all the software they currently depend on within Helios.

However, the testing also uncovered important information that we could not have learned without this testing investment, which in turn will contribute to a more robust setup for researchers. For example, Lulu has installed the Intel compiler in the /software/ directory, with the proper, legal licensing.

Upcoming to do's for Hoffmann researchers, as of 12/9/13:

...

researchers, as of 12/12/13:

Helios and Sol will be unavailable to researchers between Monday, Jan. 6th at 9am and approximately Thursday, Jan 9th, at noon.

• Before the shutdown, all data of value must be copied off of Sol.
  • All user directory data will be removed.
• Before the shutdown, all jobs on both Sol and Helios must be stopped.
  • Any remaining jobs will be stopped by ChemIT staff
• Only Sol will be up after this work is done.(Helios will no longer be available after all this is done.)
  • Data will be moved from Helios to Sol.
  • And Helios's nodes will be added to Sol.

Detailed information for cluster migration:

1) Reduce the amount of researchers’ temporary, test user data on Sol

...

to the bare minimum during the break.

• After the break, ChemIT will be deleting the content within everyone's test home directories on Sol. (They Those directories were originally copied there as a testing convenience during our test phase.)
  • We do this deletion before the final copy to ensure a clean import of researchers' researchers’ current production home directories from Helios.
  • Lulu advised researchers of this step before the Thanksgiving break, so hopefully no surprises.
• During the winter break, Sol's data will be backed up.
  • As with Helios, Sol's data will be copied and versioned to a dedicated, internal hard drive.
  • NEW: EZ-Backup will be enabled on Sol, providing off-site backup protection.

Bottom line: If there are any files on Sol which researchers cannot afford to lose (hopefully not- it was in test mode, and it is not being backed up!), researchers must move that data to Helios right away.

2) On a predetermined date, researchers will stop having access to Sol until it goes into production, replacing Helios.

• Tasks for ChemIT to do:
  • Get the server's versioning hard disk recognized within the new head node.
  • Ensure central software is installed properly and legal.

3) On a predetermined date, researchers must plan for 1-2 days of downtime for Helios (and before Sol is available).

At this time, Sol will still not be available to researchers. Thus, during this period when Helios, too, is turned off (to allow for its final set up), both Sol and Helios will be off-line to researchers.

...

(or elsewhere) before Sol is turned off.

2) Monday, Jan. 6th at 9am: Helios AND Sol being turned off to researchers

Researchers must know that:

• ALL jobs must cease by then. Any remaining jobs will be turned off by ChemIT.
• Also, there must be no remaining research data on Sol.

Tasks for ChemIT to do:

• Copy researchers' production home directory data from Helios to Sol.
• Move Helios's compute nodes into Sol's rack and reconnect all networking and power.
  • Add This allows us to add the older compute nodes from Helios, making Sol more powerful.
• Testing by ChemIT.

  4) About 1-2 days after Helios's shut down (3, above), Sol will be again be available, in a production capacity, to researchers.

  After Sol is in production, researchers must confirm all is working as expected. And they should improve their software installations.

  Tasks for ChemIT:

  • • Coordinate with Huayun (Deng) for initial testing, perhaps sometime Wednesday.

  3) Thursday, Jan 9th, at noon: Expect Sol available to researchers.

  • At this point, Sol will be up in full production and open to all researchers, with some or all of Helios's compute nodes attached.

  The Hoffmann group is encouraged to improve their software installations in order to create a more robust environment and to improve support outcomes.

  Tasks for Hoffmann group:

  • Confirm that Sol is "good". That Once Hoffmann researchers confirm Sol is "good" (that is, there is no need to "roll back" to using Helios due to some failure), Helios's head node can be added as a compute node to Sol.

  Tasks for Hoffmann group:

  • and for us to start again.
  • Hoffmann group Hoffmann group members are encouraged to install their shared group software in the /datahome/ Hoffmannhoffmann/bin/ directory. Including:
    • Materials Studio software. (Who? If Prasad is not available to do this, let's let’s discuss your options- thank you.)
    • vasp
    • gulp
    • etc.

  Tasks for ChemIT:

  • Once Hoffmann researchers confirm Sol is "good", Helios's head node can be added as a compute node to Sol.

  NOTE: Helios will no longer be available after all this is done.

  • Data will be moved from Helios to Sol.
  • And Helios's nodes will be added to Sol.

  Thank you !  -ChemIT

  ...

  Older or other notes:

  Data rates

  12/11/13: Users data to transfer from Helios to Sol is about 80GB. The transfer time, using rsync, is expected to take about 24 hours.

  Next steps

  • Meet to review all options and confirm desired direction and expected timing.
  • Review resources. Huayun Gen has cluster management experience, including set-up.

  ...