TIP: This is an easy-to-remember web address to this page:

• http://tinyurl.com/MatrixChemITDocs

(1) Information related to Phase 2 Matrix testing (Full Scheraga group)

How is the new

...

2014 Matrix different than the old Matrix?

• Link to graphical Graphical representation comparing the old Matrix to the new Matrix, for researchers.
  • Matrix Storage Layout 20141003.pdf
• Lin to text page containing full Full details on the new Matrix configurations. For both researchers and support staff.
  • Matrix Partitioning Configuration 2014-09-15.pdf
  • Includes conventions, intentions, and associated technically-imposed limitations.

...

=> Notify ChemIT and include the INC... INC000001223799 ticket number in the email. Thank you!

...

Deadlines for researchers' testing

DATE 1:

...

Monday, 10/

...

19/14: By this date, all researchers are expected to at least simply login (verifying that their account itself works).

• KEEP? Email us once you are in and working by replying to this message using your Cornell account and keep the subject & tracking number.

DATE 2:

...

Thursday, 10/

...

22/14: By this date, all researchers will need to test and either approve, or report problems with, the software they depend on.

• Report details of any questions or problems you find to the appropriate researcher.
  • See section above, "Each researcher must confirm that the applications they depend on work for them on the new Matrix.", for who to report to.

DATE 3:

...

Monday, 10/

...

27/14: By this date we expect to have all testing completed, as long as everything is indeed working as expected.

• Any delays in your testing will extend the project schedule.

...

Upcoming scheduled events (dove-tailing with above deadlines for researchers)

• DayOfWeekThursday, dateOct. 15: ChemIT makes a copy of the end-user's home directory (on the old Matrix) onto the end-user's storage directory (on the new Matrix), for testing purposes.
• Thursday, 10/Oct. 16: End-user testing starts.
• DayOfWeekFriday, dateOct 31: ChemIT deletes copy of end-user's storage (on the new Matrix). ChemiT retains copy of end-user's home directory (on the new Matrix).
  • See section below, "Important notes in what data is erased. And what data is saved."
• Monday, November 3 - old Matrix goes off-line to begin converting all nodes and transfer user data to new cluster.

Notes to get you started and keep you going

During testing, there is a the new and an old cluster are both available to you at the same time.

...

• Thus, you will be able to log into two different home directories, within two completely different systems:

  • SSH to the new (Test) system at matrixtest.scheraga.chem.cornell.edumatrixtest.

  • The old system remains at scheraga.chem.cornell.edu
• Your account ID and password on the new Matrix are the same as the old Matrix.
  • See section below, "Get access to your account", within the "(2) Information relevant for testing and after testing" section.
• Continue to use the old Matrix cluster for your production research work.
• Use the new cluster ONLY to confirm it will work for you once we cut-over from the old Matrix. Do not use it for production research.
  • Interruptions to the new Matrix during this testing phase may occur at any time, with no advanced warning.

...

• Selectively choose (from storage) just the files you need for running jobs.
• For your convenience, ChemIT has pre-configured your home directory in the following way:....by copying your .ssh/ directory and your .tcshrc file from your /storage directory into your /home directory.

A recent snapshot copy of your data from old Matrix has been put on the storage system in  "/storage/netID".

• Storage is where non-actively used, Scheraga-related, research files belongshould be saved.
• In production (NOT DURING TESTING!), you will move results and data you want to save long term back to storage
  • REMEMBER: All date in "/storage/netID" will be deleted after this testing phase.
    • This deletion will allow us to move your current, production data from the old Matrix, when we cut-over.
• Phase One researchers only: Your /storage/netID has been carried forward from your initial testing.
  • REMINDER: It, too, will be deleted before the cut-over from the old Matrix.

...

• Leave files in storage which you don't need for your jobs.
• Keeping your home directory small aids in disaster recovery for the entire group.!

ChemIT will maintain a chart to help the group track what does and does not work regarding MPI.

...

• See the storage chart for more details on disks & partitions in the new cluster, and how that compares to the old Matrix.
  • All research applications are installed under “/software”.
  • Get instantly more space, for temporary use, in your home directory, by using the "/notbackedup" disk.
  • See section above, "How is the new Fall 2014 Matrix different than the old Matrix?", for links to more details.
• Get info on the nodes on the new Matrix which are available during testing.
  • See chart with full details:
    
    • Matrix compute nodes
  • Summary info, true during Phase Two testing:
    
    • Initially about 18 17 nodes will be made available to all researchers.
      • This represents the newly purchased nodes and two of each of 5 types of old node.
      • There are 8 new compute nodes (m108-m115) accessible. Each new node has 20 cores available.
    • For testing, select old Matrix nodes will be have been moved to the new Matrix to ensure adequate computing capacity during testing.
    • During testing, GPU's are not available in the new cluster.However, 2 of their At this point, 1 or 2 of GPU nodes will be running as just CPU nodes. GPU functions will be added at a later date if possible.
• Queuing information
  • No Express queue available for testing.
    • An Express queue will be provisioned when new Matrix is in production.
  • The default queue is dque instead of express as no node has been assigned to be available for Express queue.

...

• ssh to:
  • During test:

    • matrixtest.scheraga.chem.cornell.edu

  • When new Matrix is in production, will be use the new cname:
    
    • matrix.chem.cornell.edu
    • N.B. We were going to use the same cname as
    is
    • was used for production
    now:
    • before (scheraga.chem.cornell.edu).
• Use your username and your Matrix-specific password.

...

Learn to effectively use your home directory, along with /storage, /notbackedup, and /software.

• Your home directory is "/users/netID".
  • In this location, just retain the files you need for running jobs.
  • Place files into storage which you don't need for your current jobs.
  • KEY: Keeping your home directory small aids in disaster recovery for the entire group.
  • Get instantly more space, for temporary use, in your home directory, by using the "/notbackedup" temporary disk.
    • See section below, "How to run jobs requiring lots of space in ones home directory".

• Your storage directory is "/storage/netID".
  • Storage is where non-actively usedactive, Scheraga-related, research files belong.
  • In production (NOT DURING TESTING!), you will move results and data you want to save long term back to storage
    • REMEMBER: All date in "/storage/netID" will be deleted after this testing phase.
    • This deletion will allow us to move your current, production data from the old Matrix, when we cut-over.
• All research applications are installed under “/software”.Get instantly more space, for temporary use, in your home directory, by using the "/notbackedup" disk.See section below, "

How to run jobs requiring lots of space in ones home directory

...

How to run jobs requiring lots of space in ones home directory

• Explain (to be added) purpose and use of /notbackedup partition to expand a user's effective home directory's storage capacity.Elaborate on how-to, here...
• If using the /notbackedup is not adequate, researcher can explain this and request a larger quota through their Group Cluster Representative (Gia, as of 10/2014)

Matrix user quotas

• Matrix users information ( name, netid, status, quota )

New user's default quotas

The group has instructed ChemIT to make the following defaults on a new user's account:

• 50GB 10GB /home
• 50GB /storage

A new user can request a larger quota through their Group Cluster Representative (Gia, as of 10/2014.)

...