Excerpt |
---|
The new Matrix is faster, but it is different. Learn about the differences here to reduce your aggravation. |
Table of Contents |
---|
TIP: This is an easy-to-remember web address to this page:
(1) Information related to Phase 2 Matrix testing (Full Scheraga group)
End-user reporting ticket number is INC000001223799.
|
---|
How is the new
...
2014 Matrix different than the old Matrix?
- Link to graphical Graphical representation comparing the old Matrix to the new Matrix, for researchers.
- Lin to text page containing full Full details on the new Matrix configurations. For both researchers and support staff.
- Matrix Partitioning Configuration 2014-09-15.pdf
- Includes conventions, intentions, and associated technically-imposed limitations.
...
Deadlines for researchers' testing
DATE 1:
...
Monday, 10/
...
19/14: By this date, all researchers are expected to at least simply login (verifying that their account itself works).
- KEEP? Email us once you are in and working by replying to this message using your Cornell account and keep the subject & tracking number.
DATE 2:
...
Thursday, 10/
...
22/14: By this date, all researchers will need to test and either approve, or report problems with, the software they depend on.
- Report details of any questions or problems you find to the appropriate researcher.
- See section above, "Each researcher must confirm that the applications they depend on work for them on the new Matrix.", for who to report to.
DATE 3:
...
Monday, 10/
...
27/14: By this date we expect to have all testing completed, as long as everything is indeed working as expected.
- Any delays in your testing will extend the project schedule.
...
Upcoming scheduled events (dove-tailing with above deadlines for researchers)
- DayOfWeekThursday, dateOct. 15: ChemIT makes a copy of the end-user's home directory (on the old Matrix) onto the end-user's storage directory (on the new Matrix), for testing purposes.
- Thursday, 10/Oct. 16: End-user testing starts.
- DayOfWeekFriday, dateOct 31: ChemIT deletes copy of end-user's storage (on the new Matrix). ChemiT retains copy of end-user's home directory (on the new Matrix).
- See section below, "Important notes in what data is erased. And what data is saved."
- Monday, November 3 - old Matrix goes off-line to begin converting all nodes and transfer user data to new cluster.
Notes to get you started and keep you going
During testing, there is a the new and an old cluster are both available to you at the same time.
...
- Thus, you will be able to log into two different home directories, within two completely different systems:
SSH to the new (Test) system at matrixtest.scheraga.chem.cornell.edumatrixtest.
- The old system remains at scheraga.chem.cornell.edu
- Your account ID and password on the new Matrix are the same as the old Matrix.
- See section below, "Get access to your account", within the "(2) Information relevant for testing and after testing" section.
- Continue to use the old Matrix cluster for your production research work.
- Use the new cluster ONLY to confirm it will work for you once we cut-over from the old Matrix. Do not use it for production research.
- Interruptions to the new Matrix during this testing phase may occur at any time, with no advanced warning.
...
- Selectively choose (from storage) just the files you need for running jobs.
- For your convenience, ChemIT has pre-configured your home directory in the following way:...by copying your .ssh/ directory and your .tcshrc file from your /storage directory into your /home directory.
A recent snapshot copy of your data from old Matrix has been put on the storage system in "/storage/netID".
- Storage is where non-actively used, Scheraga-related, research files belongshould be saved.
- In production (NOT DURING TESTING!), you will move results and data you want to save long term back to storage
- REMEMBER: All date in "/storage/netID" will be deleted after this testing phase.
- This deletion will allow us to move your current, production data from the old Matrix, when we cut-over.
- REMEMBER: All date in "/storage/netID" will be deleted after this testing phase.
- Phase One researchers only: Your /storage/netID has been carried forward from your initial testing.
- REMINDER: It, too, will be deleted before the cut-over from the old Matrix.
...
- Leave files in storage which you don't need for your jobs.
- Keeping your home directory small aids in disaster recovery for the entire group.!
ChemIT will maintain a chart to help the group track what does and does not work regarding MPI.
...
- See the storage chart for more details on disks & partitions in the new cluster, and how that compares to the old Matrix.
- All research applications are installed under “/software”.
- Get instantly more space, for temporary use, in your home directory, by using the "/notbackedup" disk.
- See section above, "How is the new Fall 2014 Matrix different than the old Matrix?", for links to more details.
- Get info on the nodes on the new Matrix which are available during testing.
- See chart with full details:
- Summary info, true during Phase Two testing:
- Initially about 18 17 nodes will be made available to all researchers.
- This represents the newly purchased nodes and two of each of 5 types of old node.
- There are 8 new compute nodes (m108-m115) accessible. Each new node has 20 cores available.
- For testing, select old Matrix nodes will be have been moved to the new Matrix to ensure adequate computing capacity during testing.
- During testing, GPU's are not available in the new cluster.However, 2 of their At this point, 1 or 2 of GPU nodes will be running as just CPU nodes. GPU functions will be added at a later date if possible.
- Initially about 18 17 nodes will be made available to all researchers.
- See chart with full details:
- Queuing information
- No Express queue available for testing.
- An Express queue will be provisioned when new Matrix is in production.
- The default queue is dque instead of express as no node has been assigned to be available for Express queue.
- No Express queue available for testing.
...
- ssh to:
- During test:
matrixtest.scheraga.chem.cornell.edu
- When new Matrix is in production, will be use the new cname:
- matrix.chem.cornell.edu
- N.B. We were going to use the same cname as
- was used for production
- before (scheraga.chem.cornell.edu).
- During test:
- Use your username and your Matrix-specific password.
...
- Your home directory is "/users/netID".
- In this location, just retain the files you need for running jobs.
- Place files into storage which you don't need for your current jobs.
- KEY: Keeping your home directory small aids in disaster recovery for the entire group.
- Get instantly more space, for temporary use, in your home directory, by using the "/notbackedup" temporary disk.
- See section below, "How to run jobs requiring lots of space in ones home directory".
- Your storage directory is "/storage/netID".
- Storage is where non-actively usedactive, Scheraga-related, research files belong.
- In production (NOT DURING TESTING!), you will move results and data you want to save long term back to storage
- REMEMBER: All date in "/storage/netID" will be deleted after this testing phase.
- This deletion will allow us to move your current, production data from the old Matrix, when we cut-over.
- All research applications are installed under “/software”.Get instantly more space, for temporary use, in your home directory, by using the "/notbackedup" disk.See section below, "
How to run jobs requiring lots of space in ones home directory
...
How to run jobs requiring lots of space in ones home directory
- Explain (to be added) purpose and use of /notbackedup partition to expand a user's effective home directory's storage capacity.Elaborate on how-to, here...
- If using the /notbackedup is not adequate, researcher can explain this and request a larger quota through their Group Cluster Representative (Gia, as of 10/2014)
Matrix user quotas
New user's default quotas
The group has instructed ChemIT to make the following defaults on a new user's account:
- 50GB 10GB /home
- 50GB /storage
A new user can request a larger quota through their Group Cluster Representative (Gia, as of 10/2014.)
...