You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

The new Matrix is faster, but it is different. Learn about the differences here to reduce your aggravation.

(1) Information related to testing

How is the new Fall 2014 Matrix different than the old Matrix?

End-user reporting ticket number is INC...

Work to be done during testing: Who is responsible for what?

Each researcher must confirm that the applications they depend on work for them on the new Matrix.

If and when you can confirm that all works as expected:

=> notify ChemIT and include the INC... ticket number in the email. Thank you!

If you have a problem with an application or MPI issues:

=> Try working it out by yourself.
    • You should understand how and why your application works.
    • Almost all problems are not the application itself. Instead, it's almost always how a researcher has their scripts configured to use an application.
=> If you can't fix it yourself, please contact the appropriate group member; see below for who to contact. (Do not contact ChemIT for this, please.)
    • Researchers in Poland should contact Czarek. Follow his instructions, please.
    • All others should contact the group member assigned as lead tester for the application they are having problems with. Follow their instructions, please.

Deadlines for researchers

DATE 1: DayOfWeek, 10/xx/14: By this date, all researchers are expected to at least simply login (verifying that their account itself works).

  • KEEP? Email us once you are in and working by replying to this message using your Cornell account and keep the subject & tracking number.

DATE 2: DayOfWeek, 10/xx/14: By this date, all researchers will need to test and either approve, or report problems with, the software they depend on.

  • Report details of any questions or problems you find to the appropriate researcher.
    • See section above, "Each researcher must confirm that the applications they depend on work for them on the new Matrix.", for who to report to.

DATE 3: DayOfWeek, 10/xx/14: By this date we expect to have all testing completed – as long as everything is indeed working as expected.

  • Any delays in your testing will extend the project schedule.

Testing schedule activities by ChemIT

What's been done, through Phase One

  • The system, with all requested software, has been installed.
  • All hardware has been tested.
  • All applications have been test and confirmed working by designated group researchers.

Phase Two's schedule (dovetailing with above deadlines for researchers)

DayOfWeek, date: ChemIT makes a copy of the end-user's home directory (on the old Matrix) onto the end-user's storage directory (on the new Matrix), for testing purposes.

Thursday, 10/16: End-user testing starts.

DayOfWeek, date: ChemIT deletes copy of end-user's storage (on the new Matrix). ChemiT retains copy of end-user's home directory (on the new Matrix).

  • See section below, "Important notes in what data is erased. And what data is saved."

Notes, to get you started and keep you going

During testing, there is a new and an old cluster available to you at the same time.

  • Thus, you will be able to log into two different home directories, within two completely different systems:

    • scheraga.chem.cornell.edu

    • matrixtest.scheraga.chem.cornell.edu

  • Your account and password on the new Matrix are the same as the old Matrix.
  • Continue to use the old Matrix cluster for your production research work.
  • Use the new cluster ONLY to confirm it will work for you once we cut-over from the old Matrix. Do not use it for production research.
    • Interruptions to the new Matrix during this testing phase may occur at any time, with no advanced warning.

Particulars you need to know about new Matrix cluster, for testing:

Your home directories, "/users/netID", are basically EMPTY of your files to start.

  • Selectively choose (from storage) just the files you need for running jobs.
  • For your convenience, ChemIT has pre-configured your home directory in the following way:....

A copy of your data from old Matrix has been put on the storage system in  "/storage/netID".

  • Storage is where non-actively used, Scheraga-related, research files belong.

You thus need to move or copy files needed to run jobs from storage to your new home "/users/netID" to test jobs.

  • Leave files in storage which you don't need for your jobs.
  • Keeping your home directory small aids in disaster recovery for the entire group.

In production (NOT DURING TESTING!), you will move results and data you want to save long term back to storage

ChemIT will maintain a chart to help the group track what does and does not work regarding MPI.

For testing, some old Matrix nodes will be moved to the new Matrix to ensure adequate computing capacity during testing. Initially 10 nodes will be made available.

Other details
  • All research applications are installed under “/software”.
  • See the storage chart for more details on disks & partitions in the new cluster, and how that compares to the old Matrix.
  • The default queue is dque instead of express as the node is not available for express queue.

Important notes in WHAT DATA IS ERASED after testing. And what data is saved after testing.

ChemIT will save all end-user's home directories as they build them up on the new Matrix.

  • This is to preserve all researcher's investments in getting their research files to work on the new Matrix.

ChemIT will erase all end-user's storage directories on the new Matrix, at end of testing.

  • On the day we cut-over from the old Matrix to the new Matrix, ChemIT will make a final copy of each user's old Matrix home directory into their new Matrix storage directory.

ChemIT will create a snap-shot copy of each end-user's current home directory (from the production Matrix) and place it into the end-user's storage directory (on the new Matrix).

(2) Information relevant for testing and after testing

Overall information

  • Text containing full details on the new Matrix configurations. For researchers and support staff.
    • Includes conventions, intentions, and associated technically-imposed limitations.

Get access to your account

  • ssh to ....

New user's defaults

The group has instructed ChemIT to make the following defaults on a new user's account:

  • 50GB /home
  • 50GB /storage

A new user can request a larger quota through their Group Cluster Representative (Gia, as of 10/2014.)

How to run jobs requiring lots of space in ones home directory

  • Explain purpose and use of /notbackedup partition to expand a user's effective home directory's storage capacity.
    • Elaborate on how-to, here...
  • If using the /notbackedup is not adequate, researcher can explain this and request a larger quota through their Group Cluster Representative (Gia, as of 10/2014)
  • No labels