You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 16 Next »

The new Matrix is faster, but it is different. Learn about the differences here to reduce your aggravation.

(1) Information related to testing

How is the new Fall 2014 Matrix different than the old Matrix?

End-user reporting ticket number is INC...

Work to be done during testing: Who is responsible for what?

Each researcher must confirm that the applications they depend on work for them on the new Matrix.

If and when you can confirm that all works as expected:

=> notify ChemIT and include the INC... ticket number in the email. Thank you!

If you have a problem with an application or MPI issues:

=> Try working it out by yourself.
    • You should understand how and why your application works.
    • Almost all problems are not the application itself. Instead, it's almost always how a researcher has their scripts configured to use an application.
=> If you can't fix it yourself, please contact the appropriate group member; see below for who to contact. (Do not contact ChemIT for this, please.)

Deadlines for researchers

DATE 1: DayOfWeek, 10/xx/14: By this date, all researchers are expected to at least simply login (verifying that their account itself works).

  • KEEP? Email us once you are in and working by replying to this message using your Cornell account and keep the subject & tracking number.

DATE 2: DayOfWeek, 10/xx/14: By this date, all researchers will need to test and either approve, or report problems with, the software they depend on.

  • Report details of any questions or problems you find to the appropriate researcher.
    • See section above, "Each researcher must confirm that the applications they depend on work for them on the new Matrix.", for who to report to.

DATE 3: DayOfWeek, 10/xx/14: By this date we expect to have all testing completed – as long as everything is indeed working as expected.

  • Any delays in your testing will extend the project schedule.

Testing schedule activities by ChemIT

What's been done, through Phase One

  • The system, with all requested software, has been installed.
  • All hardware has been tested.
  • All applications have been test and confirmed working by designated group researchers.

Phase Two's schedule (dovetailing with above deadlines for researchers)

DayOfWeek, date: ChemIT makes a copy of the end-user's home directory (on the old Matrix) onto the end-user's storage directory (on the new Matrix), for testing purposes.

Thursday, 10/16: End-user testing starts.

DayOfWeek, date: ChemIT deletes copy of end-user's storage (on the new Matrix). ChemiT retains copy of end-user's home directory (on the new Matrix).

  • See section below, "Important notes in what data is erased. And what data is saved."

Notes, to get you started and keep you going

During testing, there is a new and an old cluster available to you at the same time.

QuickStart info
  • Thus, you will be able to log into two different home directories, within two completely different systems:

    • scheraga.chem.cornell.edu

    • matrixtest.scheraga.chem.cornell.edu

  • Your account and password on the new Matrix are the same as the old Matrix.
  • Continue to use the old Matrix cluster for your production research work.
  • Use the new cluster ONLY to confirm it will work for you once we cut-over from the old Matrix. Do not use it for production research.
    • Interruptions to the new Matrix during this testing phase may occur at any time, with no advanced warning.
Detailed info, particularly if QuickStart above is insufficient

Elaborate on "We will provide a second email with instructions for getting to the new cluster and setting up your home directory, and how storage is set up."

Particulars you need to know about new Matrix cluster, for testing:

Your home directories, "/users/netID", are basically EMPTY of your files to start.

  • Selectively choose (from storage) just the files you need for running jobs.
  • For your convenience, ChemIT has pre-configured your home directory in the following way:....

A copy of your data from old Matrix has been put on the storage system in  "/storage/netID".

  • Storage is where non-actively used, Scheraga-related, research files belong.
  • In production (NOT DURING TESTING!), you will move results and data you want to save long term back to storage
    • REMEMBER: All date in "/storage/netID" will be deleted after this testing phase.
      • This deletion will allow us to move your current, production data from the old Matrix, when we cut-over.

You thus need to move or copy files needed to run jobs from storage to your new home "/users/netID" to test jobs.

  • Leave files in storage which you don't need for your jobs.
  • Keeping your home directory small aids in disaster recovery for the entire group.

ChemIT will maintain a chart to help the group track what does and does not work regarding MPI.

Other details
  • See the storage chart for more details on disks & partitions in the new cluster, and how that compares to the old Matrix.
    • All research applications are installed under “/software”.
    • See section above, "How is the new Fall 2014 Matrix different than the old Matrix?", for details.
  • Info on the nodes on the new Matrix which are available during testing.
    • See chart with full details:
    • Summary info, true during Phase Two testing:
      • Initially about 16 nodes will be made available to all researchers.
        • This represents the newly purchased nodes and two of each type of old node.
        • There are 8 new compute nodes (m108-m115) accessible. Each new node has 20 cores available.
      • For testing, select old Matrix nodes will be moved to the new Matrix to ensure adequate computing capacity during testing.
      • During testing, GPU's are not available in the new cluster.
  • Queuing information
    • No Express queue available for testing.
      • An Express queue will be provisioned when new Matrix is in production.
    • The default queue is dque instead of express as the node is not available for express queue.

Important notes in WHAT DATA IS ERASED after testing. And what data is saved after testing.

ChemIT will save all end-user's home directories as they build them up on the new Matrix.

  • This is to preserve all researcher's investments in getting their research files to work on the new Matrix.

ChemIT will erase all end-user's storage directories on the new Matrix, at end of testing.

  • On the day we cut-over from the old Matrix to the new Matrix, ChemIT will make a final copy of each user's old Matrix home directory into their new Matrix storage directory.

ChemIT will create a snap-shot copy of each end-user's current home directory (from the production Matrix) and place it into the end-user's storage directory (on the new Matrix).

(2) Information relevant for testing and after testing

Overall information

  • Text containing full details on the new Matrix configurations. For researchers and support staff.
    • Includes conventions, intentions, and associated technically-imposed limitations.

Get access to your account

  • ssh to ....

New user's defaults

The group has instructed ChemIT to make the following defaults on a new user's account:

  • 50GB /home
  • 50GB /storage

A new user can request a larger quota through their Group Cluster Representative (Gia, as of 10/2014.)

How to run jobs requiring lots of space in ones home directory

  • Explain purpose and use of /notbackedup partition to expand a user's effective home directory's storage capacity.
    • Elaborate on how-to, here...
  • If using the /notbackedup is not adequate, researcher can explain this and request a larger quota through their Group Cluster Representative (Gia, as of 10/2014)
  • No labels