Grace is a shared resource, and for the most part, a 'self-service' resource. In order to be fair to all users, we have set the following rules. Ignorance of the rules is not an acceptable excuse for bad behavior. To request a new account please e-mail hpcadmin@uams.edu after reading these rules acknowledging that you agree to abide by them.
- The login nodes are not meant for computation. Computationally intensive programs should be run as jobs using the queueing system.
- Users caught running computationally extensive programs (including containers) on the login nodes without permission from the system administrators may have their privileges on Grace suspended temporarily until after they have a conversation with a system administrator.
- The system administrators may kill computationally intensive programs running on the login nodes without warning to insure that other users are not adversely affected by the computationally intensive program.
- Compiling programs is allowed, if you have no facilities on your personal computer for compiling programs.
- Users may not login (e.g. use ssh) directly into any of the computational nodes, unless the user has an active job running on that node.
- Users caught logging into computational nodes may have their Grace accounts temporarily suspended until they have a conversation with a system administrator.
- The one exception is if you have a non-interactive job running on a particular computational node, you may ssh into that node to check that job's status, nothing else.
- Users should only log into the login nodes, or utilize the Open OnDemand portal (https://portal.hpc.uams.edu), and should only access the computational nodes through a scheduled and running job, nothing else.
- If you need an interactive shell or similar on a computational node, please run an interactive job..
- Launching a non-interactive job to do interactive work is strongly discouraged, and may lead to a conversation with a system administrator. Interactive work includes, for example, manually typing in shell commands, and should only be done from an interactive job, not a regular non-interactive job.
- Grace's Web Portal has a number options for graphical interactive jobs, such as Jupyter Lab/Notebook, R Studio Server, or a Linux desktop.
- Users shall accurately estimate the resources needed for a job, such as number of cores, amount of memory per core, wall clock time needed to run the job, etc.
- Failure to accurately estimate these resource requests, especially underestimating them, could lead to problems both for you and others in running jobs. Jobs that are exceeding their requested resources can be immediately terminated by the system administrators, to avoid damaging the system.
- Inaccuracies in resource requests also can lead to inefficiencies in the scheduler, which potentially impacts everyone.
- Users who consistently overestimate their resource requests, essentially reserving large blocks of resources with a job, but then not using them, could be penalized, for example by having the priorities of all of their jobs lowered. Flagrant violators, particularly users who 'camp' on an interactive job (e.g. Jupyter Lab or Notebook) without actively using it may have their accounts temporarily suspended until they have a conversation with a system administrator.
- Users may not log into any of the management nodes by any means for any purpose without permission from a system administrator.
- Users caught doing so may have their Grace accounts temporarily suspended until they have a conversation with the system administrator.
- Grace's storage system (e.g. /home, /scratch, /storage) is not an appropriate place to archive or permanently store data or programs.
- Data or programs that are not in routine use by running jobs should be offloaded by the user to other storage, such as ROSS (the Research Object Storage System), the Research NAS, a user's own workstation, lab or departmental storage, or even to cloud storage, such as Box.
- Keep in mind that Grace's storage system, though highly redundant and quite reliable, is not backed up. The system operators take no responsibility for any data left on Grace that hasn't been backed up by the user, or that cannot be recovered by external means.
- The system administrators have the option to remove data in any directory on Grace's storage systems that has not be accessed for 4 months or more (14 days if in /scratch, /local-scratch or /tmp).
- The data or programs in a user's or a group's home directory, up to 1 TB, is exempt from this rule and may be kept indefinitely.
- Users or groups that maintain more than 1 TB in their home or group directory may be asked to remove data to come within the 1 TB exemption limit. If the user or group does not drop usage to less than 1 TB after the request from the system administrators, then the 4 week rule will apply to that user's or group's home directory.
- If you have a special project that requires that data be held on Grace's internal storage systems for more than 4 months, please submit a proposal in writing to the system administrators (e-mail to hpcadmin@uams.edu is sufficient) , detailing:
- Succinctly, what the project is and why the space needs to be held on Grace's internal storage for more than 4 months.
- A short name for the project or group that will be used as the name of the top level directory where this data will reside.
- For a user-specific project, the top level directory is the user's home directory.
- For a group project, the top level directory is the group's shared directory, typically found in /storage.
- How much space the project anticipates needing.
- For how long the project or group anticipates keeping the space before archiving it elsewhere.
- What is the backup or archiving plan for that data. (The plan could be, 'this data does not require backup nor archiving.')
- Requests for additional space or longer storage durations will be reviewed, with possible further negotiations, before being approved. Please wait for an approval notice before exceeding the space limitation or expecting longer durations. Once approved, only data in the project directory (the short name) will be exempted from the 1 TB rule.
- Users who attempt to circumvent the space use policies, for example by creating bogus accounts, projects or groups, or by running scripts that 'touch' files to make them look like they are being used, when really they are not in active use, may have their accounts suspended until after they have a conversation with a system administrator.
- You should never put sensitive information, such as fully identified patient records, onto Grace or any of her storage systems. Grace is not considered a HIPAA or FERPA repository.
- Similarly, you should never attempt to access sensitive data on the UAMS network via Grace. Such activity would be reported, and could result in disciplinary actions. If you attempt unauthorized access to data protected by HIPAA or FERPA regulations, there could also be criminal charges brought by the government. Bottom
...
- line – just don't do it.
- You agree to maintain the confidentiality of any information that you may encounter that does not belong to you. If the data is not yours, you may not disclose it to anyone without permission.
- Although the upper limit to the number of jobs that slurm can comfortably manage is large, it is not infinite. Please check with a system administrator before submitting a bolus of more than 2,000 jobs within a 24 hour period, so that the system administrator can check to make sure sufficient resources are available. Also be aware that you could easily overwhelm the slurm scheduler if you submit jobs too rapidly, which could impact other users.
- If you are submitting a large number of related jobs, please consider using a Job Array to submit the entire set at once, rather than submitting all of the jobs individually. Job Arrays are much easier to administer and monitor, and often are more efficient than submitting individual jobs. Grace is configured to handle millions of tasks in a job array.
- The system administrators do have the right to look into your directories and jobs as needed to assure the smooth operation of Grace. They may suggest potential improvements or changes to the way you work, either to help your jobs be better "HPC Citizens" or to improve their efficiency. Since Grace is a shared resource, it is important to use the system gracefully (pun intended). Please do not feel offended if a system administrator approaches you. They are only trying to help you complete your work in a fashion that is fair to everyone.
- If Grace is misbehaving, please let the system administrators know. E-mail to hpcadmin@uams.edu
...
- is currently the best way to reach the administrators. They cannot fix things that they do not know about.
- The HPC system administration staff certainly is willing to answer questions and to assist with small problems, as long as such questions are infrequent. But the system administrators do not have the bandwidth to do major work for you, or to answer dozens of tiny questions whose answers could be found by simply reading the documents or searching the internet. Being largely 'self-service', Grace's users are expected to do their own research, do their own programming, do their own optimizing, and manage their own education on how to use an HPC. Users should only bother the system administrators if there is a problem with the system, and occasionally when the user is stuck or needs a little hint.
...
- If you need extensive assistance, there may be a need to set up a research partnership or project, e.g. with faculty or staff in the Department of Biomedical Informatics, or in the UAMS IT department. Obviously such partnerships or projects often require a financial arrangement to cover people's time.
Some general information about Grace that might be helpful:
...