Get Started

Stern users

For those in the Stern School of Business, you can use your existing activated Stern account, i.e., Stern NetID and password, to access SCRC services and resources. If you haven’t already, activate your Stern account.

Non-Stern NYU users

Those already at NYU but not in Stern (and not enrolled in a Stern course) must have a full-time Stern employee make a request on their behalf by contacting the Stern IT Help Desk at helpdesk@stern.nyu.edu or 212-998-0180 and provide the following information:

• Full Name
• NYU NetID
• NYU UniversityID (N#)
• Date of birth
• Sponsor’s name (Stern faculty or employee)
• Sponsoring department
• Expiration date (max 1 year; extensions may be requested yearly)

Activate your Stern account to use SCRC services.

Non-NYU users

Non-NYU affiliated users must first obtain an NYU account by having an authorized sponsor request an NYU NetID on their behalf by filling out the IT Onboarding Form. It is accessed by logging into the NYU VPN, then logging into identity.it.nyu.edu, clicking the hamburger menu, selecting “Request Affiliate”, and then “Request Affiliate User”. For additional help contact NYU IT support at AskIT@nyu.edu or 212-998-3333.

Log in to a Terminal Command Line Interface

The most common way to access the SCRC servers is to use a Command Line Interface (CLI) from a terminal application. A CLI is a text-based interface that allows users to interact with a host by entering text commands at the prompt of a terminal application.

The SCRC hosts that you connect to are running Linux. You will use your Stern NetID and password to authenticate to either one of SCRC’s remote login hosts: rnd.scrc.nyu.edu or vleda.scrc.nyu.edu.

If Off Campus, First Connect to the NYU VPN

If you are connecting from a remote location that is not on the NYU network (your home for example), you need to set up your computer to use the NYU VPN. Once you’ve created a VPN connection, you can proceed as if you were connected to the NYU network. 

If you’re on the NYU network already, you can proceed directly with your connection.

Log in From Mac/Linux

Open the terminal application and use the ssh command to make a secure connection to one of SCRC’s remote login hosts. E.g., if your Stern NetID is xy12,

ssh xy12@rnd.scrc.nyu.edu
OR
ssh xy12@vleda.scrc.nyu.edu

Log in From Windows

Windows users have several options:

• Command Line: Open the cmd program on an updated Windows 10 or later machine. Type the ssh command the same as shown above for Mac/Linux.
• WSL2: If you run Windows 10 or later, you can install Windows Subsystem for Linux (WSL), and then install Ubuntu or other Linux distributions on your machine. You will get a fully functional Ubuntu distribution with a terminal application. Type the ssh command the same as shown above for Mac/Linux.
• PuTTY: PuTTY is a free and open-source terminal emulator and network file transfer application that supports logging in with the ssh command.

Note: If you are using WSL2, you may not be able to access the internet when Cisco AnyConnect VPN, installed from the .exe file, is activated. A potential solution is to uninstall Cisco AnyConnect and install AnyConnect using Microsoft Store and then set up a new VPN connection using the settings described on the IT webpage.

Getting and using PuTTy

PuTTY is free and can be downloaded from PuTTY Download.  Download and run the latest installer file from the website. Follow the on-screen install instructions. Once installed, start PuTTY, then type rnd.stern.nyu.edu or vleda.stern.nyu.edu in the “Host Name” field and click “Open”. 

Note: The first time you log in a dialog may appear which starts with “The server’s host key is not cached…”.  Click Yes in this dialog.

Type your Stern NetID and password at the “login as” and “password” prompts, respectively. You should now be logged in and see a command line prompt from which you can enter Linux commands or run programs.

On Windows, Linux and Mac (using FileZilla)

Download, install, and start FileZilla. If off-campus, first log in to the NYU VPN.

Enter the following for a new connection:

• Host: rnd.scrc.nyu.edu
• Username: <your Stern ID>
• Password: <your password>
• Port: 22

Click Quickconnect.

On the left, you should see your local computer, while on the right, your Linux homedir.

Drag-and-drop files to transfer between the local and the Linux server.

For an in-depth tutorial, you can navigate to this Linux guide: Learning the Shell.

This tutorial demonstrates the basic Linux commands a user would need to use and run programs on the SCRC Linux computers. The Linux machines use a command line interface called a Shell. The Shell is the interface where you enter Linux commands. There are many types of shells. By default, SCRC Linux machines use the BASH shell. The first thing you see after logging in is the shell’s command prompt. For example, Christine Thomas’s prompt might look something like this:

[ct27@rnd ~]$

Your Home Directory

Once logged in, you are deposited into your home directory. Every user has a home directory. Your home directory is where you will store most of your files.

Show the present working directory

To see the complete name and path of the current working directory, use the command

$ pwd
/homedir/employees/c/ct27

This prints the full directory path.  Here, Christine’s home directory name is ct27 which is contained within a directory called c which is inside the directory employees inside the directory homedir. The Linux directory structure is a hierarchical tree structure.

Show contents of a directory

To see a list of the files and directories in the current directory type ls -l (list directory contents using long format), e.g.,

$ ls -l
total 180
drwxr-xr-x 3 ct27 nobody 4096 Apr 5  2023 mywork
drwxr-xr-x 3 ct27 nobody 4096 Apr 3  2023 archived-work
-rw-r--r-- 1 ct27 nobody 0    Mar 18 2023 prog.sas7bdat
-rw-r--r-- 1 ct27 nobody 1421 Mar 18 2023 prog.sas7bdat.log

ls is the command and -l is, in this example, the option for the command. The -l option formats the results into long format. Long format lists more than just the names of the files which is what ls alone would show. In this example, the output shows two directories – mywork and archived-work (notice the d at the beginning of the line), and two files – prog.sas7bdat and prog.sas7bdat.log. Some other information we see, e.g., the size of the log file is 1421 bytes, its owner is ct27, it’s last modification date is Apr 5 2023.

Auto Completion in the Shell

Press the `Tab` key after enough of the word you are trying to complete has been typed in. If when hitting tab the word is not completed there are probably multiple possibilities for the completion. Press tab again and it will list the possibilities.

Create a sub-directory

Say you would like to work on a project. It is probably convenient to create a sub-directory to hold the project’s files, i.e., program, data and results. To create a directory use the mkdir (make directory) command, e.g.,

$ mkdir costProject

This command creates a new directory called costProject inside the current directory.
Verify this by typing ls -l.
Note: Linux is case sensitive, so for example costproject is not the same as costProject.

Change to a subdirectory

To change into the costProject directory, i.e., make it the current directory, use the cd (change directory) command, e.g., 

$ cd costProject

Verify that we are in the costProject directory using pwd. 

$ pwd
/homedir/employees/c/ct27/costProject

Create or edit a file

nano is a simple text editor used to create or edit a data or program file, etc. For example, to edit a file called costdata.dat type, 

$ nano costdata.dat

This picture shows the nano editor editing the file costdata.dat. Along the bottom of the nano editor are the editor’s commands. The character ^ is the ctrl key. To save the file you type ctrl-o and then to exit nano type ctrl-x.

List the contents of a file

The more command displays the contents of a file, one screen-full at a time. Press the space bar to move forward screen by screen until the end of the file is reached or press the letter q, for quit., E.g., 

$ more costdata.dat

If the file contents are larger than can be shown in one page, the more command shows you the first page and halts. To see the next page press the space bar and to quit type q.

Breaking out

If you get stuck or “hung” after typing a command, etc., then use ctrl-c to break out or cancel your current command. ctrl-creturns you to the command prompt.

Linux Tutorial Continued

Commands

Most Linux commands are short, typically an abbreviation or mnemonic for what the command does. For example, the command to delete a file is rm which is short for remove. Recall: Linux is case-sensitive, i.e., it distinguishes between lowercase and uppercase (recognizing commands in lowercase only).

Most commands follow a common syntax:

$ command -options arguments

Here are some examples of basic Linux commands and command syntax.
To list all of the files in your current directory, type 

$ ls

To get a long listing of the files in your current working directory including information on size, date of modification and permissions, type 

$ ls -l

To make a copy of a file and give it a new name, type 

$ cp costdata.dat costdata.dat.bak

To remove (delete) a file, type 

$ rm costdata.dat

Note: There is no undo in Linux therefore once a file has been deleted there is no easy way to recover it.

To rename (move) a file, type 

$ mv costdata.dat.bak newcostdata.dat

To create a sub-directory, type 

$ mkdir study1

To change (move down) into the new directory, type 

$ cd study1

To change (move up) to the previous directory, type 

$ cd ..

To remove a directory, type 

$ rmdir study1

To remove ALL files in the current directory whose name begins with costdata, type

$ rm costdata*

The asterisk (*) is a wildcard character that matches one or more characters. Be VERY careful with removing files and using the * wildcard character. You might end up permanently deleting more than you intended. Always do a list first, e.g., ls costdata* to show the files that would be deleted with the rm costdata* command.

Redirecting Input and Output

Commands usually display their results to the screen. Also, commands normally operate on data as you type it in from the keyboard. A right angle-bracket > (called an “into”) on the command line indicates that the next word is the name of a file or device in which to place, or redirect the output of a command, e.g., 

$ ls > list

will place the output of the ls command in a file named list. If a file named list existed before you entered this command, any previous contents will be over written.

You can append to the end of a file using a double right angle-bracket >> (called an “onto”). For example, if the next command entered was: 

$ date >> list

The output of the date command would be added to the bottom of the file called list.

Pipes

The output of one command can be fed as input into to another command. The symbol for the input/output (I/O) connection is a vertical bar | called a pipe.

For example, a directory containing many files will scroll off the screen if just ls is used. If a pipe is used to direct the output of ls to be the input of more, then the directory listing will be shown one screen-full at a time and will not scroll off the screen. 

$ ls | more

Online Help

To get on-line help in Linux, use the man command. It provides access to a comprehensive online Linux manual. Type man followed by a command or topic on which you want information: 

$ man <command>

To make the online manual more helpful, an index is provided. The index is accessed with the -k option of the man command. For example: 

$ man -k directory

will display a one-line synopsis of all manual pages having to do with directories.The Linux File System

Linux uses a hierarchical file structure, which is made up of files, directories and sub-directories. A file can hold text, data, or a program. Directories contain files and sub-directories. A sub-directory is a directory that has been created within another directory.

You can also use the ‘–help’ to get a description of the command’s usage. Simply type your command whose usage you to know in the terminal with –h or –help after a space and press enter. And you’ll get the complete usage of that command as shown below. An example below:

$ cat --help

Permissions

Since Linux is set up to let users share files, you have the option of allowing or denying access to others on the system. Permissions determine who may access your files and directories or what may be done with a file or a directory. Use ls -l to see what permissions your files and directories have.

-rw------- 1 ct27 devel Aug 20 10:15 logon
-rwx------ 1 ct27 devel Aug 19 15:23 a.out
-r-xr-xr-x 1 ct27 devel Aug 28 09:48 ls-list
drwx------ 1 ct27 resch Aug 27 15:45 sas-one/
drw------- 1 ct27 resch Aug 27 15:45 tex/

The characters d, r, w, x, and - at the far left indicate the permission of each file and directory. There are 10 positions. Position 1 is the directory indicator. Positions 2,3,4 apply to the owner (creator) in this case ct27. Positions 5,6,7 apply to the group. Here, there are two different groups shown, devel and resch. Positions 8, 9,10 apply to all users.

The meaning of the letters are:

d (directory)

If the first letter is a “d”, the file is a directory. If the first character is a hyphen (“-“), then it is a regular file.

r (readable)

A file must be readable to be looked at or copied. A directory must be readable for you to list its contents.

w (writable)

A file must be writable in order for you to modify it, remove it, or rename it. A directory must be writable in order for you to add or delete files in it.

x (executable)

A file with executable permissions is one you can run, such as a program or a shell script. A directory must be executable in order for you to move into it (using the cd command), list its contents, or create or delete files there.

The hyphen (-) appears when the permission is switched off. For instance, if a hyphen (-) appears in place of an r, then the file or directory is not readable.

Some Examples

The file is read/write for owner, read only for others. 

-rw-r--r--

The directory is read/write/search for owner, read/search for group, searchable only for others. 

drwxr-x--x

Changing File and Directory Permissions

The file and directory permission levels in Linux can be changed to allow or deny access to other users. The chmod command is used to set the protection level. You can add or remove protection levels by using either + (add) or - (remove) with chmod and a letter signifying a class of users:

• u for the file owner
• g for a system defined group
• o for users in neither u or g
• a for all users
• and the desired access settings:
• r for read access
• w for write access
• x for execute access

For example to change the file called my.dat with the permissions:

 -rw-r--r--

to read/write/execute for all users (owner, group, and other) type the command:

 $ chmod a+rwx my.dat

The resultant permissions would be: 

-rwxrwxrwx

Now, change the permissions of “my.dat” so that only the owner can read, modify or delete it, i.e.:

$ chmod u-x my.dat
$ chmod g-rwx my.dat
$ chmod o-rwx my.dat

The resulting permissions are: 

-rw-------

History

The most recent commands typed at the command line interface are saved and can be viewed or retrieved for re-use. Access the history by pressing the up arrow. To search for a command in the history type ctrl-r, type the search text, then press enter to re-execute or ctrl-c to abort

Basic Cursor Movement, Cut, Paste and Undo

Command Summary

Introduction

This tutorial is intended to familiarize you with the process of running and scheduling jobs on the SCRC Slurm cluster. You will use the Simple Python Job to run a basic Python script that outputs the text “Hello World!”.

Make sure you’re logged in to a Linux terminal with SSH. Instructions can be found at Accessing the SCRC Systems.

Writing Python code

Using a text editing program such as nano, vim, code, or emacs, create a python script ‘hello-world.py’.

'''
hello-world.py

Print string "Hello World!" to output

'''

# --------------------------------
# main
# --------------------------------  
def main():
  
  print("Hello World!")

if __name__ == '__main__':
  main()

Creating the Slurm Script

To run a Python script use a Slurm sbatch file to allocate the necessary compute resources. It contains directives for Slurm to interpret as well as programs for it to execute. Here is an example sbatch file, ‘hello-world.sbatch’, that submits the python job to the Slurm cluster.

#!/bin/bash
#
# [hello-world.sbatch]
#
#SBATCH --job-name=hello              # Job name
#SBATCH --output=hello.out            # Output file name 
#SBATCH --export=ALL                  # Export all environment variables
#SBATCH --time=00:01:00               # Set max runtime of job = 1 minute
#SBATCH --mem=4G                      # Request 4 gigabytes of memory
#SBATCH --mail-type=BEGIN,END,FAIL    # Send email notifications
#SBATCH --mail-user=you@stern.nyu.edu # email TO
#SBATCH --partition=test              # Specify the partition to submit the job to


module purge                          # Start with a clean environment
module load python/3.9.7              # Load python module
python hello-world.py                 # Run the script using the loaded Python module

Lines beginning with #SBATCH are Slurm directives that specify job options, resource requests, and scheduling parameters for a job submitted to a Slurm cluster.

The module command is used to manage environment modules, which allow users to easily load and unload software and environment settings on systems such as HPC clusters. The purge subcommand clears all loaded modules, while the load subcommand loads a specific module.

The python hello-world.py command executes the Python script named “hello-world.py” using the Python interpreter. The script is run with the version of Python that was loaded via the environment module.

Executing the Slurm Script

Submit the job with the command:

sbatch hello-world.sbatch

The job is now queued and as soon as the requested resources are available, Slurm schedules the job onto one of the Slurm compute nodes. If the requested resources cannot be met, the job waits in the queue. To check the status of your submitted jobs:

squeue -u $USER

To check all jobs in the queue and running on the cluster:

squeue