Home > Online Help > Unix > Computing and Processing Practices

Computing and Processing Practices

(Revised 5/10/2010)


Contents
  • Introduction
  • Interpreting Loads
  • Running Jobs Remotely
  • Putting Jobs in the Background
  • Computing Etiquette

  • Introduction

    The DECF computing labs and clusters are available for use by anyone with a DECF account for running jobs and processes. There are 2 computing labs, and 3 computing clusters: Both the 1111 and 1171 Etchverry labs are reserved for class and in-lab uses. DECF users should always use the Archipelagos, Reindeer, and Ice clusters for running jobs remotely.

    However, during school holidays and off-hours, users may use the 1111/1171 machines for short and light computing jobs, as long as the jobs do not interfere with the users in the lab.

    Be adviced that since the 1111/1171 machines are accessible by lab users, it is beyond our control when machines are somehow shut-down, which will kill any jobs that are running.

    Always think twice and check the 1111/1171 class schedules before sending any jobs to the 1111/1171 machines.

    If you are uncertain if you should or when you can run jobs on 1111/1171 machines, email consult@newton.berkeley.edu


    Interpreting Loads

    Check Ganglia for computers with lower loads to run your jobs on. The red computers have HIGH loads and the blue computers have LOW or NO load. You can check the load on a computer by typing "top" at the command prompt. "top" displays the active processes, how much CPU and memory they are using, and the load.


    Running Jobs Remotely

    If you are new to SSH and remote connection, please see the How to SSH for software installation and general instructions.

    To run jobs remotely, simply SSH to kepler.berkeley.edu and authenticate with your DECF account. Once logged into kepler, ssh to one of the client machines, then start the program you want to run.


    Putting Jobs in the Background

    If you are running jobs that will take more than 10 minutes to complete, you should always run it in the background. Connections that are idle (with no active user interactions) for longer that 10 minutes are automatically disconnected by the server for secruity reasons.

    You can only put command-line jobs in the background. If you are running a program with a graphical interface, you will not be able to put the job in the background. We will focus on 2 ways that will allow users to put command-line jobs in the background: the "&" and the screen utility.

    Using the "&" (ampersand) is the easiest way to put a job in the background. Simply putting the ampersand at the end of your command will put that command in the background.

    For example, we want to run Matlab with inputfile and want the output to be written to outputfile. Since we know our job is going to run for a while, we want to put it in the background so that we can leave it running on the remote computer and simply come back for the output file once it's done. To accomplish all that, we can simply issue a one-line command:

    /usr/bin/nohup matlab-nogui < inputfile > outputfile &
    This will run Matlab in the background with the specified input and output files.

    Each program has its own syntax on input and output files. You will need to research on your particular program to find out how to write one-line commands like the one above.

    The second way to put jobs in the background is to utilize the screen manager. Simply put, you will start the screen session and run your job within the screen session. You can then detach the screen session while the job continues to run. You can re-attach the screen session in a later time to see your job that is running.

    See the "Unix Command: Screen" for details on how to use screen to run jobs in the background.


    Computing Etiquette

    DO NOT RUN JOBS ON KEPLER

    RUN ONE JOB AT A TIME

    Users found running multiple jobs at the same time will have their jobs killed and their accounts locked.

    Users running resource-intensive jobs on 1111/1171 machines during class time will also have their jobs killed.

    This is a necessary step to keep from overloading our computers and servers, and to make sure that users in our labs can work during class.