FAQs

ASPIRE 1 is a supercomputer operated by the National Supercomputing Centre (NSCC) Singapore.

It is built on x86 based architecture with initial benchmarks of 1.01 PFLOPS of compute throughput, 13 PetaBytes storage, and I/O performance of up to 500GBytes/sec. This supercomputer has 1,288 compute nodes with more than 30,000 cores, including 128 GPU nodes, and 9 large memory nodes ranging from 1TB to 6TB RAM.

ASPIRE 1 has login nodes in NUS, NTU and A*STAR extended with DTN-InfiniBand-longrange link.

ASPIRE1 is built on a CentOS Linux platform and currently does not have any Microsoft Windows OS.

PBS Compute Manager is a graphical user interface which facilitates users to run, monitor, and manage the jobs with ASPIRE1. The PBS Compute Manager can be accessed from here:
https://aspireweb.nscc.sg

PBS Display Manager is a graphical user interface which facilitates users to run graphics intensive applications in ASPIRE1. The PBS Display Manager can be accessed from here:

For A*STAR users – https://astar.nscc.sg

For NTU users – https://ntuweb.nscc.sg

For NUS users – https://nusweb.nscc.sg

For SUTD users – https://sutd.nscc.sg

For other users – https://aspireweb.nscc.sg

NSCC’s enrollment system is federated with SingAREN to facilitate users from NUS, NTU, SUTD, SMU and A*STAR to register seamlessly to ASPIRE1. Please follow the instructions presented in the User Enrollment Guide for the users from above mentioned organizations.

If you don’t belong to one of the organizations above, please contact [email protected] with the following details:

• User ID (preferably the same login ID used in your organization, limited to 8 characters):
• First name:
• Last name:
• E-mail address:
• Organization name:

*Note: Login ID creation is subjected to availability and approvals.

NUSEXT domain users are considered external users for NUS. Please contact [email protected] for registration to ASPIRE1.  

To utilize ASPIRE1 resources, you need to login to the cluster. Please follow the below table to determine how to access ASPIRE1.

Follow the below instructions to connect to ASPIRE1 through portal.

• Open the web browser and browse for the URL provided above.
• Use your ASPIRE1 credentials to access the web portal.

Below are instructions to connect using SSH/SCP on UNIX/Linux or Mac:

• Open any SSH terminal
• Type “ssh <userid>@<Host/FQDN>”
• Enter the password (characters are invisible while typing the password)
• Once logged in successfully you should be able to see a “$” prompt that allows you to execute Linux commands

If you want to use the X11 interface, replace the above ssh command with ssh -X.

For X11 – ssh -Y, make sure you have XQuartz installed for OS X 10.8 or higher.

Below are instructions to connect using SSH/SCP on Windows:

• Open any SSH terminal software (e.g. putty, MobaXterm)
• In case of putty, type the login host name and click on Open button (Putty may allow you to login automatically without asking for password. In case if putty asks for username/password, you need to type the login ID and password from your university.)
• In case of MobaXterm, type to command at the prompt “ssh [email protected]”
• Upon successful login, you will be prompted with “$” to use NSCC Supercomputer

Internet access from remote login nodes are disabled due to security policies at each entity. However, users will still be able to access files on the Internet using the following methods.

A*Star Users - login to astar.nscc.sg and download files from Internet

NUS users - ssh to nscc04-ib0 after logging in to nus.nscc.sg, and download files from Internet

NTU users - ssh to nscc04-ib0 after logging in to ntu.nscc.sg, and download files from Internet

Users accessing through aspire.nscc.sg will be able to access the Internet, once logged in.

Please login to https://user.nscc.sg/ with your organization credentials and click on set/reset your password. Once a new password is set you can login to NSCC with the new password. 

Please follow the Password Reset guide and follow the step-by-step instructions.

Note that the password must contain:

• a minimum of 8 characters;
• a mixture of upper and lower cases;
• numbers (0-9); and
• at least one special character: [email protected]#$%^&*+=?><

If you still cannot access your account after resetting your password, please contact our help desk at [email protected]

The NSCC account password expires every 90 days. Please be reminded to reset your password before the expiry of your password. If you did not reset your password after 90 days, your account will expire. Upon expiry of the account you will not be able to login to NSCC system and the messages as shown below will be displayed when using the ssh connection. 

• Your account has expired; please contact your system administrator
• You are required to change your LDAP password immediately.
• Connection closed by xxx.xxx.xxx.xxx

In order to resolve the issue you need to login to https://user.nscc.sg/ with your organization credentials and click on set/reset your password. Once a new password is set you can login to NSCC with the new password. 

If your account is still not re-activated after resetting your password, please contact our help desk at [email protected]

The personal quota of 100,000 core hours and 50 GB of storage is granted to all NSCC accounts upon creation. The quota is fixed, non-transferable and can not be extended. Once the personal quota is exceeded, you can only submit job through an approved project.

Users are encouraged to apply for projects only if your resources request exceeds the personal quota, or if you have depleted your personal quota. 

You may apply for a project during a project call window. A call for project allocations on NSCC ASPIRE 1 will be published twice a year. Please lookout for the announcement through the email blasts nearer to the call period, or check our website for the latest information on project application calls. 

1. National Initiative Projects
2. Stakeholder's Industrial Partnership Projects
3. Grant Projects
4. Individual Projects

Yes, but please note that only the PI and/or the project applicant can transfer the ownership of a project.

Please email [email protected], copying the PI, previous owner and new owner of the project to request for a transfer.

• NSCC project ID
• NSCC user ID(s) of the user(s) to be added

The PI/applicant of the project should send an email to [email protected] with the following details:

1. Project ID 
2. New resource request
3. Reasons for the new request (e.g. how was the resources used, applications used etc.)
4. Utilisation plan over the remaining months till the end of the project cycle

Please note that requests made outside the project allocation calls will be reviewed by the Resource Allocation Committee within 2 weeks to 3 months depending on the scale of the request. 

• Project quota will be set to zero (0) hours and you will not be able to dispatch any jobs. 
• All project data will remain in the system for 31 days.
• After 31 days, all project data will be archived.
• Charges will be incurred for retrieval of archived data at $0.022/GB.
• All archived data will be retained for 5 months from the date of archival.

1. All resources granted in the previous project cycle will expire by 30 June 2018.
2. On 1^st July 2018, all project quota will be set to zero (0) hours and you will not be able to dispatch any jobs. All project data will remain in the system for 31 days.
3. On 1^st August 2018, all project data will be archived.
4. Charges will be incurred for retrieval of archived data from 1^st August 2018.
5. All archived data will be retained until 31 December 2018.

1. All resources granted in the previous project cycle will expire by 31 December 2018.
2. On 1^st January 2019, all project quota will be set to zero (0) hours and you will not be able to dispatch any jobs. All project data will remain in the system for 31 days.
3. On 1^st February 2019, all project data will be archived.
4. Charges will be incurred for retrieval of archived data from 1^st February 2019.
5. All archived data will be retained until 30 June 2019.

Please submit a project renewal application during the project call window.

In order to request for a project renewal:

1. Login to the Project Portal
2. Update the project deliverables under the "Deliverables" tab
3. Click on the "Renew" button, complete and submit the application form.
4. All applications are subjected to the approval of the Policy and Resource Allocation Committee and the Technical Resource Allocation Committee. 

Please note that all unutilised resources will not be brought forward. 

At the end of the project cycle, you will be requested to update your project deliverables. 

1. Please login to the Project Portal and update the deliverables achieved for all projects under the “Deliverables” tab for each project.
2. Please note that all project allocation grantees must acknowledge and cite NSCC for any published manuscripts and presentations and submit the proof of deliverables achieved. All academic journals and conference papers shall cite NSCC with the following: “The computational work for this article was (fully/partially) performed on resources of the National Supercomputing Centre, Singapore (https://www.nscc.sg).”

The disk quota allocated is limited as stated below respectively :

/home    - Home directory    - 50GB.
/scratch - Scratch directory - 100 TB

This will not be extended on any terms. If more disk space is needed, you should apply for a project through the Project Portal.

You can check your current disk usage on /home by executing the “myquota” command.

$ myquota

===============================================================
|File System               Usage (GB)               Limit (GB) |
|nscc_home                      30.10                    50.00 |
|/scratch                    43204.67                102400.00 |
===============================================================

Purging policy is implemented on scratch directory and files unused for more than 30 days will be purged automatically.  

File under home or project directories are untouched.

You can check your current CPU usage by executing the “myusage” command.

$ myusage

Usage in 7 days to 2017/09/19:
Number of jobs on CPU nodes: 300
Core hours on CPU nodes: 81,592.3
Number of jobs on GPU nodes: 0
Core hours on GPU nodes: 0.0

Usage in month to 2017/08/31:
Number of jobs on CPU nodes: 131
Core hours on CPU nodes: 778,539.6
Number of jobs on GPU nodes: 0
Core hours on GPU nodes: 0.0

Usage since system start to 2017/08/31:
Number of jobs on CPU nodes: 6,368
Core hours on CPU nodes: 15,577,192.9
Number of jobs on GPU nodes: 0
Core hours on GPU nodes: 0.0

Upon logging in to ASPIRE 1, you will be able to see the amount of core hours remaining in all the projects in which you are a member of, as seen below:

Core hours remaining for project: Total Grant: 1000000.00
Total Used: 200000.00
Total Pending: 50000.00
Total Avail: 7500000.00

“Total Grant” corresponds to the amount of resource that was granted.
“Total Used” corresponds to the amount consumed so far.
“Total Pending” corresponds to the current amount requested (i.e. #running_jobs * per_job_core_requested * per_job_walltime_requested)
“Total Avail” corresponds to “Total Grant” - “Total Used” - “Total Pending”

Be aware that once the (Used+Pending >= Granted), the subsequent jobs will be ON-HOLD, and no new jobs will be allowed.
The new jobs will be allowed when it fit into the Avail, which is (Avail >= Granted - Used - Pending).

For example, if you request 50 hours for your job but your job actually only needs 10 hours to complete, the AMS will hold back the requested 50 hours to itself (and mark them as “pending core hours”) until the 10 hour-job is completed. The remaining 40 hours will only be released and brought back to the user at that point of time. What is categorized as “pending core hours” will be deducted from your “available core hours”, so if you’re not requesting for the right amount of walltime in your job, you might be temporarily seeing less available core hours than you actually have.

When logged into the NSCC login nodes, you will be able to see the utilization of projects assigned to the user in the MOTD page.

In order to check the usage at any point of time, please execute the command:

project_summary <project ID> 

For example, to view the project summary of the project 10000001, execute the command:

project_summary 10000001

A similar output as seen below will be displayed:

Resource: CPU-hrs, System: aspire1
Total Grant: 7000000.00
Total Used: 51392.84
Total Pending: 7488.00
Total Avail: 6941119.16
Resource: CPU-hrs-budget, System: aspire1
Total Grant: Infinite

In order to check the storage disk usage, execute the command:

myquota -p <project ID>

For example, for project 10000001, execute the command:

myquota -p 10000001

A similar output as seen below will be displayed:

Project Quota for 10000001

===============================================================
|File System Usage (GB) Limit (GB) |
|nscc_home 8572.90 30000.00 |
===============================================================

The word MPI stands for Message Passing Interface. One of the most used technologies to parallelize the applications. There are various implementation of MPI such as OpenMPI, MVAPICH, IntelMPI, etc.,
MPI Interface is a wrapper to C/Fortran compiler, which means that all the codes which are used in C/Fortran can be used with MPI in addition to penalization techniques.
ASPIRE1 is powered with IntelMPI for better performance of applications.

One of the techniques to run your code in parallel is OpenMP. Codes using OpenMP are normally restricted to run in one physical server, unless you are running a hybrid code made out of MPI and OpenMP. OpenMP codes themselves are not capable of communicating through a network infrastructure such as Infiniband. OpenMP code is simple to use and does not need any wrappers, standard compilers like GCC/Gfortran or Intel C/Fortran can use OpenMP. However, the program must have OpenMP directives.

A serial application is an application which can use only one process at a time to perform calculations, while a parallel application can scale to multiple processes and make use of cores/CPUs in one or multiple servers using high speed Interconnects. For this reason, parallel applications produce faster results in general.
To promote the HPC experience, it is strongly recommended to use parallel codes on ASPIRE1 if applicable.

Checkpoint is a technique used to save the output periodically during a runtime of an application. The advantage using checkpoint is that it is possible to restart the program from the last checkpoint, in case if the execution of program terminates abruptly due to any reason.
You are advised to use the checkpoint and restart technique while writing your application so that in case of eventualities, you should be able to restart the job from the last checkpoint.

The available compilers are as follow:

• gcc (v4.9 & v4.4)
• Intel (v17.0 & v16.1)
• NAG 

The applications, libraries and compilers are very dynamic and are powered with environment modules in ASPIRE 1. If you wish to list the available modules, use the command "module avail" from the prompt.

NSCC does not support compilers other than the versions mentioned above at the moment. If you’re keen to use other compilers, you are more than welcome to install it into your own home directory. Please contact our service desk (access https://servicedesk.nscc.sg/ or contact [email protected]) if you face any difficulties during the installation.

Yes, you can request for additional libraries or applications through the Service Desk portal. However the installation of libraries is subjected to various conditions such as compatibility, the time required to make the library available, dependencies and our software policies. Please contact Service Desk for further clarification. Our Technical specialist will respond to make the most of your request.

Commercial applications can be installed in ASPIRE1, but they need to be installed and used following the licensing terms of each software application. First of all, the license and the software modules of the intended application need to be provided by you to NSCC for installation. Based on the license terms and conditions, we will determine whether and how the software should be installed in ASPIRE1. The same rule applies to academic licenses.

Please note that the accountability on the usage of the software should be held by the owner of the license. For more information, please contact our service desk.

Yes. Jupyter notebook is not in the default supported applications, but it is available with anaconda package. Follow the instructions below to use the Jupyter notebook.

1. Login to display manager (Please see the display manager instructions provided under user guides)
2. Double click on GlxSpheres app to launch the application 
3. Close the GlxSpheres by clicking on "x" on top right corner
4. Open terminal through Menu -> Tools -> Terminal
5. Execute the below commands at the terminal 
   module load anaconda/3
   export PYTHONHOME=/app/anaconda/3
   export PATH=/app/firefox:$PATH
6. Open jupyter notebook with the command "jupyter notebook"

There are many reasons jobs may be prevented from starting. The first thing to do is to run “qstat -s ”; this will print the comments from the job scheduler about your job.

If you see a “–”"Q" in the column "S", it means the scheduler has not yet executed your job. Be patient.

There could be several reasons for this, few common reasons listed below:

1. The operating system that you are using is different from that in ASPIRE1
   Solution:
   • if you are running in your local Linux machine, a simple recompilation on the ASPIRE1 login nodes will solve the issue.
   • if you are running in Windows PC and you want to run inASPIRE1, you need to either obtain a copy of the software for Linux or port the code from Windows to Linux.
2. Different compiler/library stack: if  your job is not running although your operating system in your own environment is the same as ASPIRE1, it means the compiler/libraries are not compatible. You need to recompile your codes on ASPIRE1 login nodes.

Please contact our service desk (access https://servicedesk.nscc.sg/ or contact [email protected]) if you still have any issues with running your job on ASPIRE1.

This error could be because you have forgotten to source the appropriate system .rc in your personal .rc file.

• If you are using an sh-derived shell for your jobs, edit the .bashrc file to ensure it contains the line “. /etc/bashrc”.
• If you are using a csh-derived shell for your jobs, edit the .cshrc file to ensure it contains the line “source /etc/csh.cshrc”.

If you have accidentally deleted the .bashrc file, you can copy back the original .bashrc file from /etc/skel/.bashrc .

Example:

$ cp /etc/skel/.bash* ~/

If your batch job is named, runjob.sh and your output is not redirected in the batch script, then your job output will appear in runjob.sh.o****, where the digits at the end of the file name represent your job ID. The final entries in the .o file give you the details on wall time and virtual memory used by the job.

Similarly, error messages of your jobs will be recorded in runjob.sh.e****, where the digits at the end of the file name represent your job ID.

In a PBS job script, the memory you specify using the -lmem= option is the total memory across all nodes. However, this value is internally converted into the per-node equivalent, and this is how it is monitored.
For example, since NSCC Supercomputer has 24 cores per node, if you request -l slect=2:ncpus=24, mem=10GB, the actual limit will be 10GB on each of the two nodes. If you exceed this on either of the nodes, your job will be killed.
Please note, if a job runs for less than a few minutes, the memory use reported in your .o file once the job completes may be inaccurate. We strongly discourage people running short jobs of, e.g. less than 1 hour. This is because there is significant overhead in setting up and tearing down a job and you may end up wasting large amounts of your grant. Instead, if you have many short jobs to run, considering merging them together into a few longer jobs

Please note that you are not supposed to run any job on the Login nodes. We will automatically kill if we see any user running a job on the login node. If you want to run an interactive session, please submit and interactive job into the compute nodes using the qsub command.

$qsub -I

The queues available for job submission are :

• normal
• gpu
• largemem

There are several queues in the system with different priorities which works for different workloads. Below are some of the queues in the system:

However these queues does not accept jobs directly rather jobs will be routed once submitted to normal/gpu queue. 

E : Job is exiting after having run
F : Job is finished. Job has completed execution, job failed during execution, or job was deleted.
H : Job is held. A job is put into a held state by the server or by a user or administrator. A job stays in a held state until it is released by a user or administrator.
M : Job was moved to another server
Q : Job is queued, eligible to run or be routed
R : Job is running
S : Job is suspended by server. A job is put into the suspended state when a higher priority job needs the resources.
T : Job is in transition (being moved to a new location)
U : Job is suspended due to workstation becoming busy
W : Job is waiting for its requested execution time to be reached or job specified a stagein request which failed for some reason.
X : Subjobs only; subjob is finished (expired.)

Please refer to the PBSPro manual for more details:
http://resources.altair.com/pbs/documentation/support/PBSProUserGuide12.1.pdf

E : Job is exiting after having run
F : Job is finished. Job has completed execution, job failed during execution, or job was deleted.
H : Job is held. A job is put into a held state by the server or by a user or administrator. A job stays in a held state until it is released by a user or administrator.
M : Job was moved to another server
Q : Job is queued, eligible to run or be routed
R : Job is running
S : Job is suspended by server. A job is put into the suspended state when a higher priority job needs the resources.
T : Job is in transition (being moved to a new location)
U : Job is suspended due to workstation becoming busy
W : Job is waiting for its requested execution time to be reached or job specified a stagein request which failed for some reason.
X : Subjobs only; subjob is finished (expired.)

Please refer to the PBSPro manual for more details:
http://resources.altair.com/pbs/documentation/support/PBSProUserGuide12.1.pdf

How do I submit array jobs on ASPIRE1?

We recommend using loops like this:

#!/bin/bash
#PBS ...
for i in {1..10}; do
qsub -v PBS_ARRAY_INDEX=$i <job-script>
done

You could also run 24 single core jobs in parallel if they all use similar resources and will finish around the same time by using the following in the jobscript:

#!/bin/bash
#PBS -l select=1:ncpus=1, walltime=24:00:00
#PBS ...
for i in {1..24}; do
./run_my_program args ... &
donewait

Please note that there is a ‘&’ at the end of the command line, and the ‘wait’ for all background tasks to finish.

By default, any file is restricted to individual and the project/group. In case you need to let other users access your files , please contact the service desk (access https://servicedesk.nscc.sg/ or contact [email protected]). 

The scratch disk is a temporary space for your working or transient data. Any files that have not been accessed for more than 30 days may be removed without notice. Therefore please do regular housekeeping and move your files to your home or project space if you still need them.

Please refer to our software list (https://help.nscc.sg/software-list/). Once you’re logged in to ASPIRE1, you can load the module of the application using the following commands:

$ module avail                                             # check on available modules

$ module load <application name>        # load application

Yes, you can install applications by yourself. Please feel free to install your intended application into your own home directory.

Yes, but in order to do so, you will need to have a project registered with NSCC. When requesting, please make sure to let us know your project ID. In general, we will ask you to install applications in your home directory by yourself, unless there are any dependencies that can only be solved by the system administrators.

You can use one of the file transfer protocols which supports copying over SSH (e.g. scp).Please note the speed of copy may depend on the network speed of your local environment.

The easiest way to transfer files from your organization HPC is to use rsync. For example, to transfer /scratch/johnsmit/project1 directory in your existing cluster to ASPIRE1’s /project/johnsmit/ directory, you can run the following command on your cluster:
$ rsync -arvz /path/to/the/files <login>.nscc.sg:/project/johnsmit/
*Please replace login with the respective login node mentioned in the table access methods

From PC: 

For Microsoft Windows/Mac the preferred/suggested method is to use FileZilla or any other file transfer utility to transfer your local files to desired location in NSCC. 

From any Linux machine/Mac terminal, you can either use scp or rsync command to copy files from your local pc to NSCC. The simple syntax for rsync should be:

<localpc>: rsync -arvz /path/to/files <login>.nscc.sg:/destination/path/in/nscc

From a cluster in your organization:

This is tricky, and unless your organization cluster is published in internet, there's no straight forward method. But if your cluster is in A*Star extranet, this can be possible. Please try the below syntax:

<your cluster>: rsync -arvz /path/to/files astar.nscc.sg:/destination/path/in/nscc

For all other users, apologies, you need to copy your files to your PC and transfer back to NSCC. 

Please use Filezilla software to use GUI for file transfer

Please refer to the Linux Tutorial.