Parallel Processing with Sun HPC Developer

The other day I thought I’d try out the parallel programming environment in Sun’s HPC Developer. I selected a very popular example from the LANL Website. The example by Blaise Barney  “Simple Heat Equation Parallel Solution 1” posted here was an ideal example. It requires processes in multiples of 4. The web site goes into more detail about the science of code which is a very interesting read.

Retrieve Sample Code 

Building Parallel Program

Start SunStudio in Developer VM and create a new project. File –> New Project. Choose Categories: GridEngine, Projects: Grid Engine C Project. Click Next. Project Name: GridEngineCProject_HeatExample and set the Parallel Environment to orte. Project is created. Now open up the project tree so you can see the Source Files. A new C source file is created named gridEngineMain.c. Double click on this file and it will open. Copy the source Sample Code from Blaise Barney and paste it over all the code in the gridEngineMain.c file.

Setup Header and Library file locations by right mouse clicking on the project name in Projects Tree. Select Properties from the menu list. This will bring up a configuration panel. Under Categories select C Compiler. Under General Set the Include Directories with the browse button and go to /opt/SUNWhpc/include. In Categories now select Linker in Libraries browse to add library file dynamic /opt/SUNWhpc/lib/libmpi.so. Click OK

Now in main SunStudio Menu select Run –> Clean and Build main project. This should build successfully with Exit value 0.

Running the Parallel Program

To make the program run in a parallel environment for the first time we will use mpirun. Open a console and change directory to the executable.

$cd /export/home/hpcuser/SunStudioProjects/GridEngineCProject_HeatExample/dist/Debug/SunStudio-Solaris-Sparc

Do a list

$ ls

You should see an executable program we just built gridenginecproject_heatexample

Set the Environment variable LD_LIBRARY_PATH so the environment will know where to find the mpi library

$ LD_LIBRARY_PATH=/opt/SUNWhpc/lib

$ export LD_LIBRARY_PATH

Run the mpi example program locally with 4 processes.

$ mpirun –n 4 ./gridenginecproject_heatexample

You should see something similar to this output:

MPI task 3 has started...
MPI task 2 has started...
MPI task 1 has started...
MPI task 0 has started...
Initialized array sum = 1.335708e+14
Sent 4000000 elements to task 1 offset= 4000000
Task 1 mysum = 4.884048e+13
Sent 4000000 elements to task 2 offset= 8000000
Task 2 mysum = 7.983003e+13
Sent 4000000 elements to task 3 offset= 12000000
Task 0 mysum = 1.598859e+13
Task 3 mysum = 1.161867e+14
Sample results:
  0.000000e+00  2.000000e+00  4.000000e+00  6.000000e+00  8.000000e+00
  8.000000e+06  8.000002e+06  8.000004e+06  8.000006e+06  8.000008e+06
  1.600000e+07  1.600000e+07  1.600000e+07  1.600001e+07  1.600001e+07
  2.400000e+07  2.400000e+07  2.400000e+07  2.400001e+07  2.400001e+07
*** Final sum= 2.608458e+14 ***

 

Now we want to run the program on the integrated grid compute nodes. Setup parallel environment orte 2 slots for length of 10 mins
$ qrsh -pe orte 2 -l h_rt=00:10:00

This sets up a console with parallel environment onto the compute grid. There are 2 compute machines.

Now run mpi example program starting 4 processes
$ /opt/SUNWhpc/bin/mpirun -n 4 ./gridenginecproject_heatexample

 

Parallel environments Grid Engine uses the concept of a parallel environment which defines how a parallel job should be ran. A parallel environment (PE) is used to initialize the cluster for parallel execution of your code. To specify a parallel environment use -pe <pe_name> <num>, where pe_name is one of the defined PEs and num is the requested number of CPUs. The <num> parameter can be an integer or an interval. If the parameter is an interval, -pe dmp4 4,8,12, the scheduler tries to allocate at least 4 CPUs and at most 12.

 

 

 

Sources:

“Simple Heat Equation Parallel Solution 1” from LANL site written by Blais Barney
https://computing.llnl.gov/tutorials/parallel_comp/#ExamplesHeat
C Code Example
https://computing.llnl.gov/tutorials/mpi/samples/C/mpi_heat2D.c

Same Code written in Fortran should you like to try it.

https://computing.llnl.gov/tutorials/mpi/samples/Fortran/mpi_array.f