6.7 Calling HPF Subprograms from non-Parallel Fortran or C Programs

Digital Fortran 90 makes it easy to incorporate HPF subprograms into existing programs, including programs that aren't written in Fortran. In most cases it is not necessary to make any changes to the existing code. Using this capability, you can parallelize your existing code one subroutine at a time. This is particularly helpful if you have only a few subroutines that are responsible for most of the execution time of your program. When the new HPF subprogram is compiled and linked in to the remainder of the existing program, the HPF subprogram runs in parallel on a PSE cluster. The remainder of the program executes as it did before. For example, if it were a serial program, it executes serially on one processor.

6.7.1 Building Programs that Have Both Parallel and non-Parallel Routines

Parallel HPF subprograms called from non-parallel routines must be compiled using the -nowsf_main compiler option. Non- parallel routines are routines that run on only one processor. A non-parallel routine can be one of the following:

  1. A Fortran routine that was compiled without the -wsf compiler option, whether or not it contains HPF language features
  2. A routine written in some other language
  3. An EXTRINSIC(HPF_SERIAL) routine, whether or not it was compiled using the -wsf option.
  4. An EXTRINSIC(HPF_LOCAL) routine called from an EXTRINSIC(HPF_SERIAL) routine, whether or not it was compiled using the -wsf option.

An EXTRINSIC(HPF_LOCAL) routine called from a global HPF [or EXTRINSIC(HPF)] routine is considered parallel code, whether or not it is compiled with the -wsf option.

Table 6-2 shows the compiler options to use when compiling the called routine based on the characteristics of the calling routine:

Table 6-2 Compiler Options for Mixing Parallel and non-Parallel Routines

    Called Routine 
    Non-Parallel  EXTRINSIC (HPF) [1]  EXTRINSIC (HPF_LOCAL) [2] 
  Non-Parallel  Serial, no -wsf option  Use -wsf and -nowsf_main   Serial, no -wsf option 
Calling         
  EXTRINSIC (HPF)  [3]  Use -wsf   Use -wsf  
Routine         
  EXTRINSIC (HPF_LOCAL)  Digital extension [4]  Illegal  Use -wsf  

Note 1: An EXTRINSIC(HPF) routine that has two or more callers, one of which is a non-parallel routine and another which is an EXTRINSIC(HPF) routine, should be compiled using both the -wsf and -nowsf_main options.

Note 2: An EXTRINSIC(HPF_LOCAL) routine that has two or more callers, one of which is a non-parallel routine and another which is either an EXTRINSIC(HPF) or an EXTRINSIC(HPF_LOCAL) should be compiled using the -wsf option.

Note 3: The only non-parallel code that can be called from an EXTRINSIC(HPF) routine is an EXTRINSIC(HPF_SERIAL) routine. The actual EXTRINSIC(HPF_SERIAL) routine need not be written in Fortran, but it must have an explicit interface declaring it to be EXTRINSIC(HPF_SERIAL). The called routine, in this case the EXTRINSIC(HPF_SERIAL), can be compiled with or without the -wsf option.

Note 4: As an extension, Digital allows (non-HPF) Fortran routines to be called from EXTRINSIC(HPF_LOCAL) routines. See Section 6.6.2.1.

Once the various pieces of the program are compiled, they must be linked to produce an executable program.

Digital recommends using the f90 command with the -wsf option to link such programs whenever possible. The examples directory (/usr/examples/hpf ) contains a sample program (twin*.f ) that uses the -nowsf_main option, and is linked using the Fortran 90 command.

6.7.2 Linking Programs Without Using the f90 Command

In certain cases (such as a main program written in C++) it may be advantageous to use a different compiler driver (such as cxx). If you choose not to use the f90 command to link your program, you will need to follow these steps:

  1. Compile the pieces of the program with the appropriate compilers as described above to generate .o files.
  2. Replace the asterisks ( * ) in the following program with the numerical argument that will be given to the -wsf option at compile time. If no argument will be given, replace the asterisks with zero (0). Save the program in a file called numnodes.f90 . 
      SUBROUTINE _hpf_compiled_for_*_nodes
  COMMON /_hpf_numnodes_compiled_for/ i
  DATA i /*/
  END
  3. Compile numnodes.f90 using the Digital Fortran 90 compiler without the -wsf option, as follows: 
    % f90 -c numnodes.f90

    This produces the file numnodes.o , which can be linked with the rest of the program, as described below.

  4. Link the program as follows:

    Follow the linking instructions appropriate for the compiler driver or linker you have chosen. Be sure to include the numnodes.o object generated in step 3. In addition, all libraries needed by Digital Fortran 90 must be included explicitly on the link line. This can be done by adding the following ld options to the link line:

    1. If you are linking -call_shared (the default):
      • If you are not currently profiling with the pprof profiler, use the following ld options: 
        -tr -h/usr/lib/cmplrs/hpfrtl/ -B -lphpf -nocount -lUfor -lfor \
 -lFutil -lm_4sqrt -lm -lots -lmach
      • If you are profiling this program using the pprof parallel profiler, use the following ld options: 
        -tr -h/usr/lib/cmplrs/hpfrtl/ -B -lphpfp -lmld -nocount -lUfor \
 -lfor -lFutil -lm_4sqrt -lm -lots -lmach
    2. If you are linking -non_shared :
      • If you are not currently profiling with the pprof profiler, use the following ld options: 
        -tr -h/usr/lib/cmplrs/hpfrtl/ -B -lphpf -qlimc -limc_stub \
 -nocount -lUfor -lfor -lFutil -lm_4sqrt -lm -lots -lmach -qlc_r
      • If you are profiling this program using the pprof parallel profiler, use the following ld options: 
        -tr -h/usr/lib/cmplrs/hpfrtl/ -B -lphpfp -lmld -qlimc -limc_stub \
 -nocount -lUfor -lfor -lFutil -lm_4sqrt -lm -lots -lmach -qlc_r

      Certain options to the f90 command may change the options passed to the linker. To see exactly which link options the f90 command generates, issue the f90 command specifying a dummy executable file name and all of the options you used to compile the Fortran portions (serial or parallel) of your program, and add the -v option.

      For example, the following command line, operating on a non-existent program "dummy", shows the link options that the Fortran 90 command generates with the -v, -pprof, -non_shared , and -wsf options: 

      % f90 -v -o dummy -pprof i -non_shared -wsf

      This command line produces the following output: 

      /usr/lib/cmplrs/fort90/decfort90 -o /tmp/forAAAaauira.o \
 /tmp/forAAAaauira.f90
/usr/lib/cmplrs/hpfrtl/libp_version_val -local -peers 1
/usr/bin/cc -v -o dummy -non_shared /usr/lib/cmplrs/fort90/for_main.o\
 /tmp/forAAAaauira.o -O4 -tr -h/usr/lib/cmplrs/hpfrtl/ -B -lphpfp \
 -lmld -qlimc -limc_stub -nocount -lUfor -lfor -lFutil -lm_4sqrt \
 -lm -lots -lmach -qlc_r
/usr/lib/cmplrs/cc/ld -o dummy -g0 -O4 -non_shared \
 /usr/lib/cmplrs/hpfrtl/crt0.o /usr/lib/cmplrs/fort90/for_main.o \
 /tmp/forAAAaauira.o -lphpfp -lmld -qlimc -limc_stub -nocount -lUfor \
 -lfor -lFutil -lm_4sqrt -lm -lots -lmach -qlc_r -lc
ld:
Error: Undefined:
MAIN__
0.4u 0.4s 0:02 36% 0+29k 155+28io 98pf+0w
f90: Severe: Failed while trying to link.

      Look at the line beginning with /usr/bin/cc . In your link, you should use all of the arguments given to cc except for the .o files, -o dummy , and -v . In other words, for this example, you should use these options: 

      -non_shared -O4 -tr -h/usr/lib/cmplrs/hpfrtl/ -B -lphpfp -lmld \
 -qlimc -limc_stub -nocount -lUfor -lfor -lFutil -lm_4sqrt -lm \
 -lots -lmach -qlc_r

For More Information: