6.1 HPF Basics

HPF is a set of extensions to Fortran 90 intended to facilitate writing parallel Fortran programs. Appropriately written HPF programs that are run in an HPF-capable environment, such as a cluster running Digital's PSE (Digital Parallel Software Environment), can execute at a dramatically faster speed than Fortran programs run in a single-processor environment.

HPF is especially useful for programs that can be expressed as large scale array operations, a major class of computationally intensive programs. HPF programming involves inserting directives that advise the compiler about potentially parallelizable array operations. Lower level details of parallelization, such as message passing and synchronization, are automated by the compiler, and invisible to the programmer.

Digital Fortran 90 parallelizes array operations when these array operations are accompanied by HPF directives. The compiler uses the information given in the directives to spread data storage and computation across a cluster. For a large class of computationally intensive problems, using HPF under PSE achieves a dramatic increase in performance over serial (nonparallel) execution.

6.1.1 When to Use HPF

For many programs, HPF can produce enormous performance gains, with speed up in near direct proportion to the number of processors. Some algorithms, however, are not suitable for HPF implementation.

There is no formula or completely general rule for determining when HPF is useful, because the achievable degree of parallel speed- up is highly algorithm-dependent. Nevertheless, the following considerations can serve as a rough guide.

6.1.1.1 Existing Code

Existing codes are good candidates for conversion to HPF under the following circumstances:

• The computationally intensive kernel of the program must be expressible as an operation (or operations) on a large array (or arrays).
• The existing code should spend a long time in large DO loops that perform the array operations:
  • "Large" generally means a few thousand iterations or more. Loops on the order of a thousand iterations or less are at best marginal candidates for HPF implementation, unless each iteration performs unusually intensive computation (cosines and so forth).
  • One standard for a "long time" might be 90% of the program's run time. Another standard might be that the time in the loop (or loops) divided by the number of processors that are available is more on the order of 0.1 seconds than on the order of 0.001 seconds.
• Codes already written in Fortran 90 syntax are easy to convert to HPF.
• Existing HPF codes written for other vendors' compilers or translators may need minor modifications, such as adding interface blocks (see Section 6.4.3).
• Codes written to run well on vector machines generally perform well when converted to HPF. In particular, this means codes with large DO loops that have no inter-iteration dependencies.
• Thread-based parallel programs, or programs that rely on a process spawning other processes are not suitable for coding in HPF. However, this type of code can be incorporated into an HPF program through use of EXTRINSIC subroutines.

6.1.1.2 New Code

For new code, HPF is generally useful in the following cases:

• Problems utilizing iterative solution methods
• Signal processing
• Image processing
• Modeling
• Grid-based problems in general, especially translationally invariant grid operations - solution methods where large parts of a grid are uniformly operated upon.
• In general, most problems expressible as operations on large arrays; for the meaning of "large", see Section 6.1.1.1.

6.1.2 Syntax and Function of HPF Directives

These are examples of HPF directives:

!HPF$ DISTRIBUTE A(BLOCK, BLOCK)
!HPF$ ALIGN B(I) WITH C(I)
!HPF$ PROCESSORS P(8)

HPF directives are preceded by the tag !HPF$ , to identify them to the compiler. Because this tag begins with an exclamation mark (!), all HPF directives are syntactically Fortran comments. Except for a syntax check at compile time, HPF directives are ignored (treated like comments) in source code not explicitly compiled for execution on a PSE cluster, and have no effect on the meaning of the program. When compiled under PSE with the -wsf switch (see Section 8.1.1.1), the compiler uses the HPF directives to create a parallelized version of the program. In a parallel environment, correctly used HPF directives affect only the performance of a program, not its meaning or results. Incorrect use of HPF directives can inadvertently change the meaning of the code, producing an illegal program. See Sections 6.4.2 and 6.4.5).

HPF directives must follow certain syntax rules in order to produce meaningful results. For example, the !HPF$ tag must begin in column 1 in fixed source form, but may be indented in free source form. A number of other syntax rules apply.

For More Information:

• On the syntax of HPF directives, see the Fortran Language Reference Manual.

6.1.3 List of HPF Directives

Table 6-1 lists the HPF directives, and the sections in this chapter which explain them.

Table 6-1 HPF Directives and HPF-Specific Attribute

6.1.4 Minimum Requirements for Parallel Execution

In order to achieve performance gains from using HPF, programs must be written so that they execute in parallel. In order to be compiled to execute in parallel, certain minimum requirements must be met. Code that does not meet these requirements is not parallelized, and is compiled to run serially, with no parallel speed-up.

• Array operations are parallelized only on arrays that either:
  • Have been explicitly distributed using the DISTRIBUTE directive
  • Are aligned with an array or template that has been explicitly distributed using the DISTRIBUTE directive.
• Only data parallel array assignment statements are parallelized. The phrase "data parallel array assignment statements" refers to:
  • Fortran 90 whole array or array subsection assignment statements
  • FORALL structures
  • DO loops with the INDEPENDENT attribute
  • Certain Digital Fortran 90 array intrinsic functions and library routines