16.3 PSE cluster database Overview

A PSE cluster database contains ASCII text records that are interpreted by PSE as:

<section-type> <PSE token> <value>  . . .

Values preceded by the exclamation mark (!) character are treated as negations, such as, !user1 which is interpreted as not user1 . Each PSE cluster database contains two major sections at a minimum:

A configuration_data section
One or more partition sections

Certain PSE tokens can appear either in the configuration_ data section, in a partition section, or in both. For each case in which a token appears in the configuration_data and does not exist in a partition definition, that partition inherits the token value from configuration_data . When a token exists in both the partition definition and in configuration_data , the partition value takes precedence. The following sections cover the definition language and each database type in more detail.

16.3.1 configuration_data Information

The configuration_data definition includes:

Mandatory PSE tokens identifying the set of available partitions
Set of PSE cluster members providing load balancing information to applications
PSE cluster service port number

Optional tokens define the following:

Default partition
Set of authorized users and groups
Load update period
Selected load average sample rate
Set of file system definitions that can be verified by farmd

Table 16-1 shows an example configuration_data entry containing all allowable PSE token types. A quick overview of each token type can be found in Table 16-2 with a full description in Section 16.3.3.

Table 16-1 configuration_data Example Entries

Record Type	Entry
configuration_data	PSE_PARTITIONS compute FAST1 FAST2
configuration_data	PSE_DEFAULT_PARTITION compute
configuration_data	PSE_LOADSERVERS mach1 mach2
configuration_data	PSE_UPDATE_PERIOD 60
configuration_ data	PSE_WHICH_LOADAVG 5
configuration_data	PSE_SERVICE 6666
configuration_data	PSE_USERS !user1
configuration_data	PSE_GROUPS users
configuration_data	PSE_FILESYSTEM system1:/PSE /usr/pse
configuration_data	PSE_SCHED_POLICY sched_other

Table 16-2 configuration_data Tokens

PSE Token	Description
`PSE_PARTITIONS`	Lists the names of the partitions available in this PSE cluster
`PSE_DEFAULT_PARTITION`	Defines a default partition in which PSE jobs are run
`PSE_LOADSERVERS`	Defines a set of machines that act as load information servers
`PSE_UPDATE_PERIOD`	Defines the time interval (in seconds) that each PSE cluster member sends load information to the PSE load server or servers
`PSE_WHICH_LOADAVG`	Selects the time interval (in seconds) over which system load information is averaged
`PSE_ SERVICEPORT`	Defines the service port number for the PSE cluster
`PSE_ GROUPS`	Lists the groups with privileges to use this PSE cluster
`PSE_ USERS`	Lists the users with privileges to use this PSE cluster
`PSE_PREF_ COMM`	Lists the preferred communications mediums
`PSE_FILESYSTEM`	Lists the file systems that must be mounted by all PSE cluster members
`PSE_SCHED_ POLICY`	Selects a scheduling policy for PSE applications

16.3.2 Partition Definition

Any database record not prefixed by configuration_data is assumed to be a partition record. There should be at least one partition defined that lists all PSE cluster members. There should also be a partition record for each partition name in the PSE_PARTITIONS definition from the PSE cluster configuration_data section. Each partition must contain at least one PSE_MEMBERS definition.

Partition records are used to associate a set of hostnames and attributes to a partition name. Partition attributes are defined using the set of available tokens listed in Table 16-3.

Table 16-3 Partition Tokens

PSE Token	Description
`PSE_MEMBERS`	Lists the members of the partition
`PSE_PREF_COMM`	Lists the preferred communications media for the partition
`PSE_PRIORITY_LIMIT`	Lists the execution priority limit for programs run in the partition
`PSE_DEFAULT_ PRIORITY`	Sets the default priority for for programs run in the partition
`PSE_ FILESYSTEM`	Lists the file systems that must be mounted by all members of the partition
`PSE_GROUPS`	Lists the groups with privileges to use the partition
`PSE_ USERS`	Lists the users with privileges to use the partition
`PSE_SCHED_ POLICY`	Selects a scheduling policy for PSE applications in this partition

One of the characteristics of the PSE definition language is the ability to define partitions in terms of other partitions, such as a partition name being a PSE_MEMBERS name in another partition. This simplifies the definition of PSE_ MEMBERS and provides a form of attribute inheritance. Specified attributes of a particular partition record have precedence over inherited values.

In the example shown in Table 16-4, partitions FAST1 and FAST2 inherit the PSE_GROUPS and PSE_USERS definitions from partition compute where they are defined. Similarly, the compute partition inherits PSE_MEMBERS from both FAST1 and FAST2 .

All users belonging to the users group are specifically authorized on all partitions. The user field is specifically excluded from using any partition.

Table 16-4 Example Partition Definitions

`FAST1`	PSE_MEMBERS mach1 mach2
`FAST1`	PSE_SCHED_POLICY sched_rr
`FAST2`	PSE_MEMBERS machine3 machine4
`compute`	PSE_MEMBERS FAST1 FAST2
`compute`	PSE_GROUPS users
`compute`	PSE_USERS !field

16.3.3 PSE cluster database Tokens

In the following sections present definitions of the PSE tokens and their values.

16.3.3.1 PSE_DEFAULT_PARTITION Token

The PSE_DEFAULT_PARTITION token defines the default partition for the PSE environment. It has the following characteristics:

Only one value may be specified for this token.
If the token is not defined, PSE assigns a default value identifying the first defined partition as the pse_default_ partition .
This is an optional token. If defined, it should appear only in the configuration_data section.

The default partition is the partition in which a PSE application executes if no partition has been specified by the user. To override the value set by this token, the user must set the PSE_PARTITION environment variable or use the - partition command line option.

Digital recommends that the default partition include all PSE cluster members.

16.3.3.2 PSE_FILESYSTEM Token

The PSE_FILESYSTEM token defines a set of file systems that the PSE cluster daemon checks at startup to ensure that they are mounted. It has the following characteristics:

Multiple space-separated file system mount-specifications may be specified for this token; equivalently, multiple PSE_ FILESYSTEM records may be used to specify different file systems.
There is no default value.
This is an optional token.
This token can be defined as part of the configuration_ data section. The token may also be defined for any partition.

The PSE_FILESYSTEM token provides a means of checking required file systems to be sure they mounted and accessible. The file systems are not actually mounted. The PSE cluster daemon issues an error message during PSE cluster verification indicating that a file system is not mounted when one of the token specified file systems cannot be found.

16.3.3.3 PSE_GROUPS Token

The PSE_GROUPS token defines groups that can or cannot access the PSE environment for a particular PSE cluster or partition. It has the following characteristics:

Multiple values may be specified for this token. This is generally accomplished by using additional PSE definition records specifying different group-ids .
All groups are granted access by default.
This is an optional token.
This token can be defined as part of the configuration_ data section or the partition section.

Use the PSE_GROUPS token to grant or deny access to the PSE environment by specific groups of users. If you do not use this token or if you specify no user groups as the value, all user groups can access the environment. If you specify one or more groups, only those groups have access. You can deny access to a group by placing an exclamation mark (!) in front of the group name.

This token can be used in conjunction with the PSE_ USERS token.

16.3.3.4 PSE_WHICH_LOADAVG Token

The PSE_WHICH_LOADAVG token selects the time interval (in seconds) over which system load information is averaged. It has the following characteristics:

Only one value may be specified for this token. Acceptable values are 5 seconds, 30 seconds, or 60 seconds.
The default value is 30 seconds.
This is an optional token. If this token is defined, it should appear ONLY in the configuration_data section.

16.3.3.5 PSE_LOADSERVERS Token

The PSE_LOADSERVERS token defines the machines that act as load servers for the PSE environment. It has the following characteristics:

Multiple space separated hostnames may be specified for this token; equivalently, multiple PSE_LOADSERVERS definition records can be used to enumerate additional hostnames.
There is no default value.
This token is required and must be defined ONLY in the PSE cluster configuration_data section of the database.

16.3.3.6 PSE_PARTITIONS Token

The PSE_PARTITIONS token defines all of the partitions in the PSE environment for a specific PSE cluster. It has the following characteristics:

Multiple space separated partition-ids may be specified for this token; equivalently, multiple PSE_PARTITIONS definition records can be used to enumerate additional partition- ids.
There is no default value.
This token is required and must be defined ONLY in the configuration_data section of the database.

This token defines all the partitions available to the PSE environment. Any partition identified by this token should be further defined to have various PSE characteristics. The PSE_MEMBERS token should contain one or more hostnames for each partition specified.

16.3.3.7 PSE_DEFAULT_PRIORITY Token

The PSE_DEFAULT_PRIORITY token defines the default execution priority for all PSE applications. It has the following characteristics:

Values are specified according to POSIX priority conventions (0-63). Only one value per partition may be specified for this token. The value must not exceed 30.
The default value is 19.
This is an optional token.
This token can be defined as part of the configuration_ data section or in a partition section.
The PSE environment variable PSE_PRIORITY or the -priority command line option overrides the value of this token.

16.3.3.8 PSE_PRIORITY_LIMIT Token

The PSE_PRIORITY_LIMIT token sets an upper limit on the execution priority for PSE applications. It has the following characteristics:

Values are specified according to POSIX priority conventions (0-63). Only one value per partition can be specified for this token. To prevent conflicts with the operating system software, the value should not exceed 30.
The default value is 30.
This is an optional token.
This token can be defined as part of the configuration_ data section or in a partition section.

16.3.3.9 PSE_PREF_COMM Token

The PSE_PREF_COMM token defines a preference list for the communications medium network to be used by an application executing in the PSE environment. It has the following characteristics:

Multiple space-separated or comma-separated pref-comm-specs may be specified for this token; equivalently, multiple PSE_ PREF_COMM definition records can be used to enumerate additional pref-comm-specs.

pref-comm-spec is a list containing one or more communication-medium specifiers. The specifiers currently supported are:

shm (shared memory)
mc (Memory Channel)
atm
fddi
ethernet

The PSE cluster database can also define additional specifiers in the following form:

pref-comm-symbol=subnet.mask

For example:

PSE_PREF_COMM  shm, pse_fddi=12.144.32.0

This customization method is particularly useful when machines are interconnected using more than one adapter of the same type and where the desired adapter for PSE/HPF usage has been defined to be in a specific subnet.

Only the administrator of a PSE database can define additional specifiers of this form. It is not valid as an environment variable definition. However, users can view the available specifiers with lspart , and can select the additional specifiers in the same way as they can select the pre-defined specifiers. In the above example, users may select fddi preferentially over shm by giving the symbol pse_fddi as an argument to the -pref_comm run-time option:

% myprogram -pref_comm pse_fddi

The order in which the specifiers are listed determines PSE's preference in choosing a communications medium between any two peers. It is generally desirable to put higher-bandwidth media at the beginning of the list, and lower-bandwidth media at the end of the list.

The default setting for PSE_PREF_COMM is:

PSE_PREF_COMM  shm,mc,atm,fddi,ethernet

16.3.3.10 PSE_SCHED_POLICY Token

PSE_SCHED_POLICY is an optional token for both the configuration_data and partition records. It has the following characteristics:

Only one value may be specified for this token.
If the token is not defined, PSE applications will be scheduled using the operating system default policy (sched_ other ).
This is an optional token.
This token can be defined as part of the configuration_ data section and may also be defined for any partition.

The recognized values for this token are sched_fifo | sched_rr | sched_other; meaning real_time, first_in_first_out, real_time round robin, and timeshare , respectively.

The default scheduling policy is to use sched_other timesharing if no token is defined. For desktop workstation environments, the default timesharing policy should be used. For restricted-use, higher performance PSE cluster environments, sched_fifo can be used to approximate gang_scheduling . Under the first-in first-out, sched_fifo , scheduling policy, a running process continues to execute if there are no other higher priority processes.

Under the round-robin sched_rr scheduling policy, the highest priority process runs until either its allotted time quantum is complete or the process is preempted by another higher-priority process.

16.3.3.11 PSE_SERVICEPORT Token

The PSE_SERVICEPORT token defines the service port number for the PSE cluster environment. It has the following characteristics:

Only one value may be specified for this token; the value must be a number between 5001 and 65535.
There is no default value.
This token is required and must be defined ONLY in the PSE cluster configuration_data section of the database.
The PSE environment variable PSE_FARM or the -farm command line option identifies the PSE cluster service associated with this port number in /etc/services .

16.3.3.12 PSE_UPDATE_PERIOD Token

The PSE_UPDATE_PERIOD token defines the time interval (in seconds) that farmd waits between sends of load information to the PSE load server or servers. It has the following characteristics:

Only one value may be specified for this token. Note that the use of low valued integers (less than 10 seconds) may cause performance degradation in large PSE clusters due to high network utilization.
The default value is 60 seconds.
This is an optional token. If this token is defined, it should appear ONLY in the PSE cluster configuration_ data section.

16.3.3.13 PSE_USERS Token

The PSE_USERS token defines the users that can or cannot access a cluster or partition. It has the following characteristics:

Multiple space separated values may be specified for this token. Multiple PSE_USERS definition records can equivalently be used to specify multiple usernames.
The default value is that all users are granted access.
This is an optional token.
This token can be defined as part of the configuration_ data section or in a partition section.

Use the PSE_USERS token to grant or deny access to the PSE environment for specific users. If you do not use this token or specify an argument for this token, all users who are part of an authorized group can access the environment. You can deny access to users by placing an exclamation mark (!) in front of the username. Authorizing specific users overrides a negation when the authorized user is a part of a negated group.

This token can be used in conjunction with the PSE_ GROUPS token.