				                       Release Notes

TORQUE Resource Manager

Software Version 2.5.0

This file contains a few notes on building and installing TORQUE and 
Release notes concerning some of the new features of this release.

		                   	-------- WARNING! --------

TORQUE 3.0 Communication Changes - WARNING: TORQUE 3.0 has upgraded the 
communication protocol to version 2 in order to accomodate the multi-mom
feature (explained below). pbs_server 3.0 will not be able to communicate with
pbs_moms pre-3.0, and vice versa. Additionally, schedulers will need to use the
updated TORQUE libraries in order to communicate with the updated TORQUE

TORQUE 2.5 Array Changes - WARNING: TORQUE 2.5 Job Arrays are not backwards
compatible, you MUST NOT upgrade to this from 2.3 or 2.4 while you have job
arrays queued - See README.array_changes for more details.

---------------------------------------------------------------------------
CONTENTS

   This document contains the following sections:
     
     * Overview
     * What's New
     * Requirements
     * Installation

--------------------------------------------------------------------------
OVERVIEW

    The two main features of 3.0 are NUMA support and multi-mom capability.
    There are also several other minor enhancements and big fixes.
    For a complete list of changes see CHANGELOG.

    TORQUE 3.0 is not backward compatitble with any previous version of TORQUE.
    
    
---------------------------------------------------------------------------
WHAT'S NEW

    There is now a --enable-numa-support configure option to make pbs_mom
    compatible with NUMA systems. This has been extensively tested on UltraViolet
    and Altix hardware, and is currently in production at several sites.
    
    Essentially, this option allows pbs_mom to report "numa nodes" to the 
    server as subnodes of itself, and in turn the server reports those 
    "numa nodes" in the pbsnodes output. Numa support also creates cpusets
    around each job for its memory and cpus. There are many other features
    that accompany the NUMA support, so please refer to:
    http://www.clusterresources.com/products/torque/docs/1.7torqueonnuma.shtml

    New in 3.0 is the ability to run multiple pbs_mom daemons from the same 
    host. To learn more about the multi-mom capability see section 1.7
    of the TORQUE documentation found at http://www.clusterresources.com/products/torque/docs/index.shtml

    New from 2.5:

    Completed job information can now be logged. A new Boolean server parameter 
    record_job_info can be set to TRUE and a log file will be created
    under $TORQUE_HOME/job_logs. The log file is in XML format and 
    contains the same information that would be produced by qstat -f.

    The serverdb file which contains the queue and server configuration
    data can optionally be converted to XML format. If you configure 
    TORQUE with the --enable-server-xml option the serverdb file will
    be stored in XML format. If you are upgrading from a version of 
    TORQUE earlier than 2.5.3 the old serverdb file will be converted 
    to the new XML format.

    Munge has been added as an option for user authorization on the server.
    The default user authorization for TORQUE uses privileged ports and 
    ruserok to authorize client applications. Munge creates an alternative 
    which is more scalable and can bypass the rsh type ruserok function
    call.

    Job arrays are now suported in the commands qalter, qdel, qhold, qrls 
    in addition to the qsub command.

    Slot limits are a new feature added to job arrays which allow users and
    administrators to have more control of the number of concurrently 
    running jobs from a job array. Slot limits can be set on a per job basis
    or system wide with the new server parameter 'max_slot_limit'. 
    Administrators can also control how large user arrays can be with
    the new server parameter 'max_job_array_size'.

    New job dependecy options have been added to work with job arrays. Users
    can create dependecies based on the status of entire job arrays and
    not just individual jobs.

    qstat has also been updated to more conveniently display job array. Job
    arrays are displayed in a summary of the array by default, however,
    expanded display of the entire job can also be done.

    Special thanks to Glen Beane and David Beer for their work on the 
    new job array functionality. For more information concerning updates 
    to job arrays in TORQUE 2.5.0 refer to the README.array_changes document.

    TORQUE 2.5.0 can now be run with Cygwin. This feature was added by
    Igor Ilyenko, Yauheni Charniauski and Vikentsi Lapa. To learn how to 
    run TORQUE with Cygwin see README.cygwin. TORQUE on Cygwin was a
    community project and support for this feature will be provided by
    the TORQUE community. 
    
    For more information concerning the installation and use of TORQUE
    with Cygwin please see the README.cygwin file.

    The 'procs' keyword has been part of the qsub syntax for some time. 
    However, TORQUE itself never interpreted this argument and simply 
    passed it through to the scheduler. With TORQUE 2.5.0 the 'procs' 
    keyword is now interpreted to mean allocate a designated number of 
    processors on any combination of nodes. For example the following
    qsub command

    qsub -l nodes=2 -l procs=2 

    will allocate two separate nodes with one processor each plus it will
    allocate two additional processors from any other available nodes.
    The same allocation can be achieved with the following syntax as well.

    qsub -l nodes=2+procs=2.

    A new MOM config option was added named 'alias_server_name'. This option
    allows a MOM to add an additional host name address to its trusted 
    addresses. The option was added to overcome a problem with RPP and UDP
    when alias IP addresses are used on a pbs_server.

    'clone_batch_size', 'clone_batch_delay', 'job_start_timeout', and 
    'checkpoint_defaults' were added as new qmgr server parameters. 

    To find more information concerning the new parameters as well as other
    TORQUE features see the documentation at 
    http://www.clusterresources.com/products/torque/docs/




REQUIREMENTS
------------

An ANSI C compiler is required.   The native C compiler is recommended if it
is ANSI, otherwise use gcc.

A fully POSIX make is required.  If you are unable to "make" PBS with your
make, we suggest use of gmake from GNU.

Tcl/Tk version 8 or higher is required if you plan to build the GUI portion
of TORQUE or use a Tcl based scheduler.


BUILD AND INSTALLATION DIRECTIONS
---------------------------------

The directions to build and install are found in the PBS Administrators Guide.
A postscript and PDF copy are found in this directory.  Please read and
follow the directions CAREFULLY.

Installation instructions can also be found at http://www.clusterresources.com/products/torque/docs/


