Overview

The user visible changes implemented through these patches are covered in this document.

This document extends on the Eloquence B.07.00 release notes and reflects the changes in the Eloquence patch bundle PE70-B050531.

Contents

• database server (eloqdb6)
• image3k library
• database client library (eloqdb)
• schema utility
• dbimport/dbexport utility
• dbctl utility
• dbstore utility
• dbrecover utility
• dbfsck utility
• dbvoldump utility
• eloqcore
• eloq (Linux platform)
• eloqsd service (Windows)

database server (eloqdb6)

• A new SyncerJournalFlushInterval configuration item was added that may be used to substantially improve performance in "sync" write mode.

  Previously, when sync write mode was used (SyncMode=1), the transaction journal was flushed to disk for every commit operation (either explicitly when using transactions or implicitly for each database call). In case of a system crash only the last committed transactions could potentially be lost and the volume integrity is typically maintained.

  The new SyncerJournalFlushInterval configuration item adds the option to specify the time (in milliseconds) after that changes to the transaction journal are pushed to disk. In case of a system crash any transactions committed during the period specified by SyncerJournalFlushInterval are lost but the volume integrity is typically maintained.

  The default value for the SyncerJournalFlushInterval configuration item is 500 ms (half a second).

  NOTE: This default modifies the behavior of the current SyncMode and in case of a system crash may result in losing any transactions committed within the last 500 ms. To retain the previous SyncMode behavior the SyncerJournalFlushInterval configuration item must be added to the eloqdb6 configuration file with a value of zero. In the section [config] please add a line like below:

     [config]
     SyncerJournalFlushInterval = 0
  

  Note: Specifying a low SyncerJournalFlushInterval value may have a detrimental effect on performance. This will result in additional load on the I/O subsystem and also the operating system may block concurrent access to the lock volume(s) during a sync operation.

• The maximum number of concurrent connections has been increased to 4000 on the HP-UX platform (PA-RISC and Itanium). Windows and Linux are still limited to 1000 concurrent connections.

• The eloqdb6 database server was modified to remove the 2 GB file size limitation for dbstore/dbrestore.

Administrative enhancements

• The eloqdb6 server was changed to refuse performing a dbrestore or bimport operation when on-line backup mode is active.

  This change attempts to avoid unintentionally filling the transaction log volume which would lead to a server abort in online backup mode. Any database modification in on-line backup mode is temporarily saved in the transaction log volume.

  An error message like below is now returned in this case:

  dbrestore:

  Request refused, on-line backup mode is active and /force option not specified.

  bimport:

  Request refused, on-line backup mode is active and /f option not specified.

  The new dbrestore option /force and bimport option /f may be used to override this limitation.

• The previously undocumented dbrestore /all option allows to restore the security context with the database. Using this option currently causes dbrestore to fail if the user is not allowed to access the restored database.

• The dbctl syncmode command was enhanced to allow setting and returning the current status of the sync mode configuration item and the current value of SyncerJournalFlushInterval.

    usage: dbctl syncmode {ON [msec]|OFF|STATUS}
  

  For example:

    dbctl syncmode on 500
  

  Enables sync write mode and defines an SyncerJournalFlushInterval interval of 500 msec.

    dbctl syncmode on
  

  Enables sync mode. Equivalent to SyncMode=1

    dbctl syncmode off
  

  Disables sync mode. Equivalent to SyncMode=0

    dbctl syncmode status
  

  Returns status of sync mode. In case the sync mode is used and the SyncerJournalFlushInterval is nonzero it also returns the current value of SyncerJournalFlushInterval in msec.

TurboIMAGE enhancements

• Full support for TurboIMAGE btree modes was enabled. This causes the image3k DBINFO modes 113 and 209 to return information on master sets with "attached" indexes. It also enables the use of DBFIND mode 1 (if the btreemode1 flag is set) to locate records though the master set index ("superchains").

  Please note that index access (other than DBFIND mode 1) is always enabled with Eloquence, regardless whether the /INDEXED flag is set for a master set.

image3k library

• The image3k library was enhanced to provide additional context information if a status code is returned.
  • The DBEXPLAIN output was modified to include the database and the referenced set and item if called immediately after a failed database call. The output is truncated to 79 characters.

        DBINFO MODE 205 ON customers OF sample
        DBFIND MODE 21 ON customers.custno OF sample
    

  • The DBCALL call was modified to return the database and the referenced set and item if called immediately after a failed database call. The output is truncated to 72 characters for MPE compatibility.

• The DBOPEN call has been enhanced to check for an environment variable that matches the database name. This environment variable may be used to replace the database name. This mechanism may be used to provide a replacement to MPE file equations with the image3k library.

  To be effective the environment variable needs to start with the prefix "EQ3K__" followed by the upshifted database name. Any dot characters (.) in the database name are replaced by underscore characters (_).

  For example:

      Database Name   Environment Variable
      --------------- ---------------------
      Sample          EQ3K__SAMPLE
      sample.test     EQ3K__SAMPLE_TEST
  

• The EQ3K_MINCAPACITY environment variable may be used to specify a minimum value for capacity returned by DBINFO calls mode 202 and 205.

• The image3k library on HP-UX was modified to accept an TPI item offset of 5000. While the HP-UX version of the image3k library uses 10000 as a mininum value for a TPI item number to ensure full compatibility with MPE checking for an offset of 5000 in addition provides full compatibility to the Windows and Linux versions.

• The change in image3k patch PE70-0406020 to support an offset of 10000 for TPI items caused an ambiguity on little endian platforms (Linux and Windows). On these platforms the image3k library could no longer distinguish if an item number or name was passed. This could result in status -52 returned from DBFIND. To solve this abiguity and still support the concept to have a mininum value for a TPI item number we have changed the offset for TPI item numbers from 10000 to 5000 on Linux and Windows.

  Any application that relies on TPI items having numbers above 10000 needs to be changed to use an offset of 5000 on little endian platforms. The HP-UX version of the image3k currently retains an offset of 10000 for TPI item numbers for full backwards compatibility.

• The TPI related DBINFO modes have been modified to accept and return item numbers greater than 10000 in case an item is a TPI item. This is specified in the TPI documentation and solves potential compatibility issues in case applications specifically check for this item number range. Eloquence simply adds 10000 to the index item number.

• DBOPEN has been corrected to return status -1 if the database could not be opened. In previous image3k versions could return status -11 if the database did not exist.

• DBLOCK returns a status code 26 in case a deadlock situation was detected. With previous image3k versions DBLOCK returned a status code -35. Status code 26 is specified by te TurboIMAGE documentation.

  Please note that other intrisic calls that modifiy the database could still return status -35 in case a deadlock situation is encounted.

• DBINFO modes 811 and 812 have been modified to accept IMAGE data items in addition to index items. Previous versions only accepted index items and returned a status when a data item was specified.

• Added support for HP Fortran 90 using the +ppu compile option in HP-UX. When the +ppu compile option is used, an underline character is added to all external symbols (e.g. dbget becomes dbget_).

• Added support for the DBMAINT intrinsic call. The DBMAINT call is undocumented on MPE may be used to create or erase a database. The Eloquence implementation in addition supports purging of a database.

  The DBMAINT arguments are similar to DBOPEN:

  DBMAINT (base,password,mode,status)
  

  base

  is the name of an integer array containing a string of ASCII characters. The string must consist of two blanks followed by a left-justified database name and terminated by a semicolon, a space or a NUL character, for example, " ORDERS;".

  password

  is the name of an integer array containing a left-justified string of ASCII characters consisting of an optional password followed by an optional user identifier.

  The following constructs are valid for the password and user identifier (a _ represents a blank):

    password [{_;}[/USERIDENT{_;}]]
  

  The examples below specify an user name and an empty password

    [/USERIDENT]{_;}
    ;[/USERIDENT{_;}]
  

  The example below specifies an user name and a password

    password[/USERIDENT]{_;}
  

  The password or the user identifier string must be terminated by a semicolon, space or NUL character. Since Eloquence does not assume a fixed length, a termination character should always be present.

  Unless both the password and the user identifier string are present, the password argument is ignored.

  mode

  is an integer equal to 1, 2, or 4 indicating the type of operation requested.

    1 - create the database
    2 - erase the database.
    4 - purge the database (Eloquence specific)
  

  status

  is the name of an array of 10 halfwords in which the database returns status information about the procedure.

  DBMAINT requires exclusive access to the database and affects all sets of a database. The Eloquence dbcreate, dberase and dbpurge utility programs allow to specify a list of data sets in addition.

  Eloquence either requires that the user has administrative access to the database (eg. it is member of the "dba" group) or has the erase privilege (in case of mode 4).

• DBFIND TPI modes may also be used to access search items (equivalent to the TurboIMAGE btree modes).

• DBLOCK modes 3/4/13/14 have been modified to support the "@" qualifier to specify an effective database lock. This is supported by TurboIMAGE but was not possible with Eloquence.

• DBUNLOCK did not support the extended Eloquence modes to allow selective unlock. DBUNLOCK has been modified to follow the Eloquence documentation.

• A compatibility problem with TPI DBFIND mode 1 and 21 with numeric values has been fixed. DBFIND mode 1/21 was handling numeric arguments on an index incorrectly.

  DBFIND was expecting that numeric values are passed as text and converted it to the binary representation. However the TPI documentation defines that numeric values should be passed as a binary value. DBFIND mode 1/21 on indexes has been changed to follow the TPI documentation.

  The HP3K_TPI_COMPAT property may be used to enable the previous behavior. If the bit 1 in the HP3K_TPI_COMPAT property value is set (+2) then a numeric argument may be passed as text if the argument either starts with an operator (==, <=, >=), a positive or the negative sign. Otherwise a binary value is assumed. As this may cause ambiguities, the default is that numeric values must be passed as a binary value.

  Setting this property can be done with the dbutil utility either interactively or using the following script:

    --- snip ---
    database "<database_name>";
    create property "HP3K_TPI_COMPAT"
     value "2";
    --- snip ---
  

• The encoding of the status array has been modified in case an Eloquence call fails. In case a nonzero status is returned the status array is used as below:

    Element Description
    ------- ------------------------------------
     1      IMAGE status code
     2 - 4  unchanged
     5      zero
     6      IMAGE intrinsic ID
     7      Eloquence status code
     8      Eloquence extended status code
     9      IMAGE mode
     10     Encoded Eloquence call ID and mode
            (Format: 6 bit id, 10 bit mode)
  

• DBLOCK could return wrong status codes. A wrong status code was returned instead of status -128 and -126.

• The IMAGE3K library has been modified to consistently return status -105 in case a memory allocation failed.

• The Eloquence DBFIND mode 1/21 index handling has been enhanced with the option to handle TPI and TurboIMAGE btree search arguments more similar (#2032).

  TurboIMAGE btree indexes and TPI indexes use slightly different conventions in the DBFIND mode 1/21 search argument. With TurboIMAGE btree indexes the wildcard character can be configured through DBCONTROL or dbutil. For TPI indexes a "@" character is used. TPI also considers a space character before the "@" wildcard to be special.

  The IMAGE btree functions are currently not enabled in Eloquence and Eloquence indexes are used used as a replacement. However Eloquence indexes follow the TPI conventions which could cause problems with applications relying on the IMAGE btree behavior. As Eloquence allows to use all TurboIMAGE DBFIND modes, independendly of the index type, this option allows DBFIND mode 1/21 calls to work more closely.

  The HP3K_TPI_COMPAT database property can be used to cause Eloquence indexes to follow the TurboIMAGE btree behavior more closely. If HP3K_TPI_COMPAT is set to nonzero it causes TPI indexes to use the current IMAGE btree wildcard character and not handle trailing spaces as a special case. The default value is zero which implements the previous behavior. Setting this property can be done with the dbutil utility either interactively or using the following script:

    --- snip ---
    database "<database_name>";
    create property "HP3K_TPI_COMPAT"
     value "1";
    --- snip ---
  

  This property is currently not available in the dbutil TurboIMAGE property dialog.

• DBCONTROL mode 5 and 6 return status -82 is CIUPDATE is DISALLOWED.

• Implemented the undocumented DBVERSION intrinsic. The DBVERSION intrinsic result follows the MPE format and returns the following NUL terminated string:

     MPE/iX:     "HP30391 C.1005"
     Eloquence : "IMAGE3K B.0700.12"
  

• Mode 0 has been added to all database calls. On MPE this may used to obtain version information about a TurboIMAGE intrinsic call (eg. QUERY/3000 version command). Eloquence returns status -31 (invalid mode) but status elements 2 to 4 return the Eloquence image3k library version (w/o the patch level, for example B.07.00). It is recommended to use DBINFO mode 801 (or DBVERSION) to obtain the TurboIMAGE version.

database client library (eloqdb)

• The eloqdb database client library was enhanced to upload process information to the database server when a server connection is established. The process information is then output in the session status of the server web status and allows to identify the application.

  The process information includes the user id (not on Windows), the process id and the command line. In addition, if the environment variable EQ_AUDIT_INFO is set its content is also included. The EQ_AUDIT_INFO environment variable allows to include application specific information in the status output.

• The eloqdb database client library was enhanced to check if a process was fork()ed with a database opened. This is unsupported and resulted in undefined behavior with previous library versions.

  Any attempt to access the database in the forked child process will now abort with an internal error like below:

    Assertion failed: 
    initial_process_id == current_process_id
  

  A fork() is valid if no database is currently opened.

• The Eloquence client library was changed to return the sort item in the related detail set when a master set is passed as qualifier (#2300).

  The image3k library internally makes use of the Eloquence DBINFO mode 303 to implement DBINFO mode 301. However the Eloquence DBINFO mode 303 did not return sort item information when a master set was provided as qualifier.

schema utility

The new -w command line option allows to suppress the output of specific warning messages.

  -w id[,..]

For example:

When unrefenced items are present in a schema definition, the schema processor did output the following warning message and list all unreferenced item names.

  *** Unreferenced items:

  *** (WARNING #11) Unreferenced items:

dbimport/dbexport utility

• A new -x command line option was added to dbexport and dbimport that allows to specify a list of data sets to be excluded.

  When the -x option is not present the command line may specify a list of data sets to be processed. With the -x option present, this is understood as a list of data sets to be excluded.

• dbimport has been modified on HP-UX and Linux to handle CR/LF as a line terminator in a restructure file.

dbctl utility

dbstore utility

The dbstore utility allows to perform a dbstore operation when the server is not active or while the server is in on-line backup mode. If the eloqdb6 is in on-line backup mode the dbstore utility does not require exclusive access and may be performed while the database is in use.

  usage: dbstore [options] database {archive_file|-}
  options:
   -help   - show usage (this list)
   -u user - set user name
   -p pswd - set password
   -c cfg  - server configuration file

dbrecover utility

dbfsck utility

• An experimental option -B was added to dbfsck to enable an additional check on internal btree linkage.

• node ids (node numbers) may be specified on the command line. If present pass 2 does only process the specified nodes.

dbvoldump utility

eloqcore

• The STORE FORM statement was modified to observe the internal FORM file size limitation of 64k. An error 865 is returned in case the screen buffer exceeds the FORM file format limit.

• The error messages 862,863,864,865 were not included in the message catalog file and have been added.

    862 - Improper argument type, INTEGER expected
    863 - Inconsistent number of form fields
    864 - Internal buffer overlow while encoding 
          screen line
    865 - Screen buffer size exceeds forms file limits
  

• This patch adds the option to configure the results of the eloqcore SYSID$ function. Previous versions returned the hostname (truncated after 20 characters) with the SYSID$ function.

  The return value of the SYSID$ function may either be configured in the Eloquence configuration files (elog.config, etc.) or defined with the EQ_SYSID environment variable.

  Using the EQ_SYSID environment variable:

  If the EQ_SYSID environment variable is defined its content is returned with the SYSID$ function.

  Using a configuration file:

  The SYSID "value" clause is now recognized in Eloquence configuration files and may be used to define the result of the SYSID$ function. Please note that the configuration file entry is currently limited to 20 characters.

• eloqcore was enhanced to support the EQ_ELOQRC environment variable. When specified the environment variable defines the path to the user specific configuration file that is used instead of the default user specific configuration file ($HOME/.eloqrc on Linux/UNIX and $HOME/eloqrc on Windows).

eloq (Linux)

Support for the old BSD pty devices is no longer required in the kernel. With recent Fedora Linux distributions it is no longer required to recompile the kernel to add support for BSD pty devices.

In addition, eloq on Linux no longer needs to run as SUID root. Installing the patch bundle should drop the SUID bit.

eloqsd service (Windows)

The eloqsd on Windows now allows to configure the method how users are authenticated. This is controlled with the [Config] AuthPolicy configuration item in the eloqsd.cfg configuration file:

AuthPolicy=server

The default AuthPolicy=server configuration performs a user logon for each eloqsd connection and runs the associated processes with the privileges of the user. This allows separate privileges for each session and is preferable in most cases for security reasons. However the maximum number of concurrent eloqsd connections is limited by the Windows desktop heap (see below).

This authentication method has been used in previous eloqsd versions and is still recommended. It is therefore the default.

AuthPolicy=user

With the new AuthPolicy=user configuration, the connected user is validated using the configured user name and password in the userFile (eloqsd.user by default). However, after this validation, the user is not logged-on to Windows. Instead, all actions (for example launching an eloqcore process) are performed using the privileges of the eloqsd process. As this implies less overhead than a real user logon a higher number of concurrent connections is possible without having to adjust the size of the Windows desktop heap.

AuthPolicy=user requires the eloqsd process to run as an account different from the default Windows SYSTEM account. The necessary configuration steps are:

1. Open the Services applet from the Computer Management console in the Windows Control Panel.

2. Select the eloqsd entry in the list. Open the Properties dialog either from the menu or using the context menu (right mouse button). Select the Log On tab on top of the Properties dialog.

3. Configure an appropriate account to run the eloqsd. This account should have the necessary privileges to access all resources your application requires.

Please note:

• When using AuthPolicy=server, the maximum number of connections is limited by the Windows desktop heap, resulting in Windows error #1816 (ERROR_NOT_ENOUGH_QUOTA) when this limit is exceeded.

  To overcome this limit, the size of the Windows desktop heap for the non-interactive windowstation must be increased.

  This requires to change a Windows registry entry with the Windows regedit or regedit32 tool. Please note that this must be done with extreme care, as doing the wrong changes to a registry value can prevent Windows from operating correctly.

  The necessary procedure is documented in the Microsoft Knowledge Base document #184802.

  Summary:

  Locate the following value in the Windows registry (you must be administrator to edit this value):

   HKEY_LOCAL_MACHINE\System\CurrentControlSet\
   Control\Session Manager\SubSystems\Windows
  

  This is a string value consisting of multiple sections. Locate the section starting with SharedSection=, it should have 3 parameters which are comma-separated.

  Example: SharedSection=1024,3072,512

  The 3rd parameter (in this case 512) is the one which must be increased. If you find only 2 parameters you can add the 3rd parameter yourself.

  You can set the 3rd parameter as high as the 2nd (in this case 3072), but this is often too much. Increase it in steps of 256 until your problem is solved.

  Windows must be restarted afterwards to activate this change.

• When using AuthPolicy=user the configured users must have a password assigned in the userFile (eloqsd.user by default) to successfully connect to the eloqsd.

Configuring the Windows logon type

The new [Config] LogonType configuration item in the eloqsd.cfg configuration file is used with AuthPolicy=server (see above) to specify how a user is logged-on to Windows:

LogonType=interactive

This mode grants access to all resources the same user would have when logging on interactively. It has the least side effects and therefore is now the default.

Unless this is used on a Windows Domain Controller, this mode should not require to configure any additional user rights.

However, on a Windows Domain Controller, interactive access is denied to non-administrative accounts by default. To overcome this, the "allow log on locally" user right must be granted (see below).

LogonType=batch

This mode was used in previous eloqsd versions. It uses the "batch" logon type which is meant for background processes and is supposed to be more economical than the "interactive" logon type. However, in recent Windows versions a "batch" process is restricted so that this mode may no longer be appropriate. It requires the "log on as a batch job" user right to be granted (see below).

The new default is "interactive". To retain the previous behavior, it is required to manually configure LogonType=batch.

To grant either the "allow log on locally" or "log on as a batch job" user right, the following configuration steps are required:

1. [optional, recommended] Create a dedicated group for all eloqsd accounts (i.e. the accounts configured with UID in the userFile (eloqsd.user by default)). This will simplify the configuration. Also, it allows to grant access to common directories for all eloqsd accounts (e.g. a common program file directory).

2. Open the Local Security Policy editor from the Computer Management console in the Windows Control Panel.

   Please note: On a Windows Domain Controller please open the Domain Controller Security Policy editor instead (not the similarly named Domain Security Policy editor!).

3. In the Security Policy editor, navigate to Security Settings -> Local Policies -> User Rights Assignment.

4. In the policy list, locate the desired user right and open the Properties dialog either from the menu or using the context menu (right mouse button). If you created a dedicated group as suggested above (1), add this group to the list. Otherwise, selectively add your eloqsd accounts to the list.

The eloqsd.cfg.sam and eloqsd.user.sam configuration template files have been updated to reflect the new configuration options.