---------------------------------------------------------------------- ELOQUENCE B.07.10 - patch PE71-0704110 ---------------------------------------------------------------------- This patch adds enhancements or fixes defects of the eloqdb6 server program as released with Eloquence B.07.10. This patch will be integrated in the Eloquence B.07.10 release. Eloquence B.07.10 must be installed before applying this patch. Severity: PE71-0704110: BUG FIX, ENHANCEMENT Superseded patches: PE71-0702080: BUG FIX PE71-0610130: BUG FIX PE71-0609270: BUG FIX PE71-0609120: BUG FIX PE71-0608110: ENHANCEMENT PE71-0607210: BUG FIX PE71-0607100: ENHANCEMENT PE71-0606290: BUG FIX PE71-0603280: BUG FIX PE71-0603100: ENHANCEMENT PE71-0602280: ENHANCEMENT, BUG FIX Patch PE71-0704110 ------------------ Platforms: All * In rare cases, a DBPUT could fail with status -809:976 (#3255). This is caused by a race condition while the same automatic master entry is added/deleted concurrently. * A problem in the client request handling was fixed that could in some cases result in a slight prioritization of some connections over others (#3325). * DBGET mode 4 does not update index position (#3333) After a DBFIND on an index, a DBGET mode 4 could not be used to position the current record inside the result. The index position was not updated by a dbget mode 4. This was inconsistent with the behavior of image paths and not compatible with TurboIMAGE/SuperDex behavior. * The server was modified to solve a compatibility issue with TurboIMAGE applications (#3247). TurboIMAGE documents that a DBFIND resets the current record number for a data set. The Eloquence DBFIND call previously did not affect the current record number. This changed DBFIND behavior is enabled if both, this patch and the corresponsing client library patch PE71-0704100 (or superseding) is installed. Otherwise the behavior is unchanged. * The server was enhanced to support additional options for logging server or individual session performance information. The following config items are supported: [server] StatFile StatFile specifies a file name to be used for logging the server utilization. If enabled, this file is updated once a minute. As the file is re-opened each time it is updated it may be moved/deleted freely. This config item was introduced with the B.07.10 release. Please refer to the B.07.10 release notes for details. User visible change: The config file must specify an absolute file name. This is consistent with the corresponding dbctl command. [server] StatFileFlags StatFileFlags specifies options that influence the StatFile format. By default (StatFileFlags not set) the file content is replaced each time it is updated. Also, the content is formatted with multiple lines, each containing a descriptive text and the actual value, separated by a colon. The following flags are supported: s (single line) causes the values to be formatted into a single line. Values are separated by a space and no descriptive text is present. a (append) causes additional values to be appended to the file instead of replacing the previous content. t (local timezone) causes the timestamp to include the offset of the local timezone from UTC. If not present, the timestamp value denotes UTC. This flag allows to use the timestamp value with DSI (MeasureWare) on HP-UX without requiring a conversion. These flags may be combined (eg. StatFileFlags = sat). Example output (single line format): 1172193450 6 110 0 441 0 0 1 [server] SessionStatFile If specified, SessionStatFile is used for logging session utilization information. Depending on the SessionStatMode setting, information is logged when a session ends or after the next database call after the specified interval expires. This file is opened on the first event and kept open until a new value is specified with dbctl SessionStatFile or the SessionStatMode is changed through dbctl. The config file must specify an absolute path name. This is consistent with the corresponding dbctl command. The information logged to SessionStatFile is substantially similar to session details provided in the HTTP status and may be used to evaluate performance/behavior of an application after it has completed. Every entry in SessionStatFile consists of a single line, fields are separated by a vertical bar (|) character. The following information is provided in SessionStatFile: timestamp - the timestamp (UTC) the entry was added TID - The id of the database thread Type - Type of Entry (E or U character), E specifies the entry was logged when the thread had completed (application disconnected from the database), U specifies the entry was logged after the interval specified in SessionStatMode had expired OSuser - Operating system account used by the application DBuser - database login (most recently) used by the application ConnTime - Connect time (in ISO format YYYY-MM-DD HH:MM:SS) ConnSec - Number of seconds elapsed since connecting Stat - Three numerical values for each monitored database activity (a subset of database activity typically called from applications). The values specify two counters and the wall time for the sum of all calls of a category. The first counter specifies the number of database calls (from the client library, may be different from application calls if client side caching is used), the second counter may specify a count related to the particular call (see below). The wall time is specified in usec (1 mio per second). The follwing database activities are monitored IO_READ - Disk reads accounted to the session. This includes both reading activity as well as any disk reads required for writing activity. The count field specifies the number of IO requests, the count2 field specifies the number of pages (8K units). DBFIND - DBFIND calls. DBGET - DBGET calls (single record). DBGETB - Multi-record DBGET calls. These are used internally by the client library if client side caching is used. The count2 field specifies the number of records obtained. DBPUT - DBPUT calls DBUPDATE - DBUPDATE calls DBDELETE - DBDELETE calls DBLOCK - DBLOCK calls, count2 specifies the number of unconditional DBLOCK calls that could not be granted immediately but were blocked by a competing lock. DBUNLOCK - DBUNLOCK calls TXBEGIN - Begin Transaction TXCOMMIT - Commit Transaction TXROLLBACK - Transaction Rollback IP - IP Address and port number (separated by a colon) used to connect to the database AppEnv - Information collected from the application environment, such as process ID, operating system specific user id, (subset of the) command line, EQ_AUDIT_INFO content Please Note: The content of the SessionStatFile is subject to change without notice. Example output (single line): 1172196823|9|E|mike|public|2007-02-22 16:54:39|4|11|11|751| 0|0|0|0|2|163|45|6|10591|0|0|0|0|0|0|0|0|0|0|1|8879|0|1|6| 0|0|0|0|0|0|0|0|0|127.0.0.1:64169|uid=102 pid=4812 pname=query3k [server] SessionStatMode SessionStatMode is a numeric value that specifies when an entry is logged to the SessionStatFile. The following values are supported: 0 - (zero) The SessionStatFile is disabled 1 - (one) A log entry is written to the SessionStatFile when a session ends. Any other value is understood to specify an interval (in seconds) after which an entry is logged to the SessionStatFile in addition to the entry that is logged after the session ends. The specified value must be at least 60 seconds. * The server was enhanced to support additional dbctl commands to allow dynamically changing the logging of performance information. The following additional dbctl commands were added dbctl statfile [FileName|DISABLED] Obtain the current value of [server] StatFile or specify a new value. If used without additional file name argument this returns the current value for [server] StatFile. When a file name is present, it specifies a new value for [server] StatFile. If DISABLED is specified, the [server] StatFile is unconfigured and no longer used. Otherwise an absolute file name must be specified. The file may not exist. When a file name is specified dba capabilities are required. For example: dbctl statfile dbctl -u dba statfile /tmp/server.stat dbctl statfileflags [flags] Obtain the current value of [server] StatFileFlags or specify a new value. If used without additional flags argument this returns the current value for [server] StatFileFlags. When a flags argument is present, it specifies a new value for [server] StatFile. An empty argument may be used to reset the flags. When a flags argument is specified dba capabilities are required. For example: dbctl statfileflags dbctl -u dba statfileflags sat dbctl -u dba statfileflags "" dbctl sessionstatfile [FileName|DISABLED] Obtain the current value of [server] SessionStatFile or specify a new value. If used without additional file name argument this returns the current value for [server] SessionStatFile. When a file name is present, it specifies a new value for [server] SessionStatFile. If DISABLED is specified, the [server] SessionStatFile is unconfigured and no longer used. Otherwise an absolute file name must be specified. The file may not exist. When a file name is specified dba capabilities are required. For example: dbctl sessionstatfile dbctl -u dba sessionstatfile /tmp/session.stat dbctl sessionstatmode [mode] Obtain the current value of [server] SessionStatMode or specify a new value. If used without additional mode argument this returns the current value for [server] SessionStatMode. When a mode argument is present, it specifies a new value for [server] SessionStatMode. When a mode argument is specified dba capabilities are required. For example: dbctl sessionstatmode dbctl -u dba sessionstatmode 1 * The template config file for the database server was updated. This version adds the new config options provided with this server release and corrects some mistakes in the inline comment. Platforms: HP-UX * On HP-UX PA-RISC the listen queue backlog size was increased (#3324) Due to a build problem a smaller listen queue backlog value was used on HP-UX PA-RISC. If a large number of new connections is started concurrently this could, in some cases, cause new connections to fail temporarily (ETIMEDOUT error returned by connect). Platforms: Windows * Changed the Windows file version to "7.1.0.16" Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0702081 - dblogreset utility - PE71-0610132 - dbrecover utility - PE71-0608010 - fwaudit utility Patch PE71-0702080 ------------------ Platforms: All * A cache-miss (in the eloqdb6 buffer cache) might result in temporarily blocking concurrent access to the same set until the disk access was completed. If some structural information was not found in the buffer cache the server might fetch the data from disk with an internal lock held. This could impact performance. The server was modified to no longer hold this internal lock during disk access. In addition, internal lock handling was improved for database read operations. * A DBGET call might overly impact performance of concurrent applications when client-side caching is used (#3315). When client side caching is enabled (the default), a DBGET might request multiple records from the server in one request. The number of records requested from the server depends on previous application database use. In some situations the performance of concurrent applications might be impacted when a large number of records was requested in one call. The server was changed to improve concurrency in this case. * Aborting on-line backup mode recovery could result in data corruption (#3279). The server process (or the dblogreset utility) performs a recovery from on-line backup on startup if the server was previously shut down while in on-line backup. This recovery transfers any changes temporarily stored in the log volume(s) during the on-line backup mode to the data volume(s). Depending on the amount of data that was saved in the log volume(s) and disk performance this could take some time. If this recovery was interrupted (or aborted for some reason) in some cases this recovery could not be re-run afterwards because the generation count of the data volume would differ from the log volume(s) generation count. In this case the data volume is corrupted as the changes were only partially transferred. * Forward-log files are now created with restrictive file permissions. Only the user account configured to run the database server may access these files (#3304). * The dbctl logfile command no longer accepts existing files (#3303). * The [ForwardLog] EnableAudit and AuditOnly configuration values were not displayed in the HTTP status (#3311) Platforms: HP-UX and Linux * If the server process is started from the root user, ownership of a log file created during the server startup is changed to the user that is configured to run the database (#2368). * The eloqdb6 IO threads did not correctly change the log file after a dbctl logfile command. The previous log file was not closed (#3294). Platforms: Windows * Changed the Windows file version to "7.1.0.14" Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0702081 - dblogreset utility - PE71-0610132 - dbrecover utility - PE71-0608010 - fwaudit utility Patch PE71-0610130 ------------------ Platforms: All * A rare defect was fixed that could result in a server abort while stopping on-line backup (#3217). A message like below is written to the server log file: Assertion failed: page_addr < vol->curr_size file volume.c, line 6183 This problem could happen if during on-line backup the data volume was virtually extended and the volume list of available blocks had to be extended as well (this is required around every 500 MB). When on-line backup was later stopped, the stored free-list information was then applied in a wrong way. This problem was introduced with changes to the recovery code in the previous patch PE71-0609120. * Using dbrestore with a pipe (for example with gzip) could in rare cases cause the server process to hang (#2987). This was caused by a log message that was output asynchronously when the pipe was closed and could in rare cases result in internal memory corruption. * Changed the Windows file version to "7.1.0.13" Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0610131 - dblogreset utility - PE71-0610132 - dbrecover utility - PE71-0608010 - fwaudit utility Patch PE71-0609270 ------------------ Platforms: All * Fixed a rare race condition that could result in a server abort when extending a volume file (#2525). A message like below is written to the server log file: Assertion failed: page != 0 It is expected that this problem can only happen in on-line backup mode. As a workround the volume extension size for the log volume could be enlarged above the default 1 MB. * Fixed a corner case in the lock scheduler that could result in a server panic on a DBUNLOCK (#3204). A message like below is written to the server log file: Assertion failed: lock_granted_tail && lq->l_prev This problem was introduced with the revised lock scheduler in patch PE71-0607100. A specific sequence of DBLOCK and DBUNLOCK calls could result in a situation that violates an internal assumption. * Changed the Windows file version to "7.1.0.12" Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0609121 - dblogreset utility - PE71-0609122 - dbrecover utility - PE71-0608010 - fwaudit utility Patch PE71-0609120 ------------------ Platforms: All * Fixed a problem in btree management that could result in an internal error when a page split operation failed due to insufficient disk space (#3052). * Corrected an internal problem where a zero record number in the free-record list could in rare cases result in an internal failure during the checkpoint (#3131). * A problem was solved that could result in an inconsistency of the internal volume generation number between data and log volumes (#3129). This problem could result in an error during eloqdb6 startup as below: failed to open volume: volume #1 has inconsistent generation count This could happen under a rare condition when forward-logging was configured but disabled subsequently and afterwards the eloqdb6 process terminated unexpectedly. It could also happen if the forward log was configured and the eloqdb6 process did abort twice in a row. In both cases, the log volume header was not updated and therefore contained a wrong volume generation. * Fixed problem with the volume generation number that could get out of sync if forward-logging was restarted during on-line backup and the eloqdb6 was then stopped while still in on-line backup mode (#3130). * Fixed a potential corruption of the server catalog that could happen in some cases if the server was aborted after changing the server catalog (schema, dbcreate, dbutil, dbrestore) but before running an internal checkpoint (#3163). In this case the records allocated for the server catalog could not be marked in use by the crash recovery and become corrupted subsequently. This problem could only affect eloqdb6 patch levels PE71-0606290 and newer. * Session statistic was enhanced to separate regular DBGET calls from multiple record fetches (using client side caching). * Changed the Windows file version to "7.1.0.11" Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0609121 - dblogreset utility - PE71-0609122 - dbrecover utility - PE71-0608010 - fwaudit utility Patch PE71-0608110 ------------------ Platforms: Linux * This patch solves a compatibility issue with the glibc2.4. Patch PE71-0607210 ------------------ Platforms: HP-UX, Linux * In some situation the queue entry of an completed IO request was not released (#3118). This could after some time result in the server message tio_enqueue: TIO queue is full Patch PE71-0607100 ------------------ Platforms: All * The lock scheduler was revised to enhance scalability and performance with a large number of competing locks. * Changed the Windows file version to "7.1.0.10" Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0606291 - dblogreset utility - PE71-0606292 - dbrecover utility - PE71-0606293 - fwaudit utility Patch PE71-0606290 ------------------ Platforms: All * Improved scalability of dbunlock with a large number of blocked competing lock requests. * Fixed a problem with forward log files that could cause an internal failure with dbrecover as below (#3072). L0: Fwr_TranslatePage(page=0x3229) failed: page not found Assertion failed: page not found, file volfwr.c, line 1504 The problem was triggered by an inconsistency in the forward log that could happen in some cases if dbrestore was used while the eloqdb6 server was active. This problem could no longer happen as a side effect of installing patch PE71-0603280. This patch provides a fix for the root cause. * The forward log format that was slightly changed. This requires installing updated utilities that access the forward log. * Changed the Windows file version to "7.1.0.9" Platforms: HP-UX * Work around HP-UX problem with memory windows (#3050). When using memory windows on HP-UX with a configured buffer size of 1 GB the server process could abort with a SEGV with a message like below: ** Cought signal 11 -- traceback follows: signo = 11 addr = bffffff0 As a workaround, a configured buffer size of exactly 1 GB is internally reduced by one page. Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0606291 - dblogreset utility - PE71-0606292 - dbrecover utility - PE71-0606293 - fwaudit utility Patch PE71-0603280 ------------------ Platforms: All * Fixed a problem with forward log files that could cause an internal failure with dbrecover (#3059). Internal failure: temporary file for transaction # not found Assertion failed: Internal failure during forward-recovery This problem was introduced with patch PE71-0602280 that enables use of a more efficient format to record index changes in the forward log files. In a specific case this could result in an inconsistent transaction status recorded in the forward log file that is catched by a consistency check in dbrecover. This problem can possibly affect installations that previously installed patch PE71-0602280 or PE71-0603100 and use additional indexes. A patched dbrecover is available on request to workaround this problem if necessary. * Fixed a problem with forward logging not enabled after a server abort without a server restart (#3056). A message like below is written is the server log file: Note: forward-logging has been disabled because the server was not shutdown cleanly After a server abort any recently committed transactions are re-applied. This temporarily disables forward logging. A temporary flag was not reset after completing recovery causing forward logging to remain disabled until the server process is restarted. Notes / Related patches: - The following related patches (or superseding) should be installed with this patch: - PE71-0602281 - dblogreset utility - PE71-0602282 - dbrecover utility - PE71-0602283 - fwaudit utility Patch PE71-0603100 ------------------ Platforms: All * Improve bimport performance on master sets (#3045). The bimport performance for master sets is often dominated by creating the associated index. This may also apply to detail sets when an additional index is used. The eloqdb6 server was changed to bypass the index transaction logic during bimport to improve database load time. Patch PE71-0602280 ------------------ Platforms: All * An internal deadlock condition was fixed that could result in on-line backup requests (dbctl backup start|stop|status) to hang (#3014). A server shutdown would not succeed in this case requiring to kill the server process. An on-line backup request needs to obtain an internal lock to ensure transactional consistency. If an on-line backup request gets blocked initially due to a concurrent operation (for example a dbstore operation is currently running) and another on-line backup request is issued a deadlock condition occurred that caused both requests to hang. * The server process was enhanced to use a more efficient format to record index and meta data changes. This enhancement results in a substantial reduction in disk space for the forward-log file when index entries are changed frequently. This change required a modification of the forward-log file format. While older forward-log files are still supported, related utility programs need to be updated to handle the new format. This requires installation of additional patches on dbrecover, dblogreset and fwaudit utilities (see related patches). * A rare defect with forward-recovery was fixed that could result in a corrupted volume set (#2999). This causes an internal consistency check to fail and eloqdb6 to abort on startup with an error message like below: Assertion failed: page == new_vol->flist_tail server panic: Aborting on internal failure, file volume.c, line 3988 This problem could happen if the data volume was extended and the volume list of available blocks had to be extended as well (this is required around every 500 MB). The recovery then missed to update the flist_tail field in the volume header, causing a panic the next time the server was restarted. The dbrecover utility was fixed to correctly update the flist_tail field. In addition the eloqdb6 behavior was modified to detect and correct this problem on startup. A notice is written to the server event log file in this case. * On HP-UX and Linux some log messages regarding eloqdb6 startup and shutdown were raised in priority so they show up in the server event log file even if LogFlags=*0 (default) is used. The following messages are affected: - "Eloquence database server active" when the eloqdb6 server process started successfully. - A new message outputs the current server patch level to the log. - "Eloquence data base server terminated" when the eloqdb6 server finished shutdown. - The message when a new forward-log segment was created. On all platforms the message indicating a previous unclean shutdown was recovered is now written to the server log. Platforms: Windows * On the Windows platform, dbctl bimport refused an absolute path beginning with a drive letter (#2997). * Changed the Windows file version to "7.1.0.7" Installation: ------------- Please download the patch archive that corresponds with the installed release. The patch files follow the conventions below: PE71-0704110-hpux-ia64.tar.gz ^ ^ ^ | | Architecture / OS specific build | Operating system Patch ID HP-UX: In order to install this patch, you need to unpack it with gzip and tar. Gzip is included with HP-UX. Installation requires root privileges. cd /opt/eloquence6 gzip -dc /path/to/PE71-0704110-hpux.tar.gz | tar xf - Files: bin/eloqdb6 newconfig/config/eloqdb6.cfg share/doc/PE71-0704110-README Linux: In order to install this patch, you need to unpack it with tar. Installation requires root privileges. cd /opt/eloquence6 tar xzf /path/to/PE71-0704110-linux.tar.gz Files: bin/eloqdb6 newconfig/config/eloqdb6.cfg share/doc/PE71-0704110-README Windows XP/2000/NT: This patch should *only* be installed if you previously installed the Eloquence server components on your system. Installation requires administrative capabilities. Two options are available for patch installation. The patch is available as self extracting archive for automatic installation and as a zip archive for manual installation. Both patches are equivalent. Installation requires administrative capabilities. For autmatic installation of this patch, please download the patch file PE71-0704110-win32.exe. Before installation, please consider closing all applications, then execute the patch installation program. Installation does not require a reboot unless the patched files were active. For a manual installation of the patch, please download the patch file PE71-0704110-win32.zip and unpack its contents. Then perform the following steps: * Please make sure the eloqdb6 service has been stopped previously (in the Service Control Manager or with NET STOP eloqdb6). * Please copy the eloqdb6.exe file into the WINDOWS SYSTEM DIRECTORY (for example C:\Windows\System32). * Please copy the eloqdb6.cfg.sam file into the etc subdirectory of your Eloquence installation (for example C:\Programs\Eloquence\etc). * Please copy the PE71-0704110-README.txt file into the share\doc subdirectory of your Eloquence installation (for example C:\Programs\Eloquence\share\doc). Files: eloqdb6.exe eloqdb6.cfg.sam PE71-0704110-README.txt