---------------------------------------------------------------------- ELOQUENCE B.07.00 - patch PE70-0512060 ---------------------------------------------------------------------- This patch adds enhancements or fixes defects of the eloqdb6 program as released with Eloquence B.07.00. This patch will be integrated in the Eloquence B.07.00 release. Eloquence B.07.00 must be installed before applying this patch. Severity: PE70-0512060: CRITICAL (potential data loss) Superseded patches: PE70-0511300: ENHANCEMENT, BUG FIX PE70-0510170: BUG FIX PE70-0510100: BUG FIX PE70-0504270: BUG FIX PE70-0504150: BUG FIX PE70-0502070: ENHANCEMENT, BUG FIX PE70-0501070: BUG FIX PE70-0412282: ENHANCEMENT, BUG FIX PE70-0412171: ENHANCEMENT PE70-0412150: ENHANCEMENT PE70-0411291: CRITICAL (server abort) PE70-0411191: CRITICAL (server abort in restructuring) PE70-0411160: BUG FIX PE70-0409140: CRITICAL (data loss, corruption) PE70-0408310: BUG FIX PE70-0407280: ENHANCEMENT PE70-0406240: BUG FIX PE70-0406030: BUG FIX PE70-0401160: CRITICAL (server abort, corruption) PE70-0401130: CRITICAL (server abort, corruption) PE70-0312080: BUG FIX PE70-0311180: ENHANCEMENT PE70-0310180: BUG FIX PE70-0310082: BUG FIX PE70-0309300: BUG FIX PE70-0307161: CRITICAL (server abort, corruption) PE70-0305270: BUG FIX PE70-0304041: CRITICAL (database inconsistencies) Patch PE70-0512060 ------------------ Platforms: All * A problem was fixed that could potentially result in data loss when the eloqdb6 server was shutdown while in on-line backup mode and forward logging is used (#2936). When the server is in on-line backup mode, any database changes are temporarily saved in the transaction log volume(s). When ending on-line backup mode the saved changes are applied to the data volume(s). If the server process is shut down or aborts in on-line backup mode, any pending changes are applied on the next server startup. Due to a problem in the backup management code the server might fail in some cases to correctly identify a condition where the server was shutdown cleanly while in on-line backup mode when forward logging was used. When the server was restarted it could fail to apply any saved changes since start of the backup. A change was made to eloqdb6 to ensure this condition is handled correctly. * The server was changed to write a log message when a new forward log segment is created (#2937). A message like below is written to the server log file: created new forward-log segment (/path/file) Where path/file specifies the current fw log file. * The server was modified to no longer disable forward logging when it was shut down while in on-line backup mode (#2936). When the eloqdb6 server process was aborted a recovery of recently committed transactions is performed on server startup. If a recovery is required forward logging is diabled and a notice is written to the server log file as the forward log files can not be used to recover the server without a previous backup. However this restriction does not apply if the server was shutdown cleanly while on on-line backup mode. * Fixed a problem with the TXCOMMIT counter (new function introduced with patch PE70-0511300) that might be incorrect depending on the use of transactions by the application. Notes / Related patches: - Installation of dblogreset patch PE70-0512061 (or newer) and dbrecover patch PE70-0408312 (or newer) is required with this version of eloqdb6. - Process information is only available if database client library patch PE70-0411290 (or newer) is installed. Patch PE70-0511300 ------------------ Platforms: All * A problem during database restructuring was fixed that could cause the restructure process to fail due to a missing entry in a manual master (#2926). If restructuring fails the database could be in an inconsistent state and a subsequent DBOPEN on this database fails with status -91 (incompatible database version). The database restructuring was enhanced to create any missing manual master records. All items besides the search item are initialized with default values (space or zero). * The eloqdb6 server was enhanced to collect session specific usage information. * The HTTP status display was enhanced and a new session detail information page is available. The session detail information page also includes information in session specific database use. For the following events the number of occurrences and the total processing time is available: Disk read request, DBFIND, DBGET, DBPUT, DBUPDATE, DBDELETE, DBLOCK, DBUNLOCK as well as transaction BEGIN, COMMIT and ROLLBACK. * The eloqdb6 process could abort with an internal failure. A message message like below was written to the server log (#2847): Assertion failed: lrec != NULL file btree.c, line 1030 This was caused by a defect in the recovery code that could in a rare case trigger an internal consistency check if the volume was filled during a specific index operation. Patch PE70-0510170 ------------------ Platforms: HP-UX, IA64 based systems * A potantial stack overflow problem was corrected that could potentially result in a hang or malfunction of the server process on HP-UX for IA64 based systems. * The values on current and max. stack use are now correctly returned in a thread status list (HP-UX, IA64 based systems). Patch PE70-0510100 ------------------ Platforms: All * A number of potential latency problems were fixed. Some calls could potentailly block concurrent operations for some time in rare cases. These calls have been modified to improve concurrency. This includes - adding to a sorted chain when a large nbumber of entries need to be skipped - skipping a large number of seleted records - searching an index and returning a count on the found records - a number of internal operations * Fixed a possible problem on Windows that could block execution for some time in a rare case. * Add warning message when inserting entry to sorted chain in wrong order and more than 1000 records need to be skipped for a path. If more than 1000 records per chain need to be skipped a warning message like below is written to the server log file (once every 10 seconds). link_chain: excessive skipping DETAIL->MASTER (p=#, cnt=#)", where p=# specifies the internal index into the chain and cnt=# specifies the number of records that were skipped. * Add warning message if excessive retries are required due to an internal transaction lock conflict. Some operations are internally delayed and re-executed subsequently if a transaction lock conflict, but not a deadlock, was found. A warning message was added if an unusual number of attempts are required. A message like below is output to the server log file: dbput: excessive retries on tx conflict (SETNAME) * A rare corner case was fixed that could result in invalid FixRec freelist pointers in an on-line backup and forward recovery (#2810). The FixRec freelist is used to maintain available and deleted record numbers. * The dbctl logfile/logflags command were changed to require dba authorization (#2760). With previous versions this dbctl command could be executed without requiring a privileged account. This is a potential security problem. * A race condition was fixed that could result in an error message like below when the server was shutdown (#2132): Assertion failed: current_task != NULL file thread.c, line 372 * Fix missing end tag in HTML status output (#2779) * The eloqdb6 server was enhanced to support listing the opened databases along with some process information (#2697). usage: dbctl list dbopen database Example output: TID IP ADDR Mo User / Login ---- -------------------- --- ---------------------------------- 9 127.0.0.1:54093 9 mike / public uid{102}pid{16409}pname{query3k} - TID specifies the internal thread id associated with the process - IP ADDR specifies the IP address and port number used by the application to connect to the database server - MO specifies the database open mode (note to image3k users: This open mode is different from image3k dbopen modes) - USER / LOGIN specify the os specific user name and database login. In a second line additional information on the process may be present Notes / Related patches: - Installation of dblogreset patch PE70-0408311 (or newer) and dbrecover patch PE70-0408312 (or newer) is required with this version of eloqdb6. - Process information is only available if database client library patch PE70-0411290 (or newer) is installed. Patch PE70-0504270 ------------------ Platforms: All * DBLOCK mode 6/16 could fail to correctly identify a locking conflict if the ">=" or "<=" lock operator was used (#2701). If the lock operator ">=" or "<=" was used in a lock descriptor with DBLOCK mode 6 or 16 a lock conflict could potentially be missed, depening on locking order. In this case both (conflicting) locks were granted and any database write attempt would result in a status -12. * If the server performed a crash recovery on startup the estimated size of the transaction journal could be wrong (#2706). The database server tracks the size of the transaction journal and performs a checkpoint if it exceeds a configured size. The size of the transaction journal could be wrong if a crash recovery was performed on startup. In rare cases this could cause a performance degradation as the checkpoint operation is performed too often. * Renaming a database could cause catalog table corruption (#2702). If the new database name is at least 10 bytes shorter the catalog tables 20 or 31 might be corrupted in a rare case. A corruption results in a server panic message as below: Pool_Read(node=#20:SP) header inconsistency: header.size=38, addr->size=52, header.flags=1 (cluster=0x210, offset=7258) server panic: Fatal problem detected Assertion failed: header.size >= addr->size * A bimport could overflow the transaction log volume (#2704). The bimport function could starve the checkpoint process and eventually overflow the transaction log volume. * A limitation in the btree code was fixed that could lead to a server panic if the transaction log volume was filled (#2705) The eloqdb6 server has been modified to return a status error. * Setup time reference before initial checkpoint (#2680). In previous eloqdb6 versions the first record in a forward log file could have a zero time stamp. Patch PE70-0504150 ------------------ Platforms: All * The eloqdb6 process could abort with an internal failure. A message message like below was written to the server log (#2696): Assertion failed: !(bhp->flags & BUF_NEWPAGE) server panic: Aborting on internal failure, file mpool.c, line 2231 This problem was caused by a defect in the recovery of a transaction lock conflict in the btree code. * The eloqdb6 server was changed to refuse performing a dbrestore or bimport operation when on-line backup mode is active (#2649). This change attempts to avoid unintentionally filling the transaction log volume which would lead to a server abort in online backup mode. Any database modification in on-line backup mode is temporarily saved in the transaction log volume. An error message like below is now returned in this case: dbrestore: Request refused, on-line backup mode is active and /force option not specified. bimport: Request refused, on-line backup mode is active and /f option not specified. The new dbrestore option /force and bimport option /f may be used to override this limitation. * The previously undocumented dbrestore /all option allows to restore the security context with the database. Using this option currently causes dbrestore to fail if the user is not allowed to access the restored database. * A defect with bimport was fixed that could lead to a hang of the bimport operation upon completion when the /v option was present and the EnableIPC=0 configuration option was used. Patch PE70-0502070 ------------------ Platforms: All * Invalid arguments passed to scan_req_key() could result in an internal failure as below: Assertion failed: keydef->k_len == k_ctx->idx_keysz server panic: Aborting on internal failure, file fseq.c, line 830 This could happen due to a bug in SQL/R. The eloqdb6 server was fixed to properly verify the arguments to scan_req_key(). Platforms: HP-UX * Fixes a problem with the bmerge function that could fail to return status -96 if an invlaid chain link was encounted and cause a hang of the eloqdb6 process. * Implemented BEXP_V2 bimport/bmerge file format. Patch PE70-0501070 ------------------ Platforms: All * Fixed a small memory leak (#2582). Platforms: HP-UX * Fixed a problem with the currently undocument bulk import function. - The list of free records might get corrupted (#2586). This corruption could result in subsequent internal failure with a DBPUT call. A message like below is written to the server log: Assertion failed: *flag_ptr == FixRec_DELETED - Set names with 16 characters were not recognized (#2588). ** REVISION 02 ** Platforms: Windows * The eloqdb6.exe included with the initial revision of patch PE70-0501070 had an invalid reference to a debug library (MSVCRTD.dll). This DLL is normally not present so eloqdb6.exe could fail to start. This was caused by a build problem and was corrected by re-releasing the Windows patch. Patch PE70-0412282 ------------------ Platforms: All * Add support for new license option. * DBLOCK could return an unexpected status code (#2563). When using DBLOCK mode 6/16 to establish a database or data set lock DBLOCK could return status code 25 instead of the expected status codes 20/22/23. This could affect applications that don't expect a status code 25 from DBLOCK (such as QUERY3K). Patch PE70-0412171 ------------------ Platforms: HP-UX, Linux * The eloqdb6 database server was modified to improve performance when a large number of mostly inactive sessions are connected along with a few busy connections. When shared memory communication was enabled (EnableIPC config item set to non-zero) a large number of connected but mostly inactive sessions in combination with a few very busy sessions could cause a performance problem. A high CPU usage of the main eloqdb6 process but a low throughput could be observed. The performance impact was caused by the overhead to poll the shared memory for pending client side requests. With a few very active and a large number of inactive connections this overhead could get unproportial high and substentially impact performance. This patch implements a more effective way to check for pending client side requests and provides a protocol extenstion that is used with EnableIPC=2 to further improve performance. When EnableIPC=1 is configured or a client library version before PE70-0412170 is installed this improvement should substentially reduce the overhead. When the EnableIPC=2 configueration option is enabled and the corresponding client library patch PE70-0412170 is installed, the performance impact of connected but inactive sessions should be neglegtible. Patch PE70-0412150 ------------------ Platforms: HP-UX * The eloqdb6 database server was modified to remove the 2 GB file size limitation for dbstore/dbrestore (#2547). * The eloqdb6 database server was modified to remove an extra read access with forward logging (#2497). Patch PE70-0411291 ------------------ Platforms: ALL * The eloqdb6 process could abort with an internal failure. A message like below was written to the server log (#2542): Assertion failed: Node_GetItemPageReadOnly() failed: file is corrupt panic: Aborting on internal failure, file nodeilist.c, line 1914 This problem was caused by releasing allocated memory while it was still accessed when a data set was grown by more than 400 MB since it was first opened. This could corrupt some cache memory and eventually lead to an internal failure. This was a side effect of the changes in patch PE70-0411160 to improve performance on large data sets. * The eloqdb6 process could abort with an internal failure if a "dbctl backup stop" was executed concurrently. A message like below was written to the server log (#2540): server panic: Fatal problem detected Assertion failed: redir_mode == Vol_REDIR_ON The eloqdb6 server was modified to serialize concurrent backup start/stop requests. * The eloqdb6 database server was enhanced to collect process information submitted from the client library. The process information is then output in the session status of the server web status and allows to identify the application. This function requires the use of a newer database client library as provided with patch PE70-0411290 or newer. Notes / Related patches: - Installation of dblogreset patch PE70-0408311 (or newer) and dbrecover patch PE70-0408312 (or newer) is required with this version of eloqdb6. - Process information is only available if database client library patch PE70-0411290 (or newer) is installed. Patch PE70-0411191 ------------------ Platforms: ALL * The eloqdb6 process could abort with an internal failure during restructuring. A message message like below was written to the server log (#2499): Assertion failed: meta->ulist_cache_used <= (int)node->node.ulist.num_pages server panic: Aborting on internal failure, file volfrec.c, line 3291 The previous change in patch PE70-0411160 to improve performance on large data sets had a side effect that could cause the server to abort if a data set was restrutured and occupied less disk space than previously. The performance enhancement adds cache that accelerates access to the list of blocks used by a data set. During restructuring this cache memory was not truncated correctly in case the new data set used less block list entries than before. Notes / Related patches: - Installation of dblogreset patch PE70-0408311 (or newer) and dbrecover patch PE70-0408312 (or newer) is required with this version of eloqdb6. Patch PE70-0411160 ------------------ Platforms: ALL * A race condition was fixed on Windows that could result in a hang of the eloqdb6 service (#2518). The problem was related to the I/O completion notification and could affect Windows systems with multiple CPUs (or HT enabled CPUs, such as the Intel Xeon). It could also happen if more than 63 concurrent I/O operations were pending. * A rare race condition was fixed in the buffer management that could cause a server abort when using sync mode (#2521). When using the sync mode (with a nonzero SyncerJournalFlushInterval) the eloqdb6 process could abort with an internal failure as below in case a buffer is currently waiting on an I/O completion. assertion failed: !(bhp->flags & BUF_IO_IN_PROGRESS) file mpool.c, line 1501 * The server was modified to ensure a message is always written to the server log file if the server closes a connection due to a protocol problem or resource shortage (#2524). The log messages were also modified to include the internal thread id to allow easier diagnose what connection was related to a message. * The eloqdb6 was modified to include patch level information in the server log file, the output from the -version command line option and in the http status. * A corner case was fixed that could result in returning the status -700:-1 on dbopen when a memory allocation failed during dbopen. This was changed to return the expected status of -806:12 (where 12 specifies a system specific errno value for ENOMEM). * Performance on large data sets was improved (#2499). * The swap space that was reserved (but unused) by eloqdb6 on HP-UX was reduced by re-ordering the initialization. This specifically affects HP-UX on Itanium that uses a larger stack area for the internal threads. Notes / Related patches: - Installation of dblogreset patch PE70-0408311 (or newer) and dbrecover patch PE70-0408312 (or newer) is required with this version of eloqdb6. Patch PE70-0409140 ------------------ Platforms: ALL * A DBUPDATE on a detail set could fail to update the user accessible part the record data when changing a search item. While the user accessible part of the record data was not updated the chain links were updated possibly causing a subsequent status -18 (broken chain) or -96 (currupt chain pointer). This was caused by a side effect of the changes in beta patch PE70-0407280. This could happen when using DBUPDATE mode 2 (or DBUPDATE mode 1 with the CIUPDATE flag from the image3k library) and updating a detail record while changing a search or sort item. eloqdb6 patch levels before PE70-0407280 are not affected. Notes / Related patches: - This patch should immediately be installed to replace beta patches PE70-0407280 (beta) or PE70-0408310 (beta). - Please review the enhancements in PE70-0407280 below. - To retain the previous SyncMode behavior the SyncerJournalFlushInterval configuration item must be added to the eloqdb6 configuration file with a value of zero. In the section [config] please add a line like below: [config] SyncerJournalFlushInterval = 0 - Installation of dblogreset patch PE70-0408311 (or newer) and dbrecover patch PE70-0408312 (or newer) is required with this version of eloqdb6. Patch PE70-0408310 ------------------ Platforms: ALL * A rare race condition was fixed that could result in the wrong status code returned to the application (#2373). In case a database call that modified the database (eg. DBPUT) failed and the internal rollback operation became blocked the status code could be corrupted by a concurrent thread. A wrong status code would be returned to the application. * A rare race condition was fixed that could cause a DBPUT call on a detail set to fail with status code 43 ("duplicate key value"). Due to an internal locking problem there was a small window of opportunity where multiple threads could attempt to concurrently add the same automatic master entry (#2458). * The eloqdb6 server could exceed the valid range of the idle thread semaphore that is used internally to signal outstanding events (#2441). The semaphore value overflow could only happen in a rare case when the idle thread did not block for some time due to a large number of outstanding disk i/o completion (TIO) events. In case the idle semaphore overflowed some database sessions could terminate with status -700/-6 when shared memory communication was configured (EnableIPC set to 1 or 2). The server log had a message like below semop(UP): Numerical result out of range (errno 34) The internal event processing was changed to avoid overflowing the valid semaphore value range. * The eloqdb6 crash recovery was improved when the volume file was truncated (#2462, #2467, #2468). In case of a system crash a recent increase of a data volume might be lost. Increasing the data volume is now recorded in the transaction journal and volume size is recovered during crash recovery. * The eloqdb6 crash recovery was improved to recover the unique record id number that is used with indexes on secondary keys (#2466). Patch PE70-0407280 ------------------ Platforms: ALL * The amount of data that was written to the forward log was reduced substantially (#2398). This was done by using a different strategy to log changes to linked records caused by a PUT or DELETE operation. * A new SyncerJournalFlushInterval configuration item was added that may be used to substantially improve performance in "sync" write mode (#2400). Previously, when sync write mode was used (SyncMode=1), the transaction journal was flushed to disk for every commit operation (either explicitly when using transactions or implicitly for each database call). In case of a system crash only the last committed transactions could potentially be lost and the volume integrity is typically maintained. The new SyncerJournalFlushInterval configuration item adds the option to specify the time (in msec) after which changes to the transaction journal are pushed to disk. In case of a system crash any transactions committed in the period specified by SyncerJournalFlushInterval are lost but the volume integrity is typically maintained. The default value for the SyncerJournalFlushInterval configuration item is 500 ms (a half second). NOTE: This default modifies the behavior of the current SyncMode and in case of a system crash may result in losing any transactions committed within the last 500 ms. To retain the previous SyncMode behavior the SyncerJournalFlushInterval configuration item must be added to the eloqdb6 configuration file with a value of zero. In the section [config] please add a line like below: [config] SyncerJournalFlushInterval = 0 NOTE: Specifying a low SyncerJournalFlushInterval value may have a detrimental effect on performance. This will result in additional load on the I/O subsystem and also the operating system may block concurrent access to the lock volume(s) during a sync operation. * The dbctl syncmode command was enhanced to allow setting and returning the current status of the sync mode configuration item and the current value of SyncerJournalFlushInterval. usage: dbctl syncmode {ON [msec]|OFF|STATUS} > dbctl syncmode status Returns status of sync mode. In case the sync mode is used and the SyncerJournalFlushInterval is nonzero it also returns the current value of SyncerJournalFlushInterval in msec. > dbctl syncmode on 500 Enables sync write mode and defines an SyncerJournalFlushInterval interval of 500 msec. Patch PE70-0406240 ------------------ Platforms: ALL * The eloqdb6 process could abort with an internal failure when a client was aborted. A message message like below was written to the log (#2366): tcp_send: send failed: writecount -1, [9] Bad file descriptor tcp_async_mode: fcntl(F_GETFL): Bad file descriptor (errno 9) server panic: Fatal problem detected in close_connection Assertion failed: poll_ofs > 2 && poll_ofs < poll_set_cnt server panic: Aborting on internal failure, file event.c, line 489 This could only happen when patch PE70-0406030 was installed and the connection used the TCP communication method (connection from a remote system or EnableIPC=0) and the database call was blocked during execution (for example a DBLOCK with a wait option). In this case a cleanup operation could be executed twice which caused a problem with the changes in PE70-0406030. * The dbctl syncmode command was enhanced to return the current status of the sync mode configuration item (#2371). usage: dbctl syncmode {ON|OFF|STATUS} > dbctl syncmode status Returns status of sync mode. * Changing the server sync mode using dbctl now updates the status in the HTTP status. Notes / Related patches: - Installation of dblogreset patch PE70-0401161 (or newer) and dbrecover patch PE70-0401162 (or newer) is recommended. Patch PE70-0406030 ------------------ Platforms: ALL * The eloqdb6 process could abort with an internal failure when searching an index with multiple string segments (#2219). An internal error message like below was written to the log file: Assertion failed: len_a >= 0 && len_b >= 0 server panic: Aborting on internal failure, file nls.c, line 366 This was caused by a defect in the Eloquence DBFIND modes 6/7 that are used in the Eloquence image3k library. * Performing a DBFIND on a master set that was not created could result in an abort of the eloqdb6 process with an error message like below (#2263): Assertion failed: node server panic: Aborting on internal failure, file nodecore.c, line 1273 eloqdb6 has been fixed to return a status 80 in case an index is not created. * The http status could cause a temporary hang of the eloqdb6 server process in case of a network communication problem with the browser (#2315). This was observed with conections through a VPN gateway. * A new dbctl function was added to allow changing of sync mode without restarting the server process. usage: dbctl syncmode {ON|OFF} > dbctl -udba syncmode on Enables sync mode. Equivalent to SyncMode=1 > dbctl -udba syncmode off Disables sync mode. Equivalent to SyncMode=0 Platforms: Windows * Due to a build problem the number of concurrent connections on the Windows platfrom might be limited to 62. Additional connections would hang during establishing the connection. Platforms: HP-UX * An improper timeout value could cause a performance problem when using TCP/IP communication and few connections are active. Platforms: Linux * eloqdb6 was not compatible with the glibc2.3 library due to directly accessing the errno variable. This resulted in a warning on SUSE Linux 9.0 and a start failure with SUSE Linux 9.1. * The -s command line argument to specify the service or port nunber from the command line was corrupted during startup on Linux (#2317). This caused a failure if the ELOQDB6_SERVICE[] variable was specified in the startup configuration. Notes / Related patches: - When using this patch, installation of dblogreset patch PE70-0401161 (or newer) and dbrecover patch PE70-0401162 (or newer) is recommended. Patch PE70-0401160 ------------------ Platforms: ALL * A server abort due to an internal error during a dbrestore operation could result in disk space not being released and stale node entries (#2187,#2192). When the eloqdb6 process aborted during a dbrestore operation in some cases the disk space used by the partial dbrestore was not released. In addition stale node entries could remain that could result in warnings when running dbfsck. The eloqdb6 database server has been modified to recover disk space from an aborted dbrestore operation and to delete all non-committed node entries during restart. * In case of a server abort due to an internal error forward logging is disabled and a warning mesage is written to the log file (#2198). In case the eloqdb6 process aborted and performs a crash recovery the forward log files may not fully describe the volume changes. Consequently, forward logging is disabled. To recover, a backup must be performed and forward logging needs to be re-enabled manually. * The eloqdb6 process could abort with an internal error message like below (#2190): Assertion failed: cluster panic: Aborting on internal failure, file volpool.c, line 3182 * Insuffient disk space during a dbrestore operation could cause the eloqdb6 process to abort with an internal failure (#2189,#2192): Pool_Commit() failed on Node_AddItemToFlist() Assertion failed: Tlog_Commit() failed on Tlog_Action() panic: Aborting on internal failure, file voltxn.c, line 749 Assertion failed: !FPool_ValidAnchor(&meta->do_log_anchor) panic: Aborting on internal failure, file voltxn.c, line 734 * A server recovery after a previous server abort due to an internal error could result in an internal failure message like below (#1169): Assertion failed: f_bhp->id.node_id == node_id panic: Aborting on internal failure, file mpool.c, line 408 This was caused by a defect that could release transaction journal pages too early. In rare cases this could corrupt the transaction journal and could potentially lead to data corruption. When using this patch, installation of dblogreset patch PE70-0401161 (or newer) and dbrecover patch PE70-0307162 (or newer) is recommended. Patch PE70-0401130 ------------------ Platforms: ALL * The eloqdb6 process could abort with an internal failure when searching an index with multiple key segments of different types when providing a search key that covered multiple segments (#2175). An internal error message like below was written to the log file: Assertion failed: idx_op->seg_len == idx_op->val_sz server panic: Aborting on internal failure, file find.c, line 933 This was caused by a defect in the Eloquence DBFIND modes 6/7 that are used in the Eloquence image3k library. * A problem with collating sequences and the Eloquence DBFIND modes 6/7 was fixed (#2175). * The eloqdb6 server process could abort with a message like below when a dbrestore operation failed due to an invalid archive (#2181): server panic: Fatal problem detected in bf_flags Assertion failed: !(bhp->flags & BUF_NEWPAGE) server panic: Aborting on internal failure, file mpool.c, line 2095 The eloqdb6 code has been fixed to recover correctly from a dbrestore failure in this case. * A rare race condition has been fixed that could result in corrupted node meta data during a dbrestore operation (#2182). * The http status has been modified to use relative links (#2183). Previous versions created absolute links (including host ip address) that could cause a problem on system with multiple network addresses and in NAT environments. * A server abort during a dbrestore operation could result in a circular page link in the server node directory (#2186). The server recovery procedure has been modified to avoid and fix circular page links in the node directory. In case a circular page link in the server node directory is recognized an informational message like below is output: N1: Node_CommitNodePage(page=0xe396): page is already present - ignored * The eloqdb6 server could abort with an internal failure during if the available disk space (as defined by the volume files) was exausted in a dbrestore operation. This could result in volume corruption (#2187). * An ongoing dbrestore operation could block concurrent requests or cause large delays (#2188). Patch PE70-0312080 ------------------ Platforms: ALL * DBLOCK could ignore a conflicting lock. In rare cases DBLOCK could ignore a granted lock when granting a previously blocked lock request. This could cause status -12 on a subsequent database write (#2167). * DBGET mode 1 returned status 12 if no current record existed. It has been modified to return status 17 (#2168). * DBFIND mode 1 on a search item (chained access) could result in a memory leak if the corresponding master set was not created. * Database restructuring could fail to initialize new items (#2171) * The status return for DBFIND mode 1 on index items was inconsistent with the documentation. Status element 8 returned the first matching record number and status element 10 the last matching record number. The documented behavior is to return the first matching record number in status element 10 and the last matching record number in status element 8. DBFIND mode 1 on index items now follows the documented behavior. * The Eloquence DBFIND modes 6 and 7 may be used with detail search items. This enables the use of TurboIMAGE btree modes with the image3k library ("superchains"). * The data set /INDEXED flag is now returned to the image3k library (#1358). This enables the TurboIMAGE DBINFO modes 113 and 209 to return information on master sets with "attached" indexes. It also enables the use of DBFIND mode 1 (if the btreemode1 flag is set) to locate records though the master set index ("superchains"). Please note that index access (other than DBFIND mode 1) is always enabled with Eloquence, independendly if the /INDEXED flag is set for a master set. Patch PE70-0311180 ------------------ Platforms: ALL * The maximum number of concurrent connections has been increased to 4000 on the HP-UX platform (PA-RISC and Itanium). Windows and Linux are still limited to 1000 concurrent connections. * The performance for larger configurations has been improved. - The eloqdb6 process has been changed so that a large number of configured threads does not affect the performance. - The performance of the eloqdb6 process has been improved when a large number of (mostly) idle connections is used. - The number of system calls have been reduced. * The Eloquence DBFIND modes 6 and 7 may be used with master sets. The previous implementation was limited to detail sets. To use the image3k DBFIND with master sets, installation of patch PE70-0311181 (or newer) is required. Patch PE70-0310180 ------------------ Platforms: ALL * A deadlock situation could cause the eloqdb6 process to abort with a a panic message (#2136) D0: Assertion failed: tcp->marker != current_marker D0: server panic: Aborting on internal failure, file thread.c, line 362 This panic is caused by a deadlock situation that is not detected immediately and only becomes apparent after another process executes a DBUNLOCK. However at this time, the deadlock resolving code is bypassed instead of flagging one of the processes involved in the deadlock. The resolution is to notify one of the threads involved of the deadlock and return a db status to the application. Patch PE70-0310082 ------------------ Platforms: ALL * The eloqdb6 process could abort with an internal failure when on-line backup is initiated (dbctl backup start) (#2128) eloqdb6 could abort with the message below: D0: server panic: Fatal problem detected in Fwr_Setup D0: Assertion failed: rc == 0 D0: server panic: Aborting on internal failure, file volfwr.c, line 541 When forward logging is used but currently disabled due to an error (eg. disk full, permission problem) and is re-enabled subsequently (dbctl forwardlog enable) writing to the log file is delayed until the server is restarted or an on-line backup is performed. Initiating on-line backup resulted in an abort of the server process because the previous failure status was not reset properly. Patch PE70-0309300 ------------------ Platforms: ALL * The eloqdb6 server process could abort with a SIGSEGV when forward logging is used and a log file reached its maximum size (#2126). * The maximum number of concurrent connections has been increased from 1000 to 2000 on the HP-UX platform (#2125). Patch PE70-0307161 ------------------ Platforms: All * DBGET mode 6 after an image3k DBFIND 1/21 on an index could return status 14 (#2046) A DBGET mode 6 after an image3k DBFIND mode 1/21 on an index should retrieve the last matching record in index order. DBGET mode 6 failed with status 14 when the record was the highest key in the index. * The eloqdb6 process could abort with an internal failure or cause database corruption (#1966, #2030) Assertion failed: f_bhp->id.node_id == node_id Restructuring a database with dbutil could cause the server process to abort with an internal error. In rare cases it could cause corruption of the database catalog during crash recovery. * Restructuring a database could cause an internal failure during forward recovery (#2051) Restructuring a database might cause an internal failure in the dbrecover utility when applying the forward log files. The dbrecover utility could abort with an error message like below: panic: Fatal problem detected in FixRec_FinalCommitUpdatePut Assertion failed: *flag_ptr == FixRec_USED || *flag_ptr == FixRec_DELETED Patch PE70-0305270 ------------------ Platforms: All * The eloqdb6 process could abort with an internal failure (#1973) eloqdb6 panics with the message: panic: current_task->waiting_for == NULL (thread.c, line ~413) This was caused by a race condition which could happen when a number of application processes are terminated at the same time while accessing the database. The eloqdb6 server has been modified to solve this problem. The problem can be triggered in the following scenario: - Application thread A holds a lock. It is in the process of releasing database locks. - Application thread B is waiting for the lock held by thread A. - Application thread C is wating for the lock held by thread A (and thread B). Thread C gets interrupted (application process has been killed) during the wait. - There is a situation where thread A might become runnable before thread C during the same scheduler quantum and might unexpectedly update a scheduler variable used in the deadlock detection code ("waiting_for" which is used to track dependencies among application threads). * Connecting to the database server could fail with status -700:-6 when EnableIPC=2 is used on HP-UX (#1982). A message like below is printed: P0: Unable to down semaphore P0: semop(DOWN): Identifier removed (errno 36) P0: Failure during wait on server response This was caused by a missing initialization when a shared memory segment was re-used by a subsequent process. Due to a race condition, a new connection could become disconnected by the server immediately. * Recovery in case of abnormal server termination has been improved. An incomplete transaction journal could corrupt the database volume(s) for btree and catalog node types (#1966). * In case of inconsistent meta information (such as number of records) the eloqdb6 process could abort with an internal error message like below (#1941). D0: server panic: Fatal problem detected D0: Assertion failed: meta->num_records == 1 D0: server panic: Aborting on internal failure, file volfrec.c, line 3191 This situation is is now handled more gracefully and does no longer cause a server abort. * When forward logging is enabled, the eloqdb6 process will refuse to override an existing file (#1952). An existing file is considered a logging failure and will either result in diabling forward logging or an internal abort (depending on the configuration). When using forward recovery and restoring the volume files from the backup and errorneously starting the eloqdb6 process could could overwrite forward log segments and cause potential data loss. The eloqdb6 has been modified to refuse overwriting existing files. * When forward logging is anbled and the eloqdb6 process is killed using kill -9 (eg. due to patch PE70-0305090 is not installed), it might fail to flag the volume file(s) as consistent. This might cause the server to refuse to start (#1985). A message like below is printed: failed to open volume: volume #1 has inconsistent generation count 8 (should be 9) volume.c line 828 The eloqdb6 server code has been modified to handle this this situation correctly. Patch PE70-0304041 ------------------ Platforms: All * Changing the data set structure with dbutil could result in an inconsistent internal highest record number. As a consequence, new records which are added subsequently could become inaccessible. This could cause database status -96 on DBPUT or status 17 and status 18 on DBGET, depending on the use of the particular data set (#1880). Please note: Installing this patch will prevent this problem to occur in the future. However, it will not correct your existing database environment(s). In case you used the Eloquence B.07.00 dbutil program to modify the database structure we recommend to check and correct your database environment(s) with the dbfsck utility: 1) Shut down the eloqdb6 database server instance(s). 2) Write access to the volume files is required. On HP-UX and Linux logon as root or the user configured in the server configuration file. 3) Check and correct your database environment(s) with dbfsck: dbfsck -way This will automatically fix any inconsistencies caused by dbutil data set restructuring. You may need to specify the -c /path/to/configfile command line option in case multiple database server instances are used. Installation: ------------- UNIX: In order to install this patch, you need to unpack it with gzip. Gzip is included with HP-UX and Linux. Installation requires root privileges. cd /opt/eloquence6 gzip -dc /path/to/PE70-0512060-hpux.tar.gz | tar xf - Files: bin/eloqdb6 share/doc/PE70-0512060-README Windows XP/2000/NT: This patch should *only* be installed if you previously installed the Eloquence server components on your system. Download the PE70-0512060-win32.zip file and unpack its contents with WinZip or PKUNZIP. Installation requires administrative capabilities. PLEASE MAKE SURE THE eloqdb6 SERVICE has been STOPPED previously (in the Service Control Manager or with NET STOP eloqdb6). Please copy the eloqdb6.exe file into the WINDOWS SYSTEM DIRECTORY (for example C:\Windows\System32). Please copy the PE70-0512060-README.txt file into the share\doc subdirectory of your Eloquence installation (for example C:\Programs\Eloquence\share\doc). Files: eloqdb6.exe PE70-0512060-README.txt