.
contact contact

B.08.30 / Release Notes / Database

Database

 
.
  This document provides an overview on B.08.30 database changes. For details please refer to the documents below:


Database enhancements overview

The B.08.30 release comes with a number of substantial database enhancements.



Database server scalability improvements

The eloqdb database server performance and scalability was further improved. For a variety of work loads the Eloquence B.08.30 database server can noticeably improve throughput using less CPU cycles.

An optimization of the internal locking strategy results in reduced overhead, improved performance and concurrency. Most work loads are expected to see a noticeable improvement in throughput with lower CPU utilization.

The internal buffer cache management was changed to perform efficiently with large buffer cache configurations. As a result, the size of the buffer cache should only have a negligible effect on the management overhead and allows to configure larger buffer cache sizes.

More efficient skipping over large number of deleted records. Skipping a very large number of deleted records could impact performance with previous eloqdb versions. While may still result in additional disk I/O it should be of little consequence otherwise.

Speculative read-ahead and asynchronous I/O

The database adds speculative read-ahead to improve disk I/O rates and concurrency. The database server may read-ahead data content that is likely to be used, depending on application access patterns. This speculative disk I/O is performed asynchronously in parallel or while the application is processing the previous data. For example, reading a record in chained order may trigger disk I/O on subsequent records.

The replication slave disk read is now executed in parallel to improve replication performance bounded by disk I/O.

The AIOThreads configuration item must be set to enable use of the asynchronous disk reads. On HP-UX IA64 the kernel AIO functionality is used to handle disk read-ahead with little overhead, requiring only few internal I/O threads. On other platforms, dedicated I/O threads are used to perform disk read-ahead asynchronously and the number of I/O threads defines the number of concurrent async disk read operations.

Improved FTS scalability and new FTS dictionary pools

A number of FTS indexing scalability and performance enhancements were implemented with B.08.30, specifically improving performance for large environments.

Scanning the dictionary index for a keyword range or wildcard uses a more efficient approach to iterate the index. Instead of obtaining keyword entries sequentially, an entire index page is fetched and internally processed at once. This reduces the overhead and locking contention.

Scanning the dictionary index was further optimized to avoid internal locking on keywords that have no reference in the searched context.

Caching some FTS internal results makes access to FTS references substantially more efficient and reduces contention of the buffer cache.

Handling of FTS keywords with a large number of references was improved to be more I/O efficient. When creating new FTS indexes (or re-creating existing ones) an additional storage pool is created to maintain keywords with a lage number of references. Please be aware that this feature this is not backwards compatible and requires rebuilding FTS indexes when moving to an earlier Eloquence release.

FTS was enhanced to support multiple keyword pools. Before B.08.30 a single dictionary index was used to maintain any keyword for the entire database. This could result in a large dictionary index and could affect performance in some range searches where a large number of keywords exist that are not relevant in the current search context. Using a separate dictionary index makes those searches instantaneous and allows to maintain similar keywords (for example, date ranges) in a separate dictionary. Please be aware that this feature this is not backwards compatible and requires rebuilding FTS indexes when moving to an earlier Eloquence release.

For additional details please refer to the FTS index changes section.

Improved database password security

In previous versions, Eloquence used a secure hash to encode database passwords. As of B.08.30 the standard PBKDF2 algorithm is used to hash any passwords with improved security.

This is intended to provide additional protection against reverse lookup of passwords if the password hash value might have been exposed. Eloqdb password hashes are only visible to the dba user. As a related change the database protocol was enhanced to handle password salting and additional hashing schemes and to improve the encryption of passwords in transit.

As Eloquence on HP-UX (and Linux) makes use of the openssl library for crypto functionality, installation of openssl on HP-UX is now required to use the new password hashing. Password authentication will fail if the crypto functionality is not available.

For additional details please refer to the Password encoding changes section.

Compressed forward log files

When enabled in the eloqdb configuration forward log files are compressed. This will noticeably reduce the size of forward log files in exchange for additional CPU utilization to compress the data. It depends on the system CPU resources and the disk bandwidth if this is beneficial to performance.

The forward log files may also be compressed afterwards with gzip (for example, when changing forward log files). The Eloquence utilities (as well as the fwutil library) handle compressed forward log files transparently.

For additional details please refer to the FwCompress configuration option in the eloqdb configuration file.

Dynamic libraries

Eloquence internally does no longer use static linking. Instead it links the Eloquence database and support libraries dynamically. This allows for minimizing updates if a problem in a library was corrected that might also affect database utilities.

As a consequence, Eloquence utilities depend on accessing the correct database libraries. An improperly set LD_LIBRARY_PATH (or SHLIB_PATH on HP-UX) may impact the correct function.

Due to the new password hashing scheme, any application or utility connecting to the database depends on the database client library using the openssl crypto library. While previously openssl was optional and only used with database encryption it is now mandatory to use the new database password handling.

Thread safe database client libraries

The Eloquence database client libraries (eqdb, image3k and ftc) support multiple threads.

Eloquence by default assumes a thread model that is easy and safe to use and does not impose overhead. Databases are not shared among threads and each thread is required to open its databases.

With IMAGE database calls it is not sufficient to simply protect against concurrent calls by multiple threads. Database calls usually depend on a context that needs to be protected against concurrent modification. Examples include the current record, current path, chain, FTS results, database transactions or database locks.

More complex threading models are equally supported. Eloquence allows managing the association of database context and threads. A database context may be release and is then available for use by another thread. This allows ownership of database contexts for the duration of a transaction rather than individual calls.

For additional details please refer to the image3k thread safe section.


 
 
 
  Privacy | Webmaster | Terms of use | Impressum Revision: 2019-01-16  
  Copyright © 1995-2024 Marxmeier Software AG