|
This document provides an overview on B.08.30 database changes.
For details please refer to the documents below:
The B.08.30 release comes with a number of substantial database enhancements.
-
Database server performance improvements.
The Eloquence database was enhanced to further improve performance and
efficiency on contemporary hardware.
The Eloquence database is expected to deliver noticeably improved
throughput with a lower CPU utilization for a variety of workloads.
-
The database server supports
speculative read-ahead and asynchronous I/O
to improve disk I/O rates and concurrency.
-
Improved FTS scalability and new FTS dictionary pools.
-
Database security improvements.
Most visibly, a new hashing scheme is used to secure database passwords.
-
Support for compressed forward log files.
-
IPv6 support. The Eloquence database and
utilities were changed to support the IPv6 standard.
-
On the HP-UX and Linux platform support for local sockets was
introduced as an additional option to TCP sockets to connect the
database server process on the local system.
-
Thread safe database client libraries.
-
Eloquence does no longer use static linking internally for its
utilities but uses the dynamic client libraries.
-
The iamge3k library was improved to
support an additional "Big Endian" byte order on "Little Endian"
platforms. This will simplify migration of Cobol based IMAGE
applications to Linux or Windows.
-
Fixed a number of issues with more than 2 billion records in a table.
A number of database utilities had minor issues with database
tables with more than 2 billion records. query3k simply failed
and the database server could abort when exceeding 4 billion
records per table.
The eloqdb database server performance and scalability was
further improved.
For a variety of work loads the Eloquence B.08.30 database
server can noticeably improve throughput using less CPU cycles.
An optimization of the internal locking strategy results in reduced
overhead, improved performance and concurrency. Most work loads are
expected to see a noticeable improvement in throughput with lower CPU
utilization.
The internal buffer cache management was changed to perform efficiently
with large buffer cache configurations. As a result, the size of the
buffer cache should only have a negligible effect on the management
overhead and allows to configure larger buffer cache sizes.
More efficient skipping over large number of deleted records. Skipping
a very large number of deleted records could impact performance with
previous eloqdb versions. While may still result in additional disk I/O
it should be of little consequence otherwise.
The database adds speculative read-ahead to improve disk I/O rates
and concurrency. The database server may read-ahead data content
that is likely to be used, depending on application access patterns.
This speculative disk I/O is performed asynchronously in parallel
or while the application is processing the previous data.
For example, reading a record in chained order may trigger disk I/O
on subsequent records.
The replication slave disk read is now executed in parallel to
improve replication performance bounded by disk I/O.
The AIOThreads configuration item
must be set to enable use of the asynchronous disk reads.
On HP-UX IA64 the kernel AIO functionality is used to handle disk
read-ahead with little overhead, requiring only few internal I/O threads.
On other platforms, dedicated I/O threads are used to perform disk
read-ahead asynchronously and the number of
I/O threads defines the number of concurrent async disk read operations.
A number of FTS indexing scalability and performance enhancements
were implemented with B.08.30, specifically improving performance for
large environments.
Scanning the dictionary index for a keyword range or wildcard
uses a more efficient approach to iterate the index. Instead of
obtaining keyword entries sequentially, an entire index page is
fetched and internally processed at once.
This reduces the overhead and locking contention.
Scanning the dictionary index was further optimized to avoid
internal locking on keywords that have no reference in the
searched context.
Caching some FTS internal results makes access to FTS references
substantially more efficient and reduces contention of the
buffer cache.
Handling of FTS keywords with a large number of references was
improved to be more I/O efficient. When creating new FTS indexes
(or re-creating existing ones) an additional storage pool is
created to maintain keywords with a lage number of references.
Please be aware that this feature this is not backwards
compatible and requires rebuilding FTS indexes when moving to
an earlier Eloquence release.
FTS was enhanced to support
multiple keyword pools. Before B.08.30
a single dictionary index was used to maintain any keyword for
the entire database. This could result in a large dictionary
index and could affect performance in some range searches where
a large number of keywords exist that are not relevant in the
current search context. Using a separate dictionary index makes
those searches instantaneous and allows to maintain similar
keywords (for example, date ranges) in a separate dictionary.
Please be aware that this feature this is not backwards
compatible and requires rebuilding FTS indexes when moving to
an earlier Eloquence release.
For additional details please refer to the
FTS index changes section.
In previous versions, Eloquence used a secure hash to encode database
passwords. As of B.08.30 the standard PBKDF2 algorithm is used to hash
any passwords with improved security.
This is intended to provide additional protection against reverse
lookup of passwords if the password hash value might have
been exposed. Eloqdb password hashes are only visible to the dba user.
As a related change the database protocol was enhanced to handle
password salting and additional hashing schemes and to improve the
encryption of passwords in transit.
As Eloquence on HP-UX (and Linux) makes use of the openssl library
for crypto functionality, installation of openssl on HP-UX is now
required to use the new password hashing.
Password authentication will fail if the crypto functionality is not
available.
For additional details please refer to the
Password encoding changes section.
When enabled in the eloqdb configuration forward log files are
compressed. This will noticeably reduce the size of forward log
files in exchange for additional CPU utilization to compress the
data. It depends on the system CPU resources and the disk bandwidth
if this is beneficial to performance.
The forward log files may also be compressed afterwards with gzip
(for example, when changing forward log files).
The Eloquence utilities (as well as the fwutil library) handle
compressed forward log files transparently.
For additional details please refer to the
FwCompress configuration option
in the eloqdb configuration file.
Eloquence internally does no longer use static linking. Instead it
links the Eloquence database and support libraries dynamically.
This allows for minimizing updates if a problem in a library was
corrected that might also affect database utilities.
As a consequence, Eloquence utilities depend on accessing the
correct database libraries. An improperly set LD_LIBRARY_PATH
(or SHLIB_PATH on HP-UX) may impact the correct function.
Due to the new password hashing scheme, any application or utility
connecting to the database depends on the database client library
using the openssl crypto library.
While previously openssl was optional and only used with database
encryption it is now mandatory to use the new database password
handling.
The Eloquence database client libraries (eqdb, image3k and ftc)
support multiple threads.
Eloquence by default assumes a thread model that is easy and safe to
use and does not impose overhead.
Databases are not shared among threads and each thread is required to
open its databases.
With IMAGE database calls it is not sufficient to simply protect
against concurrent calls by multiple threads. Database calls usually
depend on a context that needs to be protected against concurrent
modification. Examples include the current record, current path,
chain, FTS results, database transactions or database locks.
More complex threading models are equally supported. Eloquence
allows managing the association of database context and threads.
A database context may be release and is then available for use by
another thread. This allows ownership of database contexts for the
duration of a transaction rather than individual calls.
For additional details please refer to the
image3k thread safe section.
|
