|
Revision: B.07.10.06
Document: 2008-03-19
Contents
The fwutil library enables access to the Eloquence database
forward log file that holds the archived committed transactions.
Decoding the fwlog file format is performed by the fwutil
library and a callback to a user provided function is performed
for IMAGE like operations which simplifies programming and
isolates the utility from format changes.
The fwutil library works incrementally. Transactions from a
starting point are processed. The current progress is recorded
in a status file and may be used to continue subsequently.
Once the end of the fw log file(s) is reached, the library
may either return or wait for additional changes which are
then processed.
The fwutil library makes use of the audit information in the
forward log file and the database server must be configured
to record audit information.
The fwutil library is available as patch PE71-0803190 (or later) in the download area
of the Eloquence web site. Please make sure to select the fwutil
patch that matches your installed eloqdb6 server version because
the fwutil library must be compatible with the internal format
of the forward log files written by the eloqdb6 server.
DBBEGIN/DBEND and DBMEMO
Beginning with version B.07.10.06 of the fwutil library,
it does not only provide information on logged DBPUT, DBUPDATE and DBDELETE
invocations, but also provides access to DBBEGIN/DBEND and DBMEMO entries
in the forward logfiles, if present.
Note that logging of such DBBEGIN/DBEND and DBMEMO entries
depends on the use of an image3k library patch level of PE71-0710030 or later
and on certain flags in the HP3K_API_COMPAT database property being set. For
processing such forward log entries with fwaudit you should also be using the
fwaudit patch PE71-0710031 or later.
Two different methods are available for accessing the
optional DBBEGIN/DBEND and DBMEMO entries in the forward logs:
- a) Registering a separate callback function with Fwu_set_memo_callback().
- This approach is similar to using the callback function that is invokved for
each DBPUT, DBUPDATE, and DBDELETE entry in the forward logs. It just delivers
the DBBEGIN/DBEND and DBMEMO information to a separate callback function with
slightly different parameters.
- b) Calling Fwu_get_memo() inside the regular Fwu_process() callback function.
- This approach is similar to obtaining session context info with Fwu_get_session_id()
while processing a DBPUT, DBUPDATE or DBDELETE event, and delivers the DBBEGIN/DBEND
and or DBMEMO context for the database update entry being processed.
Find details on the above approaches in the API descriptions below.
The fwutil.h include file provides the function prototypes
and types. It must be included in any file that makes use
of the fwutil library.
#include "fwutil.h"
The fwutil.h file is installed in the Eloquence include
subdirectory.
Output the fwutil library version (to stdout)
void Fwu_version(void);
Output the fwutil status file (to stdout)
int Fwu_status(const char *status_file, int verbose);
- Arguments:
- status_file specifies the name of the status file
- if verbose is non-zero then the full content is printed
- Returns:
- Completion status, 0 for success, -1 on failure.
If called with a non-zero argument specifies that Fwu_process()
should return once the end of the fwlog file(s) was reached.
void Fwu_synconly(int mode);
- Arguments:
- 0 = wait for new transactions after reaching the end
of the fw log file(s).
This is the default.
1 = return after reaching the end of the fw log file(s)
If called with a non-zero argument specifies that Fwu_process()
should output additional progress messages (to stderr).
void Fwu_verbose(int mode);
- Arguments:
- 0 = Don't print progress messages. This is the default
1 = Output additional debug messages (to stderr)
This is similar to setting the FWUTIL_DEBUG_FLAGS environment
variable to "*1".
Process archived transactions.
int Fwu_process(Fwu_callback_t cb, const char *cfg_file,
const char *status_file,
const char *fwr_start);
- Arguments:
- cb specifies a pointer to a user provided callback function.
This function is called for image-like changes(see below).
- cfg_file specifies the name of the server configuration file
(eloqdb6.cfg). This is used to locate the fwlog files.
- status_file specifies the name of the fwutil status file.
- The fwr_start argument may be used to specify a starting point
when processing any archived transactions. If set, the status file
is initialized. NULL should be specified to use the status file.
- Returns:
- Completion status, 0 for success, -1 on failure.
On failure, a message is printed to stderr.
Discussion:
By default, the fwutil library processes all enqueued changes and
then closely follows any on-going changes on the server.
If the Fwu_synconly() function was called with a non-zero argument,
Fwu_process() returns once all enqueued changes are processed.
The fwutil library uses a local file to maintain its most recent
successfully executed call. The status file may be specified with the
status_file argument and defaults to fwutil.stat in current directory.
The Fwu_status() function outputs the content of the status file.
If Fwu_verbose() was called with a non-zero argument all details are
output, otherwise only information on the most recently completed
call is output.
The fwr_start argument is used to specify the forward log generation
that the fwutil library should be starting with. It is required on the
first use and may also be used to specify a different starting point.
If not specified, the position is obtained from the status file.
The fwr_start argument uses a syntax of "gen[-seg[.action]]" where
gen specifies the generation of the forward log file, the optional
seg specifies a file segment and the optional action specifies a
specific action in this file. Typically only the generation is used.
This defines the prototype of a user-defined callback function.
typedef int (*Fwu_callback_t)(int db_node_id, const char *db_name,
const char *set_name, time_t timestamp,
enum Fwu_OP op, unsigned int recno, int recsz,
const void *bi_rec, const void *ai_rec);
- Arguments:
- db_node_id specifies the internal id of the database. This value
is unique for each database.
- db_name specifies the database name.
- set_name specifies the data set name.
- timestamp specifies the time when the transaction was recorded
(seconds since 1.1.1970 UTC).
- op specifies the IMAGE call (Fwu_OP_UPDATE/Fwu_OP_PUT/Fwu_OP_DELETE).
- recno specifies the record number.
- recsz specifies the record size in bytes (as specified in schema).
- bi_rec provides a pointer to a buffer holding the previous record
data ("before image"). This is present for an UPDATE and DELETE call.
Otherwise a NULL pointer is passed.
- ai_rec provides a pointer to a buffer holding the updated record
data ("after image"). This is present for a PUT and UPDATE call.
Otherwise a NULL pointer is passed.
- Returns:
- Completion status, 0 for success, -1 on failure.
Register callback function for handling memo records.
void Fwu_set_memo_callback(Fwu_memo_callback_t cb);
- Arguments:
- cb specifies a pointer to a user provided callback function.
This function is called for memo records, if any (see below).
Discussion:
By default, Fwu_process() invokes the user supplied Fwu_callback_t()
function for each DBPUT, DBUPDATE and DBDELETE entry in the forward log. If registered
with this function, a separate user supplied Fwu_memo_callback_t() function is invoked
for each memo entry (like DBBEGIN, DBEND and DBMEMO) that is present in the logs.
Note:
This API requires version B.07.10.06 or later of the fwutil library.
This defines the prototype of a user-defined memo callback function.
typedef int (*Fwu_memo_callback_t)(time_t timestamp, int mode,
const void *data, int data_sz);
- Arguments:
- timestamp specifies the time when the transaction was recorded
(seconds since 1.1.1970 UTC).
- mode specifies the type of memo call (Fwu_MEMO_DBBEGIN, Fwu_MEMO_DBEND, Fwu_MEMO_DBMEMO).
- data specifies a pointer to the data in the memo record.
- data_sz specifies the data size in bytes (may be zero).
- Returns:
- Completion status, 0 for success, -1 on failure.
Note:
This API requires version B.07.10.06 or later of the fwutil library.
Retrieve schema information. Obtains next item definition
in schema order.
int Fwu_get_item(const char **i_name, int *i_type,
int *i_size, int *i_count, unsigned int *i_fmt);
- Arguments:
- i_name is set to a NUL terminated pointer to the item name.
- i_type is set to the native item type (X/U/B/I/K/E/P/Z).
- i_size is set to the item size (in bytes).
- i_count is set to the item count.
- i_fmt is set to the item format.
- Returns:
- Completion status, 0 if no schema information available, 1 on success.
- Note:
- The item format flags are used to indicate the role of an item as below:
- Bit 16 (0x10000) is set if the item is a search item.
- Bit 18 (0x40000) is set if the item is a unique key.
Currently, this indicates it is a master search item.
- Bit 19 (0x80000) is set if the item is a sort item.
Obtains the current session identifier. Each session has a unique
identifier starting with 1.
int Fwu_get_session_id(void);
- Returns:
- current session identifier or 0 if undefined
Obtains the number of data elements in the session record.
int Fwu_get_session_elements(void);
- Returns:
- number of elements in session record or 0 if no data present
Session data consists of multiple elements, for example:
session_data[0]=protocol{8}os{HP-UX}ip{127.0.0.1}user{joe}login{public}
session_data[1]=uid{221}pid{18416}pname{dbimport -vs sample.exp sample}
Retrieves a session data element by index. The index is zero-based.
The returned data element is a null-terminated string.
const char *Fwu_get_session_data(int element_idx);
- Argument:
- zero-based element index
- Returns:
- pointer to null-terminated session data or NULL if undefined
Example use:
int i, cnt = Fwu_get_session_elements();
for (i = 0; i < cnt; i++)
printf(" session_data[%d]=%s\n", i, Fwu_get_session_data(i));
Note:
The returned value is statically buffered. A circular array of
10 buffers is provided, so that after 10 invocations the first
returned value is overwritten.
Retrieves a session entry by key.
const char *Fwu_get_session_entry(const char *key);
- Argument:
- key value of session entry to retrieve
- Returns:
- pointer to null-terminated session entry or NULL if undefined
The entries that are currently written to a forward-log file have
associated the key values below:
os - operating system
ip - ip address
user - OS name of the user
login - database login name
uid - numeric user id (HP-UX, Linux)
pid - process id
pname - command line
info - additional application information
Example use:
const char *login = Fwu_get_session_entry("login");
const char *pname = Fwu_get_session_entry("pname");
Note:
The returned value is statically buffered. A circular array of
10 buffers is provided, so that after 10 invocations the first
returned value is overwritten.
Obtains a pointer to the current memo context, if any.
const void *Fwu_get_memo(int mode, time_t *timestamp, int *data_sz);
- Arguments:
- mode selects the memo type (Fwu_MEMO_DBBEGIN, Fwu_MEMO_DBEND, Fwu_MEMO_DBMEMO)
- timestamp, if not NULL, references a variable for returning the memo timestamp
- data_sz, if not NULL, references a variable for returning the memo data size
- Returns:
- pointer to the memo data or NULL, if no such context exists.
Discussion:
The fwutil library keeps track of one DBBEGIN/DBEND and one DBMEMO
context per client session. These can be retrieved via Fwu_get_memo() when processing a DBPUT,
DBUPDATE or DBDELETE entry inside the user supplied Fwu_callback_t() function.
Each DBBEGIN or DBEND sets the DBBEGIN/DBEND context and clears the DBMEMO context of the
associated session. Each DBMEMO sets the DBMEMO context, but keeps the DBBEGIN/DBEND context
unchanged.
Note:
This API requires version B.07.10.06 or later of the fwutil library.
Returns the current fw log file generation.
unsigned int Fwu_CurrentLogGeneration(void);
Returns the current fw log file sequence (within a generation).
unsigned int Fwu_CurrentLogSequence(void);
Returns the sequence number of the current transaction that is
processed within the current fw log file.
unsigned int Fwu_CurrentTagSequence(void);
The following utility functions for converting numeric data into
text format are included in the library.
Convert signed integer of 2/4/8 bytes
int Fwu_fmt_signed_int(void *buf_p, int buf_sz,
const void *v_p, int v_len);
Convert unsigned integer of 2/4/8 bytes
int Fwu_fmt_unsigned_int(void *buf_p, int buf_sz,
const void *v_p, int v_len);
Convert ieee float of 4/8 bytes
int Fwu_fmt_ieee_float(void *buf_p, int buf_sz,
const void *vp, int len);
Convert value in packed format (positive/negative/unsigned)
int Fwu_fmt_packed(void *buf_p, int buf_sz,
const void *v_p, int v_len);
Convert value in zoned format (positive/negative/unsigned)
int Fwu_fmt_zoned(void *buf_p, int buf_sz,
const void *v_p, int v_len);
- Arguments:
- buf_p specifies the pointer to a (user provided) conversion buffer.
- buf_sz specifies the size of the conversion buffer.
- v_p specifies the pointer to the data value.
- v_len specifies the size of the data value in bytes.
- Returns:
- returns 0 on success, -1 on error, 1 on buffer overflow
The output buffer is NUL terminated.
Example use:
int err;
static char cv_buf[256];
int err = Fwu_fmt_signed_int(cv_buf, sizeof(cv_buf), buf, size);
assert(!err);
printf("%s\n", cv_buf);
The FWUTIL_DEBUG_FLAGS environment variable may be used to
define fwutil debug mode. Setting this variable to "R1" is
similar to calling Fwu_verbose() with parameter value 1.
Three example programs are installed in /opt/eloquence6/share/example/fwutil
and also available for download below:
fwtest.c - print some information on database transactions
fwsql.c - convert database changes to SQL like syntax
fwmemo.c - version of fwtest.c also displaying memo records
All three programs accept a -help argument to display options:
$ fwtest -help
usage: fwtest [options]
options:
-help - show usage (this list)
-c cfg - server configuration file
-v - verbose, display progress
-S - synchronize on existing log, then exit
-f statfile - status file (default: fwtest.stat)
-n generation - initialize from forward-log generation
-I - print status file (-v to output details)
The example programs process a given set of forward logfiles
based on the specified server configuration file (-c option).
If the -S option is not used, the programs will process the
contents of the existing logfiles and then follow any on-going
changes written by the database server.
On first invocation you need to pass the log generation to
begin with (-n option). On later invocations you may resume
processing at the point recorded in the status file. Status
file contents can be examined using the -I option.
To compile/link these programs, please find example command
lines below.
HP-UX (ANSI C compiler):
cc -Ae -o fwtest -I/opt/eloquence6/include fwtest.c \
-L/opt/eloquence6/lib/pa20_32 -lfwutil
Linux (GNU C compiler):
gcc -o fwtest -I/opt/eloquence6/include fwtest.c \
-L/opt/eloquence6/lib -lfwutil -Wl,-rpath=/opt/eloquence6/lib
Windows (Microsoft C/C++ compiler):
cl -o fwtest.exe -I "C:\Program Files\Eloquence\include"
fwtest.c getopt.c "C:\Program Files\Eloquence\lib\fwutil.lib"
Please note: The getopt() function is not part of the C library
on Windows, therefore it must be obtained separately. We found
a few links through Google where free getopt() source code may
be downloaded, such as:
- http://www.pwilson.net/getopt.html
- http://www.codeproject.com/cpp/xgetopt.asp
- http://www.geocities.com/ResearchTriangle/Node/9405/
Revision B.07.10.06
- The fwutil library was enhanced to reduce the number of previous
fwlog segments needed when resuming processing based on a saved status_file.
Prior versions had to re-read all fwlog segments for the current generation
to establish the context for resuming. The enhanced version only needs the
fwlog segment matching the saved status_file (in most cases) or the segment
immediately preceding it.
-
The fwutil library API has been enhanced to also provide access to memo
records like DBBEGIN, DBMEMO and DBEND, if present in the forward logs.
The functions Fwu_set_memo_callback(), Fwu_memo_callback_t() and Fwu_get_memo()
have been added for this purpose. See API documentation for usage details.
Note that forward logs will only contain DBBEGIN, DBMEMO and DBEND records,
if the application uses those Image Calls and the Image3k library is patch
PE71-0710030 or later and the HP3K_API_COMPAT database property has bit 0
and/or bit 1 set.
| 
