.
Eloquence B.08.00 contact contact

Documentation / FWUTIL Library

FWUTIL Library

 
.
 

Revision: B.08.00.00

Contents


Introduction

The fwutil library enables access to the Eloquence database forward log file that holds the archived committed transactions.

Decoding the fwlog file format is performed by the fwutil library and a callback to a user provided function is performed for IMAGE like operations which simplifies programming and isolates the utility from format changes.

The fwutil library works incrementally. Transactions from a starting point are processed. The current progress is recorded in a status file and may be used to continue subsequently. Once the end of the fw log file(s) is reached, the library may either return or wait for additional changes which are then processed.

The fwutil library makes use of the audit information in the forward log file and the database server must be configured to record audit information.

DBBEGIN/DBEND and DBMEMO

Beginning with version B.07.10.06 of the fwutil library, it does not only provide information on logged DBPUT, DBUPDATE and DBDELETE invocations, but also provides access to DBBEGIN/DBEND and DBMEMO entries in the forward logfiles, if present.

Note that logging of such "memo entries" depends on flags in the HP3K_API_COMPAT database property being set (bit 0 enables logging of DBMEMO and bit 1 enables logging of DBBEGIN and DBEND).

Two different methods are available for accessing the optional DBBEGIN/DBEND and DBMEMO entries in the forward logs:

a) Registering a separate callback function with Fwu_set_memo_callback().
This approach is similar to using the callback function that is invokved for each DBPUT, DBUPDATE, and DBDELETE entry in the forward logs. It just delivers the DBBEGIN/DBEND and DBMEMO information to a separate callback function with slightly different parameters.

b) Calling Fwu_get_memo() inside the regular Fwu_process() callback function.
This approach is similar to obtaining session context info with Fwu_get_session_id() while processing a DBPUT, DBUPDATE or DBDELETE event, and delivers the DBBEGIN/DBEND and or DBMEMO context for the database update entry being processed.

Find details on the above approaches in the API descriptions below.


Library functions

The fwutil.h include file provides the function prototypes and types. It must be included in any file that makes use of the fwutil library.

 #include "fwutil.h"

The fwutil.h file is installed in the Eloquence include subdirectory.


Fwu_version

Output the fwutil library version (to stdout)

 void Fwu_version(void);

Fwu_status

Output the fwutil status file (to stdout)

int Fwu_status(const char *status_file, int verbose);
Arguments:
  • status_file specifies the name of the status file
  • if verbose is non-zero then the full content is printed
Returns:
Completion status, 0 for success, -1 on failure.

Fwu_synconly

If called with a non-zero argument specifies that Fwu_process() should return once the end of the fwlog file(s) was reached.

void Fwu_synconly(int mode);
Arguments:
0 = wait for new transactions after reaching the end of the fw log file(s).
This is the default.
1 = return after reaching the end of the fw log file(s)

Fwu_verbose

If called with a non-zero argument specifies that Fwu_process() should output additional progress messages (to stderr).

void Fwu_verbose(int mode);
Arguments:
0 = Don't print progress messages. This is the default
1 = Output additional debug messages (to stderr)

This is similar to setting the FWUTIL_DEBUG_FLAGS environment variable to "*1".


Fwu_process

Process archived transactions.

int Fwu_process(Fwu_callback_t cb, const char *cfg_file, 
                const char *status_file,
	        const char *fwr_start);

Arguments:
  • cb specifies a pointer to a user provided callback function. This function is called for image-like changes(see below).
  • cfg_file specifies the name of the server configuration file (eloqdb.cfg). This is used to locate the fwlog files.
  • status_file specifies the name of the fwutil status file.
  • The fwr_start argument may be used to specify a starting point when processing any archived transactions. If set, the status file is initialized. NULL should be specified to use the status file.
Returns:
Completion status, 0 for success, -1 on failure.
On failure, a message is printed to stderr.
Discussion:

By default, the fwutil library processes all enqueued changes and then closely follows any on-going changes on the server. If the Fwu_synconly() function was called with a non-zero argument, Fwu_process() returns once all enqueued changes are processed.

The fwutil library uses a local file to maintain its most recent successfully executed call. The status file may be specified with the status_file argument and defaults to fwutil.stat in current directory.

The Fwu_status() function outputs the content of the status file. If Fwu_verbose() was called with a non-zero argument all details are output, otherwise only information on the most recently completed call is output.

The fwr_start argument is used to specify the forward log generation that the fwutil library should be starting with. It is required on the first use and may also be used to specify a different starting point. If not specified, the position is obtained from the status file. The fwr_start argument uses a syntax of "gen[-seg[.action]]" where gen specifies the generation of the forward log file, the optional seg specifies a file segment and the optional action specifies a specific action in this file. Typically only the generation is used.


Fwu_callback_t

This defines the prototype of a user-defined callback function.

typedef int (*Fwu_callback_t)(int db_node_id, const char *db_name, 
                   const char *set_name, time_t timestamp, 
		   enum Fwu_OP op, unsigned int recno, int recsz,
                   const void *bi_rec, const void *ai_rec);

Arguments:
  • db_node_id specifies the internal id of the database. This value is unique for each database.
  • db_name specifies the database name.
  • set_name specifies the data set name.
  • timestamp specifies the time when the transaction was recorded (seconds since 1.1.1970 UTC).
  • op specifies the IMAGE call (Fwu_OP_UPDATE/Fwu_OP_PUT/Fwu_OP_DELETE).
  • recno specifies the record number.
  • recsz specifies the record size in bytes (as specified in schema).
  • bi_rec provides a pointer to a buffer holding the previous record data ("before image"). This is present for an UPDATE and DELETE call. Otherwise a NULL pointer is passed.
  • ai_rec provides a pointer to a buffer holding the updated record data ("after image"). This is present for a PUT and UPDATE call. Otherwise a NULL pointer is passed.
Returns:
Completion status, 0 for success, -1 on failure.

Fwu_set_memo_callback

Register callback function for handling memo records.

void Fwu_set_memo_callback(Fwu_memo_callback_t cb);
Arguments:
  • cb specifies a pointer to a user provided callback function. This function is called for memo records, if any (see below).
Discussion:

By default, Fwu_process() invokes the user supplied Fwu_callback_t() function for each DBPUT, DBUPDATE and DBDELETE entry in the forward log. If registered with this function, a separate user supplied Fwu_memo_callback_t() function is invoked for each memo entry (like DBBEGIN, DBEND and DBMEMO) that is present in the logs.

Note:
This API requires version B.07.10.06 or later of the fwutil library.


Fwu_memo_callback_t

This defines the prototype of a user-defined memo callback function.

typedef int (*Fwu_memo_callback_t)(time_t timestamp, int mode,
                  const void *data, int data_sz);
Arguments:
  • timestamp specifies the time when the transaction was recorded (seconds since 1.1.1970 UTC).
  • mode specifies the type of memo call (Fwu_MEMO_DBBEGIN, Fwu_MEMO_DBEND, Fwu_MEMO_DBMEMO).
  • data specifies a pointer to the data in the memo record.
  • data_sz specifies the data size in bytes (may be zero).
Returns:
Completion status, 0 for success, -1 on failure.

Note:
This API requires version B.07.10.06 or later of the fwutil library.


Fwu_get_item

Retrieve schema information. Obtains next item definition in schema order.

int Fwu_get_item(const char **i_name, int *i_type, 
	         int *i_size, int *i_count, unsigned int *i_fmt);
Arguments:
  • i_name is set to a NUL terminated pointer to the item name.
  • i_type is set to the native item type (X/U/B/I/K/E/P/Z).
  • i_size is set to the item size (in bytes).
  • i_count is set to the item count.
  • i_fmt is set to the item format.
Returns:
Completion status, 0 if no schema information available, 1 on success.
Note:
The item format flags are used to indicate the role of an item as below:
  • Bit 16 (0x10000) is set if the item is a search item.
  • Bit 18 (0x40000) is set if the item is a unique key.
      Currently, this indicates it is a master search item.
  • Bit 19 (0x80000) is set if the item is a sort item.

Fwu_get_session_id

Obtains the current session identifier. Each session has a unique identifier starting with 1.

int Fwu_get_session_id(void);
Returns:
current session identifier or 0 if undefined

Fwu_get_session_elements

Obtains the number of data elements in the session record.

int Fwu_get_session_elements(void);
Returns:
number of elements in session record or 0 if no data present

Session data consists of multiple elements, for example:

 session_data[0]=protocol{8}os{HP-UX}ip{127.0.0.1}user{joe}login{public}
 session_data[1]=uid{221}pid{18416}pname{dbimport -vs sample.exp sample}

Fwu_get_session_data

Retrieves a session data element by index. The index is zero-based. The returned data element is a null-terminated string.

const char *Fwu_get_session_data(int element_idx);
Argument:
zero-based element index
Returns:
pointer to null-terminated session data or NULL if undefined

Example use:

int i, cnt = Fwu_get_session_elements();
for (i = 0; i < cnt; i++)
   printf(" session_data[%d]=%s\n", i, Fwu_get_session_data(i));

Note:
The returned value is statically buffered. A circular array of 10 buffers is provided, so that after 10 invocations the first returned value is overwritten.


Fwu_get_session_entry

Retrieves a session entry by key.

const char *Fwu_get_session_entry(const char *key);
Argument:
key value of session entry to retrieve
Returns:
pointer to null-terminated session entry or NULL if undefined

The entries that are currently written to a forward-log file have associated the key values below:

    os    - operating system
    ip    - ip address
    user  - OS name of the user
    login - database login name
    uid   - numeric user id (HP-UX, Linux)
    pid   - process id
    pname - command line
    info  - additional application information

Example use:

const char *login = Fwu_get_session_entry("login");
const char *pname = Fwu_get_session_entry("pname");

Note:
The returned value is statically buffered. A circular array of 10 buffers is provided, so that after 10 invocations the first returned value is overwritten.


Fwu_get_memo

Obtains a pointer to the current memo context, if any.

const void *Fwu_get_memo(int mode, time_t *timestamp, int *data_sz);
Arguments:
  • mode selects the memo type (Fwu_MEMO_DBBEGIN, Fwu_MEMO_DBEND, Fwu_MEMO_DBMEMO)
  • timestamp, if not NULL, references a variable for returning the memo timestamp
  • data_sz, if not NULL, references a variable for returning the memo data size
Returns:
pointer to the memo data or NULL, if no such context exists.
Discussion:

The fwutil library keeps track of one DBBEGIN/DBEND and one DBMEMO context per client session. These can be retrieved via Fwu_get_memo() when processing a DBPUT, DBUPDATE or DBDELETE entry inside the user supplied Fwu_callback_t() function.
Each DBBEGIN or DBEND sets the DBBEGIN/DBEND context and clears the DBMEMO context of the associated session. Each DBMEMO sets the DBMEMO context, but keeps the DBBEGIN/DBEND context unchanged.

Note:
This API requires version B.07.10.06 or later of the fwutil library.


Fwu_CurrentLogGeneration

Returns the current fw log file generation.

unsigned int Fwu_CurrentLogGeneration(void);

Fwu_CurrentLogSequence

Returns the current fw log file sequence (within a generation).

unsigned int Fwu_CurrentLogSequence(void);

Fwu_CurrentTagSequence

Returns the sequence number of the current transaction that is processed within the current fw log file.

unsigned int Fwu_CurrentTagSequence(void);

Utility functions

The following utility functions for converting numeric data into text format are included in the library.

Fwu_fmt_signed_int

Convert signed integer of 2/4/8 bytes

int Fwu_fmt_signed_int(void *buf_p, int buf_sz, 
                       const void *v_p, int v_len);

Fwu_fmt_unsigned_int

Convert unsigned integer of 2/4/8 bytes

int Fwu_fmt_unsigned_int(void *buf_p, int buf_sz, 
                         const void *v_p, int v_len);

Fwu_fmt_ieee_float

Convert ieee float of 4/8 bytes

int Fwu_fmt_ieee_float(void *buf_p, int buf_sz, 
                       const void *vp, int len);

Fwu_fmt_packed

Convert value in packed format (positive/negative/unsigned)

int Fwu_fmt_packed(void *buf_p, int buf_sz, 
                   const void *v_p, int v_len);

Fwu_fmt_zoned

Convert value in zoned format (positive/negative/unsigned)

int Fwu_fmt_zoned(void *buf_p, int buf_sz, 
                  const void *v_p, int v_len);
Arguments:
  • buf_p specifies the pointer to a (user provided) conversion buffer.
  • buf_sz specifies the size of the conversion buffer.
  • v_p specifies the pointer to the data value.
  • v_len specifies the size of the data value in bytes.
Returns:
returns 0 on success, -1 on error, 1 on buffer overflow

The output buffer is NUL terminated.

Example use:

int err;
static char cv_buf[256];

int err = Fwu_fmt_signed_int(cv_buf, sizeof(cv_buf), buf, size);
assert(!err);
printf("%s\n", cv_buf);

Environment variables

The FWUTIL_DEBUG_FLAGS environment variable may be used to define fwutil debug mode. Setting this variable to "R1" is similar to calling Fwu_verbose() with parameter value 1.


Example programs

Three example programs are installed in /opt/eloquence/8.0/share/example/fwutil and also available for download below:

 fwtest.c  - print some information on database transactions
 fwsql.c   - convert database changes to SQL like syntax
 fwmemo.c - version of fwtest.c also displaying memo records

All three programs accept a -help argument to display options:

 $ fwtest -help
 usage: fwtest [options]
 options:
  -help         - show usage (this list)
  -c cfg        - server configuration file
  -v            - verbose, display progress
  -S            - synchronize on existing log, then exit
  -f statfile   - status file (default: fwtest.stat)
  -n generation - initialize from forward-log generation
  -I            - print status file (-v to output details)

The example programs process a given set of forward logfiles based on the specified server configuration file (-c option).

If the -S option is not used, the programs will process the contents of the existing logfiles and then follow any on-going changes written by the database server.

On first invocation you need to pass the log generation to begin with (-n option). On later invocations you may resume processing at the point recorded in the status file. Status file contents can be examined using the -I option.


To compile/link these programs, please find example command lines below.

HP-UX (ANSI C compiler):

 cc -Ae -o fwtest -I/opt/eloquence/8.0/include fwtest.c \
   -L/opt/eloquence/8.0/lib/pa20_32 -lfwutil

Linux (GNU C compiler):

 gcc -o fwtest -I/opt/eloquence/8.0/include fwtest.c \
   -L/opt/eloquence/8.0/lib -lfwutil -Wl,-rpath=/opt/eloquence/8.0/lib

Windows (Microsoft C/C++ compiler):

 cl -o fwtest.exe -I "C:\Program Files\Eloquence\include"
   fwtest.c getopt.c "C:\Program Files\Eloquence\lib\fwutil.lib"

Please note: The getopt() function is not part of the C library on Windows, therefore it must be obtained separately. We found a few links through Google where free getopt() source code may be downloaded, such as:

  • http://www.pwilson.net/getopt.html
  • http://www.codeproject.com/cpp/xgetopt.asp
  • http://www.geocities.com/ResearchTriangle/Node/9405/


 
 
.
 
 
  Privacy | Webmaster | Terms of use | Impressum Revision:  2008-10-27  
  Copyright © 2006-2008 Marxmeier Software AG