Side of Software

	Home
	Contact Us

Products
	Dated Collections
	Marker Bar
	Persistence
	FAQs
	Tutorial
	API (javadocs)
	Print Preview
	Reports
	Wizards

Persistence Library Tutorial

Introduction

The Requirements

The Model

The Application without Persistence

The Application with Persistence

Additional Database Topics

Introduction

This tutorial steps you through the creation of two functionally identical banking applications—one that uses the Persistence Library and one that does not. By the end of the tutorial, we will have discussed these files:

Account.java. The interface for a simple bank account.

DefaultAccount.java. A straightforward implementation of an account.

BankExampleWithoutDatabase.java. The example application that does not use a database.

BankExampleWithDatabase.java. The example application that uses a database.

We show that adding an object-oriented database to a well-designed application requires very little additional programming. A head-to-head comparison of the two applications highlights the differences.

The Requirements

Our sample application should be simple enough to describe in a tutorial yet complex enough to show the benefits of the Persistence Library. We choose a banking application because the domain is well understood and the programs are often multi-threaded.

Our application will have the following capabilities:

Open a new bank account
Look up a bank account by number
Deposit and withdraw money from an account
Transfer money from one account to another
Generate a report that shows the number of accounts and the total balances in the accounts

The application creates 10 accounts each with an initial balance of 100 units. It launches 5 deposit threads, 5 transfer threads, and 5 report threads. Each deposit thread deposits 10 units in each account 10 times. Each transfer thread moves 20 units from Account i to Account i+1 10 times. Each report thread queries the number of accounts and sums the balances in all accounts before printing a statement to System.err. When all threads have finished, each account should have a balance of 600. By stressing the system with many concurrent transactions, we can test if the accounts remain consistent and if the transactions execute atomically.

The Model

The object model of our banking system is quite simple. Each account is an object that implements the Account interface. We use the java.util.Map interface to hold the accounts and to serve as an index into them. The map associates an account number with an account, and each account appears in the map exactly once.

The Account Interface

Here is the code for the Account interface (Account.java):

  public interface Account

/**

     * Returns this account's balance.<p>

*/

    public double getBalance();

/**

     * Sets this account's balance.<p>

     * @param balance this account's new balance

*/

    public void setBalance( double balance );

For our purposes, an account simply contains a balance and a way to set and retrieve it. The method getBalance retrieves the account’s balance without modifying the account, while the method setBalance modifies the account by updating its balance.

In a real-world application, the account interface would be much more complex. It would contain account numbers, references to its owners, and a history of balances (see the Dated Collections Library for an easy way to add a history of values). Furthermore, it may support property and vetoable change listeners.

The DefaultAccount Implementation

We implement our Account interface in a straightforward manner in the DefaultAccount class (DefaultAccount.java):

  public class DefaultAccount implements Account, Serializable

    private double balance;

    public double getBalance()

      return balance;

    public void setBalance( double balance )

      this.balance = balance;

The DefaultAccount class implements java.io.Serializable so the database can save it in the second version of this application.

Rule 1: The Persistence Library requires that all database objects be serializable.

The Application without Persistence

We first write the application without persistence and then show how adding persistence requires little effort. You can find the complete database-less application in BankExampleWithoutDatabase.java.

Threads and Atomicity

Since our application uses multiple threads, we must ensure that the map and the accounts are accessed safely. A thread invoking setBalance on an account may not access the account concurrently with any other thread. Threads invoking getBalance on the same account, however, may access the account concurrently because getBalance does not modify the account. Similarly, a thread iterating through the map may not execute concurrently with a thread opening a new account.

To achieve this behavior, we could associate a read-write lock with each account. Before a thread accesses an account, the thread acquires either its read lock or write lock, depending on the thread’s intentions. Unfortunately, this approach can easily lead to deadlock. Consider a transfer transaction. Suppose Thread A wishes to transfer from Account 5 to Account 16, and Thread B wishes to transfer from Account 16 to Account 5. If Thread A acquires the write lock for 5 and Thread B acquires the write lock for 16, each thread prevents the other thread from acquiring the next required lock (assuming a write lock cannot be obtained until its corresponding read lock has been released by all other threads).

It turns out to be extremely difficult to write this application with maximum concurrency. The application would not only have to handle the read-write locks but also implement a policy to avoid or detect deadlock, increasing the application’s code significantly.

It is much easier to use Java’s synchronized keyword and synchronize on a single object. This approach sacrifices concurrency but simplifies the programming and avoids deadlock. We take this approach here and synchronize on the application object. Since all access to the accounts and map will occur while holding this object’s lock, there can be no data races. Since all threads synchronize on a single lock, the chance of deadlock vanishes.

The Open Account Transaction

The application first creates 10 accounts (numbered 0 through 99) with an initial balance of 100:

    // create the accounts

    for( int i = 0; i < NUM_ACCOUNTS; i++ )

      openAccount( i, INITIAL_BALANCE );

To create an account, we construct an instance of DefaultAccount and put it in the map of accounts. It is implemented as follows:

  private synchronized Account openAccount( int accountNumber,

                                            double initialBalance )

    // create the account

    Account account = new DefaultAccount();

    // store this account in the map indexed by account number

    Object key = new Integer( accountNumber );

    accounts.put( key, account );

    // set the initial balance of this account

    account.setBalance( initialBalance );

    return account;

If the method were not marked synchronized, the map could be accessed concurrently or the account’s balance could be seen as 0.

The Lookup Transaction

Given an account number, we can retrieve the corresponding account by doing a synchronized lookup in the map:

  private synchronized Account lookupAccount( int accountNumber )

    Object key = new Integer( accountNumber );

    return (Account)accounts.get( key );        // search the accounts

Access to the map must be synchronized. If a different thread were trying to open a new account at the same time, we would have a conflicting data race.

The Deposit / Withdrawal Transaction

We can implement both deposits and withdrawals with a single method. Withdrawals are simply a deposit of a negative amount.

  private synchronized void deposit( Account acct, double amount )

    double balance = acct.getBalance();   // get the current balance

    balance += amount;                    // adjust the balance

    acct.setBalance( balance );           // set the new balance

Again, we make the method synchronized to ensure that two threads depositing to the same account at the same time set the correct amounts.

The Transfer Transaction

The transfer operation is comprised of two transactions—a withdrawal from one account and a deposit to another.

  private synchronized void transfer( Account fromAcct, Account toAcct,

                                      double amount )

    deposit( fromAcct, -amount );   // take money from one account

    deposit( toAcct, amount );      // add money to the other account

Even though the deposit method is synchronized, transfer must also be synchronized; otherwise, another thread (such as the report thread) may find the transfer amount missing if it accesses the accounts while the transfer thread is between the deposits.

The Report Transaction

The report thread iterates through the accounts, summing their balances as it goes:

  private synchronized void generateReport()

    int numAccounts = accounts.size();          // how many accounts?

    double total = 0;

    // for each account

    Collection values = accounts.values();

    Iterator acctIterator = values.iterator();

    while( acctIterator.hasNext() )

      Account account = (Account)acctIterator.next();

      double balance = account.getBalance();    // get the balance

      total += balance;                         // keep a running total

    // print the report

    System.err.println( total + " for " + numAccounts + " accounts " );

The entire iteration must be synchronized; otherwise, a thread opening an account may alter the map in the middle of the iteration.

The Worker Threads

The main body of our application spawns numerous threads, starts them, and waits for them to finish:

// create threads that will test the banking system

int numThreads = NUM_DEPOSIT_THREADS + NUM_TRANSFER_THREADS +

NUM_REPORT_THREADS;

Thread[] threads = new Thread[numThreads];

int index = 0;

// create threads that repeatedly deposit money to all accounts

for( int i = 0; i < NUM_DEPOSIT_THREADS; i++ )

threads[index++] = createDepositThread( AMOUNT_TO_DEPOSIT );

// create threads that repeatedly transfer money among accounts

for( int i = 0; i < NUM_TRANSFER_THREADS; i++ )

threads[index++] = createTransferThread( AMOUNT_TO_TRANSFER );

// create threads that repeatedly report account totals

for( int i = 0; i < NUM_REPORT_THREADS; i++ )

threads[index++] = createReportThread();

// start the threads

for( int i = 0; i < threads.length; i++ )

threads[i].start();

// wait until the threads stop

for( int i = 0; i < threads.length; i++ )

threads[i].join();

The code that creates a deposit thread (createDepositThread) is:

  private Thread createDepositThread( final double depositAmount )

    Thread thread = new Thread( new Runnable() {

      public void run()

        for( int acctNumber = 0; acctNumber < NUM_ACCOUNTS; acctNumber++ )

          Account acct = (Account)lookupAccount( acctNumber );

          for( int i = 0; i < NUM_DEPOSITS; i++ )

            deposit( acct, depositAmount );   // make the deposit

    } );

    return thread;

The code that creates a transfer thread and report thread is similar and can be found in the application’s source code.

A Sample Execution

You can find the complete database-less application in BankExampleWithoutDatabase.java. An execution of the application may produce the following output:

4140.0 for 10 accounts

4150.0 for 10 accounts

...

[82 reports omitted]

...

4300.0 for 10 accounts

4310.0 for 10 accounts

4330.0 for 10 accounts

The actual values printed depend on the thread ordering. Each execution may yield a different order. We can see from this execution that deposits and transfers were made before, during, and after the numerous report generations. The final account balances are verified with the code:

    // verify that the account balances are what we expected

    double expectedBalance = INITIAL_BALANCE + NUM_DEPOSIT_THREADS *

                             NUM_DEPOSITS * AMOUNT_TO_DEPOSIT;

    for( int i = 0; i < NUM_ACCOUNTS; i++ )

      Account acct = lookupAccount( i );

      double balance = acct.getBalance();

      assert balance == expectedBalance

        : "Incorrect balance for account " + i + ": " + balance;

Since no assertion error was thrown, the accounts had the expected balance.

The Application with Persistence

Saving the map and accounts in an object-oriented database requires little additional programming. To add persistence to our application, we:

Open and close a database
Add new objects to the database and use the returned proxies
Identify transactions
Register the read-only methods

The complete application can be found in BankExampleWithDatabase.java.

Rule 2: The Persistence Library requires you to program by interface. This means that all of the domain classes should be completely defined by one or more interfaces and the types of all mutable method parameters should be interfaces. The domain model of our bank account system—as well as the model of most well-designed applications—follows this convention. The Java Collections API also satisfies this requirement. Consequently, the application may not invoke a method that is defined in a class but not in an interface (such as java.util.LinkedList.addLast). This rule is enforced at run-time.

Opening and closing the database

To incorporate a database, we must first decide what type of database our application should use. The Persistence Library provides two types: FileDatabase and InMemoryDatabase. The former saves objects in a file and therefore is persistent. Objects survive application runs. The latter, on the other hand, only saves objects in memory, so it is not persistent. As soon as the application terminates, the memory is flushed and the database is cleared. The in-memory database is useful for applications that want database and transaction functionality but do not require persistence. It can be used in an applet without needing access to the host’s file system. Since memory I/O is much faster than file I/O, an InMemoryDatabase will outpace FileDatabase considerably.

In our application, the type of database does not matter since each run re-initializes the database with 10 accounts. We choose to use a file database, but you may uncomment the line above it to use an in-memory database:

    // construct the database

    // db = new InMemoryDatabase( "Account Database" );

    db = new FileDatabase( "Account Database", "test\\accts.db" );

The name of our database is “Account Database” and it resides in the file “test\accts.db” relative to the current working directory.

Before we can use the database, however, we must open it. It cannot be opened unless it exists, so our application first creates the database and then opens it:

    // create the database with the account map as the root object

    db.create( new HashMap() );

    // open the database

    db.open( false );

In the call to create, we must pass a root object to serve as the entry point into the database. For us, the root object is the map of accounts. In general, all database objects should be reachable from the root. A java.util.Map instance is always a safe choice for a root because it can adapt to an application’s changing requirements.

To open the database, we simply invoke open, passing it false to indicate that we do not want the database to be read-only. After the application finishes, we invoke close to disallow any subsequent database activity.

    // close the database

    db.close();

Adding objects to the database

All objects to be saved in the database must be registered with the database via a call to addObject. After registration, these objects are automatically resaved when they change. The library uses a proxy object, which is returned from the call to addObject, to detect when a registered object potentially changes. This proxy implements the same interfaces as the registered object and forwards all method invocations to it.

Rule 3: After an object is passed to addObject, the application may no longer access it directly. Instead, the application must cast the returned proxy to the appropriate interface and access it through the proxy.

Following this rule, the openAccount method in this version of our banking application becomes:

  private Account openAccount( int accountNumber, double initialBalance )

    db.startTransaction();

    // create the account

    Account account = new DefaultAccount();

    account = (Account)db.addObject( account );

    // store this account in the map indexed by account number

    Object key = new Integer( accountNumber );

    accounts.put( key, account );

    // set the initial balance of this account

    account.setBalance( initialBalance );

    db.commitTransaction( null );

    return account;

As soon as the account is instantiated, it is added to the database. It is also okay to initialize the account before adding it to the database, although it is good practice to add the object immediately after instantiation. Notice that, after the account is added to the database, the variable is reassigned to the downcast proxy. This means that setBalance is actually invoked on the proxy, which forwards the update to the instance of DefaultAccount. It is essential that we store the proxy, not the underlying account, in the map.

Rule 4: Immutable objects (like instances of java.lang.Integer) need not be registered with the database.

The root object also requires a proxy that must be retrieved via the getRoot method.

    // load the root object

    accounts = (Map)db.getRoot();

Identifying transactions

Just as we did in the first version of the application, we must identify groups of operations that should execute as a unit. Instead of using Java’s built-in synchronization, however, we use the database to identify transactions. Each database transaction exhibits the ACID properties (Atomicity, Consistency, Isolation, and Durability).

We invoke startTransaction to signal the beginning of a transaction and commitTransaction to signal the end. All database activity between these two statements by the same thread must succeed as a unit. Additionally, no transaction by another thread may appear to overlap the transaction.

Rule 5: A transaction must be carried out by a single thread.

Using startTransaction and commitTransaction instead of the synchronized keyword, the deposit, generateReport, and transfer methods become:

  private void deposit( Account acct, double amount )

    db.startTransaction();

    double balance = acct.getBalance();   // get the current balance

    balance += amount;                    // adjust the balance

    acct.setBalance( balance );           // set the new balance

    db.commitTransaction( null );

  private void generateReport()

    db.startTransaction();

    int numAccounts = accounts.size();          // how many accounts?

    double total = 0;

    // for each account

    Collection values = accounts.values();

    Iterator acctIterator = values.iterator();

    while( acctIterator.hasNext() )

      Account account = (Account)acctIterator.next();

      double balance = account.getBalance();    // get the balance

      total += balance;                         // keep a running total

    db.commitTransaction( null );

    // print the report

    System.err.println( total + " for " + numAccounts + " accounts " );

  private void transfer( Account fromAcct, Account toAcct, double amount )

    db.startTransaction();

    deposit( fromAcct, -amount );   // take money from one account

    deposit( toAcct, amount );      // add money to the other account

    db.commitTransaction( null );

Transactions can be nested. We can invoke startTransaction any number of times as long as each is paired with a commitTransaction. The top-most transaction determines when its nested transactions succeed. The transfer transaction is an example of a nested transaction. The two deposits must execute together and are committed only when the outer transfer transaction commits.

The lookupAccount method does not need to be enclosed by startTransaction and commitTransaction because it contains only one line of database activity. One-liners are automatically executed atomically by the database:

  private Account lookupAccount( int accountNumber )

    Object key = new Integer( accountNumber );

    return (Account)accounts.get( key );        // search the accounts

Registering read-only methods

The database uses a method’s read-only classification to determine if a registered object changes as a result of a method invocation. A method is read-only if it is guaranteed not to alter the receiver in any way. Our getBalance method is an example of a read-only method, as is java.util.Collection.size. A method is a write method if it may modify the receiver. Our setBalance method is an example of a write method, as is java.util.Collection.add. Some methods, like java.util.Collection.iterator, do not modify the receiver but return a new object that may. Through this new object, the application may modify the original object (by invoking java.util.Iterator.remove, for example). We classify these methods as deferred write methods. If the database does not know a method’s classification, it conservatively assumes it is a write method.

Although it is not necessary for correctness, for efficiency we should inform the read-only manager that the getBalance method does not modify the account:

    ReadOnlyManager.setMethodStatus( Account.class, "getBalance",

      null, ReadOnlyManager.READ_ONLY );

Without this statement, each invocation of an account’s getBalance method would cause the database to resave the account unnecessarily.

A Sample Execution

You can find the complete application in BankExampleWithDatabase.java and a line-by-line comparison of the two versions here. An execution may produce the following output:

Mar 3, 2004 9:17:08 PM sos.db.AbstractSerializationDatabase logDatabaseOpened

INFO: Database "Account Database" opened.

1090.0 for 10 accounts

1140.0 for 10 accounts

...

[82 reports omitted]

...

2320.0 for 10 accounts

2430.0 for 10 accounts

2620.0 for 10 accounts

2990.0 for 10 accounts

3280.0 for 10 accounts

3670.0 for 10 accounts

Mar 3, 2004 9:18:20 PM sos.db.AbstractSerializationDatabase logDatabaseClosed

INFO: Database "Account Database" closed.

Again, since no assertion error was thrown, the final account balances matched the expected balances.

Additional Database Topics

The above example discusses the minimum steps needed to incorporate mostly transparent persistence in an application. This section discusses the following additional topics:

Transaction failures
Transaction policies
Logging

Transaction Failures

Transactions may fail. They may fail because of an underlying I/O error or because of a transaction conflict. For example, the database may be corrupted or it may decide to abort a transaction because it has deadlocked with another transaction.

Whenever a transaction fails, the database throws a TransactionAbortedException and rolls the database back to its state when the transaction started. We can invoke getCause on the thrown exception to examine the cause of the abort.

Our banking application does not handle transaction failures. To make it more robust, we would have to examine each transaction and determine an appropriate action to a failure. Common actions include ignoring it, quitting, retrying, and informing the user. The following code shows how the deposit thread can forever retry aborted transactions:

  private Thread createDepositThread( final double depositAmount )

    Thread thread = new Thread( new Runnable() {

      public void run()

        for( int acctNumber = 0; acctNumber < NUM_ACCOUNTS; acctNumber++ )

try

            Account acct = (Account)lookupAccount( acctNumber );

            for( int i = 0; i < NUM_DEPOSITS; i++ )

try

                deposit( acct, depositAmount );   // make the deposit

              catch( TransactionAbortedException tae )

                i--;    // try again

          catch( TransactionAbortedException tae )

            acctNumber--;    // try again

    } );

    return thread;

Failure of nested transactions requires attention as well. If an inner transaction fails, should we abort the outer transaction? In the following code, we modify transfer to abort if either of its deposits fails:

  private void transfer( Account fromAcct, Account toAcct, double amount )

    db.startTransaction();

try

      deposit( fromAcct, -amount );   // take money from one account

      deposit( toAcct, amount );      // add money to the other account

      db.commitTransaction( null );

    catch( TransactionAbortedException tae )

      db.abortTransaction();

      throw tae;

The above code uses abortTransaction to programmatically stop the outer transaction.

Transaction Policies

It is the database’s responsibility to ensure that transactions exhibit the ACID properties. The library’s implementation actually delegates transaction behavior to an instance of TransactionPolicy. This transaction policy decides the locking scheme as well as the deadlock detection scheme.

The constructors of both FileDatabase and InMemoryDatabase can take a client-specified transaction policy. In our example, we did not specify one, so a default—an instance of DefaultTransactionPolicy—is provided.

DefaultTransactionPolicy behaves like the synchronization in the version without the database. Each transaction is willing to wait forever to acquire a single, shared lock, thereby yielding a strict sequential ordering of transactions. Since this policy avoids deadlock, the chance of TransactionAbortedExceptionbeing thrown is not very high.

As we mentioned before, this approach does not allow any concurrency, causing transactions that touch disjoint portions of the database to unnecessarily interfere. The Persistence Library provides another transaction policy, called DefaultReadWriteTransactionPolicy, which locks objects individually to achieve greater concurrency. It uses read-write locks to allow the multiple-readers single-writer paradigm. It uses a method’s read-only classification to determine whether it should acquire the read lock or write lock.

Since DefaultReadWriteTransactionPolicy locks database objects individually, the chance for deadlock is great if two transactions work with the same objects. The policy detects deadlock by using lock timeouts; if a transaction fails to acquire a lock in a specified amount of time, the policy assumes it has deadlocked and aborts it.

This policy is not ideal for our application because of the high number of conflicting transactions. If you change the instantiation of the database to:

    // construct the database

    TransactionPolicy policy =

      new DefaultReadWriteTransactionPolicy( 200, 100 );

    db = new FileDatabase( "Account Database",

                           "test\\accts.db", policy );

then you can see that many TransactionAbortedExceptions are thrown.

Logging

In the sample execution above, you may have noticed the log messages that the application outputted. The persistence framework uses Java’s Logging API to log database activity. By default, the following database activities are logged to the logger named “sos.db”:

Logged Database Activity	Severity
The database is created	INFO
The database is closed	INFO
The database is deleted	INFO
The database is opened	INFO
A transaction is started	FINE
A transaction is committed	FINE
A transaction is aborted	FINE
An object is added to the database	FINER
An object is resaved in the database	FINER
An object is removed from the database	FINER
A read lock is acquired	FINEST
A read lock is released	FINEST
A write lock is acquired	FINEST
A write lock is released	FINEST
A method on a database object is invoked	FINEST

We can adjust our application’s logging behavior either statically or at run-time, using the approaches described in the Logging Specification. For example, we can lower the log level to FINE by adding the following code to main:

Logger.getLogger( "" ).setLevel( Level.FINE );

Handler[] handlers = Logger.getLogger( "" ).getHandlers();

for( int i = 0; i < handlers.length; i++ )

handlers[i].setLevel( Level.FINE );

With this change in place, messages regarding transaction starts, aborts, and commits are also outputted:

  Mar 4, 2004 8:57:05 AM sos.db.AbstractSerializationDatabase logDatabaseOpened

  INFO: Database "Account Database" opened.

  Mar 4, 2004 8:57:05 AM sos.db.AbstractSerializationDatabase logTransactionStarted

  FINE: Transaction started for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:57:05 AM sos.db.AbstractSerializationDatabase logTransactionStarted

  FINE: Transaction started for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:57:05 AM sos.db.AbstractSerializationDatabase logTransactionCommitted

  FINE: Transaction committed for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:57:05 AM sos.db.AbstractSerializationDatabase logTransactionStarted

  FINE: Transaction started for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:57:05 AM sos.db.AbstractSerializationDatabase logTransactionCommitted

  FINE: Transaction committed for Thread[main,5,main] in database "Account Database".

...

  [Zillions of messages omitted]

...

  Mar 4, 2004 8:58:48 AM sos.db.AbstractSerializationDatabase logTransactionCommitted

  FINE: Transaction committed for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:58:48 AM sos.db.AbstractSerializationDatabase logTransactionStarted

  FINE: Transaction started for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:58:48 AM sos.db.AbstractSerializationDatabase logTransactionCommitted

  FINE: Transaction committed for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:58:48 AM sos.db.AbstractSerializationDatabase logTransactionStarted

  FINE: Transaction started for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:58:48 AM sos.db.AbstractSerializationDatabase logTransactionCommitted

  FINE: Transaction committed for Thread[main,5,main] in database "Account Database".

  Mar 4, 2004 8:58:48 AM sos.db.AbstractSerializationDatabase logDatabaseClosed

  INFO: Database "Account Database" closed.

Developers can use the logging to aid in debugging, and system administrators can use it to monitor client usage.

For additional information on the Persistence Library, consult the Persistence Library API.

Home | Contact Us | Privacy Policy

Persistence Library Tutorial

Table of Contents

Introduction

The Requirements

The Model

The Account Interface

The DefaultAccount Implementation

The Application without Persistence

Threads and Atomicity

The Open Account Transaction

The Lookup Transaction

The Deposit / Withdrawal Transaction

The Transfer Transaction

The Report Transaction

The Worker Threads

A Sample Execution

The Application with Persistence

Opening and closing the database

Adding objects to the database

Identifying transactions

Registering read-only methods

A Sample Execution

Additional Database Topics

Transaction Failures

Transaction Policies

Logging