Documentation

Apologies to anyone not interested in reading my description of the inner workings of Kowari.

There's obviously a lot happening while I try to get used to a new city and lifestyle, so I haven't returned to my research yet. I'm also working on commercial code, which means I have to be more circumspect about what I'm doing. This all means I haven't had the time or motivation to do new things in my own time, making it hard to write about anything interesting!

As an exercise in re-familiarizing myself with Kowari, and keeping in the habit of writing (which can be hard to start again once you've stopped) I decided that I should write about how each class works. I mentioned this shortly after presenting all of this for NGC, but it's taken me this long to get around to it.

I'll write about more interesting things as they come to hand. For instance, I was told yesterday that a major impediment to the adoption of semantics in some areas has been the lack of a scalable statement store. Argh!

AbstractBlockFile

This class does not add much to the semantics of BlockFile, but it has a few internal details worth mentioning.

AbstractBlockFile contains a static block which determines the byte ordering to be used for all files. This is generally the native format for the platform, but maybe overridden by setting the system property tucana.xa.useByteOrder to native (the default), big_endian, or little_endian. This should usually be left alone, but it may be necessary if files have been moved from one computer to another with a different CPU architecture. Upgrading an Apple from PowerPC to the new Intel Core Duo is an good example of the need for this option.

The constructor for AbstractBlockFile performs several checks. It starts by checking if the file is already present in a static set of files. If so, then this JVM has already opened the file, so an exception is thrown. Otherwise, the file is added to the set once it has been opened.

The second check in the constructor is to ensure that the file is a multiple of the block size. If not, then it tries to truncate the file to an appropriate length.

Truncation is an awkward operation for a mapped file, as a mapping could still exist from the file being opened and subsequently closed earlier in the lifecycle of the program. If this is the case, an IOException will be thrown, and the mapping must be removed. Running the garbage collector is usually enough to clean up any unused file mappings, but on Windows it is necessary to run the collector several times for this technique to work.

The code to do this looks like:

for (;;) { 
  try { 
    fc.truncate(fileSize);
    break;
  } catch (IOException ex) { 
    if (retries-- == 0) { 
      logger.error(
        "Could not truncate file: \"" + file + 
        "\" to size: " + fileSize, ex
      );
      throw ex;
    } 
    System.gc();
    try { Thread.sleep(100); } catch (InterruptedException ie) { } 
    System.runFinalization();
  } 
}

This technique is duplicated in clear(), close() and delete().

Factory Methods

The openBlockFile() methods each take an IOType to determine which of the implementing classes to return an instance of. However, these methods also check the tucana.xa.forceIOType system property to see if this value should be overridden. If the property is set to mapped or explicit then that will dictate if MappedBlockFile or IOBlockFile should be used. Otherwise, the requested IOType will be used.

It should be noted that it is unusual for an abstract class to have information about its implementing subclasses like this. If another type of implementation were to be added, then this super class would need to be updated as well. This sort of dependency is undesirable in object oriented code, but the benefits here were deemed to make it worthwhile.

Opened Files

The constructor for this class does not open the file, but rather calls the ensureOpen() method. This method makes sure that a file is open, and if it is not, then it attempts to perform the open operation. The operation is called in this way to try to cover occasions where a problem occurred (such as a "rollback" operation), and there is some code in the method to handle these conditions.

Several other methods in AbstractBlockFile also call ensureOpen before performing file operations, in case anything has happened to close access to the file.

Note also, that access to ensureOpen and the isOpen set is all synchronized.