State of Origin
Tonight's post will be short. State of Origin was on tonight, and my brother invited me along to watch it with a few drinks. Queensland lost in extra time - the first occasion extra time has been used.

So now I'm back home, and in no fit state to talk about much, so I won't... much.

JXUnit tests
While testing the transitive statement code last night, one of the JXUnit tests failed. After confirming with others that I was the only one with the problem, I spent all morning trying to work out why.

The problem finally showed up as a NullPointerException thrown while logging. Because I've been working in LocalQuery, I had logging turned on for this class. Some of the recent changes for Tuples objects have occasionally resulted in "Empty" sets being returned when a Tuples object has no data in it. This means that a Tuples with specified columns can claim that it has no columns if there are no rows. One of the methods which reports the contents of a Tuples was not compatible with this change, and caused the exception seen. However, it was only called when logging was enabled, which is why I was the only one to see it.

After resolving that issue I moved on to writing my own JXUnit tests. These are going fine, though I'd like to write a few more.

I spent the last couple of hours asking questions and clarifications for AN's ontology plan for Kowari. I'm finally starting to get an idea of what to do, though I don't know if everyone agrees on the approaches to take. In the meantime, I might just end up trying my own over the top of JRDF, but that will just to test some ideas. Following that path in my own time will let me do my own thing without being constrained on business issues. Hopefully that will let me get some things done that wouldn't happen otherwise.

JRDFmem
JRDFmem is coming along nicely. I have a BIG list of unit tests to perform, and only a few more to write. I'm hoping to finish them shortly. Once this is done I can get onto my new datastore.

Spent some time considering the structure of the new store today. In the first instance I won't worry about ordering at all. I realised that this won't be a problem unless I try to perform join operations on data. Without a query layer this is quite unnecessary, so I may never need it.

Since the datastore is to have a Java interface, I realised that I should work out what data I need to store in a class-owned struct on the binary side of the JNI. For a start, I'll need to store file handles. Then I realised that separate sessions will need to have their own access to the files, and I didn't want to be overusing file handles! For the index files I'll be OK, as these files will be mapped. For indexes I'll need to store the map pointer, and every instance of the session can use the same pointer.

Normal I/O is a different matter though. I can't just store a new file offset for each session, or else I will get a race for lseek and read/write file accesses between different sessions. This means I either need to lock access with a mutex, or else I use multiple file handles. Locking for file access is not scalable, so I'll go with the multiple handle option. I don't like the idea of restricting the number of objects based on the number of file handles allowable, but since this is just supposed to be small, scalable and fast I won't let it bother me to start with. My other option is to provide a pool of connections, with a mutex protecting access to the pool, though this seems like overkill for the simple system I'm aiming for.

Forgetting, for a moment, all the advantages I was expecting to get with JNI, DM was performing some tests on reading and writing strings to files, using C and Java. It turns out that we have a difference of much more than an order of magnitude, with C the clear winner, though this does vary depending on the platform. I'm now more convinced than ever that I need to create a store written in C.

I'm also starting to think about the new string pool. I need to read up on TRIEs, as it's been a couple of years. I'm curious to see if I can make one work, and how well it will go.

JRDF Transactions
Had an idea about JRDF transactions today. Create a new method on Graph called createTransaction which returns a Transaction object and can throw UnsupportedOperationException. Transaction will then define an interface that extends Graph, and introduces the methods of commit and rollback.

I'll have to run it by AN to see what he thinks. If anyone else has a comment then please email me or add it to the comments below...