Thursday, March 31, 2005

Testing
Testing on OS X is still impossible. The JVM always ends up dying somewhere in the process. I don't know any way around this, so for the moment I have to run all my tests on a Linux server. That's a little annoying, as I need to manually synch the files between my PowerBook and the server.

I'd love to find a small subset of tests that will run just the "store" tests, but for the moment I can't find it. At the moment I'm running the full suite in a scrollable window, so I can try and work out a containing target that might do the job for me. I haven't been able to do that before now, because I've used "screen" on the Linux server, and that loses the whole output buffer. And anyone who has ever looked inside the build.xml (and package build.xml files) from Kowari will know that it is not possible to work out what the necessary targets are for a single set of tests.

The test code also ended up more complex than I expected. This was because I hadn't really thought about every possibility for the data.

To start with, I wanted to check when the variables in the minuend and subtrahend had a null intersection, and also when they had intersections of 1 and 2 variables (and hope by induction that 3 or more would also work). I also wanted to check when the intersection equaled the minuend, and equaled the subtrahend, and then equaled both.

Once these parameters were set up, I then needed to consider the bindings. I needed to check when individual statements are removed, runs of statements are removed, statements at the beginning are removed, and statements at the end. I also need to check prefix matching on data that exists and does not exist, for most of the above conditions.

Finally, I ran into an interesting problem while comparing these tests to the UnboundJoin tests. This code makes a lot of use of unbound variables and the effect that they have on the results. I've tried to include several such tests for these as well, though their validity is going to depend on how well prefix matching works on the subtrahend.

Anyway, I've spent some of Tuesday, all of Wednesday and part of Thursday writing these tests. Last night and today has been spent fixing some of the bugs shown up, but I still have a few to go. While I'm still stuck running the full test suite on a remote server I'm spending a lot of time just waiting for it... hence this entry. I can't continue having a turnaround at this pace though, so I hope to improve the process on the next iteration.

No comments: