Monday, January 24, 2005

I'm sure Andrew will have blogged this years ago, but I just found Jess. This is a Java implementation of the Rete algorithm.

I still don't need Rete for Kowari yet, but I'm wondering if it will work well when considering ABox changes, as opposed to a full inference run like I'm trying to do. Rete is supposed to be good at that, but it makes all of these assumptions about having to process individual statements one at a time. Kowari is not like that, so I wonder if Rete is really the best solution, even for the case of deltas in the data.

I'm expecting to hear what's going on with work shortly. My ideal is work on OWL full time. However, I have to eat (more important, so do Anne and Luc) so I have to consider the options. Kowari/OWL will continue, but it would be a lot easier if I get paid to do it, rather than have to chew into my copious free time!

This leaves me really busy at the moment, making it difficult to work on any of this stuff. At least it has been a break from the usual, which I've appreciated.

In the mentime, I'm off to my brother's wedding in Maui this week. We really can't afford to go (think "extending the mortgage"), but he's not likely to get married again (I hope not anyway!), and besides, how many times do I get to go to Hawaii? :-)

The DSTC asked that I give a talk on Kowari for them soon. I'm looking forward to it, and dreading it at the same time. I'm really out of practice with this stuff. At least it will be good practice for my Masters confirmation.

Speaking of the Masters, my supervisor, Bob, works with the DSTC, so the odds are good that he will attend. This just gets better and better. :-}

Tuesday, January 18, 2005

The Deafening Silence
No, I haven't died or gone away. I've just been really, really busy. I want a job again just so I can slow down! :-)

To those of you who took the time to write to me (for one reason or another), thanks for the support. It's all a bit frustrating, even when you know that it has nothing to do with you.

The last few weeks I've been doing "Daddy Daycare" while Anne has been back at work. It's actually a lot of fun, but I don't get all that much done with Luc around. Fortunately Luc has started at daycare this week, so I've started to catch up on what I'm trying to do. Unfortunately, we were struggling before Luc started at daycare. Now that he's there we are rapidly going backwards again. Hopefully I'll have work again soon.

For a start, I'm writing lots of job applications. Ideally, I want to continue the OWL work (more on that shortly), but if I can't organise something that can pay me soon, then I'll have to take something more immediate that will take me away from it. Hopefully not, but I have to keep my options open.

Ideally, I'll continue on OWL development for Kowari until I have a working, scalable implementation. This should also get me to a point of submitting my Masters thesis. Once I have accomplished both of those, I'd like to use whatever contacts I've made to try to find work in America. I have only done very short working trips overseas before, and I want to get the experience that only a few years abroad can provide.

I'm going to have to take the time to write up a lot of technical details on what I've learnt recently, but for the time being I'll try give an overview.

I'm definitely continuing OWL inferencing work. That is the basis for the Masters, so it is important to me to keep it up. How much time I can put into keeping it up will be determined by the work I find, but I'll be doing it nonetheless.

The Rete algorithm seems to be the best approach I can take to the rules system. I was wondering how far it would get me after having used Drools for RDFS, but I had a conversation at the W3C day of the Evolve conference which made me look at it more carefully. Consequently I found the original paper by Charles Forgy, and the contents were quite enlightening.

For a start, Rete is for managing changing data. That's very interesting for adding and removing data, but it means that it isn't useful for doing a full set of inferences over an entire system. My plan has been to do the full set of inferences, with change management to come later, so Rete would appear to be of little interest to me at this point.

However, a closer analysis of what Rete is trying to do shows that it can offer Kowari something after all. For a start, it makes the assumption that finding and processing data means iterating over all of the data. This is normally the case, but it doesn't apply to Kowari. Because each index is an instance of the data, and because we index in every possible direction, we can find all the data which matches a constraint with a pair of binary searches. We can also count the size of the result quickly and easily by stepping over the blocks in the AVL tree. This means that there is very little cost to finding all the data again.

Rete is only supposed to be useful for changing data because it avoids finding all of the data for each node again. However, if all the data on each node can be retrieved on each iteration, then the algorithm would also be useful for performing the rules on a complete data set. Using semi-naƮve evaluation would also permit the nodes with unchanged results to be avoided most of the time. The result is that a hybrid of the Rete algorithm may allow Kowari to scalably perform all the rules on the entire data set.

I've also worked out that the nodes in this Rete hybrid are not the rules at all. Instead, they are generally the constraints which make up individual queries. The efficiencies of the Rete algorithm mean that any shared nodes will only execute their work once. This will have real benefits to the execution of the full queries within the rules framework, though it will need some re-working of the query code, particularly in terms of caching.

Having a scalable solution operating on the entire dataset will certainly be a first step. It will be inefficient for minor modifications on large systems, but it will still work. Once this is going then, I can move onto the "change" problem. At this stage it looks like a standard Rete graph will perform the rules in these circumstances quite well. It rules data will look similar to the hybrid, but the execution would differ.

Anyway, it's really late now and I'm barely coherent. I'll put this away for the moment and come back to the description when I can actually look at the screen without my eyes closing involuntarily. :-)

Saturday, January 01, 2005

It's been a while since I last wrote. Being near Christmas I was just interested in getting things in order before I took a break, and blogging was just low on the list of priorities. That didn't mean I wasn't working or researching inferencing (I know a lot more about Rete now, plus how I could take advantage of it for Kowari), I just wasn't blogging it.

However, now that Christmas has passed I find myself in a very different position.

For a start, the devestation in southern Asia has been quite large in my mind. The loss of life is so large that it's stunning to think that there was ever a single natural disaster larger than this one (why had no one ever told me about the '76 earthquake in China?). What frustrates me the most is hearing news reports that of the 100,000 or so casualties, about 10 of them were from Australia. I know that the news services just want to report things in a way that people can relate to more, but the way it's put, it sounds like those 10 people somehow rate more highly than the other 100,000.

I'm not a big fan of the media (ask me some time about personal experiences), but this time they seem to be doing some good. Sure, they're exploiting the situation, and they're desperately trying to find individuals who've flown back with personal stories of loss that will give you a lump in the throat, but this has led to people being generous in their donations. Indeed, I think it has been the media's contribution which has forced various governments to increase the level of aid being given.

Speaking of governments, Australia is a small country, and the money we have here is an order of magnitude less than we see overseas. When we spend a million dollars, it's a really big deal. So a $65 million dollar contribution is actually a significant amount, to my way of thinking. I was really pleased to hear it. The fact that big countries (who's budgets are in the trillions) could have donated less really did seem stingy to me. I'm pleased to hear that the US increased the value "tenfold" to $350 million, but it really appeared that they were embarressed into it. I don't know what really went on behind the closed doors, but in politics perception is reality, so they get to wear that particular mantle.

Back home
In our own little lives things haven't been all that happy either, though on a different scale.

Anne's brother, Ian, has been partnered to Amanda Bardon for over 10 years. 3 months ago she woke up with chest pain, which turned out to be a tumour (caused by a skin cancer that hadn't been treated in time). A short sequence of new tumours and operations ensued, and on the 27th she passed away at the age of 34.

Anne and Luc left for the memorial in Melbourne today. We'll miss you Amanda.

After several successful contracts, numerous negotiations in the final stages, and gaining interest in Kowari, one of the principal investors got cold feet. After announcing that all was well on the 22nd, the company was closed down on the 26th (27th in Australia). We get our last fortnightly paycheque, and that's it. With everyone on holidays it took 4 or 5 days before everyone could be told about it.

This is frustrating for several reasons. The first is that no one is working over this period. It has not been possible to speak to anyone to see what my prospects will be like. I'm told that business continues in America over the Christmas/New Year period, but that's not what happens here.

Second, it appears that we will not be getting any entitlements. I spent several years trying to take holidays at various times, but for numerous reasons I was refused. Consequently I've lost quite a bit of holiday time. While I'd have liked to spend it with my family, at the moment the money would have allowed me a little time to look for a job. The Tsunami puts this into perspective, but it's still frustrating.

Another frustrating is that Tucana was the best job I'd had in years, which is why I stayed so long (5 years, including 4 months when I took unpaid leave to go contracting when Tucana ran out of money). We all know quite a lot about RDF and OWL now, and I think we've made something pretty good in Kowari and TKS, but that expertise is all about to go to waste. While some people in Australia are looking at RDF, there are no commercial organisations in Brisbane who have moved that far ahead yet. As a result, everyone will end up working in completely unrelated areas.

From my perspective, I was enjoying learning how to infer for OWL in Kowari. Since my Masters relies upon this I'll be continuing to work on Kowari in my own time. I'll just have to be disciplined as I won't get the opportunity to work on Kowari in my work time.

Working Notes
So what am I doing now?

Well my blog won't be describing my work in the same detail, as I don't think that this is appropriate in many companies. It will keep up with OWL and Kowari, but I won't have the same incentive to write that I used to.

I've stayed in Brisbane to look for work while Anne goes to the memorial in my stead. I have to update my resume today... if I can even find an old copy.

Other than that, I'm trying to keep expenses down (I could only afford a small donation to the Red Cross for the tsunami work), helping a friend renovate and move house, and running around looking for work. I've been feeling a little down, which has led me to not do any triathlon training this week, but I suppose I should start on that again too. I'll have to skip swimming I think, as I can't really afford to be paying pool entry for the moment. I've also avoided using the computer, since I was on a "break", but I'll eventually have to get back to Kowari development too.

If anyone has any ideas, then let me know! :-)