Commitments

It's quite difficult to blog while working and moving residence at the same time. I've had some big things happening at work at the same time, so the preference is usually to get those things done rather than write.

On the other hand, I've let slip so many things that I wanted to talk about, that I'm getting frustrated with my lack of blogging. I don't have a lot of time tonight, but I thought I should at least get something written.

FAT_MAGIC

Following a link off Slashdot, I found an interesting discussion of otool in OS X. The author had been looking for an ldd replacement for finding the dynamically linked libraries that a binary uses, which is something I've wanted as well. I just didn't know about otool.

otool seems to be a Swiss-army-knife for inspecting Mach binary files. The array of switches seem quite large, particularly since each function seems quite unrelated to most of the others. It makes the tool feel a little overloaded.

The article points out that the file /usr/bin/crt1.o contains the entry point code for all programs in OS X. I've been interested in the structure of universal binaries, particularly dynamic libraries, so this seemed like a good library to apply otool to.

Looking at the numerous parameters, I discovered that:

  otool -f

and

  lipo -detailed_info

produce nearly identical output. I suppose this makes sense, given that both are looking at the fields in standard Mach binary structures, but it's so close that I have to wonder if they might share some code. If that is the case, then I wonder if any other otool flags duplicate the functionality of other programs?

While looking at all the information available from otool, I discovered that all of the programs I was looking at start with a magic number of: 0xcafebabe. I was shocked at this, as this is the same magic number used by Java class files. Why would they share this value?

I suspect that FreeBSD took this number first with their definition of the Mach file format, and Sun's Java team weren't aware of it when they were defining Java. Or perhaps they knew, but didn't care about a small, non-commercial operating system (little did anyone know what would become of FreeBSD in the future!). So where did FreeBSD get this format from? Did they define it themselves, or did they inherit it?

Whatever the reason, it obviously created conflict. I found a reference to it in the change log for the cctools package in the Darwin project:

Changes for the 5.16 release (the cctools-520 release):
- Fixed a bug that caused otool(1) to crash when it was called with a bad fat
  file (a java class file).  The calls to ofile_first_arch() in ofile.c was not
  checking its return value and later causing the code to crash when it should
  have returned in case of an error.  Radar bug #3670740.

Looking at the code in question, it now looks for bytes at a particular offset to try to avoid mistaking a Java class file for a Mach binary. I don't know if it's justified, but I always feel uneasy when a programmer leaves a note to say that the implemented approach is a hack.

These magic numbers are supposed to help a system differentiate between different file types, particularly executables. Having the same number on two major file types seems like a problem, but it's not one that could be changed now.