Now blogging at diego's weblog. See you over there!

why meta-structured storage matters

Reading Scot Hacker's foobar blog I found this entry pointing to a cool article on O'Reilly Network's site that talked about Apple's "iLife" apps and the need for them to have an integrated database. He says:

To me, all of these database issues point to a similar need -- find a more efficient backing store for the iApps. The more I ask around, the more it seems that XML is the smoking gun on iLife performance drags - it's a great format for interoperability, but horribly inefficient and resource consumptive. Maybe, just maybe, it's time to reconsider using XML for the iApps. Maybe, just maybe, Apple should consider using some of the highly efficient open source database code out there -- MySQL would do nicely I'm sure.

And since the iLife apps are all so wonderfully integrated now, why not place all of my media in a single, integrated media database? Whether such a database would store media objects themselves (allowing full export to original formats of course) or just references to them (with iTunes-style non-breaking inode references) is unimportant to me. With modern Mac hardware, I should be getting modern media database performance where it counts the most -- when using my Mac as the digital lifestyle hub it's touted as.

When designing spaces, the question of storage was always foremost on my mind. Most applications don't require databases of any sort, but those are usually old apps. New applications tend to store information, index it, and so on, allowing connections between the data to exist either implicit (via soft references through indexed searches) or explicit. When you are dealing with tens of thousands of objects, you need an index, when you have an application with multithreated data access you need recovery, logging, and so on. This means a database, or if you prefer a term that doesn't remind you of a 1.5 Gb Oracle 9i installation, meta-structured storage ("meta" since the storage system imposes new structures, such as indexes, upon the original structure of the data). New applications today try to leverage connections between pieces of data, and this will only grow as the amount of data that we store grows. Using a database on a consumer product sounds scary, since most of our experiences with databases come from the corporate world, where databases usually are a nightmare to install and require an army to maintain, but it needen't be a problem if the storage system is properly designed. Many products are based on databases today, and most users don't even know. Whether the data is stored in the database or not is irrelevant, as Scot says in his article. What matters is that the program supports a fast, reliable mechanism to provide access to that data.

Thousands of digital photographs and MP3s. Tens of thousands of emails. Tens of thousands of webpages. Hundreds of documents and images. Hundreds of appointments and contacts. Hundreds of notes. What more can I say: I think that in the future meta-structured storage integrated within applications will become the norm, rather than the exception.

Categories: technology
Posted by diego on February 2, 2003 at 11:49 PM

meta-meta tagging

Russ goes has a good set of pointers today about the different metatags that are emerging for websites, for weblog configuration, RSS, geographical location, etc. He mentions the need for meta-meta tags. I agree. I don't think is has to end up being something as generic as RDF. An expansion of RSD would probably be enough. Food for thought. Yum!

Categories: technology
Posted by diego on February 2, 2003 at 7:35 PM

some more coverage on the shuttle

Obviously it was all over the news, but I found the New York Times' coverage to be pretty good. In their lead article they go over possible causes and mention something that I thought was interesting:

The space agency, which spent tens of millions of dollars improving safety after the Challenger accident, has estimated the risk of a calamitous event on re-entry as 1 in 350.
Now, if that's the estimate (I imagine based on a lot of guesswork, since a re-entry event implies a catastrophic failure of some kind) then considering that this was the 113th flight of the shuttle, it doesn't seem to be terribly off the mark. It does "feel" a bit low (specially considering that the original plan was to fly it once a week at least, then at that rate of failure you could expect one "calamitous event on re-entry" ever six years or so. Not good.

A lot of the coverage has focused on the Israeli astronaut on board. This article has a paragraph that caught my eye:

In a twist of nomenclature that would seem plausible only in fiction, a craft carrying Col. Ilan Ramon of the Israeli Air Force apparently broke up near an East Texas town called Palestine.
Blind coincidence has a way of making us feel strange, doesn't it? I mean, a town in Texas called Palestine...

William Gibson also has an entry on his blog to which I relate a lot.

Another article talks in more detail about the possibility that it was a failure in the heat shield tiles, which I assume was what was on most people's minds when they found out about it. I've seen many news reports today going over the much-mentioned "lift off incident" where foam debris from the external tank below the orbiter when the shuttle left a little more than two weeks ago hit the left wing, and it's incredible that they are (simply by repetition) "convincing themselves" that this was the cause. It's unlikely that it will be just that. At a minimum, a series of things must have gone wrong. It might be something else altogether. Too bad this time Richard Feynman is not around to straighten things up for all those bureaucrats. (There is an excellent account of his role in the Challenger disaster investigation in his book What do you care what other people think?).

This is probably my last entry on the Columbia for a while. It's sad, and tragic, but we humans have a bad tendency of focusing on dramatic events in places that give them enough coverage. Just yesterday, as the first reports of the Columbia were coming in, a bomb exploded in Lagos, Nigeria, killing dozens of people. And yet hours after that it was clear what had happened to the shuttle was a catastrophic failure and all the astronauts had perished, hours after it was clear that little could be said that hadn't already been said (or speculated out loud), coverage of the bomb in Africa in non-US news was limited to (literally) a three-second bit of "By the way, a bomb exploded in Nigeria. Dozens are feared dead." Coverage in supposedly more "international" US-news stations like CNN was nil. I assume that in part it's also a matter of expectations and where we put our dreams. Technology plays its part too, we naturally gravitate to its origin (namely, the US). But it always gives me a deep sense of unease that we seem to worry more about the fate of certain people than that of others. Not that I'm immune to that, I'm just aware of that in myself (and certainly on the news) and I don't like it too much, and I don't even know if it could change.

Anyway, that's a different topic. We now return to our regularly scheduled programming.

Later: Just saw Russ has an entry talking about similar things. I agree that the media is a big driver behind this. But there's also something deeper. Before Big Media existed people still felt affected when, say, the King died, but barely lifted a finger when their neighbor passed away, as it was 'a fact of life'. This ties in with what I was saying a couple of paragraphs above about expectations, dreams, etc. Hopefully over time we'll learn to find some balance...

Categories: technology
Posted by diego on February 2, 2003 at 7:21 PM

Copyright © Diego Doval 2002-2011.