Now blogging at diego's weblog. See you over there!

why meta-structured storage matters


Reading Scot Hacker's foobar blog I found this entry pointing to a cool article on O'Reilly Network's site that talked about Apple's "iLife" apps and the need for them to have an integrated database. He says:

To me, all of these database issues point to a similar need -- find a more efficient backing store for the iApps. The more I ask around, the more it seems that XML is the smoking gun on iLife performance drags - it's a great format for interoperability, but horribly inefficient and resource consumptive. Maybe, just maybe, it's time to reconsider using XML for the iApps. Maybe, just maybe, Apple should consider using some of the highly efficient open source database code out there -- MySQL would do nicely I'm sure.

And since the iLife apps are all so wonderfully integrated now, why not place all of my media in a single, integrated media database? Whether such a database would store media objects themselves (allowing full export to original formats of course) or just references to them (with iTunes-style non-breaking inode references) is unimportant to me. With modern Mac hardware, I should be getting modern media database performance where it counts the most -- when using my Mac as the digital lifestyle hub it's touted as.

When designing spaces, the question of storage was always foremost on my mind. Most applications don't require databases of any sort, but those are usually old apps. New applications tend to store information, index it, and so on, allowing connections between the data to exist either implicit (via soft references through indexed searches) or explicit. When you are dealing with tens of thousands of objects, you need an index, when you have an application with multithreated data access you need recovery, logging, and so on. This means a database, or if you prefer a term that doesn't remind you of a 1.5 Gb Oracle 9i installation, meta-structured storage ("meta" since the storage system imposes new structures, such as indexes, upon the original structure of the data). New applications today try to leverage connections between pieces of data, and this will only grow as the amount of data that we store grows. Using a database on a consumer product sounds scary, since most of our experiences with databases come from the corporate world, where databases usually are a nightmare to install and require an army to maintain, but it needen't be a problem if the storage system is properly designed. Many products are based on databases today, and most users don't even know. Whether the data is stored in the database or not is irrelevant, as Scot says in his article. What matters is that the program supports a fast, reliable mechanism to provide access to that data.

Thousands of digital photographs and MP3s. Tens of thousands of emails. Tens of thousands of webpages. Hundreds of documents and images. Hundreds of appointments and contacts. Hundreds of notes. What more can I say: I think that in the future meta-structured storage integrated within applications will become the norm, rather than the exception.

Categories: technology
Posted by diego on February 2 2003 at 11:49 PM

Copyright © Diego Doval 2002-2011.
Powered by
Movable Type 4.37