Now blogging at diego's weblog. See you over there!


Looking for Java Web Start-related things I found Lopica. Tons of information on JWS and related things. Very, very useful.

Posted by diego on November 28, 2003 at 11:32 PM


Don Park has released BLogStAX, an StAX-based RSS 2.0 parser. Very cool.

Posted by diego on November 28, 2003 at 2:24 AM

ACM: an interview with MSN Messenger's chief architect

I got this on one of those ACM pesky subscriptions that I end up never unsubscribing from (sometimes they have something useful as well!). Read it. Thought about blogging it. Promptly forgot about it. Luckily, Slashdot reminded me today.

So, without further ado: A Conversation with Peter Ford, MSN Messenger's Chief Architect. Very, very interesting. Going over most of the main topics that relate to IM, SIP/XMPP, P2P, voice... excellent article. Particularly interesting that he ties people liking IM to the inherent "feature" of whitelisting in it, but of course past that you have the problem of bootstrapping the connection. I think that properly implemented digital identities would go a long way towards solving some of the problems he points out. I find it interesting that this solution hasn't been discussed much, although I have no doubt that it has crossed the mind of anyone who works with IM at least once.

Categories: technology
Posted by diego on November 27, 2003 at 3:01 PM

comments, comments, comments

No doubt I'm going to forget about some of the things I wanted to comment on, but here are a couple.

First, Scott in a funny-titled entry Diego, Diego, Diego or "A Conspiracy Theorist's View of WinFS" or Scott Supports Microsoft comments on my views on WinFS, Cairo,

[...] but isn't the answer here, what it always is, $$$? I mean if you are a product manager on OS stuff at Microsoft, you're not only concerned about the current release of the OS but, just as much, the next release
Actually, when I talked about "lack of resources" in my previous posts I meant exactly what Scott is saying. They have admitted as much at least regarding security:
Microsoft is considering charging for additional security options and acknowledges that it didn't move on security until customers were ready to pay for it.
So I actually don't take a conspiracy theorist's view (although maybe Scott was saying I'm not pro or con, but "other" :))---It's all about resources (in the end, money, as Scott says). Sometimes though, money is not a factor---or does anybody doubt that MS was ready to spend whatever it was necessary to stop Netscape? With WinFS and things of that nature, there is no inherent threat to which they're responding, which theoretically means that it might not be released, as has happened many times in the past (and no, Google doesn't qualify, in my mind, for something that spurs WinFS development, although how Google affects the resources of MS's new Search effort is another matter--Pure search and "the quest for metadata" are related, but separate). On the other hand, at some point the engineers would get anxious to get this out so I think the odds are on WinFS's favor. :)

Related to this, Jon has some interesting comments to an entry by Dare on the relationship between XML, Databases, and the applicability of different tools for different problems.

Second, Patrick, regarding my "as we may think" post, said:

[Diego] says:

Two of the most influential people in the history of computing have been, without a doubt, Ted Nelson and Doug Engelbart.

Big call. They were certainly infuential in the history of the internet, but in computing? I wouldn't put them in the same ballpark as Alan Turing or Claude Shannon. Would you?

I think I didn't say what Patrick read, but just to make sure: I said "two of the most influential" not "the two most influential". Partly, the difference is because Patrick considers the Internet as separate from computing. Which is a valid view. But I think that the Internet, Networking, GUIs, etc, etc, are central to computing as we know it today (In my mind, for example, people like Bill Joy or Tim Berners-Lee were also highly influential, though for more "practical" reasons). They certainly wasn't important in 1950--they didn't exist. But the same can be said of many things, including databases, microprocessors, and even stuff that we take completely for granted, such as rasterized displays (for a while, Vector-based displays were hot stuff). All of these ideas are intertwined over time, crossing back and forth and building on each other. But of course, in the end the further back we go the shorter the list of "influential people" becomes, and the more theoretical it becomes: As Patrick mentioned Shannon and Turing, I could add a few others: Von Neumann, Shockley, Eckert, Mauchly, Wiener, McCarthy, Minsky... ok, I'll stop. That's not the point :) Patrick's comment did lead me down memory lane in other directions and made me realize that I implicitly consider computing to be broader than other (maybe most!) people.

Update: The comment I attributed to Patrick was actually from Justin (here) and Patrick was just quoting it. Thanks Justin for noting it--sorry about the misquote.

Categories: technology
Posted by diego on November 27, 2003 at 2:48 PM

the antidote... know, for when the night is going down for whatever reason...

U2 Live From Slane Castle, recorded 1 September, 2001. The boys might be Southsiders now, but they've still got Northside painted on their soul. Nice combination. (You wondering where that came from? Me too. But what the hell.) :)

And I still haven't replied to those comments. Damn. Blog procastination.

...We're gonna go over to London... ...and we're gonna score ourselves a record deal... ...And when we get our record deal... ....We're not gonna stay in London... we're not gonna go to New York City... ....We're gonna stay and base our crew in Dublin! .... ...'cause these people... ...THIS IS OUR TRIBE!...

I've got big ideas, I'm out of control!

Yes, this has got to be the best version of 'Out of Control' I've ever heard.

Ok, enough with the Momentary Lapse of Reason (hey, that sounds like a good follow-up). Back to worky.

Later: And, btw, what is up with 'Where the streets have no name' giving me goosebumps every single time I listen to it performed live? Some switch failing in my head, I'm sure.

Categories: personal
Posted by diego on November 27, 2003 at 12:23 AM


Yesterday, late at night, on a break, I suddenly remembered this short story I wrote a while ago. A few months back I printed it out and made some corrections, then put it away. It resurfaced on Monday after I put some order to the living room (sometimes I can't believe how messy things can get on their own). It resurfaced, but I left it aside, and forgot about it again for a few hours.

Then it's on my mind again, and I open the document and start correcting it (in automatic mode). I stop for a moment. Remember the corrections I've already made. Go look for the paper copy. Changes I've just made are the same as those I did months ago, as if whatever is there of the story is already there and time won't change it (this has happened before, a number of times). Anyway, I finish with the changes, then look at the time. A little past one. I wonder. When did I write this? I check out the versioning information on the document. Created on 24 November, 2000, and once I substract the timezone difference (I was on PST when I wrote it) I realize it's been three years, exactly. To the hour.


Then later today (yesterday) I decide I want to watch a movie. I rent 25th Hour, incredibly good. Too good maybe. I'll have to see it again to confirm I wasn't hallucinating. Set in Manhattan, post-September 11. Some of the themes in the movie reminded me of the story (after having set it aside, again). Or the general feeling. Or (more likely) I fancied they did.

In case you want to take a look, here is the story.

That aside, there are a number of other comments to previous posts that I wanted to follow up on. I'll leave those for later today.

Categories: personal
Posted by diego on November 26, 2003 at 1:40 AM

rethinking the Internet, part one: "as we may think"

"The summation of human experience is being expanded at a prodigious rate, and the means we use for threading through the consequent maze to the momentarily important item is the same as was used in the days of square-rigged ships."

from As We May Think, Vannevar Bush, July 1945.

(First of an intended "loosely coupled" series of posts on the history of the Internet and revisiting some of its concepts and the seminal works that made it what it is today.)

Two of the most influential people in the history of computing have been, without a doubt, Ted Nelson and Doug Engelbart. Nelson, in defining with relative precision ideas that we use today (such as hyperlinks) and Engelbart leading a team that created basically everything that we use today. Nelson's ideas didn't beyond prototype stage, and Engelbart's and his team's work had to wait until they were revisited (in many cases by the same people) at Xerox PARC and through it exposed to the wider world via Apple and then Microsoft. (Which ideas? Let's see: The Mouse. Wireless Networks. LANs. Windowing systems--for starters!).

Both Nelson and Engelbart were influenced by Vannevar Bush's "As We May Think". Engelbart read Bush's article in a copy of the Atlantic Monthly he found at a Red Cross library in Manila, while waiting for his transport home after World War Two. Nelson went as far as reprinting the entire article in his first book, Literary Machines. I'll come back to Nelson's and Engelbart's in another post---For now on to Bush's article.

The last time I read "As We May Think" was about two years ago. Back then, I noted that Bush was embedded in an "analog worldview" but not much else (He was an analog kind of guy: the computer he worked on, the Rockefeller Differential Analyzer, was an analog computer that helped in calculations of artillery firing tables and radar antenna profiles--and promptly rendered obsolete with the arrival of digital computers). The ideas were great, but limited because of that. But in re-reading it this time I noted something else: Bush was describing a digital system, without digital technology.

There's a difference between a) "thinking analog" and b) being trapped by the limitations of analog technology. And Bush seems to be on the second category. This wasn't apparent to me the last time I read the article. Once you shift your view a little from "analog" to "trapped by analog" the ideas he discussed become even more astonishing. Take, for example, digital photography.

Will there be dry photography? It is already here in two forms. When Brady made his Civil War pictures, the plate had to be wet at the time of exposure. Now it has to be wet during development instead. In the future perhaps it need not be wetted at all. [There have long been films ... which form a picture without development, so that it is already there as soon as the camera has been operated ...] The process is now slow, but someone may speed it up, and it has no grain difficulties such as now keep photographic researchers busy. Often it would be advantageous to be able to snap the camera and to look at the picture immediately.
Bush goes back to this idea of "dry photography" a few times more. The problem in the article (if it can be called a problem) is that he tries to imagine the device constraining himself with the technology available at the time and in some cases an order of magnitude better in one or more axis of improvement (such as resolution of the film). But as I read it, what I saw more and more was someone saying "we need to get film out of the way. Send pictures directly into the machine's memory". In this case, the machine memory was also film (microfilm) and so it's a vicious logical circle.

But replace all the analog ideas with digital. He was talking about digital photography, albums, and massive databases that would contain them forever. We do this today without thinking twice about it. Had Bush not constrained himself by the technology available on his day, describing only function "As We May Think" would doubtlessly be considered a lot more than what it is today.

This technology constrain appears again and again. When Bush talks about compression, he is describing size-reduction on the microfilm, building on increased precision on light reflection techniques. When he talks about Voice I/O

At a recent World Fair a machine called a Voder was shown. A girl stroked its keys and it emitted recognizable speech. No human vocal chords entered into the procedure at any point; the keys simply combined some electrically produced vibrations and passed these on to a loud-speaker. In the Bell Laboratories there is the converse of this machine, called a Vocoder. The loudspeaker is replaced by a microphone, which picks up sound. Speak to it, and the corresponding keys move. This may be one element of the postulated system.
he is similarly trapped by analog technology. But all the ideas are there.

Nearing the end of the article, Bush puts together all of his concepts into his now-famous Memex device:

Consider a future device for individual use, which is a sort of mechanized private file and library. It needs a name, and, to coin one at random, "memex" will do. A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.

It consists of a desk, and while it can presumably be operated from a distance, it is primarily the piece of furniture at which he works. On the top are slanting translucent screens, on which material can be projected for convenient reading. There is a keyboard, and sets of buttons and levers. Otherwise it looks like an ordinary desk.

In one end is the stored material. The matter of bulk is well taken care of by improved microfilm. Only a small part of the interior of the memex is devoted to storage, the rest to mechanism. Yet if the user inserted 5000 pages of material a day it would take him hundreds of years to fill the repository, so he can be profligate and enter material freely.

Most of the memex contents are purchased on microfilm ready for insertion. Books of all sorts, pictures, current periodicals, newspapers, are thus obtained and dropped into place. Business correspondence takes the same path. And there is provision for direct entry. On the top of the memex is a transparent plate. On this are placed longhand notes, photographs, memoranda, all sorts of things. When one is in place, the depression of a lever causes it to be photographed onto the next blank space in a section of the memex film, dry photography being employed.

A central point of the Memex is that it records any kind data. Text. Images. Sound. In his view, everything would be reduced to microfilms, which would then be manipulated by a mechanical system.

The only difference is that storage happens on analog devices.

Even the "hyperlinks" (a term coined by Nelson). Everything that Bush describes there is basically what we do today.

Bush describes the concept of "trails" which are constructed through links on the microfiche:

When the user is building a trail, he names it, inserts the name in his code book, and taps it out on his keyboard. Before him are the two items to be joined, projected onto adjacent viewing positions. At the bottom of each there are a number of blank code spaces, and a pointer is set to indicate one of these on each item. The user taps a single key, and the items are permanently joined. In each code space appears the code word. Out of view, but also in the code space, is inserted a set of dots for photocell viewing; and on each item these dots by their positions designate the index number of the other item.
As I read the article this time, all I could think of was "Arrgh!! You're so close! Come on, say it! Digital! Digital!"

But he didn't say it. :)

Regardless. There are two things that I took away from this re-reading.

The first is that analog technology is vastly underrated. The Memex is crearly a device that can work exactly as Bush described it, particularly with today's micro- and nano-scale technology. (most of which, paradoxically enough, has been driven basically by digital technology). Of course, I'm not saying that we should go back to analog, but we barely think about it anymore. Are we missing something because of that? One problem with analog is that unless you're both a physicist and a computer scientist you're not likely to go anywhere. Digital allows us to separate the domains more clearly. You give me the chips, I'll write some cool software for it. That distinction would be less strong (or even disappear) with analog technology.

The second, and more important is: re-evaluating old ideas is useful. Who knows what else we are missing? Even if it's quite possible that the great ideas live on, the viewpoint that a different kind of technology gives you is priceless. Sometimes there are simpler solutions to things--we just can't see them because we know too damn much. Why use a nail and a hammer when a particle accelerator will do? :) (Another problem is that it's much cooler to ask for funding for particle accelerators than for boxes of nails).

Problem is, there's too much information, and even though our retrieval and correlation capabilities have increased, the amount of information itself has grown even more, putting us back into the "days of square rigged ships" that Bush mused about. We are still some way off the Memex vision, but we're closer. When we get there, it will certainly be worth spending some time using it to wade through the historical and scientific record once more to see what viewpoints and ideas can grow out of the past and help us build new visions of the future.

Categories: technology
Posted by diego on November 24, 2003 at 1:28 PM

Cairo (and WinFS) revisited

Interestingly enough Jon Udell, whom I mentioned in my post about Cairo/Longhorn the other day, was writing up a great article with similar ideas for InfoWorld, including his own take on Cairo, Longhorn, and what the web is and how MS can affect it:

[...] the Web is much more than the browser. It's an ecosystem whose social and information structures co-evolve. Innovation bubbles up from the grassroots; integration can happen spontaneously; relationships cross borders. Cairo Version 1 wasn't designed to nourish that ecosystem or to flourish in it. Let's hope Microsoft remembers the past and avoids being condemned to repeat it with Cairo Version 2.
Agreed 100%.

In an entry on his blog he points to my own post, and talks about the history of how he took it on himself to maintain Byte's archives, and some of the problems of maintaining "digital continuity." (I've been thinking about that too--more on soon).

Then, in comments, Ole pointed to X1 which does full text search on Windows PCs as a similar idea to WinFS, Longhorn's new filesystem et. al. I have tried X1 and found it interesting, but it freaked me out by showing up all the time and being surprisingly difficult to remove. So I stopped using it. Others might have a better experience though :)

Finally, Robert replied to my post (mis-spelling my name once again--by now this has all the makings of an in-joke ;-)), saying that WinFS vanished because

I've heard that this is the fifth time that we've tried something like WinFS, but previous tries never got out of the lab. Why not?

Simple: customer testing. Whenever we come up with an idea, we have real customers go into a lab here and try out the product. If the performance, or the UI, or something else keeps it from being useful it doesn't get released.

Hm. This explanation is not complete. Suppose you show it to users. They hate it. What do you do then? Ditch it, or go back to the drawing board? Well, it depends (as I said on my post) on resources. With a company of MS's size and resources, the only thing that explains ditching WinFS is a change of priorities. Otherwise you keep at it until it works. (And, again, I refuse to entertain the notion that "it couldn't be done"--especially when others have done it!) I'll stick with this idea until I am convinced otherwise. :)

This still doesn't explain what happened to all that code either, or what, exactly, didn't work back then. Was it the speed? (Ole, btw, has been asking about that issue for a while now, and hasn't seen a good reply for it AFAIK). And that aside, the fifth time?? Really??? If that is true, it's a scary thought. The technology is definitely doable. I certainly can't believe (as I said) that Microsoft can't do it, so it sounds like there's a lot of resource-shifting going on. Weird.

Categories: technology
Posted by diego on November 23, 2003 at 4:56 PM

JFK: 40 years later

I'm wondering if I should start a category for "history"... :-)

Early this morning I was reading more of The Sword and the Shield, The Secret History of the KGB (that I've mentioned before, for example, here) and, what do you know, I get to a chapter that deals largely with the KGB's "active measures" (disinformation, black ops, etc) against the "Main Adversary" (the US). I finish reading the chapter and later I realized that today was the day. Weird.

Anyway, then I'm reading news, and of course today there was a lot of coverage on JFK all over the place. The New York Times has a special section with many articles and tons of information, particularly interesting is the note that the event signaled the arrival of TV as the most powerful medium, something that is not obvious now but scratching a little under the surface becomes quite impressive: round-the-clock coverage, more than 72 hours of live TV with no interruptions, all networks cancelled their programming---capturing not only everything that happened right before and after the shooting (though, in a nod to conspiracy theories, not the shooting itself!), to the killing of the suspect, Oswald, to the burial. Now we don't think twice about it, but TV before November 22, 1963, was a whole different universe.

Back to topic, over at the Washington Post I see an editorial that deals with the role of the KGB on conspiracy theories and such, and sure enough the information from The Sword and the Shield features prominently in it. Example:

Five months later, in June 1964, a freelance journalist named Joachim Joesten posited a strikingly similar analysis in his book "Oswald: Assassin or Fall Guy?" Following a chapter on "Oswald and the CIA," Joesten asserted that the agency was beyond presidential control and bitterly opposed to Kennedy's policy of "easing the Cold War." It has long been a matter of record that Joesten's book was the first published in the United States on the subject of the assassination. Until the notes of a former KGB archivist named Vasili Mitrokhin were published in 1999, however, it was not known that Joesten's publisher, the small New York firm of Marzani & Munsell, received subsidies totaling $672,000 from the Central Committee of the Communist Party in the early 1960s.
A lot of the information in the article is a direct quote of the information in the book, even though it does not mention the book explicitly. (Example, there's a reference to a document cited by Boris Yeltsin in his 1994 memoirs which is also referenced in the book.)

Now, this defense of the CIA by the Washington Post is fine (notwhistanding that I've seen interviews of many people, including reporters, that were there when it happened, that have mentioned that the idea of a conspiracy or a "coup d'etat" had run through everyone's minds at some point immediately when they heard of the shooting). But what is really interesting is what the editorial doesn't mention: that the KGB actually, truly, really believed that the Kennedy assassination had seen some CIA involvement:

The KGB reported that a journalist from the Baltimore Sun "said in a private conversation in early December that on a ssignment from a group of Texas financers and industrialists headed by millionaire Hunt, Jack Ruby, who is now under arrest, proposed a large sum of money to Oswald for the murder of Kennedy." Oswald had subsequently been shot by Ruby to prevent him revealing the plot. Khrushchev seems to have been convinced by the KGB viwe that the aim of the right-wing conspirators behind Kennedy's assassination was to intensify the Cold War and "strengthen the reactionary and aggressive elements of American foreign policy."

The choice of Oswald as Kennedy's assassin, the KGB believed, was intended to divert public attention from the racist oil magnates and make the assassination appear to be a Communist plot. The Centre [KGB Headquarters] had strong reasons of its own to wish to deflect responsibility for the assassination from Oswald. It was deeply embarrassed by the fact that in 1959 Oswald had defected to Russia, professing disgust with the American way of life and admiration for the Soviet system. Initially the KGB had suspected that he might have been sent on a secret mission by the CIA, but eventually concluded that he was an unstable nuisance and were glad to see the back of him when he returned to Texas with his Russian wife in 1962.

There is more mentioned afterwards that details how the KGB's own records show that they were suspicious of Oswald and his motives for defecting first and then attempting to re-establish contact with CPUSA (the US branch of the communist party) later.

So. None of this means, obviously, that there was such thing as a "CIA plot". But it does mean that the intentions of the Soviets were, quite likely, a lot less sinister than that the editorial from the Post describes. In large part, the KGB and by extension the Soviet Union where putting out information that would pre-empt a possible attempt to link them to the assassination. They saw themselves as being set up along with Oswald, and everything in that particular conspiracy theory fit their own paranoid view of the US (which I described in the previous post about the book, linked above).

More importantly, (and this is a matter of public record as well) the book describes how Khrushchev and Kennedy had established "backchannels" of communication (which were used extensively to resolve the Cuban Missile Crisis in 1962). The Soviet Union knew that they were outgunned (the "missile gap" that Kennedy touted so effectively for the 1960 elections was in fact working to the US's advantage---the US had at that time ten times more intercontinental ballistic missiles than the Soviet Union), had hints that American military leadership was working on "first strike" plans and, in keeping with their pervasive paranoia, thought that the assassination was an "obvious" attempt to get Kennedy "out of the way". The strong possibility that Kennedy was thinking of withrawing from Vietnam starting in late 1963, with or without victory, and whiffs of that information only added the icing on the cake for the Soviets. (While the idea that Kennedy had decided to withdraw from Vietnam is heavily disputed, there's a couple of interesting articles on this here and here, and Robert McNamara himself, Secretary of Defense at the time, has said publicly that this was the case).

Since, in fact, there have been suggestions that Johnson pressured the Warren Commission to come up with a simple, "lone gunman" explanation for fear that anything else would lead to world war three, we could say that at a minimum the Soviets were not alone in their paranoia. Regardless of whether those suggestions are true or not (there's not a lot of evidence as far as I'm concerned) I think that anyone with half a neuron firing would see that this was a distinct possibility given the tension at the time, and that, had LBJ actually done that, it would not have been irrational of him to do so.

I find it interesting to see how far we can keep peeling layers on this. The KGB believed in a CIA plot. It spreads (dis)information to that effect. Then others react, saying that it was KGB disinformation, sometimes trying to cover their tracks, which spreads even more disinformation. Then the KGB tries to cover its tracks...

But who killed Kennedy then?

I'd respond: Does it matter?

What comes to mind is a small piece of dialogue in V for Vendetta written by Alan Moore, which is in my opinion one of the best graphic novels ever written. The main character, V, pulls a gun out of a would-be killer's hands, and says:

Did you think to kill me? There's no flesh or blood within this cloak to kill.

There's only an idea.

Ideas are bulletproof.

And isn't that what really matters?

"[...] in the final analysis, our most basic common link is that we all inhabit this small planet. We all breathe the same air. We all cherish our children's future. And we are all mortal."

John F. Kennedy, June 10, 1963

Categories: geopolitics
Posted by diego on November 22, 2003 at 5:35 PM

tony blair and the simpsons

and this from the we-don't-want-you-making-any-comparisons dept:

homer and tony blairA few months back I read that Tony Blair was going to guest-star in an upcoming Simpsons episode. As it turns out, the episode is premiering in the US this week, and yesterday The Times of London had a cover story on how 10 Downing Street flexed its lobbying muscles to stop the episode from airing coincidentally with Bush's Sate Visit: The US Visitor Blair didn't want us to see -- yet. Quoted from the article:

DOWNING Street really did try to prevent an oafish American causing trouble on a visit to London.

Tony Blair, a longstanding fan of The Simpsons, recorded his dialogue for an episode of the cartoon series at Downing Street in April.

But the episode, entitled The Regina Monologues, has been the subject of intense negotiation with Fox TV, which has claimed that No 10 prevented any tapes being released before George Bush’s visit to avoid any possible embarrassment.

Aides denied this but confirmed they had requested that the show should not be trailed in advance because “we didn’t want a lot of hoo-ha about it”.

Alastair Campbell, Mr Blair’s former communications director, is understood to have vetted the script and suggested a number of changes.

In one draft, Homer announces: “I’m going to act the way Americans act best — unilaterally.”

He then breaches security to crash a Mini Cooper through the gates of Buckingham Palace, catapulting the Queen out of her horse-drawn carriage.

(my emphasis) Heh. Too funny. I wonder if they will avoid the comparisons anyway. I think it's likely, given the astonishing short memory and attention span of the media these days.

Posted by diego on November 21, 2003 at 4:32 PM

fall is beautiful

No question. Fall is my favorite season, especially if it's not too cold. :) Just for the colors... a certain warmth. Like the world cuddling under the covers, ready to sleep.

Dublin in the Fall reminds so much of New York it's scary. Makes me miss it a bit. Strolls across Central Park, drives up the Hudson Parkway, or walking down Fifth or Third, the city seemingly extending forever (especially when looking South). Sometimes I think of Dublin as a 1/10th model of New York for some reason (or is New York a 10:1 copy of Dublin rather?). Phoenix Park is way bigger than Central Park though, and it's impossible to beat the combination of trees surrounded by skyscrapers, but the woods and ponds and trails make up for it with natural beauty. It's like two sides of the same coin.

Here are some of the pictures I took last weekend (click to see a larger version). Is that sky unreal or what?





Categories: personal
Posted by diego on November 21, 2003 at 1:27 PM

will I stop with Microsoft? and other things

Short answer is, yes. :) Even though I'm fascinated by all of this (When I put on my history buff hat, almost everything is fascinating, but this more than many other things!) I think I will let it rest for a while. I'll wait and see before saying more. (And I think that last post drained me out a little :)).

That aside, tonight I took a break to watch (on TV) Remember the Titans. Like with Any Given Sunday, the movie is still effective for me even though I have absolutely no idea of what is going on with the game. I don't understand the plays, the code words, I don't understand anything at all. "56! 7! 56! Huh! Huh! Huh!" or whatever. That kind of thing. I have watched parts of actual games and I find them absolutely, stunningly boring. It's like watching a Sumo match, a lot of jockeing around for position, a lot of posturing, and bursts of furious and violent activity. But on movies, American Football looks great (when it's well done of course). I understand that it's a game with a heavy strategic and tactical tilt, and probably that helps--kind of like a movie about war. Regardless, I was repeatedly amazed at how this happened (of me not understanding anything and still somehow relating).


I still haven't posted the pictures I took on Saturday. I've been looking at them, there's a couple I really like. Tomorrow then (why not now? Because I don't want to start checking JPEG compression sizes at the moment).

I should also be blogging more about what I'm doing, what's going on with clevercactus and so on. Mea culpa. I find it difficult to discuss things mid-process for some reason. Nothing comes out. It will come in time I guess.

Now to read a bit, and then a brief slumber.

Categories: personal
Posted by diego on November 20, 2003 at 1:00 AM

on Microsoft: a walk down memory lane (aka wading through the Byte archives)

"But it may be that although the senses sometimes deceive us concerning things which are hardly perceptible, or very far away, there are yet many others to be met with as to which we cannot reasonably have any doubt, although we recognise them by their means. For example, there is the fact that I am here, seated by the fire, attired in a dressing gown, having this paper in my hands and other similar matters."

Rene Descartes, Meditations On First Philosophy, 1641.

Wow! Quoting Descartes! This must be good!

Actually, that was probably the high point of this post. :) But it does tie in with what I wanted to talk about, it's not that I engage just for kicks in quoting philosophers with whom I don't agree at all.

Besides, I am not sitting by the fire (the warm glow of the LCD doesn't count, I'm sure), and I am not attired in a dressing gown (now that's a thought! Who needs WiFi or distributed object systems? Dressing gowns for all! Forget high tech!).

Boy, am I a riot tonight.

What was the point again? Oh, right. Longhorn, Microsoft, and that other magic word that for tech people, for a short period of time, became more than the name of the Capital of Egypt and started to embody The Future (in Technicolor). Those were the days, when Microsoft OSes were named after cities--remember Chicago and Daytona? (Win95 and WinNT 3.5 respectively.) How about Memphis? (Win98). And by the way, since Cairo ended up being NT 4, Memphis clearly did not point the State of Tennessee, to the city of the Blues and Civil Rights struggles in the 60's, but rather, it was a reference to the ancient Capital of Egypt. Memphis and Cairo, the old and the new. City names were cool.

Certainly better than Longhorn. I mean, come on!.

(Yes, I anticipate running for cover when people start explaining what's in reference to--I'm sure there's a reason).

Anyway, it's not about the names, although that's kind of interesting. This is about the technology (or I think it is), as I'll try to explain below.

Through the afternoon I kept thinking about the long-winded history of all the "innovation" that will be showered on us in a couple of years when Longhorn is released, particularly all the brouhaha surrounding its object filesystem and such.

Descartes came to mind at first because I remembered his leisurely attitude when writing his Meditations. Through the text, Descartes repeatedly goes back to the whole sitting-by-the-fire thing to use as examples and so on, impressing us with his (flawed to me) logic, but he also tends to create the unwelcome impression that he's just a basically a well-off guy (consider when he wrote it...and the conditions in general back then) with too much time on his hands.

Both that situation and the quote have some parallels to what Microsoft is doing methinks.

The Cairo-Longhorn connection has been raised before (I remember seeing it mentioned in at least one weblog recently). This is not new. But the similarities are just so startling that it's interesting to take a closer look.

Part of what I thought about were those excellent articles in Byte through which I gathered a lot of useful information (PC Magazine was always crap as far as I could see, except for their lab tests). I started wading through the Byte print archives, looking for some of those articles.

Let's begin with this one (which, as I remember, wasn't an article but a box in a bigger section on OO technology) entitled Signs to Cairo. Choice quote:

Now peek into the future. The top level will no longer be a separate application such as PowerPoint, but the Cairo desktop itself. The streams comprising the compound document will no longer be inside a DOS file allocation table (FAT) file system. Cairo's Object File System (OFS) makes the whole hard disk a single huge docfile that exposes its internal objects to the user.

That was in November, 1995. Eight years ago.

And another one, from Jon Udell (now at InfoWorld): Exploring Chicago and Daytona:

In Daytona's successor, Cairo, OLE structured storage will be able to attach to, and extend, the file system. As the Explorer navigates from a file store into an object store, control will be transferred from Explorer's viewer to an object-supplied viewer. Object internals won't be stored in user-visible directory structures, so users won't trip over them.

And more, from Inside the Mind of Microsoft:

OLE DB, the newest member of the OLE family, interfaces OLE to multiple databases. Among them is Microsoft's future object-oriented file system for Windows NT (see the sidebar "A Peek at OFS"). Ultimately, we could be looking at a distributed file system based on this technology.

Almost all this technology is expected to converge in Cairo. By then, 16- or 24-MB systems will be the baseline, so hardware shouldn't be a limitation. Cairo will inherit desirable features from Windows 95 and Memphis, until finally the day arrives when Microsoft can offer a single OS to all desktop users.

Even more interesting is Cairo Inside, an article from 1996 (with the tagline "An object-oriented, next-generation operating system called Cairo may never ship. However, future versions of Windows NT will enjoy the fruits of the Cairo development effort."). Most interesting of all in this page is the following description of Cairo's OFS:

Lets you create a pseudodirectory that unifies local, network, and Internet files.
Internet files. Interesting no?

Even when it was clear that Cairo would never be "Object Oriented" at all, it was still commonly described as "Microsoft's Next Generation Object Oriented OS". This is almost certainly due to the fact that NextStep was seen as the coolest thing around and it was, well, yes, truly Object Oriented.

Now, this is another Byte column from Jon (hosted at his site), from 1999, when NT4 had been released for some time, and the promises of Cairo's OO attitude were just a memory: From Virtual Memory to Object Storage. Quote:

MS Cairo was headed in this direction, and I saw early demos of some of these ideas back in 1993.
By the way, there's something about those articles that makes them fascinating to read, even now that their vaporware roots have been exposed in all their guts and glory.

Now, these people were not hallucinating, even though the breathable air at COMDEX was probably dwindling by then. This is what they were told by Microsoft. This was the promise.

I think about what Dave said in the comments to my post about how the browser is not the web: "it's Microsoft's dream to turn the clock back to 1993, the end of their brief period of total and utter domination of the computing world," and the history shows that the parallels are quite striking.

Specifically regarding that post, Robert replied by saying that there was an RSS aggregator in Windows among other things, a show of support for the web/openness in part, but even though that's cool, I don't see how it changes the potential ambitions that MS might have. (By the way, on that particular point of whether the browser is the web or not, for more viewpoints check out Karlin, who agreed (if briefly :)), Dare, who agreed only partially, and Christian, writing in Lockergnome, who did not agree with me).

Now, I was basically saying that Robert's argument that RSS was taking him away from the web was flawed. Dare's answer in particular was trying to nail down a pretty exact definition, but I don't think that's what's at stake here (even though it's of course useful and important).

Comparing Microsoft's proclamations 10 years ago from those today makes the little hairs on the back of my neck stand up. Notice how everything seemed going along this exact same path that MS is on today (the parallels don't stop with OFS) until the Internet happened. Then MS had to divert itself for a few years to crush Netscape and so on, and now it's back to the old game.

But what is the old game really? "Providing a better user experience" Microsoft will say. "Taking over the world" will say others. I personally think both are mixed :).

And, set the arguments aside for a moment: what happened to all of this technology?. Why didn't OFS ship, if they had demos going back to 1993? It would be interesting to know, just for historical reasons; it's weird that all of this just vanished. Maybe portions of this eventually made it to the product, but certainly not the whole enchilada.

As far as the reason for not seeing it through, my theory is that (as I said) what happened was the Internet. Suddenly OO wasn't all that hot anymore, and why deliver technology that doesn't let you create cool brochures?

Okay, I'm being a bit flip here. Seriously now: To anyone that might say that the technology could not be built... please. Microsoft is one of the top engineering organizations in the world. NeXT could do it. Why not Microsoft? The only reasonable explanation, as far as I can see, would be a realignment of priorities and the consequent starving of resources that go along with it (which is what killed both OpenDoc and Taligent, for example). Which is all well and good.

But then the question is: could it happen again?

Probably not--then again, never say never.

Just to close with some constructive criticism, since Robert's spirited defense (though a bit flawed in my opinion) deserves it.

We've heard a lot about how Microsoft intends to push these new technologies. Ok. But I'd also like to hear what, exactly, they will do to strengthen the Web and its foundations. I think that right now there's a lot of uncertainty because all of these new technologies seem to imply that Microsoft is back to its old tricks after the brush with the DOJ and the European Commission (something that's still not over yet). But I think that a lot of people would give Microsoft a chance if they announced, publicly and clearly, that they will commit to respecting web standards and support them. Examples: That Microsoft Word will stop generating HTML files that look terrible on browsers whose only problem is that they're not IE. That a future Microsoft blogging tool, if any, will not start embedding MS Office documents and such in the middle of RSS files by default (users embedding it at users' whim is another matter). And so on.

Put another way (Hopelessly idealistic as all of this may sound): I think that a lot of people would give Microsoft a chance if they made it clear that they will do a good job of supporting web standards for both for reading and writing, that most people would give Microsoft a chance if they came up with all the innovation they liked, but didn't force it on anyone, and just played fair on web standards, and, that most people would accept the challenge if Microsoft, for once, really stood up to competition based on product quality rather than on leveraging their market-share.

I know I would.

Posted by diego on November 20, 2003 at 12:40 AM

back to windows (for now) part 3

Okay, things are more or less back to normal. Installed most -not all- the software I need, definitely everything I need to run the tests (now completed as well). Two things. One, once I finished I got this nagging feeling in the back of my head, a little voice telling me that, if I could go through a Windows install, or any install, in "automatic" mode, trying things until they worked, more or less with all the answers... then something was wrong. I should be using braincells for more useful stuff than this. Oh well.

The second thing was that as soon as I was finished I experienced a moment of total confusion, as in "Now, what the hell was I doing?". The install became an end in itself. Not that it can be avoided when you have to be doing so many things just to get stuff to work normally.

A bunch of comments on the other two posts, will get to them in a moment. For now, I can do what I need and that's enough. I wish Java support on Red Hat et al would be exactly on par with Windows and Mac (there are other small annoyances like the microphone not working under Java on my machine). Hopefully soon I'll be able to get back to Linux! I already miss playing with all the fancy X stuff. Windows Powertoys are a sorry excuse for tweaking the system. :)

Posted by diego on November 19, 2003 at 12:46 AM

brian eno on the long now

A fantastic article by Brian Eno on The Long Now project:

an we grasp this sense of ourselves as existing in time, part of the beautiful continuum of life? Can we become inspired by the prospect of contributing to the future? Can we shame ourselves into thinking that we really do owe those who follow us some sort of consideration – just as the people of the nineteenth century shamed themselves out of slavery? Can we extend our empathy to the lives beyond ours?

I think we can.

Yep. I think so too.

Categories: science
Posted by diego on November 18, 2003 at 4:17 PM

back to windows (for now) part deux

It's now about 6 hours or so since I began the reinstall. Seeing the install/update/patch process all at once is quite an experience. I've spent now close to two hours downloading updates and patches (at 50 KBytes/sec!). First, there was a batch of about five "critical" updates (10 MB). Warning! Your PC may do bad things if you don't install it! and so on. Then Windows Update suggested Service Pack 4. 50 MB. Right after SP4 installed, another check (this time thinking that was it), and now there were twenty (TWENTY!) Critical-install-this-right-now-or-it's-the-end-of-the-world-as-you-know-it patches. Another 50 MB. Plus, I'm not even done with the "recommended" patches (rather than those that are "critical"), which also fix problems for various calamities that might visit you or your loved ones if you don't apply them.

Can anyone in their right mind think that this is normal? We have gotten used to this whole patching idea, but it's ludicruous. By now, every security warning, every patch, elicits a "oh, another one of those...". Mind you, lots of those patches are not just security problems, many are bugfixes that apparently have various disastrous consequences under different circumstances.

Windows is not going away. Would it be much to ask of Microsoft that instead of drooling all over XAML or whatever new thing they are planning to conquer the world with, they would put their considerable resources and smarts to find a solution? You know, I think that Longhorn would be fantastic if instead of all the thingamagic promiseware that it will supposedly have, it was simply Windows XP (or even 2000) and it just worked. Who cares about 3D icons if I'll probably need to find a new "3D Icon critical patch" every fifteen seconds?

Sorry, I know that this has been discussed to death, everyone knows this, Microsoft knows this... but the experience of seeing this whole process in the space of a couple of hours has activated my gripe-cells. We now return to our original programming.

Posted by diego on November 18, 2003 at 4:13 PM

back to windows (for now)

I've been a happy camper since I switched back to Linux (Red Hat 9) on my Thinkpad T21 laptop about three months ago. Everything worked fine. And aside from some annoyances, such as the tendency of Gnome to crash a few times a day, it was great.

But yesterday I needed to test some of the sound features in clevercactus and Linux bailed. For some time I thought this was a Linux problem, after all, the Gnome sound recorder crashed when I recorded more than once and didn't record anything at all. Then (this morning) I realized that the problem was in the internal microphone (not supported) and using an external mike worked ok. The sound recorder still crashed, but at least it worked. Once.

Problem is, I need Java to work with it, not just a native Linux app. And Java sound support has been spotty outside of Windows and the Mac (Sun is devoting basically no resources to it). Even though output worked ok, microphone input did not. LineUnavailableException.

At the moment I really don't have time to spend two days fixing whatever the problem is. I think that with enough tweaking it should end up working (that's the Linux way after all) but that's not an option right now.

Back to Windows it is, at least for the moment. I dusted off the original Windows 2000 Pro installation disk that came with the notebook (after I found it :)) and I am now in the middle of the install. Disgusting experience. FDISK. FORMAT. Abort, Retry, Fail? messages. I'm now doing the recovery of the install (the IBM recovery disk creates its own partition setup, one more reason to wince) and the file copy is in progress.

*Sigh*. I hope to be able to go back to Linux soon in the machine. Running cygwin is a poor excuse for it.

Update: Linux doesn't want to let go. After reformatting, Fdisk, and such, the Linux boot loader is still there, except that now it just hangs. Damn. Trying again...

Update 2: running "fdisk /mbr" took care of the problem.

Posted by diego on November 18, 2003 at 12:29 PM

funny, funny ad

The screen turns black, and the verses from Bryan Adams' Everything I do (I do it for you) start to sound. Sentences flash on the screen, white letters on black background, one sentence every three seconds or so:

This is Bryan Adams

He is Canadian

This tuesday

It's payback time.

Ireland v. Canada

From an ad on Ireland's TV3, regarding the football match this tuesday.


Posted by diego on November 17, 2003 at 12:34 AM

ain't that the truth :)

"[Welcome] to the twenty-first century. It's pretty much like the twentieth, except that everyone's afraid and the stock market is a lot lower."

Lisa, in The Simpsons' Treehouse of Horrors XIV episode.

Posted by diego on November 16, 2003 at 7:04 PM

"it's full of stars!"


Okay, maybe I'm exaggerating a bit. :)

Before going back to read a bit, before sleep, I just went out to the balcony to feel some of that cool (or cold) early morning fresh air and I saw something I haven't seen in a while: stars.

What a rush. You sort of forget about them since it's not really that common to see them, with the light going up from the ground, fog, clouds (lots of that here in Dublin), but tonight's surprinsingly clear, and Orion right "outside" my window, so I took a long exposure shot of it and here it is. Obviously I can see more stars than those that appear in the shot (the whole constellation, including the "left elbow" of the archer) but not that many --in fact the shot looks kind of lame, doesn't it? Oh well. That's city life. Note to self: travel out to the country for a night at least and spend some time looking at the galaxy that's out there.

I took a couple of nice pictures at sunset yesterday that I'll post tomorrow or at some point during the week as well. And some comments to reply to--will leave that for tomorrow. I mean today. Later. After sleep that is. :)

Categories: personal
Posted by diego on November 16, 2003 at 2:40 AM

self-organization, cyberspace, and conspiracy theories

I was just reading The Sword and the Shield: The Secret History of the KGB by Christopher Andrew and Vasili Mitrokhin, when I came across this passage:

Until almost the end of the Cold War, no post-war Soviet leader, KGB chairman or foreign intelligence chief had either any personal experience of living in the West or any realistic understanding of it. Accustomed to strong central direction and a command economy, the Centre [KGB Headquarters] found it difficult to fathom how the United States could achieve such high levels of economic production and technological innovation with so little apparent regulation. The gap in its understanding of what made the United States tick tended to be filled by conspiracy theory. The diplomat, and later defector, Arkadi Shevchenko [noted]:
Many are inclined to the fantastic notion that there must be a secret control center somewhere in the United States. They themselves, after all, are used to a system ruled by a small group working in secrecy in one place. Moreover, the Soviets continue to chew on Lenin's dogma that bourgeois governments are just the "servants" of monopoly capital. "Is not tha tthe secret command center?" they reason.
However much the Centre learned about the West, it never truly understood it. Worse still, it thought it did.
As soon as I read this I was struck of how true this is in general, applied to almost any situation where a "reason" is not readily identifiable. Especially with "artificial" constructs (governments, corporations, whatever) but with the world itself as well, we tend to imagine someone is pulling the strings. It's comfortable, in a way, it defines whatever it is that we are seeing, regardless of whether the reality, most of the time (if not all the time), is that things happen through simple rules that determine the self-organization of groups. Birds fly in recognizable flock patterns not because there's an AWACS bird that flies higher up and directs them using a wireless microphone, but because they use simple local rules (say, "stay no less than 2 feet and no more than 4 feet from the next bird, and never be surrounded by more than 4 other birds"). Simple rules, applied locally and consistently on groups generate incredibly complex behavior. With humans, the rules aren't simple, and they aren't applied consistenly. More complexity. We see something we can't put our finger on and we go the easiest route: someone must be behind it. Purposefully. Because the option seems to be that things happen "by themselves". But that's a false option. Things don't happen "by themselves" but there doesn't have to be a master plan either. Groups and individuals, going in similar directions, sometimes closely aligned, sometimes not, sometimes at odds with each other, sometimes not even aware of what the other is doing---this is the essence of most events that tend to be "explained" by conspiracy theories.

But we don't like that idea, do we. It takes away part of our cherised egotism. It places success, in part, in the unseen dynamics of groups. Dynamics that are, at the moment, more or less beyond our understanding. Suddenly "what makes things tick" is no longer individual actions, or plans, but interactions. What's in between. Just like in a phone conversation, when you are talking to someone, where is the conversation actually taking place? In your head? The other person's head? The phone line? The switches? Of course not. But without each component there is no conversation, yet it's not contained in any of them. It's not about the sum being greater than its parts, it's about the sum being something altogether different from its parts. A phone conversation is fundamentally meta-human (sure, this applies to a "regular conversation as well since air is just another medium, but it's easier to see with a phone I think).

But we've had a word for that "place" where conversations or more generally data exchanges and interaction happen, for twenty years now: Cyberspace. And just now I saw that this concept of the land of meta, all those things that emerge as entities on their own from a group of apparently disconnected parts, is a very similar concept, but applied to everyday life, less bound by the digital.

Just an early Sunday blinding flash of the obvious. :)

Categories: personal
Posted by diego on November 16, 2003 at 1:45 AM

the web is not the browser

Yesterday Robert Scoble, reacting to comments by Dave, Jon, and others, was saying something really, really weird which involved comparing Microsoft with Dave.

Robert's comment was centered on the idea that "Dave Winer has done more to get me to move away from the Web than a huge international corporation that's supposedly focused on killing the Web."

Robert's thesis is summarized in the sentence that follows: "[...] what has gotten me to use the Web less and less lately [is] RSS 2.0."

He goes on to describe the wonderful advantages of Longhorn's components, for example: "And wait until Mozilla's and other developers start exploiting things like WinFS to give you new features that display Internet-based information in whole new ways."


On to the debunking. :)

There are two points here. Number one, let's take Robert's thesis at face value. It is a standard Microsoft tactic to say "see? The little guy is doing it, so why can't we?". How MS can't see the difference between "the little guys" and themselves is beyond me.

But that aside, there's a bigger (much bigger) problem.

Robert: the web is not the browser.

Robert says that he's "using the web less and less" because of RSS. He's completely, 100% wrong.

RSS is not anti-web, RSS is the web at its best.

The web is a complex system, an interconnection of open protocols that run on any operating system. Robert reads RSS on a Windows client (I assume), through a protocol originally developed in Geneva and now maintained in Cambridge, MA. He reads from a variety of servers, Linux, Windows, Apache, IIS, what have you. The RSS files he reads are generated by a multitude of software systems, all of them connected through the simplicity of a few lines of XML code.

Now, if that is not "the web", then I don't know what "the web" is.

Consider the alternative: what if Robert is right? What if in 2007 or whenever Longhorn leaves the realm of promiseware developers start switching in hordes to it? Take Robert's example: of "Mozilla's and other developers start exploiting things like WinFS to give you new features" and consider: what server will that run on? Linux? Not a chance. What if developers start using XAML? What client will that run on? Macintosh? A Symbian mobile phone? Of course not. You'll need a Windows device to see it.

Longhorn is anti-web because it locks down everything back into Microsoft's control. It has nothing to do with HTML. It pushes a system where you'd be forced to using Microsoft servers and Microsoft clients.

Let me say it again. The web is not the browser. The web is protocols and formats. Presentation is almost a side-effect. And that's what people like Dave and Jon are talking about.

Categories: technology
Posted by diego on November 14, 2003 at 12:18 PM

the bladerunner soundtrack

Enhance 34 to 46.

(click) (click) (click) (click) (click) (click)

Pull back. (click) (click) (click) (click) (click) (click) Wait a minute, go right.

(click) (click) (click) (click) (click) (click)


Enhance 57 to 19.

(click) (click) (click) (click) (click) (click)

Track 45 left.

(click) (click) (click) (click) (click) (click) (click) (click) (click) (click) (click)


Enhance 50 to 23.

(click) (click) (click) (click) (click) (click) (click) (click) (click) (click) (click)

Give me a hardcopy right there.

For some reason, the Bladerunner soundtrack never fails to give me the chills; it's almost unbelievable that it wasn't released for ten years (it came out in conjunction with the Director's Cut in 1993, back when the media conglomerates weren't so good at the multiple medium thing, and were less... well, "conglomerated"). There is something about Vangelis' music together with the pieces of dialogue and the ambient sounds that go with it that puts you immediately there. No images required. Not even closing your eyes. Just listening.

So good it hurts.

I should watch the movie again one of these days. It's been a while since the last time (months even!) :-)

-Do you like our owl?

-It's artificial?

-Of course it is.

-Must be expensive.


The undertones. The atmosphere. The depth of the story in just a few lines. Must... resist...

Posted by diego on November 14, 2003 at 1:11 AM

wanted: a breakthrough for testing distributed systems

The last few days I've been heads-down working most of the time on a distributed system that involves going deeply into the arcana of TCP/IP. It's an eye-opening experience. First, because Java proves once more to be a rock, obtaining pretty high data transfer rates (currently 37 KB/sec on a 50 KB/sec connection) even with all the twists and turns the code is taking. Second, but most important, it has reminded me of how little we know of distributed systems, how to build them properly, and how to test them.

By distributed systems, I mean truly distributed. I mean that you can't count on a server to be happily taking over stuff for you with a bunch of TCP ports open and a four-way processor core ready to handle incoming tasks. We might be tempted to call this peer-to-peer (as opposed to client-server) but not really, since I could easily see this being used on a "traditional" client-server server environment. The difference is subtle, in terms of what you assume on the server side, and how you get around the constraints imposed by today's Internet.

That aside, being a test-first-code-later kind of person, I tend to put the burden on testing, or the testing framework rather. So I thought I'd write down my wish-list for a distributed testing framework (as food for thought more than anything else). This framework would work as follows: you'd have a "test listener" that can run on any machine, and a "test controller" app that can run on your desktop. Once the listener is running on the other machines (and maybe even on your desktop too) you can easily choose a JAR to deploy to all the target machines, then run it. The system automatically routes the output (System.err and System.out) to your "test controller" in multiple windows. You can control any of the clients through simple play/pause/stop/restart buttons. Clear the consoles, etc. You would be able to script it, so that this whole process can be run in loops, or automatically every day or every week, or whatever. You would be able to define output values to check for that can alert you of results that don't match expectations.

Looking around, I found the DTF at SourceForge, but it seems to be dead (no binaries, and no updates since February this year). I found papers (if you look hard enough, you can find papers on every conceivable topic I guess, so this doesn't mean much), like this one. But not much, really. Or is there some vast download area somewhere that I'm overlooking?

In any case, I know for a fact that CS curricula still don't pay enough attention to testing, much less to distributed testing. For one, distributed testing is difficult to generalize. But there should be more in this area happening, shouldn't it? Or does anyone doubt that half the future lies with large scale distributed applications? (The other half is web services :-)).

Posted by diego on November 13, 2003 at 12:58 PM

3 am

Suddenly everything looks bigger. The monitor. The walls around you. The sky outside has no end; you can't even look at it.

The cup, half empty, is still steaming. You pick it up. You look at it. You set it down again. Slowly. No sounds allowed now.

These moments can come at any time, day or night, but especially at night. When the city sleeps, its only sounds remote sirens that come and go rhytmically like the pulse of a slumbering giant.

Head spins. You are about to turn on some music, but the finger hesitates over the play button.

You wonder.

It's a good feeling, almost mystical. You wish for a moment that it would never go away but quickly change your mind. It's because these moments are rare that they are precious.

Soon, it will be sleep, and, for a while, darkness. And silence.

Categories: personal
Posted by diego on November 13, 2003 at 2:59 AM

short presentation on blogging

This is from a few days ago, and in the storm created by a number of things I forgot to link to it, but here it goes. Following my two introductory articles on weblogging, Charles published a presentation he gave some time ago, short, to the point, and geared towards a more technical audience. Nice!

Categories: technology
Posted by diego on November 8, 2003 at 5:57 PM

lunar eclipse

There's a lunar eclipse tonight (full Earth/Sun alignment at 1 am Zulu Time). But it's cloudy out there tonight, and I doubt that it will clear up in time. Looks like Dublin (and by extension, me) is going to miss it. Too bad.

Categories: science
Posted by diego on November 8, 2003 at 5:52 PM

The Matrix Revolutions: a review

Okay, I thought I had put the subject to rest (for me) with my parody script, but there was a factor I didn't count on: exposure. The script had tons of reads and links very quickly and I keep getting comments and emails asking questions (generally good-natured, though not always) and I wanted to say what I actually thought about the movie so I could refer people to this and save some time :).

Again, spoiler warning. Don't read what follows if you haven't seen the movie or don't want to know what happens.

Before we begin: to those that liked the movie anyway and are willing to flame others. All the "defenses" of the movie I've seen are like a comment posted to my script by Daniel:

Its easy to make fun of something you don't understand. You did a damn fine job of that.
Essentially, the defense boils down to "people that didn't like it are stupid. I am not stupid, hence, unlike you, I understood and appreciated the movie." (I have seen this same theme in a few--very few-- other places. Why assume that people that didn't like aspects (or all) of the movie did not understand it? Personally, I understood Reloaded with only one viewing. Are you telling me this movie is more layered and more deep and complex than Reloaded? And even if it was, saying that everyone's an idiot doesn't really explain anything. Yes, sorry, but unless you actually refute some of what I (or others) say, or unless you offer a clear explanation for some of the ludicrous twists of logic that we have to endure, you are simply clinging on and you are not willing to see the movie for what it is: Just a movie, hollywoodish science-fiction stuff that does not "respect" the basic tenets of science fiction. Saying I (or others) did not understand it is not good enough.

"Ah!" One of these critics might say. "But of course it's just a movie. You are the one who put the burden of proof on them to produce your imagined story-of-the-ages."

Touche, I'd say then. Very, very true.

I think most of the people that were not satisfied with Revolutions were hoping that this would become an all-time SF/Fantasy classic, way up there with the Foundation series and The Lord of the Rings. Most definitely, we put ourselves in that position. But we had good reasons I think, and I'll get back to them in a bit.

What the movie was about

First, for what it's worth, my take on the movie: it's entertaining. Nice picture. Great battle scenes. I think it's worth seeing in a theater, because it's a cinematic experience. Things can be explained. I have no doubt about that (as I make clear below with my own set of explanations).

But... but... it requires too much suspension of disbelief to qualify among the great creations of science fiction. The explanations are not satisfying. Not unlike ID4: Independence Day, or Armaggedon: entertaining, but not self-consistent enough. Sure this one has more twists and turns, and more ideas (not original though as I have mentioned before, the Brothers lifted sequences from Anime, and other classics such as Alien or Bladerunner. I also saw [via Alan] this scene-by-scene comparison of Matrix v. Ghost in The Shell which is very good). The Machine city shots as well as several others reminded me of both Bladerunner and Star Wars. Too much. Waay too much.

The plot would seem to be summarized as follows (given information basically present in the last scene of the Oracle and the Architect). The themes are chaos/order as well as religion. The Oracle is Chaos/Creativity. The Architect is Rationality/Order. The Oracle basically instigated this whole revolution because she wanted to see a new kind of balance emerge (Remember the Architect telling her at the end: "This is a dangerous game you're playing". Although it's not 100% clear how this balance is actually achieved in the end. If the machines let the humans go, don't they lose their power? and so on). Similarly, Neo/Smith are Order/Chaos figures, and it's all framed in terms of a battle of opposites. (Which would theoretically explain why Smith could not survive merging with its opposite at the end). The little girl, Sati, is very possibly a representation of the Matrix itself. If not, it's a program that, because it was "born" within the Matrix, can manipulate it at will but more than Neo could (no more than Smith though, since the hellish climate at the end could easily be attributed to Smith expressing himself after taking over all humans in the world).

The religious themes are back with a vengeance: Sacrifice, Martyrdom, A new world is born after the death of the Chosen One, the Chosen One dies but not really, (note the Machines taking Neo's bodies at the end, as well as the references by The Oracle), etc.

The information given in the second movie amounted to giving us a hint that this is what was happening. That the characters were sort of unwitting players (almost unwitting, since Neo makes it very clear at the end that it's his choice to do what he does) in a game played by the forces of chaos and order, the Gods (in the Platonic sense--there are lots of references to Plato) that play with humanity, a parallel to our "real" world.

Neo can "see" the Matrix both in its "virtual" form and manifested through the appearance of the Matrix in the machines that are plugged/depend on it, like in the machine city, and he can affect it even though he is unplugged. Of course, if he really was a "natural" occurrence of the "choice" flaw in the Matrix, and he is really fully human as they keep saying, then this implies superpowers, but that's ok (within the suspension of disbelief theme). In the end, peace is achieved through balacing of opposites all the way. If you wanted to take the religious analogy further, you could say that there are historical parallels with our own history: First, Christianity and Islam, the age of the Messiahs (the first movie), then deconstructionism, the age of rationality (the second movie) and finally chaos/order or yin/yang, the age of Eastern Philosophy or "new age" beliefs.

Though probably close to the truth, this is just one interpretation of what is basically a Rorsarch test of a movie (I would challenge anyone to come up with one that is substantially different though). In the end, You see ... what you want to see.

Which brings me to the problems I have with it.

But... but...

When we walked into theaters four years ago to watch The Matrix, the overriding question was: What is the Matrix?

Coming out of that movie, the sense was that we had an answer: a prison for the mind, the Matrix was a device created by Machines to win a war against Humans, creating yet another war, this one just for freedom from the shackles of a virtual world.

Then came Matrix Reloaded. The question going into the movie then was: How will humans win the war? (Note: How, not If). The answer was, in essence, "There is no spoon." Or rather, "There is no war." The Wachowski Brothers turned everything on its head and destroyed all our preconceptions. The rebels were actually being controlled. Their revolution was a sham. Another lever of control. We were pulled out of the Christian and even Muslim parables of Neo-as-Savior (Muslim because Neo is much more a "Warrior Messiah" like Muhammad, than a Christ-like character of peace and understanding), into a new level of pure science-fiction possibility. Just as the first movie studiously created a fictional reality, the second dedicated itself to proving the first one wrong. Just as the first one required us to suspend disbelief more than once, the second one gave potentially reasonable explanations for everything that was going on. Reloaded, more than anything else, revived the question: What is the Matrix?

And so we start The Matrix Revolutions essentially with the same question as the series begun. We have a lot more information, but there's one big difference. Now we don't trust anything we see. Anything. Every statement is parsed, analyzed. Somewhere deep down, we expect the second movie to be another layer of fabrication as well, and we dread the moment when it might all turn out not to be a fabrication, but the pretense of truth. And we are enticed to pick it apart like few movies before. With all the pretense of "deep meaning", we are told: "this is deep stuff. You have to think damn it!". But then there is no "deep stuff". When analyzed closely, we're left with just bad dialogue, a lot of obvious ideas that are a rehash or outright steal of other things, and a lot of overacting.

Let me go back to a moment when I was walking to the movie theater on Wednesday and I thought about something I had just written:

And what is up with characters not telling others what they've seen? Example: Neo is all cryptic just after meeting the Architect. Why not tell Morpheus the whole thing? Just because he has condemned humanity to extinction? (Supposedly). Or: When Neo stops the Sentinels at the end of Reloaded. He clearly says "I can feel them" to Trinity. Then he stops them. Morpheus arrives. "What happened?" he says. Trinity replies: "I don't know." You don't know? Come on. "He said he could feel them, and then he stopped them." Is it too hard to say that? It's as if characters play the same game between each other as the one they are playing with the audience.
The more I thought about this, the more I thought it was a symptom. Consider that twice in Reloaded we are treated to this drivel from Link. He is watching the Matrix. Neo is flying. Suddenly Link goes "What is that?" or "I don't know what it is, but it's moving faster than anything I've ever seen" when we all know that it's Neo flying, when he has already seen him fly. Etc. He does this both in the freeway chase, and at the end when Neo saves Trinity. Sure you might say that as Neo gets more powerful his Matrix-pattern becomes more difficult to discern, but this happens all over the place, like the Trinity/Neo/Morpheus example I mentioned, with the Sentinels, at the end of Reloaded.

Another example is all the "mystery" surrounding the "new" Oracle. Now, I know that they had to come up with something to explain the problem that the original Oracle (Gloria Foster) died while filming was incomplete. I appreciate that. But instead of pointing to something reasonable (for example, they had already hinted at a reason when the Merovingian said in Reloaded that her time was "almost up") the explanation just dissolves into a bunch of generalities that "hint" that something deep happened but it's never fully explained (maybe it's explained in the game Enter The Matrix, I don't know). Why make a mystery out of something that can be explained away easily with a million different reasons? Why not trust the audience? The audience wants to believe, just like Morpheus. :-)

My point is, this is a symptom for writing where the mystery is created by making weird and ambiguous statements, rather than having something true to tell. It's very easy to do. Consider:

Sam stared at the scopes in astonishment. All the screens had suddenly gone blank. The visual showed a flash, growing, where the Sun used to be. Was it...? "Oh my God."

The Captain interrupted. "Sam, what is it?"

"I don't know Cap." Sam replied, lips quivering. "Something I've never seen before. But it's approaching us."

"Can we get out in time?"

"No way. It's too fast." Sam doubted for a moment. "Do you think the ship will hold?"

The Captain rested his hand on Sam's shoulder and pressed reassuringly. "Don't worry, Sam. We'll know soon enough."

Now, I just made that up, so don't start criticizing the writing :) but I'm trying to show that it's really easy to get off explanations by making characters appear dazed and confused. Think about yourself in real life. Whenever something strange happens, you don't just sit there and say "Well, beats me." You use your experience. You talk with others. In this case, it wouldn't be so hard to think that the Sun had just gone supernova. And we get a hint that a character might have an explanation with the "Was it...?" but then he doesn't say anything. Why? You better have a good reason, because if you don't, it's just an empty device that you are using to create suspense, and eventually it wears out.

Just as it does in The Matrix.

And even worse trick to pull is creating the suspense for the answer, and then giving it, but the answer raises yet more questions which are never explained. In my example, the Sun (our star) doesn't have enough mass to go supernova. It physically can't. So once the characters survive, and say "Our readings indicate it was a supernova, Captain". And you leave it at that. Why? Why did the Sun go supernova?

Apparent plot-holes such as the lack of offensive weapons on the Nebucadnezzar (Morpheous' ship) or not using EMPs to defend Zion can be explained, but only raise more questions that force the viewer to extrapolate with no information whatsoever. For example, we could say that the "Neb" didn't have offensive weapons because it was never assigned to be on the offensive, while the Hammer was. But that was really a last minute change. Lock intended to use every ship in the offense. Morpheus bailed out of that at the last minute. So why wasn't it fitted properly? Sure, you could come up with answers. But at this point, we are already well beyond what the story says, we're just inventing reasons, speculation based on speculation. But I think that's not good enough. And Reloaded raised the bar on the whole story by implicitly saying that it was fully self-consistent.

And this is the essence of why we were led to believe there was a cool explanation behind all of this. We were caught by the 'Matrix' created by the Wachowskis you say? We were given a lesson-in-action of how we let ourselves be deceived by appearances you say? Maybe. But if so, if I can see through it so easily, it's a clumsy attempt. And if you're doing that, forcing you to, say, play a PC game just to get part of the story, buy the Animatrix, etc, etc, would imply that all of this relentless franchising of the story is not done by a mega-corporation (Time-Warner), but on purpose by a bunch of guys who don't care about money with too much time on their hands.

Which one is right? Occam's Razor. The simplest explanation tends to be the right one.

Now, I don't have anything against plots that don't fully make sense. In fact, the first Matrix on its own, had so many holes that it was hilarious. But I liked it. I willingly suspended disbelief to enjoy the ride. And a big part of that was that the first movie was not pretentious.

But Reloaded was. Reloaded said to the audience: "See? We've thought about this stuff. There's more levels than you imagine, even though you can't see them all. Here is the depth you sensed in the first movie."

And so we got to Revolutions waiting to see the final twist. But instead of a twist, Revolutions essentially goes back to the original Matrix. Why do I say this?

Think about the three movies. Now imagine you completely remove the second movie. Ignore all that information. Ignore everything that happened in it. Does it change anything?

Be honest. Does it?

No. All the flaws of logic and plot consistency are still there. (In fact, I'd say that just the first and third movies on their own work better, and by "consistency" here I mean J.R.R. Tolkien-level of consistency, not "ID4: Independence Day" level of consistency--though both are valid). But at least there is no pretense that there is something deep behind all this. We are not given lectures on causality or whatever. Things just happen, which is just fine. I can ignore all the flaws of logic in making programs have emotions for some things (e.g., Agent Smith is angry or greedy) but not for others (e.g., The Architect saying that he could never betray a deal he made). I can ignore how weird it is to use humans for power (why not cows? or chipmunks? Oh, right, "because the machines create a symbiotic relationship with their enemy through it"). I can ignore the ludicrous explanations of The Second Rennaisance in which we are told that humans nuked the hell out of Zero-One but nothing happened (Nuclear weapons create electromagnetic pulses of the same type of those we've seen "kill" machines time and again in the movie). I can forget about the fact that first the Hovercrafts "only have EMPs as weapons against the machines" (and this is also the case in Reloaded when Morpheus loses his ship to an attack----but by the third movie they have so much ammo (though not many guns!) that you could film Commando all over again with Keanu taking the role of Schwarzenegger. Or I could forget about the other miriad plot holes, many of which I mentioned in my parody script.

But I can't.

Again, these plot-holes depend on your expectation of the movie. Not on the movie itself. The movie never promised explicitly to match our expectations. But in my opinion, there's a sort of "contract" that happens when you dive into fiction of any kind. For example, in ID4 or Indiana Jones the contract is "shut down your brain for a while and we'll show you a good time". The first Matrix walked a fine line between deepthink and pure entertainment. But the second one, with all of its pretentiousness, did not. The second one cemented the promise of the first one, which was "Look, I know there's tons of philosophical dialog and repetitiveness, but it's all for good reason. Just hold on there." And so the washed-up explanations of the third, basically in line with the first, don't add up. Why? Because all the repetitiveness is to explain something simple, not something complex, and that amounts to telling the viewers: "See, you are too stupid to understand this. So we'll say it over and over again. We will put Neo in the guise of a martyr, bandages and all, so you can see how much he suffers and don't miss the Jesus analogy. We will make you questions words like "Love" by making machines say "they're just words". We will make obvious references to everything under the sun until you can't ignore them."

But I think that people would rather see something that is a) either simple and to the point, or b) something non-obvious that can not be fully expressed, but hinted at.

The Matrix, as it stands, is just a bunch of obvious points driven into our heads with a sledge hammer. And that's not what the "contract" specified. Furthermore, all the ideas have been used before. Nothing new here. It's all one big cop-out. Entertaining? Yes. A Masterpiece? Most definitely not. Treating your audience like idiots is not a very good idea.

As Smith said in the first movie. "[Humans] refused the program. Entire crops were lost."


Don't try and bed the spoon. That's impossible. Instead, realize the truth. There is no spoon. So it is not the spoon that bends. It is only yourself.

Replace the world "spoon," by "rational explanation" or "self-consistent plot" and you're in business with The Matrix.

Phew! Okay this should do for now. :)

In closing, a line from Futurama:

... and so life returned to normal... or as normal as it gets on this primitive dirtball inhabited by psychotic apes.

Posted by diego on November 7, 2003 at 1:16 PM

linux confusion

A bunch of Linux-related news has hit the ... err... "newsstands" recently. Novell announced they would acquire SUSE for 210 million. Predictably, this generated a lots of comments, including this great (if brief) analysis from Charles Cooper over at A less-noticed element of the Novell announcement was that IBM is buying into Novell as well, getting about 5% of the stock. This sounded strange to me at first, but IBM is clearly covering their bases. Now they have "deep" alliances with both Red Hat and Novell, which are by now the two most prominent Linux "vendors". Since IBM is so intertwined with the Linux thing, this makes some sense. But I still wonder exactly what it means. After all, you don't sell 5% of the company just for money if you have revenue streams, etc (or do you?).

Red Hat, on the other hand, has made some strategic changes to their product line that are still confusing to me. First, they are discontinuing the Red Hat professional line (support of any kind for RH9 ends in April next year). Focus is now profitability, which they go after with their Enterprise Edition. But does this mean they killed the "workstation"-type product? Apparently not. There's a new RH "for hobbyists", down the pipeline. I don't get it. Why announce that they would kill RH9 and then say they'd release another one? What exactly is the difference between the current "workstation" version and the new one that's coming up? Faster changes? Fees? Maybe I'm missing something, or maybe this will become clear in the next few months. In the meantime, I'm already wondering if I will have to go back to the days of chasing around the net for updates and patches to the version of Red Hat 9 I already have installed.

Posted by diego on November 6, 2003 at 11:19 AM

the matrix revolutions script (abridged)

Suddenly it was clear to me that the best way to express how I see the movie was through something like this, that would let me have some fun with it. So I sat down and just wrote the thing, in the spirit of the abridged script for Reloaded.

Disclaimer: I am not a Matrix-basher. If you have any doubts, read here, or here, or here, or here, or here.

Update (7/11): to address questions on what I really thought about the movie in more detail, as well as my interpretation of what happens in it, I've posted an actual review here.

Warning: This contains major spoilers, essentially the whole plot of the movie. If you don't want to know what happens, don't read it until you've seen it.

Ready? Read on...

The Matrix Revolutions script (abridged) by Diego Doval

INT: Inside the Hammer Hovercraft. We begin were Reloaded left off six months ago.

Come on Keanu, wake up.

His brain pattern looks as if he's jacked in to the Matrix. Even though it's ridiculous, since there is no way that could happen, that seems to be how things are. That's okay though: who even remembers that we're in a ship called the Hammer, if we only refer to it by name?

I'm still worried about Keanu.

Yeah. I would be too.

They exhange SERIOUS GLANCES. The Medical officer leaves. Laurence Fishburne SHOWS UP.

Let's go see the Oracle. She's always fun.


They go see THE ORACLE.

INT: The Oracle's crappy apartment.

The Oracle looks DIFFERENT. It's ANOTHER actress.

Who are you? And where is Gloria Foster?

I am the eventuality of an anomaly...--- no, no. Strike that. I'm the Oracle, like Gloria Foster was, but there was some kind of thingamagic that turned me younger. Or I was deleted and subsequently rebooted. Or something.

Ok. (doesn't believe her). Where is Neo?

Neo is in a train station. The train station is used by programs who want to escape into the Matrix.

What? Into? The Matrix is a "prison for the mind"---How can someone escape into a prison?

Poetic, isn't it? And the Merovingian is involved, which is handy since his contract hasn't run out yet. So you have to go see him to a nightclub. This guy here (points to the Seraph) will go with you, since his contract hasn't run out either.




There will be lots of leather, plastic, and S&M stuff.

INT: The train station. All white and clean. Kinda comfy.

Keanu Reeves WAKES UP. There's a LITTLE GIRL standing NEXT TO HIM

Hi, I'm Sati. I am an all-powerful entity disguised as a child. This helps us pretend that this whole thing is still mysterious. Somehow.

You're not as bad acting as that kid who played Anakin Skywalker in Star Wars Episode I, are you?


Yes you are.

The little girl's FATHER shows up. Her MOTHER too.

Hey, long time no see.

Do I know you?

Sure you do. I appeared for a split-second in Reloaded, remember?

Ah, right. I remember you. (he doesn't).

I am a program, as is my wife. But we had a child anyway. Thought it would be fun. All you need is love, like John Lennon said. We are waiting for a train to let our child escape, and only she will go, but inexplicably we will get on the train with her. That aside, we are here for no particular reason.


INT: Garage, the entrance to the Merovingian's club, ominously called Club Hell.

Although it looks like a nightclub, this is a heavily armed facility full of deadly programs. We're going to get through them anyway after fighting them upside down.

They DO get through them. The upside down FIGHT looks COOL. They reach the MEROVINGIAN.

Give us Keanu.

Ok. But you have to give me the eyes of the Oracle in exchange.

She's a program. It's all a simulation. How could I give you the eyes of a piece of software? Read the script goddamn it!

Oh, well, it was worth a try. Nevermind.

Something HAPPENS. Carrie-Anne Moss gets the UPPER HAND. Apparently. She points a gun at the Merovingian but has 20 guns POINTED AT HER.

I said, give us Keanu, or we all die.

She's in love.

(curses in French) Ok.

Carrie-Anne RESCUES Keanu from the train station. They are about to LEAVE, and go back to the REAL WORLD.

EXT: The Matrix

I have to go see the Oracle.

Are you crazy? You know how long it takes us to set up an escape point? Do you remember how we ran around half the Matrix to get to one in the previous two movies?

Yeah, but that's not a problem anymore. We've done enough chase scenes already.

Ok. (shrugs) Go see the Oracle then. You're not plugged in anyway, plus, you can fly.

INT: The Oracle's crappy apartment.

Hi Keanu.

Hey Keanu. (to the little girl) Sati, go somewhere else now. We have already established that you are here even though you were supposed to have arrived in the train which is controlled by the Merovingian, and people in the audience should be confused enough already.

Ok. (Leaves)

(to the oracle) Who are you?

(rolls her eyes). Again with that? Look, I like candy. Isn't that enough to make you think I'm the same as before, even though I look, talk, and act differently?

Ok, I believe you. (he believes her). Tell me why you didn't say anything about all the other versions of the Matrix, the Architect, and so on, when we talked in Reloaded.

Well, the audience wasn't... --- I mean, you were not ready to know yet.

And how come I can see and affect the Matrix from the real world now? What was up with the Sentinels at the end of the previous movie? How could I stop them?

It's a mystery. You were not ready then. That's why you went into a coma. You should have died. But you were not ready to die. Besides, if you died, we couldn't do this movie. Also, I don't see that as a problem since you obviously have a wide range of super-powers, such as hacking into a computer system without any equipment.

Right. And what should I do now?

I see darkness spreading. I see empty movie theaters. I see us making millions and millions of dollars anyway. Everything that has a beginning has an end. You have to try to stop Hugo Weaving, because even though he's a piece of software running in a simulated reality, he is impossible to stop. The holes are too big.

The holes?


Cool. See you.


INT: The Hammer

Keanu WAKES UP in the ship, PLUGGED IN, even though HE WASN'T before, when he showed up at the TRAIN STATION.

Phew! Glad we got THAT ONE out of the way.

INT: The Oracle's crappy apartment

Seraph, take Sati away. Hugo Weaving is coming here for no particular reason.


Hugo Weaving arrives at the Oracle's apartment. He also surrounds the Seraph and the little girl with more copies of himself, in some other apartment of the same building.

INT: A crappy apartment in the same building as The Oracle's.

Hello. I am here to take you hostage for the rest of the movie until you inexplicably show up again at the end.

I seem to have misplaced the keys that let me run around the Matrix through backdoors in the last movie. So go ahead and take me and the little girl.

INT: The Oracle's crappy apartment


Come on.

Hugo Weaving doubts for a moment. This seems too easy. But he copies himself into the New New Oracle anyway, and keeps laughing. This PROBABLY has some MEANING.

INT: The Hammer

Keanu retires to think. Bane wakes up.

Hi, I'm Hugo Weaving, only you can't tell because it's a different actor whose body I hijacked in a split-second scene in the previous movie. I mysteriously survived the botched attack on the machines, and so I seem to be a traitor, yet I won't explain anything and appear to have a bad memory. This could mean something, or it could not. However, I have a goatee. Note that the bad guy from the first movie also had a goatee. Now, please leave me alone unrestrained with your medical officer and this scalpel here.

Ok. See you. (they leave)

Bane KILLS the medical officer. Meanwhile, Jada Pinkett Smith and her ship, the Logos, show up.

Hey, glad you could join us.

No prob.

Keanu emerges from meditation.

Hi. Sorry I took so long. I know we are about to face imminent extinction, but I wanted to spend a few hours and see if I could get out of this plot-mess nicely. But I can't. So I'll go to the Machine City, which at least should look cool.

Okay, take my ship. The Oracle said this would happen. Sort of.

When was that?

Oh, it was in the videogame, Enter the Matrix, available for PS2, XBOX, and PC. Sorry if you didn't play it.

This is stupid.

Do you have a better idea?


Okay then.

Hey Laurence. Sorry I didn't explain anything at all of what is really going on here and once I'm gone you'll still think this is the year 2199, and you still believe in the prophecy and such, even though we know that's not true. Gotta go!

(hugs Keanu). That's allright. I get paid anyway.

Carrie-Anne Moss and Keanu Reeves go into the LOGOS. Bane is ALREADY THERE. The Hammer LEAVES.

Hello Keanu. I am here to blind you with some implausibly well-placed electric cables so you can see the machines through your Matrix sense once your eyes are all burned up.

Nicely done. Anything else?

No, not really. Oh, wait. I have to say this: "Not impossible. *Inevitable*".


INT: Zion.

Hey, we're about twenty guys against a quarter million of these squid things, I mean Sentinels. But that's ok. We have lots of ammo. And they won't really try to kill us too fast anyway, so it will look cool.

Hey, I want a part in this movie too. I don't get residuals otherwise.

Stick around. I'm sure you'll be able to save the day at some point.

A huge battle follows. The Sentinels fly around THE DOCK without going further. They MOVE in SWARMS. It looks COOL.

INT: The Hammer.

Jada Pinkett Smith is PILOTING. She is an AWESOME pilot. The OTHERS in the ship note this loudly and often in SURPRISE, even though they've KNOWN her for YEARS.

Hey boys, go fire at those Sentinels that are chasing us.

But I thought we didn't have any weapons against the machines other than the EMP. At least that's what Carrie-Anne told Keanu in the first movie.


Cool. This is like a videogame.

Damn right. Just wait until we start releasing the spin-offs.

They KEEP the Sentinels at BAY with the guns they never had before. They approach Zion. They do LOSE the RADIO, so they won't be able to tell Zion that it's THEM arriving. Massive suspense builds. EVERYTHING looks VERY COOL.

Commander, something strange is on the scopes.

Who cares? It's not as if I can do anything right? We are surrounded from all sides and we almost have no weapons left.

It's a ship. Let's open the gates.

No. Besides, there would be no suspense if I just let you open the gates.

Come on... Please? Pretty Please?

Ok. Open the gates.

The SUSPENSE goes NOWHERE. The door mechanism is broken however. The Kid shows up and SAVES THE DAY, releases the mechanism of the door by shooting it up. Link's wife shows up ALSO to HELP HIM.

Yay! I saved the day!

The gates open. As soon as the Hammer enters the dock, they blow the EMP. All the Sentinels DIE.

Wow. We hadn't thought of that to defend the city.

You fool. Of course we thought of that. But it would have been too easy to defend the city by triggering a set of timed EMPs one after another, or to rig the tunnels with EMPs. There would be no fight. And what would we do then with all the workstations and the CG software we got for the previous movie? Besides, you'll see there are more Sentinels ready.

Right. Sorry, sir. I forgot it was obvious to use EMPs way back when at the beginning of the war too, instead of screwing up the climate for no reason.

Now you're getting it.

More Sentinels ARRIVE. The Humans descend to the LOWEST LEVEL of Zion, but the Sentinels ARE STILL DIGGING. The end approaches.

Keanu is still up there. He will save us through some implausible means.

EXT: The Earth's surface, approaching Machine City.

Keanu is BLIND, his eyes covered with CLOTH.

With this get-up, I hope everyone sees that I'm supposed to be a martyr.

Don't worry, they will. Hey, there's the city. Look, they are lining up thousands of cannons to destroy us.

No problemo. I'll stop them.

He STOPS THEM. The bombs explode in MID AIR. It looks COOL. A few Sentinels LATCH onto the BACK of the SHIP as well..

Sentinels. Kill them too.

I can't. There's too many.

What? There are only a few Sentinels. You're stopping thousands of bombs already. What could possibly be the problem with a few more machines?

It doesn't work like that. Fly into the upper atmosphere instead.

This is a *hovercraft*. Not an airplane.

Who are you going to trust? Sir Isaac Newton or me? Point this thing up.

The Hovercraft SHOOTS UP past the cloud cover, and we see THE BEAUTIFUL SKY. Then, like Wile E. Coyote, they REALIZE they can't fly, the ship starts to fall. They have, however, FLOWN PAST the city's defenses, which apparently are only pointed to the outside and can't turn around. The ship FALLS, accelerating rapidly.

Weird. The engines are off now, even though they were on a second ago.

Of course. We need to crash so you can die.

But wait, the city is hundreds of square miles in size! How will we get to the right place?

Don't worry about that now. It's all taken care of.

They CRASH. Carrie-Anne Moss is IMPALED with some sort of metallic pole which somehow showed up in the COCKPIT.

You know, when I was dying in Reloaded my last words where 'I'm sorry'. But that wasn't what I wanted to say. Couldn't remember my lines. Luckily enough, you revived me so I could die properly again... (she goes on talking about how she didn't say what she actually wanted to say for a few minutes). Ooops! Almost no time again! D'Oh! I love you. Kiss me.

He KISSES her. She DIES. We are reminded of the Merovingian's Theory of everything being CAUSE and EFFECT.


EXT: The Machine City

Keanu Walks off the ship, seeing everything through his Matrix-sense. We see a lot of COOL-LOOKING spidery machines. A bigger machine SHOWS UP floating at the EDGE of a CLIFF and turns a SWARM of smaller machines into a HUMAN FACE. Suddenly the face can TALK, too, even though it obviously has no vocal chords and there's no reason for it to have SPEAKERS.


Wait a minute. If you're the Matrix, which is a meta-human entity, how come you can anthropomorphize yourself? Isn't it impossible for a hive mind like you to appear as a single entity? Plus, since there's no Matrix-within-Matrix, all of this makes little sense.


Nevermind. Look, here's the deal. I have to go into the Matrix again. We have a bunch of cool special effects to show off yet. I will kill Hugo Weaving somehow, even though I couldn't before when he was less powerful and even though you can't kill him either, and in return we'll have peace.

Good idea. Off you go.

Keanu GOES INTO THE MATRIX. There are millions of Hugo Weavings. It's RAINING and it looks COOL. In Zion, the Sentinels immediately STAND DOWN, even though Keanu hasn't held up to his part of the bargain yet. Laurence Fishburne APPROACHES one of the Sentinels, who is now CUTELY squirming on the GROUND.

INT: Zion

Pet me.

EXT: The Matrix

Hello, Mr. Anderson.

Again with that? That stopped being funny ten reels ago.

Okay. Let's get on with it then. Here's how it is. There are millions of us, but only one of us will fight you, while the others look on with interest. Plus, now I can fly too.


They fight. It looks COOL. They fly. It looks COOL. They fight. It looks COOL. They open up CRATERS on the ground with their IMPACTS and they still survive. It looks REALLY COOL. Keanu looks beaten.

Why do you keep fighting?

Because I *choose to*.

And that's supposed to explain something?

I'm just reading my lines, you know. Look. Let's end this thing. You can copy yourself into me now.

Hugo Weaving COPIES HIMSELF into Keanu. He smiles. Keanu displays NO EMOTION, as usual.

I win! (a second later). Something's wrong.

We see Deus Ex Machina doing something with Keanu's BODY, and this might or might not have MEANING. It was a TRAP. Keanu let Hugo Weaving get into him so that he could destroy all the Hugo Weavings through their other-wordly connection to each other. Keanu can do this though he's ALREADY dead.

Nooooooo! This doesn't make any sense!

You figured that out just now?

All the Hugo Weavings EXPLODE. The GOOD guys WIN.

INT: Zion.

Yay! It's over!


Oh Keanu.

Oh Keanu.

They embrace lovingly, even though Jada Pinkett Smith's boyfriend, shown a second before, is standing two feet behind them, off camera.

The Prophecy was true.

What? No, no, no. There was no prophecy, remember?

Shush, honey pops. All is well.

EXT: A Park, Presumably inside the Matrix. Or somewhere altogether different.

We see the Matrix apparently being rebuilt, but we can't be sure. The Deja-Vu effect with the black cat we saw in the FIRST MOVIE shows up AGAIN, though this time it looks COOLER.


Hello. Glad we could all be here for this gratuitous scene that explains nothing and further confuses the audience.

The SERAPH and the LITTLE GIRL also appear.

I'm glad we could all be here too, particularly since we seemed to have been consumed by Hugo Weaving earlier. (to the Architect) So? Are you really going to let those saps be free?

Of course. What do you think I am, human? Even though we have already seen software programs display a wide range of emotions throughout the trilogy, including Love, Greed, and Rage, and some have also had children, I am incapable of betrayal.


The sun raises.

(to the little girl) Nice sunrise, Sati.

Thanks, I did it for Keanu. Do you think he'll be back? Or not?

What? And kill a multi-billion dollar franchise like this? Of course he'll be back. If not him, I'm sure we can convince other actors. Plus, we have the comic, books, and other things in the pipeline. You'll see. It will all be allright. Here, have a candy.

The little girl SMILES. They are all HAPPY. The sun SHINES.


Posted by diego on November 5, 2003 at 9:53 PM

matrix revolutions: first impression

I hate to be right.

Here's some reviews: NY Times, CNN, Washington Post, Salon, and they are pretty accurate, especially the one from Salon. I'll write more later (to get it out of my system).

But what's on my mind right now is what Switch says in the first movie when she's about to be unplugged (and killed) by Cypher:

"Not like this. Not like this."

Posted by diego on November 5, 2003 at 6:11 PM

off to see...

...The Matrix Revolutions. But before I go, a few thoughts. I have avoided reading reviews and comments and such, but that doesn't mean I can shut down my brain. :)

The other day I watched The Matrix and a couple of things caught my eye. First, Tank, the operator of the Nebuchadnezzar, survives the final attack at the end of the first movie. But in the second, he's dead. Why? How? Unknown. (Dozer, his brother, died in the first attack. Link, their brother in law, is the operator in Reloaded. I say brother in law though this in never clear--his wife/girlfriend is definitely sister of both Tank and Dozer, so I assume she's not Link's sister too... :-))

The more I hear "Zion", the more I think about "Zero-One", or 01, the machine city shown in The Second Rennaisance. Connection?

The Zion control room, which we see in Reloaded (which I saw again recently), all white and pristine, is actually run as part of a Matrix-like construct. We see the people plugged in for a few seconds... I don't know why I didn't notice that before. Without that information, the whole control room thing feels weird.

The monitor that watches Neo in the first movie, as he is questioned by the agents. It looks like one of the Architect's monitors no? Plus, in one of the trailers of the first movie, there was Neo but in the real world seen through those same monitors. Coincidence?

Unrelated but not much: here's an interesting Wired article about the Wachowski Brothers and their penchant for secrecy.

I have mentioned before how ridiculous the whole "Broadcast a pirate signal to hack into the Matrix" is ... but let's say for a moment that it's reasonable that the Matrix would leave holes open, since the whole point of Zion is that the humans think they're rebelling, when they are not. However. Why do you need to run around in the hovercrafts for that? Whatever happened to setting up antennas and relay stations?

And what is up with characters not telling others what they've seen? Example: Neo is all cryptic just after meeting the Architect. Why not tell Morpheus the whole thing? Just because he has condemned humanity to extinction? (Supposedly). Or: When Neo stops the Sentinels at the end of Reloaded. He clearly says "I can feel them" to Trinity. Then he stops them. Morpheus arrives. "What happened?" he says. Trinity replies: "I don't know." You don't know? Come on. "He said he could feel them, and then he stopped them." Is it too hard to say that? It's as if characters play the same game between each other as the one they are playing with the audience.

revolutions1.jpgA final thought on the whole matrix-within-matrix theory, et. al. Number one, some time ago (when it was clearly speculation--whatever you read these days could easily be the truth and so it is to be avoided :)) I read of other theories being bandied about in which the whole matrix is a prison for machines rather than for humans. Nice try, but this theory has as many holes and any other (starting with the why would humans create a prison for the mind ... for machines... and then... give them consciousness and feeling... and then... make them believe they are human... and then ... try to satisfy their desire to escape by building the Zion-level matrix.... anyway). We could go on with the infinite-matrixes theory for quite a while. About the only thing on which that theory hinges is the Animatrix clip in which humans modify a machine to make it feel like humans. Not a lot. But who knows, everything is possible. It would feel a bit like a cop-out methinks.

That said, I think it's narrowed down to two main possibilities.

Everything hinges on one key question. Is Neo human?

If he is not human, then this theory requires a Matrix-within-Matrix. Because otherwise he, as software, could not "operate" in the Zion-World. This theory explains his superpowers quite nicely, etc, and leaves very few holes open (like that ridiculous idea that the machines use humans for power when nuclear power would do just fine---it involves the machines taking a conscious "weird" choice to both subjugate and use their enemy but also become symbiotic with it, but as much as it can be explained, I've never been fully comfortable with it---in the case of Neo-as-software and Matrix-within-Matrix the actual real world is one level above and you could cook up any reasonable explanation for why things are what they are).

If Neo is human, on the other hand... things get ugly. Because, first, the matrix-within-matrix theory loses luster. Why? Consider: if Neo is really a human, then he would need some kind of extra-sensory, mythological superpower to do what he does inside the matrix. Bioengineering to his body to the level that he can manipulate software "with his hands" is utterly ridiculous (of the two options I'd even prefer the "superpower" one). If you change Neo so much that you don't rely on "powers" then he's not human and we're back to what I said in the previous paragraph. If he is human, and those are superpowers of some sort, then there is no need for matrix-within-matrix--already all bets are off. After all, if he has superpowers within the Matrix, why not some weird unexplained connection to the Matrix direct from the real world as well? Or why not explain the direct connection to the matrix through some other sort of weirdness?

So, if Neo is human, then it's all as we've seen, and there is only one Matrix. If he's a program, then Matrix-within-Matrix is necessary. Then again, this is a logical conclusion but the movie doesn't really have to be logical. And so it could easily be wrong. :-)

In other words: even though I'd like the matrix-within-matrix theory to be true, I get the feeling I am in for a dissappointment. Occam's Razor: of all the possible explanations for something, the simplest one tends to be true.

With the stuff I've seen in the trailers, all the "I believe in him, he believes in us, we believe in each other..." which sounds more like practice for verb conjugation than anything else, I'm getting the feeling we've hyped ourselves to expect too much from the Wachowskis. I certainly thought that this could be to the level of internal consistency of The Lord of the Rings or The Foundation Series, or even "smaller" works (in size, not in scope) like Neuromancer, Bladerunner, Alien, or The Diamond Age. This problem of overhyping appears in subtle ways. For example, the story of The Kid, from Animatrix, is completely passed over in Reloaded. This is ridiculous, of course. A work of art has to be self-contained (within a series at least). You go to the additional stories for more information, not to understand what it was all about. For example, the whole thing with the Osiris and its warning was done properly: you get the idea from watching Reloaded but you can learn more by watching The last flight of Osiris. The story of The Kid and possibly some of the stuff that's on Enter the Matrix (the game) are referenced in the movie with no background whatsoever, which is bad because not everyone has the possibility of, say, buying a game, or the time to play it. It's the difference between letting the depth of the story shine through the cracks (something that, for example LoTR does extremely well) and simply inserting references that leave you "uh?" which is easy to do to pretend there is depth, even when there is little or none of it.

Anyway, rant over.

Something I said back in May still applies: "[...] if Revolutions follows closely the tradition of Anime, we should prepare ourselves for an ending that might be ambiguous, even possibly unsatisfying by Hollywood's standards".


More in a few hours!

Posted by diego on November 5, 2003 at 11:31 AM

my feedster plugin @ mozdev

Jarno from Mycroft (a site under mozdev that deals with search plugins for Mozilla) stumbled today or late yesterday on my Feedster plugin for Mozilla Firebird and asked if I was going to submit it to the site. I hadn't thought of submitting it--I don't know why. Zoom! Went past me. Anyway, Ricky from mycroft stepped in and added it himself. The result is that if you now go to the Mycroft home page and search for "Feedster", you'll find my plugin. Nice!

Categories: technology
Posted by diego on November 4, 2003 at 4:27 PM

mozilla/netscape RSS ticker toolbar

Very cool... for those using Mozilla or Netscape (it doesn't seem to work with Firebird), you might be interested in checking out the RSS News Ticker Toolbar---an interesting use for RSS.

Categories: technology
Posted by diego on November 4, 2003 at 12:44 PM

quote of the day

"At a few hundred kilometers altitude, the Earth fills half your sky, and the band of blue that stretches from Mindanao to Bombay, which your eye encompasses with a single glance, can break your heart with its beauty. Home, you think. Home. This is my world. This is where I come from. Everyone I know, everyone I ever heard of, grew up down there, under that relentless and exquisite blue.


In the daylight, though, it's hard to see any sign of human habitation. But at night, except for the polar aurora, everything you see if due to humans, humming and blinking all over the planet. That swath of light is easter North America, continuous from Boston to Washington, a megalopolis in fact if not in name. Over there is the burnoff of natural gas in Libya. The dazzling lights of the Japanese shrimp fishing fleet have moved toward the South China Sea. On every orbit, the Earth tells you new stories. You can see a volcanic eruption in Kamchatka, a Saharan sandstorm approaching Brazil, unseasonably frigid weather in New Zealand. You get to thinking of the Earth as an organism, a living thing. You get to worry about it, care for it, wish it well. National boundaries are as invisible as meridians of longitude, or the Tropics of Cancer and Capricorn. The boundaries are arbitrary. The planet is real.

Spaceflight, therefore, is subversive. If they are fortunate enough to find themselves in Earth orbit, most people, after a little meditation, have similar thoughts. The nations that had insituted spaceflight had done so largely for nationalistic reasons; it was a small irony that almost everyone who entered space received a starting glimpse of a transnational perspective, of the Earth as one world."

From Contact, by Carl Sagan (1985).

Posted by diego on November 3, 2003 at 5:59 PM

on longhorn

Ole has posted a comprehensive comment on Microsoft's Longhorn. Great analysis, I agree with basically everything he says. The uptake?

Longhorn will certainly hurt speed. Whether it helps robustness remains to be seen; we can only hope it will, given that this is a big problem for Windows currently.
Microsoft always seems to design OSes for the next generation of technology (for whatever reason). I remember how impressed I was the first time I installed Linux on a 386. Even running X Windows worked well. I think this is something that shouldn't be underestimated. In any case, Longhorn will take a long, long time to have any impact. Most big MS customers are well-known for waiting until the first Service Pack to change to the new technology. If they release in 2006 (as they say) then it won't be until 2007 until it is reasonably deployed.

Four years. A lot can happen in that time.

Update: Scoble responds to Ole's piece. Interesting read.

Categories:, technology
Posted by diego on November 3, 2003 at 4:52 PM

daypop, blogdex and

Amazingly enough I forgot to include both Daypop, Blogdex and in my introduction to weblogs. I'm making the changes now--they're useful resources. They reminded me of their existence themselves, since the intro is at the moment in the top ten of of all three. Cool. :)

Categories: technology
Posted by diego on November 3, 2003 at 4:13 PM

alien: the director's cut

Yesterday it was my intention to see Alien: The Director's Cut, but it was sold out. In any case, I have seen the cut (I own it on DVD--it was released when they came out with the Alien DVD Box set a few years ago). It's pretty good--new transfer, surround sound and an additional scene that shows people being "cocooned" to create the Alien eggs (later, in Aliens, James Cameron would take advantage of the fact that this didn't make the first cut and change the creation of the eggs to be done by an Alien queen, and the people that were "cocooned" were only food for the eggs. In the original, people turn into the eggs. Yuck.) Maybe I'll get to see it next weekend after Revolutions (never underestimate the importance of watching a movie on a big screen with sound that blasts your ears).

Btw, if you were wondering why they re-released Alien now, here's the reason: Fox is coming out with Alien Vs. Predator in the summer of 2004 (Teaser trailer here) and I assume they want to build it up a little (After all, who remembers Alien Resurrection? :)). For those that are wondering if this Alien v. Predator thing is a marketing gimmick, not really. For years now Alien & Predator have battled it out in comics and even video games. I guess it was a matter of time until they made a movie out of it.

Posted by diego on November 3, 2003 at 11:37 AM

and in yet other news...

... my blogbrain is still a bit frazzled from writing the two pieces (articles?) on intro to blogs. Not even two full days out, and I've gotten great feedback and tons of links. Thanks all! Glad it was useful. I'll try to reply to some of the comments (and email) tomorrow as well.

Just writing them was quite an experience. It is quite amazing how easily we get used to things and we end up thinking they're obvious--only they're not. Once you start to peel back ideas and concepts, things that we (some of us) use and work with and talk about every single day, like "RSS", or even "server". The main thing I've learned all over again writing those posts is that we need to do better. Way better. Come up with examples. Simpler software. And so on.

Coincidentally, Dave points to a tutorial that he wrote back in May last year: How To Start a Weblog (For Professional Journalists). Very cool.

I'm wondering what other things seem obvious but are not, and might be a good topic for another introduction. For example, there are some things that I didn't go into detail in the articles, like blog posting from other clients...

That aside, today was a beautiful day, just a bit cold (not as cold as yesterday, when it was really cold!), sunny, breezy. I went out for a walk... and... I got ticket for Matrix Revolutions. Wednesday. November 5. At 2 pm. Yay!

Yes, that's when it opens. Supposedly it's synchronized worldwide: the first showing of the movie starts everywhere in the world at about the same time. 2 pm GMT... (I assume that in California they'll start at 12:00 am or 1 am at most. Kinda like they did with Star Wars Episode 1).

Going to sleep now, or rather, to read for a while. Lots of things today. A bit tired. Not bad for a Sunday! :)

Categories: personal
Posted by diego on November 3, 2003 at 12:39 AM

and in other news...

...Happy Birthday to my brother Sergio, who turns 24 today! When we talked today he said, so, you're not saying anything on your blog? I started mumbling stuff about not wanting to invade his privacy and so on. But it didn't work. :) He was having a good time, my parents, sister and his fiancee (yes!) were there.

Too bad I can't be there too, but I'll be seeing all of them soon... Travelling in December!

Anyway, again: happy birthday little brother! :)

Categories: personal
Posted by diego on November 2, 2003 at 11:47 PM

an introduction to weblogs, part two: syndication

The first part of this introductory guide was basically about publishing, but there is a second component to weblogs, perhaps as important, to cover, that of reading weblogs.

Note: For those that already know about weblogs, syndication, etc., I will greatly appreciate any feedback on this piece. This is a bit more technical that I would have liked, but there are some issues that, in my opinion, can't be ignored. If you have ideas on how this can be improved for end users (both technical and non-technical), pointers to other descriptions that they can go to and get a different take on this, please send them over. Thanks!

Moving on...

The need for syndication

Wait a minute (I hear you say) what do you mean reading? Don't we use the web browser for that? What would I need to know about reading weblogs?

Well, the answer is, technically, web browsing is just fine. But there is another component of weblog infrastructure that is quite important today, and that will probably become even more important as we have to deal with an ever-increasing number of information sources. This component is usually referred to as syndication. Syndication is also usually known as aggregation or news aggregation. The exact definition of the concept, or how and what it should be used for, is something that people could (and do) discuss ad infinitum, but without getting into specifics we can say that at least everyone agrees that certain things represent the idea of syndication or news aggregation quite well.

In the "hard copy" publishing world, syndication implies arrangements to republish something. Popular newspaper comic strips, for example, are usually syndicated, as are some news articles. While the meaning is similar in the web, it is primarily concerned with the technology, rather than with the contracts, or syndication agencies, etc.

Generally speaking, we could say that syndication is a process through which publishers make their content available in a form that software (as opposed to people) can read.

That is, if a site is supports syndication, and you are using appropriate software, you can subscribe to a certain site using that software. This allows updates on the site to be presented to you by the software, on your desktop (or web site that you use for that purpose) automatically.

This means that you can forget about checking certain websites for updates or news: the updates and news come to you.

Syndication is a dry, unassuming word for a powerful concept (as far as the web is concerned at least). It ties in together many ideas, and it is instrumental in sustaining the 'community' part of weblogs that I talked about in part one.


For an answer, let's go to an example. You have started your weblog, and you have been running it for a bit of time. You have found other weblogs that you enjoy reading, or that you find useful; you are also reading weblogs of friends and coworkers. Very quickly, you might be reading maybe ten or fifteen different weblogs. Additionally, you might also regularly check news sites, such as CNN or the New York Times. Suddenly, it's difficult to keep up. Bookmarks in the browser don't seem to help anymore, and you find yourself checking sites only every so often. Sometimes you miss a big piece of news that you'd liked to hear about sooner---or sometimes you find yourself wading through stuff simply because you haven't kept up. If you are a self-described 'news junkie' (as I am) you might already know about this problem, since keeping up with multiple news sources is also difficult. But with weblogs, the problem is greatly amplified: weblogs put the power of publishing on the hands of individuals, and as a result there are millions of weblogs. There are simply too many publishers. The problem of just 'keeping up' with what others are saying becomes unavoidable.

This is the problem that syndication solves. And the software that does the magic is usually called an aggregator.

Simply put, an aggregator is a piece of software designed to subscribe to sites through syndication, and automatically download updates. It does this regularly during the day, at intervals you can specify, or only when you are connected to the Internet. If the aggregator is running on your PC or other device, once you have the content you can read it in "offline" mode (unless the aggregator is web-based, which will require connectivity to the Internet at all times). For a more detailed take on what aggregators are, I recommend you read Dave's what is a news aggregator? piece. As usual in the weblog world, there is discussion about these definitions, see here, for example, for comment on Dave's piece.

A word of caution before we go on: for non-technical people, the issues surrounding syndication, aggregators and such can appear to get complicated if you start reading some of the links I provide here. There are acronyms and terms used here and there that you might see, such as "XML", "RSS", "RDF", "namespaces", and so on, that can be confusing. Let's skip that for the moment. I will (try) to go into them below (when necessary), in the section below, 'using your aggregator'.

Which aggregator?

Aggregators (or "news aggregators") come in different "shapes and sizes", and there are two main categories of aggregators on which everyone generally agrees on: 'webpage style' and 'email style' (also referred to as 'three-pane aggregators'). 'Webpage style' aggregators present new entries they have received as a webpage, in reverse chronological order (and so the end result looks very much like a weblog on the web does, but of pieces that are put together dynamically by the software). 'Email style' aggregators generally display new posts as messages (also in reverse chronological order) that you can click on and view on a separate area of the screen.

As in other cases, there are good arguments for preferring one over another, and in the end it comes down to personal choice. Reading different weblogs you might find people that are for one or for the other, and other people propose to do away with the whole thing and come up with something completely new. As with other things with weblogs: reading different opinions, and coming to your own conclusions is best. This is probably good in life in general :) but with
web-related things it becomes so easy to do that you generally end up doing it, sometimes without realizing  Look on search engines, other weblogs you like, leave comments, ask people you know, then try some of the software out. You'll find the one you prefer in no time (and, likely, as your usage changes and you have different needs, you might end up switching from one to another).

As it is the case with weblog software, all aggregators are invariably free to try, and many of them have to be purchased after a trial period (usually a month). Aggregators and weblog software are complementary, you could use both, but you could use one and not the other. It's quite possible that there are more people that use aggregators than people that have weblogs. (Certainly there are more people that read weblogs than people that write them).

If you have a weblog, chances are you also have a news aggregator already as well, because some weblog software includes news aggregation built in (as you'll see below). There are lots of news aggregators (and by lots I mean more than fifty, probably a lot more), and more on the way. Additionally, the underlying technology for syndication is simple enough that many software
developers implement and use their own aggregators.

This means that I can't possibly list all aggregators that exist here, and besides, there are other pages that do this already, such as this one, this one, this one, or this one. As it was the case with weblog directories, no listing of aggregator software is 100% complete (and probably can never be). However, I will mention a few aggregators that I know about and have tried myself, or have seen in action (and, in the case of clevercactus, that I developed :-)). (Lists in alphabetical order).

Some webpage-style aggregators

Some email-style, or 'three-pane' aggregatorsAll have one or two distinguishing features that make them unique. In the end, which one works better for you is all about personal taste and work patterns. Check out the aggregator listings I mentioned above, and look for something that grabs your attention. Try them out, and see which one you like best. Note: in many cases, aggregators are ongoing projects. Some are open source and are updated often. My advise to save time in choosing an aggregator is to go for the simplest route possible at first, and then with time try out new things. For example, if you're a paying-LiveJournal or Radio user, it will probably be better to use the built-in aggregator at first, especially as you try to find your way around all of these new concepts. If you want to try other options, or you use different weblog software, I think that a quick glance at the webpage of a product is usually a good indication of how much knowledge you need to set up something: if you don't understand what the page says, it's probably not for you. It all depends on your knowledge and how much time you want to spend on it. But, by all means: if you have the inclination or the time or both, spend some time looking at the different options. You'll be surprised at some of the cool features that some aggregators have, even if they are sometimes 'experimental'.

Using your aggregator

Once you've installed an aggregator (or decided to use the built-in aggregator of your weblog software), it's time to subscribe to some feeds.

Feeds (or newsfeeds) are usually the name given to sources of information (used by aggregators) to obtain the content they display. Feeds are technically similar web pages, like those that are displayed in a web browser. Web pages, however, are written in HTML (HyperText Markup Language) which is designed to create pages readable by humans. Feeds, on the other hand,
used for syndication, are intended to be "read" (or rather, processed) by software, and so they have different type of information, are more structured and strict in the data they can contain. Feeds are written using a language called XML (eXtensible Markup Language) using a de-facto standard "dialect" of it called RSS.

Aggregators let you 'subscribe' to to these feeds in different ways. Most pages identify the feeds as 'Syndication', or 'RSS', or 'RSS+version number'. (See the next section if you'd like to know more about these differences). Many pages have an orange icon that says "XML" like this one: . Depending on the software you are using, subscription itself can be easier or more difficult. In all cases, the following set of steps will work to subscribe to a feed:

  • Find the link on the page that says "Syndication", "Syndicate this site", "XML", "RSS", etc.
  • Right-click (or press-hold in Macintosh) over that link. Your browser will show a menu of options, and one of them will be "Copy Link Location" or "Copy Shortcut". Select that option.
  • Now go to your aggregator and find the option to Add or Subscribe to a new feed. Select it and when you are requested to type in the URL (link) of the feed, right-click (or press-hold in Macintosh) again on the field and select "Paste". This will make the URL be pasted on to the field. If right-click doesn't work, you can try with keyboard options: Ctrl+V or Shift+Insert on Windows, or Command+V on the Mac.
Now, just to be clear. These instructions are the most basic of all, and supported everywhere, and they are important so that you can use them "when all else fails". Many aggregators come built-in with a choice of feeds that you can subscribe to with only the click of a button. Many aggregators allow you to type in simply the URL for the page you are visiting (as opposed to that of the feed) and then discover the feed for you. Others also establish a "relationship" with your web browser so that when you click on the icon or link for a feed, they give you the option of automatically subscribing to that feed.

Okay, now for a bit of a detour. If you'd like to know a bit more about RSS and related technologies and have an interest in the technical background, or are technically proficient, please read the next section. Otherwise, skip to the following section.


Okay, tell me more about RSS.

Before I start: this is a highly charged (and even emotional) issue in the weblog developer community. People have very different opinions, and this is just my take on the situation. By all means, go to different search engines and search for "history of RSS", "history of syndication", "RSS politics" and similar terms to find pointers to different sides of the argument.

Politics? Did I say "Politics"?

Yes. Yes I did.

Sometimes people mean different things when they say "RSS". Some people see it only as a way to syndicate web content. Others see it as a way to pull all sorts of information into clients. There are different opinions as to how it should be used, how it should do what it does, etc.

In the majority of cases RSS stands for "Really Simple Syndication" but you might come across other places where it is described as meaning "RDF Site Summary" (RDF, which stands for "Resource Description Framework" is yet another XML dialect, that is more flexible, but also more complex). I prefer to separate them clearly and call RSS-based feeds RSS feeds and RDF-based feeds RDF feeds, but I might be in the minority (So when I say "RSS" I mean Really Simple Syndication, not the RDF-based format). There is another syndication format being developed at the moment (also XML-based) called "Atom". Additionally, there are different versions of RSS: 0.91, 0.92 ... 2.0 (the current version of RSS is 2.0)... and RDF-based syndication is sometimes called RSS 1.0 (yes, this last one in particular is quite confusing). These are various formats for syndication. If we lived in a perfect world, we'd only have one format. But that is not the case.

I can imagine you're thinking: So, even if I know about technology, why do I care about all of this "XML mumbo-jumbo"?

Well, if you start your own weblog and begin to discover new weblogs and new feeds, and are curious about the technology, more likely than not you will read about this, about people passionately arguing about these things, mentions of RSS of this version and that, and so on. And so it's a subject that can't really be completely avoided. If you're interested in knowing more, not telling you about this would be like pretending that you can fly across the Atlantic and think that you'll never have to know about the fact that you are likely going to experience some kind of delay on departure.

But, as the Hitchhiker's Guide to the Galaxy says: Don't panic. :-)

More specifically, I'm mentioning this for two reasons:

  1. Because you, as a user (that is nevertheless aware or interested in the technology behind this), are likely to encounter this in subtle forms. For example, you might go to one news site and see that they say they provide "RSS 0.91 Feeds". Or you might see the XML orange icon shown above. Or you might see they say just "RSS", or "RDF". You will quite possibly see mention of all of these names and acronyms when you're looking at aggregator software. So, in seeing all this, you might wonder: is one better than the other? Which one should I choose? To that I'd say: start by not worrying too much. RSS is the most common format by a mile, and that is good because it's the one you'll be most likely to encounter and so you won't have to think about this much. Additionally, all aggregators support the most used formats, and many of them support all the formats in existence. In general, you don't really have to even know which of these formats is actually being used. But once you get a grasp on things, it might be a good idea to read about the history behind these differences and make up your own mind about them. (If you are a software developer, and know about XML, good starting points are Dave's RSS 2.0 political FAQ and Mark's What is RSS? and History of the RSS Fork. Beware: these discussions tend to get quite technical very quickly).
  2. Because as you're perusing weblogs or news sites, it's better to be aware of what these things mean to avoid getting confused. Once you have this bit of information, it becomes easy to look for the XML orange icon, or some link that says "RSS" or "RSS+version number" or RDF or whatever.
Again, if we lived in a perfect world only a few people would ever have to deal with this stuff. But this is relatively new technology, and we are still trying to figure out all of its uses, and in some cases, what it is exactly. The important thing is, in my opinion, that you as a technically proficient user don't feel as if things are beyond your grasp. My pet peeve is when people using say "I don't know what I did, it seems I broke something". But when (say) the washing machine stops working, we never say "I broke the washing machine" simply because we were using it. We say "the washing machine broke down". So why is that? I can think of many reasons: error messages in computers generally put the burden on the user, for a start. But regardless of that, what I would say is: if something seems complicated (like all of this "XML mumbo jumbo") it's not a problem with your knowledge of computers. It's our problem, a problem of the software developers. (If you get involved in the technology or the community in any way, then it will be your problem too :-)).

And so what? You ask. Well. Weblogs allow a new level of interaction. You can make a difference. Perhaps for the first time ever, users can actually influence and participate directly in the creation of the tools they use everyday through the tools they use every day. So if there is something that is difficult to use, something confusing, it's likely that you can find a weblog or reference for the software author(s). Post a comment. Write your own post about it. Get involved if you can, and by that I don't mean 'develop software'---simply giving opinions and ideas is a good start. People will listen, and the problem might even be fixed!

Now back from the technical depths of this section, and to simpler things.

So how do I find these 'feeds'? And how do I create them?

First of all, let's deal with feed creation. Just as your weblog software automatically generates the HTML page that is displayed in a browser (when you post an entry), most weblog software also generates the feeds for you, and places a link for the feed in your homepage. All of this is done automatically by default---if you are not sure of whether or how this is happening, check your weblog software's help page for "Syndication" or "RSS" and you should be on your way.

Finding feeds to subscribe to is not so difficult. If you're reading other weblogs or you find one of them that looks interesting and would like to keep up with what they are writing, just look for the link or icon that identifies their feed and subscribe to them "as you go". In some cases, the aggregator software will come pre-subscribed to some feeds, or will suggest new feeds to subscribe to. The methods used to find weblogs (mentioned in part one) apply to finding feeds as well, for example, both Technorati and Blogstreet (as well as other sites) allow you to find new weblogs, and hence new feeds to subscribe to. There are "Feed directories" like Syndic8, that allow you to find feeds of certain topics easily. Finally, there's Feedster which is a cool search engine that deals specifically with syndication. All the results in Feedster come from feeds, and so lets you not only look for information but also find new feeds that you'd be interested in looking at.

At the beginning you mentioned news sites. Are there 'feeds' for news sites too?

Yes. Many news sites and organizations today support news feeds. Examples: the BBC, Rolling
Stone Magazine
, and Look in your favorite news site, or use the feeds recommended by your aggregator (if any), or use some of the directories mentioned in the previous section to find more.

Why did you say that syndication is 'instrumental' for weblog communities?

I think that weblogs are cool but syndication+weblogs is really cool. It's a case of 1+1 = 4. Because syndication allows you to subscribe to many sources, you can keep up to date with a lot more and so they allow to maintain people up to date easily on what others are doing in their particular community. Things like Feedster and Technorati reinforce the "loop" that feeds create. These loops are "loosely coupled," connected through links and with people notified of updates through feeds, both done in unobtrusive ways. The conversation moves across sites, as people find the time or have the interest to do it.

Posting from your aggregator

Since a big part of weblogs is the 'conversation' that is established between different sites, it would be great if you could just re-post a piece of something you've read, or comment on it, no? Many aggregators let you do just that. For example, since Radio is both weblog software and an aggregator, you can use them 'in tandem', to post comments on things you're reading about. NetNewswire, NewsGator, FeedDemon, clevercactus (as well as others) all allow you to post to weblog software as well as reading feeds. I won't go into the details of how to do this mainly because the configuration varies from software to software, but I just wanted to mention it as something that exists, and that you might find useful as you get more comfortable with weblogs and aggregation.

Final final words

Both part one and two are an overview of concepts that (as I said) are still relatively new. As a result, things are still evolving, and new applications are being created all the time. Sometimes the technology can appear to be daunting, but there's lots of people working on making it better, and easier to use. Once you are more 'embedded' in the world of weblogs, you will start finding new uses and applications---things that were simply not possible only a few years ago---to communicate, collaborate and express yourself.

See you in the blogsphere! :-)

Categories:, technology
Posted by diego on November 2, 2003 at 7:33 PM

Copyright © Diego Doval 2002-2011.