Now blogging at diego's weblog. See you over there!

server platforms: choice, or lack thereof

One of the things we're doing in the process of upgrading our infrastructure is getting a new public server and a development server. And one of the biggest questions is (as usual) which platform to base it on.

The first problem that comes up is that pre-install choices for OSes are pretty much limited to either Red Hat Linux or Windows Server 2003 (In some cases Windows 2000 Server). For obvious reasons, the development server should be a copy of the deployment server, which means that if, if you have different providers, you have to settle for the "intersection" between the various offerings.

Note: all of these comments are taken from the POV of multiplatform server applications (Java, Perl, etc) with small server clusters. That is, if I say, "Windows and Linux are roughly similar in X" it means roughly similar in that context. Java in particular is sort of an equalizer in that sense, in fact helping Microsoft by taking Windows Server more up to par (who would have thought?). Additionally, it's only been in the last few years that Sun has been more aggressive in ensuring parity between platforms, there was a time (say, 6, 7 years ago) when Sun's Windows VM was the best all-around VM (remember it took a while for Sun to implement Java native threads on Solaris?). I have no doubt that in other contexts, for other types of webapps or webservices, or at different scales (say, deploying 100 servers instead of 10) both the parameters and the results of comparing these choices would be quite different.

Some might say that Windows "is not an option", but I prefer to make decisions based on objective information when possible (with personal preference a factor, of course, but not an overriding one), and I think it depends on how much money/resources you've got to deal with it. As far as one Internet server is concerned Windows is more expensive than Linux but not by a huge amount (as a comparison, Dell.ie pre-installs Windows 2003 Server for Euro 700, and RH9 for Euro 170). But when you deploy it in an internal network you have to start thinking about Client Access Licenses (CALs) which cost $50 a pop or more, installation licenses, and so forth, and this is where the "resources" come in: if you install Windows you need someone to spend a lot of time figuring out licensing and making sure that you are using the licenses properly, etc. So aside from Windows being more expensive, it's also more difficult to manage from the licensing point of view. Additionally, going beyond a few servers complicates matters even further. Needless to say, startups usually will not have the time for that. I know we don't.

A few years ago I deployed+used Windows Server in the company I was working for then and it worked okay (that is, it did what it had to do, nothing earth-shattering). The company was basically using Windows on the client (IE was the primary target platform) and that sort of dictated that the servers be Windows as well, particularly because there weren't that many servers. Downsides were mainly that you had to keep up with the neverending stream of security updates and the licensing stuff, but in that case there was a person who took care of the IT infrastructure.

I haven't used Windows 2003 Server, but Windows 2000 was pretty stable (Again, all of this in the context of multiplatform server apps that connect to an SQL DB in the backend). I think it's interesting that now that Windows is more on par in terms of stability and features (clustering, remote terminal, etc) with Linux what really becomes a bigger barrier of adoption is both the price and the "management" of licenses. Linux is just easier in that sense: make as many copies as you want, install as many servers as you need, with as many clients as you like. Microsoft is, in effect, shooting itself in the foot (for a certain segment of the audience at least) by making their licensing more convoluted than it should be.

Now, though, the comparison has become a little more complicated, since Red Hat has discontinued Red Hat Linux and split the process in two. On one hand we've got Fedora, which is sort of the "spiritual heir" of RH9, and on the other hand you've got Red Hat Enterprise Linux (RHE for short) which is their commercial offering. RHE comes in three flavors: WS (Workstation), AS (small-medium server) and ES (enterprise servers). As far as servers are concerned, the main differences between AS and ES are (btw, the information on RH's site isn't nearly as clear as it should be):

• AS supports only x86, while ES also supports Itanium, AMD64, and others.
• AS supports only one processor, while ES supports 1+.
• ES has round-the-clock tech support, while AS's is more limited.
• ES supports more than 8GB of memory on x86, while AS does not.

The problem with any of these options is, of course, that you have to pay a subscription that on the surface rivals the price of Windows Server. I say "on the surface" because the licensing is simpler, you basically pay the subscription, you get the updates when you want, and that's it. One subscription per install, and everything else is pretty much Linux as usual, which is substantially simpler (and so less costly) than Windows. (Btw, Russ also had some comments yesterday on his views on the Red Hat transition).

Ah, but why not go with another Linux distro you say? SuSE? Gentoo? or Debian? (Debian has lots of fans :)). Why not FreeBSD? Well, here we are back to what I mentioned at the beginning, that many providers pre-install either Windows or Red Hat. They don't preinstall, say, SuSE, or whatever.

Fine, you'd say: get a clean machine and install your favorite free distro yourself.

Which is an option, yes, but, but... if you are getting dedicated servers that are pre-installed on a remote location you're faced with the prospect of either a) going to the remote location to do the reinstall or b) doing the reinstall remotely, without being at the console. Both are possible (maybe b is not in some cases) but neither is very appealing, especially when you want to work on what you need rather than spend time making the OS boot properly. If we had more time to work on that things might be different, but that's not the case.

So.

For the moment, Red Hat is a better option for this kind of usage, since there is an upgrade path (even if it's convoluted) to either RHE or Fedora (although Fedora might be too much in flux as a distro to base a production system on--I guess that time will tell). If these moves by Red Hat make some hardware or Internet providers pre-install systems other than Red Hat there might more choice in the future (for when you need things to "just work"). And, if Microsoft changed their Licensing scheme to be something like "here, sign this two-page contract and pay us $500 a year and you get your updates and you can use this with as many clients as you want, etc" (kind of like what Sun did with their new licensing scheme) then Microsoft would be more of a contender in this area I think.

Oh, and btw, either Mac OS X Server or Solaris are indeed good options, but the hardware is simply more (sometimes a lot more) expensive, at least here in Europe, which makes it difficult to justify cost-wise. I hope that someday I'll understand why, if everything is built in factories all over the world, shipping stuff to Dublin instead of doing it to Palo Alto, CA makes prices jump 40%. Oh yeah, and software is more expensive too. I guess that the bits get tired from all that swimming and have to be compensated somehow. :-)

rebooting

There are lots of news related to clevercactus, and a number of other things I wanted to comment on. This last few weeks have been really busy (being away in December was also a factor) and as a result I haven't updated as often as I wanted. (Plus, I've had a bad cold that I haven't shaken up completely--what with the subzero temperatures and all).

Regarding clevercactus, a few things to start (I'll elaborate more on these points in future postings):

• We're "upgrading our infrastructure" (office, servers, et. al).
• We're hiring! :)
• That aside, I've been working on a new app that we plan to release soon (and I mean soon). After that there will be a new release of clevercactus pro, which, contrary to what it would seem (since the external release hasn't seen updates in months), is not dead :-). Lots of things have changed/improved since beta2, and it's time to get the new rev out of the gates.

Things are just about to get a lot more interesting.

ps: the category for this entry is still "spaces"--it should be changed to "clevercactus". Will probably do that tomorrow. :)

yet another email virus

"Fastest ever to spread", so far at least. Some coverage here from News.com. I have some thoughts on this, but will leave them for later (busy busy!).

weather chaos

I've been thinking about writing this for a few days now, maybe more than a week, and for some reason I never get around to it. I start writing down a primer for chaotic dynamical systems and then I think that it will be too much and not very interesting for most people ... but anyway, here are some thoughts.

The recent harsh cold weather in the Northern Hemisphere has lead to some global-warming naysayers to use this as "proof" that global warming is not happening.

Right.

The problem is that global warming will not simply make the Earth "warmer." By raising the temperature, several things happen. For example, glaciers start to melt (which is happening now at an alarming rate) and the cold water not only raises water levels but affects warm currents that are vital to preserving elements of global climate (a big factor is reducing the salinity --and thus the density--of the water, others are that the increased temperatures affect wind patterns which a;sp play a role in determining ocean currents). This could for example slow down or shutdown key oceanic streams such as the Gulf Stream. The imbalance created by the higher temperatures and changes in ocean currents would create extreme weather patterns all over the globe: superstorms on some parts, droughts on others, floods, and so on.

Global warming is, then, a misnomer. In our MTV-3-seconds-a-news-clip culture we probably need a new phrase to describe what global warming does. "Weather chaos" is pretty accurate, but it doesn't have the 'zing' I think.

Additionally, these changes reinforce each other, bigger storms pour even more rain on the oceans, which affects the water even further, just as the winds start behaving in more violent and uncommon patterns. The increased cloud cover brought on by increased temperatures also feeds the greenhouse effect, trapping heat and increasing the temperatures even more, making the climate even more unstable. In the end, we might not see it coming. Dynamical systems have a way of shifting directions dramatically and without warning.

None of this is 100% certain of course. But what ever is? The real question is not "are we sure this could happen?" but "What can we do to stop it from happening if possible?" Assuming it happens, once it starts there will be no turning back, no quick fixes. The weather will be out of control, and we can kiss our precious little all-singing all-dancing civilization goodbye.

All of this is a long prelude to say that I simply don't understand what is the problem in getting some action behind trying to curtail emissions, etc. I can't understand at all why some people argue that trying to cut back on CO2 emissions would "hit the bottom line". What happens when there's no bottom line to hit anymore? Why, why, why is it that western societies, that are so conscious of "health care" seem to worry little when a disaster of massive proportions maybe not too far ahead? Only because it's 20, 30, 100 years into the future? We can all hope that these scenarios are indeed wrong. But the evidence is piling up to the contrary. And if they happen to be right... what then? Will we just limit ourselves to pouting and moaning about it? This is potentially catastrophic as few things are, and not taking it more seriously seems to me a massive folly.

What comes to mind is a paragraph from Richard Preston's The Hot Zone:

In a sense, the earth is mounting an immune response against the human species.

In Preston's book it is related to the emergence of new viral threats, but it might as well apply in this case.

Now, as it happens, the other day when I saw The Return of the King I also saw the trailer for The Day After Tomorrow which is basically, from what I can see, about this scenario. One good thing about the movie is that it might increase awareness; the bad thing is that people might say "oh, it's just a movie" and since it's from the director of Independence Day reach the conclusion that this is as likely destroying an invading alien race by infecting the Alien's mother ship using an Apple Powerbook with software written in about a day. What is a;sp weird is that the first thing you hear (on the trailer at least) is that "Meteorologists are at a loss to explain what is causing this weather". This is complete crap of course. We will know. We will know that the delicate balance of the Earth's atmosphere has been broken, and that a new balance has to be found, and it will most likely come in the form of another Ice Age.

Maybe for the first time in history we have primitive knowledge that enables us to see this problem coming and take action (and yes, that same knowledge creates technology that is likely, in fact, to be exacerbating something that would have happened naturally anyway, or even creating it artifically). But instead of taking action we are sitting comfortably arguing about it. Even in a strict analysis "by the numbers" it seems to me that the potential cost (investment in cleaner technologies, paying for more serious research on the topic, intergovernmental cooperation on the matter) would be insignificant compared to the consequences.

Sorry if this seems a bit depressing. I'm just venting a little. I do hope that we'll turn around. Humans. You never know what we're going to do next!

the impact of the macintosh

On my entry about the Mac's 20th birthday there were a couple of comments that are interesting enough to echo here.

First, Chris posted a link to the 1984 commercial (thanks Chris!).

Second, Doug posted a long comment that I'll quote verbatim before replying:

As I recall, the '1984' commercial did not have much impact. It was advertising a product that most viewers had never heard of, but failed to introduce the product or suggest any of its benefits. The commercial was never run again.

Also, the Mac wasn't really that 'innovative'. It didn't have much of anything that the Apple Lisa didn't have... except that the Mac was at least somewhat affordable.

Finally, I would note that the 1984 Macintosh was generally considered to be a flop. Sales were abysmal when compared with the then-ancient Apple ][ -- it had taken 74 days for the Mac to sell its first 50K units, but when the IIc was introduced a few months later, Apple sold 50K of them in 7 hours. Worse, Mac sales were miniscule when compared with the IBM PC. The Macintosh failed to stop Apple's decline from #1 microcomputer maker to tiny niche player. The 9" black-and-white low-res screen and undersized keyboard marked it as a rich man's toy.

It was the introduction of the Mac II in '87, with available 13" 640x480 color display and a real keyboard, that finally gave Apple a system that was attractive to professionals.

What Doug says is all true. However, I disagree with his implied conclusion (that the Mac, or the launch even, weren't as important as they were).

Specifically on the points Doug mentions. The Lisa was about one third of the price of a Xerox Star. The Mac was one-fifth to one-fourth of the price of the Lisa. The Mac-to-Lisa jump was done in part due to hardware advances, but more importantly, due to top-notch engineering. Price matters. Also, the Mac improved on the Lisa in several aspects and to a degree it was an independent project that was running in parallel and that Jobs took over when the Lisa project started to sink under its own weight.

Regarding sales, well, the first Mac was in part underpowered (a year later the addition of a hard drive among other things made the product a much better proposition, and it sold accordingly), which hurt its sales. But at the heart of the difference in sales there's also the "Apple factor": not licensing the OS, using proprietary components, pushing for very high margins, etc., which had a big effect IMO.

As far as the impact of the 1984 commercial is concerned, I would just ask how many other commercials for computers are still known to the level that one is, and leave it at that.

Finally, if we start comparing things on the basis of what had already been proposed or developed to a degree (but not seriously marketed) before a product was launched, then the Lisa wasn't really that 'innovative' either because it borrowed a large number of concepts from the Xerox Star (which in turn borrowed from Engelbart's work), as I mentioned in the entry. There's a big differecence between doing something for a tiny audience or playing with it in a lab and designing it so you can manufacture hundreds of thousands of units a year of it. We could also argue (for example) that as crucial as the Mac II was the Laser Printer and PageMaker, which made the idea of "desktop publishing" a reality.

The original Mac planted the seeds for what was to come, and the Mac II and everything after it, on all other sides of the aisle (yes, for example, Windows), resembles to a large degree what was in that "rich man's toy" as Doug describes it.

A "desktop". Icons. Mouse. Graphical filesystem navigation. Applications running inside windows. Menus. Bit-mapped graphics. The idea of a common "Look and Feel", established through published guidelines (this one is fading now though, what with our modern skinnable apps and such :)). An API (The Toolbox) for developing apps against it, with high-level OS services. Most of these things existed before it, but the original Mac had it all, and in some respects it was several years ahead of its time (we all talk about computers as "appliances" now given the right context, but Jobs always saw the Mac as that, a machine that could sit comfortable in any living room without looking out of place).

What is sad is that we haven't really moved too far beyond those basic ideas, particularly since many of them were not designed for the massive amounts of information we deal with today (say, the number of files on our hard drives) and so in some sense some ideas have been as much a problem as a solution. But that's how it played out.

The impact of the Mac is therefore, in my opinion, difficult to underestimate. It defined what computers should be rather than bringing up a fancier version of what they were. And that's what's important, I think.

Update: [via Peter] Andy Hertzfeld's Folklore page dedicated to the early days of building the Macintosh. Very, very cool.

apple: beyond the mac

An News.com article that looks at the past and present of Apple and how its recent move into "digital lifestyle" products might play out. Interesting read.

dsl half-broken, again

For the last couple of days I've had the same problem I saw about two weeks ago with my DSL connection. Some sites were unreachable while others worked fine. So I bit the bullet and called up support (again) and this time I got some more details. Apparently Eircom added a pool of IPs recently and one of their transatlantic provider (AT&T) doesn't know about them yet. So whenever you go through them, they drop the packets. After some changes and trying out other usernames, it started to work again. Hopefully this problem with AT&T will be solved soon (they said this week, I'm skeptical); for some reason I keep getting assigned to that broken pool and half the internet is innaccessible for me (and people can't reach me either, which is bad for P2P apps). We'll see.

atomenabled.org

[via Don] Sam released AtomEnabled.org. Missing: a feed to keep up with updates. Site looks great though!

the mac turns 20

Almost forgot: January 24, 1984, was the launch of Apple's Macintosh. Yes, a lot of the ideas were already present in Xerox's Star, but the Mac did have include many inventions (I guess that today we'd call them "innovations") and it did mean that all of these things were within the realm of the affordable. More importantly, the Mac forced the rest of the industry to improve.

Some time ago I found online a copy of the famous '1984' ad that launched the Mac, directed by Ridley Scott. It's really great. I can only imagine the impression it must have caused at the time. I mean, a computer being advertised, quite literally, as part of a revolution? These days all "revolutionary" icons have been turned into marketing gimmicks (say, images of Che Guevara being used to sell T-Shirts, Jeans and such), but 20 years ago that must have been quite the thing to see. And to sell a computer no less.

One more thing: something interesting from this CNN article:

Twenty years ago, on January 24, 1984, Apple Computer launched the Macintosh. It contained virtually unknown features, including simple icons, and an odd little attachment called a mouse.

Many newspaper stories at the time had to include a definition. Silicon Valley's newspaper The San Jose (California) Mercury News, for example, described the mouse as "a handheld device that, when slid across a table top, moves the cursor on the Mac's screen."

Heh.

the lord of the rings: the return of the king

And so it ends.

I just came back from seeing The Return of The King, and I must say: Wow.

The Two Towers had been (I admit) slightly dissapointing. Too many liberties taken with the story, for my taste. Mostly, I think that the second movie suffered because it didn't end with a massive cliffhanger (as the book does) but rather carried the "cliffhanger moments" into the beginning of RoT. But RoT being what it is, all is forgiven.

The battle of Minas Tirith has to be the best battle scene I've ever seen, and maybe the best ever put on the big screen. When the Rohirrim charged against the hordes massing outside the city I could only hope that it would last just a little longer. Frodo's and Sam's journey from the gates of Mordor to Mount Doom is a bit cut here and there (no encounter with a company of Orcs, for example), but it retains its essence (I can clearly remember the overpowering dread throughout those 60 pages in the third book).

How the tension was maintained across the wide range of things that were happening at the same time was also very impressive. Eowyn's confrontation with the Nazgul was great (although they skimped on the consequences for both her and Merry. Okay, maybe not skimped--ignored :)).

As far as other things that were missing, well, the "Scouring of the Shire" was the single biggest no-show, but I understand why they had to cut it. I can only hope that the "director's cut" on DVD will include it (assuming they filmed it, that is). No mention of the effect of the Ent's drink on Merry and Pippin, and an oversimplification of what the Palantiri did (including no mention of the role Gondor's Palantir had to play in Denethor's madness). The corniest, most off-place moment was Aragorn asking Gandalf "What does your heart tell you?" which made me roll my eyes. Come on! He's a demigod for crying out loud! (a Maiar, like the Balrog, Saruman and Sauron) What is up with this sappy Titanic-like moment? (DiCapio wasn't around, I checked). I guess it made sense dramatically. Anyway. A few oversimplifications here and there, the most notable being at the end, with a simplified version of the fate of The Fellowship ommiting a number of important details (e.g., Sam's ultimate destination, having been a ringbearer if only for a short time, or some more details about the Three Rings of the Elves). But those are small problems compared to the achievement that was putting LoTR on the big screen. I probably count as one of those "die hard readers" that are usually so hard to please.

So. Great, great movie, and a worthy conclusion to the trilogy (movie-wise that is). If you haven't seen it, try go see it in a theater, it's what it deserves. Now, I just have to wait for the box set with all the extended versions and watch it all over again... :-)

porting swing to swt: a tutorial, and the Tiger is out

Must check out when I have the time (In a couple of days? Early next week? Maybe). Looks interesting, both items via Erik. First, a tutorial from IBM DeveloperWorks on porting Swing apps to SWT. Would be a good way to update on my first impressions.

And, in other Java-related news, (I saw this a couple of days ago, but was reminded again today) a pre-release of Tiger (J2SE 1.5) is now available for testing, very quietly (here's the download page). From the looks of it the installation is still a pain, particularly when juggling multiple JDKs in a single machine (The other day I tried out JBuilder Foundation for a moment and it decided to set itself as the default JDK. Reminds me of the days when media apps like Windows Media, Quicktime and Realplayer used to take control of the registry entries for media formats without asking). Hopefully Sun plans to fix these problems at some point.

comment spam filtering - it's all about the IPs

Sam describes his new comment spam filtering system. Quote:

Then it struck me: from an ip address I have never seen before. Light bulb.

Hit and run. Strangers. People who never have been here before. These people are unlikely to be seen again.

And if they don't come back, it is not possible to have a two way conversation, is it?

So I set to work. I wrote a script to scan my Apache logs for everybody who has ever visited my weblog within the last week. Bots, aggregators, and an occasional carbon based life form, I make no distinction.

Then I add in everybody who has left a comment in the last ninety days. And not just ip addresses, but also urls.

All these people are welcome to comment freely.

Very, very cool idea! Use the logs to establish an implicit community effect, a kind of automatic self-updating whitelist. It leaves me thinking "and where else can this be applied?". Mmm...

rethinking weblogging

[via Danny] Bruce Ecker: rethinking weblogging (and everything else). Great piece. Hard to summarize. Go read it! :)

c-span.org

One of the most interesting things to watch right now in politics world-wide is, for me, the US presidential race. I get the good stuff from prints and blogs (including, for example, cool things like channel Dean), but there's the live side that you just have to see However, being in Ireland, and not having any US News networks, coverage on TV is pretty sparse. So it was great to find yesterday CSPAN.ORG which carries video of all sorts of political and other events in the US. For example, just this afternoon I watched yesterday's Dean Campaign Rally in Iowa in which his wife made her first appearance there. It was all incredibly interesting... it had the feeling of a rock concert somehow. Lots of energy. (Some of the other rallies too!).

Anyway, Very cool resource.

feedexplorer

Over the weekend I released clevercactus feedexplorer, a simple free app to browse the data from the Share Your OPML commons (thanks Dave for making this resource available!) and choose feeds that you find interesting, then allowing you to save them into OPML files that can be imported into a news aggregator. It runs on Windows, Mac OS X, and other OSes.

Here is the page with installation instructions and a short user guide.

If you can, take a moment to read the user guide as it explains how to change the sorting, perform searches, etc (Btw, I think the UI is pretty self-explanatory, but reading the doc should leave little doubt as to how to do something :)).

Note: if you have any problems with the installation, please take a moment to read the installation page, as it answers common questions and problems.

Another Note: the first time the app loads it will obtain the data from the site, but afterwards it only downloads the changes (through combined use of Etags and Last-Modified HTTP headers with the data in the main feed, which also includes change dates). Additionally, transfers use GZIP compression to minimize both server load and download times.

Yet Another Note: I find the incremental search function to be strangely mesmerizing. :)

Screenshots
Here are a couple of screenshots (click on the images to see a larger version).
feedexplorer running on WinXP:

And under Mac OS X (thanks to Erik for the image):

A bit of background
As I noted a few days ago, Dave had released Share Your OPML. After that he released an SDK to allow others to tap into the data and provide new applications. I had an idea last week for an app that would use the data, but was too busy to do it. Finally, on Saturday morning I decided to let off some steam by coding something else, and this application seemed like a good idea: it had a simple goal, and I could do it quickly. In the end it took me about three hours to write the app, and a couple of hours more to finish the docs and the install pages. :)

So what's the idea?
The idea is that you can peruse subscription lists in two ways, one by looking at them starting from the feed and being able to see who subscribes to it, and two by looking at people that have shared their sub lists and see what they subscribe to individually. As you look through the list you can choose feeds you find interesting and add them to your own subscription list, which you can then save into an OPML file in your local hard drive to import into your news aggregator.

Now, there are other ways of getting subscription lists, but what I found interesting about this dataset is that it tells you who's reading what, which maybe leads you to find feeds (that you'd otherwise not look at) simply because someone you know is reading them--sort of an implicit recommendation system. If your own feed is listed, you can find out some of the people who are subscribing to you.

And what's feedexplorer have to do with clevercactus pro?
Well, eventually functionality like this would be added to cc pro. I think that it would make it easier for people to subscribe to feeds within the app, find out what's going on in the blogsphere, etc. feedexplorer, however, stands on its own as a simple, free utility that is generic and not tied to any particular product.

So that's it! And, as usual, comments welcome (if I have to close the comment section of this entry due to spam, you can always send me an email).

digital law conference @ Yale

The other day I got an invitation to the CyberCrime and Digital Law Enforcement conference to be held at Yale Law School, March 26-28. I won't be able to attend, but it looks interesting, with some good sessions and speakers. Mr. Lessig would seem to be missing from the roster of speakers, but Jennifer Granick (who is also at Stanford Law School) will be there.

movies, movies, movies

Speaking of movies. The worst two movies of 2003 were, by far, SWAT and Tears of the Sun. Both were an incredible disappointment. SWAT is supposed to be a thriller I guess. Tears of the Sun, some kind of adult drama. Both made me laugh more than most comedies (now that I think of it, I'd rate them as best comedies of the year). They are so terrible in so many senses that it's hard to know where to start describing them. At the center of the problem in both cases was the story, then the script, and then it radiated from there. Anyway, avoid them if you can. SWAT might make interesting viewing for late-night TV. Tears of the Sun, with all of its pretentiousness, doesn't even get to that level (which is a major disappointment, since the director was Antoine Fuqua, whose previous movie was the incredibly good Training Day). The Hulk was also a big disappointment. The editing is fantastic though, recreating comic-book feel on the screen--very well done but since the story is pretty bad in the end it just looks like an empty gimmick.

Among movies that left something (or a lot) to be desired (aka the "watchable" category, aka the "meh" category) that I saw were:

• The Italian Job. A strangely happy, well-dressed group of thieves that made me feel as if I was watching an episode of Friends rather than a Heist movie.
• Matrix Revolutions. No comment.
• Terminator 3. I guess that watching a movie thinking it's going to be crap improves its chances of rating it as "Meh" later...
• The Recruit. Too predictable, much ado about nothing. Pacino is good as usual, Farrell as well, but the material is not that great.
• Identity. My expectations were too high for this one--Cusack is excellent in it though.
• The life of David Gale. Not bad, but again pretty predictable, which takes the fun out of it, especially when you figure out the plot in the first five minutes.
• Confidence. Also predictable.
• Daredevil. Funny Colin Farrell, nice CGI, not much else.
• Anger Management. Not very funny as far as I was concerned--the final scene in the Yankee Stadium was memorable though.
• 28 days later. An excellent movie until they blew it by losing their nerve at the end. It should have had an open ending.
• The league of extraordinary gentlemen. Having read the graphic novel (which predates the movie), the film version feels like sanitized garbage. On its own, it rates a "meh" :).

Now for good movies I saw last year, more or less in the order I remember them, which is probably a relatively good measure of how much I liked them :) -- many of these can't be compared to each other though:

• The 25th Hour. Some notes on it here.
• Frida, which is the best performance by Salma Hayek I've seen. Great display of how editing & post-production can be great if used properly--such as in transitions between Kahlo's paintings and live-action sequences to show influences, how she saw things, etc.
• The Hours. With this movie everyone was talking about Nicole Kidman and her famous nosejob, it truly had everything: great acting (Kidman, Julianne Moore, Meryl Streep and Ed Harris all turning in great performances), great script, great editing, great directing. Can you tell I liked this movie?
• Gangs of New York. Daniel Day-Lewis, directed by Martin Scorcese, with music by U2--what more can we ask for? :)
• The Dancer upstairs (directed by John Malkovich). Simply fantastic.
• Kill Bill Vol. 1. Tarantino is back. Need I say more?
• Chicago. And I don't like musicals that much!
• Hollywood Ending. Classic Woody. Woody aficionados probably all really liked it, everyone else probably thought "Meh".
• Phone Booth, in which Colin Farrell shows that his acting in the excellent Tigerland was no fluke.
• Avalon. An unknown movie that should rightfully be a cult-classic (and maybe it is :)). (review).
• 8 mile. Eminem didn't have to act much for this, if we are to believe the press releases, but it was good nonetheless.
• Pirates of the Caribbean, in which Johnny Depp shows that he can create magic with a bit of eye liner. Seriously, without Depp this movie would have been crap (I could see him thinking "Take that Errol Flynn!" Heh). A movie of Grade A entertainment with no brains, and comfortable about it.
• X-Men 2. Well done, understated, and an improvement over X-Men, which had already been good. Hopefully the final installment of the trilogy will maintain this tradition. I really want to see The Phoenix rise.
• Matrix Reloaded. Regardless of my disappointment with Revolutions, this movie opened up so many possibilities that I think it was pretty good.
• Adaptation was good as well, if a bit too self-conscious for me. There comes a point when I've had enough of the in-your-face self-referencing.
• Seabiscuit. A strange movie in that it starts slow and picks up the pace a lot. I think they did it on purpose, to mirror the qualities of a horse race (particularly as the horse in the movie runs them). Regardless of whether I'm right about my interpretation or not, really enjoyable once you get past the first half-hour.
• Equilibrium. This is Fahrenheit 451 meets The Matrix (I couldn't resist, I find these one-line analogies funny for some reason). Entertaining and well done.

I'm probably forgetting a few, but that's ok. Btw, many of these movies were actually released in 2002, but they were only released here in Ireland at the beginning of 2003 (e.g., Gangs of New York).

still some server problems

I woke up this morning to find the site down (again), this time because the server rebooted overnight (again) and there is some problem with the init scripts (again). Same thing that happened yesterday, and a few days ago. Damn. I have no idea what might be happening, I wonder if it's a hardware problem, or a software problem, or a reaction to some kind of attack, or just the load on the server that is straining things.

At least now I've got the routine down cold: "start this, start that". I need to sit down for a bit and fix it. Hopefully this will stop soon... or at least the server will decide to reboot when I'm awake (this shouldn't be hard, since I don't sleep much).

Apart from that, I was planning to go see LoTR RoK last week but didn't. I was planning to see The Last Samurai but didn't. I wanted to go see Paycheck, but didn't. I was planning to work a lot, and that went fine :).

What else. Yesterday morning I took a break from what I was doing and wrote another application (yes, the whole thing), just for fun. It might sound strange, but to me it is actually relaxing to spend a few hours coding something completely different, especially if it's simple and well-defined. In the process, I managed to fix several problems that I had with the other app, mainly framework and UI things. Funny how changing the context sometimes makes difficult problems seem easy. Archimedes probably knew that, didn't he, when he went to take that famous bath.

I also took yesterday as an "offline" day. No IM. No IRC. No weblog. It's not some kind of master plan, but it just happened. Another way of taking a break for me.

lower DSL prices in Ireland?

Karlin reports that:

Eircom said today that it has applied to ComReg to further lower the price of its main broadband DSL product, from Euro 54.45 to Euro 39.99 monthly (both incl VAT). Good news; I can't imagine ComReg will refuse.

(That's from USD 70 to USD 50 at current exchange rates, for those on the other side of the Atlantic!). Good news indeed. Fingers crossed...

half-oops

I don't know what happened but the server rebooted overnight, and some services didn't come back up, in particular naming, so some machines might have failed to resolve this domain. All seems to be well now. It seems this has been one of those weeks...

file sharing = piracy? Not really.

An interesting Salon article: Is the war on file sharing over?:

If one is willing to believe the happy talk from music business executives, the tide has finally turned against file sharing, thanks to the get-tough tactics employed by the Recording Industry Association of America.

Last fall, the RIAA began filing lawsuits against individual users of peer-to-peer trading sites, and the strategy, the RIAA says now, has paid off. The group is careful not to declare a final victory over file trading, but things are finally beginning to look up for a business long in decline, say industry representatives. After years of scoffing at copyright laws, Americans are finally beginning to understand the gravity of file trading's offense against copyright.

The article is interesting. But what I find most interesting is this automatic alignment that is made in the media discourse between file sharing and piracy. There are many, many uses other than those the RIAA defines as illegitimate for file sharing (note, I am not saying anonymous file sharing, although there worthy uses for that too). Sure, the media loves a good fight and that's why the focus on this comparison. But the uses of sharing should, can, and will move beyond those in dispute. And not just for files, either.

Why am I saying this? Well, can't you guess?

Stay tuned. :-)

fly me to the moon...

Finally the long-rumored announcement from the Bush administration happened yesterday, and the New York Times has both an article and analysis (more coverage from CNN, the Washington Post 1, 2, 3, and space.com). At first I was excited, since as I've expressed before I wholeheartedly support spaceflight. True spacefaring abilities is be among the short list of things mankind should strive to achieve in this century. (Along with tending to some...err... tiny problems we still seem to have when taking care of our home planet).

The plan is (apparently) to phase out what's left of the Shuttle fleet (STS, or Space Transportation System). There are three Shuttles left: Discovery, Atlantis, and Endeavour. (an early model of the orbiter, the Enterprise, only performed tests flights). Additionally, NASA space science programs will be downsized, including cancellation of further servicing to the Hubble Space Telescope. The STS phase-out would be complete by 2010 (which would also be the "date of completion" of the International Space Station), and the new transportation vehicle would be ready by 2014.

And herein lies the first problem with this plan. Are we seriously saying that the US will stay out of space for four years? I find this very hard to believe, considering that the Chinese are certain to have made some progress by then on their own goal of landing on the moon. (And let's not forget Russia...).

After the new launch, a lunar base would be established, "at most" by 2020, and subsequently used as additional research, development and launch platform for launching a manned Mission to Mars.

This "schedule" seems to me slow, and with many of its targets are so far off that (as the NYTimes analysis makes clear), easy to derail. Not to mention that the announcement provided basically no new funding for the program ($1 billion, plus the money that would come from phasing out the STS fleet).

A big factor in this seems to be "safety". For example, the NY Times analysis mentions that the shuttles have been "prone to catastrophic failure". This statement appears to imply that other space vehicles have not been prone to catastrophic failure. Mmm. Let me see. The Shuttle has flown over a hundred missions (STS-107 was the last flight of the ill-fated Columbia) with exactly two catastrophic failures. In contrast, the Apollo program flew less than 15 manned missions (with a few more unmanned) and it had two massive failures, the first in Apollo 1 (which killed the crew during a test) and the second with Apollo 13, which barely made it back to earth. The number of Soviet failures at the same time is difficult to know with a high degree of confidence, but no one thinks that it was a walk in the park. The Soviet Union, after all, never managed to put a man on the moon, and Soviet technology, though constantly a bit behind the times, was never that bad.

This reminds me of one of Steve Buscemi's lines in Armageddon: "You know, Harry, we're sitting on 4 million pounds of fuel, one nuclear weapon, and a thing that has 270,000 moving parts built by the lowest bidder. Makes you feel good, doesn't it?"

Setting aside the nuclear weapon for the moment (Flying to Mars and beyond may well involve some sort of nuclear- or even antimatter-powered spacecraft), this is one of those "funny 'cause it's true" jokes.

What I'm saying is: I don't get it. Can't they get astronauts to fly? What's the problem? If they can't find anyone, sign me up! But of course, they can get astronauts to fly. They would, under whatever circumstances and whatever risks. But of course this whole obsession with safety is something that has been growing and growing in the Western world, with the US "leading the way" but with Europe particularly in the same boat. Apparently, people are just not supposed to die anymore.

And what about the technology? Does it really take more than 10 years to create a new moon crew transport vehicle? Of course not. Our science and technology has advanced by leaps and bounds since the 70s, particularly computer technology which is crucial to this whole endeavor. As the Washington Post notes:

Bush has outlined a tortoise-like pace, dictated by severe budget constraints, that allows a full decade just to develop a vehicle that would, once again, deliver people to the moon -- something Apollo engineers accomplished, starting from scratch, in about eight years.

The problem is not technology, it's political will, and funding. In fact, this new project is a mirror of something that was proposed ten years ago, which went nowhere, as one of the articles from the NYT describes:

In 1989, in a speech honoring the 20th anniversary of the initial lunar landing, the first President Bush proposed that the nation establish a base on the Moon and send an expedition to Mars to begin "the permanent settlement of space." He set the Mars goal for 2019 but the effort soon fizzled when the cost estimates hit $400 billion.

In today's western culture (but it's really happening all over the world) with our instant-satisfaction, one-click-shopping, celebrity-obsessed and 24-hour-of-irrelevant-news media, it's hard to think that popular support will keep steady over the course of the 15-25 years required for this project.

I must say, though, without cynicism, that I hope I'm wrong. I really, really hope that the US can stick with it. It's the one country that has the knowhow and the resources (and, at times, the spirit) necessary to pull it off. And for all the criticisms, it has maintained a continuing space program, to its credit. Does anyone think that the International Space Station would be anything but a blueprint by now if it wasn't for the time, money, and energy (however misdirected) that the US has spent on it?

And, by the way, why does the US have to do this by itself? The Chinese are moving forward, but if they keep at it there will be questions as to how much international aid they need, as this article from the economist notes. And, where's Japan, where's Russia? More importantly, where's the EU? There's been lots of talk about the potential world power the EU can become. But instead of talking about worthy goals, like using the European Space Agency for a daring multinational space exploration program, we keep discussing agricultural subsidies and whether one country has more votes than the other. It's not of course that those are not important issues, but there is zero attention, money, or "political capital" put forward for anything other than those things. I mean, Germany, France, the UK, and all the other great countries. Come on! Europe has to stop running scared from its past of internicine warfare and truly look forward to the future. The US can't be left alone holding the bag with this.

I suddenly think of part of a Sagan quote I posted sometime ago:

Spaceflight, therefore, is subversive. If they are fortunate enough to find themselves in Earth orbit, most people, after a little meditation, have similar thoughts. The nations that had insituted spaceflight had done so largely for nationalistic reasons; it was a small irony that almost everyone who entered space received a starting glimpse of a transnational perspective, of the Earth as one world.

We are not that far away. We can only hope that we, as a society, can for once look just a little beyond our noses and truly make it happen.

the story of an outage

a tale of mistakes, backups, recovery (by a hair), and why permalinks are not so permanent after all

out·age (ou?tij) noun

1. A quantity or portion of something lacking after delivery or storage.
2. A temporary suspension of operation, especially of electric power.

   When I woke up yesterday after a brief sleep I started to log back in to different services and as I'm seeing something's funny with my server, Jim over at #mobitopia asks "is your site down?".

   Damn.

   As I checked what was happening, I could see that all sorts of things were not working on the server. I was starting to fear the worst ("the worst" in abstract, nothing specific) when I remembered that I had seen similar symptoms a couple of months ago, and back then it had been a disk space problem. I run "df" and sure enough, the mountpoint where a bunch of data related to the services (including logs) is stored was full (since November the number of pageviews a month has increased to over 200,000, which creates pretty big logfiles). As the last time, the logs were the culprits. Still half-asleep, I start to compress, move things around and delete files, when suddenly after a delete I stop cold: "No such file or directory".

   What? But I had just seen that file...

   I look up the console history and four rm commands had failed similarly.

   Uh-oh.

   I run "pwd". Look at the result. "That's not right...". I was not where I thought I was.

   At that point, I woke up completely. Nothing like adrenaline for shaking off sleepiness.

   I look through the command history. At some point in my switching back and forth from one directory to another, I mistyped a "cd -" command and it all went downhill from there. Adding to the confusion was the fact that I used keep parallel structures of the same data on different partitions, "just in case". I stopped doing that once I got DSL back in May last year, opting instead to download stuff to my home machine, but the old structure, with old data, remained. And, even more, my bash configuration for root doesn't display the current directory (the first thing I did after I realized that was add $PWD to the prompt, but of course by then it was too late).

   I had just wiped out the movable type DB, the MT binaries (actually, all the CGI scripts), the archives, and a bunch of other stuff in my home directory.

   I took a deep breath and finished creating space, and moved on.

   First thing I did was restart the services, now that disk space wasn't longer an issue. Then I reinstalled the binaries that I had just wiped out, which I always keep in a separate directory with some quick instructions on how to install them. That turned out to be a lifesaver, one of the many in this little story.

   After that I put up a simple page that explaining the situation (here's a copy for... err... "historical reference"), plus a hand-written feed and worked on the problem in breaks between work.

   Then I realized that all the links that were coming in from the outside (through other weblogs, google, etc) were getting a 404. So as a temporary measure I redirected the archive traffic to the main page through a mod_rewrite clause:

   RewriteRule /d2r/archives/(.*) /d2r/ [R=307]

   That would return a temporary redirect (code 307) while I got things fixed (one fire out! 10 to go).

   So what next? The data of course. When I came back to Ireland at the beginning of January I started doing backups of different things (a "new year, new backups" sort of thing), and I backed up all the server data directories on Thursday, and then on Saturday I did what I thought was a backup of my weblog data, through MovableType's "Export" feature. As things turned out, the latter proved useless, and it was the "binary" backup that saved the day.

   Why? Well, as I started looking at things, I went to MT's "import" command in cavalier fashion and was about to start when the word "permalink" popped up in my head. Then it grew to a question: "What about the permalinks?".

   The question was valid because my permalinks are directly based on the MT entry ids. Therefore, if an import changed the entry IDs, it would also break all the permalinks. I started cursing for not switching over to using entry-based strings for permalinks, but that didn't help. So I did a little digging and I realized that I was right. MT assigns entry IDs on a system-wide basis. So if you have multiple weblogs on the same DB (which I have, some of them private, some for testing, etc) OR if you have to recover the data from an export (which I had to do) you're out of luck. More likely than not, the permalinks will not work anymore. The exported file did not include IDs. Re-importing would generate the IDs again. Different IDs. Different links. Result: broken links all over the place, both within the weblog and from external sources.

   This is clearly an issue with the MT database design, which doesn't seem too well adapted to the idea of recovery. To be fair, however, I am not sure how other blogging software deals with this problem, if at all. I think this is one big hole in the weblog infrastructure that we haven't yet completely figured out, both for recovery and for transitions between blog software (As Don noted recently).

   This is when I started thinking that things would have been much easier if I had written my own weblog software. :) That thought would return a few times over the next 24 hours, but luckily I was busy enough with other things not to indulge in it too much.

   After looking online and finding nothing on the topic, I came to the conclusion that my only chance was to do a direct restore of the "binary" copy (that is, replacing the clean database with the backup directly) I had from last Thursday. I did the upload, put everything in place, and things seemed to go well, I could log in to MT and the entries up to that point where right where they had to be. So far so good. I was going to do a rebuild and I thought that maybe now was a good time to close off all comment threads in all entries (to avoid ever-increasing comment spam) and I spent some time trying to figure out how to use the various MT tools to close comments on old entries. However, they all seem to be ready for MySQL rather than BerkeleyDB. It wasn't a hard decision to set it aside and move on.

   So I started a full rebuild. The first 40 entries went along fine, albeit slowly. Then nothing happened. Then, failure. I thought for a moment that, for some strange reason, the redirect I had set up yesterday was causing the problem, so I removed it, restarted the server, and Tried again. Failed again. No apparent reason.

   I got angry for a second but then I remembered that the "binary" backup was of everything, including the published HTML files. Aha! I uploaded those,crossed my fingers, and did a rebuild only of the index files, and everything was up again. Actually, this was important for another reason, since the uploaded images that are linked from the entries end up by default in the archives directory, you need a backup of that or the images (and whatever else you upload into MT) will be gone if you lose the site.

   So the solution up until this point had been a lot simpler than I thought at the beginning.

   But wait! All the entries after last Thursday were missing, and I didn't have a backup for those. That was when RSS came to the rescue in three different forms: 1) I download my own feeds into my aggregator, so there I had a copy up to a point. 2) Some kind souls, along with their condolences for the problem, sent along their own copy of the latest entries (Thanks!!--and Thanks to those who sent good wishes as well). 3) Search engines, (Feedster was the most up to date--btw, it was Matt that suggested yesterday, also on #mobitopia, that I check out Feedster as a source of information, a great idea that really applies to many search engines if their database is properly updated), had cached copies that I could use to check dates and content. So armed with all that information I set out to recreate the missing entries.

   Here the problem of the permalinks surfaced again. I had to be careful on the sequencing, or the IDs wouldn't match. So I re-created empty entries, one-by-one, to maintain the sequencing (leaving them unpublished), actually posted a couple of updates of what was going on, and then I published the recovered entries as I entered the content and set the right dates.

   So. All things are restored now (except for the comments from the last week, which are truly lost--this makes me think that setting up comment feeds would be a good idea. However, that doesn't address how would I recreate the comments given what happened. Would I post them myself under the submitter's name? That doesn't seem right at all. Another problem with no obvious solution given the combination of export/ID issues with MT).

   What's strange is that there's been slight a breakdown in continuity now, because I did "post" some updates to that temporary index file, but it couldn't be part of the regular blogflow. Hopefully this entry fixes that to the extent possible.

   Okay, lessons learned?

   1. Backups do work. :) I am going to do another full backup today, and I'll try to set up something automated to that effect. (Yes, I know I should have done it before, but as usual there are no simple solutions, and then you leave it for the next day... and the next...). Plus, backups for MT installations, should always be both of the DB and the published data, to make recovery quick. (I have about 1500 entries, which amount to something like 20MB of generated HTML--additionally, the images are posted directly on the archives directory, so if you're not backing that up, you've lost them).
      
   2. For MovableType, the export feature is not so great as far as backups are concerned. The single-ID-per-database problem is a big one IMO, and I don't think MT is alone in this. We need to start looking at recovery and transition in a big way if weblogs are going to hit the mainstream (and we want permalinks to be really permanent)
      
   3. Solutions are often simpler than you think, if you have the right data. Having a full backup makes recovery in this case easy and fast.
      
   4. This stuff is still too hard. What would a less technically-oriented user do in this situation? Granted, it was my knowledge (since I was fixing stuff directly on the server) that actually created the problem in the first place, but there are lots of ways in which the same result could have been "achieved", starting from simple admin screwups, hardware failures, etc.
   Overall, this has been a wake-up call in more than one sense, and it has set off a number of ideas and questions in my head. How to solve these problems? I'll have to think about it more.

   Anyway. Back to work now, one less thing on my mind.

   Where was I?

   Categories: technology
   Posted by diego on January 14, 2004 at 5:45 PM

all entries restored

Okay, all entries since last thursday have been restored. Coming up: the story of an outage.

back up--partially

Okay. Getting back to normal now. Everything seems to be back up until last thursday. Now I'm recovering the entries I had posted since then. Entries will start showing up as I update them. And I say "update them" because I had to create the entries first as empty drafts to maintain the ID sequencing. Many lessons learned from this whole thing. More later.

on OPML

On my previous entry about Postel's Law, Danny Ayers made a comment, and to a part of it I said I'd reply in a separate entry. To make the question clear, I'll restate it here in a different way to involve the technology only.

Essentially, Danny was asking "If you have used OPML, would you agree that OPML does not follow this route you are advocating of adding as many constraints as possible to a spec, to make interoperability easier?" (Danny, if I misunderstood the question please let me know, but I'm pretty sure that was the essence of your comment, personal matters aside).

My answer to that question would have to be no, I do not agree. Let me explain.

I have implemented both readers and writers of OPML when used for RSS subscription lists for an end-user product (ie, clevercactus). And there is one main point that I've found frustrating, namely that the attributes used on the "outline" element vary between tools. I have previously noted, in another context the elements that would "complete the spec" by properly specifying these attributes.

However, I've come to the conclusion that this is not a problem with the spec itself, but rather a problem of what are we using it for. As far as I can read in the spec, it was designed to be a very simple and flexible storage mechanism. The first sentence in the spec says "This document describes a format for storing outlines in XML 1.0" (my emphasis). It doesn't say "This is a format for interchange of outlines" or anything like that.

That is, creating an interoperable format for RSS subscription lists was not part of the original "charter" of OPML.

Which is why I can't agree with Danny's statement, because the interoperability problems we all know about pop up when using OPML outside of its original intended domain.

As such, that is, as a format for local storage of outlines, the OPML spec might have done a good thing by keeping things very open. Note that the spec explicitly says, in its goals: "Outlines can be used for specifications, legal briefs, product plans, presentations, screenplays, directories, diaries, discussion groups, chat systems and stories." -- that's a big set of apps, and I'd be hard pressed to define a consistent set of common attributes for all of them. To be honest, if it was me designing it maybe I would have chosen a different path (like for example target less applications), but that's not really the point. Design is at its core subjective.

So. Given that OPML was not originally designed as an interoperable way to store feed subscription lists, the current situation is logical, almost predictable. It seems to me (given what I've seen--I might be wrong of course) that this is a use of OPML that grew in ad-hoc fashion and as such created some incompatibility problems. But is this a problem with OPML itself? I don't think so. Usage grew beyond its original intended target, and things got a bit messy.

Okay, that's my answer to Danny's question, but I just want to be clear on what I think about OPML given the current situation, as what I said above might seem a bit too ... err... "theoretical".

That is, we still have the interoperability problems for feed subscription lists.

However, now that it's clear that it has become accepted for that use, I noticed that Dave recently put up a short RFC that clearly states "Using OPML to exchange subscription lists". My comments from October last year would, then, apply in this new context, and the new RFC already covers part of them (the most important in my mind, which is the issue of standard attributes).

This new spec of "Interchangeable OPML Subscription Lists" plus the OPML spec itself (which doesn't necessarily need to change, since it is still relevant in its original intended domain) make a simple combined spec that is useful and already deployed (granted, some aggregators might be generating different attribute names that those on the RFC, but that's a tiny change, and none of the other items under discussion that I'm aware of are in any way "deal-breakers").

Hence, OPML applied to the domain of feed subscription lists in particular is a good solution, simple and to the point. And to me that's what matters: if something does what I need, it's simple, and it works, I'm all for it.

alarm! alarm!

I was woken up this morning (before 7 am) to the horrible screeching of the building's fire alarm (yes, the building's not my apartment's). It's a pretty bad way to wake up. I was about to go down and see what was happening when it stopped. This has been a problem once before in the past. Needless to say, I couldn't go back to sleep, and it took me a while to get the sound out of my head and to quell a rising headache. Then I got to work, write a bit, etc.

A few minutes back I was right "in the zone" and the alarm went off again. Damn. I went down. "They are fixing it". Right. Then it stopped.

About 10 minutes ago it started up again and they are stopping it and starting it intermittently as they try to fix it. It's impossible to concentrate. If this keeps up, I'll go for a walk. And hopefully they'll get it fixed for the rest of the day at least, so I can get some work done!

Ah, the wonders of technology.

Later: The Alarm stopped. I went out for a walk anyway. Brrr it's cold. Weather.co.uk says "5 C, feels like -1 C". I believe it. Plus, the weather's crazy, there's very strong winds so whole stormfronts move in and out of sight within minutes (I'm not exaggerating). When I got back the alarm started up again. Still fixing it, it seems, and we building-dwellers can do nothing but wait (and complain :)).

postel's law is for implementors, not designers

Another discussion that recently flared up (again) is regarding the applicability of constraints within specifications, more specifically (heh) of constraints that should or should not be placed in the Atom API. The first I heard about this was through this post on Mark's weblog, where among other things he says:

Another entire class of unhelpful suggestions that seems to pop up on a regular basis is unproductive mandates about how producers can produce Atom feeds, or how clients can consume them. Things like “let’s mandate that feeds can’t use CDATA blocks” (runs contrary to the XML specification), or “let’s mandate that feeds can’t contain processing instructions” (technically possible, but to what purpose?), or “let’s mandate that clients can only consume feeds with conforming XML parsers”.

This last one is interesting, in that it tries to wish away Postel’s Law (originally stated in RFC 793 as “be conservative in what you do, be liberal in what you accept from others”). Various people have tried to mandate this principle out of existence, some going so far as to claim that Postel’s Law should not apply to XML, because (apparently) the three letters “X”, “M”, and “L” are a magical combination that signal a glorious revolution that somehow overturns the fundamental principles of interoperability.

There are no exceptions to Postel’s Law. Anyone who tries to tell you differently is probably a client-side developer who wants the entire world to change so that their life might be 0.00001% easier. The world doesn’t work that way.

Mark then goes on to describe the ability of his ultra-liberal feed parser to handle different types of RSS, RDF and Atom. (Note: I do agree with Mark that CDATA statements should be permitted, as per the XML spec). In fact I do agree with Mark's statement, but I don't agree with the context in which he applies it.

Today, Dave points to a message on the Atom-syntax mailing list where Bob Wyman gives his view on the barriers created by the "ultra-liberal" approach to specifications, using HTML as an example.

I italicized the word "specifications" because I think there's a disconnect in the discussion here, and the context in which Postel's Law is being applied is at the center of it.

As I understand it, Mark is saying that writing down constraints in the Atom spec (or any other for that matter) is something to be avoided when possible, because people will do whatever they want anyway, and it's not a big deal (and he gives his parser as an example). But whether his parser or any other can deal with anything you throw at it is beside the point I think, or rather it proves that Postel's law is properly applied to implementation, but it doesn't prove that it applies to design.

Mark quotes the original expression of Postel's Law in RFC 793, but his quote is incomplete. Here is the full quote:

2.10. Robustness Principle

TCP implementations will follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others.

(my emphasis). The comment in the RFC clearly states that implementations will be flexible, not the spec itself. I agree with Mark's statement: there are no exceptions to Postel's law. But I disagree in how he applies it, because it doesn't affect design, but rather implementation.

Getting a little bit into the semantic of things, I think it's interesting to note that placing a comment like that on the RFC is actually defining accepted practice (dealing with reality rather than the abstractions of the spec) and so it is a constraint (a constraint that requests you accept anything, rather than reject it, is nevertheless a constraint). So the fact that this "Robustness principle" is within that particular RFC as an example shows that placing constraints is a good idea.

Implementations can and often do differ from specs, unintentionally (ie., because of a bug) or otherwise. But the less constraints there are in a spec, the easier it is to get away with extensions that kill interoperability. So I don't think it's bad to say what's "within spec" and what is not within spec. Saying flat-out that "constraints are bad" is not a good idea IMO.

One example of a reasonable constraint that I think would be useful for Atom would be to say that if an entry's content is not text or HTML/XHTML (e.g., it's a Word document, something that as far as I can see could be done on an Atom feed according to the current spec) then the feed must provide the equivalent text in plain text or HTML. Sure, it might happen that someone starts serving word documents, but they'd be clearly disregaring the spec, and so taking a big chance. Maybe they can pull it off. Just as Netscape introduced new tags that they liked when they had 80 or 90% market share. But when that happened, no one had any doubts that using that tag was "non-standard". And that's a plus I think.

So, my opinion in a nutshell: constraints are good. The more things can be defined with the agreement of those involved, the better, since once something is "out in the wild" accepted practices emerge and the ability to place new constraints (e.g., to fix problems) becomes more limited, as we all know.

What I would say, then, is: Postel's law has no exceptions, but it applies to implementation, not design.

syndication-land happenings

A week has gone by since I came back and I'm still catching up with some of the things that have happened in syndication-land. For example, recently, Dave released a new "subscription aggregator" (meta-aggregator?) that allows members to see who's subscribing to who and do other interesting things. I haven't had time to digest all of its implications so I'll leave the comments for later, but from what I understand (and I haven't registered yet) it looks very cool.

Another example: Feedster has released an RSS feed for their "Feed of the day" feature, and they are doing a number of other interesting things that I haven't yet had time to fully explore. And, btw, my weblog was one of the first "feeds of the day" that was chosen (look in this page near the bottom). This happened months ago! Totally missed it. Thanks!

More later as my brain processes it. :)

from components to modules

Right now I'm refactoring/rebuilding the user interface of a new release coming out soon (oh right... Note to self: talk about that) and I'm facing the fight against "sticky" APIs. Or, in more technical terms, their coupling.

Ideally, a certain component set that is self-contained (say, and HTML component) will be isolated from other components at the same level. This makes it both simpler, easier to maintain and, contrary to what one might think, often faster. While I was at Drexel, at the Software Engineering Research Group, I did work on source code analysis, studying things like automatic clustering (paper) of software systems, that is, creating software that was able to infer the modules present on a source code base using API cross-references as a basis. Since then I've always been aware (more than I was before that, that is) of the subtle pull created by API references.

The holy grail in this sense is, for me, to create applications that are built of fully interchangeable pieces, that connect dynamically at runtime, thus avoiding compile-time dependencies. In theory, we have many ways of achieving this decoupling between components or component sets; in practice there are some barriers that make it hard to get it right the first time. Or the second. Or...

First, the most common ways of achieving component decoupling are:

1. Through data: usually this means a configuration file, but it could be a database or whatever else is editable post-compilation. This is one of the reasons why XML is so important, btw.
2. Through dynamic binding: that is, references "by name" of classes or methods. This is useful mostly with OO languages, as you'll generally end up dynamically allocating a superclass and then using an interface (or superclass) to access the underlying object without losing generality (and thus without increasing coupling).

Achieving decoupling in non-UI components is not too difficult (the data model has to flexible enough though, see below). But UIs are almost by definition something that pulls together all the components of a program so they can be used or managed. The UI references (almost) everything else by necessity, directly or indirectly, and visual components affect each other (say, a list on the left that changes what you see on the right).

In my experience, MVC is an absolute necessity to achieve at least a minimal level of decoupling. Going further is possible by using a combination of data (ie., config files) to connect dynamically loaded visual components removes the coupling created at the UI level, but that is difficult to achieve, because it complicates the initial development process (with dynamically loaded components bugs become more difficult to track, the build process is more complex, etc.) and development tools in general deal with code-units (e.g., classes, or source files) rather than with modules. They go from fine-grained view of a system (say, a class or even a method) to a project, with little in between. We are left with separating files in directories to make a project manageable, which is kind of crazy when you think how far we've come in other areas, particularly in recent years.

The process then becomes iterative, one of achieving higher degrees of decoupling on each release. One thing I've found: that the underlying data model of the application has to be flexible enough, be completely isolated (as a module) and relatively abstract, not just to evolve itself but also to allow the developer to change everything that's "on top" of it and improve the structure of the application without affecting users, etc.

Yes, this is relatively "common knowledge", but I'm a bit frustrated at the moment because I know how things "should be" structured in the code I'm working on but I also know that time is limited, so I make some improvements and move on, leaving the rest for the next release.

Final thought: Until major development tools fully incorporate the concept of modules into their operation (and I mean going beyond the lame use of, for example, things like Java packages in today's Java tools), until they treat a piece of user interface as more than a source file (so far, all of the UI designers I've seen maintain a pretty strict correspondence between a UI design "form" and a single file/class/whatever that references everything else), it will be difficult to get things right on the first try.

the digital life

Today's New York Times magazine has an article, My So-Called Blog on weblogs and the impact of our digital lives on the "real" world:

When M. gets home from school, he immediately logs on to his computer. Then he stays there, touching base with the people he has seen all day long, floating in a kind of multitasking heaven of communication. First, he clicks on his Web log, or blog -- an online diary he keeps on a Web site called LiveJournal -- and checks for responses from his readers. Next he reads his friends' journals, contributing his distinctive brand of wry, supportive commentary to their observations. Then he returns to his own journal to compose his entries: sometimes confessional, more often dry private jokes or koanlike observations on life.

Finally, he spends a long time -- sometimes hours -- exchanging instant messages, a form of communication far more common among teenagers than phone calls. In multiple dialogue boxes on his computer screen, he'll type real-time conversations with several friends at once; if he leaves the house to hang out in the real world, he'll come back and instant-message some more, and sometimes cut and paste transcripts of these conversations into his online journal. All this upkeep can get in the way of homework, he admitted. ''You keep telling yourself, 'Don't look, don't look!' And you keep on checking your e-mail.''

Well, we've all been there, haven't we? Okay, many of us have. Okay, would you believe me if I said I have? :-)

The article has a certain focus on "teenagers" or "young adults" for some reason. But that aside, it has some interesting comments and some good insights that apply to all groups I think. Everyone that is involved with new tools (using them... building them... whatever...) is trying to feel their way around.

And this is just text, and maybe pictures. A video here and there at most. And that creates a certain tension IMO, which won't really be gone until we can superimpose cyberspace with meatspace.

Right now if you want to "be online", mostly (and I emphasize mostly, as we all know that you could be IM'ing on your cellphone these days) you need to be sitting at a computer, and that means not being with others, or doing other things. The display, the keyboard, the whole UI experience pulls us in and demands a large part of our attention.

Result: a disconnect.

But, as I said, if the "real" (I keep putting real between quotes because I'm a subjectivist) and "virtual" worlds were superimposed things would be different. When that superimposition happens, there will be very little tension between interacting digitally and otherwise.

How do I mean? Science Fiction moment: You look at a restaurant and your glasses (or a retinal implant) superimpose a translucent image of its website. You get a person's business card and it contains a bluetooth chip that tells your PAN (Personal Area Network) about the person's email, etc, and their company webpage pulls up next to their smiling face and you see there that the product he's talking about hasn't been released yet. Or you have embedded a few key details into a wireless implant in your arm and everyone that sees you through the glasses (or the implant!) can see your weblog too, and see that you just posted a picture of them, taken with your cameraphone. Posting, browsing, and chatting, all from your local pub, pint in hand.

Okay, this is pretty lame as science fiction goes. I should brush up on my Snow Crash, Neuromancer, The Diamond Age, and all the rest...

Wait a minute. How did I get here from a New York Times article? Oh Well. :-)

quote of the day

The most exciting phrase to hear in science, the one that heralds new discoveries, is not "Eureka!" but "That's funny ..."

Isaac Asimov

reading “Voynichese”

Here's something weird and interesting from this week's Economist: an article on the Voynich manuscript. Quote:

THE Voynich manuscript, once owned by Emperor Rudolph II in 16th-century Bohemia, is filled with drawings of fantastic plants, zodiacal symbols and naked ladies. Far more intriguing than its illustrations, however, is the accompanying text: 234 pages of beautifully formed, yet completely unintelligible script.

Modern scholars have pored over the book since 1912, when Wilfrid Voynich, an American antiquarian, bought the manuscript and started circulating copies in the hope of having it translated. Some 90 years later, the book still defies deciphering. It now resides at Yale University.

The manuscript is written in “Voynichese”, which consists of strange characters, some of which look like normal Latin letters and Roman numerals. Some analysts have suggested that Voynichese is a modified form of Chinese. Others think it may be Ukrainian with the vowels taken out. But Voynichese words do not resemble those of any known language. Nor is the text a simple transliteration into fanciful symbols: the internal structure of Voynichese words, and how they fit together in sentences, is unlike patterns seen in other languages.

The other alternatives are, as the article notes, that the manuscript is either in code, or simply a hoax. Nevertheless, my geek-sense flares up when reading about something like this. Oh boy! An entire manuscript to decrypt, and a few centuries old to boot! Does that sound like fun or what?

bittorrent is nice, but...

...there's always a 'but' isn't there?

I had attempted to use BitTorrent a couple of times before, but never spent more than a few minutes with it, not enough to understand what was going on. Yesterday night though, I gave it a little more time and some tips from Russ and Matt I could get it going. I had to adjust some settings, such as the bandwidth allocated for uploads, which defaulted at 12 KB/sec and immediately started to suck up my entire upload capability (I set it at 7 KB/sec). I chose a couple of files (three actually) and let it download overnight. This morning, things were well on their way, two files done, the remaining one halfway through. But then it hit me: my transfers are limited!

I have a 4GB transfer limit (as it's common here in Ireland) on my DSL connection. So now I have downloaded, in one day, over 1.5 GB of data, and still have 1 GB to go. Then, there's the uploaded data, which also counts. EEk! By the time the second transfer is finished I will have spent over 75% of my monthly bandwidth allotment. With 60% of the month still to go!

Damn. I want to go back to my good old days of DSL in the Bay Area, where I had a symmetric 768 KB/sec DSL connection, with no transfer limits, at $40 a month. Okay, that's not realistic. :) But on the other hand, until transfer limits are removed (or at least raised) here, I won't be able to do much with BitTorrent. Too bad.

And, btw, this clearly has to have an impact on broadband usage. Forget about BitTorrent specifically, other types of media transfers are also quite heavy, and having that sword hanging over your neck (the sword being whatever they charge per megabyte after you cross the transfer limit) users will be more likely to treat broadband as a kind of always-on modem, rather than as true broadband. Ireland is great, for technology in particular, but it definitely needs some serious improvements to both infrastructure and access to that infrastructure (see my post on mobile handset costs yesterday) to be truly competitive. There's a qualitative jump (both on the supplier and the consumer side of a market) that happens when connectivity is pervasive, always-on, fast, and relatively inexpensive, and Ireland isn't there yet. Here's hoping we won't have to wait much longer.

handset prices in ireland

A few months back I commented on the user UNfriendly behavior of mobile operators in Ireland when it came to phone upgrades. I put an upgrade out of my mind for a while (since, quite simply, I couldn't afford it), but today as I passed by a carphonewarehouse store I saw that they had the SonyEricsson T610 at "only" Euro 139 with a contract with Vodafone. I thought "Hey, maybe prices have gone down for some mysterious reason" and I went into the store to check things out.

Long story short, I was wrong. Prices are still outrageous. Subsequent visits to O2 and Vodafone stores confirmed that this was indeed the case.

How outrageous? Consider, as just one example, the price for the Nokia 3650. Vodafone and O2 pricing is basically Euro 300 (= 380 USD, or 210 British Pounds). Both with contract, upgrade prices are exactly the same, although in some cases they might knock off Euro 10 of the price if a) you've been a customer for more than 2 years and b) you've spent more than Euro 1000 in calls during that period. As a comparison, the prices of carphonewarehouse UK for the Nokia 3660 starts at 80 GBP, that is, Euro 100. Pricing in the US is similar, as is in all other European countries that I could find. That is, price here is basically more than double (and depending on the contract, three times as much) as that of anywhere else. SIM-free phones are similarly more expensive than in other countries. The Nokia N-Gage (which is probably the most "consumerish" device you could find, a device that, given its target, cries out for a low price) is priced at Euro 300 here and found in other countries at half the price. Older handsets, like the Nokia 6310, still sell here at Euro 200 apiece with contract.

Even more, note that when I quoted the carphonewarehouse UK price it was for the Nokia 3660, not the 3650. Why? Because they don't even sell the 3650 anymore. So while the 3650 is already being phased out in some places, in Ireland they only started selling it less than two months ago.

The point of this rant: I wish that the much-vaunted European integration would take hold in the supposedly fluid and borderless market that is consumer electronics. Even if Ireland's market is too small to sustain low prices, Europe as a whole shouldn't be, and prices would be, if not the same, at least roughly equivalent across borders. Now, that's not much to ask for is it? :-)

autoclose comments on MT

I had mentioned the idea of using a script to close comments automatically in July last year and then thought about it when I started getting hit with comment spam later in October.

Now, a couple of weeks ago Jeremy posted the script he has been using for quite a while to do exactly that. Thanks! He uses mysql and I use Berkeley DB, so the script might need changing (honestly, I have no idea :)). Hopefully at some point within the next few days I'll have some time to look at it and start using it.

connection back up

This morning the DSL was still acting up (ie., not working, with traceroute dying at random points on different routes) on many websites worldwide, including Eircom's own DSL support page (dsl-support.eircom.net) which is hosted here in Ireland. So I called up technical support and immediately they told me that the problem was that I had an "invalid IP" assigned. Apparently they have a number of these invalid IPs in the pool (for the PPPoE connections) which they are "trying to remove" and if you get one of those assigned for your dynamic IP a number of sites on the Internet will not be reachable. The solution is to turn off the modem and wait for a while... and if that doesn't work (as in my case, since I did that in the morning) call up tech support and they will release the IP and make sure that the new IP assigned will not be one of the invalid ones. Sounds weird that an IP in the pool might work for some sites and not others. I wonder what kind of routing boxes they are using.

Now, if you ask me, it shouldn't be so hard to write a little program that would check each IP in turn and remove those that don't work no? Why wait until customers complain?

Anyway, if you're in Ireland, use Eircom DSL and you have problems reaching some websites, this might be the reason.

weird referer

Recently (but I just noticed it today) I started getting HTTP referers that are a variation of the following: "XXXX:+++++++++++++++++++++++" (the number of plus signs varies). A google search with appropriate terms quickly turned up discussions like this one that suggest that the referer is someone using an anonymizer or internet security product of some kind. Without that information it smells like an attempt at an exploit of some kind... but of what kind (and if so, I've never heard of it)?

Anyone knows about this? Has anyone else seen it? I'm curious. :)

the wal-mart way

An inside look from Fast Company at Wal Mart and how it achieves its low-low prices. Interesting, considering the implications it has for the US economy and (both consequently and directly) for the world economy as a whole.

watching jobs' macworld keynote

And on the 20th anniversary of the launch of the Mac no less...

First, It is amazing that an audience can clap, hoot and make lots of noise over someone saying "We've added a G5 to our XServe line" and things of that nature. It is kind of surreal.

Aside from XServe upgrades and such and the new versions of iLife apps (iTunes, iPhoto, iMovie, iDVD), Jobs announced GarageBand, software for making music, and what I can say is: WOW. Great synthethized instruments. Loops. Effects. Amps (!). Great Quality. Jobs' demo of this product was really, really impressive. It's not as if I can compare this to more professional tools that are out there, since I'm not aware of what the market is like (I just remember playing with CakeWalk on the PC a while ago, but it was crap compared to this), but since this is included in a $49 package, or free with new Macs, it seems to me that it's a big deal. Or at a minimum, another reason to lust after a Mac. :)

BTW, the "iLife ad" they showed after the demo was terrible, especially compared to the live demo, even more considering that many of the lines that the people in the video say had been used verbatim by Jobs before, which takes away a lot of the enjoyment of the keynote (What? You mean that the keynote was scripted?!? -- you're reminded of what you already knew but Jobs had made you forget). Ditch the pathetic ads apple, Jobs on a stage is enough. :)

Finally, the "iPod mini" (and it comes in colors!). 4 GB of memory (!!!). This is clearly not using Flash, but a microdrive, although Jobs didn't mention it, I'm sure that is the case. Very cool and not that expensive ($249) compared to flash players, after which they are going. However, the high-end flash players now are about one quarter of the size of the mini and no doubt 1/4 of weight too, and you can get around 512 MB for reasonable prices. So, I still prefer flash players but the iPod mini will probably be a good choice for many people.

Cool stuff.

Update: Matt's link roundup and comments on the announcements.

spambots get smarter

Since today started as a "spam kind of day"...

Something I noticed over the last few weeks is that I've started to receive spam that is way more targeted than before. In what sense?

Well, let's say this: I'm getting spam that not only knows my full name, but also my address. Okay, not my current address, but I've already gotten spam that explictly mentions both my New York address (from 5+ years ago) and my SF Bay address (from 2+ years ago). This is bad, not only they know my email address, but they also know where I live(d)! Yes, we know that with time and money you can get a lot of information on anyone, but this has to be done automatically and massively, or otherwise it wouldn't be a practical option for spammers.

Clearly, one way this could happen is if someone (say, buy.com) has been selling their customer information. Since I usually take care of buying online only when my privacy is more or less protected, this is unlikely, though certainly possible.

There's a more likely way in which this connection was made: Google.

Google not only knows the web, it also knows other information... like phone numbers (at least in the US). Jon mentioned this some time ago.

A spambot to get "connected information" would work like this. Say you write an automated script to go through phone numbers on Google. Then the script takes the address data and the person's name, and then googles the person's name. It takes the first few results (or maybe only the first one) and scans the resulting pages to match an email's name to the person's name. Sure, this won't be 100% correct, but spammers don't care about that. And Google's reach makes it reasonable to think that you'd have a reasonably high hit rate. You could even write a program that uses the GoogleAPI for it.

Sure, we could say, as Scott McNealy does, that "you're privacy is gone, get over it". Even if you agree with that statement (and I don't, at least I want to resist it!), this is nevertheless disturbing. And the question that follows is: does Google have any responsibility for this? They'd probably say that they're providing a service by integrating yellow pages information, which would be true.

I'm not picking on Google, rather Google is the example here because of its reach and pervasiveness, but I'm sure that similar things can be done with other search engines and if not it won't be long before you can. Can we fix this at all? If so, how?

Since this is the tip of the iceberg, my main thought at the moment is that I'm a character from Lost in Space and all I hear is "Danger Will Robinson! Danger!".

connection problems

While I finished deleting the spam comments a few hours ago, the Internet connection problems remain. I can't reach several sites, including news sites and weblogs, and neither IM or IRC can connect. Other sites, both in Ireland and abroad, work fine. Definitely weird.

yahoo goes after google (finally)

Well, well. Finally, a year after purchasing Inktomi, and six months after buying Overture it seems that Yahoo has finally digested both acquisitions and is ready to move on from using Google for its searches and begin growing on its own in the area. The Wall Street Journal reports today (subscription required) that Yahoo is now widely expected to dump Google "within a few months". Quote:

Some marketing firms, which help advertisers manage their online campaigns for search-related ads, say they have been told Yahoo will switch from Google to its own technology as early as the first quarter.

Now, I suppose we all expected that to happen at some point (I for one am surprised it didn't happen sooner) but there's more meat to it, including mentions of Yahoo preparing to leverage its various web properties in making search a more personalized affair. Quote:

Yahoo isn't discussing many of its search plans in detail. But some steps toward independence can already be seen on the shopping section of Yahoo's site, which is now using Inktomi's technology. Type in "digital camera," for example, and the site shows pictures of specific cameras along with their prices, flanked by "shopping tools" that allow users to quickly call up price comparisons, fuller specifications and user reviews. By contrast, the same search on Yahoo's front page, which still uses Google's technology, returns a familiar text-based list of links, starting with those sponsored by retailers and followed by other camera-related sites ranked by popularity.

In the future, Yahoo officials say, searches could become much more personalized. They could be tailored to return results that reflect users' past Web-surfing behavior, for example, or preferences or interests they list in a profile.

With Google now expected to go public sometime before mid-year, things are surely going to get interesting in the search space. AOL might have to do something too to keep up with the other portals (ie., MSN and Yahoo), especially as dial-up erodes as a market advantage over time. AOL-Google anyone?

swt and swing, cont'd.

Yesterday Russ was ranting (his term :)) on how Sun was botching it by not getting behind SWT, because SWT is, in his view, better than Swing. I have written about both a few times before, more recently in this short review of my initial impressions of developing with SWT, and earlier here, here and here among others. Specifically on what Russ is saying I had a couple of things to add. One is that, although I'm obviously partial on this :), I think that clevercactus shows that Swing interfaces need not feel out of place, or be slow, or whatever. And I think it looks better than LimeWire too :). IDEA is also a fine example IMO. However, it's true that all of that is subjective and that for hard-core Windows users there are small differences. For power users in particular the differences might indeed be difficult to accept. The situation is much better in other platforms though.

That aside, there is the other matter that Russ mentions, that of Sun not joining the Eclipse consortium. The main reason given for this is that, for all its platform appeal Eclipse is still, at heart, an IDE toolkit. If you doubt that's true, spend some time perusing the Eclipse APIs, and you'll notice how many times you have to use components from within the IDE package rather than "platform" packages (e.g., "org.eclipse.swt"). Restated, what I mean is that the boundaries between platform and IDE APIs are not clear at all, and I guess that some people would say that's precisely the point, Eclipse is both an IDE and a platform, and that's fine. Fine indeed, but what does that matter? Well, keep in mind that Sun has NetBeans to take care of. With its own community, and plugins, and additional tools, and so on. Were Sun to ditch NetBeans in favor of Eclipse as a platform, they would have to a) port all sorts of plugins and code to the new platform, not to mention "convert" their community, both of open source developers and third party developers, to Eclipse. This is by no means impossible, but it's not easy either.

Then there is the small matter of SWT. If Sun joined Eclipse, SWT would have to be included in the JDK would it not? Sun would have to maintain and release simultaneously three different windowing toolkits for each release: AWT, Swing, and SWT. That doesn't sound good either. And while I like some things of SWT, ditching Swing completely is to me not an option.

Why?

First, Swing does run on every single platform that the full JDK runs on. For example, some users today are running clevercactus on OS/2. That would be impossible if cc was written in SWT.

Second, Swing is, for all its complexity (or perhaps because of it) and incredibly rich and flexible toolkit. Much more so than SWT. Surely this will change as SWT evolves, but that's the reality at the moment. With SWT you are forced to write custom components more often than with Swing, as I discovered when I worked for about a week replicating the clevercactus UI using SWT.

And, finally (although this is a small matter compared to the two above), SWT still requires release of resources "by hand". I find this a horrible step back. Moreover, debugging becomes more difficult. Something might fail not just on your java code, not just on the SWT-to-Native code (say, if you're running it on Windows), but something might also fail at the Native component level. Suddenly bugs have to be tracked on three levels. SWT will be buggy for a while, particularly on non-Win32 platforms (Win32 support is pretty good). And Native errors are very difficult to pin down.

Please note, these are not reasons why "Swing is better than SWT" but reasons why I think Swing can't be discarded at the moment and for some time to come. And that puts Sun in a difficult position.

Ideally, yes, Sun would join Eclipse, ditch AWT in favor of SWT keeping the latter as an alternative to Swing, plus using something like the SWTSwing project to bridge between both worlds. But for the moment, staying out of Eclipse might have been a good choice by Sun to avoid creating even more confusion.

Update: more news today on Sun's efforts regarding standarization on the non-Eclipse side of development tools.

a spam kind of day

Feeling better today. For some reason I'm having trouble at the moment reaching several sites on the Internet. Unclear if this is a widespread problem, or just one of my provider. Plus, as if mail spam was not enough, this morning I woke up to a barrage of spam comments posted on this weblog, more than sixty in total. I've been deleting them for the last hour (halfway through now) since movable type doesn't have (that I know of) a "master" comment list where I can delete all at once and rebuild all the entries in a single clikc. Plus, it's slow to rebuild on this machine (which means that posting all those comments must have taken a good time). Nothing wrecks workflow like having to deal with enormous amounts of spam. As I'm deleting comments I'm also closing comment threads to prevent more spam postings. Sigh.

more or less

Here, that is. More or less here. In body, definitely, but in mind it's still "more or less". I have been battling a cold for the last couple of days, ups and downs and still adjusting after so much travel. All sorts of "--aches" and general dizziness. Waking up this morning was truly a hellish experience. It took me half an hour to feel even remotely normal.

So.

I've been reading some interesting stuff, mostly as I can, since it's hard to read on the screen or even a book with a headache always there about to break out. But this fast company article on Apple and "innovation" caught my eye. Interesting, though I don't agree with all of its conclusions. On CNN I saw this mention of new search interfaces, which also seemed interesting.

And as far as books, I'm re-reading Chesterton's The man who was Thursday, which is of course fantastic.

Anyway, hopefully by tomorrow I'll feel better, and be able to getting back "into it" ("it" being, well, everything!) ... there are a number of cool things coming up this week--and many ideas and thoughts that have surfaced in the last few weeks of non-Internet life.

happy new year!

I got to Madrid okay yesterday, but, in the end, I couldn't get the modem connection to work properly... and in truth I didn't spend that much time with it (I fixed that a few minutes ago problems with a DHCP connection that required some fixed TCP/IP settings)...

... but now we're about to leave, so I'll be brief: Happy 2004 everyone!

PS: Digital life resumes tomorrow afternoon, after I get back to Dublin. :)