broken mac

Some time after I got my Macbook, a CD got jammed in its DVD drive. Blast! As it happens, we had a refurb Macbook at the office so to avoid spending a week or more without it while Apple fixed the drive I swapped hard drives and continued using the other Mac. Things got busy at the office and my Mac stayed with the jammed CD for a few weeks -- since I had a replacement there wasn't a reason to rush it.

Now, a few weeks ago, the Macbook started hanging. First, it wouldn't come out of sleep. Then, it started locking itself up even when waking up the display (i.e., it was connected to the adapter and so only the display turned off). It started doing this once a day. Then twice a day. Then it started to lock up mid-use, sometimes even a few minutes after rebooting.

Don't get me wrong, when I talk about locked up I don't mean I could Force Quit whatever was dead. I am talking about a complete OS-wide lockup that left the colorful spinning thingy spiraling as if the machine was trying to hypnotize me. The only solution was to hold down the power button for 5 seconds and do a hard-shutdown that way.

Yup. Not good.

A couple of times, both Safari and Firefox (not simultaneously) had gone out to lunch and for some reason were consuming 100% of CPU (usually just one of the cores, but that seemed to be enough for the machine to stop responding). That was my leading theory until lockups happened without the browsers loaded. Then I decided that the refurb had a circuit that was berserk. So we got the original Mac fixed and I swapped the drives again. Yes! Now everything would be alright. All the original parts were reunited.

No such luck. After a few hours I was back in lockup land, and if anything things were getting worse. I couldn't understand what was going on -- I use exactly the same software on my Mac Pro at the office and it's never locked up like this. I looked online and found references to lockups due to a corrupted "sleep image file", which if deleted could restore sanity.

So this morning I decided that enough was enough, and that if I was going to try anything else radical now was the time. I didn't want to try the image file thing since sometimes lockups could take hours, and I was in no mood for waiting. I backed up everything (easy process between Apple's Backup software and the fact that I've centralized my data in my home directory for years), and I reinstalled OS X.

I did a clean install, which took about 2 hours total, mostly unattended. Most of the install didn't require my attention, and installing the OS updates at the end, while still a bit of a pain, was a single-step process, compared to the multi-hour nightmare that is Windows Update right after you do a clean install of XP or even Vista.

Another big difference with reinstalling a machine, compared to Windows, is restoring the apps you use. I just went to the Applications directory, TARred each ".app" directory that I wanted and copied it off to the network server, then uncompressed and moved back in each dir to the newly installed copy when it was done. The whole thing took about 20 minutes (then of course, I had to re-add all the license keys and such, but I keep good track of those). Windows would have required endless hours of switching CDs and DVDs, one after another, until your setup was complete. I know. I've done it.

Anyway -- the machine now appears stable -- it hasn't locked up all day. We'll see if this continues, if not, I'll try the sleep image file thingy.

I feel like I've gone through some sort of twisted rite of passage. OS resintalls! Looking forward to the time when I have to reinstall the software on my coffee table. :-)

the next blog you'll add to your feed reader...

...is Marc's.

Yup. Go read, and stop wondering what I'm talking about. :)

ubuntu server 7.04's paltry default packages

There are some basic packages that the basic distro of Ubuntu Server (as of 'Feisty' 7.04) does not include. I was just documenting a bit the sequence of apt-get commands I used right after the install was done:

apt-get update
apt-get upgrade
apt-get install ssh
apt-get install lynx
apt-get install links
apt-get install vim
apt-get install gcc
apt-get install make
apt-get install sun-java6-bin
apt-get install sun-java6-jdk
apt-get install subversion
apt-get install smbclient
apt-get install smbfs

The update and upgrade commands are to update apt-get's lists and then upgrade packages that were just installed from CD, respectively.

Some of these are perhaps a bit less common -- smbfs maybe. But vim? gcc? make? Really? Not to mention ssh. The client of SSH comes in pre-installed, but you have to install the server.

I imagine there's some weird reason that has to do with copyrights or encryption, or the copyrights of encryption, but it's still a pain. Especially if you forget about doing it...

jpc: holy emulators batman!

JPC is a pure Java emulation of an x86 PC with fully virtual peripherals. You can go to their site and run the applet demo, which runs FreeDOS and then lets you execute various classic PC-DOS games such as Lemmings or Prince of Persia. And it supports protected mode, so you can run Windows 95 and --gasp!-- Linux.

Because it's an emulator and not simply a hypervisor, you can run it anywhere in which a Java 5 or higher JVM can run.

Mindblowing.

ps: in the same vein, check out this Browser emulator which simulates the experience of older browsers within your... browser. Right.

at javaone tomorrow!

Martin, Brian and myself will be at JavaOne tomorrow presenting Building a Web Platform: Java Technology at Ning. We'll talk about the evolution of the Ning Platform over the last two and a half years and how Java and some specific design choices let us continually grow and expand the platform, replacing and upgrading infrastructure, without affecting users or developers.

The session is TS-6039, in Esplanade 301, at 4:10 pm, so if you're around come say hello. I'll post the slides after and talk a bit more about that and other interesting things. :)

128-bit storage: are you high?

As Reverend Lovejoy would say: "Short answer, Yes with an if.... long answer, No, with a but."

Nevertheless: great article on ZFS and storage limits (theoretical and otherwise). Recommended!

yes, you can use that which you had paid for

[via Slashdot]:

Up until now, manufacturers have been wary of building a device to allow this type of usage because they've been afraid a lawsuit. The DVD Copy Control Association had claimed this was contractually forbidden, but now a judge says otherwise [...].

Up next: the Department of Justice decides that it's ok to put a PC in the living room and connect it to a TV, solving the problem that has puzzled many a tech geek.

Radical notion, that of us being able to decide how to best make use of our own property... :-)

Ubuntu Feisty on OS X and VMWare Fusion

So I tried running Feisty on Parallels but no dice -- it would boot but go no further. Additionally Parallels doesn't seem to know Ubuntu exists (they know about Xandros, but not Ubuntu?).

So I tried it in VMWare Fusion (Beta 3) and it worked perfectly. The VMWare Tools installed flawlessly as well. Performance is great, but then again, the machine does have two dual core Xeons. :-)

Running Ubuntu inside OS X is mostly useful if you want to test browsers for example or verify something platform-specific. OS X is too good a UNIX to make me miss Linux. Parallels does come in handier for XP/Vista tests, which also run pretty fast. Most of the time, though, is OS X all the way. Right now the only disadvantage that has is the lack of an official, final version of Java 6 (which we all assume will come with Leopard...), but you can get the developer preview from Apple's Developer Connection, so API-wise, at least, you're mostly covered.

Edgy -> Feisty

So it turns out that the upgrade to Feisty Fawn was actually fairly simple, even in a headless environment. I happened to check out the Ubuntu Upgrade page and it was fairly simple.

I ran

sudo apt-get install update-manager-core

Which failed, even though I had just installed Edgy yesterday. I figured all apt-get needed was a refresh, so

sudo apt-get clean
sudo apt-get update

did the trick. After that

sudo apt-get install update-manager-core

worked fine, followed then by

sudo do-release-upgrade. After that, I left it running for a couple of hours, and at the end (naturally) it rebooted, but didn't come back. Horror! All was lost!

Well, not so much, I hard-rebooted the machine and it came back happily. All is well. Pretty good!

OS X -> Edgy

So during my migration of old server to new stuff, I decided to experiment with using OS X as a server environment. With Apple Remote Desktop, it wasn't half bad really (although ARD is really crappy, speed-wise, compared to Microsoft's Remote Desktop Connection, thanks to its VNC roots).

It took a while, especially since I was unfamiliar with a lot of the OSS-on-OSX subculture in the Intertron, but in time I had everything set up including Fink, a LAMP stack, and such, but weirdness remained. For example, I kept fighting mysqld and its tendency to decide to not startup on boot, no doubt my own lack of knowledge of some particular OS X magic rather than an inherent lack of the feature by the OS or Mysql.

Anyway, so I was talking to Russ the other day and he very reasonably said that I was nuts for not using simply Debian or something of the sort, and last night I decided to give it a try. I was very impressed. In about 45 minutes I had Edgy server installed, with a full LAMP stack, Mysql properly configured and everything migrated. I shutdown, reconnected everything to replace the Mac, and I was done!

There is some sluggishness when posting right now, probably as a result of the crappy disk or perhaps not enough memory (although the machine seems perfectly happy), or maybe it's the fact that I'm using LVM. I will try a switcheroo to Feisty (which came out today) at some point next week when I have another hour to spend on this and perhaps I'll also try out a couple of different options. :)

a couple of great tools

At Ning we use JIRA as our bug-tracking system (when we started a couple of years back we were originally on Bugzilla, but we switched to JIRA fairly quickly), and it's really a great tool, part of our development and release process really (I'll talk more about that in another post).

For intranet doc-keeping we have mostly been using Confluence (also from Atlassian) and it's good enough, but about a month ago I discovered Clearspace from Jive Software (the guys that wrote Wildfire, now Openfire) and we've been evaluating it since, well, really using it. It's a fantastic product, seamlessly integrating discussion boards with a wiki, simple ways to turning a discussion into a wiki doc, group and individual internal blogs, etc. Also, it has some nice features like searching inside PDFs that you uploaded.

Confluence is a bit more wonky than Clearspace, but it still does the job if all you need is a Wiki. However, for the combination of blogs, forum, and wiki docs, Clearspace wins hands down (except in one point: Wiki syntax in Confluence is better).

Regardless, depending on your needs, all great tools. Highly recommended!

mowser!!

So, Russ has taken the wraps off his project Mowser!

It's a "mobile browser" plus a mini mobile portal, including useful links, feeds and commands (which he calls keywords) all rolled into one.

He's still polishing some details, so you should expect some minor things to be weird for a while, but it's all there!

He's been working on this for months-- so congratulations Russ! It's awesome.

and the prize for most misleading headline of the day goes to...

... News.com, with their article "Yahoo to give away email code", from the article:

The move to open up the underlying code of Yahoo Mail--used by 257 million people--is designed to spark development of thousands of new e-mail applications built not only by Yahoo engineers but by outside companies and individuals.

Hmm... now, let's see what Jeremy has to say about this:

Our Browser Based Authentication (BBAuth) is a generic mechanism that will allow users to grant 3rd party web-based applications access their Yahoo! data. There's already a similar mechanism in place on Flickr and used by services like MOO. BBAuth is the protocol that's going to open the door to doing the same thing for many Yahoo! branded services in the coming months. Stay tuned for those announcements. :-)

The first two Yahoo! services supporting BBAuth are Yahoo! Photos (API) and Yahoo! Mail (API only available to Hack Day attendees at the moment).

Okay. So, Yahoo is opening up its authentication API, and probably other APIs to be able, to, well, do something useful with the Yahoo auth API aside from signing on. This isn't peanuts, it's definitely interesting, but ... "giving away email code"? Please. Someone needs to talk to the News.com guys and explain the difference between "giving away code" and "exposing an API", pronto.

did you know...?

... that Ning is hiring? But of course you did! Well, here's a reminder then. :) If you're looking for something, go check out our list of current openings at http://jobs.ning.com/. From Java developers/architects to QA engineers and product management, there's something for everyone!

(Did I say Java? Wasn't Ning about PHP? Well, the apps are written in PHP. But there's a ton of Java in there --some really cool stuff-- even if it's not obvious... but that's a topic for another post).

And, hey, if you don't find what you want in there, but you think you want to work with us, send us an email anyway. :)

microsoft gets it

I think it's about time we dropped the "Microsoft doesn't get web 2.0" meme. Scoble today has a post responding to Kaliya in which she says that to Microsoft we're just "customers". Please. Everyone tries to do what is best for both their customers and company. MySpace may "invite dudes to contribute" but trust me, News Corp. doesn't give a damn if they are called "customers" or "enablers" or "contributors". Microsoft still has some vestiges of its predatory behavior in the past (particularly on pricing of most software that hangs to stratospheric monopoly heights, ever notice that Office and Windows are the only two Microsoft products that are not at lower prices than before? Look at the stuff where MS has competition.) But on the other hand they'd probably be sued by shareholders if they lowered earnings "for no reason". Anyway, that's not really at issue here.

Look at Bill Gates at Mix06, talking to Tim O'Reilly and Mike Arrington. Look at Windows Live, and the stuff they're doing with gadgets. Look at the integration in Vista. Look at Channel 9. Look at On10.net -- whatever that's about, it's certainly not the Microsoft of old. Look at what Microsofties are discussing in blogs, from Ray Ozzie, to Scoble (of course :)), to Dare, to Mini-Microsoft, to hundreds of others (Quiz: Assumming you don't work for Google, how many high-profile Google bloggers can you name vs how many from MS? I'm not saying it's good or bad, btw, really, to each his own, I'm just pointing out the difference in, um, "engagement"). Look at Ray Ozzie's LiveClipboard stuff, which from what I've seen underwhelmed many but it just blew me away. This didn't look like a Microsoft demo at all! It looked like the demo of some dingy startup, three guys just kicking cool stuff around! Is it small, perhaps a bit of a trifle given Microsoft's resources? Maybe. But damn! There it is: A screencast, done on Flash, and running all on FireFox, using standard Internet formats. Three years ago, you'd probably had to endure some insane ActiveX plugin and a demo of how IE and Office could do stuff together using COM or some such.

If anything, that's what tells me that Microsoft as an entity gets that it must adapt, and it gets where it should go, and it's trying, really, really hard. They've opened up the floodgates to some degree -- Microsofties are doing a lot of stuff that may not be necessarily "sanctioned" or perfectly aligned with the different BU requirements of Windows or Office. This was done by necessity rather than out of some high-minded pursuit of "innovation," but that's ok, that's how these things work.

Microsoft may not be out of the woods yet with respect to the threat that web 2.0 (and, let's not forget Google) represent to their "traditional" business. But let's give credit where it's due. They're moving.

And they didn't need an "Internet Tidal Wave" memo to get them going. So hats off to them!

PS: Btw, let's not get hung up on whether web 2.0 is hype or not. Is there some hype in there? Sure. Is there something real behind it? Yep. Does it accurately describe a market space? At this point, yes. Ok. Good. :)

plugging the dns recursion hole

Via this Slashdot article I was reminded about a vulnerability in DNS configs that allow recursion and therefore let the server act as an open relay that could be used in a DDoS attack. I verified my DNS using DNS Report and this matched what I saw in my config files -- my DNS server was open. Rogers had a post last week on the topic which outlined the steps he took and served as a quick guide, and along with this page of the BIND9 manual I had the whole plugged in a few minutes, confirmed by the DNS Report tool. Phew!

evernote

One app I've been trying out for a few days now is EverNote. It's truly, really well done. It does what it's supposed to do, and it does it well. Great UI, including some cool UI concepts like their accelerating scrollbar. I've also tried OneNote, but I've been underwhelmed. Too much complexity really, for a simple task like taking notes. EverNote also has good integration with browsers, including a plugin for FireFox 1.5. If you need a good piece of note-taking software, look no further. Highly recommended.

Update: Russ tries out EverNote but is confused by the non-standard UI. Good point, I forgot to mention my take on this. EverNote does use a different UI than we're used to, and it definitely has a bit of a learning curve. My opinion, however, is that our current UI paradigms are broken. They just don't work. So we will need to change into new ones, and given that pretty much anything new and more efficient will involve some sort of shift and relearning. I think EverNote pushes the envelope enough to make things better but not so much that you have to spend hours learning what to do. That's my 2c at least. :)

filebox: a quick way to share files

Many many times I want to quickly send a file to someone for them to look at, and I can never remember the names of the services that let you do this. But there's Ning! :)

So my 1-hour hack for tonight was to create filebox.ning.com, which allows you to upload files and then share the link with others, and it's deleted after a few days. Basically I cloned Brian's filedrop, modified some things in the code, made the uploads private, added messaging, and made it a little easier on the eye. The power of Ning at work. :)

windows vista: me likey!

For about a week now I've been using Windows Vista (CTP, Build 5308) in my machine at the office--at home I am still running XP-- which we got through MSDN. The CTP is of the "Windows Ultimate" flavor which includes Pro+Media Center+Tablet PC functionality. Nice. I dared to install it at the office because if that machine couldn't handle it, I wasn't sure which one could (Dual Core P4, GeForce 6800, 3 gig of RAM... you get the idea). It's an early beta (they're probably still six to nine months away from release) but it's pretty stable save for a few wrinkles here and there and the occassional slowdown. But I'm getting ahead of myself.

The installation process is really, really good. Pop in the DVD, click "install" and wait for a couple of hours. In my case, I did that as I left the office one night (yep, perhaps I'm a bit kamikaze that way) and when I got back the next morning, the machine was botted on Vista. That was it. No clicks, no questions, it just worked.

Once I booted the first thing I loaded was IE 7. I wanted to see how well it worked. Boy, was I impressed. Network transfers were super-fast. We huddled around it and wondered if IE was doing something funky, but whatever was going on also extended to FireFox, which loaded webpages at least 50% faster than before. I use FasterFox, so I know I wasn't dreaming. We compared the speed of loading the same pages on similarly configured machines (both Linux, Mac and Win) and Vista blew them all away. Sometimes twice as fast. WOW. I guess that the rewritten TCP/IP stack does make a difference after all.

There were a couple of problems with the install. First, I had Norton Antivirus running and it just can't be uninstalled for some reason. I can't stop it from getting in the middle of other processes, such as downloads either, and I haven't had the time to look for a solution. Probably a registry setting. At any rate, it's very annoying. The really annoying feature is the new user security they put in, essentially downgrading user privileges so you have to "sudo" for some operations. What's frustrating is that my user is an Administrator, there is apparently some hidden Administrator user that I don't have the password for, and there are folders that I can't delete. Period. I can't even look into them. That is just plainly idiotic. It's MY machine. Give me all the warnings you want, but let me access my own files damn it! I could bet that NAV is getting confused with that too, since it plugs in at a pretty low level. Other minor problems: the video driver Windows installed on its own was old -- but getting the update for Vista from nVidia fixed that. Then Powerarchiver wouldn't work at all, but WinZip did just fine. Oh, another tiny thing: when I shut the PC down, it bluescreens for just a second. I'd bet it's Norton again. But it doesn't affect anything. So it's fine. :)

Those wrinkles aside, the experience has been pretty good. Vista is fast and stable. Search has suddenly become useful. I can now search Outlook messages from the OS (and open them) faster than from within Outlook, and I didn't have to install or configure anything. Good one.

The hardware requirements are pretty steep at this point to run it properly, but I think those will come down as it gets optimized. As I understand it, build 5308 is the first one that is close to being "feature complete," so there still must be a ways to go in the optimization front.

Let's see, other cool things. The new sidebar (a copy of Konfabulator) is cool, and there's a fairly web2.0-ish site that deals with "gadgets" that apply (the site, not the gadgets) to both Windows Vista and Windows Live. The famous "Flip 3D" view is pretty cool and even (gasp!) useful. But, I searched aaaall over the place to see how to use it, and everyone talked about how cool it was but NO SITE said how. So here it goes: Windows Key + Tab. Alt-Tab is the regular switch, but with "live views" of the windows scaled down. WinKey+Tab is the Flip 3D thing. There. I said it. The secret's out (hold on for a million comments telling me where the obvious place to look was). :) You'll need a spiffy video card to use it, but it will work well if your system can handle it.

So far then? Aside from the ludicrously bad file permissions thing, I am very impressed by vista -- in fact, I'm sold. No WinFS, true, and it took a hell of a lot longer than it should have, but Vista is definitely going to be a good upgrade from XP I think (damn, I sound like Scoble!). Anyway.

As the Firefly characters would say: "Shiny." :)

A free VMWare?

Now that would be cool. VMWare, for its reliability, features, and simplicity, is one of my favorite sofware products of all time. News.com says the free server edition is just around the corner. Now what about the workstation? I'd bet that half the development world would rush to download it and start tinkering with it. Speaking of tinkering. VMWare APIs, anyone? :)

opera mini: awesome

On Tuesday Russ sent me an SMS with a link to get Opera Mini as soon as it came out and I have to say I was massively impressed. This is not really a review but just a small comment -- Russ has more here. Even considering the install was a bit wonky due to the typical hoops we must jump through to install non-carrier-sanctioned stuff on a phone, and that afterwards it ends up being shoved into the "games" section (at least on my RAZR) it's still worth it. Fast and usable, finally a browser for phones that doesn't suck. If you have high-speed data services for your phone and were wondering what to do with all that paid-for bandwidth, check it out.

blogs and discourse in software and politics

Salon's Scott Rosenberg muses on What journalists can learn from software developers and points to my recent exchange with Mike Arrington as an example. Thanks!

Which reminds me of what I wrote a while back in rethoric, semantics, and Microsoft and some of the stuff (such as the "rules to posting") in my introduction to weblogs.

Political discourse definitely feels a bit shrill these days. Then again, when reading the history about other times of relative upheaval (about, say, the late 1700s/early 1800s period in US history, the Civil War, or World War II) it's striking to see that they weren't all that "nice" back then either, in fact the early days of the republic were pretty vicious in terms of political rethoric and even backstabbing (witness the falling out among some of the Founding Fathers). I think it's just that it was much harder for tempers to flash out of range, since by time involved in sending messages back and forth allowed for some cooling-off period -- and live debates weren't watched by millions of people at once, who also had to respond asynchronously. I wonder if it's just a matter of slowing down a bit then... :)

movable type 3.2: best MT yet

I've been meaning to post this for sometime now, but I keep forgetting: MT 3.2 rocks. The comment/trackback spam management is excellent, and pretty effective at stopping 99% of spam coming in. The new template management was a bit confusing at first, but all's well now (I still haven't converted everything or even all that much, and only this morning I fixed search, which was broken due to misconfiguration). I remember the conversion to 3.2 to be mostly uneventful except for a few minor hiccups. All-in-all, a must-have upgrade. :)

quote of the day

10:40 Outside, a guy talking into his phone: "Well Steve Jobs is a fucking Jedi Master of this shit compared to these other clowns." from Engadget's coverage of Yahoo's CES Keynote this morning.

movabletype 3.2

I recently switched over to MT 3.2 (which came out in August I believe) and it's been an improvement on a number of fronts. In particular spam management for both comments and trackbacks has become way easier, not painless, but less of a pain. There are some new features in it that I haven't had time to explore (the template setup is different) and for example search within the blog is now broken and I'm not sure why (an "alternate template" is missing). I guess I'll have to do some digging this weekend.

xbox 360: a review

While I am indeed pretty busy that doesn't prevent me from, say, sleeping even less and every once in a while using that "extra" time to to play games, on an original xbox. Initially I wasn't going to get an x360, but it so happened that a nearby Best Buy decided to allow preorders and off we went last Friday.

booting up
By now I've got a fairly good handle on the dozen or so different ways to connect A/V systems (composite, component, Svideo, DVI, optical or not optical sound input and all the variants they create), and more assorted lengths of wire than Professor Farnsworth. So once I got home connecting the system was fairly painless. On first boot things looked pretty good, although the image quality was a little off, even with HD settings (my TV supports 480p and 1080i)--but more on that later.

Before that: oh, the power brick. It feels like a brick, both in size and weight. Probably a third of the volume of the xbox itself (on the left in the picture). It is completely anachronistic next to the sleek design of the rest of the system, but so be it. Power has to come from somewhere no?

The controllers themselves include a few crucial improvements over the original xbox controllers, most notably the option buttons that used to be to the left and right of the joysticks are now better distributed -- start in the center, and the white/black option buttons are now on the front, above the triggers. Much better. The remote control (included as a bonus right now in the pro configuration) is also pretty good, although for some strange unfathomable reason the remote is infrared whereas the control is wireless.

Simple things, like being able to turn on the xbox from the controllers or the remote, are actually big improvements. The dashboard can be accessed easily from within a game, navigation is easier, etc. The network setup was buried behind several screens, but I was glad to see that it supports both WEP and WPA, WPA-PSK in particular (although they call it "WPA password" for some reason).

Overall, the initial setup experience was fairly painless, but I think that non-techies may find it confusing, particularly for the wireless setup options, and especially if they've got secure wireless setup by someone else.

that media center thing

Configuring the 360 to do streaming off my PC's music collection took about 10 minutes, although a couple of false starts were involved (PC not finding the xbox, then the xbox not finding the PC), and I think that people without PC experience will probably find the process fairly confusing, but in my case it worked out well. The XP-based application is easy to install and configure (at least by MS standards, again - having to dive into the control panel to change settings is something that will confuse a lot of people methinks). So suddenly I have a single repository for my music and photos--not bad!

Video support seems to be limited to Windows Media Center (I have XP pro), but I don't have high hopes on that, I assume they support MPG2 and WMA, which is not usually the format in which I store video anyway. Strangely enough, there's a section for "movies" but it contains a few trailers that come built in, and that I can't seem to be able to get rid of. GIven that the x360 comes from the factory with only 10gig free of disk space, that's pretty bad. If they hadn't included PC integration, no one would use it for media playback -- there's just not enough space there to do anything useful.

But I digress. The integration, once set up, is pretty seamless, although I'm so used to the ipod's waterfall menu mechanism/navigation that the linear behavior on the xbox felt somewhat clunky. Regardless, it was really cool to get more juice out of the music on the PC as a bonus (you could do that with the original xbox, but you needed another CD and I never took the time to set it up--or you needed a modded box).

the games

Holy moly, the difference in game experience with HD and surround is amazing, particularly with a large screen. Project Gotham Racing 3 is fantastic ... once you actually get HD set up. I mentioned above that I configured the box for HD/widescreen within the dashboard, but there was another (annoying) step to get through: flip a tiny switch on the video component output cable from "TV" to "HDTV". In setting stuff up and naturally not reading the manual you miss this crucial step and things look good but not that good. Anyway, with the switch in its proper location, the image was just gorgeous. For once, the game trailers really reflect well the astonishing quality of the graphics. The menu navigation of the game is actually more confusing than PGR2, but they've made some small improvements that make gameplay better.

As for first-person shooters: Call of Duty 2 is excellent, but it suffers from lack of a cooperative multiplayer mode. After having tried Ghost Recon 2: Summit Strike on the original xbox, I'm much more interested now in those kinds of games than in standard deathmatch (although that's definitely fun :)). Quake 4 is good, but nothing shockingly new: think Doom to the nth power. Then there's Perfect Dark Zero, which has some interesting new takes on the FPS genre, and looks great as well.

The surprise for me was the "Arcade" game category in the xbox live section: you can access lists of games and download trials or buy full versions of classics with "microsoft points" (on this topic, btw, I agree with Russ: I don't like them for multiple reasons, but I don't think they'll get rid of it anytime soon).

backwards compatibility

The x360 runs a good amount of original xbox games, but not nearly enough. When they do run, however, it's pretty amazing. Halo 2 for instance looks better and runs perfectly well. This is definitely an achievement considering that the original xbox ran on a P3/Celeron and the x360 one runs a multicore PowerPC. There definitely seems to be some fairly complex emulation code there, assumming that what they're doing is indeed pure or mixed emulation.

final thoughts

Overall, the x360 is really, really well done and the games are excellent. Microsoft has definitely done a good job with it, wrinkles notwhitstanding. There's been some discussion for example on this slashdot story (and then echoed all over the place) on how the 360 is "very unstable." If shreds of anecdotal evidence is all we're going with, I know of 5 xbox 360s, 3 pro and 2 core systems, all of which run without a glitch. It very well maybe a heating problem. That would not be surprising considering that this little box packs as much processing power as a modern PC (and look at the fans and heat exchangers on those things).

One thing that does bother me a bit is that there seems to be a lot of information about what you're doing and what you've done broadcast by the system to others (just entering into an xbox live game once seems to grab that list of people forever, and then you can see their status) with no obvious way of turning that off. I assume privacy settings are in there somewhere, I'll have to look for them later.

Anyway, given the price and supply issues it's certainly not a device for the masses yet, but it's definitely something worth trying if you can. The next big game to be released will be IMO Ghost Recon: Advanced Warfighter. But that's in February... in the meantime, there's a bunch of cool stuff to try out :).

grokster shuts down

Grokster has shut down as a result of the Supreme Court decision a few months ago. The website now reads:

" There are legal services for downloading music and movies. This service is not one of them.

Grokster hopes to have a safe and legal service available soon."

Right. Because that strategy worked great with Napster didn't it? Even if the new, "safe and legal" Napster continued limping along, its user base was completely gone, and so it will be with Grokster. I honestly cannot understand how the RIAA and others think this plays out in the end. You can't uninvent technology, and all the lawsuits accomplish is to push it further underground... and through these aggressive moves they only stifle investment and research on the topic, which is really the only way a happy balance can be found.

There's no way out, there's only a way through.

synergy: wow

Here's something really cool: synergy. A GPL'ed tool to share mouse and keyboard across different systems with Windows, Linux, or OS X. A virtual KVM, if you will. You need two monitors, and two machines. But if you've got that, this tool is a must-have. Most excellent.

@ foo!

Got to Foo Camp relatively late, around 8:30 pm, after picking up Drew Endy in SF (he was at a conference there, and since I was on the way north, I gave him a ride up to Foo Camp. Drew (and those in his lab at MIT) are doing mind-blowing stuff in trying to figure out how to code DNA to make it do what they want. "DNA on Rails". So even though we got here late, that unique part of Foo had already started, at least for me (maybe not so much for Drew, since I basically pestered him with questions the whole time). I was thinking that what I'd like to do from the top down (make software behave more like organisms) is what they're working on doing from the bottom up (figure out how to code to the software that organisms already have). From one end it's getting the black box and tries to figure out how to make it do what you want. From the other, it's trying to emulate the external behavior of the black box and make other things do what the black box does. Box are way too interesting to make them justice at midnight after an exhausting day. :)

I did miss Tim's kickoff presentation, which was incredibly interesting when I saw it at EuroFoo last year.

All rooms have been taken and I couldn't get a hotel, so I'll be sleeping in the car. Since I won't be sleeping much, I don't anticipate it will be too bad. :)

how to install comcast high speed internet: a quick guide

I would generally not post about something as specific as this, but after seeing the vast amounts of confusion and misinformation that's out there on the topic I felt that my 2c in this case would be worthwhile.

Today I got cable along with Internet service from Comcast, and one of the potential that I wondered about was the setup. I've got a Linksys WRT54G for both wired and wireless routing. Most of the comments out there mentioned a process that usually included first connecting the computer directly to the cable modem, then registering it, then disconnecting it, then connecting the router configured to spoof the MAC address to that of the computer.

However, this didn't seem to make a lot of sense. While the MAC address of the cable modem does need to be registered with comcast, it seems less obvious why they would need the MAC address of the PC itself. Maybe at some point it was necessary, but as I discovered today that's no longer the case. )As a sidenote, the cable guy that did the install of the line also said that the "pc first, router later" process was a requirement... so maybe that's how this got started).

To make sure this remains a quick guide, here's what I did (as far as I can tell the procedure would be the same anywhere in the US for a Comcast high speed Internet install):

• First, I got my own cablemodem, a Linksys BEFCMU10 since the price was equivalent pretty much to a year of renting whatever Comcast was going to give me, it seemed like a good investment. I don't think this makes a difference though.
• Once the connection was complete and running, I connected the Coaxial cable to the modem, then connected the modem via ethernet to the WRT54G's WAN input, and the PC to one of the ethernet ports of the WRT54G, then I powered up both the cable modem and the router. (The router must be set in DHCP mode).
• After a bit of waiting, the router directly obtained a dynamic IP and all seemed well. Loading up any browser page in the computer redirected me to a comcast page which included a link to download the software that comes with the Comcast self-install kit. I don't even need to insert a CD!
• Downloaded the software, then executed it. It turns out it's basically a specialized IE window, doing HTTPS requests to Comcast in the background. Here they actually activate routing for your cable modem, but at the same time they're assuming that the PC is connected directly to it, and so they seem to reconfigure the modem to talk to the PC's network card (which isn't connected directly to it). When the registration and configuration is done, they reboot the modem automatically, at which point nothing works.
• Aha! The reboot seemed to be key.
• So, I did a hard restart on both the cable modem and the router (first disconnected the router, then reset the modem, then reset the router, then reconnected the router) at which point the cable modem seemed to find the router agreeable once more.
• And that's it!

From what I gather the modem does attach itself to one MAC address, but that is reset when it reboots, so spoofing the MAC address (which sounded like a weird requirement in the first place) doesn't appear to be an actual requirement. The MAC address of the cable modem itself is registered with Comcast, so you'll need to let them know if you get it replaced.

After all that, speeds are pretty great, 3 Mbps down, 768 Kbps up (or more), and it's working well so far, keyword here being so far since there were some signal strength issues that made installation more tedious than it should have been, and those gremlins have a way of showing up again...

PS: I also got HDTV with Dolby Digital. Not that many channels, but one word: Wow.

back to PST!

I just realized that the weblog was still "on" GMT (where I have spent most of the last 4 years) but now it's time to switch that, too. And done!

Sure that was easy. But I wonder, now. MT seemed to be intent on rebuilding all entries based on the new target time, but that was just appearances. It offered a rebuild but did nothing of the sort. No changes. All the dates that were wrong are still wrong. I wonder if there's a button to push somewhere. Has the timezone info been lost?

And what's the right etiquette? Plus, the timezone in itself contains some interesting information (for me at least), but only if it's accurate. Accuracy there would be hard to guarantee though, the browser can obtain some of that information from the OS, but that's not even accurate sometimes... Hm. Now if a GPS was permanently connected and feeding data into the system permanently and seamlessly...

quote of the day

"What I really need is minions." -- Russ.

geography, software, and context

While I've been sort of away from blogging (and keeping up with blogs as well, I'm now beginning to catch up) as always I've spent time running around looking at the new apps and ideas that are thrown out there all the time. Of these, I've spent a fair amount of time thinking about mapping applications, for one reason: they puzzled me.

All that follows is probably obvious to many people, but hey, I won't mind that much for arriving late at the party and just be happy I showed up at all. :)

The puzzle was the sudden explosion of use and why it had caught me blindsided. Blindsided, in the sense that they'd be so popular. I've been looking at geolocation apps for a while now (the wireless ad hoc research community, where I spent some time during my thesis research, has been babbling about geoloc for quite a while now). But it had never caught my attention all that much. Partially the blinders that come from focusing on one topic are responsible, but on the other hand I found it hard to see them as more than niche applications. GPS: useful if you're traveling but most of us don't spend our lives in the car or train, and usually quickly develop enough knowledge of our transportation paths that a GPS can become redundant quickly when a route is coupled with habit. This is less true in the US, given that there's higher mobility and greater distance between places, but still you can't see using geolocation as something more than something that would be use ocassionally at best. And so on.

Geolocation is cool, I thought, but it's not massive, as say an RJ-11 plug is massive or as the Internet is massive. True, there is value is niches, and we could start yet another socratic discourse on the long tail, but...

When I started thinking about this, I began by questioning whether these apps where actually niche apps. In the end I realized that the apps are niche apps, but not just any kind: they connect meatspace with cyberspace.

That is, they provide context. Many people (me included) enjoy the geography of the virtual. We can see the towering structures, or as Gibson put it on Neuromancer, "lines of light ranged in the non-space of the mind, clusters and constellations of data. Like city lights, receding..." But that's only one half of the story.

If there's anything truly unique to pervasive computing, sensor networks, clothing with embedded circuitry, and a million other things that are going on right now, it is that they seek to provide a seamless bridge between the real and the virtual. Not a one-way connection mind you, but bidirectional. The ability to interlock devices with what's around us in a way that creates something that neither alone can provide. Example: object geotagging. Leave a marker using a webservice, automatically labeled with your Long/Lat, and others that walk by can find it. Have your clothes keep track of your movements and of who you meet, and let your shirt buzz softly against your skin when it realizes that you have an appointment in 30 minutes across town and you are still at a cafe, sitting still, with someone you know nearby ("someone you know" being defined in quick and dirty fashion as the same marker showing up in your travel history for the last 6 months, which makes it likely you won't just get up and leave that easily).

It's all about context, and it's not just geography--geography is just what we've suddenly got a critical mass of data on and the APIs to go with it). Geolocation is the tip of the iceberg. In itself, it's interesting, yes, and useful to varying degrees depending on your habits. But it's way more interesting for what it portends: finally, the emergence of real-world applications of context to software, and of software to the real world.

the age of the mix

William Gibson, writing for Wired: God's Little Toys:

[...] I already knew that word processing was another of God's little toys, and that the scissors and paste pot were always there for me, on the desktop of my Apple IIc. Burroughs' methods, which had also worked for Picasso, Duchamp, and Godard, were built into the technology through which I now composed my own narratives. Everything I wrote, I believed instinctively, was to some extent collage. Meaning, ultimately, seemed a matter of adjacent data.

Yep. Technology is freeing creativity in part because it makes the collage technique much easier. What we create (write, compose, paint, even code) obviously has some precedent in our lives--one way or another. Woody Allen does collage with his own life, and what we usually call a reference in literature walks a thin multidimensional line between collage, homage and inspiration.

My favorite paragraph from the article, however, is this one:

Today, an endless, recombinant, and fundamentally social process generates countless hours of creative product (another antique term?). To say that this poses a threat to the record industry is simply comic. The record industry, though it may not know it yet, has gone the way of the record. Instead, the recombinant (the bootleg, the remix, the mash-up) has become the characteristic pivot at the turn of our two centuries.

Concise, yet encompassing. Pure Gibson.

The age of the mix is here.

nvu

An open source website editor: nvu now in 1.0. Pretty good CSS integration (a bit clunky, but useful) and straightforward UI. Plus, it's available on Linux, OS X, and Windows. Nice.

lemmings!

[via Dare] DHTML Lemmings. Perhaps predictably there are legal questions surrounding it (even if no one has bought that game in years, you'd think that the owner of the copyright would be rushing to buy this thing and put it online...). It's really, really cool. Go check it out.

the network is the disk drive

Sean McGrath: "XML is not - repeat NOT - a 'file format' in the sense that most people use the phrase 'file format'."

Here's his article. Of course he's right. XML is a specification for formats, not a file format itself. It has high-level semantics for defining specific semantics.

But, in any case, when so much of the data we consume flows across the network without ever settling in a well-defined filesystem location, the idea of a "file" stops being so important methinks. Even on the desktop side we are thinking more and more in terms of pieces of information, emails, IMs, webpages--not files.

I am reminded of a short-term backup system someone once described to me: every number of hours a backup would be created, then sent out through SMTP to the company's subsidiary in Australia, where another program would pick it up and then, through direct streaming, send it back. At any point in time there were a number of copies flowing through the system--and if something happened and you needed a certain copy for a certain date, all you had to do was wait for the right piece of data to circle back to you, and you were done (there's a case where a super-fast network would actually be a liability). The same could be easily done today through a couple of daemons and webservices running on top of apache--Rather than Sun's "The network is the computer" it would be "The network is the disk drive".

I forget what my point was.

Ah yes: "A keyboard! How quaint!" (That's Scotty speaking for all out there not fully up to date on your Star Trek arcana).

Let us all put the quaint notion of files and formats behind us and think about where that may take us...

msr's "bittorrent killer"

Kevin Schofield, on the recent brouhaha surrounding Avalanche. Quote:

Um, let me get this straight. In six days, a research project went from some algorithms in a paper to Microsoft's competitive answer to BitTorrent, to "vaporware" to an evil conspiracy."

MS/MSR is getting a bad rap on this one. They've been working on p2p for some time now, it's just that they've never gotten attention for it. Pastry for example is a well-known (at least within the p2p research clique) overlay network project that has been around for more than two years. And their stuff is pretty good, too.

Now that I'm getting an education on certain effects of sensationalist press coverage, I can't say I'm that surprised. At least weblogs help in getting the other side of the story out there...

anne's consulting practice

Anne now has a new website for her consulting practice. Ethnographer, anthropologist, ui-social-network-wireless-pervasive-computing-thinker-designer (yeah, buzzwordy, but true), all in one. When she's done with your project, she can also explain the social structure of the Incas, which apparently didn't have computers, cellphones, or wore Nikes. My theory is that this is why they couldn't stop the Conquistadores, but she says I'm wrong for some reason I can't quite follow.

Anyway, if you need someone to analyze and improve whatever cutting-edge product you're cooking, she's your woman!

the AlwaysOn/Technorati 100

Congrats to Russ on his appearance in the AlwaysOn/Technorati 100 list in the 'practitioners' category! Also to "the Scotts" (Feedster), Om, Jon, Rael, Jeremy, Robert, Matt and... well, everyone else on the list -- I'll stop before I just duplicate a list that seems to strangely match my aggregator subscription.

Russ comes first, yes. He's a good friend. Plus I want to be on his good graces for the next Halo match. :)

24 hour laundry: the view from inside

Well, well, well. :)

There's been a lot of discussion recently about a certain new startup called 24 Hour Laundry. It pretty much got started with this CNET article, then as highlights we've got Om, Mark. Even (perhaps predictably) Slashdot.

24HL, as it happens, is where I work. Remember this?

Yep. It's true. Aaaaaall this time and I didn't say anything. Outrageous! How could I?

Well, that's kinda the point.

You see, we didn't want to make any noise. CNET decided that they wanted to "scoop" a story that didn't exist (and is still not all that exciting at this point). We didn't have anything to do with that article.

Then, in the process of not asking for any press and minding our own business, we get branded a certain way, and told we are doing something wrong by focusing on our product.

What is confusing to me is that some of the comments out there begin with "Well, I don't know what they're doing but [insert your thought about why it's wrong here]".

It is one thing to speculate (which we all do a lot of, don't we) and draw tentative conclusions based on that, but it's another to take those assumptions and then categorically "paint a picture". I know: to a certain degree, these are the rules of the game. But there is a difference between saying "If X is doing W, then here are the problems I see" and saying "X appears to be doing W. They're crazy!" This was partially Russ's point with his great post yesterday. (Update 6/23: Jeff Clavier also makes good points on the topic).

For example, Mark Fletcher said:

[...] But creating a new web service is not rocket science and does not take a lot of time or money. My rule of thumb is that it should take no more than 3 months to go from conception to launch of a new web service. And that's being generous. I'm speaking from experience here. I developed the first version of ONEList over a period of 3 months, and that was while working a full-time job. I developed the first version of Bloglines in 3 months.

In other words: "whatever it is you're doing, you should be able to do it in three months."

Ah, those pesky generalizations--but this is actually an interesting point to bring up. Last year, it took me about 3 months to write the first version of clevercactus share, which didn't just include a website/webservice, but also an identity server, a relay server (to circumvent firewalls) as well as a peer to peer client app that ran on Windows, Mac and Linux.

One person, three months. Webservice, servers, clients, deployment systems, UI/design, architecture, code, even support.

Which proves... absolutely nothing.

You have to fit the strategy to the company and not the other way around. In our case, we're doing something a little different (not better, just different) than the next web service, so we're just trying to keep our heads down until we have something that makes sense.

Of course we want to release as quickly as we can. Of course we know that when we launch there will be dozens of features we wanted to add but didn't have time for. Of course we keep in mind that we can't release a "perfect" product.

We absolutely want to involve users in the product's eventual evolution. We just want to make sure that we have a few things figured out before we start sending out press releases to announce our video-blogging social scooter company.

We appreciate the patience, and the interest (even if in some cases it's a bit misguided!). We are working as hard as we can, as fast as we can, to come up with a good product.

Sounds reasonable? :-)

PS: this may be a good time to add "This is my personal website and blog. The views expressed here are mine alone and do not necessarily reflect those of my employer."

the apple switch

Now that the details are out, it's a good time to really see what's what. There was some feverish speculation over the weekend regarding the switch, what it meant, and whether Apple really was ditching PowerPC or merely going to ask Intel to produce powerPC chips, or something entirely different. The PPC-by-Intel idea never had legs, in my opinion, because even if Apple could take the IP away from IBM/Freescale (Freescale is the Motorola chip spinoff) Intel wouldn't have been able to get up to speed on actually improving those chips better than IBM without, well, buying IBM microelectronics. The problem for Apple with PowerPCs had less to do with production (although that was a factor too) than with power consumption and the roadmap of PowerPCs going forward. And given that Apple represents some estimated 3% of revenues and 2% of profits for IBM Micro, it was hard to see how they could get them to do Apple needed, particularly since the aggregated yearly market for consoles is going to be some ten times bigger than Apple's.

The news "leak" was clearly done in a professional manner, to "soften up the ground" shall we say. It got everyone in the computer world to focus on Apple for days, before and after the event. Masterstroke, really.

This so-called "secret life" of OSX is a bunch of baloney, I've heard for years that OSX ran on Intel, and it absolutely made sense from a technical perspective, because of its Mach microkernel and BSD core. Keep in mind OSX had already been ported out of Moto's 680x0 to PowerPC (from its previous incarnation as Nextstep).

That, and the universal binary idea (ie., single binary for both PPC and Intel) is an example of why Apple can pull this off. Not only they're the only computer company to have survived major transitions in the past, they have time and again delivered environments that executed previous code with fairly good accuracy. And--these days processors ain't what they used to be in terms of lock-in. Linux is proof enough of the underlying shift that has made this possible: higher performance, which leads to most of app's code being written in high level languages (C/C++/Objective C and up) not to mention the rise of Java and scripting languages.

Additionally, the majority of end-users really care about a few things: web, email, document processing, and maybe games. Then there's specialty apps, like Photoshop, but those will get ported no matter what--and many of those already have Windows+Intel versions, so it's not as if the in-house knowledge doesn't exist. Everything else can run on the emulation environment.

And it is the web, particularly, that has really enabled this transition. Even if data isn't going into the web, it is at least going through the web to a large degree. Local datastores can get converted these days between different formats, which are way more open than they were 10 or even 5 years ago.

The web is, to me, the embodiment of the real shift that's happened in the last five years. We have gone from caring about applications to care about data. And it's a good shift. We aren't done with this transition, but things are definitely going in that direction. Data rules, and the device with which you access it matters less. All of which creates more competition, lower prices, and, one can hope, better quality. Hear, hear!

report: apple to release OSX for abacus, paper

from the fake-as-news dept.

CUPERTINO, CA -- On the heels of rumors of Apple's intention to switch from PowerPC to x86 processors, comes the stunning revelation that the company is looking further ahead to make an even bolder move: a change to what Apple "iCEO" Steve Jobs described as "the shift to neural computing."

Apple enthusiasts who on the Internet had described the port to x86 alternatively as "undisputably moronic" and "the greatest thing since the last Finder update" where stunned to the point silence for several seconds when hearing the latest news.

In its most recent quarterly investor newsletter, Apple had cryptically disclosed it had acquired a controlling share of The Friendly Woodchuck Paper Mills, a Tuskegee, OK-based paper production concern. This move had investors and analysts scratching their heads, until this week's disclosure of Apple's skunkworks project dubbed "iWritePad".

"The iWritePad is going to change the way we think about computing," said an Apple official that wished to remain anonymous. "You get a blank slate on which to scrible free form text, maintain ToDo lists, record your thoughts, even create architectural drawings, using the iPen. It will also come with an iAbacus built in the package for quickly performing mathematical operations. Unlimited storage, and all natural product. Acid free, too."

But is this a brilliant move, or folly?

"The next logical step after moving from PowerPC to x86 would appear to be a switch to 680x0," said an Apple spokesman. "But Apple is all about innovation, so we're skipping steps. And those that say that these are moves in the wrong direction just don't know the meaning of the words 'wrong' or 'direction'. Or of 'word' for that matter."

OSX for iWritePad will include an advanced "iInk-based" "icon" "system" to "navigate" between the different "pages" of the "device". While pricing has not been finalized, the plan is to release the product at $299 per pack, which would include one iPen and 50 "letter" sized "pages".

"300 bucks seems a little bit much to charge for pen and paper," A skeptical analyst remarked, to which the Apple official snikered "Does paper come with shiny blue-white headings and a cute little Apple logo? I think not!"

The Apple official also hinted at something even bigger in the pipeline, an update to their fledging digital music product line, and one which has fueled the company's recent profits. Asked to describe this project, the official refused initially but then added. "I have one word for you: Vinyl."

x86, watch out: PowerPC is coming!

In all the hoopla in the last couple of weeks about the new game consoles announced at E3 by Sony (PS3), Microsoft (XBOX 360) and Nintendo (Revolution), one item has received little attention: every one of them runs on PowerPC chips.

Am I the only one that finds this significant? Think about it: every console after 2006 will have a PowerPC chip in it, all of them multi-core. Cell, the processor in the PS3, will have nine processing cores. (Intel is just now getting to release Pentium Extreme Edition with two cores.)

Compared to the PC market, the console market is small (about 15 million units a year I think). But Apple is also running PowerPCs, and with consoles clearly positioning as favorites for the "home media center" title (and Apple expected to do something along those lines with the Mac Mini), this seems like the beginning of an important shift, and I suspect it will become pretty significant over time. Intel in particular has been struggling to extend beyond their core market of PCs, and this is clearly another blow to them, and is under pressure from AMD as well.

IBM! Think back to its position in 1995... the tech industry certainly has surprises in store for everyone. :)

PS: btw, the PS3's specs just blow the XBOX 360 out of the water (just in terms of processing power, it's 2 TFlops against 1, and it has amazing compatibility, integration with PSP, bluetooth, BluRay DVD, and more). Now the question is whether the early launch of XBOX will make up for that or not...

an unexpected upgrade

Something unexpected happened during the time I was away: Eircom has, astonishingly enough, upgraded my DSL connection. I'm now regularly getting download speeds of 1.5 Mbps and sometimes more. The upload speed appears to be stuck at 128Kbs though, which is still a problem for Video/Voice communications.

I still have to check that they are not charging me more too (not likely, but it wouldn't surprise me one bit), but good news anyway. :)

samurize

Samurize, similar to Konfabulator (but only for Windows). Probably well-known to many, but it was news to me when Martin showed it to me last week. Cool!

so that's why I'm not getting notifications...

I was just looking at the release post on movable type 3.16 when I notice this item:

Fixed email notifications for entries and moderated comments which were broken for some users in the last release

D'Oh! And I had happily concocted the theory that SpamAssassin or some other unknown gremlin was to blame. Funny. It never occurred to me that movabletype was to blame. It never even crossed my mind! I wonder why. Not likely that I'll overlook that again though.

Anyway, I just upgraded and it all works again. Cool.

search and stuff

As I'm "walking my way back" into the blogsphere, I've been reading some of the stuff I missed the last few weeks. An interesting one was this post by Mike on the "Recent Innovations in Search and Other Ways of Finding Information" Panel. Very detailed notes, and lots of cool info there.

Ah, search, search, search, search. It's the hot thing again. And I keep wondering what kind of context information we can get automagically, beyond what cool tools like Yahoo's My Web or Google's different take of personalized search provide. And I'm not talking about the semantic web either, rather, the extraction of semantics from what's already out there, no fancy new tags required.

Yep, that's nothing new. I'd bet everybody in search is chasing that eluvise goal. Makes for interesting thinking sessions though. :)

Sixfoo! 660

What social networks would say if they could talk: Sixfoo! 660.

Hilarious.

my stereo system, circa 2005

When I was in the US last I had a 5-disc Technics CD changer, and the main reason I had gotten that was that even then I resisted the idea of having to switch CDs every single time I wanted to listen to something different. So one of the things I got recently at the Apple Store here in Palo Alto is a set of JBL Creature II speakers. Connected semi-permanently to the iPod's dock, I drop the iPod on it when I get home at night and I can listen to any of my music. I know this sounds kind of obvious, but actually doing was a pleasant surprise at first. All my music in a tiny package that can be easily moved around the house (if necessary), or taken on the road. On top of which, the Creature speakers have really excellent sound, and they're not that expensive at around $100. This idea of data portability is partially what I wanted for my dream portable of 2005.

This post reads like an ad. Oh well. Call me a satisfied customer.

PS: Also cool: at night, the green light under the satellites makes them appear to be tiny ghosts, as in old-fashion-bed-linen-covered ghosts, hovering motionless over the carpet. :)

testing y!q "search in context"

I'm going to be testing y!q search for the next few weeks and see how it goes (thanks Jeremy for the invitation). It's an interesting concept---it's been out for a few weeks I think though. You set a few words or a paragraph and you get a search related to the text. The more text you search the less relevant the results get, unless for some reason the text is very focused on a particular topic. In my case, I'm using weblog post titles as context, since I generally have titles that are fairly descriptive. I tried using post summaries but the results weren't as good--what with my tendency to start (and sometimes to continue) off-topic and all that.

Another thing I'm doing this week, when I have some free time :), is looking at the Yahoo! search APIs and related stuff. Should be interesting.

PS: I've also taken a further look at A9's OpenSearch. When I first noted its release I thought that A9 was providing their own results in RSS as well as aggregating others, but that's not the case, or at least I haven't been able to find how to do it. Although the "search exchange standard format" idea is cool, it's a bit weird of A9 to not adhere to their own output standard. Maybe this is something that will be added in the future (one hopes...)

the problem with scoble's linkblog

While I enjoy perusing Scoble's linkblog when I have time (there's pointers to a ton of interesting stuff in there) I have not been so thrilled about his full-republishing technique. In my opinion, the question who exactly created the content is going to be slightly confusing for someone arriving there from a search engine (this in particular for people that don't yet know what blogs are, much less linkblogs).

Even if it was obvious though, the fact that he is republishing articles/posts wholesale without explicit permission means that a reader that would otherwise end up in my blog suddenly has no reason to do so. I have avoided commenting publicly on this, waiting to see if it changed, but it hasn't.

For example, check out his reposting of my take on AJAX. It's a long post (something relatively common for me) and by the time you scroll down to the second paragraph, you have forgotten that URL at the top. Many people will just get to the end, and move on to the next linkblog post.

Republishing content wholesale without permission is a bad idea. And a linkblog is supposed to be made of links, not full posts.

Robert, I suggest you simply post links and titles, rather than full posts---at most, a 50-word snippet or comment would do (similar to what Kottke does for his linkblog posts). If you think that's unreasonable, I'd ask you to remove any posts of mine that you may have republished over there and to avoid republishing other posts in the future. Thanks. :)

opml, icons, and the nooked rss directory

A couple of months ago I was looking for options for identifying OPML output while going through the design stage of the Nooked RSS Directory.

The question was: if the white-on-orange XML icon is a common way to identify RSS feeds, what's the equivalent for OPML?

One answer was to use the same icon, which is in fact generic, to represent OPML as well. This solution, however, can be used when you have either one type of output or the other, but not both at the same time. Another possibility was to use one of the many different icons that turn up in a Google Image search for opml. Another option was of course to create an entirely new icon. The problem was to come up with something that a) wouldn't be unfamiliar to users, b) would be unobtrusive and c) would maintain the value of the XML icon for the RSS feed.

On one hand, the line was crossed long ago when different applications overloaded the XML icon, and even its Look and Feel, while changing its contents (Yahoo! for example has modified the original icon here and they also have a similar-looking icon with "RSS" instead of "XML" to subscribe to Yahoo! Groups, example -- there are many others, I'm not singling out Yahoo! in particular). On the other hand, there's no need to create even more confusion. Dave has advocated the use of the XML icon for the appropriate XML output for a page (be that RSS, OPML, etc), and, maintaining the value of the icon by avoiding changing its contents while keeping the look and feel intact. After some thought, I concluded that in this case the second approach made more sense: barring a particular design or business need (for which there are many good examples, but that weren't a factor in this case) simplicity was the best option. Additionally, avoiding possible user confusion that would result from a non-standard icon is definitely a good thing.

So what I ended up doing was to maintain the common XML icon to point to an RSS feed (the most accepted use of it by far) and to link to the OPML using a simple text link, enough to be unobtrusive while remaining usable for those that know what they're looking for (after all, not everyone knows what "OPML" is). As it turned out this was a fourth option: have no icon at all.

For example, check out the directory's Arts & Humanities page. The same page can be viewed as OPML and as RSS. (The link to the OPML view is still there, to the right --- Nooked replaced the link that directs to the RSS view with a small ad after I delivered the app, so essentially imagine that the white-on-orange XML icon is where the Nooked ad is).

Interestingly, this solution was also pretty good in terms of matching the functionality needs that we had (unobtrusive and at the same time easy to identify for experienced users).

Related: there's some cool stuff that can be done by using the OPML output as a start, for example, reading the OPML of feeds listed in a category and presenting a full RSS data view of them (different from showing the entries of the directory in RSS, which the directory does) with little coding. Then the loop could go on by creating OPML views of the category feeds, and so on... interesting possibilities for meta-aggregation and possibly other kinds of data processing/analysis.

gumstix

I have absolutely no idea what I'd do with a bunch of Gumstix Waysmall systems, but, like Cringely, I keep thinking that there's something cool that one should be able to get them to do. A mini-cluster, connected via bluetooth! Right, but for what exactly? Anyway. Certainly something to keep in mind for when I need a swarm of tiny computers to do something. :)

(un)structured

Matt points to a new wordpress plugin for structured blogging. Kewl. Reminds me of.

Then, completely unrelated to this, but way more fun is Ted's pointer (originally from Boing Boing) to LoTR in l33t:

**The white light fades, revealing Gandalf
Aragorn: "WTFOMGBSHAX!"
Gandalf: "Teh balrog was noob. He got pwned. I can dance all day, I can dance all day!"
Gandalf: "Boom! HEADSHOT!"
Gandalf: "I R GANDALF TEH GREY!"
Gandalf: "I R GANDALF TEH WHITE!"
Gimli: "FFS, make up your mind! ><"

To which one can only say: LOL!

Then we've got Andrew Sullivan who thinks that society is dead. Why? Because people in Manhattan listen to music on iPods rather than to the noise of traffic and such. Right. Lighten up, dude. Music players have sold, since inception, maybe, what, 100, 200 million units? There are about 6.3 billion people in the world without music players. Maybe society isn't dead quite yet? More than a billion of them don't even have running water, but they haven't "fallen under the spell of iPods" either. So it ain't all bad...

iPods and blogs. Now, that is something that should keep you up at night.

Woody Allen comes to mind: "What if everything is an illusion and nothing exists? In that case, I definitely overpaid for my carpet."

Also, Adam Curry: "Veni, Vidi, Velcro."

While working, I've spent half a day listening to U2, the other half to Sinatra. Both rock, even though Sinatra didn't. Language is funny that way. If I was really feeling recursive, I'd say funny, not ha-ha funny (like this Bono vs. Carly post is), which in itself is funny, but not...

Well, what do you know, I guess I am feeling recursive.

trackety track

[via Mitch]: Tracking PCs everywhere on the Net (paper):

The technique works by "exploiting small, microscopic deviations in device hardware: clock skews." In practice, Kohno's paper says, his techniques "exploit the fact that most modern TCP stacks implement the TCP timestamps option from RFC 1323 whereby, for performance purposes, each party in a TCP flow includes information about its perception of time in each outgoing packet. A fingerprinter can use the information contained within the TCP headers to estimate a device's clock skew and thereby fingerprint a physical device."

Kohno goes on to say: "Our techniques report consistent measurements when the measurer is thousands of miles, multiple hops, and tens of milliseconds away from the fingerprinted device, and when the fingerprinted device is connected to the Internet from different locations and via different access technologies. Further, one can apply our passive and semi-passive techniques when the fingerprinted device is behind a NAT or firewall."

And the paper stresses that "For all our methods, we stress that the fingerprinter does not require any modification to or cooperation from the fingerprintee." Kohno and his team tested their techniques on many operating systems, including Windows XP and 2000, Mac OS X Panther, Red Hat and Debian Linux, FreeBSD, OpenBSD and even Windows for Pocket PCs 2002.

"In all cases," the paper says, "we found that we could use at least one of our techniques to estimate clock skews on the machines and that we required only a small amount of data, although the exact data requirements depended on the operating system in question."

Wow.

the nooked rss directory

So, last week Nooked publicly released the application that I wrote for them during late December and January: The Nooked RSS Directory.

The directory is an interesting app: basically a resource of corporate RSS feeds (including Podcasts). Data is still being added, so there are some categories that don't yet contain many entries. One of the points of this particular app is that there is "editorial control" (similar to the Yahoo! directory) and so a level of relevance should me maintained throughout the entries.

While the nature of blogs (and the web itself) makes centralization difficult to maintain, directories are of course useful resources, and in this case, for many people that are only now approaching RSS from a corporate communications point of view, the directory would be a useful resource to get started and find information on the companies or products that they are interested in, and they could submit their own feeds.

It took a bit longer that originally planned, but it's great to finally see it deployed. The OPML view of the listings has some interesting consequences--but that's for later! :)

outlets and high speed connections

One thing that I had forgotten (or probably repressed :)) about the US: there are so many outlets in apartments! Outlets everywhere, and lots of them. I may still need expansion cords, but not so far. It's great! Such a difference from my outlet-challenged apartment in Ireland.

Another thing: happiness is a 4 Mbps connection at home. I can't say I miss my wimpy 512/128 kbps DSL in Ireland. At all.

Also, I had Internet connection in the apartment before I had phone service. An Ethernet port in the bedroom and another one in the living room.

If that's not a Silicon Valley experience, I don't know what is. :)

how to reboot an iPod photo

One of the things that happened while traveling was that about an hour into the LHR-SFO flight my iPod photo went nuts and started just to skip all songs without playing them. I came to the conclusion that it needed a reboot, but not having a notebook with me (I'm just carrying my data on a portable hard drive) I couldn't connect it and reboot it that way. I assumed (correctly as it turns out) that there should be a way to reboot it by using some key combination, but I didn't bring the iPod manual (such as it is) with me. I did have iPod: the missing manual but that was on one of the suitcases and so out of reach.

I tried a lot of options, mostly involving key combinations, but couldn't make it happen. Once I got here, I was able to check the book and the instructions to reboot both regular iPods and iPod minis were there, but, alas, no instructions to reboot an iPod photo. I decided to try the iPod mini way first (since it's a newer product) and it worked.

This is probably common knowledge to most long-time iPodders, but it was news to me. :)

Here's how to do it: put the Hold switch in ON, then back to OFF. Then press MENU and SELECT simultaneously and hold for a bit, and on release the iPod should reboot. According to the missing manual it is possible that the iPod is misbehaving due to battery charge confusion (not sure how that would happen, but anyway...) and for that you can plug it to the charger so that you can rule out that problem.

Speaking of iPod: did you know that European iPods can't play as loudly as American ones? Yep, the EU enforces "stricter" standard for maximum volumes in portable devices. Apparently, people aren't smart enough to know when they are blowing their ear drums off, or when they can't hear anything else except the music, and assumming they did want to do that, for some bizarre reason, then they shouldn't be allowed.

Now that is government regulation run amok.

today's (re-)readings

Some articles I've been re-reading, in no particular order:

• Macintosh Production Introduction Plan, part of Stanford's Making the Macintosh: Technology and Culture in Silicon Valley
• The Anti-Mac Interface and Towards the Anti-Mac
• Apophenia: Revenge of the User
• Public-key support for Collaborative Groups (Paper+Presentation)

If you're wondering why these articles, it's simple: as I've been, in the last few days, trying to impose some order on the unremitting chaos that is my apartment, I've found articles, books, and references that I had read long ago and "shelved" somewhere. Some of these, as I find them, I set aside and read again. Hence the random nature of the list. :)

a tool's influence

I keep seeing flashes of text, sentence-bursts that I have the impulse to blog and then, inevitably, evaporate in a "nah, too short." I have no doubt that in my case not just the tool (movabletype) but also my weblog's design have influenced me to go for longer, article-like posts rather than the shorter posts we see elsewhere. Similarly, weblogs based on other tools (most notably Radio) tend to be more of a mix of snippets and medium-length posts with the occassional medium length article and only every once in a while a really long post...

So what I've been wondering is, how to merge these two? Not just in terms of the tool (currently no tool seamlessly moves from one end of the spectrum to the next, at best, some provide plugins to deal with linkblogs, etc, and Radio has both "posts" and "stories" but the granularity isn't enough and stories are outside of the normal flow of the weblog). Also, in terms of the UI, navigation, and content evolution. A newspaper view comes to mind.

I've been looking for something entertaining to do with PHP. Maybe this is it.

tag surfin'!

Yesterday Russ "officially" announced a service that he and Anthony are developing, called tagsurf, a "tagging-enabled hyperforum." Russ showed me one of the very first revs when I was visiting last month. It is a very cool idea: create loosely-coupled threads (or "forums"), where the coupling is determined by tags, the new new thing that's been making the rounds for the last few weeks in the blogsphere (this Salon article has a good summary on tags). Because you can attach any number of tags to a message, you are actually enabling multi-dimensional browsing on topics, for very little extra effort. (MetaFilter started doing something similar in January).

Anyway, go and give it a try. And congrats on the release guys!

the new toy

Thanks Feedster!!!!

PS: Funnily enough, I got the iPod Photo exactly one month ago, on January 9th. I should probably decree the 9th to be "iPod day" on d2r or something. :)

so THAT'S why!

I had a sequence of reactions to this article in the New York Times which deals with Google's recent approval as an Internet registrar.

First, the obligatory rolling of eyes at people "wondering" and "worrying" and "speculating" about every single move of Google. Let the guys do their work in peace ok? If it's world domination, then fine, let them try.

Second, was a raised eyebrow as I read the following:

Eileen Rodriguez, a Google spokeswoman, hardly quelled the speculation by explaining that the whole thing was really a learning opportunity for the company.

To this, I followed with an interested (muttered) "A-ha". But then the actual quote went on:

Google "has become a domain name registrar to learn more about the Internet's domain name system," she said recently in an e-mail message. "While we have no plans to register domains at this time, we believe this information can help us increase the quality of our search results."

To this, I just laughed out loud, quite literally, for several seconds. I still giggle at it every time I read it.

Come on boys! That's the line you feed to your spokespeople? "We became a domain name registrar to learn more about the Internet's domain name system"?!?!?? Yes!

LOL LOL ROFL!

Maybe this is practice, you know.

When Google announces they are releasing, say, a web browser, they will say "We are not interested in the browser business, this is just to learn more about web browsers and stuff."

Then, if they want to release an Operating System: "We are not interested in OSes. This is just to learn more about computers. And stuff."

And Larry or Sergey will show up and whisper something in the ear of whoever is giving the interview, and then they will add: "Right. To improve search results too. You know?"

Cue Mr. Burns: "And remember... a shiny new donkey for whoever brings me the head of Colonel Montoya."

And still laughing... :)

PS: Before anyone starts "explaining". Yes, I know that some of the info available only to registrars, and that can help, etc, etc. I just find the spokeperson's choice of words quite hilarious (aside from imagining the innocent look on their eye while they essentially say, straight-faced "Thinking of a product that is not pure search engine? Us? Noooooooooooo!").

Don't you? :)

microsoft natural keyboards: evolution or devolution?

I am, like many of us, very specific about certain things in my work environment, and there's stuff that I have "requested" (read: demanded!) everywhere I've been to in the last few years. One of those things is a Microsoft Mouse. The other is a Microsoft Natural Keyboard.

Yep, that's two Microsoft products I actually like a lot (the other main one is MS Bookshelf, which sadly got discontinued in 2000 and got swallowed by Encarta, which is ok but too big for basic dictionary/thesaurus needs).

I got a Natural Keyboard when it was first released (it was one of the first USB keyboards) and then got a new one in '99 when MS released the Microsoft Natural Keyboard Pro. That very keyboard has been with me since then in every machine I have at home, crossed oceans back and forth and been in as many cities as I've been, and it's been a rock. Nice feel, good construction and key placement. I particularly like that the keyboard is huge, and since I have big hands I feel comfortable with it.

I have yet to meet another person that feels as comfortable as I do with this keyboard, I wonder if I'm the only one buying them. :) But my problem is that in recent years, as the new Wireless versions have appeared, Microsoft has been "compressing" the width of the keyboard, with the result that even the Pro version compresses the Insert/Delete/Home/End/PgUp/PgDn key set from a 3x2 configuration to a 2/1/2 configuration (the insert key is gone, the alignment is vertical instead of horizontal, and the delete key is two keys tall).

What truly drives me crazy about this is that there is no option to get the old keyboard layout (and don't even get me started on the crosshairs cursor keys of the Natural Keyboard Elite! For a while it seemed that every new keyboard came with a different key layout). You either get the Natural Keyboard layout with vertical keys over the cursors (and compressed design) or a regular keyboard layout, with more space.

Also, every new version keeps adding weird function keys, integration and whatnot. It's starting to become a problem, like the million features in Microsoft Word that no one uses. Wake up MS! You had a great product with the basic Natural Keyboard and regular keyboards, along with Intellimouse. Stop adding buttons and lights and gizmos. Just make a good, simple keyboard. Notice how Apple keeps going for simplicity? Try that for a change!

So last night I "retired" my old set of Keyboard/Mouse and now I've got a new one, the Wireless Optical Desktop Pro, which is as close as it gets to the old version. It's the Natural Keyboard, plus the Intellimouse explorer, but alas bluetooth versions don't yet come in the "Natural" split-keyboard format.

Here's hoping that when they do, Microsoft will restore the standard Insert/Delete/Home/End/PgUp/PgDn 3x2 key arrangement over the cursor keys. And enlarge it a little bit. And remove some (just a few) of the crazy functions they keep adding to the keys. That, and I'd be happy.

PS: Since I'm asking... also, there's a trend of designing Mice so that they adapt ergonomically to the hand. This is fine, but it becomes a problem for people like me, who frequently switch the mouse hand throughout the work day. I'll probably have to end up working with two mice at once. :)

russ @ yahoo! v2

Russ is now full-time at Yahoo! Congrats!

what I'll get... right after I get a powerbook

The iLap. Not expensive, and very cool (literally and figuratively). Even if I don't get a powerbook soon, I'd still want one of these for the Thinkpad, assuming that it works fine with it that is. [via Tom Yager].

funny stuff

SF Gate: Why does Windows still suck? or "how I learned to stop worrying and appreciate the 4 minutes my windows computer works without being infected". (I found this through slashdot, but I'm too lazy to look for the link).

The Onion: Google in 2005, top secret plans from the googleplex for this year, including "Enter beta testing of Google Apartment, which will let users search for shoes, wallets and keys", "Patent the idea of looking for something" and "Occasionally shut down so people stop taking them for granted."

Hee-larious. :)

firefox weirdness

I've been using FireFox for quite a while now, but suddenly something strange started happening: if I'm editing something on the browser (say, a post), and I do a "preview" I used to be able to press the back button and continue editing. Now, though, when I press the back button the form contents are wiped out. I can press "forward" though and the contents are re-posted, so they're still in there somewhere. Strange. I'm pretty sure I didn't change any settings on the browser. I wonder why this started happening...

the arrival of trackback spam

At least in this weblog. This morning there were some 50 spam trackbacks to different entries.

I've been waiting for this to happen--until today trackback had never been abused massively. But it was clearly just a matter of time, particularly since trackback allows to set snippets and in many weblogs they are rolled in with the rest of the comments.

Conclusion: I'll do what I did for comments: change the 'allow trackbacks' flag. Luckily I switched to MySQL not long ago, making it easier to access the raw data, since Movable Type still doesn't support a "close all comments & trackbacks in entries after this date" feature (and for me using SQL is easier than using a plug in).

Anyway. Another line crossed...

Update: Very strange. A couple of hours after closing the trackbacks and rebuilding the weblog, spam restarted. I verified that the entries had trackbacks closed, and yet spammers were able to post trackbacks anyway. I tested sending a trackback myself to an entry which was closed, and correctly got a message "Ping 'ENTRYID' failed: This TrackBack item is disabled." The trackback was not received. I have to assume that they have found a way to post trackbacks even if they are closed... (some unknown hole in MT's trackback implementation? Or maybe the additions were in a queue somewhere and got in anyway, since they were so many?). As a temporary measure, I've changed the name of the trackback script, so they shouldn't be able to post to the URL they have crawled.

Another update: Definitely some form of queueing was at play. I have done some more experiments and looked at my logs and enabling/disabling the trackback script returns the spammer (which is still going at it) a 404 and a 500 HTTP Error alternatively, so the check is working. Leaving the old script disabled is better, obviously, since it doesn't hit the MT db for checks.

another microsoft gem

I read this first over at the Wall Street Journal but then found a CNET article on it.

Last year Microsoft actually lost a court case, against the EU, for predatory practices (sounds like something from the Discovery Channel, doesn't it?) in the market for media players/formats. Essentially the EU forced Microsoft to distribute Windows without forcing Windows Media Player to be bundled with it, essentially allowing customers to choose.

So what did MS do? It said "fine". Then it went and called this product "Windows XP Reduced Media Edition". Then it also removed the ability to do some basic things, such as playing music CDs (something that even Windows 95 could do).

If you are a consumer, and are offered, for the same price, a machine with "Windows XP Home Edition" and another with "Windows XP Reduced Media Edition", which one are you gonna get? I can hear the Shadow-like laugh in the background when the MS representative says (in the article) "We believe this name complies with the commission's orders".

You know, it's stuff like this that pisses people off about Microsoft. Maybe you could argue that it's MS's "Darwinian" attitude that allows it to post record profits and market gains, even when most in the tech industry consider it passé.

But isn't there a point when MS just has to say, "okay guys, we've misbehaved, how do we make things better?" instead of fighting tooth and nail for every single tiny scrap of whatever. I have stated more than once here that Microsoft's desktop monopoly is not necessarily illegal in itself, it's the predatory practices and the illegal actions taken to maintain and defend that monopoly that are the problem (sadly, it's also in question whether the monopology would have gotten this far without those illegal actions). Maybe if MS started to differentiate the monopoly itself and what they do to maintain it, then things would get better, don't you think?

No, I'm not holding my breath. But hey, it's Saturday morning. I thought it was Friday. Let's give the lad some leeway.

Bonus: a post from November 2003, where I wonder if WinFS will be delayed again ("never say never" eh?), some more day-dreaming about MS embracing, or at least not actively attacking, the Web (sure), and some Descartes thrown in for good measure.

spamassassin and comment spam

I have the blog set up (as most others I suspect) to email me when a new comment is pending for moderation. Main reason for this being, of course, comment spam. Now I get lots of comment spam, but it's easy to ignore.

But I also have spamassassin set up to dump any email that scores 5 or higher (I think). And SA learns. Put two and two together and it turns out that after a while SA apparently came to the conclusion that any email that came from comments on the weblog was spam. Now I don't get notified of any comments at all. I'll have to dig into the SA config and figure out which rule is triggering this, or how to override it. I'm not even sure how it's separating them, since the trackback notifications are still arriving, and they have a similar message pattern.

You know, I'd be pissed at SA if it wasn't that it's pretty much on the mark in its decision. Hopefully eventually we'll have SA connected to something like Cyc so that it knows a bit more about the world. Now wouldn't that be a treat.

old shoe, new shoe

For some reason I just remembered an exchange between Dennis Leary and Willie Nelson in "Wag the Dog" where Dennis Leary's character utters those words. That's how random my brain is.

Dylan, a good friend whom I stayed with in California (the first few days I was there at the beginning of January), has retired his weblog. He has a number of excuses (I mean, reasons!), which I want to ignore and hope he'll get back to blogging at some point, even if haphazardly. Dylan's foray into blogging is one of the reasons why I'm blogging, so at the very least, here's a digital Guinness raised in Warmbrain's honor. :)

On the new shoe front, Martin, another friend, has finally acquiesced to my constant babbling about blogs (and my usual "start a weblog! now!") and started a weblog :). He's one of the best developers I know, so he's sure to post some interesting stuff. Check out his wide range of interests (and knowledge) in some of his initial posts, from Keystroke emulation in Win32, to Java Swing Tips to Stack trace annotations. If my brain wasn't fried (18 hour day so far and all) I'd make some interesting comments on those, but for the moment I'll just link to them. Welcome Martin! :)

ipod photo: first impressions

Okay, so I was going to do this all at once but I realized that it would end up being an impossibly long post. Better to split things up. This is stuff that I wrote down as I started using it, about a week ago.

but why oh why?

While you'd think this question has an obvious answer (e.g., hype, the "cool" factor, "just because"...), in my case it's not so obvious. You see, I generally prefer solid-state electronics over anything with moving parts, and that meant that for a while I was skeptical of an iPod for myself (even as I knew the value they had for most others). It's one of those techno-snobbish things.

In any case, what happened is that I realized that over time I started to view my digital music collection as one "set" or "unit" rather than as disjointed playlists. No doubt iTunes has a big role to play in that, but it's a natural evolution IMO. Once you have 20 gig of music on your machine you get used to selecting anything, anytime. And once you're used to that, it's pretty hard to use puny 1/2 gig flash players or whatever: you never know what to choose "for the road". My first MP3 player was the original Rio 300, then a Rio 600, then a Rio 800. I knew the flash-player use case, and in the last couple of months it became obvious that it just wouldn't cut it anymore.

In fact, not even the iPod mini (or similar) would cut it. I wanted a player that would let me take all my music with me. So last week as I walked around the Apple Store in Palo Alto, I kept drifting back to the little pile of iPod boxes in one corner. I knew that I was getting an iPod mini from Feedster soon, but I thought that they would be useful in different cases.

Eventually, I gave in. :)

Package, installation

Everyone gives points to Apple for design, but not many comment on the packaging, which I always think is an integral part of the "experience". The iPod is no exception. The little cubic box opens like a book in half and for some reason by the time you get to pulling cables out of it, you're already smiling. It's design genius.

The package includes pretty much everything you need to get started, and the included carrying case is good, but not great since you can't use the controls with the iPod in it. But still useful when the playlist has been set, or if you get a remote. As far as accessories, the iTrip (FM radio broadcast direct from the iPod) looks pretty sweet, but I didn't get it. :) Also the docks with built-in speakers.

Set up is pretty easy--the only wrinkle is that it piles on a couple of installations of different things (iTunes, Quicktime, iPod updater) and it's less simple than it should be, but still manageable. The fact that iTunes still has to be installed separately is kind of a pain, even afterwards updating iTunes is still a full download and install. Maybe it's time for Apple to come up with a Windows version of OS X's Software Update feature?

Taking it for a spin

Anyway, you're up and running fairly quickly. Copying songs over USB isn't bad (and USB2 is great, I haven't been able to try FireWire but I have no doubt it will also work well). In the case of plain-old-USB, you just have to wait a couple of hours until all those GB are transfered.

One slight issue with the dock (which comes bundled in the iPod photo, I think other models don't include it, but I might be wrong) is that while it's docked you have this message on screen of "do not unplug" or something. Which is a problem, because you don't really think about looking at the iPod's screen before pulling it out of the dock. I think that this doesn't necessarily create problems, but it might when a transfer hasn't finished... maybe better sync in the software (so that the device is released as soon as possible) and some notification at the OS level for moments when you absolutely-positively-can't-unplug it would work.

The headphones are fantastic. I am not a fan of earbuds, but the iPod's are really comfortable and sound incredibly good. Apparently the drivers are built out of Neodymium, as opposed to the more common aluminum, cobalt, or ceramic. I definitely think that there's a difference. In any case, difference or not, they sound great, and they're the first earbud earphones in years that I've used for more than a few hours without them bothering me.

Battery life is pretty good, I've definitely gotten over 10 hours but I'm not sure if I reached the advertised 15 for this model. This is also hard to gauge since battery usage varies wildly with use (ie., times the hard drive has to spin up, backlight on, etc). But I definitely had no problems with it this monday/tuesday with a nearly 20-hour trip. The battery was almost exhausted at the end, but it still had some juice.

So far so good

So, quick conclusion: it's a pretty great product. Already I've listened to stuff that I hadn't listened to in a while, just because the opportunity doesn't present itself (or rather, when it does, you're nowhere near the music you'd like to hear). I gave the output-to-TV feature (with included AV cable) a quick try and it's great, but that, and the photos (Along with the extras, calendar, notes, games, etc. :)), is something for next time.

happy birthday russ!

Today is Russ's birthday! Go send good wishes to his weblog. :)

iCaved

Yes. Yes I did. Review upcoming. :)

new design

I have succumbed to the temptation! Yes! A few weeks ago I spent some time playing around with new designs, but none of them really convinced me. I was looking at them again this morning, though, and I realized that one of them wasn't too bad, and it could be a good starting point, considering also that the previous design was nearly a year old. So here it is... let's see how it works out. I still have some pages to update, particularly for the individual entries.

The new design is not that different really, mostly it reduces clutter and organizes things a bit (and gets rid of time-dependent elements in individual entries). One "feature" I like about it is that while it uses CSS, there is some structure maintained through tables (which the CSS kinda overrides). Purists will probably scoff at this, but I did it to maintain the design parameters when looking at the site with the venerable Lynx and older browsers. Lynx-compatibility is a crucial feature! Heh.

PS: you might need to do a hard-refresh in your browser to reload the CSS. Let me know if you see problems in a particular platform. Thanks!

Later: Holy Cow! Now that each individual page doesn't include links to archives and such, the rebuild process is about two orders of magnitude faster. It's now rebuilding some 20 pages per second, whereas before it used to take 5 seconds per page (Rebuild of a single entry on its own takes longer because it also rebuilds all associated indexes, but it's also much faster!). I've just rebuilt the entire site in like 4 minutes! That was all I had to do to get it to run faster? This makes me think that maybe the default entry/daily/archive templates for MT should not include so many dynamically generated links. Another lesson there somewhere...

my dream portable for 2005

Okay, since I'm tired of waiting for someone at some PC manufacturer to come up with what I really want, and since they don't appear to have developed mind-reading yet, I thought I'd spec it out here, including some quick sketches :) (click on the images to see a larger version).

I often think about what I'd really like in a portable and surprisingly my thoughts have been pretty consistent of late. Before you start saying that this already exists, read through the specs--the devil is in the details!

The basics

To start, I don't want anything that doesn't exist today. It's all a matter of packaging and connectivity. Fairly obvious things that my dream portable should have are:

• Decent processor, 1 Gig of RAM.
• (Relatively) small built-in hard disk. A 40 Gig 2 inch platter (like what the iPod has in its drive) will do.
• High-res TFT (Widescreen would be nice but not necessary).
• WiFi, Bluetooth, Ethernet, a couple of USB2 connectors, and a couple of FireWire connectors, although I'd be happy with at least one of each of those. Throwing in GSM and CDMA chips for cell connectivity would be a plus. :)
• Touch sensitive screen (ie., Tablet mode, including display rotation).
• 2.5 pound basic package, without keyboard, 3 pound package with keyboard, and 4 pound max with the storage & adaptor (I'll get to what I mean by "storage" in a bit). Keyboard should be separate from the machine, connected to it via bluetooth, with built-in trackpoint or similar.

Some issues with this machine is that as you get lighter you need more on the base, which is why in my design the base flips and locks with the hinge to provide support for the screen.

So far I imagine few people would disagree with me, except maybe on the size of the built-in disk. Ah, but here's where what I want differs from what's out there. I separate between base storage and personal storage. Let me explain.

The Key: Storage

Base Storage is what's required to run the OS and applications. It's not your data, it's something that is machine-dependent (mostly) and relatively stable. For this, 40 gigabytes is pretty good for today's needs.

Personal Storage is for my data. These days, when we're running around with tens of gigabytes of MP3s, videos, and such, not to mention the rest of the stuff (my personal data store, not including media, is about 4 gigs), we need a lot more than what portables require, additionally, synchronization between our devices is a nightmare. Data ends up duplicated for no reason. So what I think is that we should decouple our storage from everything else, and that's where things get interesting.

This personal storage device would be reasonably big, 250 gigabytes minimum, maybe reaching into a terabyte by the end of the year on single platter drives (on multiple 3.5 inch platters we're probably there by now). Yes, I know that the idea of "Brick PCs" has been floated in the past, but I don't want a PC on a brick. I just want a drive on a brick.

More on the 'Storage Brick'

So were does the personal storage go in my dream laptop? Look at the back view:

The personal storage unit "docks" into the back. That is, instead of docking the machine into an expansion unit, just dock the drive into the machine!

This approach has several advantages. First, I should be able to selectively synchronize some important data into the permanent storage of the portable, for when portability is more important and I don't want to carry the storage brick around. Second, I should be able to dock the storage into my desktop PC, making it easy to move between machines. Third, using the PC's larger internal storage to automatically sync (i.e., backup) my personal storage brick with the local drive we'd get automatic backups! Yay! :)

Also, by splitting up the machine into more parts, you get more choice in portability. Need to minimize weight? Just take the screen and base unit and use it in tablet mode. Need more space? Plug in the brick. Easy.

What's crucial here is not the idea of portable drives, but the simplicity of "docking the drive" into the portable and into the desktop PC. Alternatively, instead of docking you could do a USB2 or FireWire connection, and handle the integration in software.

If we lived in a perfect world, we should be able to just buy a storage brick from one manufacturer, the base from another, the keyboard from someone else... but that's for later. All the technology required for this already exists. All that's required is the integration work, and, yes, the sync software for the drive dock would be a sticky point, but certainly nothing insurmountable.

Some day...

how microsoft will take over the world (according to Cringely)

A couple of weeks ago Cringely was more locuacious than usual with his article "Between an xBox and a Hard Place." He is generally entertaining and well-informed, but sometimes his machinations show an excessive fear of Microsoft. Example:

Take a long look at xBox development, the evolving PC and consumer electronics markets, and Microsoft's own need for revenue growth, and figure what that means for the xBox 3, which should appear around the end of this decade. My analysis suggests that xBox 3 will be a game system that's also a media receiver and recorder and a desktop workstation. Not that you'd use one box for all three things, but that you'd buy three essentially identical boxes and use them for all three functions. And of course you'd buy extra units for kids and spare TVs, etc. In short, xBox 3 will be Microsoft's effort to extend its dominance of the PC software industry into dominance of the PC hardware, game, and electronic entertainment industries. At that point, even mighty Dell goes down.

While there is no question that xBox3 might be that and more, and that the IBM/Lenovo deal elements he discusses are interesting, the problem with his logic is that it assumes that nobody else does anything. "At that point, even mighty Dell goes down," because, you know, Dell will stand still for five years as Microsoft supposedly prepares to anihilate its business. So will anyone else, btw, including HP.

Right.

Never mind as well that the industry is becoming much more complex, with different products and price points, multiple devices (smartphones anyone?) so that the battle for xBox and PlayStation might be more of a sideshow than anyone anticipated. After all, my phone is with me 24 hours a day, while I might be in my living room only ten percent or less of that time. And a place in which, I might add, the TiVos of the world will be comfortably settled in by 2010. The problem with multi-function devices like that is that they have "multi-competition" too.

Or maybe someone will come out with a WiFi transmitter that can connect directly to the TV from the PC and bingo, you don't need to move the PC below the TV and paint it black, just leave the TV where it is, the PC where it is, and connect them, and everything else, wirelessly.

Oh, but wait, we're already doing that! And I think that Microsoft does understand it, certainly at some levels. So how much of the possible xBox 3 grand plans is just keeping up? Especially since we may not yet have WinFS by then...

Nevertheless: entertaining. :)

ipod!

Btw, I forgot to mention this (or rather, I've been sort of 'off the grid' for a couple of days) but I won one of the iPods in the Feedster developer contest!

Many thanks to Feedster, and Scott. Very cool!

a manifold kind of day

I'm about to go out for a bit, but before that, here's what I've been talking about for days now: the manifold weblog. (About my thesis work). The PDF of the dissertation is posted there (finally!), as well as two new posts that explain some more things:

• the manifold algorithms, and
• a bit more on manifold-g

as well as the older posts on the topic, plus a presentation and some code in the last one. Keep in mind that this is research work. Hope it's interesting! And that it doesn't instantaneously induce heavy sleep. :)

a subtle problem of frontpage

As an aside, I spent some time in the last couple of days doing a favor to someone who had created a website but wanted to make it look reasonably good.

Since the site had been created with Frontpage, I had to go through the usual rigamarole of removing the extraordinary amounts of garbage that Frontpage inserts into the HTML. This was problem number one.

The 'subtle' problem though, was the UI. In the process of changing the site I of course redesigned the navigation, but I realized that Frontpage was actually doing something pretty terrible: creating a bad UI.

Frontpage automatically manages the creation and maintenance of navigation on a site. You can create the site hierarchy and FP will maintain links, etc. The problem is that the UI that FP generates is hierarchical, and it doesn't really do justice to the multidimensional nature of hypertext. It is, pretty much, a directory in HTML form. In many default FP templates, sub-pages are generated with "Up" navigation links along with the rest, which is not only ridiculous with HTML but also bad UI practice because the navigation bar changes content for every page you're in.

So my question is: can't Microsoft fix Frontpage so that it a) generates simple, CSS-based HTML and b) that the default templates include well-designed hypertextual UIs, rather than what it does today?

Or does Microsoft need a Firefox HTML Editing app that will wake up the Frontpage team, just as Firefox itself has resucitated the IE team?

PS: Frontpage is actually a product that Microsoft acquired in 1996 when they bought a company called Vermeer Technologies. The founder of Vermeer, Charles Ferguson, wrote a book about his experience, from founding to acquisition, called High Stakes, No Prisoners, which is fantastic. Recommended.

under attack

Through the last week the clevercactus site has been sporadically unavailable, and it's down right now. This means no web, no service, no emails getting through.

If you're trying to get through to clevercactus and can't please let me know through a comment or email to my personal address.

What happened is that we were attacked (I'm not sure when) and someone left a number of scripts there that are flooding the system (they do other things too, but at least one of them is clearly written simply to flood the network and disable it). This is something obviously intended to bring down clevercactus, not just a simple hacking. Why? What do they gain by bringing down the service of a small company that is going through hard times?

This kind of thing makes me sad, and is really discouraging.

I had this whole thing planned for today, getting the manifold site up and so on but now I'm going to spend time trying to see how to route around the problem for now until we can determine the extent of the hack. I don't even know how they got in yet--we constantly update our software with the latest patches. Needless to say, I'm seriously reconsidering the whole of the software I use and how to set it up so that this doesn't happen again.

Anyway. We'll see how it goes.

manifold, the 30,000 ft. view

As a follow-up to my thesis abstract, I wanted to add a sort of introduction-ish post to explain a couple of things in more detail. People have asked for the PDF of the thesis, which I haven't published yet, for a simple reason: everything is ready, everything's approved, and I have four copies nicely bound (two to submit to TCD) but... there's a signature missing somewhere in one of the documents, and they're trying to fix that. Bureaucracy. Yikes. Hopefully that will be fixed by next week. When that is done, right after I've submitted it, I'll post it here (or, more likely, I'll create a site for it... I want to maintain some coherency on the posts and here it gets mixed up with everything else).

Anyway, I was saying. Here's a short intro.

Resource Location, Resource Discovery

In essence, Resource Location creates a level of indirection, and therefore a decoupling, between a resource (which can be a person, a machine, a software services or agents, etc.) and its location. This decoupling can then be used for various things: mapping human-readable names to machine names, obtaining related information, autoconfiguration, supporting mobility, load balancing, etc.

Resource discovery, on the other hand, facilitates search for resources that match certain characteristics, allowing then to perform a location request or to use the resulting data set directly.

The canonical example of Resource Location is DNS, while Resource Discovery is what we do with search engines. Sometimes, Resource Discovery will involve a Location step afterwards. Web search is an example of this as well. Other times, discovery on its own will give you what you need, particularly if the result of the query contains enough metadata and what you're looking for is related information.

RLD always involves search, but the lines seemed a bit blurry. When was something one and not the other? What defines it? My answer was to look at usage patterns.

It's all about the user

It's the user's needs that determine what will be used, how. The user isn't necessarily a person: more often than not, RLD happens between systems, at the lower levels of applications. So, I settled on the usage patterns according to two main categories: locality of the (local/global) search, and whether the search was exact or inexact. I use the term "search" as an abstract action, the action of locating something. "Finding a book I might like to read" and "Finding my copy of Neuromancer among my books" and "Finding reviews of a book on the web" are all examples of search as I'm using it here.

Local/Global, defining at a high level the "depth" that the search will have. This means, for the current search action, the context of the user in relation to what they are trying to find.

Exact/Inexact, defining the "fuziness" of the search. Inexact searches will generally return one or more matches; Exact searches identify a single, unique, item or set.

These categories combined define four main types of RLD.

Examples: DNS is Global/Exact. Google is Global/Inexact. Looking up my own printer on the network is Local/Exact. Looking up any available printer on the network is Local/Inexact.

Now, none of these concepts will come as a shock to anybody. But writing them down, clearly identifying them, was useful to define what I was after, served as a way to categorize when a system did one but not the other, and to know the limits of what I was trying to achieve.

The Manifold Algorithms

With the usage patterns in hand, I looked at how to solve one or more of the problems, considering that my goal was to have something where absolutely no servers of any kind would be involved.

Local RLD is comparatively simple, since the size of the search space is going to be limited, and I had already looked at that part of the problem with my Nom system for ad hoc wireless networks. Looking at the state of the art, one thing that was clear was that every one of the systems currently existing or proposed for global RLD depends on infrastructure of some kind. In some of them, the infrastructure is self-organizing to a large degree, one of the best examples of this being the Internet Indirection Infrastructure (i3). So I set about to design an algorithm that would would work at global scales with guaranteed upper time bounds, which later turned out to be an overlay network algorithm (which ended up being based on a hypercube virtual topology), as opposed to the broadcast type that Nom was. For a bit more on overlays vs. broadcast networks, check out my IEEE article on the topic.

Then the question was whether to use one or the other, and it occurred to me that there was no reason I couldn't use both. It is possible to to embed a multicast tree in an overlay and thus use a single network, but there are other advantages to the broadcast algorithm that were pretty important in completely "disconnected" environments such as wireless ad hoc networks.

So Nom became the local component, Manifold-b, and the second algorithm became Manifold-g.

So that's about it for the intro. I know that the algorithms are pretty crucial but I want to take some time to explain them properly, and their implications, so I'll leave that for later.

As usual, comments welcome!

IBM puts the PC business on the block

It's all over the wire.

Aside from the historical significance of this, my first feeling about was a bit of sadness, and thinking "No!". My second feeling was surprise at feeling anything about a corporate acquisition! My third :) was a realization that if this happened Apple would be alone, as far as I'm concerned, in moving the ball forward on laptops. IBM was the king of laptop evolution in the PC side, and even though Dell and HP are respectable, they've never shown huge amounts of initiative there (HP has been much better in terms of its PocketPC work, or rather, the iPaq team from Compaq was and HP has kept the ball rolling).

So my thought had a lot less to do with the PC business itself than with the Thinkpad business. IBM desktops were always a bit clunky IMO, and, excellent keyboards aside, they didn't do much for me. No doubt the failed PS/2-OS/2 experiment in the late 80s (remember Microchannel?) had a huge effect on the vitality of the IBM desktop PC line for a long time.

But the Thinkpads! Almost all of my laptops have been Thinkpads. Best keyboards of any laptop. Simple, good design. Amazing reliability. I got a 560e in 1998 and in 2000 I gave it to my parents so they could keep using it. And it still works fine. The battery started failing two years ago. 4 years. No doubt we take good care of the machines, but still, that's quite a long time for these things.

More than anything, what will be missed the most will be the innovation that Thinkpads championed. While a bit dry in terms of design (certainly Apple has been far ahead of everyone else on that count, as usual), they've always moved forward in terms of functionality. Thinkpads where the first to include 10.4" color TFTs, introduced the trackpoint device (invented by IBM, the trackpad was invented by Apple), first notebook to include a CD reader, ultraportables (the 560), first to integrate DVD, and small but useful things like the "ThinkLight" the little light at the top of the LCD that illuminates the keyboard. That, plus what didn't make it in the long such as the amazing Butterfly keyboard or LCD projection that was done by removing the back cover on the display and then resting the now see through LCD on top of an overhead projector, and other things like the Thinkpad Transnote.

Anyway. Maybe IBM will keep part of research focused on that (I hope so!). But it seems more likely that from the point that this transaction happens (assuming it does) then Apple will be the main flag-bearer for innovation on notebooks.

ads in rss - not as easy as it sounds?

Last week Jeremy was talking about ads in RSS and how it seems a foregone conclusion that they will, eventually, become the norm. I agree that this is more likely than not, but I doubt that today's web ad infrastructure (as understood by what Yahoo!, Google, et.al. do) will be used directly.

The reason why I say this was actually mentioned by Jeremy, but not explored. While talking about the options (full text with ads, summaries without), he said:

I don't want to have to choose between ad-laden full-content feeds and the pain in the ass summary only feeds. Anyone whose ever tried to catch up on their reading while on an airplane or train gets this.

The problem with ads in RSS lies in the second sentence: "Anyone whose ever tried to catch up on their reading while on an airplane or train gets this."

Many RSS readers are web-based, and those would always work for web ads (unless a plugin is added to stop them, see below). But many, many RSS readers are rich clients, and clients will sometimes be working in disconnected mode.

"Disconnected mode" throws a wrench in the ad-serving business model, by either preventing the download of the ad, or preventing clickthrough.

If that's the case, then how do you serve the ads? You could embed them into the content, sure, but then you'd have the problem of a) showing relevant/uptodate ads, b) measuring ad-views and c) allowing click-throughs, which are impossible while disconnected.

Someone might say that most people are wired most of the time, and so this problem is minimal. But I have no doubt that, were ads in RSS to become pervasive, rich clients would include a simple way of working in "disconnect mode" (and those that don't would fall behind those that do), not to speak of plugins that would surely be developed, both for clients and browsers, just like adblock exists for Mozilla.

If the readers were to be integrated into the ad serving-viewing-clicking cycle (keeping stats, allowing clickthroughs, etc), then maybe things would be closer to web ads, but who is to say that users will not flock to RSS readers that will support the "ad-free" mode? Or modify their ad-friendly readers?

So even though ads in RSS might be just around the corner, I'd bet that they (and the business model behind them) will have to change at least a bit--the current way in which web ads work probably won't be enough.

looking for the next big thing

So. A week has gone by with no posting. Lots has happened, but more than anything it's been a time of consolidation of what had been happening in the previous weeks. First, the short version (if you have a couple of minutes, I recommend you read the extended version below): tomorrow is my last day working for clevercactus. And that means I'm looking for the next thing to do. So if you know of anything you think I could be interested in, please let me know.

Now for the extended version.

For the last couple of months (and according to our plan) we have been looking for funding. Sadly, we haven't been able to get it. This hasn't just been a matter of what we were doing or how (although that must be partly a problem) but also a combination of factors: the funding "market" in Europe and more specifically in Ireland (what people put money into, etc), our target market (consumer) and other things. Suffice it to say that we really tried, and, well, clearly it was a possibility that we wouldn't be able to find it.

On top of this, I haven't been quite myself in the last few weeks, maybe even going back to September (and my erratic blogging probably is a measure of that). By then I was quite burned out. Last year was crazy in terms of work, and this one was no different: between January and the end of July I only took two days off work (yes, literally, a couple of Sundays) and the stress plus that obviously got to be too much. I see signs of recovery, but clearly this affected how much I could do in terms of moving the technology forward in recent weeks. Since there's only two of us, and it's only me coding (my partner deals with the business side of things), this wasn't the most appropriate time to have a burnout like that. I screwed up in not pacing myself better. Definitely a lesson learned there.

At this point, the company is running out of its seed funding and we don't have many options left. Even though it's possible that something would happen (e.g., acquisition), what we'll be doing now is to stop full time work on the company, which after all won't be able to pay for our salaries much longer, and look for alternatives since of course we need to, you know, buy food and such things. The service will remain up for the time being, and I'll try to gather my strength to make one last upgrade (long-planned) to the site and the app, if only just for the symmetry of the thing. Plus, you can't just make a service with thousands of users disappear overnight. Or rather, you can, but it wouldn't be a nice thing to do.

Now I have a few weeks before things get tight, and I'll use that time to get in the groove again and hopefully find something new to do that not only will help pay for the bills but is cool as well. Who knows? I might even end up in a different country! As I said at the beginning, if you know of something that I might find interesting, please send it my way. Both email and comments are fine (my email address can be found in my about page).

In the meantime, I'm going to start blogging more. No, really. I have some ideas I want to talk about, and maybe I can get back into shape by coding (or thinking about) something fun and harmless.

Or, as the amended H2G2 reads: Mostly harmless. :)

russ@yahoo!

Russ will be consulting for Yahoo! starting Monday. Congratz!

Between Russ in mobile stuff and Jeremy in search (another cool move) you've got two big areas of Y! covered by great bloggers. (I wonder if someone is blogging from the services side...Y!Mail, etc.). Most excellent.

the market of one

Tangentially related to my previous post, in terms of usage patterns, context, and so on, I was thinking of this notion I call "the market of one".

The market of one is... yourself. You (in theory at least:)) have the best insights on what drives you and not, what you like about something, use patterns, etc. It is the "eat your own dog food" concept but with some insight applied, and only for one person.

The market of one seems crucial to me when either a) your organization is small (large companies being able to create focus groups, commission marketing studies, etc, and then being able to survive massive failures of product lines) or b) when you're doing something completely new (when focus groups aren't much help--people generally react badly to the unknown).

It's not just using what you're creating, but also asking yourself: how much am I using it? Does it satisfy my needs? Why, or why not? And so on.

When the product doesn't exist, it is, I think, the "standard way of thinking." We project ourselves and our own needs and based on that we evaluate whether we think something's good or not. For example, a lot of the disagreement over web-on-mobiles usage mentioned in the previous post comes from people transparently applying their own use-cases to what they think the product is, and then extrapolating from there.

I use this idea all the time when thinking about a product, or when designing it. But I think I haven't been as consistent in applying it during and after development, when reality takes over the grand designs that are in my head.

Somehow, I think that I keep seeing the product that I know it will eventually be, rather than what it is today. That future-vision can become harmful if it blinds you to the problems that exist today. Something to pay more attention to in the future.

the web is not the browser, a year later

About a year ago (a year and three days, actually) I wrote a post titled the web is not the browser. At that point the discussion was whether "RSS usage" was "web usage" or not. Russ posted his thoughts on the Mobile Web yesterday, and there's a bit of deja-vu there, but going in some new directions. So here's my thoughts, updated. :)

The Web isn't just HTML+HTTP

Russ starts:

Now, when I say "web", you know what I saying right? I'm generally thinking HTML over HTTP and though you could probably say there's a lot of "dark content" out there on the internet - like in email, etc. it's generally not publicly accessible. The web in 2004 is the lingua franca of internet based information, I don't think there's much argument on this...

Actually, I think there is some argument on this, and Russ and I basically discussed it in the comments of that entry of mine a year ago. Russ made exactly the same argument in his comment and I replied with this in another comment:

Russ: you say the web is HTTP + HTML. Okay. However, what you type in your browser is a URL. The URL is post-web (RFC 1738 is dated Dec. 1994), and as it says in the RFC: "The specification is derived from concepts introduced by the World-Wide Web global information initiative" [Just to clarify, I mean "post-web" in that the concepts in general use today were developed after the initial web was launched]. Yes, originally "The Web" was the world wide web, HTML+HTTP. But very quickly things became intermixed. Today, you'll click on a link that is "ftp://" to download a file. FTP dates back to the pre-WWW days. Is clicking on ftp:// *not* the web too? If people click on a text file obtained through FTP and read it on their browser, is that not the web? Or on a quicktime video?

Today we often use HTTP to download files, not to see HTML, or to stream video, or audio, or Flash animations, or, yes, RSS, and it's an integral part of the "web experience". Some clients take HTML and present it in different ways "syndicating" it. So what I'm saying is that, even though at the beginning the web was indeed just HTTP+HTML, it went beyond that very quickly. Even if you only consider what you see within the browser the "web experience" includes a multitude of formats and protocols.

Purely in terms of content "weight" (storage capacity + bandwidth) I'd bet that web protocols are carrying as much, if not more, content types that are not text/html. As an example of what I mean, consider that downloading the full page for Russ's post on the mobile web clocks at 35,703 bytes--34 KB or so, and it's a typical post, image, a decent amount of text, some comments. Now, about a month ago Russ was talking about that 25 GB that he saw of downloads from an MP3 he posted (size 5,649,826 bytes, or about 5.5 MB).

That single MP3 amounts to about 160 posts.

Russ generally posts once a day. So that MP3 equaled his production for half a year in terms of size (the bandwidth ratio is probably a bit lower than 1:1 though, since the MP3 doesn't get hits from search). Since Russ has also been doing audioblogging, there are several MP3s posted in his blog, which I'd wager amount for quite a lot more than all of the text/html content he's ever produced (even if you count images, CSS, etc). Then, elsewhere, there's Flash, Java, video, and everything else.

So, I think the assumption that most content is text/html is wrong. I don't see this as a big problem for the discussion that follows though, because mobile browsers will eventually support most if not all of the advanced features.

To mobile or not to mobile

Russ's next point is that delivering a "dumbed down" version of the web (or using some other kind of content delivery system) is wrong because a) devices are appropriate and b) people want "the whole web". I agree with the second, but not the first (not for any use-case+context at least, more on that in a bit).

Paradoxically, the title of Russ's post undermines his argument a bit. If you have to use an adjective for something, then it's different right? Russ's vision is not a "mobile web" but "the web on mobiles". After all, we didn't start calling the web "the video web" or "the audio web" when streaming arrived--it was just the web. This is just semantics though. :)

Russ then goes through the common arguments against browsing on mobiles. These arguments generally fall into one of two categories:

• Capabilities. These include: "The screen is too small," "Mobile phone browsers are too limited (no Javascript, no frames, etc.)," "Other platforms (Flash, Java, Brew) provide more richer experience (with less latency)," "Mobile data connectivity - even using 3G networks - has massive latency between pages."
• Usage. "People use their mobile devices "differently" - thus need snippets of data." "Why use a mobile phone when you're not more than 10 minutes away from a PC ever," "Prefer laptops and WiFi, rather than struggling with a mobile."

To this list I'd add "Navigation" in the Capabilities section. I think the small screen is less a factor than the fact that with most phones you have to use the keypad/tiny-joystick for input. The Web, however, has been designed with keyboard/mouse in mind. I think solving the navigation issue is more important (and more difficult) than worrying about the latency for example. People browsed the web with 14.4kpbs modems for quite a while, after all.

The capabilities problems might be a factor at the moment but clearly will not go on forever. If there's a need, they'll be fixed (eventually). Moore's Law and all that. So I don't see those things as a major problem.

Usage is to me the most important category, but it's generally overlooked. Capabilities matter, but assuming you fix most of them (it will be hard to get around the screen-size problem for a few years though--but even so I don't think it's a major roadblock), usage is really where the crux of the matter is, and one that can easily get muddled. When Russ mentions "People use their mobile devices "differently" - thus need snippets of data," he's no doubt writing down something he's heard many times. And while the first part is hard to argue with (People definitely use their mobile devices differently), the second part doesn't follow from it. At all. Blackberry users handle quite a lot of email. Many users enjoy streaming on phones, or online games. Those hardly count as "snippets" of data. However makes that statement is pushing forward their own assumptions about phones: "People use mobiles differently, and mobiles are small and puny, hence you need snippets of data."

But the use cases are different and it's worthwhile to spend a bit of time on that topic, because I think that's a big part of what's actually being discussed, though not directly, when someone says "I don't want the web on a mobile."

It's all about the use case and the context

Use-cases and context are what drive usage, not product features (the "capabilities" from above). Product features can enable new use cases, but given a certain base of devices (i.e., given today's technology, maybe looking a few months into the future at most), it's the use-cases+context that matter most IMO.

Consider the following use-cases+context:

1. If I'm sitting at a PC with a broadband connection, it's almost unthinkable that I'd use a phone for browsing the web. In fact, if I'm sitting at a PC, it's almost unthinkable that I'd use any other device. And why would I? Everything is right there.
2. I'm on the road, and I have a laptop that is turned on with WiFi running, and a smartphone. I'd use the laptop. Cheaper, faster, better UI, etc.
3. Same case as before: laptop and smartphone. Only this time the laptop is turned off. The phone is on standby. How long does it take to get everything up and running on the laptop? Maybe 5 minutes? The phone, though, being always-on, is nearly instantaneous. So if I just need to check something (say, the weather) the phone is probably a better choice. But if I'm going to be online for a while, I'll probably use the laptop anyway.
4. If I have nothing but a phone, that's clearly what I'd use. But would I use it just like a PC? Doubtful. A phone doesn't go well with "multitasking", e.g., I can't easily browse the web and talk, and check my calendar, and check a relevant email at the same time, and I've often found myself doing all of those things while in the middle of a conversation (real-world, phone, or VoIP). Phones are more of an immersive experience (speaking of phones being immersive, check this out :)).

My point is that what we think "people" will do or won't do is heavily influenced by our own experience and usage. I suspect that phones will find entrenched niches at first, things where the availability, mobility, and form factor takes precedence (similar to how game consoles have their place against ever-more-powerful PCs). If I'm carrying a laptop around most of the time, then it's unlikely that I'll see the phone as more than a stopgap measure. If, however, I only carry a phone around most of the time, the phone will gain importance.

Do I want the whole web on a phone? Absolutely.

Will it eventually become a much smoother experience? Certainly.

Does that mean that I'll (that is, me personally) stop using PC and PC-like devices, and use phones as my primary method of browsing? Not a chance.

But does that mean that nobody is going to use the phone as their primary interface to the web? Of course not.

For some people though, the use-case, or the context, or both, will be there, and maybe what makes the discussion more complex is that "some people" here means tens of millions of users. As I understand it, in Japan, for example, people use their phones to access online services more than their PCs, or at least nearly as much. Phones will grow in importance, and take their rightful place in the continuum of capabilities we have today--browsing the whole web, yes, but not necessarily pushing PCs out of the way because of that.

the true story of audion

[via Frank]: The True Story of Audion. Quote:

As the kids say, upon seeing some awesome frags and/or gibs: OMFG.

Must read.

the new sony ultralight pc

News.com wonders if the new Sony ultralight PC (6x4 inches, 1.2 lb, 512 MB RAM, 20 gig drive) will take on the iPod. This is a ridiculous idea for a number of reasons, starting with the $2000 price tag for the same storage capacity you get on an iPod for about one-tenth of the price (or less, considering that probably 25% of the drive goes to Windows and various apps).

What I'd be wondering, instead, is if this will be finally the ultraportable that cracks the US market, or if this is the first of many "webtop" devices people use around the house, a kind of portable display. Ebook-reading, web browsing, quick note-taking, tasks, and email would be good tasks for this machine.

Regardless, it looks fantastic doesn't it?

today's reading

The Internet as a Complex System (PDF) by Kihong Park, Chapter 1 of The Internet as a Large-Scale Complex from Oxford University Press, and Anda's Game a short story by Cory Doctorow. Both highly recommended :).

new design time?

A few days ago Dylan changed the design for his blog, now Russ has changed it as well. For the last couple of weeks, whenever I want to relax (or use CSS infuriation as a distraction, depending on how you look at it) I've been playing with a new design based on a newspaper-like view, but it hasn't convinced me. I guess I'll add it as an alternate stylesheet and FireFox-enlightened users can switch it using the little gizmo that appears at the bottom-right of the window when a page has alternate stylesheets (have you noticed that one?). Maybe this weekend...

One thing though: new blog designs are always reinvigorating for some reason, even if many readers don't see them, courtesy of syndication. :)

content, sharing, and user interfaces

A couple of days ago Russ posted an interesting entry (long, but worth the time) on what he dubbed 'communicontent':

Communicontent to me, is a byproduct of communication where traditional content is magically created. As a corollary, the forms of communication that can best be expressed as content almost naturally become communicontent. See this weblog? This is communicontent. I used to drive my friends on mailing lists crazy by writing all these long, in-depth emails. Now I just write all the same thoughts in my weblog instead. The only difference is that the viewers aren't restricted. I'm still just communicating my personal thoughts. It's communication, but because it's been captured in a fixed state to be found later, it's also content.

This is more than just the famous "user generated content." If I take a picture (content I've generated) it doesn't really matter until I decide I want to send that picture to someone. Then it becomes something different. The act of communicating that piece of content makes it more special. In practical terms, it simply adds more meta-data at the very minimum: a title, a description, a place, etc. But it also gives it an inherent value as well: I think this is important enough to send, therefore you may want to think it's important enough to take time to look at.

In general I agree that content that is communicated becomes a different sort of beast (The Google-Gmail analogy he mentionts at one point is stretching it a bit IMO). There are a couple of things I'd add, particularly what I think adds to the success of this type of shared content.

First, is that content relevance (and quality) matters, a lot. Most content people generate has relevance for themselves and a small group, even when we blog we sometimes (or maybe most of the times :)) we post about things a lot of people simply do not find interesting. Quality has a lot to do with the kind of information you're sharing, and with the kind of device/interface you use to create it. For example, there is no way someone can write a well-thought-out argument on anything using T9 on a, say, Nokia 3650. Why? Because the interface gets in the way. Similarly, you might be able to post high-resolution pictures from your PC, but not from most phones (camera quality... network speed... ability to crop/edit if necessary).

Second, as Russ notes:

In order to create communicontent, pure content needs meta-data, and pure communication needs organization.

Consider this and what I said in the previous paragraph, it brings back my recent thoughts on metadata. That is, the ability to create metadata or organization is worthless if there aren't also good ways of navigating that metadata, and viceversa. Both ends have to be covered. FOAF has, in my view, suffered from this. There's no way for non-geeks to make use of all that metadata, and conversely they don't have easy-as-pie ways to create it, which results in limited appreciation of it by non-geeks.

Putting this two thoughts together, what I'd add to Russ's ideas is that the process (which includes generation and access) by which this shared content is created matters a great deal, as does the follow-up access. Both ends of the equation have to be covered, that is:

• The content (and if possible its accompanying metadata) has to be extremely easy to create and share
• Once content is created, the content access interface has to be adequate for its purpose

I think moblogs work because it's easy to take a picture, then (relatively) easy to post them, and then the software on the server does the rest for you (organize them according to time, create a slideshow, etc), which covers both ends--and it's when both of these conditions are met that apps cross the boundary from the cool to the useful.

the switch from berkeley db over to mysql

One of the objectives of my recent server switch was to move from Berkeley DB to MySQL (aside from an expected performance improvement, I was tired of seeing plug-ins for MT that I couldn't run-- example). Plus I feel more comfortable with MySQL. I followed the instructions for this in the movable type documentation and aside from some glitches during conversion (weird messages such as "WARNING: Use of uninitialized value in string eq at lib/MT/Util.pm line 754.") and one timeout (which forced me to restart the process after deleting the old tables) everything went fine.

One thing to note though, which the documentation doesn't make completely clear: when you run the mt-db2sql.cgi script, you must leave both the previous BerkeleyDB pointer as well as the configuration for MySQL. Once the conversion is done, you can comment out the BerkeleyDB location line. During the process, I also renamed the trackback and comment scripts, to avoid overlaps.

Now all seems to be running fine, and simple tests seem to show that some things are a bit faster (example: posting takes about 20 seconds, as opposed to 30 seconds before).

One more thing I can cross off the list. :)

it's all about simplicity

The latest Economist's quarterly survey on IT is more high level than usual, but quite good anyway. Quote: "The Next Big Thing is not a thing at all: it is simplicity." Also: an article on Complexity (suscription possibly required for the links).

the return of the spammer

So today, as a background task, I did a full rebuild of the site, so that the new MT scripts were properly referenced in the archive pages and so on. And lo and behold, it only took about an hour for comment spammers to show up again. Except this time it took only three or four clicks to select all the spam they had already posted, delete them, and ban their IP address. I'm glad I switched over to MT 3.1 or I'd be pulling my hair out again already.

Now to figure out how much bandwidth those invalid posts are costing and how to stop them if possible...

the only way to beat an ipod... is to be an ipod

Reading the news of Samsung's announcement of the first 5-megapixel cameraphone brought back some questions I had over media in current smartphones. I've had a SonyEricsson p900 for more than two months now, and this I can say with certainty: media center, this phone ain't (yet).

Considering, then, that the P900 is most definitely at the high-end of smartphones, it would be reasonable to say (even accounting for the differences) that in terms of media storage, display, and management, smartphones in general aren't there yet either.

Why, oh, why, do I make a statement that will surely enrage smartphone fans? :-)

Okay, I'll correct the statement a little bit. Against dedicated devices, smartphones still have a ways to go. As ocassional-use devices they are excellent, and hopefully they'll grow to more than that in some areas, but for day-to-day use they are still lagging behind dedicated devices.

Some of my reasons:

• Poor data sync: Most smartphones today have poor data synchronization mechanisms: cumbersome, slow, erratic. Until switching, say, one playlist for another in a phone is a one-click affair (or until phones get an iPod's memory) they won't be able to out-play the pure music/media players. P900 synchronization, for example, uses serial-bus speeds--you don't want to know how long it takes to upload a single MP3 song, let alone an entire album. Bluetooth connectivity in particular is quite the random affair, particularly phone-to-PC.
• Speed/Navigation/Resolution: Scrolling through pictures on smartphones is slow. Scrolling through songs is slow. Switching functions is slow. Camera resolution is generally crappy. Audio/Video quality suffers. Result: they are good for ocassional use, rather than "first-device" use. Speed and resolution, however, are easier (and upcoming) fixes than navigation (i.e., usability), as the release of the above-mentioned Samsung phone demonstrates.
• Poor built-in media players: this really applies to music more than anything else, although I still enjoy more flipping through pictures on my digital camera than on my P900. Or, for example, switching through songs on the P900's media player without looking at the screen is an invitation to disaster (although I really like the scroll-wheel) while any respectable media players can be used "blind" without a problem.
• Stability: The final straw. I can't tell you how many times my P900 crashed while switching between the camera and other apps (for example, when a phone call arrived and it was in camera mode). While other phones might be more stable in terms of crashes, many of the current phones have issues with memory that requires restarting the phone or getting a memory management application, or both.

So. I think that big strides have been taken recently, and no doubt more will be taken in the future. For the moment, the only way to beat an ipod... is to be an ipod. :)

(possibly) made-up proverbs for interesting times

Reading this post over at Jeremy's weblog I found myself nodding profusely, and then I noted he was pointing to this page that talks about the famous (for me and many others at least) Chinese curse (or proverb) "may you live in interesting times". It is possible that it's not a Chinese proverb at all. (Here is some more information).

As it happens, testerday I was thinking about the various tidbits of "knowledge" that I have stored up in my head and that I can't trace a source for (example: "Whales can use an ultra-low frequency signal that has world-wide reach, but by now it's likely that the interference caused by ships all around the world precludes them from doing so"). First I wondered about the social mechanisms that allow these memes (whether true, partially-true, or false) to spread and take hold (e.g., do they adjust to widely shared preconceptions?). Then what I was trying to do was assign them to some sort of "unchecked" mental category that would later imply a proper reference lookup before using them (or whenever I felt like it).

Of course, right after thinking about that, I remembered the following piece of dialogue from Annie Hall between Alvy Singer (Woody Allen) and Annie Hall (Diane Keaton):

Alvy: I've got to see a picture exactly from the start to the finish, 'cause ... 'cause I'm anal.
Annie: Ha! That's a polite word for what you are.

Anyway.

Now it seems I should add "may you live in interesting times" to that list as well.

Ain't that something.

what the bubble got right

[via Sam] Another link that has, quite possibly made its rounds: Paul Graham, What the Bubble got right. A good read.

rfc 822 dates in movable type

Yesterday, when I migrated to MT 3.12 I remembered that I was using a plugin to generate RFC 822 dates which MT didn't seem to be able to do. I had arrived at this solution a year ago (damn! a year ago? What the... anyway) because the basic $MTBlogTimezone$ in MT generated timezones in the format +HH:MM, while RFC 822 generally requires timezone abbreviations (e.g., GMT, PST, etc). But yesterday, while thinking about reinstalling the plugin, I rechecked the MT docs for tags and noticed that since MT 2.5 there has been a no_colon attribute for $MTBlogTimezone$ which removes the colon and thus matches one of the possible ways of specifying a timezone in RFC 822 dates, as +HHMM. The result is:

<$MTEntryDate format="%a, %d %b %Y %H:%M:%S"$> <$MTBlogTimezone no_colon="1"$>

which gets you RFC 822 compliant dates on your RSS 2.0 feeds without requiring any plugins. With this knowledge, looking specifically for these terms tells me that this is a well-known solution, but maybe it's not as clear as it should be, it certainly wasn't obvious to me when I looked for this a year ago.

Now, this solution applies to MT from 2.51 and up. MT 3x introduced a specific RFC 822 format (basically a shorter way of specifying the tags shown above), used as follows:

<$MTEntryDate format_name="rfc822"$>

Which makes the whole thing a lot easier, and which is included by default in the MT 3 templates. Cool.

all systems go

Phew!

I'd say that the process is basically finished. I'm sure that things will keep popping up over the next few days, but most things seem to be working.

The last thing I did today was upgrade to movabletype 3.12. The new comment moderation, as well as other comment management features, and dynamic pages made it a good option for me. There was a little snafu when upgrading the database but it was a combination of me reading the docs wrong and the mt-upgrade script telling me that everything was fine when it clearly wasn't (the DB was getting trashed). A short (and suprisingly quick) exchange with MT support made me realize my mistake (I had do two format upgrades in a row, first to 30 and then to 31) but aside from the obvious thought that the upgrade script should be a bit smarter about this (obviously it has to be a common occurrence) it was a pretty smooth experience.

I made some measurements on the old machine: posting an entry (i.e., individual rebuild, plus related indexes) was taking 3:30 minutes (yes, that's three and a half minutes). Unbelievable how we get used to things and we stop noticing them. With the new machine it's taking about 30 seconds. I would expect (hope?) that after the MT 3.12 upgrade this will be even faster. We'll see.

So--to test the new new thing I'm reopening comments in this entry and re-enabling them on the weblog, with comment moderation turned on. Let's see how long this lasts! :)

If you're reading this...

...then it means that you're accessing my weblog on my new server. The DNS switch will take a couple of days to propagate fully, but it's my experience that almost everybody sees the changes within 12-18 hours.

Finally! I started a few hours ago and by now it's mostly done. Had a few interesting experiences which I'll talk about more later. I still haven't finished moving everything over, since Eircom (my home's ISP) still hasn't updated its DNS tables (I'm posting this through an alias, and not everything works).

After the transfer is done, it will take me a bit to verify that all is well and backups are in place (just in case...) but everything should be back to normal by early next week.

Finally: if I haven't replied to an email you sent recently, wait a bit more (or send it again). The migration implies changing email systems, and that will take a bit to stabilize as well.

ps: I'm still a bit sick: my left ear feels clogged ocassionally, which is extremely disorienting and quite a pain, and I'm generally pretty congested, but definitely feeling better. Funny how easy we forget how good it is to feel normal.

google desktop search: not yet for me

This is one of those things that has already burned its way through the blogsphere, but anyway...

One of the few things I did (was able to do) yesterday was install the recently-released Google Desktop Search.

Why? Two reasons.

Number one, I wanted to try the latest new new thing.

Number two, I wanted to find a particular document based on a particular set of keywords, and I was hoping GDS would make that easier. I could wait for a few hours for it to index my hard drive.

The download was quick. The installation was a snap. It apparently integrated with Firefox automatically (it wanted to close it before installing), but that wasn't mentioned anywhere. Whatever. Fine. I could see that it was a personal web server. Good solution, nice and seamless integration with Google web.

But I couldn't use it yet, because then there was the wait for Google to index some 60,000 "documents" it eventually identified in my machine. I honestly don't know how long it took--I left it running all day, and when coming back in the evening it was done, so 6-7 hours max.

Before searching, I tried closing it to see what would happen. It complained that if I closed it, any new files created or viewed (web) during that time would not be indexed at all. "Really?" I thought. "That seems kind of harsh. Anyway...". I didn't close it.

Finally I was ready. Double click on the taskbar icon. Search comes up. Type in keywords.

Scan the results.

Garbage. Images, mixed in with documents. Some things given precedence over others because, apparently, the keywords where in the directory name (I'm not sure though). A wave of dissapointment.

Hm. Of course that makes sense, you know, without hyperlinks to provide rank, it becomes more difficult to find what you want.

Okay, so I started trying more accurate keyword sequences. Different combinations.

Time after time, garbage. After a while, I started getting confused at the data I thought I knew. Hm.

I had to reply to email, etc. so I kept working on other things. I had to do web searches, which now, suddenly showed up with a first result that pointed to my own hard drive search results for that term. I mistakenly clicked on them. I did this a couple of times without looking--it took me a while to figure out what the hell was happening, and every time I ended up being left in the mess of my local result list.

It didn't take long before I simply changed my default Firefox search engine to A9.

Early this morning I uninstalled GDS. But guess what. I just realized that A9 is still my default search engine, and I'm starting to get used to the features I was mentioning the other day. Plus, I was browsing through Amazon and I discovered that using A9 gives you a small discount when ordering. Hm. Suddenly I might switch over to A9. I'm still not sure. I've switched back and forth before, but A9 doesn't seem to be as annoying as Yahoo! was with its ads.

So I started off yesterday as a Google user trying Desktop Search, and a day later I'm neither a user of GDS or of the web search? I thought: What the hell...?

I ordered the neuron to do some thinking on the subject and after a short argument, this is what I got to.

Clearly, this is, in part, because of the data I've got. I have many "subtopics" that I work on which contain references, ideas, texts, drafts... it becomes difficult to separate the tree from the forest (or whatever). But other search tools I've used on my data, such as Enfish, have never presented me with the seemingly chaotic results that GDS was showing.

So my neuron came to the conclusion that what was happening was that GDS was too adept at finding information, but, unable to discern proper ordering, it was actually making things worse. Ok, Google isn't God. We all make mistakes. No problem.

But why did I leave Google Web so quickly, and worse, almost without realizing it?

It seems to me that the answer is simple: integration. Google has gone to great lengths to make the local search experience be a seamless continuum with the web search experience, and viceversa. On principle, it's a great idea. However, I trust Google Web to deliver good results. When GDS provides the same experience, I expect the same results. But that doesn't happen (and it's doubtful that it ever can). So suddenly I don't trust Google Search, the experience in general. The web, which is now integrated with desktop results, gets dragged down into the mud by the crappiness of the desktop search results.

Consequence: Google loses a user for GDS. And then, because of the seamless integration, for all its properties. Even if that's not the case, the web results are now tarnished by sharing being equated with the local search results.

It is important to note that this is how I interpret my own actions since yesterday, things I did more or less "subconsciously". I'm not saying this would be a conscious thought process...

I also note that the GDS design integrates with the web design in a way the previous web design might not have been able to do--not cleanly at least. (Am I wrong?). This would prove that GDS has been on the works for some time---my point being that those journalists that say that GDS was "rushed" are wrong.

Okay.

Keep in mind my only functional neuron is heavily congested, so it's possible I'm missing something, but I think it may be a mistake on Google's part to take the fight to obviously and directly to the desktop. That said, everything Google does is so scrutinized that it's probably impossible to make this a "low key" release.

The desktop is Microsoft's turf. The Web is Google's. I was kind of hoping that Google would behave differently than others in the past and just keep on moving into other markets, rather than retreat (at least partially) to fight into Microsoft's. GDS, together with Picasa (and the ever-present rumors of a Google Browser), seem to indicate that it's going to be a mixed thing at best, with Google leaning on its web side to pull in desktop functionality and users with integrated "seamless navigation" features (e.g., Picasa is to Google Image search what GDS is to Google Web Search, and so on).

Anyway. It was an interesting experience. Given that it was a smooth install/uninstall cycle, I'm certainly willing to give it another try when a new release is out.

PS: My experience was similar in some respects to Don's, but it never got that bad. He noted: "I don't enjoy writing code inside a jet engine". LOL. Also, Jon discusses Firefox integration and points to alternatives. Dave makes a good point about user formats (since one would expect that other major formats, such as OpenOffice, would eventually be supported), and exposing an API (which Jon would points out could be made by "reflecting" content in a format the indexer understands, in this case XHTML). Kevin ponders a lucene-based answer, and Om compares it to Blinkx as well as pointing to other reviews.

GooglME

Another cool piece of software that was released while I was away is Erik's GooglME, which he updated to 0.2 beta last week, is a cool J2ME app that provides a front end to Google's SMS service.

The MIDP 2.0 version installs and launches successfully on my P900 but it throws an error--to be expected since the P900 isn't yet on the list of tested devices. (btw, If you have tried the app on another MIDP-enabled phone, let Erik know the results as he notes here).

adblock (and other Firefox extensions)

A must-have Firefox extension: adblock. Not only improves reading content online -- it also reduces bandwidth usage. I use it with sites that I pay a subscription for and that still insist on bothering me with ads, or when the ads have become so large that they cover a quarter of a browser's window (otherwise, I allow ads, since it's clear that it's how a site pays for its bills, something we all need to do).

Other Firefox extensions I use regularly: EditCSS, JSLib, Venkman and WebDeveloper.

Update: Luke notes, via email, that adblock does not save bandwidth--it only hides or removes the elements from the display layer. Good point, and thanks for the correction! I wonder how hard it would be to add that feature... it would probably still require downloading in some cases, to be able to calculate the width and/or height of what you're not including (if the size is not available in the tag) to maintain the layout...

just. blog. it.

A couple of days ago Dylan released JustBlogIt, a Firefox extension to allow right-click posting to a weblog. It's excellent (yep, using it right now). The only thing missing is the swoosh!

ps: it's also missing the ability to show the "preview" button in movabletype. But that's a minor detail and probably a movabletype issue more than anything.

more than just words and hyperlinks

Sometimes a post, succint or not, leads in so many interesting directions that it deserves a category on its own for thought-provoking ability alone--the post, the info to which it links, sets off a storm of ideas in my head that takes a while to get under control. Today's examples are Dare's Why I love XML-DEV and Scott's the spirit of startups past. Most excellent.

a9

Amazon's A9 search engine has been adding some intriguing new features. "I like, I like!" :)

I've been actually musing (in my head) about personalization and search during the last few weeks. This adds some more datapoints. When the ideas are organized to a minimal degree, I'll write something up.

craigslist

[via Karlin]: craigslist arrives in Dublin. Cool!

per-coffeemug is better

Don is thinking about filing "per-chair, per-desk, and per-floormat software pricing methods" in response to Sun's per-employee software pricing patent.

Don, patents are serious stuff, and you're foolish in not taking them seriously. I wish you would. The growth of patents such as these should be not just accepted but applauded. I am thus compelled to reply. If my language appears overly circuitous, it is only because I avoid vitriol through an overextended misuse of formalisms. Which also excuses this redundant explanation.

I was saying.

Patents provide "glory" as the News.com article says, and this is for a good and simple reason: patents are the work of Heroes. Paraphrasing attorney-at-law Lionel Hutz: "I don't use the word Hero very often. But patent applicants are among some of the greatest American heroes of all time."

Let me, Don, also point out some of the problems in your childish proposal.

First, floormats are not common in software development environments.

Carpets, maybe. But not floormats.

And something else: what is up with static electricity and carpets? It hurts when I touch metallic stuff and the spark or whatever goes off. I wish that would stop. Can't we invent a system to make electricity only come from some kind of socket on walls or something?

But I digress.

Second point: many programmers share desks. How are you, Don, going to get around that? How about when a chair breaks?

Third, I am already filing per-coffeemug patents which will box in youir per-chair and per-floormat patents in no time.

Per-coffeemug is also superior to per-employee, which is login based. Users are very protective of their coffee mugs, more than they are of their login/password combinations.

I will soon have the world-wide market on software pricing patents cornered. I will then sell the license at a fair price (reference for that can be the price of a box of Windows XP for example) and use the proceeds to purchase a coconut farm on an island in the Caribbean (preferably one that comes with monkey-butlers).

PS: I am also patenting this blog entry. Just in case.

comments?

Related to my comment spam problems (and the drastic solution I "implemented" of simply disabling the script), Toni recently pointed specifically to scode. I looked at something like this (and it led me to consider something similar but with a random text string instead of an image, and in a random hidden text field included in the form), the problem with an image-based solution is that it increases load on the server (every static page calls a script) and I don't want to do that until I switch servers (any day now! :)) since this server is edge already.

Then again I haven't looked for other solutions since late August. I need to get this done eventually -- at this point I am probably just being lazy. Or "spring feverish". :)

airborne madness, two years later.

This post I wrote on September 2002 could have been written two days rather than two years ago. I can't easily describe how ridiculous I find the restrictions companies place on electronic devices because they might "interfere with navigational systems" and such. Every time I travel there seems to be a new device or system that is "dangerous." Example from my recent footrip to the Netherlands was that you couldn't use any "laser-powered device, such as a CD player." Nevermind that a CD player is not "laser-powered", you could use laptops, which generally have CD or DVD players. But laptops are OK.

Right.

Cellphones have long been no-no at any time during flight (only recently some airlines have started to let you use them with the doors open at the gate, others still don't, even though they use the same airplanes). Now, in a couple of years there will probably be cellphone service on airplanes. Aha. Having a cell on the airplane and everyone using their phones, doesn't interfere. Giving you WiFi in the plane is also A-OK. Turning on a phone on its own, however, is quite dangerous. I get it.

Anyway. I wonder how they will spin the fact that WiFi & cell service on planes creates way more interference that a single device ever would.

firefox's live bookmarks

Don delves into a feature in FireFox 1.0: Live Bookmarks. I only installed FireFox 1.0PR a couple of days ago, in the process of doing a clean install on my notebook (by now my various machines have various versions of FireFox, the oldest of which is 0.7), and I noticed the Live Bookmarks but didn't pay much attention to them: I didn't find the UI very inviting (I read too many blogs to make it practical to see headlines there, and linear menus were NOT designed to hold that many items and be usable in any way, shape or form). From what Don describes the implementation would limit their usefulness as well. But I've noticed other "sudden" UI changes as well, which made me think that maybe the FireFox team was coming down with a case of featuritis. Hopefully that won't happen.

holy service pack batman!

Yeah, well, I still can't get that ridiculous image of Batman @ Buckingham out of my head, so I'm writing it in somehow.

A few days ago I installed Service Pack 2 on one of my PCs, the idea being that I could see whether it would interfere with share, etc. The actual installation went allright (it took a while to download & install though).

The first boot after the install was finished I was asked to configure the "security settings". Since I rely on an external firewall and have all the dangerous stuff disabled anyway, I left this disabled as well. After that the login appeared.

I logged in. And waited. And waited. Icons drew themselves painstakingly across the screen, as if the millions of tiny drunken leprechauns that are responsible for the behavior of Windows were suddenly more inhebriated than usual or maybe were away on a security seminar and thus unavailable to paint icons or do their usual tasks.

Eventually the leprechauns decided to get to work, but slowly. So I rebooted, hoping that things would get fixed that way. No luck. The second time, login took even longer.

I was sort of resigned to things not really working anymore, and thinking how I was going to uninstall this piece of crap, when a dialog box distracted me.

It was from one of the McAfee applications.

Now, backtracking for a moment. I purchased McAfee VirusScan a few weeks ago when my Norton License expired. I had fond memories of VirusScan from many years ago, when it was a simple program that included nice command line tools. Norton AV has become a monster in recent years, installing all sorts of background services that do NOTHING useful except take up resources and interrupt whenever it's most inconvenient. (Note: these services usually are somewhat useful when you operate in an all-MS environment; i.e., Office, Outlook, Messenger, etc. and you have no idea of what you're doing). Be that as it may, I had gotten tired of Norton's heavyhanded approach and decided to try something else, so I remembered VirusScan.

To be honest, I was a bit weary. McAfee had been through a number of corporate acquisitions and mergers and "refocused" on the corporate market (which is usually code for "we will now be able to push garbage down our customer's throats, only now 20,000 seats at a time). But I really had good memories of VirusScan and thought, "really, how much could they have screwed it up?".

As it turns out, the answer is "quite a lot". VirusScan 8 is one of the worst pieces of software I have ever seen. Difficult to register for. Difficult to install. Difficult to set up. Slow. In short, do not buy VirusScan. Norton is much better (if I find a simple product that just scans for viruses and doesn't try to set up my PC as if it was NORAD, I'll let you know :)).

And, most of all, VirusScan is intrusive as hell. You wouldn't believe how many times I've been interrupted in the middle of typing by the stupid VirusScan notification window telling me that "it has downloaded an update" and asking if I wanted to "continue with what I was doing."

I know that my topic was supposed to be XP SP2, not VirusScan, and that this appears to be too hyperbolic even for me, but I am getting somewhere.

About a week before I installed SP2, one of the VirusScan updates installed something called the "McAfee Security Center", which is basically a fancy control window (which is probably ActiveX-based) that tells me that my security in my computer sucks because I don't have any McAfee products installed. I am completely serious about this: McAfee is telling me that my computer is not secure simply because I don't have their software (i.e., their firewall, antispam, etc) installed--irrespective of whether I have other software or other solutions installed. When this security I disabled all the automatic services except the Virus definition update and promptly forgot about it (naturally, the thing kept updating itself, but it wasn't too bad).

Okay, so after SP2 is installed, and the second time I reboot, while I am wondering how to get rid of SP2, the following dialog box pops up:

Since I am suprised by the dialog box, I read it carefully. And even though I read it carefully, I am still not sure of the implications of this action. Keep in mind, I have just installed SP2. I haven't had time to see any of these features. As far as I know, there's nothing in Windows called "Security Center". And yet McAfee wants to replace the default with its own (which I am barely aware exists as it is).

Reasonable person that I am, I decide that no, I will not let McAfee take over the Windows Security center, since I want to see what this is, and maybe later I'll set it up like that. So I press "No" in the dialog above.

Clearly selecting "No" then triggers the following dialog:

Look at the text of the second dialog carefully.

Notice that, again, there's the options "Yes" and "No". However, because of text that can only be described as designed to deceive, the meaning of the buttons is inverted from what it was in the previous dialog. Normally, if you require double-confirmation, you'd say something like "are you really sure? Y/N". The McAfee guys, counting on the fact that you'd made up your mind and you'll probably click "No" again, simply invert the behavior of the buttons on the second dialog, so that you do what they want, no matter what.

Let's recap for a moment.

There's new security settings on SP2. Claims from Microsoft about "improved security" notwhistanding, as far as I can see the main improvement in this pack is that Microsoft is bundling all sorts of apps that until now have been third-party apps, such as adaptive personal firewalls. Additionally, they have disabled a bunch of stuff that is "dangerous", thus taking the path of "if something has a security hole, disable the feature, instead of fixing the hole." (granted, when the problem is the design, "fixing the hole" is much harder, but that's not an excuse on my book).

Anyway, it is clear that to appease third-party vendors (such as Symantec and McAfee) Microsoft has included an API of some sort of this Security Center stuff. And obviously the poor third-party companies do not want users to use the Windows defaults.

So they resort to terrible UIs and behavior like what I described here. They have to be both devious and in-your-face, so that you are not silently taken away to the MS-bundled elements over time, and you remember that it's McAfee (or whoever) that's protecting you, rather than Microsoft.

I think that most technical users will find most of this stuff completely confusing and they will either a) end up with a machine where nothing works, without knowing why, or b) end up disabling all the security.

These are several bad effects that are all clearly tied to the implementation of SP2 (and dependent to some degree on the different products that people already had). Bad UI. Confusing features. Software that's difficult to use, and crippled so that it's "secure".

I am sure that Microsoft can do better.

PS: In the end I did leave SP2 installed. After a few more reboots, the drunken leprechauns magically started to work properly again, and that was that. The only other strange thing was that Windows Messenger refused to start on reboot, and has been behaving erratically since I upgraded (when I say "erratically" I mean exactly that. Yesterday, for example, it kept showing its right-click menu no matter what I did anywhere on the screen. I had to kill it. Today, nothing's wrong. Two days ago, there were two taskbar icons for the same Window).

And yes, I have scanned for viruses, just to be sure that the source of the weirdness is not something else. :)

dasher

Last Saturday, during a conversation with Santiago he showed me dasher running on his Linux laptop (Here's an article on it from The Economist).

It is amazing. At first it is relatively weird to use, but that feeling goes away quite quickly. And then you start "typing" away without problems (I haven't tried adding a training file to it though, the basic package worked well enough for simple testing). But you won't really know until you try it. Go check it out--they have binaries for most platforms.

If this isn't squarely part of the future of typing on mobile and input-restricted devices (aside from Speech/Handwriting recognition and gestures), then I don't know what is.

live from EuroFoo

I just realized that in the madness of the last few weeks I neglected to mention that I had been invited to EuroFoo. I got here yesterday evening after a ten-hour trip (there were no delays, but since it was bus+plane+train it was nearly inevitable that it would take a bit).

I spent some 24 hours in electronic isolation, between the trip and subsequent rest. It was good.

Now there are about 50 of us gathered in what I'd say is the restaurant of the hotel that (I heard this morning) we've basically taken over. The hardware/human ratio is something to behold. Powerbooks are the majority, or nearly so.

We should "officially" start any minute now. More in a bit.

'comments off for now' cont'd.

Email I got (thanks all!) regarding my post yesterday comments off for now pointed to a number of solutions, most of which I knew. I neglected to mention those yesterday, so here they go:

• Adrian pointed me to this entry on his blog on which he describes a solution similar to what I was discussing yesterday--adding a field to the form, while what I was saying was to change the names of the fields that already exist. Combining extra fields with field name changes and script name change should be a pretty good deterrent I think.
  
• To close comments after a period of time, Tima's mt-closure is good but as far as I can see it rebuilds every entry it changes the values on, which is a problem with my slow machine. Ideally, it would save the values and then perform a complete rebuild, similar to what Tima's own mt-rebuild does.
• Jeremy's mysql-compatible autoclose comments script--there are a couple of others with similar functionality, and in any case I haven't been able to try them since I use Berkeley DB.
• Some suggestions centered around mt-blacklist, but I think that "blind" (i.e., depending on blacklists that are widely distributed) blacklisting of any kind (IPs, terms, etc) is a bad, bad, bad idea, inelegant to the extreme, and not very effective. If any proof is necessary, it should be enough to note that blacklisting has been done for years on email. And we can all see how well it has worked...
• Bayesian filters, such as mt-bayesian are another alternative but again they are error-prone.

When I read all this I realize it seems like a long list of complaints. Clearly these solutions all have something to offer, but the reason I haven't started using the ones that have been around for a while is simply time. While I might not be sure of which solution is best, I am sure that I don't want to install a solution that not only doesn't work all the time but that then demands more work to solve the edge cases, however few they may be, or that mixes legitimate and illegitimate comments. That is, in my view, any solution should never mark a legitimate comment as spam, while it may mark a spam comment as legitimate. Closing comments is sort of an orthogonal approach to spam detection or prevention, and I haven't used that for reasons unrelated to the script (ie., machine speed).

Okay, enough with the rant. This is all useful information for that moment at some point in the (near?) future when I will have time to deal with this properly.

Internet != internet

Wired has announced that they are ditching the capital 'I' in Internet [via Dave] as well as the capital W in Web and the capital N in Net. While I don't have an opinion about the Web and Net cases, I think that the case for Internet is different.

An internet (lowercase 'i') was initially defined to be a connected set of (potentially different) networks, with the Internet (uppercase 'i') being the internet of all internets. So Internet is a particular name for a particular abstraction. True, this might be a definition that is of historical interest more than anything, or specifically pointing at the Internet construct, where Wired is looking at the Internet as a medium. A medium implies certain homogeneity, which, while true in practice in most of the Internet today, is not what the initial use of the term "internet" implied, is not true for research edge networks (and some commercial networks) connected to it, and was not true at the beginning when a protocol had not yet won over others as a standard.

In any case, I assume that in technical terms we will continue to make the distinction, since, strictly speaking, an internet is a subset of the Internet. :)

PS: Check out the various references in the History page at the Internet Society for more detail on the early terminology and naming.

comments off for now

So I had planned to blog a bit 'for real' today but maybe that time is past: I spent most of my "alloted" blogging time (and then some) deleting spam comments. Twice in the last week d2r has been under what has to be described as a massive spam attack. A bot systematically going through every page (loading information with a Mozilla client ID so that it looks as if there was a read before posting a comment, and probably scanning the page for the comment ID) and then posting garbage to up the ranking of whatever crap of the day they're selling.

The way to stop it has been to simply remove, for now, the comment script as I look for a solution. I've found several, mostly directed towards mysql backends (which I don't use, maybe I should) but always when trying them something doesn't quite work. Also, I don't want to spend much time looking for a solution (probably switching to a faster server is part of that).

One thing I was thinking is that these scripts obviously have to rely on standard MT configurations to be effective. This means fields IDs, form name, things of that nature. Before, I could stop them by changing the name of the comment script, but they have (predictably, I might add) adapted to that by scanning the page before posting. But if the comment form uses names for the fields that are non-standard, as well as pointing to a different URL, I think the only way to post comments should be at the webpage itself, by a person that can recognize the form, since the elements that allow scripts to recognize them automatically wouldn't be there. I recoil at the idea of digging through the MT sources to find that, but maybe I'll do it. Certainly MT could come with a screen to configure the names for your setup, in that way every blog would have its own form names and format and it would be, I think, quite difficult to post comments automatically.

A simple concept, but I think it should work, no?

the new phone

So about a week ago we got a couple of smartphones to start doing some work on them and to understand their capabilities better: a Nokia 6600, and a Sony Ericsson P900, which I got.

While I understand the technologies, capabilities, etc, and have played with them in emulators, I haven't actually owned a smartphone until now. I'll write a more detailed review later (I wanted to play with it for a reasonable amount of time first), but here are some first impressions.

As far as phone functionality, the P900 is quite good. The transition from other apps to the phone is a little awkward but manageable. It comes out of the box with miniheadphones and a microphone, which is crucial considering the phone is at first a little too big to use as a normal phone (later I got used to it though).

Then there's the connectivity: Internet and bluetooth (why oh why doesn't it come with WiFi!). Opera comes in the tools CD, which is nice, but you actually have to install it. Once everything is set up (and I'll dwell on that particular point in a later entry) it works fairly well, usable for many situations, but I don't even want to think how much money it costs to see a simple webpage. Using mobile versions of things like the Google for palm site helps a lot. Too bad it's not so easy to find sites that support that (Opera, btw, is amazing at fitting regular websites in such a tiny screen without completely destryoying the design). Bluetooth works fine, but it's much too slow for file transfers (but this is probably due to the phone rather than Bluetooth's intrinsic speed). The phone also comes with a USB cradle that doubles up as charging station, which is also good, but the transfer speed is also bad, which confirms that it's the phone that is the bottleneck.

Storage: this phone uses the braindead Memory Stick, and Sony in all its wisdom has released a rash of incompatible versions (Regular, Duo, Pro, Duo Pro...). The P900 supports Duo which is limited at 128 MB--ok for a phone but not to store media. This is quite a limitation, and unnecessary IMO.

Software: decent list of basic apps, and of course access to a lot of Symbian apps. Only once, after playing with it for a while, it complained of "low memory" even though I had very few files open. Had to be a memory leak: cue Purify for the Symbian guys. A restart cleared the problem. I also got Task Manager, which has turned out to be a critical tool for looking at running apps, closing them, etc. Don't know if there's a better solution that this.

Finally, media: built-in camera for video and stills. Quality is decent, but I won't be leaving my digital camera behind any time soon. The phone can play MP3s, but the player software that comes with it is a joke (I haven't even found a way of creating playlists), and I haven't yet looked for better software--I don't even know if there is better software. This, coupled with the slow transfers makes it hard to use the phone as your regular MP3 player, but barring any other option it does the job.

Overall, pleasantly surprised on many levels, and a little disappointed at its media handling capabilities (considering this is one of the (if not the) highest-end devices running Symbian, I imagine the situation is similar for most other smartphones. I am probably being unfair in comparing it with devices that only perform a certain function, though. Then again, I don't think it's meant to replace those devices yet.

I got all the necessary tools to write code for it, but I haven't done anything with them--too busy. That'll have to wait for the next few days, or even next week. :)

feedster v2

Scott announced yesterday Feedster version 2. Excellent. It's now much faster and with several new features. Congrats to Scott & the Feedster team!

PS: by the way, I have a question: what does the version="2.0/XSS-extensible" mean in the feedster blog RSS 2.0 feed? I was recently doing a small study of feed types and out of 20,000 this is the only feed that identifies itself as that. Bug? Feature? Just wondering :).

comment spam, cont'd

On my previous post on comment spam Phil made a good point, that since I generally close comment threads once spam appears (because I can't afford to be removing comments and rebuilding too often) it makes for a Godwin's Law sort-of-thing, where someone could essentially "engineer" the closing of a thread by posting spam.

I hadn't thought of that! I can say though, that generally when I get comment spam posted I also ban the originating IP, so whoever did it will have problems posting in the future (and the IP should also let me see if a spam looks iffy by being the same as someone who just posted a comment). For people with dynamic IPs (most of us?) it won't necessarily work, and someone could re-login (to their DSL, etc) purposefully to try to change IPs... I guess it's not a perfect solution, but it's good enough under the circumstances...

comment spam

BTW, comment spam has been quite a problem recently, which accounts for the number of posts that have comments closed. Basically whenever a spam comment is posted I close the comments for the entry, which accounts for the randomness of the comments on/comments off posts. Normally I'd just delete them and be done with it, but Movable Type on my poor Celeron 700 server is dog slow and rebuilding an entry to remove the comment takes about a minute (I'm not exaggerating). Probably part of the problem is the fact that I am inching on to 2,000 posts and that makes MT slower (maybe) but in any case I can't be removing spam comments every five seconds, and that forces me to close the thread. Sorry about that.

burnout? no, just busy

So yesterday as I'm pondering why I haven't posted anything in a few days I read this wired article on blogger's burnout. Although I've experienced lack of blogflow before, this time it was something different: just being too absorbed into what I was doing to do anything else.

So what was I doing? Simple: working on a new release of clevercactus pro. Not that share is taking a back-seat or anything, mind you, this is something that we had planned for a while and finally there was time to do it. (The release will be out sometime next week).

Anyway--I have had a couple of posts swirling in my head for a couple of days now, so I'll get to that now. :)

apology accepted

Jon Udell quoted me on a piece for infoworld but somehow my name ended up being "Diego Rivera". I sent him and email and in a few hours he replied apologizing (the change will be online at some point in the near future, but the ship has probably sailed on the print copies), and he posted a correction on his weblog.

At the risk of sounding hokey, I find it a small honor to be quoted by Jon, mistaken attribution and everything (and hey, there are worse things than being confused with Diego Rivera!)--as I've said before, I learned many things from his columns, going back to the days of Byte.

It's often been said how weblogs "push back" against media (big or small), but not a lot has been said about how media itself can use them to improve itself (or maybe I just haven't read a lot about that). Had this happened on a print-only medium, a correction would probably have taken a week or more, plus no one would see it because there's no context. Of course, Jon is ahead of the curve on this, but we can hope that feedback loops of this nature eventually become the norm rather than the exception.

So, thanks, Jon, and apology accepted. :)

tracking a hoax

Wired has an article Copy This Article & Win Quick Cash! that tracks down the (by now almost ancient) "forward this email and get paid by Microsoft" hoax. The author manages to track down the originator of the hoax, who says it started out as a joke and quickly "got a little out of hand." So why hadn't he claimed 'authorship' before? He replied: "It's just a hoax. And if I admitted to it, why would anyone believe me?" --to which I can only add: Indeed.

tiger--when?

No, no that Tiger. I'm talking about OS X v10.4. Reading the feature list makes me want to install it right now, but there's no information on the page of a release date (News.com has it for "early next year"). Maybe by the next WWDC?

Update: Russ and Erik have some good comments on Apple's inclusion of a concept originally developed for OS X by a small developer, Konfabulator. Am I wrong, or this also happened with another tool for the release of Jaguar or something? A Watson add-in? Funnily enough, Dave notes this New York Times article in which Jobs is quoted as saying "They're copying our concepts, [...] I'd kind of like to get credit sometime." Hm.

stone hackers

Sean McGrath: The ancient art of hacking in Ireland. Heh.

((no single == many) && (no single != no)) point(s) of failure

So today an outage of some sort at Akamai's distributed DNS service brought down access to some major sites from various parts of the world, including Google, Yahoo, and Microsoft. Pretty quickly, as evidenced by this slashdot thread the questions over how the days of "no single point of failure" are over started to pop up.

The myth of the Internet being so resilient that it would never fail is an interesting one. More accurately, its a set of layers of myths, that go back to the often-repeated idea that "the Internet was designed to survive a nuclear attack".

One of the crucial ideas of ARPAnet was that it would be packet-switched, rather than circuit switched. With packet-based communications, clearly the packets will attempt to reach their destination regardless of the circuit used, and there is no question that packet-based networks are much more resilient to failures than circuit-switched networks.

Let me be clear: part of my argument is semantic. That is, the fact that packet-switching means "no single point of failure" doesn't mean that there are no points of failure at all. The problem, however, is that we end up ignoring the word "point" and reading "no failure". The idea of "no single point of failure" eventually ends up implying "failure proof". Which is why we are so surprised when a systemic failure does occur.

ARPAnet, however, never qualified as a failure-proof network, and the points of failure were few enough that "no single point of failure" had little meaning. In the early days you could literally take out most of the Internet by cutting a bunch of cables in certain areas of Boston and California. With time, yes, more lines of communications where available, reducing the probability of failure even further, but even today the amount of trans-continental and intercontinental bandwidth is certainly not infinite.

But, ok. Let's concede the point that a systemic failure at the packet-switching level is of very low probability in today's Internet. What about the services?

Because it is the services that create today's Internet. And many of the services that the Internet depends on are centralized.

Take DNS. Originally, name resolution ocurred by matching names against the contents of the local hosts table (stored in /etc/hosts) and when a new host was added a new hosts table was propagated across the participating hosts. Eventually, this process became impossible, since hosts were being added too fast. This led, in the 80s, to the development of DNS, which eventually became the standard.

DNS, however, is a highly centralized system, and it was designed for a network a couple of orders of magnitude smaller than what we have today. The fact that it does work today is more a credit to sheer engineering prowess in implementation, rather than design, although the design was clearly excellent for its time.

Even today, if the root Internet clusters (those that serve the root domains) where to be seriously compromised), the Internet would last about a week until most of the cached DNS mappings expired. And then we'd all be back to typing IP numbers.

And it doesn't stop with DNS. What if Yahoo! was to go offline? What if Google vanished for a week? What if someone devised a worm that flooded, say, 70% of the world's email servers?

For users, the Internet has now become its applications and services rather than its protocols. And the applications and services leave a lot to be desired.

What's missing is a shift at the service and application level in all fields, routing, security, and so on (Spam is just the tip of the iceberg). Something that brings the higher levels of networking in line with the ideas of packet switching.

So, today, Akamai sneezes and the rest of the world gets a cold. Tomorrow, it will be someone else. This will keep happening until the high-level infrastructure we use everyday becomes decentralized itself. Only then the probability of systemic failure will be low enough. Low enough, mind you, not non-existent: Biomimetism and self-organization, after all, don't guarantee eternity. :)

cringely on weblogs

[via Dave] The most recent Cringely article is on the topic of weblogs, and he makes a number of interesting points. The first is that

It takes society 30 years, more or less, to absorb a new information technology into daily life. It took about that long to turn movable type into books in the 15th century. Telephones were invented in the 1870s, but did not change our lives until the 1900s. Motion pictures were born in the 1890s, but became an important industry in the 1920s. Television, invented in the mid-1920s, took until the mid-1950s to bind us to our sofas. The PC and the Internet are both today about 30 years old, which means we are finally figuring what they are about.

While his numbers match, I have to say that I find the logic faulty. I don't think that it takes 30 years for people to "figure out" what something is good for (although the lag between early adopters and the public-at-large is definitely there), I think that the 30 years he points to is more a measure of the economic and cultural evolution of a particular technology and its acceptance rather than whether people "figure out" what something is good for or not. Also, Cringely is measuring using a US-centric view, when you go outside it becomes easier to see that sometimes a technology (or technology/science mix) evolves through different timescales. Take, for example, Norman Borlaug's work on "distilling" and then "exporting" dwarf wheat, which provided one of the keys to allow densely populated countries such as India to (quite literally) feed themselves. If you start counting with Mendel's work in the late 19th century to Watson & Crick's work on DNA, it becomes difficult to find 30 years that fit neatly in that timeline. Even for more "technological" achievements, such as air travel, it's difficult to find the 30 years anywhere. Generation and use of electricity is the same thing. And even for Cringely's examples, like TV, we could say that even though they became more used in the 50s in the US, it wasn't until the 60s, and Kennedy's assasination, that television's power was really apparent. On the other hand, it didn't take anyone 30 years to figure out that digital music had quite a number of advantages, as anyone within reach of a computer can attest.

While I do argue against his calendar-based absolutism and my contention that it's economic and curltural forces, rather than "figuring out" what something is or does, I think he has a point: technologies, or anything that affects our cultural behavior for that matter, do have an adoption curve measured in years and sometimes decades.

However, in using the "figuring out" imagery, he also implies that the technology necessarily is creating something new, and that, I think, is wrong. In many cases a technology is simply facilitating a process that already existed, and it's only after a while (sometimes quickly, sometimes slowly) that the facilitation of previous behavior leads to entirely new processes.

And, once in a while, in facilitating something that previously existed, a technology will simply bring that to the fore, help us rediscover something that might have been lost or pushed aside in the evolution of things.

That, I believe, is the case with weblogs.

For "proof" I refer you to another segment of Cringely's article:

Some people think this column is a web log, but it isn't. For one thing, it predates web logs and I'm hoping will post-date them, too. Google News classifies what I am doing here as a web log even though I predate Google, itself, by more than a decade and don't see my work that way at all. I use too darned many words to be a web log, for one thing, and too darned few links. If I write anything really newsworthy, which I like to think that I do from time to time, the only way Google News will show it is if one of their 4500 REAL news sites mentions me. Otherwise, I don't exist, or more properly I exist only in a blogosphere that I, in turn, refuse to acknowledge. I'm odd that way.

His oddity notwhistanding, the fact that he has been doing whatever he was doing before weblogs came along doesn't mean he's not weblogging, or doing something that shares qualities with weblogging. As I said in my intro to weblogs, some people are "natural-born bloggers". Cringely's style of writing always had a foot on the world of weblogging: personal, opinionated, timely but not necessarily timeless. Even his book, Accidental Empires, which is excellent and I've read more than once, tilts towards qualities we generally associate with weblogging--not that weblogs "invented them", it's just that weblogs share some qualities with certain styles of (dare I say it?) journalism, or, more generally, writing.

But most of all, I'd point to the fact that his column includes a picture of his newborn baby.

Now, where have we seen that before? :)

comPost

Dylan has released comPost, a bookmarklet to post to a variety of weblogging tools that integrates with Internet Explorer. He used it to post the announcement, so you know it works. Rockin'!

Now, if I could only get it to work with FireFox, that would rock tenfold! :)

ps: the name is... less than appealing. I get the pun, but I think a cool tool as this one is would spread faster with a cool name: "coolPost" sounds too obvious and probably too cheesy, "anyPost"... maybe. allPost? blogPost? lightPost? Anyway, you get the idea.

airtunes

Sound over WiFi, simple, and useful, as well as other things. Read more here.

First reaction when I heard about this: oh yeah. :)

quote of the day

"If any person can edit a webpage, so can any robot." [source].

the PDA field shrinks

Sony is exiting the PDA market in both the US and Europe, it seems. There original vision wasn't bad, it was simply too proprietary and couldn't keep up with non-proprietary solutions. On the handheld side, PDAs are not going to vanish anyway, they'll just be morph with Smartphones and devices of the sort (although I think there will always be a market, however small, for "pure" PDAs). Maybe Sony is planning on reviving some of the Clie stuff through its cellphone venture with Ericsson...

solar! wind! power!

If only solar power was more commonplace, and less of a luxury. Another problem is that solar cells today still take too much energy to be manufactured and they don't produce much over their lifetimes making the situation, energy-conservation-wise, something of a wash (or worse). Yes, they aren't practical in many situations. But they are still useful--and cool.

Here in Ireland it's much easier to use wind. Here's a map of some current wind farms, and a new wind farm is being built south of Dublin. When completed, it will be the largest in the world (200 turbines, three times the generating capacity of all current offshore wind farms worldwide).

not aliens--just some wireless sensors

[via Slashdot] Area 51 'hackers' dig up trouble. Quote:

[...] it turns out the truth really was out there, and the government didn't appreciate Clark digging it up.

Clark didn't find the Roswell craft or an alien autopsy room -- in fact, while officially shrouded in secrecy, the 50-year-old base is generally believed to be dedicated to the terrestrial mission of testing classified aircraft. "The U2 spy plane, the SR-71, the F-117A stealth fighter, all were flight-tested out of the Groom Lake facility," says Steven Aftergood, director of the Federation of American Scientists' Project on Government Secrecy. The myth of Area 51 memorialized in films, T.V. shows and novels is a function of the secrecy that surrounds it. "It is a concrete manifestation of official secrecy at its most intense, and that invites a mixture of paranoia and speculative fantasy that has become ingrained in popular culture," says Aftergood.

Even without aliens, the facility has its secrets, and last year while roaming the desert outside the Groom Lake base Clark stumbled upon one of them: an electronic device packed in a rugged case and buried in the dirt. Marked "U.S. Government Property," the device turned out to be a wireless transmitter, connected by an underground cable to a sensor buried nearby next to one of the unpaved roads that vein the public land surrounding the base. Together, the units act as a surveillance system, warning someone -- somewhere -- whenever a vehicle drives down that stretch of road.

Makes sense of course. I think similar technology is used around other high-security facilities, like Cheyenne Mountain (NORAD's Operation Center). Speaking of NORAD, recently I read that NORAD is a binational military organization, and its commander is appointed by (and responsible to) both the US President and the Canadian Prime Minister. Although it's the North American Aerospace Defense Command, that surprised me for some reason...

on spam

Lately I've been more irate than usual at the increasing volumes of spam my server (and inbox) has to deal with. A couple of interesting articles on News.com recently on this topic: Who owns your email address? and Attack of Comcast's Internet Zombies, which give their take on different parts of the problem.

muglia on longhorn

This also interesting from last week: a CNET interview with Bob Muglia, Senior VP at Microsoft, on Longhorn Client, Server, and almost everything in between. A little more information on WinFS (to add to my comments last week): apparently WinFS will be on Longhorn server, but it's unclear that it will show up on the client. The more I see the progression of the backtracking of announcements, the more it seems to me that the problem is that the initial announcement was too vague. Maybe MS would do well to not talk so much about features that haven't coalesced yet.

pearpc

PearPC, a PowerPC architecture simulator for PCs. (News.com article). Check out the screenshots as well. Speed is of course a problem, and it stands to reason that simulating a RISC architecture on a CISC processor would naturally be worse than simulating CISC over RISC (as Virtual PC does on the Mac). The News.com article notes that speed is about 2.2 percent of the machine it runs on. But it's still cool. :)

code that kills

Tangentially related to my ethics and computer science rant from some time ago, Scott Rosenberg has an interesting article in Salon on the problems of software on military systems. Quote:

"When everyone decides for themselves what frequency to use, what protocols to use, what standards to use, then you get systems that don't talk to each other. And it's killing us."

That sort of lament is a staple at technology conferences, and its dire language is usually a matter of executive hyperbole: Somewhere in corporate America, perhaps, a bottom line is breathing its last, and we're supposed to care.

But when the speaker is in uniform, and the incompatible systems he's describing belong to the armed forces, then you sit up straight in your seat and realize that the words are meant all too literally. As Adm. Michael Sharp of the U.S. Navy went on to say, in a talk last month in Salt Lake City, "Software errors, timing errors, can get real critical -- killing the wrong people, or not killing the right people and leaving our people unprotected."

Here's David Cook, senior research scientist at Aegis Technologies: "It has been said that, without software, the F-16C is nothing more than a $15 million lawn dart. There are stories that I know for a fact of airplanes that have been flying cross-country that land at a base that's not where they're supposed to land, and while they're there, somebody modifies the software. And the airplane flat stops in midair when they turn on the radar unit. Why? Because there are incompatible versions of this certain piece of radar software, one of which they never thought would be on that particular model."

$15 million lawn darts indeed. Tiny problem is, lawn darts usually don't run around at Mach 2, or come loaded with AIM-9 Sidewinder missiles. And that's just one example...

kapor on the google IPO

Mitch Kapor writes about the terms of the Google IPO. A good read.

the devil is in the details

In one of those random occurrences that are bound to happen everyday I notice there's a strange referer in my long. It looked something like this (And yes, it's not my typo):

http://ww.google.de/...

So I think about it for a second and I realize that this is something eminently reasonable for google to do, namely, make sure that common mispellings are taken care of so that users can get to what they want instead of seeing an unnecessary error. But then I got curious.

As we all know (?), DNS resolution depends on the dots of the names to separate host name from domain from master domain (e.g., .com or .org). If the machine name isn't configured in the DNS server then it will fail. So it requires a conscious act to support multiple (apparenlty invalid) domains, such as ww or w or whatever. But how where different systems dealing with this? How pervasive was the practice?

Google, for example, supports w.google.com, ww.google.com, but strangely, not wwww.google.com (which seems to me to be another possible common misspelling). Microsoft, in what would have to be characterized as their (perceived) usual disregard for end-users, supports none of the variants, either for their main site or for msn.com. So if you make a typing error, even a common one, such as leaving one "w", MS doesn't help you at all, you just get a browser error telling you the site doesn't exist (just like Google failing with four "w"s). Teoma supports all w, ww, www, and wwww (+1 for Teoma!).

Both Google and Teoma, however, leave you at the "wrong" address, which in my mind seems, well, wrong. Yahoo! goes one step further and redirects you to www.yahoo.com no matter what (Yahoo!, however, only supports ww, www and wwww, while sending you to an error page for only one w used). Overall, Yahoo wins in my book.

However, I wonder, is it really wrong when a user types it with one or two or four "w"s? Yahoo!'s behavior is to gently "correct you", but if you got to where you wanted, does it really matter? Hm.

Nevertheless, interesting stuff. The details, always the details...

an internet of ends

Yesterday I gave a talk at Netsoc at TCD titled "An Internet of Ends". Here's the PDF of the slides. There are many ideas that I think are in there that finally jelled in the last few days, ideas that have been buzzing around my head for quite some time but that I haven't been able to connect or express in a single thread up to this point. I thought it would be a good idea to start expanding on them here, using this post as a kick-off point.

Yes, more later. And as always, questions and comments most welcome!

CD-Rs: how reliable?

From the Independent an article on CD-Rs and their lack of long-term reliability:

You know those CD-Rs that you've trusted your most precious memories to? They could be little more use than coasters after just two years.

An interesting read, more or less along the lines of other things I've seen on the topic over time.

about privacy and stuff

I read this, then this, then this, then instead of ranting I just point to this I wrote a few days ago and then go back to whatever it is I was doing. :)

Yahoo! SoulSearch

Following yesterday's Google HeavenSearch post, the Onion has a story of its own along similar lines: Yahoo! Launched Soul-Search Engine. The return of the metaphysical? Hm... :-)

seriously though

So, before anyone throws a fit about my Google-HeavenSearch fake news item, I just wanted to say a couple of things.

No, I'm not dismissing any of the issues. I'm just saying that it seems to me pretty hypocrytical all the immense attention that is being given to the privacy concerns (which exist) regarding Google when there are way more pressing concerns in many other areas. Other search engines. Other identity systems. Airline reservation systems. Credit card systems.

Yes, even I haven't been always raising these issues in unison, but they should be. Privacy breaks down at the weakest point, not just for Google.

For example, lots of attention has been given to Google not "guaranteeing" that an email in GMail will actually be completely deleted. But let's be realistic. Have Microsoft or Yahoo! or AOL ever guaranteed that? Not that I'm aware of.

Writing in News.com, Declan McCullagh has one of the few fairly balanced views that I've seen on the topic. He rightly points to the real problem, which is centralized systems, not Google in particular. (See also here). If you want something to be fully private, then just don't use, say, a webmail account for it.

Some of the complaints, however, have had to do with the precedent that this sets. Precedent? When companies like Gator provide software that is installed on millions of machines and basically acts like a Trojan to track behavior? When most email messages exchanged travel unencrypted through the Internet?

Come on.

Yes, these are real issues. But it's not a Google problem alone. It's structural to the kind of service being provided, part of its nature. I've tried GMail (I will hopefully have time to comment on that specifically later) and Google is doing something basically similar to others, improving on several respects (and a number of very cool ideas), changing others, and still lagging behind on some.

For me the solution is clear: decentralization, plus end-to-end encryption at the application level based on public-key infrastructure. Centralized systems have their pros and cons, as anything--and it seems that these days it is too easy to imagine that they have to be good (and perfect) for everything, and even more, that a single company has to be responsible for fixing all that's wrong in the world.

Okay, diatribe finished. :-))

google announces HeavenSearch, partially disclaims deity status

This from the fake-as-news dept:

(for immediate release)
MOUNTAIN VIEW, CA, April 2004.

Google Inc. revealed today a new product called HeavenSearch (http://heaven.google.com/) that, in accordance to the company's oft-discussed reach into theological depths, will allow users to search the information contained in Paradise and alternative otherwordly venues.

"We're really excited about this product," said a Google official. "People have been talking for a while about whether Google is God and so on. And they're not totally off the mark. Our cookies see everywhere and everything, even beyond Death. Beyond Taxes too. We wanted to make this wealth of information available to users." The initial product, released as a beta (as is standard Google practice), will start off by searching through the Christian Heaven. Plans are in place, however, to provide search facilities for other major Religions' pleasant afterlife locations.

And what about Hell?

"We're not going there," said a developer that worked on the project. "Our motto is 'Do no Evil'. Obviously that precludes searching through Hell," and then noted, "we've wondered about Limbo though."

Privacy advocates were outraged at the very notion. "This is a disastrous development. Between search, mail, shopping, news, and now religion, Google's cookies are becoming all-powerful entities. The Google cookie is the greatest threat to our way of life since Oreos were invented. The Pope should be worried about the Googleplex, too."

The Vatican declined to comment.

The Google Official wondered: "So we're terrible, but, say, AOL, or Microsoft, or Yahoo aren't an issue at all? Passport? The Windows Registration System? AIM? ICQ? MSN Messenger? Ads everywhere? Pop-ups? Pop-unders? Instertitial ads? Paid-for-search results? Credit Card information in a server in Redmond somewhere? Why is it that all this brouhaha applies to Google only?"

To which the privacy advocates replied (in unison): "Oh, because AOL, Microsoft, and everyone else basically are good companies. We have nothing to fear from them!". One of them added "Plus, those companies... you know, they're pretty harmless. They just have tens of billions of dollars in cash and dominant, locked-in, fiercely defended positions on their markets. Just look at Microsoft, they just had to pay Sun Microsystems something like two billion dollars. Now they've only got 54 billion left!." Another interjected. "Right. These companies aren't God or anything like that," after which he dropped to his knees and, eyes closed, head down, hands to chin, started mumbling search-engine queries.

"I see," said the Google Official when hearing this, as he dug out some M&Ms from an open bowl on the table, then proceeded to sit down at a nearby massage chair.

So is Google, really, really God now?

"We're not like, God-God, you know? After all, we had nothing to do with the creation of the Universe and all living things." The official said, his voice vibrating in unison with the massage chair, disclaiming Google's incipient deity status, and added cryptically "That was there before us."

gmail

So, finally, Google will release its email service today (NYTimes, News.com). This follows last week's launch of Google's new Look and some Lab features like Web alerts. One interesting thing: Google calculates the cost of providing a gigabyte of email storage at $2, which I presume includes the processing I bandwidth required to use the service.

The privacy question will now become even more complex:

At Google, one official said, the company has engaged in an intense debate over how extensively to exploit the content of e-mail.

Many people inside the company are worried that users might fear that the content of their e-mail messages could be used to tailor individual advertising messages, much as ad messages are now placed on pages tied to specific responses to search inquiries. Google hopes to quell any such concerns by assuring users that the content of their messages will remain private.

not to mention other additions:

Eric Schmidt, chief executive of Google, said he ``absolutely'' has plans to integrate Orkut into Google's search engine.

Another interesting thing, from the New York Times article:

It will be "soft launched," they said, in a manner that Google has followed with other features that it has added to its Web site, with little fanfare and presented initially as a long-running test.

This has a new definition of the word "soft launch"... what with an article in the New York Times and all... :)

As far as this being an April's Fools Joke... I found this press release by Google which does sound a bit, well, iffy. But then again there's a website for gmail which looks very much like the real thing (Including privacy policy, terms of use, etc). If this is indeed a joke, then some significant effort has gone into it (note that the articles include quotes from Google employees, so if it is a joke we have to presume that either a) the journalists are in on it or b) the employees continued with the 'joke' while giving unnatributed quotes). There are no disclaimers in any of the pages, and the HTML sources look clean too. Andrew (for example) is skeptical, to say the least :). I guess we'll have to wait and see.

it's not as easy as it seems

An excellent Salon article triumph of the telcos, on what we rarely hear about developments in VoIP:

[...] the battle with Big Telecom is one front in a wider war on the oligarchies that dominate the world economy. But to the oligarchs themselves that war is a mere sideshow. The real fight is between Big Telecom and Big Cable, with both sides using Internet telephony as a weapon.

names, or lack thereof

James Gleick, writing in the New York Times: Get Out of My Namespace. Quote:

The world is running out of names. The roster of possible names seems almost infinite, but the demand is even greater. With the rise of instantaneous communication, business spreading across the globe and the Internet annihilating geography, conflict is rampant in this realm of language and of intellectual property. Rules are up for grabs. Laws regarding names have never been in such disarray.

Indeed. And who hasn't encountered this problem? Related to this (though more technical), the book Ruling the Root is excellent.

small movabletype tip

I just discovered that my RSS feeds where publishing my email unencoded. The culprit turned out to be an <$MTEntryAuthorEmail$>. As the MT Template manual explains here, this can be easily solved by adding a spam_protect parameter, as follows: <$MTEntryAuthorEmail spam_protect="1"$>. Useful and simple.

boomerangs

Gavin Sheridan posted two days ago on being threatened with legal action by author John Gray. This threat came about because of comments that Gavin made in this post which basically referenced this other post on a different weblog, give or take a few words (a few words which, btw, seem to be at the crux of the matter). Now, I saw this early today over at Karlin's weblog and was going to comment on it, but over the topic was already "in play". Dan Gillmor has commented on it. Kevin Drum has comments. Dave has comments. And on and on and on it goes.

The posts in question are from November last year. In terms of the web, they were long gone. It's difficult to see them resurfacing in any meaningful way.

Except, of course, if you did exactly what John Gray's attorneys have done.

First, it seems that the threat could potentially be in murky waters legally (It appears that it's not an open and shut case, both in what relates to the statements, how they were made, and even matters of jurisdiction).

But that's not the end of it. One would assume that Gavin's post was found through a web search, and that this threat of legal action was intended to remove those references from search engines and such (after all, if something can't be found or read by anyone, where's the problem?). However, I will bet that within a couple of weeks both Google and Yahoo! (along with other search engines) will return posts and pages related to this story when searching for "John Gray" or any number of keywords (who knows maybe this entry will show up even), whereas currently that doesn't happen at all. So instead of "solving the problem" (assuming there was a problem in the first place) this action has had the effect of multiplying it by several orders of magnitude. It is even possible that this is picked up by the media somewhere. And you can imagine the headlines, right?

So why would someone do something like this? Mystery. After all, we've already seen what can happen when us weblog-folk get, um, "agitated"... :)

links, influence, and networks

According to this list at BlogRunner that contains "the most influential reporters and bloggers on the web" I am in position 190. Apparently there were some complaints over the results (I say "apparently" because I have been a little removed from online conversations in the last few days), which led BlogRunner to calculate a revised listing changing the parameters by which "influence" was calculated (for example, "penalizing" by posting frequency). I thought I wouldn't show up in the new list, but there I'm in position 128!

Yeah, of course I'm surprised. But that's not the point of this post. :)

Now, I do read many of the journalists and bloggers listed there. List #1 is clearly "skewed" towards bloggers, while list List #2 would be skewed towards journalists (If I'm higher on #2 than on #1 then, does that say anything about me?). The author, Philippe Lourier, responded to comments by Daniel Drezner which noted the problems on the first list (and resulted in changes for the second list).

Philippe's comments make it clear that he doesn't think there's a silver bullet for this, and he probably agrees with Dave, who says that it "proves that trying to quantify influence pretty hard to do, and maybe not so important". (My emphasis).

Not so important in absolute terms, I agree, but definitely useful if we could just put it in the context of a single person, that is, creating (privately) my own network of influence.

A starting point would be to delve a little more into the word "influence". Influence is, in this case, a possible effect of the number of links, and so this ranking (or similar others) is more an indicator of possible influence rather than a direct measure of it (btw, I'm not saying anyone suggested otherwise, I'm just clarifying my POV before continuing).

I've been thinking a lot recently about this, in the context of social software (or, as Anne prefers to call it, sociable software). There are roughly three levels of "influence" that we can readily identify in our daily lives:

• Personal: friends, family, etc. We are naturally more inclined to take into account and listen to ideas and opinions that come from people we know and trust.
• Community: Community spheres are generally multiple for any given person: your neighborhood, group of online friends, bloggers you know, local politics, etc. These involve people that you don't know very well but that see their trust level increased because of the context. That is, you may not know A very well, but if B, C, and D all say that A has got a point, you might be inclined to take the idea into consideration. This could also be a form of "soft" peer pressure.
• Media: I say "media" here lacking a good term to describe influences that are massively distributed and have (more often than not) global reach. Here the context is almost all that matters, as what makes you take the time to consider something is the forum in which it is published.

Each sphere also moves, incrementally, from action to reflection. In my personal sphere my opinions (reflection) can affect my actions directly, but less so when dealing with communities, while at the media level pretty much all I can do is rant about it and then take it as information that can filter back into my other spheres of interest. The interaction between the spheres however, is largely opaque. If I read a number of articles that move me to take action, exposing that context into the other spheres will be difficult (although people that know me very well might know "where I'm coming from").

Nice theory, but so what?

First, I think that weblogs+RSS correlate and bind these three spheres of influence tighter than before. Within my weblog I can include pointers to elements of any of the three spheres, and provide a better context for each. A lot more people in the two spheres that really matter to me as an individual ("personal" and "communities") will be able to know "where I'm coming from". Which in turn changes the dynamics of communication within those spheres of influence, both online and off.

But let's backtrack a bit further though. What are these "lists" useful for? Aside from the fleeting ego-trip they provide :), I see them as useful for finding new "spheres of interest", another form of the old "related links" or "other people who have read this also read...". In other words, community formation and (re)shaping.

With weblogs those interconnections are decidedly faint (which is why it's so hard to come up with these lists), because they in their basic form don't include relationship-value information to the embedded links. Also as a consequence of their nature, weblogs have so far defied easy definition of absolute spheres of influence, and so what weblogs are and do hasn't been overshadowed, yet, by lists of various types. Anything that relates to weblogs, in this sense, is useful in some way. To find new ideas, new discussions, long-lost groups, or what others are saying about things that interest you.

Which (finally!) brings me to the point.

What would happen, for example, if we overlapped FOAF information with those links? Wouldn't we be able to obtain a clearer picture of influence as it relates to a given person? Granted, that by itself wouldn't tell us much about the overall influence of something, but that would be a meta-value that could be obtained based on the aggregate of all those values. But for most people, their own spheres of interest/influence, and that of the people they trust the most, would be enough.

More generically, this goes back to what I said a few days ago about social networks being the glue for next-generation internet applications, as it is one more example of how the personal networks defined by these applications can give new meaning to the data that's out there that we generate, and that relates what we and others create, providing us with new windows (and ways to navigate) the vast sea of information that is the Internet, with the goal of doing something truly useful for us in concrete terms, in improving communication and exchange of ideas.

The network is not the computer. The network is not the person. And because it's neither one or the other, it can help in using the first, and help us, in small ways, in "being" the second.

Lists of connections are nice, but only as far as they are useful for something. And, as with other things, weblogs lead the way.

PS: and, before I forget, happy St. Patrick's day to all! :)

timeouts in safari

While working on the Mac my default browser is Safari, and I'm very happy with it. It is fast, looks good, and its tabs implementation is excellent. However, there was this annoying 60-second default timeout for webpage loading that apparently could not be changed (which I hit often when posting, as a combination of my slow server, lots of templates on my weblog, and movable type's way of doing things, but of course the primary reason is the server speed, no question about it). A bit of searching reveals that this is a well-known problem in Safari, and most of the search results point to a fix: SafariNoTimeout 1.0 which I've just downloaded and installed--and seems to work just fine, even in Safari 1.2.

Now, what I don't get is: have the Safari engineers (which I presume use their own browser) never, ever found a page that took more than 60 seconds to load? Weird.

christopher allen on social software

Christopher Allen of Alacrity Ventures comments on social software. A good read. I've mentioned similar things in the past in other contexts, so I agree with a lot of what he says. Maybe more later when I've fully digested it [via Scoble].

two weeks with a mac

It's been almost two weeks since I got my new development machine, a Powermac G5, so I thought I'd write down some of my impressions during this time.

The first Mac I used was at my first job, one of the original Powermacs (PowerPC 601) with System 7. The experience was good, but System 7 was very good at developing System 7 apps, and running System 7 apps, but interoperability was hard. I remember I used to spend nearly all my time running X within it, since most of the work I was doing then was in Java, or C++ targeted at various UNIX platforms.

Then I used a Mac on and off over the last year (an old G3 with OS X) that we got on loan to test our software, but with only 128 megs of RAM it wasn't possible to do more than launch a program (wait... and wait... and wait...) and see how it came out. Serious debugging (of problems mostly related to layout problems) was pretty much impossible.

The G5, of course, changed all that.

Btw, you will have to forgive my sometimes starry-eyed commentary in what follows :). Even when I point out some of the problems that I've run against I sort of gloss over them; I'm sure that for others they will be more difficult to accept. So this is not terribly objective, but I think it's a good example of how the experience affected my judgment :).

First, the experience of setting up the machine is quite simply a pleasure from start to finish. The packaging is nice. Oohs and Aahs abound even as you open the box and get everything out. The machine looks nice. The cables have nice terminators that make them appear to "meld" with the machine.

Plus, there are surprises in the most unexpected places. For example, as I was setting up the LCD, I was wondering how to adjust its angle. I placed it on the desk, and looked at it for a couple of minutes. Nothing. Looked behind. Nothing that seemed to indicate how to rotate it (Mac LCDs stand on an inverted V, as opposed to PC LCDs which generally have a single stand with a swivel). I refused to look at any manuals. (Not that the Mac has many of those anyway :)). I lifted it up and tried to move it (gently), to no avail. I set it down. And then it happened: for whatever reason I pushed it slightly from the top border.

It moved.

I pushed it further, the monitor's angle was reduced accordingly. I pushed it from behind, and the angle increased.

The utter shock of that moment can't be easily explained. Here was a mechanism that simple, understated, and that worked properly when it had to. The long hours that must have gone into the excellent design of something as small (that could easily be considered "inconsequential") dropped on me like a bucket of cold water. And we tend to ignore mechanical engineering. Like turning on the machine from the monitor: there are no buttons, you just slide your finger over an area on the bottom-right of it, and on it goes.

Then, later, almost everything was were it should be. Front sockets for USB, headphones, and so on.

At around that time, during the installation, there were two things that bugged me a bit. One was the overly intrusive registration procedure that I had to go through to get the machine up and running, and which I couldn't bypass. The other was the CD tray, which could only be opened by pressing a key on the keyboard (which took me about 2 minutes to figure out). I wondered how this could affect error situations for a bit, but then I let it go.

Once I was in, the machine was more than fast, it was instantaneous. Then again, I would have been dissapointed if that wasn't the case, considering the hardware (G5, 1 GB RAM, Serial-ATA disk...). It was nice nevertheless.

The default security settings of the machine were a joke (no password for logging in, no password for the screensaver, in fact, no screensaver set up) but I quickly changed that. For finding my way around the configuration, Mark's Dive Into OS X was a good guide. Many thanks to Mark for maintaining such an excellent resource.

Safari is fantastic, I downloaded Firebird but after enabling the tabs in Safari I simply had no need for it. Safari was the default browser out of the box and it has stayed that way.

Setting up a printer was so easy it felt like cheating. We have a LaserJet 2300 with JetDirect, and while Windows was confused about it (as usual) requiring CDs, looking through the network and so forth, on the Mac it was a two-click process. Go to add printers. The LaserJet shows up. Select. Done.

After playing with some settings I updated the software (Java, iTunes, etc) and then installed Eclipse and other things I needed for development, including Xcode and the X11 server. I've had little need of anything else since then, with the exception of Office (I tried installed OpenOffice for Mac beta but it was a disaster, and it needs X11 to run... I'll just wait until they put out the native Carbon version), which meant that at some point I'll have to get Microsoft Office for Mac, and VirtualPC (also a Microsoft product). I was going to get a copy of VirtualPC in fact, but I found out that it doesn't run on G5s (!!). I guess I'll have to wait for that one.

Now for the little annoyances: coming from both UNIX/Linux and Windows, Macs are a little too "opaque". It is hard to know what's installed, and where. It's even harder to know with certainty how to uninstall things. I know that in 99% of the cases just dumping a program's folder into the Trash is enough, but what about programs that have registered themselves as MIME handlers for example? Is that taken care of automatically? I often ended up wondering if things were properly uninstalled, and sometimes checked the list of services running to make sure nothing was left as a UNIX-style daemon somewhere.

As far as the Finder is concerned, there are several inconsistencies in the navigation, and default settings are generally hard to find. Mostly it's a matter of knowing how to do something, rather than wondering if it can be done at all (Example: taking screenshots, or creating PDFs among other things).

The Command+Tab functionality that was added in OS X (which has existed in Windows as Alt-Tab for years and years) is nice, and the incredibly useful (and incredibly cool) Exposé feature is a godsend which basically negates the need of using virtual desktops.

All in all, a great experience so far. The little navigational inconsistencies might become more annoying as time goes by, but for the moment, I can live with them.

Good stuff. :)

re: ethics in cs, and identity

Anne mentions an article in the recent ACM Crossroads (I have to see if I can get a version from the digital archives, thanks to my ACM membership. Which, oops, I just remembered I have to renew!). She makes some similar points to what I discussed a few days ago.

She also points to a summary of the Urban Tapestries project which says:

The key features defining the relationships our respondents had with ICTs are the importance of control (or lack of it), socio-cultural contexts, expectation management, external or internal locus of control, and personal aesthetics.

It is clear that respondents used UT in order to negotiate boundaries and mark their territories, stake claims and identify their personal preferences ... In this sense, public authoring promotes a sense of control not only over users' territories, but also over their boundaries and their own role in those territories.

It's all too possible that I'm projecting too much of my own thinking on these issues (because it could be argue that those conclusions are not specific enough--to which I'd say that you have to put them in the context of the project), but I find that data very encouraging. I might not be that crazy after all. :-)

can you spell "hoax"?

I'll try to describe my thinking process in the two (2) seconds that followed reading this (and I recommend you get to the end of this post, where the mystery is revealed!).

As linked in the previous paragraph, Seb at Many2Many has posted a link to a message on the reputation mailing list where Orkut is "outed" as a "Master's thesis" of a random person who (they say) works for Orkut (the man, not the site).

Let's see. You are Google, right? You are a 1500+ person company, one of the most respected in the world, that is soon to be going public. So when a developer comes up from somewhere saying they want to do a "Master's thesis" using the company's name and reputation (including a link of "in affiliation with Google), you say "suure, go ahead". Furthermore, the student wishes to remain anonymous, so another developer (Orkut) is recruited (somehow) to use this own name and take all the credit and responsibility.

So far so good, right?

Now you launch, and after a few days pass the experiment is a success. So this student (presumably Eric Schmidt at this point, or his alter-ego) tells Google to throw a party (and he gets random people to talk about it afterwards) in a posh San Francisco location for hundreds of people. Since Orkut (the man) is still the "patsy", the party in question is announced as Orkut's idea, and makes it coincide with his birthday. To this, Google says "but of course! My pleasure!" and happily pays for the expenses (alternatively, this student is also a millionaire and pays out of his own pocket).

Luckily for everyone involved, someone posts a message to a mailing list quoting an "article" from an unknown writer with no links to an organization of any kind (even a personal site) to back it up.

But of course, it must be true!

Now you look for the source, and you discover it's this page at HACT (What do you mean what's HACT? you mean you've never heard of it? What rock have you been hiding under?).

Oh, but wait, this is the same page that, at the bottom, says: "Please note that this is a humor article and is not true in any way, shape or form, except in that it rings true in a scary way".

Damn. It wasn't true I guess. I thought that my explanation above was so incredibly reasonable, so universal, that you could post happily in indignation about it without thinking twice.

Hm. So you mean I shouldn't post about that message I read somewhere that said that Orkut in reverse spelled the name of the Alien race that actually sent this student to Stanford to do get a Masters and discover what the earthlings are up to?

PS: I find it interesting that something so patently unbelievable could disseminate at all without a ton of smileys and LOLs before and after the text.

what's in an IPO?

Wired has a good article in their latest issue: The complete guide to google, which actually starts by talking about the challenges any company goes through when pre-, in- and post-IPO. Just the first section alone makes it worth reading. Oh, yes. It talks about Google too. :)

the lack of an ethics conversation in computer science

In April 2000 Bill Joy published in Wired an excellent article titled Why the future doesn't need us. In the article he was saying that, for once, maybe we should stop for a moment and think, because the technologies that are emerging now (molecular nanotechnology, genetic engineering, etc) present both a promise and a distinctive threat to the human species: things like near immortality on one hand, and complete destruction on the other. I'd like to quote at relative length a few paragraphs with an eye on that I want to discuss, so bear with me a little:

["Unabomber" Theodore] Kaczynski's dystopian vision describes unintended consequences, a well-known problem with the design and use of technology, and one that is clearly related to Murphy's law - "Anything that can go wrong, will." (Actually, this is Finagle's law, which in itself shows that Finagle was right.) Our overuse of antibiotics has led to what may be the biggest such problem so far: the emergence of antibiotic-resistant and much more dangerous bacteria. Similar things happened when attempts to eliminate malarial mosquitoes using DDT caused them to acquire DDT resistance; malarial parasites likewise acquired multi-drug-resistant genes.

The cause of many such surprises seems clear: The systems involved are complex, involving interaction among and feedback between many parts. Any changes to such a system will cascade in ways that are difficult to predict; this is especially true when human actions are involved.

[...]

What was different in the 20th century? Certainly, the technologies underlying the weapons of mass destruction (WMD) - nuclear, biological, and chemical (NBC) - were powerful, and the weapons an enormous threat. But building nuclear weapons required, at least for a time, access to both rare - indeed, effectively unavailable - raw materials and highly protected information; biological and chemical weapons programs also tended to require large-scale activities.

The 21st-century technologies - genetics, nanotechnology, and robotics (GNR) - are so powerful that they can spawn whole new classes of accidents and abuses. Most dangerously, for the first time, these accidents and abuses are widely within the reach of individuals or small groups. They will not require large facilities or rare raw materials. Knowledge alone will enable the use of them.

Thus we have the possibility not just of weapons of mass destruction but of knowledge-enabled mass destruction (KMD), this destructiveness hugely amplified by the power of self-replication.

I think it is no exaggeration to say we are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, on to a surprising and terrible empowerment of extreme individuals.

[...]

Nothing about the way I got involved with computers suggested to me that I was going to be facing these kinds of issues.

(My emphasis). What Joy (who I personally consider among some of the greatest people in the history of computing) describes in that last sentence is striking not because of what it implies, but because we don't hear it often enough.

When we hear the word "ethics" together with "computers" we immediately think about issues like copyright, file trading, and the like. While at Drexel as an undergrad I took a "computer ethics" class where indeed the main topics of discussion where copying, copyright law, the "hacker ethos", etc. The class was fantastic, but there was something missing, and it took me a good while to figure out what it was.

What was missing was a discussion of the most fundamental problems of ethics of all when dealing with a certain discipline, particularly one like ours where "yesterday" means an hour ago and last year is barely last month. We try to run faster and faster, trying to "catch up" and "stay ahead of the curve" (and any number of other cliches). But we never, ever ask ourselves: should we do this at all?

In other words: what about the consequences?

Let's take a detour through history. Pull back in time: It is June, 1942. Nuclear weapons, discussed theoretically for some time, are rumored to be under development in Nazi Germany (the rumors started around 1939--but of course, back then most people didn't quite realize the viciousness of the Nazis). The US government, urged by some of the most brilliant scientists in history (including Einstein) started the Manhattan project at Los Alamos to develop its own nuclear weapon, a fission device, or A-Bomb. (Fusion devices --also known as H-bombs-- , that use a fission reaction as the starting point and are orders of magnitude more powerful, would come later, based on the breakthroughs of the Manhattan Project).

But then, after the first successful test at the Trinity Test site in July 16, 1945, something happened. The scientists, which up until that point had been too worried with technological questions that they had forgotten to think about the philosophical ones, realized what they had built. Oppenheimer, the scientific leader of the project, famously said

I remembered the line from the Hindu scripture, the Bhagavad-Gita: Vishnu is trying to persuade the Prince that he should do his duty and to impress him he takes on his multi-armed form and says, "Now I am become Death, the destroyer of worlds."

While Kenneth Bainbridge, in charge of the test, later said at that time that he told Oppenheimer:

"Now we are all sons of bitches."

Following the test, the scientists got together and tried to stop the bomb from ever being used. To which Truman said (I'm paraphrasing):

"What did they think they were building it for? We can't uninvent it."

Which was, of course, quite true.

"All of this sanctimonious preaching is all well and good" (I hear you think) "But what the hell does this have to do with computer science?".

Well. :)

When Bill Joy's piece came out, there was a lot of discussion on the topic. Many reacted viscerally, attacking Joy as a doomsayer, a Cassandra, and so on. Eventually the topic sort of died down. Not much happened. September 11 and then the war in Iraq, surprisingly, did nothing to revive it (contrary to what one might expect). Technology was called upon in aid of the military, spying, anti-terrorism efforts, and so on. The larger question, of whether we should stop to think for a moment before rushing to create things that "we can't uninvent" has been largely set aside. Joy was essentially trying to jump-start the discussion that should have happened before the Mahattan project was started. True, given the Nazi threat, it might have been done anyway. But the more important point to make is that if the Manhattan Project had never started, nuclear weapons might not exist today.

Uh?

After WW2 Europe was in tatters, and Germany in particular was completely destroyed. There were only two powers left, only two that had the resources, the know-how, and the incentive, to create Nuclear Weapons. So if the US had not developed them, it would be reasonable to ask: What about the Soviets?

As it has been documented in books like The Sword and the Shield (based on KGB files), the Soviet Union, while powerful and full of brilliant scientists, could not have brought its own nuclear effort to fruition but for two reasons: 1) The Americans had nuclear weapons, and 2) they stole the most crucial parts of the technology from the Americans. The Soviet Union was well informed, through spies and "conscientious objectors" of the advances in the US nuclear effort. Key elements, such as the spherical implosion device, were copied verbatim. And even so, it took the Soviet Union two four more years (until its first test in August 29, 1949) to duplicate the technology.

Is it obvious then, that, had the Manhattan project never existed, nuclear weapons wouldn't have been developed? Of course not. But it is clear that the nature of the Cold War might have been radically altered (if there was to be a Cold War at all), and at a minimum nuclear weapons wouldn't have existed for several more years.

Now, historical revisionism is not my thing: what happened, happened. But we can learn from it. Had there been a meaningful discussion on nuclear power before the Manhattan Project, even if it had been completed, maybe we would have come up with ways to avert the nuclear arms race that followed. Maybe protective measures that took time, and trial, and error, to work out would have been in place earlier.

Maybe not. But at least it wouldn't have been for lack of trying.

"Fine. But why do you talk about computer science?" Someone might say. "What about, say, bioengineering?". Take cloning, for example, a field similarly ripe with both peril and promise. An ongoing discussion exists, even among lawmakers. Maybe the answer we'll get to at the end will be wrong. Maybe we'll bungle it anyway. But it's a good bet that whatever happens, we'll be walking into it with our eyes wide open. It will be our choice, not an unforseen consequence that is almost forced upon us.

The difference between CS and everything else is that we seem to be blissfully unaware of the consequences of what we're doing. Consider for a second: of all the weapon systems that exist today, of all the increasingly sophisticated missiles and bombs, of all the combat airplanes designed since the early 80's, which would have been possible without computers?

The answer: Zero. Zilch. None.

Airplanes like the B-2 bomber or the F-117, in fact, cannot fly at all without computers. They're too unstable for humans to handle. Reagan's SDSI (aka "Star Wars"), credited by some with bringing about the fall of the Soviet Union, was a perfect example of the influence of computers (unworkable at the time, true, but a perfect example nevertheless).

During the war in Iraq last year, as I watched the (conveniently) sanitized nightscope visuals of bombs falling on Baghdad and other places in Iraq, I couldn't help but think, constantly, of the amount of programs and microchips and PCI buses that were making it possible. Forget about whether the war was right or wrong. What matters is that, for ill or good, it is the technology we built and continue to build every day that enables this capabilities for both defense and destruction.

So what's our share of the responsibility in this? If we are to believe the deafening silence on the matter, absolutely none.

This responsibility appears obvious when something goes wrong (like in this case, or in any of the other occasions when bugs have caused crashes, accidents, or equipment failures), but it is always there.

It could be argued that after the military-industrial complex (as Eisenhower aptly described it) took over, market forces, which are inherently non-ethical (note, non-ethical, not un-ethical), we lost all hope of having any say in this. But is that the truth? Isn't it about people in the end?

And this is relevant today. Take cameras in cell phones. Wow, cool stuff we said. But now that we've got 50 million of the little critters out there, suddenly people are screaming: the vanishing of privacy! aiee!. Well, why didn't we think of it before? How many people were involved at the early stages of this development? A few, as with anything. And how many thought about the consequences? How many tried to anticipate and maybe even somehow circumvent some of the problems we're facing today?

Wanna bet on that number?

Now, to make it absolutely clear: I'm not saying we should all just stow our keyboards away and start farming or something of the sort. I'm all too aware that this sounds too preachy and gloomy, but I put myself squarely with the rest. I am no better, or worse, and I mean that.

All I'm saying is that, when we make a choice to go forward, we should be aware of what we know, and what we don't know. We should have thought about the risks. We should be thinking about ways to minimize them. We should pause for a moment and, in Einstein's terms, perform a small gedankenexperiment: what are the consequences of what I'm doing? Do the benefits outweigh the risks? What would happen if anyone could build this? How hard is it to build? What would others do with it? And so on.

We should be discussing this topic in our universities, for starters. Talking about copyright is useful, but there are larger things at stake, the RIAA's pronouncements notwhistanding.

This is all the more necessary because we're reaching a point were technologies are increasingly dealing with self-replicating systems that are even more difficult to understand, not to mention control (computer viruses, anyone?), as Joy so clearly put it in his article.

We should be having a meaningful, ongoing conversation about what we do and why. Yes, market forces are all well and good, but in the end it comes down to people. And it's people, us, that should be thinking about these issues before we do things, not after.

These are difficult questions, with no clear-cut answers. Sometimes the questions themselves aren't even clear. But we should try, at least.

Because, when there's an oncoming train and you're tied to the tracks, closing your eyes and humming to yourself doesn't really do anything to get you out of there.

so long, ZIP

I'm spending a couple of hours today retiring the few backups I still have on ZIP (250 MB) and moving them over to CD-R. ZIP is just too slow for large amounts of data (at least compared to 48X CD drives) and keeping two separate mediums (CD and ZIPs) is too much of a pain. Plus, CD-R is simply too inexpensive these days to justify ZIPs (can't say I've tried the new 750 MB ZIPs, but I'm not inclined to either). I'm disconnecting the drive (which I haven't used in the last 3 months) and leaving it there for one of those just-in-case situations. After all, it is light and easy to carry around... so it's useful when making backups on the road (unless your notebook has a built-in CD-R that is--mine doesn't).

It was good while it lasted, ZIP was a great technology in its early days, and it certainly had a good 3-4 year run, considering how fast things move in storage technologies. Now to wait for the day when DVD-Rs replace CD-Rs...

the new yahoo search

Yahoo! has finally dumped Google as its search technology for Yahoo! Search. I like it!.

This happened as predicted a little more than a month ago. The new Yahoo! search design had been active for some time already, but using Google for results.

Now consider that Google itself is working on a new design (here are some screenshots of what it might look like, via Aaron). The new Yahoo search looks like a more modern version of what Google does today (at least to me, this is of course subjective). But Google might be changing its design soon. So Yahoo! will end up looking like a "nicer" Google, and Google will end up looking like something else. Funny, isn't it?

I can immediately tell that the results it provides are very good. Comparisons with Google's results for similar keywords show similar (though a bit slower) speeds, less obsession with trackbacks and such, and a good mix of weblog and non-weblog results. I got the Firebird/Firefox plugin for Yahoo! search and replaced my current default (Google of course, although I tried Teoma for a while, it didn't work as well). Let's see if the results are consistently good enough that they convince me to switch.

Plus: here is the link to find the search plugin for the Firebird/Firefox search bar. (Look for "Yahoo" on its own).

not everything that shines is made of gold...

Over the last few days an interesting story has developed in the US marketplace, namely Vodafone's bid for AT&T Wireless and then Cingular's counter-bid (Cingular won today). The economist has a couple of interesting articles on it (see Who's the real winner? and Vodafone's dilemma), noting that AT&T Wireless might be less of a prize than one might think at first sight. Problems are not only related to technology integration (AT&T Wireless runs two networks, on different technologies) but also to cost and the real potential of the US market.

The technology is moving so fast that business models are also very susceptible to shifts (e.g., is it content they're selling? Bandwidth? Hosted services? A platform? All of the above?), and so making it much more risky to potentially get stuck with old-to-new rather than new-to-next generation transitions. In my view, Vodafone might have actually been lucky in losing this bidding war. It's not just subscriber numbers that count.

Plus: some good comments on the topic over at Wi-Fi Networking News.

more on demo 2004

Lots of cool announcements for Demo 2004. Big focus on weblogs and decentralized communities (which I find to be intimately linked with weblogging, in spirit at least if not in practice). As a follow up to my previous post on WaveMarket's release of location-based moblogging tool (here's Russ's own entry on the topic). Doc has a good set of pointers to what went on, but here are a couple of other things that caught my attention:

• Feedster showed their search technology and Feedster builder, which is very cool. Congrats to Scott Johnson and the Feedster team! What they're doing with Feedster shows IMO that if the oft-maligned semantic web ever arrives it will be in the form of the gathering of information formed by decentralized self-organizing communities who provide context to infer the semantics, rather than forcing people to enter them on their own. Yes. Like weblogs are.
• SixApart demoed a new set of moblogging tools that work both with TypePad and MovableType. Kudos as well. I must get a decent mobile phone and start playing with this new cool stuff.

If weblogging, RSS, syndication (and related technologies) didn't come of age in 2003 (which I guess some would argue, though not me), then 2004 looks like a good bet, don't you think?

location-based blogging on mobiles

Congratulations to Russ and WaveMarket for the release of their new location-based blogging/information WaveIQ sharing system at DEMO 2004!

From the press release:

WaveIQ consists of three software products, all now available:

• WaveSpotter—a cellular map interface that allows users move about, letting users drill down to street level and post or consume blogs.
• WaveBlog—a company-hosted super blog serving as a multiple channel informational clearinghouse engineered by uber blogger Russell Beattie, WaveMarket director of blog engineering.
• WaveAlert—wireless operator infrastructure that allows users to be notified whenever they enter or leave a designated area. The server software powers a scalable system that reduces network loads and hardware requirements.

Uber-blogger eh? :-)

Sounds very cool. Congrats again, Russ. And, when can we try this on for size? :)

glancing

At ETech Matt Webb presented Glancing (slides here):

Glancing is an application to support small groups by simulating a very limited form of eye contact online. By small groups I mean about 2 to a dozen people.

Which covers part of the often overlooked area of underlying and implicit group dynamics, rather than the more overt and explicit kind. Anne has some interesting comments on it too. Very interesting ideas. Will have to think more about this (I've been saying this a lot recently both here and to myself, which is probably a measure of how many other things I have to do... so many things going on this week... :)).

movabletype and db versions

Dylan has a great post on recovering from a database version change that left his MT weblog data inaccessible. Got me thinking about my recent brush with disaster, and the possibility of moving to mysql (I didn't know BerkeleyDB had problems with large DBs, and my weblog is well over 1,500 entries right now). Not with this server but, since I'm planning to switch servers soon maybe I'll do it then.

mythtv

Adding to the List of Cool Things I Didn't Know About: MythTV. Sam explains the different things he's tried to get the system running. I had seen/read of other DIY PVR systems or projects, but nothing as sophisticated as what MythTV appears to be. Something else to keep an eye on.

faifzilla.org

Just found faifzilla.org, home to Free as in Freedom: Richard Stallman's Crusade for Free Software, a biography of sorts of Richard Stallman. Read bits and pieces of it, very, very interesting. And: an essay on the online book by Eric Raymond (linked from the main site).

The pluses of random web navigation... :)

mt-rebuild: rebuilding movabletype from the command line

My attempt yesterday at doing a full rebuild ended in pathetic failure as the normal load on the machine plus the Rebuild process meant that the page never got to the second stage. This was clearly a problem with timeouts on the web browser (through which MT is 100% controlled) because of the speed at which the process happened, rather than the process itself. So I spent some time today looking for a way to manage MovableType from the command line. I had done this before a few weeks ago but didn't get anywhere, this time I had more luck and I quickly found Timothy's excellent mt-rebuild: The rebuild script to end all rebuild scripts, which solved my problem (it did take a few hours to do a full rebuild though, which has nothing to do with the script and everything to do with the machine's load and speed) with a simple command of the form "mt-rebuild.pl -mode="all" -blog_id=xx". Only comment I'd have is that it doesn't seem to have a switch to provide feedback, so you don't know what's going on, but so what, it's not as if it's a consumer application or anything.

Yes, this is old hat (release date was almost a year ago) but I missed it when it came out and we know how it is with the web and its tendency to bury yesterday's news under a new avalanche of discussion, comments, posts, news, and other interesting stuff :).

This is exactly what I needed, thanks Timothy for making it available!! His other MT plugins are pretty cool too, including mt-publish-on, which I'll check it out when I have the time, since I've talked about something like it before.

Good stuff.

my wired | tired | expired

Since I thought the latest wired | tired | expired (which I linked to in the previous entry) was pretty lame, I decided to write my own. :) Here it goes.

Wired	Tired	Expired
Ultra-wideband	Bluetooth	IrDA
Peta	Tera	Giga
Mobile broadcasting	MMS	SMS
BitTorrent	TiVO	TV
Radians	< 6 degrees	6 degrees
RSS ads	AdSense	Banner Ads
Introverster	Orkut	Friendster

microsoft and google

Still catching up on some of yesterday's articles that I left open for reading later (does it show?). From the New York Times comes the shocking (shocking I tellsya!) revelation that Microsoft is taking on Google. Seriously though, quote:

"We took an approach that I now realize was wrong,'' [Bill Gates] said of his company's earlier decision to ignore the search market. But, he added pointedly, "we will catch them.'"

"We will catch them." Simple and to the point, don't you think? The comparisons with Netscape are the order of the day of course. Yahoo! gets a short mention (less than what it deserves IMO, after all, they are probably the one company aside from MS that has the technology reach and depth in the area to be a big factor, as they are in fact today--AOL doesn't quite have the tech know-how to make the list, even if they have the millions of users. Still, there's a few other interesting tidbits of information in the article that make it worth reading.

server platforms: choice, or lack thereof

One of the things we're doing in the process of upgrading our infrastructure is getting a new public server and a development server. And one of the biggest questions is (as usual) which platform to base it on.

The first problem that comes up is that pre-install choices for OSes are pretty much limited to either Red Hat Linux or Windows Server 2003 (In some cases Windows 2000 Server). For obvious reasons, the development server should be a copy of the deployment server, which means that if, if you have different providers, you have to settle for the "intersection" between the various offerings.

Note: all of these comments are taken from the POV of multiplatform server applications (Java, Perl, etc) with small server clusters. That is, if I say, "Windows and Linux are roughly similar in X" it means roughly similar in that context. Java in particular is sort of an equalizer in that sense, in fact helping Microsoft by taking Windows Server more up to par (who would have thought?). Additionally, it's only been in the last few years that Sun has been more aggressive in ensuring parity between platforms, there was a time (say, 6, 7 years ago) when Sun's Windows VM was the best all-around VM (remember it took a while for Sun to implement Java native threads on Solaris?). I have no doubt that in other contexts, for other types of webapps or webservices, or at different scales (say, deploying 100 servers instead of 10) both the parameters and the results of comparing these choices would be quite different.

Some might say that Windows "is not an option", but I prefer to make decisions based on objective information when possible (with personal preference a factor, of course, but not an overriding one), and I think it depends on how much money/resources you've got to deal with it. As far as one Internet server is concerned Windows is more expensive than Linux but not by a huge amount (as a comparison, Dell.ie pre-installs Windows 2003 Server for Euro 700, and RH9 for Euro 170). But when you deploy it in an internal network you have to start thinking about Client Access Licenses (CALs) which cost $50 a pop or more, installation licenses, and so forth, and this is where the "resources" come in: if you install Windows you need someone to spend a lot of time figuring out licensing and making sure that you are using the licenses properly, etc. So aside from Windows being more expensive, it's also more difficult to manage from the licensing point of view. Additionally, going beyond a few servers complicates matters even further. Needless to say, startups usually will not have the time for that. I know we don't.

A few years ago I deployed+used Windows Server in the company I was working for then and it worked okay (that is, it did what it had to do, nothing earth-shattering). The company was basically using Windows on the client (IE was the primary target platform) and that sort of dictated that the servers be Windows as well, particularly because there weren't that many servers. Downsides were mainly that you had to keep up with the neverending stream of security updates and the licensing stuff, but in that case there was a person who took care of the IT infrastructure.

I haven't used Windows 2003 Server, but Windows 2000 was pretty stable (Again, all of this in the context of multiplatform server apps that connect to an SQL DB in the backend). I think it's interesting that now that Windows is more on par in terms of stability and features (clustering, remote terminal, etc) with Linux what really becomes a bigger barrier of adoption is both the price and the "management" of licenses. Linux is just easier in that sense: make as many copies as you want, install as many servers as you need, with as many clients as you like. Microsoft is, in effect, shooting itself in the foot (for a certain segment of the audience at least) by making their licensing more convoluted than it should be.

Now, though, the comparison has become a little more complicated, since Red Hat has discontinued Red Hat Linux and split the process in two. On one hand we've got Fedora, which is sort of the "spiritual heir" of RH9, and on the other hand you've got Red Hat Enterprise Linux (RHE for short) which is their commercial offering. RHE comes in three flavors: WS (Workstation), AS (small-medium server) and ES (enterprise servers). As far as servers are concerned, the main differences between AS and ES are (btw, the information on RH's site isn't nearly as clear as it should be):

• AS supports only x86, while ES also supports Itanium, AMD64, and others.
• AS supports only one processor, while ES supports 1+.
• ES has round-the-clock tech support, while AS's is more limited.
• ES supports more than 8GB of memory on x86, while AS does not.

The problem with any of these options is, of course, that you have to pay a subscription that on the surface rivals the price of Windows Server. I say "on the surface" because the licensing is simpler, you basically pay the subscription, you get the updates when you want, and that's it. One subscription per install, and everything else is pretty much Linux as usual, which is substantially simpler (and so less costly) than Windows. (Btw, Russ also had some comments yesterday on his views on the Red Hat transition).

Ah, but why not go with another Linux distro you say? SuSE? Gentoo? or Debian? (Debian has lots of fans :)). Why not FreeBSD? Well, here we are back to what I mentioned at the beginning, that many providers pre-install either Windows or Red Hat. They don't preinstall, say, SuSE, or whatever.

Fine, you'd say: get a clean machine and install your favorite free distro yourself.

Which is an option, yes, but, but... if you are getting dedicated servers that are pre-installed on a remote location you're faced with the prospect of either a) going to the remote location to do the reinstall or b) doing the reinstall remotely, without being at the console. Both are possible (maybe b is not in some cases) but neither is very appealing, especially when you want to work on what you need rather than spend time making the OS boot properly. If we had more time to work on that things might be different, but that's not the case.

So.

For the moment, Red Hat is a better option for this kind of usage, since there is an upgrade path (even if it's convoluted) to either RHE or Fedora (although Fedora might be too much in flux as a distro to base a production system on--I guess that time will tell). If these moves by Red Hat make some hardware or Internet providers pre-install systems other than Red Hat there might more choice in the future (for when you need things to "just work"). And, if Microsoft changed their Licensing scheme to be something like "here, sign this two-page contract and pay us $500 a year and you get your updates and you can use this with as many clients as you want, etc" (kind of like what Sun did with their new licensing scheme) then Microsoft would be more of a contender in this area I think.

Oh, and btw, either Mac OS X Server or Solaris are indeed good options, but the hardware is simply more (sometimes a lot more) expensive, at least here in Europe, which makes it difficult to justify cost-wise. I hope that someday I'll understand why, if everything is built in factories all over the world, shipping stuff to Dublin instead of doing it to Palo Alto, CA makes prices jump 40%. Oh yeah, and software is more expensive too. I guess that the bits get tired from all that swimming and have to be compensated somehow. :-)

yet another email virus

"Fastest ever to spread", so far at least. Some coverage here from News.com. I have some thoughts on this, but will leave them for later (busy busy!).

the impact of the macintosh

On my entry about the Mac's 20th birthday there were a couple of comments that are interesting enough to echo here.

First, Chris posted a link to the 1984 commercial (thanks Chris!).

Second, Doug posted a long comment that I'll quote verbatim before replying:

As I recall, the '1984' commercial did not have much impact. It was advertising a product that most viewers had never heard of, but failed to introduce the product or suggest any of its benefits. The commercial was never run again.

Also, the Mac wasn't really that 'innovative'. It didn't have much of anything that the Apple Lisa didn't have... except that the Mac was at least somewhat affordable.

Finally, I would note that the 1984 Macintosh was generally considered to be a flop. Sales were abysmal when compared with the then-ancient Apple ][ -- it had taken 74 days for the Mac to sell its first 50K units, but when the IIc was introduced a few months later, Apple sold 50K of them in 7 hours. Worse, Mac sales were miniscule when compared with the IBM PC. The Macintosh failed to stop Apple's decline from #1 microcomputer maker to tiny niche player. The 9" black-and-white low-res screen and undersized keyboard marked it as a rich man's toy.

It was the introduction of the Mac II in '87, with available 13" 640x480 color display and a real keyboard, that finally gave Apple a system that was attractive to professionals.

What Doug says is all true. However, I disagree with his implied conclusion (that the Mac, or the launch even, weren't as important as they were).

Specifically on the points Doug mentions. The Lisa was about one third of the price of a Xerox Star. The Mac was one-fifth to one-fourth of the price of the Lisa. The Mac-to-Lisa jump was done in part due to hardware advances, but more importantly, due to top-notch engineering. Price matters. Also, the Mac improved on the Lisa in several aspects and to a degree it was an independent project that was running in parallel and that Jobs took over when the Lisa project started to sink under its own weight.

Regarding sales, well, the first Mac was in part underpowered (a year later the addition of a hard drive among other things made the product a much better proposition, and it sold accordingly), which hurt its sales. But at the heart of the difference in sales there's also the "Apple factor": not licensing the OS, using proprietary components, pushing for very high margins, etc., which had a big effect IMO.

As far as the impact of the 1984 commercial is concerned, I would just ask how many other commercials for computers are still known to the level that one is, and leave it at that.

Finally, if we start comparing things on the basis of what had already been proposed or developed to a degree (but not seriously marketed) before a product was launched, then the Lisa wasn't really that 'innovative' either because it borrowed a large number of concepts from the Xerox Star (which in turn borrowed from Engelbart's work), as I mentioned in the entry. There's a big differecence between doing something for a tiny audience or playing with it in a lab and designing it so you can manufacture hundreds of thousands of units a year of it. We could also argue (for example) that as crucial as the Mac II was the Laser Printer and PageMaker, which made the idea of "desktop publishing" a reality.

The original Mac planted the seeds for what was to come, and the Mac II and everything after it, on all other sides of the aisle (yes, for example, Windows), resembles to a large degree what was in that "rich man's toy" as Doug describes it.

A "desktop". Icons. Mouse. Graphical filesystem navigation. Applications running inside windows. Menus. Bit-mapped graphics. The idea of a common "Look and Feel", established through published guidelines (this one is fading now though, what with our modern skinnable apps and such :)). An API (The Toolbox) for developing apps against it, with high-level OS services. Most of these things existed before it, but the original Mac had it all, and in some respects it was several years ahead of its time (we all talk about computers as "appliances" now given the right context, but Jobs always saw the Mac as that, a machine that could sit comfortable in any living room without looking out of place).

What is sad is that we haven't really moved too far beyond those basic ideas, particularly since many of them were not designed for the massive amounts of information we deal with today (say, the number of files on our hard drives) and so in some sense some ideas have been as much a problem as a solution. But that's how it played out.

The impact of the Mac is therefore, in my opinion, difficult to underestimate. It defined what computers should be rather than bringing up a fancier version of what they were. And that's what's important, I think.

Update: [via Peter] Andy Hertzfeld's Folklore page dedicated to the early days of building the Macintosh. Very, very cool.

apple: beyond the mac

An News.com article that looks at the past and present of Apple and how its recent move into "digital lifestyle" products might play out. Interesting read.

the mac turns 20

Almost forgot: January 24, 1984, was the launch of Apple's Macintosh. Yes, a lot of the ideas were already present in Xerox's Star, but the Mac did have include many inventions (I guess that today we'd call them "innovations") and it did mean that all of these things were within the realm of the affordable. More importantly, the Mac forced the rest of the industry to improve.

Some time ago I found online a copy of the famous '1984' ad that launched the Mac, directed by Ridley Scott. It's really great. I can only imagine the impression it must have caused at the time. I mean, a computer being advertised, quite literally, as part of a revolution? These days all "revolutionary" icons have been turned into marketing gimmicks (say, images of Che Guevara being used to sell T-Shirts, Jeans and such), but 20 years ago that must have been quite the thing to see. And to sell a computer no less.

One more thing: something interesting from this CNN article:

Twenty years ago, on January 24, 1984, Apple Computer launched the Macintosh. It contained virtually unknown features, including simple icons, and an odd little attachment called a mouse.

Many newspaper stories at the time had to include a definition. Silicon Valley's newspaper The San Jose (California) Mercury News, for example, described the mouse as "a handheld device that, when slid across a table top, moves the cursor on the Mac's screen."

Heh.

comment spam filtering - it's all about the IPs

Sam describes his new comment spam filtering system. Quote:

Then it struck me: from an ip address I have never seen before. Light bulb.

Hit and run. Strangers. People who never have been here before. These people are unlikely to be seen again.

And if they don't come back, it is not possible to have a two way conversation, is it?

So I set to work. I wrote a script to scan my Apache logs for everybody who has ever visited my weblog within the last week. Bots, aggregators, and an occasional carbon based life form, I make no distinction.

Then I add in everybody who has left a comment in the last ninety days. And not just ip addresses, but also urls.

All these people are welcome to comment freely.

Very, very cool idea! Use the logs to establish an implicit community effect, a kind of automatic self-updating whitelist. It leaves me thinking "and where else can this be applied?". Mmm...

rethinking weblogging

[via Danny] Bruce Ecker: rethinking weblogging (and everything else). Great piece. Hard to summarize. Go read it! :)

digital law conference @ Yale

The other day I got an invitation to the CyberCrime and Digital Law Enforcement conference to be held at Yale Law School, March 26-28. I won't be able to attend, but it looks interesting, with some good sessions and speakers. Mr. Lessig would seem to be missing from the roster of speakers, but Jennifer Granick (who is also at Stanford Law School) will be there.

lower DSL prices in Ireland?

Karlin reports that:

Eircom said today that it has applied to ComReg to further lower the price of its main broadband DSL product, from Euro 54.45 to Euro 39.99 monthly (both incl VAT). Good news; I can't imagine ComReg will refuse.

(That's from USD 70 to USD 50 at current exchange rates, for those on the other side of the Atlantic!). Good news indeed. Fingers crossed...

file sharing = piracy? Not really.

An interesting Salon article: Is the war on file sharing over?:

If one is willing to believe the happy talk from music business executives, the tide has finally turned against file sharing, thanks to the get-tough tactics employed by the Recording Industry Association of America.

Last fall, the RIAA began filing lawsuits against individual users of peer-to-peer trading sites, and the strategy, the RIAA says now, has paid off. The group is careful not to declare a final victory over file trading, but things are finally beginning to look up for a business long in decline, say industry representatives. After years of scoffing at copyright laws, Americans are finally beginning to understand the gravity of file trading's offense against copyright.

The article is interesting. But what I find most interesting is this automatic alignment that is made in the media discourse between file sharing and piracy. There are many, many uses other than those the RIAA defines as illegitimate for file sharing (note, I am not saying anonymous file sharing, although there worthy uses for that too). Sure, the media loves a good fight and that's why the focus on this comparison. But the uses of sharing should, can, and will move beyond those in dispute. And not just for files, either.

Why am I saying this? Well, can't you guess?

Stay tuned. :-)

the story of an outage

a tale of mistakes, backups, recovery (by a hair), and why permalinks are not so permanent after all

out·age (ou?tij) noun

1. A quantity or portion of something lacking after delivery or storage.
2. A temporary suspension of operation, especially of electric power.

   When I woke up yesterday after a brief sleep I started to log back in to different services and as I'm seeing something's funny with my server, Jim over at #mobitopia asks "is your site down?".

   Damn.

   As I checked what was happening, I could see that all sorts of things were not working on the server. I was starting to fear the worst ("the worst" in abstract, nothing specific) when I remembered that I had seen similar symptoms a couple of months ago, and back then it had been a disk space problem. I run "df" and sure enough, the mountpoint where a bunch of data related to the services (including logs) is stored was full (since November the number of pageviews a month has increased to over 200,000, which creates pretty big logfiles). As the last time, the logs were the culprits. Still half-asleep, I start to compress, move things around and delete files, when suddenly after a delete I stop cold: "No such file or directory".

   What? But I had just seen that file...

   I look up the console history and four rm commands had failed similarly.

   Uh-oh.

   I run "pwd". Look at the result. "That's not right...". I was not where I thought I was.

   At that point, I woke up completely. Nothing like adrenaline for shaking off sleepiness.

   I look through the command history. At some point in my switching back and forth from one directory to another, I mistyped a "cd -" command and it all went downhill from there. Adding to the confusion was the fact that I used keep parallel structures of the same data on different partitions, "just in case". I stopped doing that once I got DSL back in May last year, opting instead to download stuff to my home machine, but the old structure, with old data, remained. And, even more, my bash configuration for root doesn't display the current directory (the first thing I did after I realized that was add $PWD to the prompt, but of course by then it was too late).

   I had just wiped out the movable type DB, the MT binaries (actually, all the CGI scripts), the archives, and a bunch of other stuff in my home directory.

   I took a deep breath and finished creating space, and moved on.

   First thing I did was restart the services, now that disk space wasn't longer an issue. Then I reinstalled the binaries that I had just wiped out, which I always keep in a separate directory with some quick instructions on how to install them. That turned out to be a lifesaver, one of the many in this little story.

   After that I put up a simple page that explaining the situation (here's a copy for... err... "historical reference"), plus a hand-written feed and worked on the problem in breaks between work.

   Then I realized that all the links that were coming in from the outside (through other weblogs, google, etc) were getting a 404. So as a temporary measure I redirected the archive traffic to the main page through a mod_rewrite clause:

   RewriteRule /d2r/archives/(.*) /d2r/ [R=307]

   That would return a temporary redirect (code 307) while I got things fixed (one fire out! 10 to go).

   So what next? The data of course. When I came back to Ireland at the beginning of January I started doing backups of different things (a "new year, new backups" sort of thing), and I backed up all the server data directories on Thursday, and then on Saturday I did what I thought was a backup of my weblog data, through MovableType's "Export" feature. As things turned out, the latter proved useless, and it was the "binary" backup that saved the day.

   Why? Well, as I started looking at things, I went to MT's "import" command in cavalier fashion and was about to start when the word "permalink" popped up in my head. Then it grew to a question: "What about the permalinks?".

   The question was valid because my permalinks are directly based on the MT entry ids. Therefore, if an import changed the entry IDs, it would also break all the permalinks. I started cursing for not switching over to using entry-based strings for permalinks, but that didn't help. So I did a little digging and I realized that I was right. MT assigns entry IDs on a system-wide basis. So if you have multiple weblogs on the same DB (which I have, some of them private, some for testing, etc) OR if you have to recover the data from an export (which I had to do) you're out of luck. More likely than not, the permalinks will not work anymore. The exported file did not include IDs. Re-importing would generate the IDs again. Different IDs. Different links. Result: broken links all over the place, both within the weblog and from external sources.

   This is clearly an issue with the MT database design, which doesn't seem too well adapted to the idea of recovery. To be fair, however, I am not sure how other blogging software deals with this problem, if at all. I think this is one big hole in the weblog infrastructure that we haven't yet completely figured out, both for recovery and for transitions between blog software (As Don noted recently).

   This is when I started thinking that things would have been much easier if I had written my own weblog software. :) That thought would return a few times over the next 24 hours, but luckily I was busy enough with other things not to indulge in it too much.

   After looking online and finding nothing on the topic, I came to the conclusion that my only chance was to do a direct restore of the "binary" copy (that is, replacing the clean database with the backup directly) I had from last Thursday. I did the upload, put everything in place, and things seemed to go well, I could log in to MT and the entries up to that point where right where they had to be. So far so good. I was going to do a rebuild and I thought that maybe now was a good time to close off all comment threads in all entries (to avoid ever-increasing comment spam) and I spent some time trying to figure out how to use the various MT tools to close comments on old entries. However, they all seem to be ready for MySQL rather than BerkeleyDB. It wasn't a hard decision to set it aside and move on.

   So I started a full rebuild. The first 40 entries went along fine, albeit slowly. Then nothing happened. Then, failure. I thought for a moment that, for some strange reason, the redirect I had set up yesterday was causing the problem, so I removed it, restarted the server, and Tried again. Failed again. No apparent reason.

   I got angry for a second but then I remembered that the "binary" backup was of everything, including the published HTML files. Aha! I uploaded those,crossed my fingers, and did a rebuild only of the index files, and everything was up again. Actually, this was important for another reason, since the uploaded images that are linked from the entries end up by default in the archives directory, you need a backup of that or the images (and whatever else you upload into MT) will be gone if you lose the site.

   So the solution up until this point had been a lot simpler than I thought at the beginning.

   But wait! All the entries after last Thursday were missing, and I didn't have a backup for those. That was when RSS came to the rescue in three different forms: 1) I download my own feeds into my aggregator, so there I had a copy up to a point. 2) Some kind souls, along with their condolences for the problem, sent along their own copy of the latest entries (Thanks!!--and Thanks to those who sent good wishes as well). 3) Search engines, (Feedster was the most up to date--btw, it was Matt that suggested yesterday, also on #mobitopia, that I check out Feedster as a source of information, a great idea that really applies to many search engines if their database is properly updated), had cached copies that I could use to check dates and content. So armed with all that information I set out to recreate the missing entries.

   Here the problem of the permalinks surfaced again. I had to be careful on the sequencing, or the IDs wouldn't match. So I re-created empty entries, one-by-one, to maintain the sequencing (leaving them unpublished), actually posted a couple of updates of what was going on, and then I published the recovered entries as I entered the content and set the right dates.

   So. All things are restored now (except for the comments from the last week, which are truly lost--this makes me think that setting up comment feeds would be a good idea. However, that doesn't address how would I recreate the comments given what happened. Would I post them myself under the submitter's name? That doesn't seem right at all. Another problem with no obvious solution given the combination of export/ID issues with MT).

   What's strange is that there's been slight a breakdown in continuity now, because I did "post" some updates to that temporary index file, but it couldn't be part of the regular blogflow. Hopefully this entry fixes that to the extent possible.

   Okay, lessons learned?

   1. Backups do work. :) I am going to do another full backup today, and I'll try to set up something automated to that effect. (Yes, I know I should have done it before, but as usual there are no simple solutions, and then you leave it for the next day... and the next...). Plus, backups for MT installations, should always be both of the DB and the published data, to make reco