Today, Joel on Software wrote about Outlook 2007’s recent search performance fixes, and mentioned Lookout. I had a couple of thoughts about the article.
First – I’m so pleased to hear such nice things about Lookout when it hasn’t been updated for nearly 3 years 🙂 Thanks!
Second, Joel wrote that “Microsoft finally put Lookout up for download…”. He makes it sound like it was down for a long time; I think the total down period was between 4-7 days. I know people like to think Microsoft buys products to squash them, and it’s easy to cite this downtime as an example of it. But I know for a fact that was not true in this case. I hope people won’t conclude that.
Third, while lots of people really liked Lookout, it did lack some key features which were put into Windows Desktop Search. Most notable is that WDS actually indexes new & changed email in real-time. Google Desktop Search does this too, but Lookout never did.
But it brings up an interesting point which requires some background. Building incremental indexing in Outlook is quite tricky. The problem is that changes can happen in your exchange mailbox while Outlook is not running. When you next start Outlook, you need to make sure these changes are reflected in the index. Because Outlook/Exchange don’t have APIs to do this efficiently, the only real way to do it is to start combing through folders trying to find what changes exist. There are a whole bunch of nuances (cached mode, exchange mode, internet mode, online/offline, etc) and many APIs available (OOM, MAPI, CDO, ECS, etc). When you try to make it work across the 4 different Outlooks (OL2000, OL-XP, OL2003, OL2007), each one presents a new set of problems.
So, Lookout punted on real-time indexing, and elected a “wake up every hour when the user is idle and index then.” While this approach isn’t perfect, it is much much simpler. And, it turns out that users usually don’t need to search for things which were sent to you less than an hour ago (although some did complain). But the biggest benefit is that because it’s not trying to be real-time, it is much less prone to bogging down your system doing indexing. And “system hogging” is a problem which both WDS and GDS took a long time to get right. Reading reviews of these products indicates that even today, they still are pretty resource intensive.
In my opinion, this is a classic example of “Better is the Enemy of Good”. The indexer doesn’t really need to be real time, even though it is “better” to be real-time. If you weren’t confined by the limits of Outlook & MAPI, my thoughts on this would probably be different. The fact that people are still talking about Lookout after both Microsoft and Google have continued to work on real-time indexers for 3 years is proof to me that in this case, better is actually worse.