Recording Windows Media Streams

Yesterday, my housemate was on Newstalk 106, a national radio station, to talk about Ladyfest Dublin, which she's involved in. Being the naive-yet-helpful type, I offered to record this for her and mp3 it, so Ladyfest could offer it as a download on their website. In the process of doing this, I downloaded 16 different pieces of software, most of which were completely useless for the job I was trying to do. That's why I've written this, to help anyone else trying to do something similar.

By the way, if anyone has a simpler way, please let me know.

Once bitten, twice shy

My previous experience of mp3'ing a radio show involved recording it to tape (yes, I still use tapes), connecting from the headphone jack on my stereo to the 'line in' jack on my laptop, and recording that. Unfortunately, this resulted in a really crackly mp3, full of static. So I figured, this is the 21st Century, there has to be an easier way to do this.

And being cheap, the easier way had better be free.

A thoroughly modern, convoluted solution

Newstalk offer a Windows Media stream of their live broadcasts. I used the shareware Net Transport to record the stream. I believe the shareware version will only record 15 minutes of a stream, but I didn't check this out. Once it had recorded the stream, I exported it as a 2.14MB .asf file.

Next, I used asftools to create a .wav of the stream. However, the .wav it created was only 2.12MB, while the actual recording was approximately 14 minutes long. Clearly asftools uses some weird codec that, despite downloading 20MBs of codec packs, I just couldn't find. Their website wasn't much help either. It addressed the problems with the .wav files asftools creates, but suggests it's "a codec problem". So I'd have to find something else.

After searching for a good hour or so, I finally stumbled across [http://www.dbpoweramp.com/](DB Power Amp Music Converter). This was able to read the busted wav, and export it as whatever I liked; a .wav or a .mp3. I was finally getting somewhere. Since I still had some editing to do on the source before I put it up as an mp3, I exported it as a "proper", 145MB .wav.

I opened the .wav file in audacity and from there, was able to trim off the useless bit I'd recorded at the beginning. I also added a nice little fade-out, for good measure. Audacity was able to export this as either a .ogg or a .mp3 file. As much as my nerd side wanted to put this out as a .ogg file, my sensible side told me that we were going for as much cross-compatibility as possible, so I exported it as a 12MB .mp3 file.

You can hear the mp3 on the Ladyfest website

del.icio.us

For the past few months, I've become increasingly fond of del.icio.us. Plugged into any half-decent RSS reader (liferea being my RSS-reader of choice), it becomes an invaluable tool to help me stay on top of my game, exposing me to tools and advice I probably wouldn't normally have stumbled upon.

But until recently, I never saw the point of signing up for an account. To me, it was a link exchange - whoop-di-doo. I don't really have enough to contribute to something like this, I don't tend to come across things by chance that other people would find interesting. Finally, in the depths of last night, I realised its true purpose and how I could help make it better whilst simultaneously scratching many of my own personal itches.

When I browse around on the internet at home, there's a lot of stuff that I just don't have time to check out on anything but a 'high' level. I'm generally up until 1am, winding down by chewing email for a while and seeing what's happening in the world, but it's absolutely impossible for me to keep my concentration levels high for some of the things I come across. For instance, last night I came across a link that explains the Ten Mistakes Writers Don't See (But Can Easily Fix When They Do), but it being almost 1am, I couldn't really digest the information. Since it's nigh-on impossible (or at least, a whole bunch of work) to synchronise my bookmarks list at home with my workstation in the office, I began the process of emailing the URL to myself, to check it when I got into work today.

I stopped before hitting the "send" button as it finally dawned on me. I realised that I was completely ignoring the fact that I'd just gotten this link from del.icio.us - it would always be there. If only some bright spark could come up with a way for me to keep a track of all the stuff I liked on del.icio.us -- one giant all-encompassing bookmark -- where I wouldn't need to maintain a file on a number of seperate machines.

But wait! Some bright spark already set this up! If I set up my own del.icio.us account, I could add the link to it, and have it available no matter where it was. I'm almost certainly pointing out the obvious to some of you here, but to me, it was like someone had finally removed the forest and I was able to say "Oh, there are the trees".

This also has a number of other knock-on effects. Now, when I want to show someone something cool I'd seen that they might be interested, I don't have to go digging through all the various machines I use, I can just point them at my del.icio.us account.

It also has the added advantage of making my bookmarks infinitely sortable. In Firefox, I have a "useful stuff" folder, where I dump, well.. useful stuff. This is to stop me getting confused "hmm.. where is that useful linux drum sequencer program I found? Is it in 'linux' or 'music'?". Now I can have it appear in a filter for 'useful', 'linux' and 'sound'.

Like I said, this might all just sound like the most obvious thing in the world to you all, but I'm glad I finally caught up.

Getting Things Done with Ecco

Inspired by Merlin Mann's amazing 43 folders, I've recently become more and more obsessed with Getting Things Done. This is at least partly due to the fact that circumstances have changed, leaving me with an increased workload and the increased possibility of spreading myself too thin. Using the basic principles of Getting Things Done (or at least, the ones I can pick up from around the place, because it's absolutely impossible to get my hands on this book in Dublin), I've managed to ensure that I'm consistently more productive. And even times when I'm not so productive, I'm still completely focused on what needs to be done.

To this end, I've found some pieces of software very useful. First is the Vim Outliner (nice, but too basic for my needs - I feel as if I'm wasting a small-but-significant amount of time wrestling with the software), then there's TomBoy (will be an indispensible piece of software (especially now that it's been hooked into Gnome's new finder-lite), but is still too early a release to be useful for me), and now finally, Ecco.

I read about Ecco on various posts about Getting Things Done. People were saying that they still can't live without it, despite the fact that it hasn't been updated in enough years for it to fall into the "ancient history" category, in internet-years. I downloaded it and gave it a go, and found that it almost perfectly suited my needs.

It manages to present the things I like about the Vim Outliner (the ability to "outline" my goals, obviously) in an well-structured way. You can easily throw a goal together, give it a "todo" date (which combines with its built-in calendar to give you a quick overview of your day's tasks) and easily mark things as done. Once something has been marked as done, it then moves into your "completed tasks" tab, so you can take it out of your TODO list.

It could almost be the perfect piece of software for my needs.

Almost.

My setup here is strange. My primary workstation is a Debian Linux machine, but since a large part of my job includes supporting Windows clients, I also have a Windows 2000 machine on my desk. I have a monitor for each, and thanks to Synergy, I can control both using one keyboard (stolen from an old iMac, because I love the action of the keys) and mouse (a Logitech optical mouse). And this causes problems with Ecc

Because Ecco is quite old, and isn't quite optimised for today's operating systems (and kick-ass TCP keyboard/mouse controllers), it barfs every so often. When I give it a date for the TODO, the mouse and keyboard go unresponsive for a couple of seconds. When I click somewhere I shouldn't, same deal. It's like Ecco prevents my Windows machine from accessing the network while it's performing some task. And perhaps that's the problem - a misconfiguration somewhere that's causing Ecco to try and access a network share or something.. I'll try to look into it.

For now, it's a nasty problem that's driving me away from Ecco.

Today, as a last resort (and maybe some over-optimism), I tried running Ecco on Linux using Wine. It went well, despite missing some of the features I liked about Ecco on Windows (like the ability to pull in highlighted text from any application), and I would have been happy to live with it, if it hadn't been for the fact that Ecco crashes each time I try to access the address book in Linux. Since my Inbox is my address book, I rarely use this feature. It's just knowing that simply clicking on that link would be enough to crash the application I'd rely on most.

Like Chinese Water Torture, it's enough to drive a man insane.

If you want to try out Ecco, it's available for free download from compusol

TomBoy

Following on from my previous post about the principle of dorks Getting Things Done (and hoping, desperately hoping that this doesn't turn into yet another self-help website), comes TomBoy, a simple note-taking application that combines elements of post-it notes and a Wiki.

Although this description belies the power underneath such an application. Simply highlight a portion of text, right-click, and you can create a new note about that particular piece of information.

How well it will fare in the long term remains to be seen. Of all the proposed changes, the only one I can actually see being of any actual use to me would be the plugging into Evolution (even though I'm edging more and more towards Thunderbird and Enigmail as my primary mail client -- something I hope to touch on at a later date).

For anyone that cares, TomBoy compiles cleanly on Debian Unstable providing you have the following packages:
gcc
libstdc++6-dev
g++-3.3
libgtk2.0-dev
libgtkspell-dev
make
mono-mcs

43 Folders

Listening to Tim O'Reilly's talk on Alpha Geeks, he mentions something that I found very interesting: he says he got started on his long, strange journey by simply documenting something that was largely an oral tradition - what it meant to be 'root' on a Unix system. By simply writing down what had been passed on verbally from one admin to the next, he started the ball rolling on what has become one of the largest, most successful and best-respected tech publishing houses in the world.

It's hard to read Merlin Mann's 43 folders and not think of the same thing. Here, Merlin is simply writing down what others have taken for granted -- their productivity habits -- and sharing them with others. In most cases, people don't really appreciate just how effective their habits can be. Or perhaps they just don't see them as significant enough to share with others. Whatever the case, I can only voice my support for 43 folders (and Danny O'Brien's slow-coming life hacks), and hope that my self-discipline allows at least some of these habits they are suggesting to seep into my daily routine.

Right now, my routine in work is this:

  • Each day, we have a team 'huddle' which, as sexual as it sounds, is remarkably useful, we remind each other of what we were supposed to do yesterday, and announce our plans for today
  • Once the huddle is over, I immediately open up ~/work/TODO.otl in Vim
  • .otl files are Outline files, which work well with the Vim Outliner.
  • I give each actionable item a topic by itself, and each sub-action gets another sub-topic
  • I try to give each action at least one line of a description as to what is involved
  • Once an action is completed, I update it with what was done (e.g. text from an email I sent, a snippet of code I wrote), and 'collapse' that item. Because I've set Vim to give 'collapsed' items a different background, I can quickly scan over it and see what still needs to be done.

Here's what an entry from last week looks like:

`2004-09-14 Machines for Customer Conference | handed off to JohnB Asset Register for Eamon Service Pack 2 screenshots from Ouzo Mailman archives for Andrew | MHonArc: http://www.mhonarc.org/archive/html/ | Smart Archiver: http://smartarchiver.sourceforge.net/ | Mailman2rss: http://taint.org/mmrss/`

Within the Vim Outliner, this is displayed in a easy-to-follow, colour-coordinated format, and makes a lot more sense.

This still needs a lot of work, but thanks to people like Merlin Mann and Danny O'Brien actually writing down all these things most people take for granted, I'm coming up with all sorts of new ideas as to how I can make myself more productive.


Update October 4th

Okay, so it's been a couple of weeks. And in those couple of weeks, I've been playing about with a few different systems for Getting Things Done. Things like TomBoy and Ecco have both grabbed my attention in a big way (Ecco is so nice, I'm thinking of writing a post about it). But still I keep coming back to my Vim Outliner.

What I've discovered

I really only started using Vim since I started my current job. Before then, I used nano, because I mainly wrote text instead of config files and large, unweildy shell scripts. So I'm starting to discover and harness some of its power.

Linking files

My TODO.otl still follows the same format (although I've written a shell script to "archive" off the top few entries, so I'm not left with a gigantic text file. However, I've found that I can link to external files by wrapping them in square brackets. This has proved remarkably useful for larger projects, ones that go on for a while, with a set of tasks so long that I can't really keep importing them into my current day's worklist.

So now, it looks something like this 2004-09-22 Finish Asset Register [AssetRegister.otl] Install laptop for John Doe [JohnDoeLaptop.otl] Remove machines from domain | use PHPLdapAdmin - http://xxxxx Investigate LDAP password policies

To get to the external documents, I simply place the cursor between the square brackets and press Ctrl+], which opens it. Once inside the new external document, I put a [TODO.otl] at the top, so I've got an easy route back to my TODO list. This way, I never really have to leave my task list.

Back to Debian

A couple of weeks ago, my main workstation went insane. I put this down to the fact that I was running Debian unstable, which made it, well... unstable. I'd always had a lot of trouble with it. Occasionally, files would become corrupt (mostly apt's status file), or programs would randomly segfault. The most hilarious instance being where fsck segfaulted on startup. Finally, after putting it off for too long and an hour or two of repeated crashes, I decided to force a full fsck.

I wish I hadn't.

There were so many tragic errors. My /bin directory got wiped and replaced with a single shell script I'd written myself. My /home directory also got wiped and replaced with a 415-bit text file.

I wept and wept. And then went about the process of getting it back on-line.

The reason I chose to run Debian Unstable as opposed to, say, Debian Stable is because I liked the better choice of packages. Debian Stable is stuck with Gnome 2.4, whereas Debian Unstable is up to Gnome 2.6 (2.7 if you include the stuff from Experimental). But not wanting to risk killing my machine again, I decided to look elsewhere. The first thing that came to mind was [fedora][1]. I grabbed the isos from bittorrent and started installing.

Initially, I loved Fedora. Its crisp interface (much nicer than Debian thanks to the improvements in the Xorg that Fedora use) was very easy on the eyes, and the fact that everything just worked really scored big points with me. Between you and me, even the graphical boot screen made me go "Ooh". For a while, I was pretty happy.

Until, that is, I tried to get it working the way I was comfortable, with the software I like.

It was then that I realised the simple beauty of Debian. For example, I compose my entries into this blog using BloGTK. To install it on Fedora was a matter of

  • Fire up a web browser
  • go to google
  • search for fedora blogtk
  • check a few false-hits
  • download the package
  • rpm -Uvh fedora_package.rpm
  • find out it was missing a few depencies
  • go to google
  • search for fedora dependcyname
  • download the package
  • ...

Here's me installing BloGTK on Debian:

Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
  libzvt2.0-0 python-glade2 python-gnome2 python2.3-glade2 python2.3-gnome2
  python2.3-pyorbit
Recommended packages:
  aspell
The following NEW packages will be installed:
  blogtk libzvt2.0-0 python-glade2 python-gnome2 python2.3-glade2
  python2.3-gnome2 python2.3-pyorbit
0 upgraded, 7 newly installed, 0 to remove and 10 not upgraded.
Need to get 477kB of archives.
After unpacking 2048kB of additional disk space will be used.
Do you want to continue? \[Y/n] y```

Do you see how simple that is?

I read somewhere that Fedora has its own apt-alike, called Yum, but I disliked this intensely.  I asked it to install a particular piece of software, something trivial and without too many dependencies, and I watched as the machine sat there for a good ten minutes, calculating its depencies.  Clearly this would not do.

So on Monday, I decided to switch back to Debian.  This time, I swore I would be more careful, I would stay on _stable_ for as long as I could.  I think I lasted an hour or two before I was upgrading to _unstable_.  And almost immediately, I was back to where I began - my `apt` was crashing randomly, programs were segfaulting for no reason.  I went home on Monday night an unhappy bunny.

Yesterday, before I did anything else, I made sure I had all the latest packages, especially `e2fsprogs` and forced a full fsck on reboot with

`shutdown -F now`

It took a while (full fsck on a 120GB disk is about 10 minutes).  But since then, I haven't had an ounce of trouble.  Of course, I'm touching wood with every part of me that isn't needed for working right now, but I'm pretty hopeful about the stability of my _unstable_ box.

I'm not much for morals, but I'd say that if there had to be a moral to this story, it would be: fsck regularly.  Especially if you've just upgraded from _stable_ to \_unstable.

[1]:	http://fedora.redhat.com/

Helpdesk Systems: Eventum

For the last two years, our IT helpdesk has been "powerered" by Zope's 'Tracker' Product. I laughingly refer to it as 'powered' because it's anything but. Right now, we've managed to create an ad-hoc system based around Tracker but at the same which tries to avoid it at every step, because Tracker is just plain nasty.

We're trying to make things better, so for the past couple of weeks, we've been trying out a few different products, such as Footprints (good, fully-featured, very expensive) and Auscomp's IT Commander (cheap, bizarre feature-set). We still haven't had success finding something that's suited to our needs, but we're still looking.

Since I had nothing better to do all weekend, I went looking for other replacement candidates. I stumbled across eventum. Eventum is currently in use by the boys in MySQL AB to handle their technical support. I liked the look of the screenshots, so I installed it on a linux machine at home and tried it out. Here's what I learned:

  • Very simple install Just unpack the .tar.gz file into a web-accessibly directory on a server with Apache and MySQL
  • Very fast It's doing a lot, but seems to be pulling the data out pretty quick. And the server I'm running it on isn't beefy.
  • Intuitive workflow No faffing about trying to figure out how to do certain things.
  • Powerful report generator Provides many different views of your data, which is useful for say, weekly summaries.
  • Multiple projects Create a different eventum tracker for different tasks (IT Helpdesk, Mail Server Upgrade Project etc.)
  • Role-based accounts Unlike ITCommander, these roles are actually useful and tied into your account, so you're never confused as to what you can do.
  • Anonymous posting No need to sign in to report a problem.
  • Email integration Didn't get this working at home (because.. well.. I really don't have a mail server set up at home), but this is tied heavily into the issues. You have the option to notify the person who opened the issue at every change (and also change the list of people who get notified)
  • Time tracking Complete time-tracking integration for proper project management.
  • Phone call tracking There's an option to update an issue with details of phone calls you've made/received regarding this issue. I really like this idea. No more ("Hey, did you ring John Doe about buying that software?"). This is also nicely tied into the time tracking system.
  • Easy to configure/tinker with It provides a powerful administration interface, but it's also written in really simple PHP. I was able to get my head around the code in an hour or so.

Here are the things I didn't get to configure and play with, but sounded really bloody nifty from the INSTALL file:

  • Reminder System The reminder system was designed with the objective as serving as a safe net for issues that need attention.
  • Heartbeat Monitor The heartbeat monitor is a feature designed for the administrator that wants to be alerted whenever a common problem in Eventum is detected, like the database server not being available anymore.
  • Command-line interface The Eventum command-line interface allows you to access most of the features of the web interface straight from your command shell. From a personal perspective, this means I can easily automate many things without having to write some custom web-scraping script.

Now, it's not all roses. Here are the problems I've noticed:

  • Possible bugs When I go to close an issue, it will sometimes not budge, and look like it's not done anything. But it has, it just hasn't told me. Similarly, when I put in an anonymous issue, it didn't move. I hit submit five or six times, then finally checked the issue list, and there were five or six anonymous issues. Although this COULD just be the fact that I've configured email integration without a working email setup. I'll have to check this out.
  • Very developer-centric Out of the box, it looks as if it's geared towards software developers rather than IT helpdesk. However, after an hour or so of changing options in the Admin interface, I managed to make it look and feel more like what we'd need.
  • Open source project As an open-source project, you're pretty much on your own with very little documentation to guide you. Although there is an active eventum-users mailing list, which could be a pretty good source of support for.

Mini-hops →

We're switching our network provider. Before we did, we decided to do some speed tests, to find out which was faster. We did this by pinging ireland.com. Strangely, we found that our old provider was faster, in spite of the fact they're supposed to be a slower link, with antiquated technology.

So, we dug about a bit more. Using our old provider:

[johnke@oasis johnke]$ traceroute www.ireland.com
traceroute to www.ireland.com (195.7.33.37), 30 hops max, 38 byte packets
 1  www.ireland.com (195.7.33.37)  256.913 ms  4.886 ms  9.690 ms

No wonder we it was faster. We're on the same subnet.

Anyway, I was just amused by this. I've never seen a one-hop traceroute to a major site before.

Wario Ware Inc.

Strip the flashy graphics, DTS surround sound and heavy production costs from most of today's best-selling video games. Strip them right down to their bare-bones essentials and what are you left with? A truly interesting game will manage to maintain its element of "fun" without these things. Nintendo understand this and this is why, pound for pound, they're producing the most fun of the next-generation giants.

Wario Ware Inc. is a perfect demonstration of this understanding. There are no flashy graphics. The few glimpses of 3D come in the form of static menu screens or non-interactive FMV. What they have produced is stripped-down, bare-bones fun in the form of 200 mini-games. Since most of the first video games were also stripped-down, bare-bones fun, Nintendo have decided to emulate this for their graphical style. Many of the mini-games achieve a visual aesthetic similar to classic Atari games. With a knowing grin, Nintendo acknowledge this, having you play all of these games through an on-screen GameBoy Advance.

In keeping with the faux-GBA interface, this is perfect opportunity gaming. Waiting for a kettle to boil? No problem. Turn on your GameCube and bash about for a couple of minutes - if you can put it down, that is. As well as being opportunity gaming, it's also a perfect example of just-one-more-go gaming. You fly through the mini-games at such a furious rate that it's hard to put down. And as your gaming pride kicks in at being beaten by something so simple as glorified "wack-a-mole", it becomes even more difficult to say no at the "play again" screen.

Also entertaining is watching its effect on non-gamers. I live in a house of non-gamers, and each of my housemates' reactions to the game has been the same:

"What's this?"
"Haha, this looks insane"
"Wow, they're really going for the old-school graphics"

At this point, I usually step out for a cup of coffee or something to keep me twitching like a ten-year old who ate too much sugar (the essential state of being for many of the mini-games) and come back in to find them playing my game.

Project Idea no. 5629

Another idea for a project I may or may never get around to completing (or in this case, starting):

  • list of configuration files/binaries to 'check'
  • in the case of text-based configuration files, read the file into the database
  • nightly, check the files to see if they've been modified (MD5sum)
  • if the configuration file has been modified, diff it and the file already in the database
  • put the output of diff into a database

This would allow an admin quick and easy access to view any changes made to the configuration structure of their machine.

Advantages over using CVS for monitoring

  • automated, no chance of 'forgetting' to check a change into CVS
  • non-intrusive
  • easy to view changes
  • easier configuration, easier to roll out onto multiple servers
  • "server roles" automatically selecting default packages - eg "Apache web server" would automatically add /etc/apache/httpd.conf, /usr/sbin/httpd, etc.
  • all done via a central database, eg select samba_config from db_machinename;

update

I've been doing some more thinking about this. Here's one proposed database structure and what each column should store:

db_machinename

  • file_name (name of the config file we're backing up)
  • config_orig (config file as it was originally)
  • date_orig (date the config file was read in)
  • config_cur (config file as it is today)

db_machinename_diffs

  • file_id (id of the config file we're dealing with)
  • diff_date (date we took the diff)
  • diff_text (text of the actual diff)

LookOut 1.2 →

It seems Microsoft has made LookOut available as a free download on their site. LookOut was a pretty fancy plugin for Outlook that made it possible to search a giant mailbox in a couple of seconds. Word on the street says that Microsoft bought LookOut to get at the search algorithm, which they intend to use on msn.com to compete with google.

Anyway, you can download LookOut here: microsoft.com

404 Cheat

When our CEO came to me and told me that I had two weeks to design and implement a new website, I knew I was in trouble. In the years since our site first went live, it had grown a lot of 'cruft', information that was now completely useless. I started by going to our head of marketing and discussing what didn't need to be transitioned across to the new website.

Although the CEO was happy with the look of the new site and liked the way we'd trimmed it down, he still insisted that all content be available. So I cheated a little.

I moved the old website to another directory (called 'oldsite'), and set up a new virtualhost for 'oldsite.ourdomain.com'. I gave this its own 404 page saying "Your page could not be found". In the new site, I also gave it a custom 404 page - actually, a PHP script, which would redirect to oldsite.ourdomain.com

So, someone requesting a document we hadn't transitioned to the new site:

  • browser requests http://www.ourdomain.com/somepage.html

  • Server 404's, goes to custom 404.php page

  • 404.php redirects the request to http://oldsite.ourdomain.com/somepage.html

It's a simple procedure, but one which saved my neck. In the hopes of saving someone else's neck, here's that simple 404.php page

<?php
$uri=getenv("REQUEST_URI");
header("Location: http://oldsite.ourdomain.com$uri");
echo $uri;
?>

Outlook Quotefix →

Outlook Quotefix is a standalone module that 'fixes' many of the problems people have with Outlook - specifically the way it handles quoting. Goodbye top-quoting! Goodbye shoddy line-breaks!

From their site:

Outlook-QuoteFix can modify MS Outlook's message composition windows on-the-fly to allow for correct quoting and to change the appearance of your plain-text replies and forwards in general: move your signature, use compressed indentation, customize your quote header, etc.

On Why We Don't Use Challenge-Response

I was recently sent an email from one of our users, evidently impressed by a challenge/response mechanism set up by one of our clients on their mail server (even more impressed by the client's claim that he received "no spam, ever"). He asked us why we didn't implement something like this.

Hi Luke,

We've been keeping an eye on the challenge/response (C-R) debate for quite some time now. I remember we spent a good while debating over whether we should include it in our anti-spam arsenal. After a lot of consideration, I think we're going to leave it alone for now, and treat it as a "last line of defense".

A few of the reasons we are choosing not to roll out a C-R solution:

  1. Increases the amount of non-legitimate mail traffic. This is actually contrary to the goals of an anti-spam solution.
  2. Doesn't provide as much protection as you'd think. I doubt Eric's claim of "no junk mail ever", especially since we regularly get spam emails that are 'spoofed' to be from @ourdomain.com addresses.
  3. Trivial to work around. Spammers, for all their misdeeds, are inventive, creative little sods. For example, there was a story recently about spammers getting around Yahoo's automated-account-creation-prevention tool. When you try to create a Yahoo account, you're given an image with a word on it, which is hard for machines to easily guess. So what the Spammers redirected this image onto their pornography sites. People joining these sites would type in the word they saw, and this would be fed directly into Yahoo. Sneaky, but impressively so.
  4. Any kind of automated response will just lead to the auto-responding address being added to the spammer's list of "active" emails. This results in more spam hitting the address.
  5. This, in turn, results in heavier burden on the system.
  6. Speaking of which, most spam comes from non-working or false email addresses. A C-R response to each of these could easily result in a DOS attack on our system.

I could go on, but I think you should see by now that there's a lot to be said AGAINST C-R systems.

However, one of the things we're keeping a very close eye on for our anti-spam toolkit is the idea of 'greylisting' (www.greylisting.org). A brief rundown on the greylisting method:

  • Unknown person (john.doe@unknowndomain.com) sends an email to myaddress@ourdomain.com
  • ourdomain's mail server responds with "oops, temporarily unavailable, try again in a minute"
  • ourdomain's mail server notes that it's got unknowndomain.com's mail server in its queue of mails
  • if unknowndomain.com is a proper mail server, it will wait a couple of minutes and try again
  • if unkowndomain.com is using spam software, it will just barf
  • unknowndomain.com's mail server tries sending the mail again, ourdomain.com's mail server notes that it passed verification, and "whitelists" @unknowndomain.com

It's like C-R, but without any of the nasty downsides I listed above. One thing I particularly like about this system is that it doesn't involve any human interaction. My Grandmother could email me and not get confused by the Challenge-Response mechanism.

We'll probably be testing out greylisting on our secondary mail server soon, and if all goes well, we'll roll it out onto our primary mail server.

Open-Source groupware

Something that's come up quite a bit in work recently has been the idea that we need groupware. What we specifically need is:

  1. shared address book
  2. shared calendar
  3. shared mail directories

The third of these, we've managed to hack together using courier-imap. Unfortunately for us, most of our users are very reluctant to move away from POP3, so they're largely unaware of the availability of this really cool technology.

The second of these, we've... well... sort of managed to hack together. Using Outlook 2000/XP/2003's Free-Busy publishing tool, our users publish to a shared folder which they can all read from. I even wrote a nifty little hack for our 'resources' (meeting room, projector, etc.). But one of the limitations of the free-busy information is that it doesn't list the reason for being busy. It wil just say "John is busy from 10am until 11pm tomorrow", not "John is in a meeting with Jane and Bob from 10am until 11pm in Meeting Room 1". So again, this gets underused.

Playing about with Mozilla's Calendar, we have a tool that does what we want, and allows us to share calendars amongst a team. And it does it all using open standards, so we can get in there and hack around it, if we want. Unfortunately, Sunbird (the codename for the calendar software) is still at a very early stage (0.0.2?), and is barely-usable. Definitely not usable within a production environment.

Shared address books? Nightmare. We have a company-wide address book, with all our email addresses in LDAP (one of these days I'll get around to integrating this with Sendmail, I promise). But we can't add to this remotely. Perhaps we can, and I just haven't figured out how yet. Either way, we need something better.

Yesterday, a crazy thought entered my head. Bear with me, because it sounds a little 'out there'. Would it be possible to remove Outlook completely, and have everyone work through a 'thin-client' (read: browser-based) solution? We already provide a much-loved web interface to mail (using the horde application framework), so would it be possible to extend this some more? Horde offer a module that lets people browse CVS, which I'm sure will appeal to developers. It also allows them to set up filtering rules and vacation notices and, and, and...

Well, this is useless. Of course I can see the advantages of it. I'm already sold on it. Based on their reluctance to give up POP3, I'm worried about how reluctant they'd be to give up their Outlook. People get remarkably attached to their email.

Free X Server for Windows →

X-Deep/32 has been released as a freeware X Server for windows since its parent company (pexus) went into voluntary dissolution.

As someone who's spent months trying to find a decent free X server, and each time had to settle for the giant cygwin/X11, all I can say is: hurrah!

First Post!

Finally got around to watching Danny O'Brien's Life Hacks talk from NotCon (video). This talk is essentially a "Seven Habits of Highly Effective People" for dorks - Habits of Highly Prolific Geeks. It's terrifying how many of these habits I seem to have formed myself, and yet am nowhere near being what you would call 'prolific' (I sometimes stick a mirror under my nose to make sure I haven't died).

One of the figures that stood out for me from this talk was that "7% of all posts to livejournal are marked as 'private'". These are the posts that are visible to noone but the poster. I use these a lot on my own livejournal to keep track of important memories, or even nuggets of code that I might use a lot (the most-used being wget -r -l1 -H -t1 -nd -N -np -A.mp3 http://www.site.com). There are also countless nerd-related thoughts that run through my head on any given day that I'd love to expand upon, but don't want to do it through livejournal. I've got everyone thinking I'm some kind of puerile semi-moron, and I like that. I wouldn't want to spoil that by posting something that's actually slightly useful.

So that's the purpose of this blog. Hopefully I won't get bored too quickly.