Thanks for visiting my blog - I have now moved to a new location at Nature Networks. Url: - Please come visit my blog there.

Monday, December 31, 2007

Personal Email Sever on Ubuntu 7.10

Well, this has been a good year for learning, so I thought I'd close out by sharing some of what I learned over the past week, in the process of setting up an email server. I don't know if this post will really be of any use to the people who usually read my blog (if there still are any out there), but it might just help those who stumble across it through google.

My basic idea was to consolidate all of my various Pop3 email accounts into one single IMAP account. I have several personal accounts (google,, etc), as well as a work account, and now that I'll be accessing my email from 4 separate computers (my work computer, my home computer, my laptop, and the one at my girlfriend's place), keeping track of emails on a pop account is a pain in the rear end.

The main lesson I'd like to go over here is the architecture of how this works. There are tons of tutorials on setting up IMAP accounts on Ubuntu (and some of them were really good) - but no one tells you how to fit all the pieces together.

The core of the whole process is that you need to set up what's called a Mail Transfer Agent or MTA. This agent or program will accept emails on port 25, and transfer them on to whatever other system should receive them. In a simple setup, the MTA will accept emails, and then pass them on to your ISP's mail server, and do nothing more. Essentially, you're simply making your MDA into a personalized SMTP server.

Unfortunately, this is severely useless. Why create your own SMTP? (simple mail transfer protocol server.) Well, if you were in a large organization, this might be useful, or if you wanted to send a lot of spam, then sure. Otherwise, it's pretty pointless - the vast majority of people don't need this functionality.

However, it does become useful if you attach the MTA to a mail delivery agent. The mail delivery agent is a local postman - it takes mail that comes in to your computer, and instead of handing it off to another computer, it delivers it into your local mail box. Thus, each user (or virtual user) can have their own personal mailbox on the local computer.

For my setup, I began by using Postfix as my MTA. After playing with it for a while, I discovered that some of the lines blur. By configuring it properly, Postfix can act not only as an MTA, but also as a MDA - it not only receives email, but also passes it into the appropriate mailbox. (A subprogram called "local" will take over this function, if it's set up in the This allows you to use Postfix as, essentially, a stand alone mail server.

Incoming mail arrives on port 25, to Postfix. Postfix hands it off to the "local" program, which delivers it. Neat.

Unfortunately, my system couldn't be quite so simple. My ISP blocks port 25, so no mail will ever come to computer via the regular route. No matter whether I configure the MX record correctly, and the local delivery options, email will never make it to my computer, and into my mailbox.

Thus, the next element is something called "fetchmail". Fetchmail is a program that helps get around this problem. Instead of forwarding email from my other accounts to my new local IMAP server, I have to reverse the process, and use my IMAP server to fetch my email from the other accounts, avoiding the blocked port 25. Enter Fetchmail.

Using Fetchmail, I configured my local computer to poll the servers at some interval (1.5 minutes), to grab my email on the regular POP3 port as if it were a mail client (ie. thunderbird, outlook, etc.). Unlike a mail client, Fetchmail doesn't deliver the mail to my mailbox, it hands it off to port 25 on my local computer, where it finds the above mentioned Postfix server! Tada!

Ok. So now, we have a system where email is accessed from my accounts on another machine, and dropped into my local email system. Great. 2 parts left: getting my email, and making sure it's delivered into the right folder in my email box. Fortunately, they're both handled by two parts of the same program. Dovecot.

I took advantage of Dovecot for two main reasons: it has a very easy to set up IMAP server, so I can access my email remotely, and because it's integrated with something called "sieve".

As an IMAP server, Dovecot is remarkably easy to set up. Tell it where and what form the local mailbox is, and what protocol to serve it up on. Tada, you'll have an IMAP server up very quickly. A little more work, and you can set it up with some excellent security, too.! (Highly recommended!)

That other piece, "sieve" is a little more complicated. It's essentially a language for filtering emails, though it's very simple to get the basics of it. The only caveat is that you can only use Sieve if you're using Dovecot as your MDA. This isn't a big deal - almost no setup is required, and again there are tutorials on it. By switching away from the "local" program I mentioned above, and configuring Postfix to use dovecot instead, you'll be able to create some simple filtering rules that pass each email into the appropriate folder. And there you go - you'll be all set up. (Don't forget to tell thunderbird to check each folder for new mail, however!)

So, my system works like this:

  1. Email goes to my pop3 server

  2. Fetchmail gets new mail from my pop3 server, and brings it down to my local IMAP server, where it's handed off to Postfix.

  3. Postfix looks at the email, and hands it off to Dovecot for local delivery

  4. Dovecot looks at the email, and processes it with Sieve rules

  5. it goes into the right folder


Sunday, December 23, 2007

Merry Xmas from the puppy

Normally, I'm against putting costumes on my dog, but hey, sometimes you've got to let Elaine have her way... and then this was just too cute not to post, once I'd already taken a picture.

Merry Xmas everyone!


Friday, December 21, 2007

One last picture

I've spent part of the afternoon setting myself up to be productive on the laptop, between working on other projects, so I thought I'd show the end result of this process:

Gotta love the bad photography. Anyhow, it's now all setup running Ubuntu, with wireless, accelerated graphics, eclipse, Evolution, xmms, pidgin, and just about anything else I could think of. The final touch was creating a wall paper for it - I kinda like the green theme.

Anyhow, for Olena, who said I wasn't "stalkable enough" because I wasn't writing enough blog entries, there you go, 3 in one day!

Now I can shift to writing more useful blogs - I've been saving up a few Java rants for a couple of days, so those will come next.

It's here!

It finally arrived, after 3 weeks of agonizing waiting.

It took me a bit of time to open the box, boot the laptop, refuse the Microsoft Windows license, and get ubuntu started, since I was taking pictures the whole time.... but it's now working, so I should get back to work. Removing windows from a computer is one of the best ways I can think of to spend a lunch hour.

So, there are a few things I've noticed instantly about this Vostro 1000. The first is that it's much lighter than I expected. The last few laptops I've used have all been beasts - this one weighs a lot less, and the screen is by far the heaviest part of it.

As an addendum, it's also a lot quieter than the other laptops - except when the fan is going full power, or the CD is running - but I expect those won't be quite as common.

The second is that someone FINALLY created a decent power pack, with a built in cord wrapper. I definitely like that!

I'd write up a tutorial on how to install linux, but it's just too damn easy. The only glitch I've come across so far is that the BIOS is defaulted to not allow Booting from the CD. You have to press F2 to get into the bios, go to the boot menu, and then change the boot order. From there, it's smooth sailing.

Finally, I'd just like to point out something of interest for those people interested in getting a refund on the Vostro 1000 for a copy of Windows you'll never use. (Assuming you want to install linux, instead - and who wouldn't!)

Shipping update... (again)

Yay!!!! it's on the truck. (=

2007/12/21 08:07 On vehicle for delivery
2007/12/21 07:25 Shipment In Transit
2007/12/21 07:10 Shipment In Transit
2007/12/20 13:16 Shipment In Transit
2007/12/20 13:15 Picked up by Purolator
2007/12/17 13:30 In Transit in U.S.

Thursday, December 20, 2007

Dell shipping update... thank you purolator

After 3 days, my laptop has finally made it to Canada! Should I get my hopes up that it might arrive before xmas?

Status     Shipment In Transit
Date/Time  DECEMBER 20, 2007 AT 13:16
Depot      CALGARY, AB

Monday, December 17, 2007

Definitive proof that Linux is ready for the masses

Bear with me... this article has a long introduction, which sets the stage for my two conclusions, which are that Linux really is ready for mainstream uses, and that there are still reasons to install Microsoft Windows OSes.

Yes, my preamble goes back a long way... way back to when I was learning to program, when I was 9. It was on a coleco-vision, with an Adam expansion module. The kind that loaded cassette tapes. My older sister was learning to program, and, being an arrogant punk, decided that I had to try it myself. It wasn't that hard, since I never really got past control loops, and the like. Still, it was a good exposure to computers and programming (and it was a skill that got me hired for a job programming in basic, back in the Y2K days.)

When I moved in with my father, 4 years later, he had just purchased a brand spanking new 386/8SX. It was blazing fast, had a 10Mb hard drive and 1Mb of ram. It was my father who showed me how to use DOS, Windows 3.0, and to create batch files as well as use many of the early programs available at the time. I was a big fan of windows, and I was the one who pushed hard for us to upgrade to 3.1, when it became available. Still, my father was the trailblazer, and the one who taught me how to use computers and to tame them when they were misbehaving. For years he was my computer mentor.

Somewhere along the line, however, my father decided to stop following computer technology and forgot most of what he'd learned. So long as I was around to help, that worked out pretty well. He left it to me to install the whopping 4Mb upgrade, the modem, and he didn't even watch when I added the sound card. Help was never more than a phone call away - even after moving 5000km away, long distance is a lot cheaper than professional support (and I'd like to think that the quality difference isn't that great.) (-;

Personally, I switched full time into Linux in 2002 (Slackware), and haven't looked back. Slowly, I became out of touch with the windows world, which made my support somewhat less than ideal. The distance meant that smaller problems were often saved up for months at time, with the occasional bi-monthly crisis phone call, so that my 1 week vacations to visit my parents took 3 days of support work - virus removal, upgrades, staring at odd bugs in programs. The usual windows oddities. (My father used to keep a list of "weird occurrences" by the computer to ask my about on my vacations.)

To stop this flood of annoyances, I moved my father to (Fedora Core) linux (dual boot), in 2004, which stemmed the tide of viruses, and changed the nature of the support calls. (Invariably, it was either a modem or printer problem.) There were weird things happening - substituting random windows errors for random Linux errors. The following year, I upgraded to an Ubuntu distribution (breezy), which had just come out. Touted as a much more friendly distribution, it fit my father's work style pretty well, and had fewer bugs. That OS lasted 2 years - and my father tells me he liked it.

Of course, that translated to fewer support calls. And this year, I just upgraded Breezy Badger to Ubuntu 7.10, Gutsy Gibbon. All of the weird printer bugs appear to be gone, networking was easy to set up, wine was able to run his last Windows program (Quickbooks) in Linux, and the new vista computer was upgraded so that Vista was removed entirely. Both computers boot to Ubuntu Linux by default and my father even asked me to remove the old Win2k partition from the older computer: He claims he doesn't need it now. (I figured we should leave it there for one more year, just in case.)

My father actually started pushing for my step-mother to switch to Ubuntu.

What prompted this story, though, is the support call I got from my father this morning - the first in months, really. Just so you don't think my father is a computer guru or anything, the question from my father went something like: "I'm in windows, and we'd like to attach an URL to an email... after I click on attach, what do I do?" (The solution: copy and paste the link from the email I sent him the other day.)

Anyhow, that brings my to my last point. The value of Windows: Now that I have my father running Linux, I don't hear from him so often. I'm going to leave my step-mother running windows, just to make sure I get a few more phone calls, now and then. (-;

Labels: ,

Friday, December 14, 2007

Dr. Randy Pausch's lectures

I often use my 3 in-boxes as task lists, and I had one item in particular that's sat around for ages: watching the Last lecture of Randy Pausch. I finally watched it last night - in fact, I stayed up late to watch it, despite my intention to get to work early to get a head start in a few projects (which I managed to accomplish anyhow). All I can say was that it was worth it.

Tonight, I was debating going to bed early, but ended up clicking on a link that took me to Dr. Pausch's web page, where I discovered a new lecture he's posted: "Time Management". It's a fantastic reflection on how to focus on your goals, and to achieve the things you want to achieve. He gives plenty of pointers, of which a number are so clearly good ideas that it would be silly not to follow up on them.

Taking his advice, I've already done two things this evening that I was procrastinating on... and I have several good ideas on how to improve my working environment.

Anyways, if you have the time, each of the lectures I've linked to is about an hour and a half, and worth every minute of it. Invest the time and watch them.


Monday, December 10, 2007


I had a ton of things to rant about, but the one that's annoying me the most, at the moment is the random delay of my laptop. I'm eagerly awaiting it, as I expect my productivity to rise significantly once I have it up and Ubuntu'd. Anyhow, I went to the web page go check on the order status of my laptop (which I've been doing at least once a day - order tracking is one of the worlds best inventions, yet also one of the biggest destroyers of productivity ever created) only to find the following message:

Delivery date of your product might have been changed.
This could have been prompted by the following circumstances:

* Mismatch between billing and credit card information.
* A system issue that interrupted the order entry flow

If the above occurred as a result of mismatched billing information, you will be contacted directly by one of our representatives within the next 2-3 business days.

Of course, I haven't been called, and there is a new order number listed, so I can't figure out why my order was canceled and restarted. I did a quick search online and discovered a few horror stories of people who've had this error just get lost in production limbo, where new order numbers are repeatedly spawned, until the order is finally just terminated. Wonderful.

I guess I'll just give them a call tomorrow to find out what's up.


Tuesday, December 4, 2007

Committee Meeting #1

I had my first PhD committee meeting this afternoon, and it went reasonably well. I don't know what I was expecting, but having spoken to 3 of the 4 committee members about my project multiple times over the past 5 days, I guess I wasn't exactly expecting to be raked over the coals.

My only real comment is that I should have spent a bit more time practicing my presentation - but who has time for stuff like that? I'm going to try to get the Paired End Tag code working tomorrow, and the paired Eland reader - somehow it's just way more important to do the work than to talk about it, which is it feels like I've been doing lately.

Anyhow, so much for taking the evening off - I just spent the last half hour trying to figure out why someone in Wisconsin can't seem to get my FindPeaks application to work. I don't really get it - I can run my code with his file just fine. Anyhow, another thing to look at tomorrow.


Monday, December 3, 2007


Planning and reality often don't mix in grad school, I'm discovering. Since I have a committee meeting tomorrow, I thought I'd use today to finish my presentation and written documentation for it. Instead, I was told I had to have code running by 1pm that would process transcriptome data using paired end solexa reads. 5 hours later, I had code analyzing Single end transcriptome data, but nothing finished for my committee meeting.

On that note, I'm not going to blog about paired end data - I'll save that for another day, and instead, I'm going to go work on my presentation and report. However, I did want to provide a neat link Elaine passed to me: made with molecules. Jewelry for the rich and geeky. (I'm geeky enough to like it, but not rich enough to pay THAT much for it, even if I wore ornaments.)

Oh, and one last parting note... my laptop is being assembled today. If I wasn't going to be so busy tomorrow, I'd probably be checking it's status every 10 minutes. The curse and blessing of real-time tracking data.

Labels: , , ,

Sunday, December 2, 2007

New Laptop... with a refund?

Well, I took the plunge today. After struggling to get through grad school without a laptop for the past year, I've decided I can't take it anymore. The computer I use at work is an ancient IBM, which does an admirable job of running Ubuntu, but really just isn't up to the tasks I set it. So, with pending conferences, presentations, committee meetings and some serious programming likely in the next few months, I figured I've got shell out the money for a laptop. In fact, I've purchased a Vostro 1000 for $649 CDN... not too shabby, I think.

My plan A was to buy an Ubuntu laptop from Dell Canada, but after waiting for 8 months for that program to come north of the border, I've realized it's probably not going to happen in a time frame I'm comfortable with. It's only for a few select markets. So, I had to kick into plan B: I've bought the laptop, and I'm going to try to get an O/S refund from Dell.

It's not malicious or trying to get revenge - I just don't want to pay for software that I will be deleting straight away, once I turn on the laptop. I think it's wrong to bundle in an operating system, and force you to pay for it - so I'm going to try to get my money back.

This isn't a wild crusade - I have heard of people getting back money from Dell before, anywhere from $10 - $110, with no real regularity to what's received. Such as these links:

here, here, and here.

The internet is full of personal accounts of this working - so I think it's my turn to try. Besides, as a grad student, getting back $100 is worth about 10 hours of my time - I'm willing to give it a shot. Watch out Dell Service Reps... here I come.

Labels: , , , ,

Saturday, December 1, 2007

BC Cancer Annual Cancer Conference 2007

I didn't have a chance to blog yesterday from the BC Cancer Annual Cancer conference, but I figured it was worth mentioning a few highlights.

I "finished" the chapter I was working on in the morning, on ChIP-Seq, and sent it off today. I probably could have used a little more guidance on it, meaning that it focuses on things I found interesting/important. Who knows if that's what everyone else will want, hence the quote on "finished."

Around noon, a few other grad students and I cabbed down to the Westin Bayshore, where we enjoyed one of the better free buffet lunches I've had in a while. (Why more people don't use free *good* food as a way to bring graduate students to events is a constant source of amazement. Pizza is good... but smoked salmon is SO much better.)

In the afternoon, I did my usual thing, and went off to see a track that really had nothing to do with my research, and ended up learning a tremendous amount about pathology and cancer tissues. Some of my favorite facts:
  • up to 30% of pathologists diagnoses based on histology are misunderstood by the doctors

  • Pathologists generally agree about what is cancer and what is not... but when diagnosing which type of cancer it is, it's usually a "best guess" scenario. They frequently disagree.

  • Cancer cell morphology is generally rule of thumb based, with emphasis on the size and shape of the nucleus, and the location of the cell.

  • Pathologists often can't distinguish between types of cells that they see. "if you didn't tell me, I wouldn't know if these cells were skin, vagina, ovary..." (The pathologist was a specialist in women's tissues.)

Either way, after about 2 hours of talking with a pathologist, I have a new respect for the lab work done in hospitals, and a few ideas about places I can take my projects to solve major issues pathologists have... I'll see where they take us.

Anyhow, it's time to work on preparing for my committee meeting on tuesday. While waiting for my next post, feel free to check out my poster on applications for transcriptome and ChIP-Seq analysis. (Don't mind the two mistakes I'm aware of!) Cheers!