As mentioned in our previous installment, when I installed this machine in Jan 2003, I started with the RedHat Enterprise Linux 3 clone of White Box Enterprise Linux 3. It has since switched to CentOS 3.
Our departmental e-mail server handles a lot of chores for us. It is where the mail for Mark, Todd, and myself goes. It also handles all the CRT mailing lists, such as ezRETS-users and newsletter-broker. We also have it host some virtual mailboxes such as our help line of info@crt.realtors.org.
The hardware is a dual Pentium III 700 with 2GB of RAM. Its slightly overpowered for most our needs. There are two situations that call for the heavy amount of RAM, at least: 1) When the mailing lists are very active and 2) when we have heavy incoming mail. With the mailing lists, every one mail coming in turns into a large number of outbound mails. With the incoming mail, we spam and virus filter them which can take a good amount of CPU time and active memory.
One of my goals in building this mail system was “ubiquitous e-mail access.” By that I mean that if I can get on to the Internet, I can check my staff e-mail. We achieve that by allowing access on three fronts: IMAP over SSL, POP3 over SSL, and a web-based IMAP client.
The heart of any unix-based mail server is the mail transfer agent aka the MTA. To quote from wikipedia
A mail transfer agent or MTA (also called a mail server, or a mail exchange server in the context of the Domain Name System) is a computer program or software agent that transfers electronic mail messages from one computer to another.
For our MTA, I choose postfix. Postfix is an extremely popular open source MTA. I started using it on all my mail servers in 1998. After dealing with sendmail and its object-code-like .cf files for many years prior, postfix was a welcome change. The first time I set it up, I had it installed and accepting mail in 5 minutes. It normally took me 45 minutes to get to that point with sendmail, due to reading and rereading docs, so I spend the next hour trying to figure out what I did wrong because it couldn’t have been that easy. It was. It still is, for that matter. Throughout the time I’ve been using it, it has consistently hit its goals of being “fast, easy to administer, and secure.” Postfix comes with the operation system. However, I wanted to use new features in a later version than what ships with RHEL3, so I hand maintain a more current version. As of this writing we’re using Postfix 2.2.5.
For real mail accounts i.e. they have a shell account on the system, we use procmail as our mail delivery agent aka MDA. An MDA takes mail from an MTA and distributes them to the users/recipients mailbox on the mail server. Using procmail, we can do powerful server side filtering, such as putting all mail from the ezRETS-users mailing list directly into an ezRETS-users folder, instead of the main INBOX. procmail also allows us to do trick like doing a store-and-forward on the mail in certain conditions. I have it set up to store-and-forward anything that hits my INBOX. By default, the postfix that comes with recent RHEL releases uses procmail as the MDA.
At this point, we have a pretty typical unix mail system, but we’re limited to reading mail on the server using text pasted tools like mutt, pine, or, if you’re feeling particularly old school, mailx. (Actually, I still use mutt for 99.9% of my personal e-mail. I haven’t found any other program that will allow me to rip through mail as fast as mutt does. However, for work, and with the setup I’m describing, I’ve been using Thunderbird) Another bad thing, if we’re only set up this far, is that we’re being flooded with spam and viruses as we have nothing set up to block that.
Lets take care of the easier problem first, getting access to our mail via IMAP or POP3. For this, we’ve gone with Courier-IMAP. Courier-IMAP was chosen due to its small memory footprint, support of virtual e-mail accounts, and support of LDAP for authentication. Its fairly easy to set up for local users, but we also want to support virtual accounts and aliases.
To make courier (and postfix) handle virtual accounts and aliases, we look to CRT’s own Jamm. We wrote Jamm a few years ago to bring E-mail Redirection to REALTOR® masses. That didn’t work out quite as we hoped, however, Jamm is still an extremely useful tool and it is in use in many ISPs. We’ve set it up to manage @crt.realtors.org and some of its virtual accounts such as LDAP server. In our case, we run OpenLDAP. In the future I might consider using the RedHat Directory Server.
Now that we can accept mail and read it via IMAP, we want web access for the times when we can’t get to our IMAP client. There’s a number of web-based IMAP clients out there, we went with SquirrelMail due to familiarity. SquirrelMail is incredibly capable and has most of the features you’d expect from a web mailer. As a performance enhancement, we’ve set up IMAP Proxy to persist IMAP connections. Normally, a script based app, such as SquirrelMail, has to open and close IMAP connections per web hit to get the information they are serving to you. IMAP Proxy runs as its own server and stands between the web app and the imap server. It intercepts the opening of IMAP connections, and simulates a normal IMAP open and close, while maintaining a real/persistent IMAP connection. The performance different is staggering and I wouldn’t run an web-based IMAP client without it.
At this point we have a fairly complete e-mail setup, but we’re still being buried in spam and viruses. Luckily, we have solutions for this as well.
With the requirement that non-technical people should be able to configure their spam and virus settings without editing a text file on the server, we came across Maia Mailguard. Maia is a web-based configuration front-end for the spam and virus filters. Originally it controlled the amavisd-new content checker. Now it has its own fork of amavisd-new as the goals of Maia Mailguard and amavisd-new have diverged. amavisd-new (or its replacement) get the mail from the MTA and then run it through the spam and virus filters. Maia also requires that we use a database to store the configuration. For a reason that will be made appeared in a few paragraphs, we are using MySQL. So we have the webapp and controlling deamon to run the anti-spam and anti-virus apps, but which ones should we use?
For spam, we have no choice, we have to use SpamAssassin. amavisd-new will only work with SpamAssassin. This is okay as SpamAssassin is very widely used and trusted. It has all the popular features we expect in modern anti-spam software, in fact, it pioneered many of them.
We have a large variety of choices for the anti-virus application. Many popular commercial anti-virus packages have been ported to Linux. They are mostly used in the capacity for scanning e-mail that is being served to people who run e-mail clients on that popular platform prone to viruses. There is also an excellent open-source anti-virus app called Clam AntiVirus. clamav, as its own, has a large community around it and is often a few hours ahead of the commercial virus scanners. We run with Clam, update the virus definitions every 4 hours, and have never had a problem. One interesting thing we don’t do, but is possible, is to run multiple anti-virus scanners through amavisd-new. The theory is that if one anti-virus app misses it, the other will catch it. As we’ve never had a problem, we’re happily running with just the one.
One last trick in our anti-spam sleeve is greylisting. As we recently wrote about, greylisting is an invaluable tool. It stops 90 to 95% of spam we get before it even hits the anti-spam software. Since we’re using postfix, we went with gld which is a greylisting deamon designed to work with postfix and only with mysql. (See, I told you we’d have a specific reason to stick with mysql.) gld is extremely easy to set up and makes a dramatic difference.
So, now we’re all set, we can read our mail and we’re only getting mail that we went to get. Well, mostly, no anti-spam technology is perfect. So, occasionally we have to go into Maia Mailguard and help train the Bayesian filter.
The final piece, which is really kind of a bolt on as we don’t need it to integrate with the previously discussed pieces, is our mailing list software. As mentioned above, CRT runs many mailing lists. Again, there is a large variety of software out there to do mailing list, but we went with one that seems to have significant traction in the open-source world, Mailman. Mailman does a great job, and offers some simple archiving as well. We haven’t had much of a complaint about it from users, and its fairly easy to run as administrators.
This is a pretty good rundown on how we have our mail server set up and this is a much longer post than I intended. I also run a very similar setup on my “personal” mail server that runs the mail for about 20 people. However, this setup can be made to scale to a much larger environment. I know at least one ISP runs something similar as its admin and myself traded notes. (He’s the one that turned me on to Maia Mailguard.)
I tried to keep this article fairly non-technical, but if anyone has any questions, please don’t hesitate to ask.
0 Responses to “CRT’s Infrastructure Part II: The mail server”