August 20, 2010

Dovecot 2.0 install

Ran into some minor hurdles in upgrading to Dovecot 2.0. Definitely need to read the docs on upgrading a bit better next time... But after some work, I have it running here.

Found an undocumented configuration option called service(dns_client), this was logged when it was trying to use the user 'dovecot' instead of the OS X user '_dovecot', which I had already listed in my config for some other services. Simple fix here, I took out the lines specifying 'user = _dovecot' for my listed services, and instead used this line at the top of my config file:

default_internal_user = _dovecot

Second issue that I ran into was the default install looks for a user called dovenull. This user didn't yet exist on my system, so I needed to create it (as opposed to defining a different existing user for this function, which I didn't want to do). My concern is that when Dovecot 2 is rolled into OS X, a new dovenull user will be created, or more likely, _dovenull. So, I can create the user now, and have it later overwritten, and who knows what issues might arise, or I create this user with a different name to avoid the whole issue.

Creating unix level users with OS X is a bit more involved than other systems, but once you read up on the dscl command, you can find some quick examples of this. The trickiest part is picking an appropriate UniqueID for the user.

Here's a handy command for listing all of the used UniqueIDs:
sudo dscl . -list /Users UniqueID | awk '{print $2}' | sort

We'll use a similar command to get a list of all Group IDs that are used:
dscl . -readall /Groups | grep PrimaryGroupID | awk '{print $2}' | sort

It seems best to avoid the range 0-99, and the low 200's are slowly filling up for UserIDs. There's nothing from 100-199, and OS X starts creating new users (the ones used by the Finder) at 501. Why 100-199 was left blank, I don't know, and who knows what other 200 range codes might be used in a future OS update. For the Group IDs, 0-100 were full up, again the break from 101-199, starting again with the low 200's, then starting with the 400's. So, throwing caution to the wind, I'll use 301 for my new user and the group associated with it, created with the following commands:


sudo dscl . -create /Users/_dovenull
sudo dscl . -create /Users/_dovenull UniqueID 301
sudo dscl . -create /Users/_dovenull PrimaryGroupID 301
sudo dscl . -create /Users/_dovenull UserShell /usr/bin/false
sudo dscl . -create /Users/_dovenull RealName "Dovenull"
sudo dscl . -create /Users/_dovenull NFSHomeDirectory /var/empty
sudo dscl . -append /Users/_dovenull RecordName _dovenull

sudo dscl . -create /Groups/_dovenull
sudo dscl . -create /Groups/_dovenull PrimaryGroupID 301
sudo dscl . -append /Groups/_dovenull RecordName dovenull

I set the UserShell above so that the user has no shell access, and set NFSHomeDirectory so that there is no home directory.

Once I had the _dovenull user created, I was successfully able to get dovecot running, so far, no issues.

Posted by Jim at 10:00 PM | TrackBack

April 18, 2010

Upgrade to 10.6.3 complete

As I mentioned last week, I've been working on upgrading my system from 10.5.7 to 10.6.3, performing as clean of an install as possible to clear out years of crud under the hood. I've wrapped up the last of my upgrades, and am up and running on a freshly built system.

The only real hiccup was with the Postfix compile, once that was sorted, everything else was fairly straightforward, simply a matter of grabbing all of the various packages to make everything here run, run through all the necessary compiles, then finally transfer over various changed files since the last copy.

Here is a list of the software currently installed for mail and web services:

Xcode 3.2.2
PHP 5.3.1 (10.6 built in version)
Apache 2.2.14 (10.6 built in version)
MySQL 5.1.45
Postfix 2.7
Dovecot 1.2.11
Cluebringer 2.0.10 (Policyd 2.0)
PCRE 8.0
DBI 1.609
DBD-mysql 4.014
Net-Server 0.97
Net-CIDR 0.13
Config-IniFiles 2.57
Cache-FastMmap 1.35

Initially I needed to copy the etc/postfix directory to preserve configuration files when installing the new Postfix,also copied over /var/mail to bring over the mail stores used by Postfix/Dovecot, and a handful of other config files, /Library/Webserver for the web pages.

To get Apache running, there were some simple edits to enable the built in PHP, and setting up the correct vhosts again; I hand edited the files to match the old config to keep from introducing any unneeded changes.

I think that the only real surprise was that there weren't more surprises. I now have everything running in 64 bit mode on the new server, with the exception of some 3rd party apps. Sweet.

Posted by Jim at 10:36 PM | TrackBack

April 14, 2010

Great site with Mac server info

In my web searching recently, I came across DIYMacServer, a site who's focus is all about running Postfix, Apache, PHP, Dovecot, and related code on the Mac. There are numerous articles about each software update that comes down the line, and the author, Richard Valk, does a great job at documenting everything he can about changes each update brings, and how it effects his system.

Posted by Jim at 9:08 PM | TrackBack

Compiling Postfix on Mac OS X 10.6

So, I'm trying to compile Postfix 2.7 on my new 10.6 system. During make, I get this:

In file included from dns_lookup.c:152:
dns.h:23:29: error: nameser8_compat.h: No such file or directory
make: *** [dns_lookup.o] Error 1
make: *** [update] Error 1

A google search found a suggested fix, in /src/util/sys_defs.h, the following line should be commented out:

#define RESOLVE_H_NEEDS_NAMESER8_COMPAT_H

With this line commented out, I'm able to get a good build, but at what cost? Presumably this is going to break some of the name resolution that Postfix uses, which would not be good. After reporting this on the Postfix mailing list, I spent some additional time researching the issue, but ironically kept coming up with various pages that mirror the Postfix list, and kept coming back to my own posting... Time to switch gears.

More searching online found some similar reports for software other than Postfix, but no hints at fixes. Digging into the OS, I found that Mac OS X 10.6 no longer has an include file named nameser8_compat.h, which is the source of the issue. The equivalent file now seems to be arpa/nameser_compat.h. Updating the Postfix dns.h file (line 23) to include this file instead finally resulted in a good build. It may be another day or so before I'm able to put this server online to test.

This information has also been reported to the Postfix mailing list, it is likely that the 2.7.1 version of Postfix should have this change and compile properly out of the box.

Posted by Jim at 10:58 AM | TrackBack

April 11, 2010

Upgrading to 10.6

It's upgrade time again here, and I've made the decision to set up a brand new system on the Mac Mini here. The current system originally started out under 10.3, was then upgraded to 10.4, then 10.5, and also saw at least one hardware switch in there. While fairly stable, I know there's a lot of crud under the hood left over from all the various installs, upgrades, and software changes. So, I'm setting up a new server completely from scratch on Snow Leopard, Mac OS X 10.6.3. I've set up a spare Mini for this work, so I can have both systems up and running simultaneously.

That said, I do need to preserve the web site, and keep the old mail stores around, so this will need to be copied over to the new drive I've set up. After setting up the new drive, I've used Carbon Copy Cloner to copy the web directories, mail storage, and the Postfix directory to the new system, this gets me most of the data that I'll need.

Next I'll need the MySQL data. Since this process is likely to be ongoing for a few days as I test and retest, I'm not going to want to make several data moves from MySQL, so the simplest solution was to set up a Master/Slave relationship on the two systems, so the data will always be in sync between both boxes. I know that at the end I'll need to clone over the web and mail data again, but that will be a quick process at that point.

Now to figure out what works and what doesn't...

Posted by Jim at 8:41 PM | TrackBack

July 6, 2009

Updates - Postfix, Dovecot, 10.5.7

Finally got around to upgrading from 10.5.6 to 10.5.7, good news, nothing broke Postfix this time!

Took the opportunity to also upgrade Postfix, now at 2.6.2 here. Nothing tricky about the upgrade, just follow the standard build instructions. Also upgraded Dovecot from 1.13 to 1.2, just released. Again, nothing terribly tricky, just read the docs.

Posted by Jim at 10:03 PM | TrackBack

March 24, 2009

Courier migration to Dovecot

I've dumped the old Courier IMAP software that I had been using for POP/IMAP in favor of Dovecot. I had no real reason to do so, Courier IMAP was running fine (though I did have issues during install last time around). However, Dovecot does seem to be the new hotness, so I decided to give it a go.

Much has been said elsewhere about Dovecot's superior performance, but with my little server that isn't much of a concern, but there are fewer files to mess with, and the overall operation does seem more streamlined. A single process handles authentication, IMAP, and POP, spawning what it needs automatically.

The tricky part was integrating with Postfix Admin and the virtual user setup, but there were guides online that assisted with most of this. As I had stretched this upgrade process out over several weeks of fiddling with the server, I can't point to any single definitive source for upgrade advice, but some Googling will find what you're looking for.

My major headache came with the courier-dovecot-migrate script, and its insistence that I had no mailboxes. Online advice claimed that this was normal, and that it would work when the actual conversion was done, but even then it still reported zero mailboxes, but apparently DID do its thing.

The last hurdle was specifying the exact path for my virtual users, in my case, /var/mail/vhost/%d/%n, and after that it was working fine.

There are some good debugging options, which helped greatly.

Posted by Jim at 12:13 PM | TrackBack

March 11, 2009

More updates break Postfix

I'm assuming it was the 10.5.6 update, possibly a security update, I don't know. But Postfix broke again after the install finished. Same nonsense with many log entries about files not being owned by Postfix, as before, running a new 'make upgrade' fixed that up.

Had one last problem that was new, Postfix launched, but wasn't receiving mail. main.cf had an entry for inet_interfaces that had previously been set to 'all', but was no longer. Changing that back and relaunching Postfix cured the issue.

Posted by Jim at 12:03 AM | TrackBack

February 7, 2009

Postfix Admin and error 105

Spent some time struggling to upgrade my old Postfix Admin install from 2.1.0 to 2.2.1.1, kept running up against the following error:


DEBUG INFORMATION:
Invalid query: Can't create table './postfix/vacation_notification.frm' (errno: 150)

Seems like others have had this, but I didn't find a solution posted. Some time Googling and investigating MySQL docs finally had me stumbling across the answer. The MySQL user 'postfixadmin' wasn't originally granted sufficient privileges to the database, and a prior ALTER step hadn't executed, and more importantly, hadn't been flagged as an error. Granting this user the ability to Alter, Create, Drop, and all other structure related commands then running the setup.php again resolved the issue.

Hope this helps someone out there.

Posted by Jim at 7:59 PM | TrackBack

October 16, 2008

Apple Security Update 2008-07 breaks Postfix

At least, if you have compiled your own version it does... More info on this update here.

After running this update and restarting, not only did Postfix not launch correctly, but my logs quickly filled up with warnings about every single file on my drive not being owned by Postfix. Several reboots later I was finally at the point where I could try troubleshooting this, and the easiest fix was to just run a new 'make upgrade' on my current postfix install.

Also, be sure to remove /System/Library/LaunchDaemons/org.postfix.master.plist before rebooting, as this file won't work for a real mailserver, if you were using a .plist in this location to start postfix previously, you'll need to restore a backup of your original file or recreate it from scratch.

Posted by Jim at 8:47 PM | TrackBack

September 15, 2008

Mail server upgraded to 10.5.5

Made the upgrade to Mac OS X 10.5.5 on the mail/web server here, so far so good, no errors that I'm aware of. Made a fresh backup of the main drive just prior to the upgrade, just in case...

Posted by Jim at 8:03 PM | TrackBack

July 11, 2008

More server upgrades

A new version of Policyd is out, it runs as a Perl module and not as a daemon, I've begun looking at upgrading to this version. More modules will have to be installed first, though, to support this, including configuring amavisd-new, which I'm not currently running.

I think I've also found a workable install process for imagemagick, my past attempts at getting this going have all failed. The steps I've found include a number of additional modules which appear to be either required or strongly recommended, and these may not have been installed previously. My last attempt at doing this via MacPorts failed miserably, including having two separate version of Perl installed, which I think I've now managed to remove all traces of.


Posted by Jim at 5:35 PM | TrackBack

February 13, 2008

getnameinfo fixed with 10.5.2

I have been able to confirm that the getnameinfo issue with Mac OS X 10.5 has been fixed with the 10.5.2 update. I never did confirm if this was an issue with 10.5.2 Server or not, but the fix is in the server version also.

Posted by Jim at 8:11 PM | TrackBack

December 19, 2007

Postfix and iPhones

My wife received an iPhone for Christmas this year, and she's been having fun playing with her new toy. She was very excited to find out that she could check her mail from the phone, and as I had never bothered to set up mail on mine, I took a few minutes to set her up. But, this simple exercise took a bit longer than I expected, since my mail server here wasn't set up for this just yet.

I have both POP and IMAP configured, but haven't tested IMAP extensively, and have only used it via the webmail apps I run, and the iPhone was trying to talk on ports that I hadn't set up yet.

The first order of business was to allow mail to be received on port 587, this is done by uncommenting the line starting with #submission in the postfix master.cf file. The submission port is a standard designation for port 587. If you would like to have postfix be able to receive mail on other ports, simply copy/paste that line, and change the word 'submission' to the desired port number, restart postfix, and postfix will now be listening on this new port. This may be handy for getting around certain networks that block the standard mail ports and want you to use their mail gateway. Having an alternate port you can send on may be handy to have in those rare occasions.

The next trick was editing the account setup on the iPhone. After entering all of the account info, the initial connection failed. I began tweaking settings on the mail server, but couldn't easily get back to change my wife's account setup on the phone. I found that by going to the phone's Settings, then Mail, I could select the account, and at the bottom of this screen was an Advanced option, that let me tweak all of the settings.

Despite declining to use SSL encryption, this option was still selected for both incoming and outgoing, very simple to turn off. This screen also lets the default ports be overridden, again, useful to get around certain network blocks.

Once I had tweaked the necessary settings on the iPhone, I had her mail running just fine. Thank goodness for unlimited data plans...

Posted by Jim at 11:05 AM | TrackBack

December 8, 2007

Courier-IMAP 4.2.1 compiled successfully

Well, I resolved my issue with the 4.2.1 release... I had originally mentioned that 4.2.1 just couldn't be made to compile here, and I therefore reverted to the older 4.1.3 release without issues. Well, it turns out that I apparently had some bad data saved from a prior compile attempt, and it was as simple as running a 'make clean' before doing another make that cleared everything out and let me get a good compile.

A 'make install' later, I was up and running with Courier-IMAP 4.2.1. It's the little things that get you, sometimes...

Posted by Jim at 9:00 PM | TrackBack

December 4, 2007

getnameinfo issue with Mac OS X 10.5

I found a rather obscure bug with Leopard while troubleshooting my postfix logs. It seems that mail that I had been getting from some of the mail lists I usually receive was being bounced by my server because it couldn't resolve the IP address based on the DNS supplied by the sending server. If you've read my prior Postfix postings, you know that I'm fairly strict about the servers I accept mail from, and misconfigured servers generally don't get any mail delivered here.

So, this came as somewhat of a surprise that formerly working servers were now being rejected after my upgrade to Leopard. Some troubleshoot assistance from the postfix mail list uncovered the issue, the getnameinfo function in the OS was not resolving DNS addresses that resolved to a CNAME record, or anything other than a PTR record. The unix nslookup and host commands though, worked fine, but postfix relies on the getnameinfo function.

The good news here is that this bug has been reported to Apple, and signs are good that this should be fixed in Mac OS X 10.5.2 when it is released. I'll report back on that after release. For now, my temporary workaround is to identify servers that aren't resolving, and whitelist them in my helo_access list.

Posted by Jim at 8:11 PM | TrackBack

November 22, 2007

newsyslog Revisited

Earlier this month, I wrote about a new utility that handles log rotation in Leopard, and gave a tip on fixing logging for the mail.log. It turns out that my fix wasn't quite right...

The original line in the configuration file was as follows:


/var/log/mail.log 640 5 100 * J

This results in the log file being rotated when it reached 100Kb in size. What I wanted was for the logs to roll weekly as they had with prior systems, and my assumption was that this would continue to happen as part of the periodic.weekly script. Bad assumption.

I don't usually have to scroll too far back in my logs when researching things, but tonight discovered that I had entries going back more than a week, and that the log file wasn't rotating as I thought it should have been. A quick check of the periodic.weekly script revealed that log rotation wasn't there anymore, so I revisited the newsyslog.conf file, and made the following change:

/var/log/mail.log			640  5	   *	$W0D0     J

The asterisk there is in the size column, meaning don't worry about the log size, the $W0D0 is under the when column, this means to rotate weekly, on day 0 (Sunday), at hour 0 (12am).

The man page for newsyslog.conf gives a wealth of info on configuring this utility, and is well worth a read.

Posted by Jim at 11:35 PM | TrackBack

November 4, 2007

New log rotation utility in Leopard

Most folks have no need to ever check their system logs. Some folks check their logs religiously. Mac OS X 10.5 has thrown a new tool into the mix, and it might bite you if you don't know about it.

There is a new command line command called newsyslog, it is called every minute by the file /System/Library/LaunchDaemons/com.apple.newsyslog.plist, and it's config file lives in /etc/newsyslog.conf.

Tonight, I needed to check my mail server logs for some information, and had to search prior logs. At first, my searches made no sense, as I kept coming up with today's date in the data, but my mail logs normally contain a week's worth of data. Well, not anymore, thanks to newsyslog, now they only contain 100Kb worth of data before they roll over to a new log. Ack! A simple fix, commenting out the log cycling for the mail.log file. Hopefully this tip might help out anyone else out there that gets bitten by this.

Posted by Jim at 1:08 AM | TrackBack

November 3, 2007

Details on upgrades

To recap my recent upgrades here, I was transitioning my old web/mail server from a G4 box running OS 10.4 to a new Mac Mini running OS X 10.5. Funny that I wrote about using a Mini as a server back in 2005, and I'm only now finally getting around to putting one in here...

So, the basic process here was shutting down Postfix, then using Carbon Copy Cloner to clone my existing server to the Mac Mini (booted in Target Disk Mode), then rebooting the Mini into the Mac OS X 10.5 Installer. The Installer had absolutely no problems upgrading a PPC version of OS 10.5 to an Intel OS running 10.5, which was great. I really did not want to do a clean install, which would have been more of a hassle in converting mail files and other lower level items.

The next necessary step after installing 10.5 was to install Xcode 3.0, in order to compile all the apps I needed. Once that was done, I was finally able to start getting things up and running.

From prior dry runs, I had done a lot of testing of various packages to make sure that things would compile properly, and run without errors. There was a good bit of trial and error, and lots of googling. And thanks to someone else googling and finding an earlier entry of mine, a helpful tip out of the blue (Thanks to Paul S.) that helped massively. I had partitioned my drive so that I had a nice workspace partition to hold files between attempts at cloning and upgrading, and I had saved a few helpful notes there as well, which was very handy.

As I had mentioned a few days ago, the unix system accounts for postfix, mysql, www, and others, now for some reason all begin with an underscore character, so I had to edit a few config files where these accounts were specifically used to make sure that they reflected the current users. Also, 10.5 now runs Apache 2.2.x and not Apache 1.x, so I had to do some reading up on how this gets configured in order to migrate my config files, there were few surprises there, once I paid attention to the sample config files. Having saved copies of my working config files from earlier runs, it was a simple matter to copy these over before starting other work.

In retrospect, I should have worked on getting the mail server up and running before the web server, I didn't lose any mail, but I just hated it being offline as long as it was...

Starting with the web side, I installed MySQL 5.0.45 using a pre-build package, I used the one built for 10.4 Intel, and plan to upgrade that to a 10.5 specific build once one is available. After installing this, I simply copied my data folder over, ran the mysql_upgrade script, and all was well. Next, I compiled DBI-1.601 and DBD-mysql-4.005. For some reason I wasn't able to track down, DBD insisted on looking for mysql/lib files in /mysql/lib/mysql, even though my install never mentioned this path anywhere. Some googling finally revealed that the easiest fix was simply to fake it with some symlink trickery:

cd /usr/local/mysql/lib
sudo mkdir mysql
cd mysql
sudo ln -s ../*

DBI compiled fine, DBD threw up an error about incompatible pointers, which I was stuck at for a day or two before finding out that this was just a warning and could be ignored. Sure enough, it ran just fine, and I found that MovableType was now working fine. During the final install of everything, I discovered that I had to reset access privs for my web folder in order for MT to be able to write files, but after doing that, it worked fine again. I'm saving my upgrade to MovableType 4.x for another day.

Compiling Postfix was fairly straightforward, as before, I built Postfix according to the standard install docs to include MySQL and PCRE support, but this time included SASL in the mix. It is very important to read the SASL docs, there was a bit of needing to create symlinks and make sure that header files were in the right locations, but once I followed all the steps outlined, it compiled fine.

The Courier-IMAP pieces drove me nuts for several days. Courier-IMAP 4.2.1, the latest build, just couldn't be made to work here, I eventually tried building an older version, 4.1.3, and that worked just fine. Courier-Authlib 0.60.2 compiled but had problems running, the trick mailed in my Paul S. was to enter the following before doing the compile:

export MACOSX_DEPLOYMENT_TARGET=10.5

This handy command has been around for a few OS releases now, and forces some settings that apparently don't get set otherwise, a quick google search found many packages needing this to compile properly. Once set, AuthLib compiled properly and more importantly, ran properly.

Despite doing the 'migrate' steps, though, my old Courier settings never made it over, and so I had to edit the authmysqlrc and some other Courier files by hand using my older versions as templates, but this work was done in short order.

One site that helped a lot in checking over some of my steps was this one:

http://switch.richard5.net/isp-in-a-box-v2/building-the-mail-server-components/building-the-courier-imap-server/

The versions used there weren't current, but helped to validate what I was trying to do here, and setting the proper CFLAGS and compile arguments. His setup there was very similar to mine, virtual domains, MySQL authentication, etc, which was a great help.

With this done, I was now actually able to check mail the last necessary step, which made a good stopping point for the night with a fairly functioning server.

The next day, I tested a few more functions of the system, and found that one of the web packages I had installed was having problems with MySQL. This turned out to be a PHP issue connecting to MySQL, it was looking for the mysql.sock file in /var instead of /tmp. The easiest fix here was to create a /etc/php.ini file, consisting of the following:

; Default socket name for local MySQL connects.  If empty, uses the built-in MySQL defaults.
mysql.default_socket = /tmp/mysql.sock
; Default socket name for local MySQL connects.  If empty, uses the built-in MySQL defaults.
mysqli.default_socket = /tmp/mysql.sock

The second section for mysqli was required for version of MySQL 4.1 and later, once this was in place and Apache stopped and started, this problem was now history.

The last hurdle I had was getting policyd running, this is the greylisting package I use with Postfix. I had been struggling for some time to get newer builds of this running. I had somehow hacked the 1.7.x version into running previously, and was never able to duplicate my success with later builds. Thanks to some outstanding work by the developers, the final fixes to this are now available in the latest SVN builds, and I was able to get the 1.9.x experimental build to compile successfully, and more importantly, to run successfully as well.

In closing, what I'd like to say here is that when you're rolling your own code, patience is your best friend. Take things one step at a time, make sure you have a backup, and when you hit a wall, do searches and ask questions on mailing lists until you find the answers you need. If all else fails, post about your failures, and someone else might stumble across your post and supply the answers you need, it's amazing how things like that work out sometimes.

Posted by Jim at 9:27 PM | TrackBack

November 2, 2007

Server upgraded successfully

The server here has been successfully upgraded to Mac OS X 10.5. A few tips rolled in earlier this week that resolved the last of my compile issues (details to follow soon), so last night I cloned everything over to the new box and started the upgrade process.

One important tip, installing Xcode is kinda important. It's the little things you forget to do... :)

Posted by Jim at 11:34 AM | TrackBack

October 28, 2007

Upgrades... Hmmm.

A relatively sleepless weekend, and not in a good way. Here's an interesting tidbit, the common unix accounts such as postfix, mysql, www, and others, are now prefaced by an underscore character in OS X 10.5. Why, I have no idea, but when configuring scripts, make sure to change the usernames.

The switch to Apache2 for the most part went pretty well, it took a bit of trial and error to get my virtual domains working, but once I went back and poured over the sample configs, it all started to make sense. Just copying and pasting relevant bits from my old config files was not the way to go. :)

Minor issues compiling the DBD::mysql module, I had to use a slightly older version and it worked fine.

Courier-IMAP is what drove me absolutely nuts, I finally thought I had it all going, and then discovered that the auth module was throwing errors in the log, and I'm still trying to track that down. Also, despite attempting to migrate my older Courier settings, this apparently didn't happen, best thing may be to build it all up from scratch.

Posted by Jim at 10:51 PM | TrackBack

October 26, 2007

Leopard Day...

Mac OS X 10.5 (Leopard) is available in stores, and of course I've been playing with it for a bit. The new tabbed Terminal is great, having several terminal sessions all open in separate tabs instead of multiple windows is great, it really helps keep things organized.

Anyway, my first pass at upgrading my mail/web server from PPC 10.5 to Intel 10.5 went pretty well, everything actually seemed to launch and run correctly, web services worked, postfix was running, etc. Of course, I want to compile Intel binaries and not use the PPC codes, so I've been testing some installs. So far, most of them are going well, a few minor snags though, but I hope to have things working on the new hardware by the time the weekend is done.

Posted by Jim at 9:19 PM | TrackBack

October 18, 2007

Upgrade prep

I had forgotten what a pain in the ass a major upgrade can be... Made even worse by switching platforms, going from a PowerPC based Mac to an Intel based one.

I did a number of 'test compiles' on the Mac Mini just to make sure that things looked like they'd be working come Leopard day, and figured out that in order to clone my old server over to the new box, I'd have to format the drive with a GUID partition map, something not done by default when formatting from a G4 system... It's the little things that get ya.

So, quick checklist of things to do:

Format Mini's drive as GUID
Download latest MySQL 5, MovableType, Courier-IMAP, Courier AuthLib, Postfix, PCRE, PHP, Cyrus SASL, policyd
Shut down all services
Clone drive from G4 to Mac Mini
Boot 10.5 Installer
Upgrade system
Install MySQL, PHP (built for 10.4.x, will upgrade to 10.5 versions when available)
Build/Install PCRE, Cyrus SASL, Courier AuthLib, Courier IMAP, then Postfix (order probably important)
Build/Install policyd
Keep fingers crossed

Hopefully I haven't forgotten anything that's a dependency, if so, I'm sure I'll find out about it.

I'll probably try for a dry run this weekend, and see what happens.

Posted by Jim at 11:38 PM | TrackBack

September 18, 2007

Major Upgrades

I'm in the beginning stages of some major upgrades to the machine running the web/mail system here, every major piece of software on the back end is being upgraded, and the hardware is seeing some major changes as well Gone will be the old G4 system, in favor of a new Intel Mac, and a whole host of software upgrades for the new hardware to bring everything current with the latest releases (MySQL 5, MovableType 4, Courier-IMAP 4.1.3, Postfix 2.4.5, pcre 7.3, PHP 5.x, etc, etc). Oh, and of course, Mac OS X 10.5, when it ships, hopefully next month.

The plan at this point is to prep the new hardware, install all the software, then migrate the data from the old server, followed by much testing to make sure that everything is working as it should be. If all goes well, shortly after Mac OS X 10.5 ships, I'll be able to wipe the drive and install all the latest bits and be able to go live in early November. There should be no noticeable difference (unless I play with MovableType a bit!), but my UPS will have a lighter power load to deal with, at least. :)

Posted by Jim at 3:04 PM | TrackBack

October 9, 2006

Updated Whitelist

It has been a while since I last posted my whitelist that I use with greylisting, and I've since converted to Policyd. So, here's my current whitelist:

+----------------+--------------------------------------------------------------+
| _whitelist     | _description                                                 |
+----------------+--------------------------------------------------------------+
| 127.%.%.%      | # localhost                                                  |
| 192.168.%.%    | # private netblock                                           |
| 10.%.%.%       | # private netblock                                           |
| 12.5.136.%     | # Southwest Airlines (unique sender, no retry)               |
| 12.107.209.244 | # kernel.org mailing lists (high traffic, unique sender per  |
| 12.107.209.250 | # sourceware.org mailing lists (high traffic, unique sender  |
| 63.82.37.110   | # SLmail                                                     |
| 64.7.153.18    | # sentex.ca (common pool)                                    |
| 64.12.137.%    | # AOL (common pool) - http://postmaster.aol.com/servers/imo. |
| 64.12.138.%    | # AOL (common pool)                                          |
| 64.124.204.39  | # moveon.org (unique sender per attempt)                     |
| 64.125.132.254 | # collab.net (unique sender per attempt)                     |
| 64.233.170.%   | # gmail (common server pool)                                 |
| 65.82.241.160  | # Groupwise?                                                 |
| 66.100.210.82  | # Groupwise?                                                 |
| 66.135.209.%   | # Ebay (for time critical alerts)                            |
| 66.135.197.%   | # Ebay (common pool)                                         |
| 66.162.216.166 | # Groupwise?                                                 |
| 66.206.22.%    | # PLEXOR                                                     |
| 66.218.66.%    | # Yahoo Groups servers (common pool, no retry)               |
| 66.218.67.%    | # Yahoo Groups servers (common pool, no retry)               |
| 66.218.69.%    | # Yahoo Groups servers (common pool, no retry)               |
| 66.27.51.218   | # ljbtc.com (Groupwise)                                      |
| 66.89.73.101   | # Groupwise?                                                 |
| 68.15.115.88   | # Groupwise?                                                 |
| 194.245.101.88 | # Joker.com (email forwarding server)                        |
| 195.235.39.19  | # Tid InfoMail Exchanger v2.20                               |
| 195.238.2.%    | # skynet.be (wierd retry pattern)                            |
| 195.238.3.%    | # skynet.be (common pool)                                    |
| 204.60.8.162   | # Groupwise?                                                 |
| 204.107.120.10 | # Ameritrade (no retry)                                      |
| 205.188.139.%  | # AOL (common pool)                                          |
| 205.188.144.%  | # AOL (common pool)                                          |
| 205.188.156.%  | # AOL (common pool)                                          |
| 205.188.157.%  | # AOL (common pool)                                          |
| 205.188.159.7  | # AOL (common pool)                                          |
| 205.206.231.%  | # SecurityFocus.com (unique sender per attempt)              |
| 205.211.164.50 | # sentex.ca (common pool)                                    |
| 207.115.63.%   | # Prodigy (broken software that retries continually with no  |
| 207.171.168.%  | # Amazon.com (common pool)                                   |
| 207.171.180.%  | # Amazon.com (common pool)                                   |
| 207.171.187.%  | # Amazon.com (common pool)                                   |
| 207.171.188.%  | # Amazon.com (common pool)                                   |
| 207.171.190.%  | # Amazon.com (common pool)                                   |
| 213.136.52.31  | # Mysql.com (unique sender)                                  |
| 216.136.226.0  | # Yahoo Mail?                                                |
| 216.157.204.5  | # Groupwise?                                                 |
| 217.158.50.178 | # AXKit mailing list (unique sender per attempt)             |
| 209.237.227.%  | # SpamAssassin mailing list                                  |
| 66.35.250.%    | # lists.sourceforge.net                                      |
| 196.25.240.%   | # saix.net                                                   |
| 196.4.160.%    | # internet solutions (business smtp)                         |
| 196.35.77.%    | # internet solutions (dialup smtp)                           |
| 196.25.69.%    | # telkom                                                     |
| 196.2.50.%     | # mweb (dialup smtp)                                         |
| 196.2.49.%     | # mweb (business smtp)                                       |
| 196.2.24.%     | # mweb (business smtp)                                       |
| 205.174.22.%   | # Delta                                                      |
| 64.12.136.%    | # AOL (common pool)                                          |
| 209.104.48.%   | # Ticketmaster                                               |
| 209.104.56.%   | # Ticketmaster                                               |
| 216.154.234.%  | # Tickets.com                                                |
| 216.155.201.%  | # Yahoo                                                      |
| 216.155.203.%  | # Yahoo                                                      |
| 63.240.36.%    | # Priceline.com                                              |
| 66.163.187.%   | # Yahoo                                                      |
| 66.94.237.%    | # Yahoo                                                      |
| 207.171.160.%  | # Amazon.com (common pool)                                   |
| 64.125.133.202 | # collab.net (unique sender per attempt)                     |
| 63.169.44.140  | # Southwest Airlines (unique sender, no retry)               |
| 144.9.158.%    | # American Airlines                                          |
| 151.193.203.%  | # US Airways                                                 |
| 165.212.65.%   | # United Airlines                                            |
| 207.235.15.%   | # Continental Airlines                                       |
| 66.142.137.%   | # Continental Airlines                                       |
| 139.72.190.%   | # Northwest Airlines                                         |
| 68.142.200     | # Yahoo                                                      |
| 68.142.199     | # Yahoo                                                      |
| 209.191.126    | # Yahoo                                                      |
| 64.4.240.%     | Paypal                                                       |
| 216.113.188.%  | Paypal                                                       |
| 209.104.46.%   | # Ticketmaster                                               |
| 209.104.33.%   | # Ticketmaster                                               |
| 206.165.246.%  | # Hotwire.com                                                |
| 64.95.142.%    | # Hotwire.com                                                |
| 66.211.168.%   | Paypal                                                       |
+----------------+--------------------------------------------------------------+
86 rows in set (0.00 sec)
Posted by Jim at 6:52 PM | TrackBack

September 10, 2006

Another Webmail client

After playing around with SquirrelMail for a while, I ended up not being terribly happy with it, mainly because it was all text based, no fancy icons for functions, etc. After a bit of web searching I came across the RoundCube Webmail Project, which I liked a lot, as it looked somewhat similar to Mail in OS X.

Roundcube was very easy to get installed on my OS X based mail/web server, and even though the software is only beta2 currently, seems to be running just fine. There seems to be a pretty healthy user base already running this, so I have high hopes that it will continue to be developed to a 1.0 release and beyond.

Posted by Jim at 9:45 PM | TrackBack

August 11, 2006

SquirrelMail

This afternoon, I finally broke down and installed a webmail application here to read mail when away from our usual machines, after scouring the web for info, I finally settled on SquirrelMail, a well rounded package for doing email via the web.

Installation was a snap, the online info on installing under Mac OS X 10.1 still worked fine even though the mail server here is 10.4, and aren't really any different than a standard install, so I'm kind of missing the point of having separate install instructions... I'll put this through its paces over the next few weeks and will report back if any signifigant issues are uncovered.

Posted by Jim at 3:19 PM | TrackBack

June 19, 2006

Policyd for Postfix

I've decided to switch the greylisting software in use here from Gld to Policy Daemon. Policyd has a lot more flexibility with greylisting, including some automated blacklisting and whitelisting functionality, and is overall a lot more robust.

I may be the first user of this running it on OS X, the author hadn't heard of any other Mac users using it, but with some very minor tweaks to the MAKE file it was up and running with no problems, these changes should be part of releases after 1.77. As I've done with most other utils like this I run on the server here, I created a StartupItem for it, and something new for me, created a LaunchDaemon to handle the cleanup instead of a cron job, as cron is depreciated with OS X 10.4 Both were easy enough to do, plenty of docs available elsewhere on setting these up.

Posted by Jim at 11:27 PM | TrackBack

February 18, 2006

Upgrade to 10.4.5

The server here has been upgraded from 10.4 to 10.4.5. I was a little apprehensive about this update as I didn't have a chance to test prior to performing it, but everything went well, Postfix kept chugging, web services uninterrupted, other miscellaneous non-standard compiled code went well. Looks like Apple didn't stomp on anything that I'd upgraded. ;)

The big hangup was an issue with Carbon Copy Cloner, under 10.4, it wasn't able to clone my drive due to an authentication issue, normally I'd clone my drive and test out the upgrade first, so I just had to chance it. The main reason I needed to upgrade was 10.4 apparently had some issue that would keep log files from rotating properly, and possibly other scheduled tasks from running, so it was high time to get that issue fixed.

Posted by Jim at 3:03 PM | TrackBack

January 8, 2006

Postfix Pop-Before-SMTP

While on vacation recently, I found myself unable to send email while at a wireless hotspot, I couldn't connect to my usual ISP's outbound mail server, and the hotspot wasn't providing this access either. I never actually send mail through my own domain, and with my access restrictions, was unable to do so from a foreign ISP.

After some digging, I installed Pop-Before-SMTP, a handy little utility that integrated with Postfix to create an access list that can be checked to allow outgoing mails to be sent. It works by creating a list of IPs that have successfully logged in to check mail (via POP, or IMAP), then allows those specific IPs to send outgoing mail, and will automatically expire those IPs in a configurable number of minutes. Also, to avoid issues with mail rejections, I configured my Postfix to relay all outgoing mails through my home ISP rather than delivering directly. It worked great the few times I needed to use it.

Posted by Jim at 11:17 PM | TrackBack

September 17, 2005

Postfix & Tiger upgrade

I've finally made the upgrade to Tiger on my mail server. That is, I've finally 'successfully' upgraded, it took a few attempts to get everything just right for what I wanted, and now that I'm done I thought I'd share a few details.

Probably the most important step is to make sure that you have a good backup before starting. For me, this step was critical, as it allowed me to back out at any time and get running again from the backup. I wasn't just upgrading from 10.3 to 10.4, but was also upgrading Postfix, MySQL, and PCRE all at the same time...

Tiger (10.4) includes version 2.1.5 of Postfix, for most folks this is probably more than sufficient. However, I wanted to upgrade to the Postfix 2.2.5 release, as it includes a few more bells & whistles that I wanted, I wanted to bring MySQL current, and also bring PCRE (Perl Compatible Regular Expressions) up to date. Basically, this meant upgrading my system to Tiger, upgrading/compiling each new piece, and finally getting it all up and running.

MySQL was a snap, thanks to a ready made Installer built for Tiger. I was upgrading from an earlier 4.1.x release, so there were no worries about my databases not working correctly, and it was a simple matter of copying over the data directory from the old path to the new and getting MySQL running.

PCRE compiled well, but prior to compiling I also upgraded to the latest version of Xcode to get all the latest libraries installed. Standard docs on how to install worked just fine.

The last and trickiest step was getting Postfix compiled properly. After trying a few times and having problems, I finally took a step back and found my error, a simple typo caused from a copy/paste error when trying to get both the MySQL and PCRE code compiled in. For reference, here's the correct MAKE instruction for that:

make -f Makefile.init makefiles \
    'CCARGS=-DHAS_MYSQL -I/usr/local/mysql/include -DHAS_PCRE -I/usr/local/include' \
    'AUXLIBS=-L/usr/local/mysql/lib -lmysqlclient -lz -lm -L/usr/local/lib -lpcre'

After that, it was just a matter of running Make, and then Make Upgrade.

The last and trickiest step was to get Postfix running at Startup again. The Tiger upgrade removed my Postfix StartupItem (included with the Postfix source code), so that just needed copied back to /Library/StartupItems, and I also removed the /System/Library/LaunchDaemons/org.postfix.master.plist file that Tiger includes for starting Postfix on demand, as I wanted this running all the time, and it also didn't seem to be launching my newly compiled version of Postfix properly. I also needed to edit the /etc/hostconfig file to change MAILSERVER=-NO- to -YES-, and after that Postfix launched perfectly.

In hindsight, I probably should have upgraded 10.4 to 10.4.2 before upgrading everything else, but no mail related updates seem to be part of the upgrade, so this shouldn't be a problem. But if you're planning on going to 10.4 and recompiling other software, get your OS fully current first, then start in on your other work, it'll save you the trouble later.

I used Mike Bombich's excellent Carbon Copy Cloner to clone my working system to a backup drive, upgrade that and was able to do all my testing on a backup drive prior to the actual upgrade, and at any time I could simply reboot from my primary drive and be up and running. Very useful when you can't have a production system down long but need to do some 'live' testing.

Posted by Jim at 10:55 PM | TrackBack

May 15, 2005

Postfix and road warriors

Ok, so you've set up your own mail server on your PowerBook for when you're on the road, and don't know what ISP you'll be connecting to next. Now you can send mail from anywhere, as long as you've got an internet connection, and it'll go through just fine, right? Wrong. An incresing number of mail servers worldwide are restricting where they accept mail from, dynamic or dial up IP ranges are blocked in many cases, mail from IP addresses without a resolvable DNS name, or even ANY DNS name, etc. And some mail is just dropped, you'll never know it bounced. So, what to do, you ask?

The answer is to relay your mail through another mail server. Generally, this is something that most mail servers should be set up to avoid doing (as Postfix does), but in this case, what we want to do is get mail from our mobile mail server to a fixed mail server elsewhere. If your regular ISP has a mail server set up to allow SASL (Simple Authentication and Security Layer) access, Postfix can be configured to support this, and once authenticated, mail can then be relayed to your ISP's mail server, which should then go through fine (assuming your ISP has a properly configured mail server).

A second option would be if you've already set up Postfix as a mail server at home (as I have), and your ISP doesn't provide SASL access, would be to pass mail to your home system, which can then send the mail out. A bit convoluted, but it will get the job done, and you won't need to worry about mails not going through.

Not quite the simple plug and play solution Mac folks are used to, but mail servers are not known for being simple, nor should they be. Anyone attempting to set up their own server should understand the technology involved, and be prepared to support themselves, and probably consider joining one of the mailing lists available.

Posted by Jim at 10:47 PM | TrackBack

Postfix Enabler for Mac OS X 10.4

Postfix Enabler for Tiger is now available, finding this out has saved me some time trying to test the prior release and figure out if it worked. This will activate the built in version of Postfix and within moments give you a fully functioning mail server.

Posted by Jim at 10:27 PM | TrackBack

May 1, 2005

Tiger mail server

Now that Mac OS X 10.4 is shipping, I know that some folks running mail servers will be interested in upgrading their systems. I've heard that 10.4 includes Postfix 2.1.5, the last official release in the 2.1.x series. Even though Postfix 2.2.x is available, light duty servers can still manage just fine with 2.1.5, but I'm assuming that MySQL support and other goodies probably weren't compiled into the build.

I'm going to get my server here upgraded in the next few days to Tiger, which for me will involve recompiling Postfix (I'll probably upgrade to 2.2.3), but I'll also see about doing some testing of the base 10.4 config and report my findings here.

Posted by Jim at 1:39 PM | TrackBack

January 27, 2005

Virtual domains fully functional

Well, it took a while, and a lot of testing, but I now have all the pieces in place. Postfix, MySQL, Courier IMAP, Postfix Admin, and virtual domains, all working together. See my prior posting for links to the various packages and more tips.

The last hurdle I took care of last night, eliminating the old mailbox format accounts on my server, and instead going with maildir format accounts that would work properly with all the above software. For anyone setting up a new server, save yourself some trouble and use the maildir format from the beginning.

Posted by Jim at 7:47 PM | TrackBack

January 25, 2005

Courier IMAP working

After doing some struggling and discovering that the latest versions of Courier IMAP (4.0.x) won't compile under OS X, I went back and got the 3.0.8 release installed and functional thanks to setup instuctions here and a handy .pkg installer of the Courier IMAP software from here. After struggling through understanding all the options, I had it up and running. Mostly.

One thing that quickly became painfully obvious was that Courier IMAP was designed only to access Postfix Maildirs, not the default mailbox files used by unix system accounts. So, my next step will be to get all users mail transitioned over to this format, and get Postfix to recognize my local domain as well as my virtual one...

Posted by Jim at 11:26 PM | TrackBack

January 23, 2005

Postfix, MySQL, and Virtual Domains

Well, after a few days of struggling, I've finally gotten Postfix Admin up and running here, and as part of that setup (and the handy HowTo posted on their site), I've got MySQL properly working with Postfix, and have virtual domains working as well.

The Postfix Admin software is a collection of PHP scripts for creating and maintaining domains and mailboxes, and eliminates all that mucking about with MySQL commands. They've got a good support forum with some great users there, which helped tremendously in getting this running.

Posted by Jim at 11:53 PM | TrackBack

January 12, 2005

Upgrades and more upgrades

Last week, I wrote of my attempts to get the latest MySQL and MovableType working together. Well, looks like I now have things going properly. When I upgraded MySQL this time, I left all the password information in the old format, and the upgrades went without a hitch.

Of course, I also went ahead and upgraded to Mac OS X 10.3.7, which meant (read here for more info) having to reinstall Postfix 2.1.x, so I took the opportunity to upgrade to 2.1.5 and added in MySQL support there too...

So, after several attempts at getting it all going, it looks like I'm finally all up to date on the various bits of my server now, finally! The Postfix part was easy (oh, also upgraded the PCRE software to 5.0), the main trick was making MovableType 3.14 happy with MySQL.

It looks like I can finally get virtual domains going, now that I can work with MySQL in Postfix. But I think I'll save installing Cyrus IMAP for another day...

Posted by Jim at 11:26 PM | TrackBack

November 20, 2004

More Whitelisting

I just had a problem yesterday caused by my greylisting, my wife missed a confirmation email from Delta on a flight she booked at the last minute because the greylisting temporarily bounced the email. I've since added Delta's server to my whitelist, and I'm searching for any other Airline's mail servers to add them in as well. I'll be posting an updated whitelist when I've compiled a bit more info.

Posted by Jim at 11:26 PM | TrackBack

November 17, 2004

Updated greylisting whitelist

Over the last few months, I've been tweaking the whitelist I use in conjunction with my greylisting software on my mail server. The whitelist is a list of known mail domains that I don't wish to delay mail from. Most greylisting packages will include a sample whitelist, I've found a few posted on the web, and have collected the most common settings, but over time have added a few new entries to several of the larger emailers.

If you're running a greylisting process, I'll leave it to you to figure the proper way to import these entries into your system...

mysql> select * from whitelist;
+----------------+---------------------------------------------------------------+
| mail           | commentaire                                                   |
+----------------+---------------------------------------------------------------+
| 12.5.136       | Southwest Airlines (unique sender, no retry)                  |
| 64.12.136      | AOL                                                           |
| 64.12.137      | AOL                                                           |
| 64.12.138      | AOL                                                           |
| 152.163.225    | AOL                                                           |
| 205.188.156    | AOL                                                           |
| 205.188.157    | AOL                                                           |
| 205.188.159    | AOL                                                           |
| 64.125.132.254 | collab.net (unique sender per attempt)                        |
| 64.125.133.202 | collab.net (unique sender per attempt)                        |
| 66.135.209     | Ebay                                                          |
| 66.135.197     | Ebay                                                          |
| 66.218.66      | Yahoo Groups servers (common pool, no retry)                  |
| 66.218.67      | Yahoo Groups servers (common pool, no retry)                  |
| 66.218.69      | Yahoo Groups servers (common pool, no retry)                  |
| 195.238.2      | Skynet                                                        |
| 195.238.3      | Skynet                                                        |
| 204.107.120    | Ameritrade (no retry)                                         |
| 205.206.231    | SecurityFocus.com (unique sender per attempt)                 |
| 207.115.63     | Prodigy - broken software that retries continually (no delay) |
| 207.171.168    | Amazon                                                        |
| 207.171.180    | Amazon                                                        |
| 207.171.187    | Amazon                                                        |
| 207.171.190    | Amazon                                                        |
| 213.136.52.31  | Mysql.com (unique sender)                                     |
| 217.158.50.178 | AXKit mailing list (unique sender per attempt)                |
| 66.94.237      | Yahoo Groups servers (common pool, no retry)                  |
| 205.188.144    | AOL                                                           |
+----------------+---------------------------------------------------------------+
28 rows in set (0.00 sec)
Posted by Jim at 11:54 PM | TrackBack

November 7, 2004

Great Mac/Postfix site

I came across the site for the ECM Mail Server System a while back, it combines Postfix, MySQL, Courier IMAP, and some other cool utilities into what looks like a pretty solid package, all running under OS X. They've recently revised their code to be up to date with 10.3.5, and the latest Courier IMAP software. Definitely worth a look if you're running Postfix as a mail server and want to add more bells and whistles. When I get around to doing my upgrade here, I'll be following their steps closely.

Posted by Jim at 10:06 PM | TrackBack

October 6, 2004

Correction on ipop3d

The other day I mentioned that ipop3d was the pop3 server built into OS X, well, I was mistaken. It turns out that this version of ipop3d was actually installed by Postfix Enabler, this particular version is from the UW-IMAP project, and was in fact not a built in part of OS X.

Sorry for any confusion.

Posted by Jim at 9:14 PM | TrackBack

Apple updates Postfix

Yesterday, I read about Apple's latest security update, and noticed that it updates Postfix. After doing some checking on the Postfix mailing list, it appears that this update includes a new version of the Postfix code, and not just a minor config change.

What this means is that anyone (like me) who has updated their version of Postfix from the original Apple code will have their version stomped by intalling this update...

All is not lost, though. In my entry Postfix 2.1 on OS X, I mentioned an article at AFP548 which covers the install steps to get Postfix 2.1.1 running on OS X (article currently 'offline' there but still accessible, there are issues with the SASL implementation mentioned that are keeping the document from being officially sanctioned), and a short bit of code is listed to archive the current Postfix code.

This code was meant to archive the original Apple code so you would have a backup, this same code could be used to backup your current Postfix install, so that you can run the Software Update, then restore your Postfix.

Of course, you also have the choice of ignoring this particular update, but it's likely that when/if Apple released a 10.3.6 update, that the Postfix code will be rolled into that as most updates are, so you may get it down the road without realizing it.

My choice here is I think to just recompile Postfix using the AFP article's steps, and install per that article's instructions. This will also be an opportunity to upgrade to Postfix 2.1.4 (I'm running 2.1.3), and include MySQL support, which I'm sure I didn't do before, to help support the virtual domains that I want to implement.

Posted by Jim at 12:19 AM | TrackBack

September 27, 2004

Virtual Domains, almost there

After some strange errors caused by a late-night typo, I've got virtual domains about 50% working. That is, they're working fine in Postfix, mail seems to be getting delivered properly, Postfix is accepting the virtual accounts, etc. But there's one important piece missing... Mail clients can't login to read their mail!

I'm using POP3 for mail delivery, and the POP3 client built into OS X, ipop3d, apparently is only capable of handling local accounts, and not virtual accounts. So, I'm off to find a replacement POP3 client...

At this point I'm leaning towards using Cyrus IMAP, which handles both POP3 and IMAP, and also provides Secure Socket Layer (SSL) support, mainly because Cyrus is what's bundled with Mac OS X Server, but I'm still early in the process on deciding what to run. The other popular choice is Courier-IMAP, this will take some reading to see what seems to run better under OS X.

Posted by Jim at 11:28 PM | TrackBack

September 24, 2004

Virtual Domains with Postfix

Since I picked up a few extra domains earlier in the month, I wanted to play with adding one of them to Postfix as a virtual domain. Basically, this lets me set up mail accounts using the other domain names, but does not require that local unix level accounts be created for each user.

I'm still doing some reading on this and probably won't do anything for a month yet, as work has been incredibly busy lately, but I'll definitely post all about it when I get something going.

Posted by Jim at 12:11 AM | TrackBack

July 28, 2004

gld upgraded to 1.3

Gld, the greylisting system I use here, seems to have received an update while I was away. Seems the prior version crashed on my system while I was out, it was down about 24 hours before I noticed and brought it back online, so I went in search of an update once I was back.

No problems so far, but it's still a pain to compile on the Mac, and getting it going still involved some editing of the make file to get it working. I still hate the command line, but it's a necessary evil at times...

Posted by Jim at 11:38 PM | TrackBack

July 15, 2004

More Postfix setup notes

Well, I learn a little more about Postfix every day, and most recently it was to pay attention to the order you run your restrictions in... I had an odd problem that wasn't getting fixed like I intended, and had to run to the experts on the Postfix mailing list for help, and was set straight in short order. Read along for notes and my latest config...

One of these days, I need to put together a full tutorial on setting up Postfix and then just keep that updated, but for now, here's where I'm at.

First, the proper order of restrictions. If you get this wrong, you'll make mistakes and never know why things aren't working right. I was under the impression that the HELO restrictions happened first (since HELO happens first in SMTP), but that isn't how Postfix handles things. So, here's the actual order for restrictions:

smtpd_client_restrictions
smtpd_helo_restrictions
smtpd_sender_restrictions
smtpd_recipient_restrictions


Did you score 100%? Good for you if you did. ;) My problem was that I was trying to filter a spammer that was proclaiming himself to be sending mail from a mail server with my own DNS name. As if! The blocks I put in place weren't working (it was hitting my other spam filters first and getting bounced), so that's where I started digging.

It turns out that Postfix Enabler, which I run to help manage Postfix on my Mac, set up its own smtpd_client_restrictions with the rbl filters and some other things, and in my other settings I didn't set up my own list of restrictions for that block figuring that PE had it under control, and thinking my HELO restrictions would happen first. So, once I was straightened out on that, I decided to rework my set of restrictions to make things flow a bit better.

One thing that I will eventually go in and change is the access restrictions, and create separate lists for recipient, client, and sender restrictions. Postfix Enabler manages the one list called 'access' and rebuilds it every time I update (using the postmap command in the background, I'm assuming), and for now I'd rather not have to do that manually, but if my lists grow much more I may need to separate them to avoid filter errors.

So, here's my current config, with a few notes:


###Start PostfixEnabler###
alias_maps=hash:/etc/postfix/aliases
alias_database=hash:/etc/postfix/aliases
smtpd_sender_restrictions=hash:/etc/postfix/access
inet_interfaces=all
mynetworks_style=subnet
message_size_limit=10240000
mydomain=wrightthisway.com
myhostname=wrightthisway.com
smtpd_recipient_restrictions=permit_mynetworks,check_recipient_access hash:/etc/postfix/filtered_domains
unknown_local_recipient_reject_code=550
###End PostfixEnabler###

As indicated, this is what Postfix Enabler puts in on its own, most of those settings I'm not overwriting, but the restrictions definitely get changed. Also, I removed all the rbl listings from Postfix Enabler, so it now no longer creates its own client restrictions setting.


###Start Custom Config###
disable_vrfy_command = yes
default_process_limit = 10
smtpd_error_sleep_time = 30

strict_rfc821_envelopes = yes
smtpd_helo_required = yes

Same as what I've had before here, being strict with the protocols, setting some values to better manage bad servers, etc.


smtpd_helo_restrictions=
check_recipient_maps,
check_helo_access hash:/etc/postfix/helo_access,
check_client_access hash:/etc/postfix/access,
check_sender_access hash:/etc/postfix/access,
reject_unknown_hostname,
reject_invalid_hostname,
reject_non_fqdn_hostname,
reject_unknown_client,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,

Now, here we have some real changes. I had copied my previous settings for client restrictions (that Postfix Enabler created for me) and brought those into the helo restriction block, and placed these after the checks I wanted in there first.

Specifically, the check_recipient_maps will make sure that the recipient actually has an account on my server, the check_helo_access is what I wrote about here, then the client and sender access filters kick in so that I can specifically allow certain email addresses or server names to pass through without going through further filters (for users on misbehaving servers that their admins won't fix, or some mail lists that happen to show up on a spammer listing, etc.), then my usual checking to make sure that the server I'm talking with checks out and is well configured.


smtpd_recipient_restrictions =
reject_unauth_destination,
check_recipient_access hash:/etc/postfix/access,
check_client_access hash:/etc/postfix/access,
check_sender_access hash:/etc/postfix/access,
check_policy_service inet:127.0.0.1:2525,
reject_rbl_client sbl-xbl.spamhaus.org,
reject_rbl_client list.dsbl.org,
reject_rbl_client relays.ordb.org,
reject_rbl_client dnsbl.njabl.org,
reject_rbl_client bl.spamcop.net,
reject_rbl_client dnsbl.ahbl.org,
reject_rbl_client dnsbl.sorbs.net,
reject_rbl_client relays.visi.com,
reject_rhsbl_client blackhole.securitysage.com,
reject_rhsbl_sender blackhole.securitysage.com,
reject_rhsbl_client rhsbl.ahbl.org,
reject_rhsbl_sender rhsbl.ahbl.org,
reject_rhsbl_client rhsbl.sorbs.net
reject_rhsbl_sender rhsbl.sorbs.net,
reject_rhsbl_client block.rhs.mailpolice.com,
reject_rhsbl_sender block.rhs.mailpolice.com,
reject_rhsbl_client dynamic.rhs.mailpolice.com,
reject_rhsbl_sender dynamic.rhs.mailpolice.com,
reject_rhsbl_client bogusmx.rfc-ignorant.org,
reject_rhsbl_sender bogusmx.rfc-ignorant.org,
reject_rhsbl_client dsn.rfc-ignorant.org,
reject_rhsbl_sender dsn.rfc-ignorant.org

smtpd_data_restrictions =
reject_unauth_pipelining,
permit

Here we can actually check the recipient access list, and I also check client and sender again for good measure, but I think perhaps that's not needed again here. Next is a check using the Gld Greylisting software, this will force a server to try a second time to deliver the message, which most spammers won't, then once the mail comes in again, we run it through the various block lists. If it passes those, it should be pretty clean.


header_checks = regexp:/etc/postfix/maps/header_checks
mime_header_checks = regexp:/etc/postfix/maps/mime_header_checks
body_checks = regexp:/etc/postfix/maps/body_checks

unknown_address_reject_code = 550
unknown_client_reject_code = 550
unknown_hostname_reject_code = 450

default_rbl_reply=$rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason} - see http://$rbl_domain for additional info. If this was actually a legitimate email to a real user, please forward this message to postmaster@wrightthisway.com for assistance.

###End Custom Config###

Here we stop the pipelining that some spammers use to flood a server, then do the header/body checks I've covered previously, set some reject codes (I'm now doing a 450 instead of a 550 on bad hostnames, this has helped some mails come through from larger ISPs that might have one bad server out of several, often redeliveries will come through from a different box and make it in, so the 450 gives them another chance), and lastly a custom rbl_reply, with some added text to help legitimate users reach me in the case of a problem. Spammers won't ever read these even if they do get the bounce.

And to guarantee that my postmaster account gets all mail sent to it, I've assed that account to my access list, so when the recipient checks run, this will pass those mails right on through without filtering.

So, that's this month's round of changes, we'll see how far this gets me. ;)

Posted by Jim at 1:40 AM | TrackBack

July 11, 2004

Blocking forged HELO in Postfix

I came across this article on Blocking spammers with Postfix HELO controls after finding a log entry for a (rejected) spam that appeared to be getting sent from my own mail server. My own server never sends email (all that routes through my ISP), so this is all instant spam.

In the case of the mail that appeared on my system, a different spam filter caught it before it was even delivered, but I figured having one more hurdle for mail to pass wouldn't hurt, so this seemed a good one to try out.

I followed the steps in the article to set this up here, pay special attention to the 'Making it so' section about half way through. When this file is created or changed, you need to do a 'postmap helo_access' in the terminal for the changes to take place. If you don't, not only won't your changes take place, but postfix will log a warning during each server connection telling you that the source file is newer than it's database.

Posted by Jim at 11:55 PM | TrackBack

July 7, 2004

Anomy's overzealous filtering

I'd completely forgotten that Postfix Enabler also installed a little utility called Anomy for helping to filter email. I'd forgotten until I had to troubleshoot a problem that was effecting some mail my wife was receiving and it brought up a problem I'd forgotten about...

Anomy has the ability to 'defang' parts of HTML code embedded in messages, usually this will keep rogue code from executing on your machine, but in this case was effecting how mail messages were being displayed.

I'd first noticed this when I was forwarding a certain email back to myself, the quoted section had several zeros on there that I couldn't figure where they came from. Eventually I discovered that this was some oddity in the HTML formatting, so I just turned that off in Mail (OS X's mail application) and forgot about it.

Today, my wife received a mail with several numbers at the start from someone who's mailed her before, and she never saw anything like that. After scratching my head then looking at the raw source of the message, things finally started clicking.

I need to dig into this a bit further, but I'm thinking with all the other spam filtering I'm doing, I should be safe to just disable this feature in Anomy, it's a quick fix, just change the following in the anomy.conf file:

feat_html = 1

change to:

feat_html = 0

That should do it! I'll investigate more and see what else I can find out about this problem. I figured it was easier to deactivate this one setting than to try removing Anomy itself.

Posted by Jim at 12:05 AM | TrackBack

July 3, 2004

Greylisting working well

The Gld greylisting utility I'm running with Postfix is working pretty well, and it's already saving me some headaches. I'm still getting other headaches from postmasters that don't quite understand how their mail servers should be configured and some ISPs that just don't care, but that's a different issue. One great thing about greylisting is that it can help BIG if your mail server happens to get hit with a dictionary attack, because of greylisting, all attempts, even to valid email account, will get a temporary bounce, and this is usually enough that the spammer won't bother running through their list again.

I'm going to be looking into cutting back on some of the filtering I'm doing now that greylisting is working, I'll post my updated Postfix config in a week or so with some new notes.

Posted by Jim at 12:29 AM | TrackBack

June 24, 2004

Gld docs coming, also new version

I'll soon have online my docs on getting Gld, a greylisting utility for Postfix, to compile and run under Mac OS X. It took a bit for me to get this working, but I'm learning as I go. ;)

Also, the author informs me that a new release of gld is in the works, and I hope to be running a beta of that shortly. The 1.0 release was pretty raw, the 1.2 version on the way should be a fair step forward.

Posted by Jim at 12:18 AM | TrackBack

June 23, 2004

Gld greylisting working

After some false starts and some tips from the author, I now have gld working to greylist incoming mail. It took a bit to get this all going, I plan to write it all up in a few days and will post the steps involved here.

Posted by Jim at 12:56 AM | TrackBack

June 19, 2004

Postfix 2.1.3 on OS X working

Well, after dorking around for a while tonight, I finally got Postfix 2.1.3 up and running, using the basic steps from AFP548.com, but without the problematic SASL code incorporated. So far, it seems to be working fine, but I've learned a few lessons...

I should start out by saying that I'm not a unix geek, I'm a Mac guy. I know enough about unix to be dangerous, which is to say, I don't really know that much, hence the danger. ;)

The box my mail and web server is running on is a dedicated system, I'm not using it for anything else. When I first set it up and was working on getting things up and running, I had to download a number of various routines and bits of code to make everything I wanted work and compile properly, and of course when I downloaded these, everything wound up on the desktop. Not minding much, I left it all there.

So, tonight I decided to clean up some folders, and organize things a bit. In the process of doing this, I created a folder called 'Mail code' and put the new Postfix and some other items in there, and then proceeded to try to get Postfix to compile. All went well until the last step, the 'make upgrade' command. Well, that one kept erroring with a line telling me I didn't have write access to the postfix folder the new code was in, even though I was running as root. After fussing with it for over half an hour, I finally discovered that the problem was that the Make command didn't like the space in the middle of the folder name, after I then changed it to an underscore, and recompiled everything to have the new path set, it worked great.

Second lesson... All those bits of code on my desktop, I have no idea if after compiling the final code was moved someplace more permanent (/etc/bin?) or if the live code is still living in those folders. I don't want to move anything for fear of breaking something, so for now I'm leaving them where they are. But, in the future I'll be a bit more careful about what goes where.

I'm going to leave things running like they are for a bit before messing with anything else. Once I'm sure that Postfix is working properly, I'm going to try out the Gld greylisting utility for Postfix and see how that works. Since it required Postfix 2.1 or later, I'll finally be able to give it a spin.

Posted by Jim at 10:29 PM | TrackBack

June 15, 2004

No do-not-spam list

I think this is good news... The FTC will NOT be creating their do-not-spam list. This is one battle that needs to be better fought with technology, and not by legistlation.

Posted by Jim at 7:41 PM | TrackBack

June 14, 2004

No SASL for Posfix, yet...

The AFP548.com article I mentioned over the weekend is offlne for now, though that link will still work since it was direct to the article. The author is having some SASL issues that he's still working to resolve. The steps in the article are still good for compiling a newer version of Postfix, if you don't need the SASL support.

Apparently the newer versions of SASL were causing problems with OS X Server, but 2.1.15 'seemed' to work, but still apparently has issues.

Posted by Jim at 8:20 PM | TrackBack

June 11, 2004

Postfix 2.1 on OS X

While surfing around for anyone that got Postfix 2.1 running under OS X tonight, I came across this article at AFP548.com. Not only does it give info on getting this running under Mac OS X 10.3, it also includes info on enabling PCRE, or Perl Compatible Regular Expression, a useful feature if you have more involved mail filtering rules set up. Most basic setups will use regex, more involved rules will require pcre, I've seen this most with some user created Spamassassin filters.

Also described are including support for SASL (Simple Authentication and Security Layer), but the article includes a (bad) link to an older version of this software, and isn't clear why the latest version (available when the article was published) wasn't used. I'm trying to contact the author for clarification. Also, the article seems geared towards OS X Server, but I'm assuming it should work well under standard OS X.

Posted by Jim at 11:22 PM | TrackBack

June 10, 2004

Doh!

Argh! Well, on the good news front, I finally got Gld, a greylisting Postfix add-on, to compile, and it looks like it's pretty much ready to go. The bad news is, greylisting requires Postfix 2.1 or later, and of course Mac OS X 10.3.4 only has Postfix 2.0.10.

Maybe I'll upgrade this weekend...

Posted by Jim at 12:59 AM | TrackBack

June 6, 2004

More Greylisting

I've been giving more thought to greylisting, a process for temporarily delaying emails sent to my server, to help cut down on the spam that ultimately makes it through to my server. Some of my filters are being a bit problematic, and I've had a very small number of legitimate mails that didn't make it through, so I'm needing to revisit things again.

I've found an interesting piece of code called Gld, it uses MySQL so it should be fairly efficient, but I'm having trouble compiling the darned thing for OS X. Unfortunately, it's poorly documented and I'm waiting for the author to respond to my plea for help. But, it sounds promising, if I can make it work here.

Posted by Jim at 10:42 PM | TrackBack

May 29, 2004

Greylisting via Postfix

I've been exploring a new (to me) idea called Greylisting, which is kind of a middle ground between blacklisting (denying) and whitelisting (accepting) mail sent to a server. Basically, what this does is instruct the system trying to send you mail to 'try again later'. Legitimate mails will be retried at whatever interval the other system is set to, but most junk mails won't even try resending, effectively blocking those.

The postfix implementation of this seems a bit spotty currently, but may improve once a few more folks start playing with it. However, at least one person questions its effectiveness based on the amount of spam hitting his Hotmail account, and that fact that those servers are generally hard to reach anyway.

Posted by Jim at 10:07 PM | TrackBack

May 24, 2004

ISP Wall Of Shame posted

The ISP Wall Of Shame is up. These are ISPs that have had mails bounced because of problems with the way that their mail servers have been configured. Most of these are because of problems with how they handle the abuse@xxx.xxx email accounts, some others because of problems with the postmaster@xxx.xxx accounts. More info on these checks can be found at rfc-ignorant, but others are from other types of errors from other spam filters I'm running. Users at any of these systems can try sending me mail (see my contact info at right) and then forward the bounce error you get back to the appropriate support staff at your domain for them to address.

No mail admin wants to block legitimate mail, however the amount of junk mail floating around makes such filtering a necessity to avoid having user's accounts flooded with junk mails. I should point out that only one of the domains listed was actually legitimate mail to my server, the rest are all actually from spams and viruses attempted to be sent here. There were a number of other domains that sent mail that was rejected, but the list I posted were the worst offenders.

There is really no good excuse for not having a properly running mail server, there are a variety of tools to check outgoing mails as well as incoming to keep spam from actually leaving your own server, so these folks are just being enablers for malicious software and users, and it's even more troubling that most of these systems are charging their users for the privledge.

I'll update this list from time to time as I scan through my mail logs.

Posted by Jim at 11:42 PM | TrackBack

ISP wall of shame

Well, thanks to some excellent spam filtering courtesy rfc-ignorant, virtually no junk mail is making it through my mail server. However, it seems that a number of fairly large ISPs have some problems with their mail servers...

I've tried contacting postmasters at several of these with little/no success, so for easy reference I'm creating another side section listing ISPs that have mail server problems, any users trying to email me from those systems will have their emails automatically bounced (sorry). Please contact your support folks with those ISPs to try to help them resolve the problem.

Mail admins that don't run tight ships aren't really doing anyone any favors, and are in my opinion a large part of the junk mail problem we have today.

Posted by Jim at 3:13 PM | TrackBack

May 22, 2004

Wacky mail servers...

I came up with an interesting issue with Google this evening in how it was returning search results, and after some additional Googling to try to find out what was going on, decided it would be best to write their folks and ask for a clarification. So I fire off a note, and almost instantly a mail gets bounced on my server. Sure enough, what was probably a form response from Google got bounced because it came from an improperly set up server.

But, I don't blame Google... It seems that they're dealing with an outfit called Trakken, who according to their web site is a 'world-class eSupport solution'. I'm assuming that this knowledge doesn't extend to mail servers...

Posted by Jim at 11:09 PM | TrackBack

May 19, 2004

My current Postfix config

I must say at this point I seem to be blocking most everything coming my way, whenever something does manage to sneak through, I then go out in search of more tools to try blocking these new junk mails.

I thought it might be good to report my Postfix config since I've updated it quite a bit since last time I listed it here.

In the config below, I'm doing some additional header_checks and mime_header_checks, the files referenced can be found at SecuritySage, just save their sample files to your local system (make sure the names are exactly right and don't have a hidden extension when you save them, I made that mistake the first time and of course they didn't work.

I'm also doing a body_checks, which like it sounds looks for specific text in the message body, my current file only has one line:




/this is the latest version of security update,/ REJECT Confirmed spam. Go away.


That line will catch a nasty email that is supposed to look like it's coming from Microsoft and they're being nice enough to send all of their customers a 'critical update', which I'm sure actually contains a worm that keeps spreading this nonsense. Generally this one gets blocked by some of my other filtering which eliminates mails with .exe, .com, and other PC executables, but some ISP's are actually nice enough to strip the executables off this mail since they know they're a virus, but they then leave the message intact instead of deleting it! So the spam gets through my filter since it'd been monkey'd with, that one line above seemed to be common in each of those mails, so if I see it, that message gets killed.

The folks at Declude have a great site with lots of filters for blocking mails, I've incorporated a number of these, but lately the best ones are the ones at rfc-ignorant.org, these are killing off a number of things that weren't being caught before, including a number of those MS critical update ones.

I'm also running Spamassassin which is installed by the latest build of Postfix Enabler for OS X, and just recently installed Vipul's Razor which works with Spamassassin.

The code below is a clip from my main.cf config file for Postfix, the first block is what Postfix Enabler does on its own, the various reject_rbl_client servers listed there need to be specified individually in Postfix Enabler, just enter them on the line given with commas separating each one. The second block is my 'custom Postfix settings', that whole thing gets pasted into the appropriate field in PostFix Enabler.


###Start PostfixEnabler###
alias_maps=hash:/etc/postfix/aliases
alias_database=hash:/etc/postfix/aliases
smtpd_sender_restrictions=hash:/etc/postfix/access
inet_interfaces=all
mynetworks_style=subnet
message_size_limit=10240000
mydomain=wrightthisway.com
myhostname=wrightthisway.com
smtpd_recipient_restrictions=permit_mynetworks,check_recipient_access hash:/etc/postfix/filtered_domains

smtpd_client_restrictions=hash:/etc/postfix/access,reject_rbl_client sbl-xbl.spamhaus.org,reject_rbl_client list.dsbl.org,reject_rbl_client relays.ordb.org,reject_rbl_client dnsbl.njabl.org,reject_rbl_client bl.spamcop.net,reject_rbl_client dnsbl.ahbl.org,reject_rbl_client opm.blitzed.org,reject_rbl_client dnsbl.sorbs.net,reject_rbl_client relays.visi.com,reject_rbl_client cbl.abuseat.org
default_rbl_reply=$rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason} - see http://$rbl_domain.
smtpd_helo_required=yes

unknown_local_recipient_reject_code=550
###End PostfixEnabler###

###Start Custom Config###
###Keep spammers from discovering real email's and alias expansions###
disable_vrfy_command = yes
default_process_limit = 10
smtpd_error_sleep_time = 30

strict_rfc821_envelopes = yes
smtpd_helo_required = yes
smtpd_helo_restrictions =
permit_mynetworks,
check_helo_access hash:/etc/postfix/access,
reject_unknown_hostname,
reject_invalid_hostname,
reject_non_fqdn_hostname

smtpd_recipient_restrictions =
permit_mynetworks,
reject_unknown_client,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
reject_unauth_destination,
reject_rhsbl_client blackhole.securitysage.com,
reject_rhsbl_sender blackhole.securitysage.com,
reject_rhsbl_client rhsbl.ahbl.org,
reject_rhsbl_sender rhsbl.ahbl.org,
reject_rhsbl_client rhsbl.sorbs.net
reject_rhsbl_sender rhsbl.sorbs.net,
reject_rhsbl_client block.rhs.mailpolice.com,
reject_rhsbl_sender block.rhs.mailpolice.com,
reject_rhsbl_client dynamic.rhs.mailpolice.com,
reject_rhsbl_sender dynamic.rhs.mailpolice.com,
reject_rhsbl_client bogusmx.rfc-ignorant.org,
reject_rhsbl_sender bogusmx.rfc-ignorant.org,
reject_rhsbl_client abuse.rfc-ignorant.org,
reject_rhsbl_sender abuse.rfc-ignorant.org,
reject_rhsbl_client postmaster.rfc-ignorant.org,
reject_rhsbl_sender postmaster.rfc-ignorant.org,
reject_rhsbl_client dsn.rfc-ignorant.org,
reject_rhsbl_sender dsn.rfc-ignorant.org,
check_recipient_access hash:/etc/postfix/access,
check_sender_access hash:/etc/postfix/access,
check_client_access hash:/etc/postfix/access,
check_recipient_access hash:/etc/postfix/filtered_domains

smtpd_data_restrictions =
reject_unauth_pipelining,
permit


header_checks = regexp:/etc/postfix/maps/header_checks
mime_header_checks = regexp:/etc/postfix/maps/mime_header_checks
body_checks = regexp:/etc/postfix/maps/body_checks

unknown_address_reject_code = 550
unknown_client_reject_code = 550
unknown_hostname_reject_code = 550

###End Custom Config###



If you have questions about any of the commands, you can do a simple Google search on that command and usually find out what it's about in short order. Please be aware that the various RBL and RSHBL servers out there do tend to come and go, so it's a good idea to keep a list of what you're using bookmarked in your browser and check their home pages from time to time to see what might be new or changed, or if there's still there at all.

Posted by Jim at 1:02 PM | TrackBack

Vipul's Razor

I've decided to try adding one more spam tool to my arsenal, it's an add in to Spamassassin called Vipul's Razor. I found a nice page on Apple's Developer site that gave some good steps on installing this. Installing the CPAN stuff was probably the most difficult, and that wasn't really hard at all.

Maybe there's a difference in what OS X Server installs from regular OS X, but apparently the Digest::SHA1 code wasn't the right version, so I had to install that separately in CPAN, but after that all was well with installing Razor. My only problem now is that no spam is making it through to my test mailbox so I can't test it! LOL

Posted by Jim at 12:17 PM | TrackBack

May 16, 2004

Great anti-spam helper page

I just came across a page at Declude detailing a large number of current DNS based spam databases, also known as block lists. These include the usual DNSBL lists that block receiving mail from specific servers, and also include RHSBL lists as well.

The RHSBL lists are something I just recently discovered, they are used to filter mail based on the sender's email domain, as some spam may actually pass through legitimate servers, this will catch some additional junk mail that might pass through other filtering. I'm trying out some additional filtering based on RHSBL, I'll report on the results in a few days.

Posted by Jim at 10:48 PM | TrackBack

May 11, 2004

Wow, still

Ok, I've been running the filters posted at SecuritySage doing header_checks and mime_header_checks in Postfix for the last for days, and I've had a grand total of zero junk mails delivered to my dummy account. That's zero delivered, mind you, the mail server is rejecting these outright, so they never hit the mail account at all. In the same amount of time, my mac.com has been hit about 60 times, and that's just the ones that were actually delivered.

No, not quite a fair test since my mac.com account is in a bit wider circulation, including to some (unfortunately) PC email users, and we all know how often viruses and worms come along and steal entire mail lists from those folks... Still, if you're running your own mail server with Postfix, do yourself a favor and check out those filters. ;)

Posted by Jim at 12:01 AM | TrackBack

May 8, 2004

Much better...

All I have to say so far is, wow. The spam filters I borrowed from SecuritySage have really done the trick on the mails I was getting, most of which were fake Microsoft 'patches' and other garbage. Now they're all getting rejected before they even get to my mailbox, or rather, the mailbox of the spam catcher account I set up.

Sweet.

Posted by Jim at 9:04 PM | TrackBack

May 7, 2004

More spam blocking

I just came across the SecuritySage web site, it has some good info on fighting spam, specifically some header and mime checks to actually reject mail from hitting your server, rather than waiting for Spamassassin to try catching it.

I'm trying out some additional filtering to see if this cuts down on the mails going to the account I set up for spam. If anyone's curious about that, I created a new account, then set up my newsreader with that account, and posted a number of emails in various newsgroups wtih a clear subject and body that this message was being sent so spammers would get the email address. Within an hour, I was already getting spams, so obviously that trick worked. If you're setting up a new spam filter, you might want to do this to get a dummy address out in the wild for testing, just don't post with it too much or you'll be guilty of spamming too...

Posted by Jim at 11:44 PM | TrackBack

May 4, 2004

Postfix and Spamassassin

I've written previously about using Postfix Enabler to set up the mail server I'm using for the site. The author, Bernard Teo, was nice enough to hook me up with a beta of version 1.1 that I've been running here for a bit over a week, and it is now available via the link above for anyone that wants to give it a shot. It now includes an optional setup for Spamassassin, as well as a handy Mail Stats generator to keep track of what your server is doing. Also included is a new field to set a RBL (Realtime block List) server, such as Spamhaus to help with spam checking.

I thought I had the Spamassassin part set up correctly, but after setting up a new account here for testing and then going out of my way to put that account where spammers would find it, I found that Spamassasin wasn't checking my mail at all. Read on for how I fixed this, and learned a bit about Postfix's main.cf configuration file.

The main think I discovered with Postfix and its config file (main.cf), was that if you give the same config line twice with two different sets of parameters, the second instance will replace the first. I suppose my work with CSS style sheets had me confused since one line could build on top of what came before, but this isn't the case with Postfix.

What was happening was that the Postfix Enabler was setting up the necessary commands for Spamassassin to filter the mail using the smtpd_recipient_restrictions control, but it turned out that I was using this for some additional filtering of my own, and Postfix Enabler was putting my custom config after its own settings, thereby overriding the settings for Spamassassin.

Once I finally realized what was going on, it was a simple matter to copy the relevant line from the config and put it into my custom settings and restart Postfix. A quick test mail then confirmed that Spamassassin was alive and well, filtering my mail.

For anyone curious, here are the custom commands that Postfix Enabler is setting, and my own custom settings below that. This sets some fairly strict filtering, so be warned.

###Start PostfixEnabler###
alias_maps=hash:/etc/postfix/aliases
alias_database=hash:/etc/postfix/aliases
inet_interfaces=all
mynetworks_style=subnet
message_size_limit=10240000
mydomain=wrightthisway.com
myhostname=wrightthisway.com
smtpd_recipient_restrictions=permit_mynetworks,check_recipient_access hash:/etc/postfix/filtered_domains

smtpd_client_restrictions=hash:/etc/postfix/access,reject_rbl_client sbl-xbl.spamhaus.org
default_rbl_reply=$rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason} - see http://$rbl_domain.
smtpd_helo_required=yes

unknown_local_recipient_reject_code=550
###End PostfixEnabler###

###Start Custom Config###
strict_rfc821_envelopes = yes
smtpd_helo_required = yes
smtpd_helo_restrictions =
permit_mynetworks,
check_helo_access hash:/etc/postfix/access, reject_unknown_hostname,
reject_invalid_hostname,
reject_non_fqdn_hostname

smtpd_recipient_restrictions =
permit_mynetworks,
reject_unknown_client,
reject_non_fqdn_sender,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
reject_unauth_destination,
check_recipient_access hash:/etc/postfix/access,
check_sender_access hash:/etc/postfix/access,
check_client_access hash:/etc/postfix/access,
check_recipient_access hash:/etc/postfix/filtered_domains

smtpd_data_restrictions =
reject_unauth_pipelining,
permit

unknown_address_reject_code = 550
unknown_client_reject_code = 550
unknown_hostname_reject_code = 550

Posted by Jim at 9:37 PM | TrackBack

April 27, 2004

450? Do I hear 550?

After going through my documentation on Postfix again, and trying to figure out why those problem mails keep resending to my system, I discovered that the error code I had set, 450, was basically telling the other end that there was a problem, and to try again. I've now changed this to 550, which should now reject those mails.

This seems to be good in that those sending me mail should not receive the error mails that indicate a problem mailing me, letting them know about the problem. I've begun sending postmasters of mail servers with issues notes containing clips from my logs in the hope that this might help them fix their systems. We'll see what happens.

This last error I saw from actually from what should have been a Mac system, so hopefully they'll get their act together quickly. ;)

Posted by Jim at 10:58 PM | TrackBack

April 26, 2004

Reverse DNS, or lack thereof

In setting up my mail server, I made the decision to use some fairly strict settings in order to reduce spam. I was aware that there might be some legitimate mails that may not make it through from improperly configured mail servers, and I figured the chances of this would be fairly slim. Last night it looks like I bounced my first legitimate mail.

It doesn't seem to be a problem with this party's mail server, but rather something with their DNS, it isn't providing a reverse lookup for their IP address, so when my mail server sees a message coming from their IP address, it tries to determine the name of their mail server (ex. mail.yourcompany.com), and when no results are returned, the message is rejected.

I know of some companies that have set up restrictions on their mail systems to only accept mail from known hosts (those with reverse lookups) to help block junk mail, who quickly backed away from this, the article linked above specifically mentions that AT&T Worldnet ran this setup for only 24 hours in January 2003 before they had to turn it off because of legitimate mail being bounced.

So, I'm now left with a dilema, so do I disable this setting so this company's mail can finally reach me (I'm getting a bounce each hour right now as they attempt to resend the mail), giving in to the terrorists, er, technological nonconformists, or do I stand my ground against them like Spain, er, AT&T Worldnet didn't? For now, I'm standing my ground.

If someone can give me some legitimate reason why a full time mail server NEEDS to be misconfigured or have improper network settings, I'll be happy to listen, and possibly revise my opinion. At this time, I haven't seen any legitimate cases for this, all I've found so far is various companies and mail admins who have tried to enforce standards and given in because of others who continue to run out of date or sloppily written software. Granted, those that gave in seem to have done so because of complaints from paying customers (I suppose I'd be mad at my mail provider if they were blocking legitimate mail) or from employees who were unable to conduct business, but it seems that a line in the sand must be drawn, so for now, consider the line drawn.

Posted by Jim at 9:00 AM | TrackBack

April 23, 2004

Spam fighting, the Mac way

I've blogged previously setting up Postfix on my server, and wanted to share this link about Spamhaus, and how they're making use of Macs, including a G4 cube, to help ISPs around the world, as well as small time mail servers like mine, keep up the war against unwanted junk mail.

My mail logs show that I'm regularly hit from systems looking for an open mail relay for sending junk mail, Postfix by default blocks these. But I'm getting an increasing number of messages that are being rejected thanks to Spamhaus. So far, not one junk mail has made it to my inbox from a spammer.

Posted by Jim at 11:04 PM | TrackBack

April 11, 2004

Setting up my mail server

Mac OS X 10.3 includes the Postfix , a low level email server utilty for sending and receiving mail. Prior version of OS X relied on Sendmail for this task, but Postfix is superior in a number of ways, mainly performance and security.

Since I was setting up a site under my own domain name, I though that it'd be nice to set up my own mail server, fully under my own control. To help in this, I'm using a great utility called Postfix Enabler .

Postfix Enabler lets you activate Postfix under OS X, and configure it for both sending and receiving mail. Many folks have set their systems up to simply allow them to sent mail and bypass the outgoing mail servers on networks they're connected to. This is primarily useful for mobile users who connect to the internet at a variety of locations, and may not always be able to determine the proper mail settings needed. Some ISPs or corporate networks may only allow outgoing mail traffic to pass through their own servers, so your mileage may vary.

I've elected to set this up for incoming mail only, and use my ISP's own outgoing mail server, there is no real advantage in setting up my own for that. But, since I'm running my own mail server, my primary concern will be spam. No, not that tasty meat from Hormel , I'm talking about junk mail. Over on the right hand side I've got a few links to some sites that were very helpful in getting Postfix set up to filter junk mail, and the excellent spam blocker Spamhaus, all of which help to keep unwanted mail from even appearing in my In Box to begin with.

Since I'm not receiving any mail right now, it's a bit hard to check how well this is working, but I'll report back over the next few weeks as mail starts to roll in and detail my findings.

One word of caution, the settings I've selected to perform some fairly strict checking of the mail protocols used between servers to communicate. Spammers and virus writers are notorious for not following standards, topping even writers of Windoze applications, so the checks I have in place should filter out a good bit of unwanted mail before the actual checks for spam even kick in. There's a good chance that some 'legitimate' mail might get bounced from misconfigured mail servers or Windoze users, but I think I can live with that.

Posted by Jim at 7:44 PM | TrackBack