February 13, 2008
getnameinfo fixed with 10.5.2
I have been able to confirm that the getnameinfo issue with Mac OS X 10.5 has been fixed with the 10.5.2 update. I never did confirm if this was an issue with 10.5.2 Server or not, but the fix is in the server version also.
December 19, 2007
Postfix and iPhones
My wife received an iPhone for Christmas this year, and she's been having fun playing with her new toy. She was very excited to find out that she could check her mail from the phone, and as I had never bothered to set up mail on mine, I took a few minutes to set her up. But, this simple exercise took a bit longer than I expected, since my mail server here wasn't set up for this just yet.
I have both POP and IMAP configured, but haven't tested IMAP extensively, and have only used it via the webmail apps I run, and the iPhone was trying to talk on ports that I hadn't set up yet.
The first order of business was to allow mail to be received on port 587, this is done by uncommenting the line starting with #submission in the postfix master.cf file. The submission port is a standard designation for port 587. If you would like to have postfix be able to receive mail on other ports, simply copy/paste that line, and change the word 'submission' to the desired port number, restart postfix, and postfix will now be listening on this new port. This may be handy for getting around certain networks that block the standard mail ports and want you to use their mail gateway. Having an alternate port you can send on may be handy to have in those rare occasions.
The next trick was editing the account setup on the iPhone. After entering all of the account info, the initial connection failed. I began tweaking settings on the mail server, but couldn't easily get back to change my wife's account setup on the phone. I found that by going to the phone's Settings, then Mail, I could select the account, and at the bottom of this screen was an Advanced option, that let me tweak all of the settings.
Despite declining to use SSL encryption, this option was still selected for both incoming and outgoing, very simple to turn off. This screen also lets the default ports be overridden, again, useful to get around certain network blocks.
Once I had tweaked the necessary settings on the iPhone, I had her mail running just fine. Thank goodness for unlimited data plans...
December 8, 2007
Courier-IMAP 4.2.1 compiled successfully
Well, I resolved my issue with the 4.2.1 release... I had originally mentioned that 4.2.1 just couldn't be made to compile here, and I therefore reverted to the older 4.1.3 release without issues. Well, it turns out that I apparently had some bad data saved from a prior compile attempt, and it was as simple as running a 'make clean' before doing another make that cleared everything out and let me get a good compile.
A 'make install' later, I was up and running with Courier-IMAP 4.2.1. It's the little things that get you, sometimes...
December 4, 2007
getnameinfo issue with Mac OS X 10.5
I found a rather obscure bug with Leopard while troubleshooting my postfix logs. It seems that mail that I had been getting from some of the mail lists I usually receive was being bounced by my server because it couldn't resolve the IP address based on the DNS supplied by the sending server. If you've read my prior Postfix postings, you know that I'm fairly strict about the servers I accept mail from, and misconfigured servers generally don't get any mail delivered here.
So, this came as somewhat of a surprise that formerly working servers were now being rejected after my upgrade to Leopard. Some troubleshoot assistance from the postfix mail list uncovered the issue, the getnameinfo function in the OS was not resolving DNS addresses that resolved to a CNAME record, or anything other than a PTR record. The unix nslookup and host commands though, worked fine, but postfix relies on the getnameinfo function.
The good news here is that this bug has been reported to Apple, and signs are good that this should be fixed in Mac OS X 10.5.2 when it is released. I'll report back on that after release. For now, my temporary workaround is to identify servers that aren't resolving, and whitelist them in my helo_access list.
November 22, 2007
newsyslog Revisited
Earlier this month, I wrote about a new utility that handles log rotation in Leopard, and gave a tip on fixing logging for the mail.log. It turns out that my fix wasn't quite right...
The original line in the configuration file was as follows:
/var/log/mail.log 640 5 100 * J
This results in the log file being rotated when it reached 100Kb in size. What I wanted was for the logs to roll weekly as they had with prior systems, and my assumption was that this would continue to happen as part of the periodic.weekly script. Bad assumption.
I don't usually have to scroll too far back in my logs when researching things, but tonight discovered that I had entries going back more than a week, and that the log file wasn't rotating as I thought it should have been. A quick check of the periodic.weekly script revealed that log rotation wasn't there anymore, so I revisited the newsyslog.conf file, and made the following change:
/var/log/mail.log 640 5 * $W0D0 J
The asterisk there is in the size column, meaning don't worry about the log size, the $W0D0 is under the when column, this means to rotate weekly, on day 0 (Sunday), at hour 0 (12am).
The man page for newsyslog.conf gives a wealth of info on configuring this utility, and is well worth a read.
November 4, 2007
New log rotation utility in Leopard
Most folks have no need to ever check their system logs. Some folks check their logs religiously. Mac OS X 10.5 has thrown a new tool into the mix, and it might bite you if you don't know about it.
There is a new command line command called newsyslog, it is called every minute by the file /System/Library/LaunchDaemons/com.apple.newsyslog.plist, and it's config file lives in /etc/newsyslog.conf.
Tonight, I needed to check my mail server logs for some information, and had to search prior logs. At first, my searches made no sense, as I kept coming up with today's date in the data, but my mail logs normally contain a week's worth of data. Well, not anymore, thanks to newsyslog, now they only contain 100Kb worth of data before they roll over to a new log. Ack! A simple fix, commenting out the log cycling for the mail.log file. Hopefully this tip might help out anyone else out there that gets bitten by this.
November 3, 2007
Details on upgrades
To recap my recent upgrades here, I was transitioning my old web/mail server from a G4 box running OS 10.4 to a new Mac Mini running OS X 10.5. Funny that I wrote about using a Mini as a server back in 2005, and I'm only now finally getting around to putting one in here...
So, the basic process here was shutting down Postfix, then using Carbon Copy Cloner to clone my existing server to the Mac Mini (booted in Target Disk Mode), then rebooting the Mini into the Mac OS X 10.5 Installer. The Installer had absolutely no problems upgrading a PPC version of OS 10.5 to an Intel OS running 10.5, which was great. I really did not want to do a clean install, which would have been more of a hassle in converting mail files and other lower level items.
The next necessary step after installing 10.5 was to install Xcode 3.0, in order to compile all the apps I needed. Once that was done, I was finally able to start getting things up and running.
From prior dry runs, I had done a lot of testing of various packages to make sure that things would compile properly, and run without errors. There was a good bit of trial and error, and lots of googling. And thanks to someone else googling and finding an earlier entry of mine, a helpful tip out of the blue (Thanks to Paul S.) that helped massively. I had partitioned my drive so that I had a nice workspace partition to hold files between attempts at cloning and upgrading, and I had saved a few helpful notes there as well, which was very handy.
As I had mentioned a few days ago, the unix system accounts for postfix, mysql, www, and others, now for some reason all begin with an underscore character, so I had to edit a few config files where these accounts were specifically used to make sure that they reflected the current users. Also, 10.5 now runs Apache 2.2.x and not Apache 1.x, so I had to do some reading up on how this gets configured in order to migrate my config files, there were few surprises there, once I paid attention to the sample config files. Having saved copies of my working config files from earlier runs, it was a simple matter to copy these over before starting other work.
In retrospect, I should have worked on getting the mail server up and running before the web server, I didn't lose any mail, but I just hated it being offline as long as it was...
Starting with the web side, I installed MySQL 5.0.45 using a pre-build package, I used the one built for 10.4 Intel, and plan to upgrade that to a 10.5 specific build once one is available. After installing this, I simply copied my data folder over, ran the mysql_upgrade script, and all was well. Next, I compiled DBI-1.601 and DBD-mysql-4.005. For some reason I wasn't able to track down, DBD insisted on looking for mysql/lib files in /mysql/lib/mysql, even though my install never mentioned this path anywhere. Some googling finally revealed that the easiest fix was simply to fake it with some symlink trickery:
cd /usr/local/mysql/lib sudo mkdir mysql cd mysql sudo ln -s ../*
DBI compiled fine, DBD threw up an error about incompatible pointers, which I was stuck at for a day or two before finding out that this was just a warning and could be ignored. Sure enough, it ran just fine, and I found that MovableType was now working fine. During the final install of everything, I discovered that I had to reset access privs for my web folder in order for MT to be able to write files, but after doing that, it worked fine again. I'm saving my upgrade to MovableType 4.x for another day.
Compiling Postfix was fairly straightforward, as before, I built Postfix according to the standard install docs to include MySQL and PCRE support, but this time included SASL in the mix. It is very important to read the SASL docs, there was a bit of needing to create symlinks and make sure that header files were in the right locations, but once I followed all the steps outlined, it compiled fine.
The Courier-IMAP pieces drove me nuts for several days. Courier-IMAP 4.2.1, the latest build, just couldn't be made to work here, I eventually tried building an older version, 4.1.3, and that worked just fine. Courier-Authlib 0.60.2 compiled but had problems running, the trick mailed in my Paul S. was to enter the following before doing the compile:
export MACOSX_DEPLOYMENT_TARGET=10.5
This handy command has been around for a few OS releases now, and forces some settings that apparently don't get set otherwise, a quick google search found many packages needing this to compile properly. Once set, AuthLib compiled properly and more importantly, ran properly.
Despite doing the 'migrate' steps, though, my old Courier settings never made it over, and so I had to edit the authmysqlrc and some other Courier files by hand using my older versions as templates, but this work was done in short order.
One site that helped a lot in checking over some of my steps was this one:
http://switch.richard5.net/isp-in-a-box-v2/building-the-mail-server-components/building-the-courier-imap-server/
The versions used there weren't current, but helped to validate what I was trying to do here, and setting the proper CFLAGS and compile arguments. His setup there was very similar to mine, virtual domains, MySQL authentication, etc, which was a great help.
With this done, I was now actually able to check mail the last necessary step, which made a good stopping point for the night with a fairly functioning server.
The next day, I tested a few more functions of the system, and found that one of the web packages I had installed was having problems with MySQL. This turned out to be a PHP issue connecting to MySQL, it was looking for the mysql.sock file in /var instead of /tmp. The easiest fix here was to create a /etc/php.ini file, consisting of the following:
; Default socket name for local MySQL connects. If empty, uses the built-in MySQL defaults. mysql.default_socket = /tmp/mysql.sock ; Default socket name for local MySQL connects. If empty, uses the built-in MySQL defaults. mysqli.default_socket = /tmp/mysql.sock
The second section for mysqli was required for version of MySQL 4.1 and later, once this was in place and Apache stopped and started, this problem was now history.
The last hurdle I had was getting policyd running, this is the greylisting package I use with Postfix. I had been struggling for some time to get newer builds of this running. I had somehow hacked the 1.7.x version into running previously, and was never able to duplicate my success with later builds. Thanks to some outstanding work by the developers, the final fixes to this are now available in the latest SVN builds, and I was able to get the 1.9.x experimental build to compile successfully, and more importantly, to run successfully as well.
In closing, what I'd like to say here is that when you're rolling your own code, patience is your best friend. Take things one step at a time, make sure you have a backup, and when you hit a wall, do searches and ask questions on mailing lists until you find the answers you need. If all else fails, post about your failures, and someone else might stumble across your post and supply the answers you need, it's amazing how things like that work out sometimes.
November 2, 2007
Server upgraded successfully
The server here has been successfully upgraded to Mac OS X 10.5. A few tips rolled in earlier this week that resolved the last of my compile issues (details to follow soon), so last night I cloned everything over to the new box and started the upgrade process.
One important tip, installing Xcode is kinda important. It's the little things you forget to do... :)
October 28, 2007
Upgrades... Hmmm.
A relatively sleepless weekend, and not in a good way. Here's an interesting tidbit, the common unix accounts such as postfix, mysql, www, and others, are now prefaced by an underscore character in OS X 10.5. Why, I have no idea, but when configuring scripts, make sure to change the usernames.
The switch to Apache2 for the most part went pretty well, it took a bit of trial and error to get my virtual domains working, but once I went back and poured over the sample configs, it all started to make sense. Just copying and pasting relevant bits from my old config files was not the way to go. :)
Minor issues compiling the DBD::mysql module, I had to use a slightly older version and it worked fine.
Courier-IMAP is what drove me absolutely nuts, I finally thought I had it all going, and then discovered that the auth module was throwing errors in the log, and I'm still trying to track that down. Also, despite attempting to migrate my older Courier settings, this apparently didn't happen, best thing may be to build it all up from scratch.
October 26, 2007
Leopard Day...
Mac OS X 10.5 (Leopard) is available in stores, and of course I've been playing with it for a bit. The new tabbed Terminal is great, having several terminal sessions all open in separate tabs instead of multiple windows is great, it really helps keep things organized.
Anyway, my first pass at upgrading my mail/web server from PPC 10.5 to Intel 10.5 went pretty well, everything actually seemed to launch and run correctly, web services worked, postfix was running, etc. Of course, I want to compile Intel binaries and not use the PPC codes, so I've been testing some installs. So far, most of them are going well, a few minor snags though, but I hope to have things working on the new hardware by the time the weekend is done.
October 18, 2007
Upgrade prep
I had forgotten what a pain in the ass a major upgrade can be... Made even worse by switching platforms, going from a PowerPC based Mac to an Intel based one.
I did a number of 'test compiles' on the Mac Mini just to make sure that things looked like they'd be working come Leopard day, and figured out that in order to clone my old server over to the new box, I'd have to format the drive with a GUID partition map, something not done by default when formatting from a G4 system... It's the little things that get ya.
So, quick checklist of things to do:
Format Mini's drive as GUID
Download latest MySQL 5, MovableType, Courier-IMAP, Courier AuthLib, Postfix, PCRE, PHP, Cyrus SASL, policyd
Shut down all services
Clone drive from G4 to Mac Mini
Boot 10.5 Installer
Upgrade system
Install MySQL, PHP (built for 10.4.x, will upgrade to 10.5 versions when available)
Build/Install PCRE, Cyrus SASL, Courier AuthLib, Courier IMAP, then Postfix (order probably important)
Build/Install policyd
Keep fingers crossed
Hopefully I haven't forgotten anything that's a dependency, if so, I'm sure I'll find out about it.
I'll probably try for a dry run this weekend, and see what happens.
September 18, 2007
Major Upgrades
I'm in the beginning stages of some major upgrades to the machine running the web/mail system here, every major piece of software on the back end is being upgraded, and the hardware is seeing some major changes as well Gone will be the old G4 system, in favor of a new Intel Mac, and a whole host of software upgrades for the new hardware to bring everything current with the latest releases (MySQL 5, MovableType 4, Courier-IMAP 4.1.3, Postfix 2.4.5, pcre 7.3, PHP 5.x, etc, etc). Oh, and of course, Mac OS X 10.5, when it ships, hopefully next month.
The plan at this point is to prep the new hardware, install all the software, then migrate the data from the old server, followed by much testing to make sure that everything is working as it should be. If all goes well, shortly after Mac OS X 10.5 ships, I'll be able to wipe the drive and install all the latest bits and be able to go live in early November. There should be no noticeable difference (unless I play with MovableType a bit!), but my UPS will have a lighter power load to deal with, at least. :)
October 9, 2006
Updated Whitelist
It has been a while since I last posted my whitelist that I use with greylisting, and I've since converted to Policyd. So, here's my current whitelist:
+----------------+--------------------------------------------------------------+ | _whitelist | _description | +----------------+--------------------------------------------------------------+ | 127.%.%.% | # localhost | | 192.168.%.% | # private netblock | | 10.%.%.% | # private netblock | | 12.5.136.% | # Southwest Airlines (unique sender, no retry) | | 12.107.209.244 | # kernel.org mailing lists (high traffic, unique sender per | | 12.107.209.250 | # sourceware.org mailing lists (high traffic, unique sender | | 63.82.37.110 | # SLmail | | 64.7.153.18 | # sentex.ca (common pool) | | 64.12.137.% | # AOL (common pool) - http://postmaster.aol.com/servers/imo. | | 64.12.138.% | # AOL (common pool) | | 64.124.204.39 | # moveon.org (unique sender per attempt) | | 64.125.132.254 | # collab.net (unique sender per attempt) | | 64.233.170.% | # gmail (common server pool) | | 65.82.241.160 | # Groupwise? | | 66.100.210.82 | # Groupwise? | | 66.135.209.% | # Ebay (for time critical alerts) | | 66.135.197.% | # Ebay (common pool) | | 66.162.216.166 | # Groupwise? | | 66.206.22.% | # PLEXOR | | 66.218.66.% | # Yahoo Groups servers (common pool, no retry) | | 66.218.67.% | # Yahoo Groups servers (common pool, no retry) | | 66.218.69.% | # Yahoo Groups servers (common pool, no retry) | | 66.27.51.218 | # ljbtc.com (Groupwise) | | 66.89.73.101 | # Groupwise? | | 68.15.115.88 | # Groupwise? | | 194.245.101.88 | # Joker.com (email forwarding server) | | 195.235.39.19 | # Tid InfoMail Exchanger v2.20 | | 195.238.2.% | # skynet.be (wierd retry pattern) | | 195.238.3.% | # skynet.be (common pool) | | 204.60.8.162 | # Groupwise? | | 204.107.120.10 | # Ameritrade (no retry) | | 205.188.139.% | # AOL (common pool) | | 205.188.144.% | # AOL (common pool) | | 205.188.156.% | # AOL (common pool) | | 205.188.157.% | # AOL (common pool) | | 205.188.159.7 | # AOL (common pool) | | 205.206.231.% | # SecurityFocus.com (unique sender per attempt) | | 205.211.164.50 | # sentex.ca (common pool) | | 207.115.63.% | # Prodigy (broken software that retries continually with no | | 207.171.168.% | # Amazon.com (common pool) | | 207.171.180.% | # Amazon.com (common pool) | | 207.171.187.% | # Amazon.com (common pool) | | 207.171.188.% | # Amazon.com (common pool) | | 207.171.190.% | # Amazon.com (common pool) | | 213.136.52.31 | # Mysql.com (unique sender) | | 216.136.226.0 | # Yahoo Mail? | | 216.157.204.5 | # Groupwise? | | 217.158.50.178 | # AXKit mailing list (unique sender per attempt) | | 209.237.227.% | # SpamAssassin mailing list | | 66.35.250.% | # lists.sourceforge.net | | 196.25.240.% | # saix.net | | 196.4.160.% | # internet solutions (business smtp) | | 196.35.77.% | # internet solutions (dialup smtp) | | 196.25.69.% | # telkom | | 196.2.50.% | # mweb (dialup smtp) | | 196.2.49.% | # mweb (business smtp) | | 196.2.24.% | # mweb (business smtp) | | 205.174.22.% | # Delta | | 64.12.136.% | # AOL (common pool) | | 209.104.48.% | # Ticketmaster | | 209.104.56.% | # Ticketmaster | | 216.154.234.% | # Tickets.com | | 216.155.201.% | # Yahoo | | 216.155.203.% | # Yahoo | | 63.240.36.% | # Priceline.com | | 66.163.187.% | # Yahoo | | 66.94.237.% | # Yahoo | | 207.171.160.% | # Amazon.com (common pool) | | 64.125.133.202 | # collab.net (unique sender per attempt) | | 63.169.44.140 | # Southwest Airlines (unique sender, no retry) | | 144.9.158.% | # American Airlines | | 151.193.203.% | # US Airways | | 165.212.65.% | # United Airlines | | 207.235.15.% | # Continental Airlines | | 66.142.137.% | # Continental Airlines | | 139.72.190.% | # Northwest Airlines | | 68.142.200 | # Yahoo | | 68.142.199 | # Yahoo | | 209.191.126 | # Yahoo | | 64.4.240.% | Paypal | | 216.113.188.% | Paypal | | 209.104.46.% | # Ticketmaster | | 209.104.33.% | # Ticketmaster | | 206.165.246.% | # Hotwire.com | | 64.95.142.% | # Hotwire.com | | 66.211.168.% | Paypal | +----------------+--------------------------------------------------------------+ 86 rows in set (0.00 sec)
September 10, 2006
Another Webmail client
After playing around with SquirrelMail for a while, I ended up not being terribly happy with it, mainly because it was all text based, no fancy icons for functions, etc. After a bit of web searching I came across the RoundCube Webmail Project, which I liked a lot, as it looked somewhat similar to Mail in OS X.
Roundcube was very easy to get installed on my OS X based mail/web server, and even though the software is only beta2 currently, seems to be running just fine. There seems to be a pretty healthy user base already running this, so I have high hopes that it will continue to be developed to a 1.0 release and beyond.
August 11, 2006
SquirrelMail
This afternoon, I finally broke down and installed a webmail application here to read mail when away from our usual machines, after scouring the web for info, I finally settled on SquirrelMail, a well rounded package for doing email via the web.
Installation was a snap, the online info on installing under Mac OS X 10.1 still worked fine even though the mail server here is 10.4, and aren't really any different than a standard install, so I'm kind of missing the point of having separate install instructions... I'll put this through its paces over the next few weeks and will report back if any signifigant issues are uncovered.
June 19, 2006
Policyd for Postfix
I've decided to switch the greylisting software in use here from Gld to Policy Daemon. Policyd has a lot more flexibility with greylisting, including some automated blacklisting and whitelisting functionality, and is overall a lot more robust.
I may be the first user of this running it on OS X, the author hadn't heard of any other Mac users using it, but with some very minor tweaks to the MAKE file it was up and running with no problems, these changes should be part of releases after 1.77. As I've done with most other utils like this I run on the server here, I created a StartupItem for it, and something new for me, created a LaunchDaemon to handle the cleanup instead of a cron job, as cron is depreciated with OS X 10.4 Both were easy enough to do, plenty of docs available elsewhere on setting these up.
February 18, 2006
Upgrade to 10.4.5
The server here has been upgraded from 10.4 to 10.4.5. I was a little apprehensive about this update as I didn't have a chance to test prior to performing it, but everything went well, Postfix kept chugging, web services uninterrupted, other miscellaneous non-standard compiled code went well. Looks like Apple didn't stomp on anything that I'd upgraded. ;)
The big hangup was an issue with Carbon Copy Cloner, under 10.4, it wasn't able to clone my drive due to an authentication issue, normally I'd clone my drive and test out the upgrade first, so I just had to chance it. The main reason I needed to upgrade was 10.4 apparently had some issue that would keep log files from rotating properly, and possibly other scheduled tasks from running, so it was high time to get that issue fixed.
January 8, 2006
Postfix Pop-Before-SMTP
While on vacation recently, I found myself unable to send email while at a wireless hotspot, I couldn't connect to my usual ISP's outbound mail server, and the hotspot wasn't providing this access either. I never actually send mail through my own domain, and with my access restrictions, was unable to do so from a foreign ISP.
After some digging, I installed Pop-Before-SMTP, a handy little utility that integrated with Postfix to create an access list that can be checked to allow outgoing mails to be sent. It works by creating a list of IPs that have successfully logged in to check mail (via POP, or IMAP), then allows those specific IPs to send outgoing mail, and will automatically expire those IPs in a configurable number of minutes. Also, to avoid issues with mail rejections, I configured my Postfix to relay all outgoing mails through my home ISP rather than delivering directly. It worked great the few times I needed to use it.
September 17, 2005
Postfix & Tiger upgrade
I've finally made the upgrade to Tiger on my mail server. That is, I've finally 'successfully' upgraded, it took a few attempts to get everything just right for what I wanted, and now that I'm done I thought I'd share a few details.
Probably the most important step is to make sure that you have a good backup before starting. For me, this step was critical, as it allowed me to back out at any time and get running again from the backup. I wasn't just upgrading from 10.3 to 10.4, but was also upgrading Postfix, MySQL, and PCRE all at the same time...
Tiger (10.4) includes version 2.1.5 of Postfix, for most folks this is probably more than sufficient. However, I wanted to upgrade to the Postfix 2.2.5 release, as it includes a few more bells & whistles that I wanted, I wanted to bring MySQL current, and also bring PCRE (Perl Compatible Regular Expressions) up to date. Basically, this meant upgrading my system to Tiger, upgrading/compiling each new piece, and finally getting it all up and running.
MySQL was a snap, thanks to a ready made Installer built for Tiger. I was upgrading from an earlier 4.1.x release, so there were no worries about my databases not working correctly, and it was a simple matter of copying over the data directory from the old path to the new and getting MySQL running.
PCRE compiled well, but prior to compiling I also upgraded to the latest version of Xcode to get all the latest libraries installed. Standard docs on how to install worked just fine.
The last and trickiest step was getting Postfix compiled properly. After trying a few times and having problems, I finally took a step back and found my error, a simple typo caused from a copy/paste error when trying to get both the MySQL and PCRE code compiled in. For reference, here's the correct MAKE instruction for that:
make -f Makefile.init makefiles \
'CCARGS=-DHAS_MYSQL -I/usr/local/mysql/include -DHAS_PCRE -I/usr/local/include' \
'AUXLIBS=-L/usr/local/mysql/lib -lmysqlclient -lz -lm -L/usr/local/lib -lpcre'
After that, it was just a matter of running Make, and then Make Upgrade.
The last and trickiest step was to get Postfix running at Startup again. The Tiger upgrade removed my Postfix StartupItem (included with the Postfix source code), so that just needed copied back to /Library/StartupItems, and I also removed the /System/Library/LaunchDaemons/org.postfix.master.plist file that Tiger includes for starting Postfix on demand, as I wanted this running all the time, and it also didn't seem to be launching my newly compiled version of Postfix properly. I also needed to edit the /etc/hostconfig file to change MAILSERVER=-NO- to -YES-, and after that Postfix launched perfectly.
In hindsight, I probably should have upgraded 10.4 to 10.4.2 before upgrading everything else, but no mail related updates seem to be part of the upgrade, so this shouldn't be a problem. But if you're planning on going to 10.4 and recompiling other software, get your OS fully current first, then start in on your other work, it'll save you the trouble later.
I used Mike Bombich's excellent Carbon Copy Cloner to clone my working system to a backup drive, upgrade that and was able to do all my testing on a backup drive prior to the actual upgrade, and at any time I could simply reboot from my primary drive and be up and running. Very useful when you can't have a production system down long but need to do some 'live' testing.
May 15, 2005
Postfix and road warriors
Ok, so you've set up your own mail server on your PowerBook for when you're on the road, and don't know what ISP you'll be connecting to next. Now you can send mail from anywhere, as long as you've got an internet connection, and it'll go through just fine, right? Wrong. An incresing number of mail servers worldwide are restricting where they accept mail from, dynamic or dial up IP ranges are blocked in many cases, mail from IP addresses without a resolvable DNS name, or even ANY DNS name, etc. And some mail is just dropped, you'll never know it bounced. So, what to do, you ask?
The answer is to relay your mail through another mail server. Generally, this is something that most mail servers should be set up to avoid doing (as Postfix does), but in this case, what we want to do is get mail from our mobile mail server to a fixed mail server elsewhere. If your regular ISP has a mail server set up to allow SASL (Simple Authentication and Security Layer) access, Postfix can be configured to support this, and once authenticated, mail can then be relayed to your ISP's mail server, which should then go through fine (assuming your ISP has a properly configured mail server).
A second option would be if you've already set up Postfix as a mail server at home (as I have), and your ISP doesn't provide SASL access, would be to pass mail to your home system, which can then send the mail out. A bit convoluted, but it will get the job done, and you won't need to worry about mails not going through.
Not quite the simple plug and play solution Mac folks are used to, but mail servers are not known for being simple, nor should they be. Anyone attempting to set up their own server should understand the technology involved, and be prepared to support themselves, and probably consider joining one of the mailing lists available.
Postfix Enabler for Mac OS X 10.4
Postfix Enabler for Tiger is now available, finding this out has saved me some time trying to test the prior release and figure out if it worked. This will activate the built in version of Postfix and within moments give you a fully functioning mail server.
May 1, 2005
Tiger mail server
Now that Mac OS X 10.4 is shipping, I know that some folks running mail servers will be interested in upgrading their systems. I've heard that 10.4 includes Postfix 2.1.5, the last official release in the 2.1.x series. Even though Postfix 2.2.x is available, light duty servers can still manage just fine with 2.1.5, but I'm assuming that MySQL support and other goodies probably weren't compiled into the build.
I'm going to get my server here upgraded in the next few days to Tiger, which for me will involve recompiling Postfix (I'll probably upgrade to 2.2.3), but I'll also see about doing some testing of the base 10.4 config and report my findings here.
January 27, 2005
Virtual domains fully functional
Well, it took a while, and a lot of testing, but I now have all the pieces in place. Postfix, MySQL, Courier IMAP, Postfix Admin, and virtual domains, all working together. See my prior posting for links to the various packages and more tips.
The last hurdle I took care of last night, eliminating the old mailbox format accounts on my server, and instead going with maildir format accounts that would work properly with all the above software. For anyone setting up a new server, save yourself some trouble and use the maildir format from the beginning.
January 25, 2005
Courier IMAP working
After doing some struggling and discovering that the latest versions of Courier IMAP (4.0.x) won't compile under OS X, I went back and got the 3.0.8 release installed and functional thanks to setup instuctions here and a handy .pkg installer of the Courier IMAP software from here. After struggling through understanding all the options, I had it up and running. Mostly.
One thing that quickly became painfully obvious was that Courier IMAP was designed only to access Postfix Maildirs, not the default mailbox files used by unix system accounts. So, my next step will be to get all users mail transitioned over to this format, and get Postfix to recognize my local domain as well as my virtual one...
January 23, 2005
Postfix, MySQL, and Virtual Domains
Well, after a few days of struggling, I've finally gotten Postfix Admin up and running here, and as part of that setup (and the handy HowTo posted on their site), I've got MySQL properly working with Postfix, and have virtual domains working as well.
The Postfix Admin software is a collection of PHP scripts for creating and maintaining domains and mailboxes, and eliminates all that mucking about with MySQL commands. They've got a good support forum with some great users there, which helped tremendously in getting this running.
January 12, 2005
Upgrades and more upgrades
Last week, I wrote of my attempts to get the latest MySQL and MovableType working together. Well, looks like I now have things going properly. When I upgraded MySQL this time, I left all the password information in the old format, and the upgrades went without a hitch.
Of course, I also went ahead and upgraded to Mac OS X 10.3.7, which meant (read here for more info) having to reinstall Postfix 2.1.x, so I took the opportunity to upgrade to 2.1.5 and added in MySQL support there too...
So, after several attempts at getting it all going, it looks like I'm finally all up to date on the various bits of my server now, finally! The Postfix part was easy (oh, also upgraded the PCRE software to 5.0), the main trick was making MovableType 3.14 happy with MySQL.
It looks like I can finally get virtual domains going, now that I can work with MySQL in Postfix. But I think I'll save installing Cyrus IMAP for another day...
November 20, 2004
More Whitelisting
I just had a problem yesterday caused by my greylisting, my wife missed a confirmation email from Delta on a flight she booked at the last minute because the greylisting temporarily bounced the email. I've since added Delta's server to my whitelist, and I'm searching for any other Airline's mail servers to add them in as well. I'll be posting an updated whitelist when I've compiled a bit more info.
November 17, 2004
Updated greylisting whitelist
Over the last few months, I've been tweaking the whitelist I use in conjunction with my greylisting software on my mail server. The whitelist is a list of known mail domains that I don't wish to delay mail from. Most greylisting packages will include a sample whitelist, I've found a few posted on the web, and have collected the most common settings, but over time have added a few new entries to several of the larger emailers.
If you're running a greylisting process, I'll leave it to you to figure the proper way to import these entries into your system...
mysql> select * from whitelist; +----------------+---------------------------------------------------------------+ | mail | commentaire | +----------------+---------------------------------------------------------------+ | 12.5.136 | Southwest Airlines (unique sender, no retry) | | 64.12.136 | AOL | | 64.12.137 | AOL | | 64.12.138 | AOL | | 152.163.225 | AOL | | 205.188.156 | AOL | | 205.188.157 | AOL | | 205.188.159 | AOL | | 64.125.132.254 | collab.net (unique sender per attempt) | | 64.125.133.202 | collab.net (unique sender per attempt) | | 66.135.209 | Ebay | | 66.135.197 | Ebay | | 66.218.66 | Yahoo Groups servers (common pool, no retry) | | 66.218.67 | Yahoo Groups servers (common pool, no retry) | | 66.218.69 | Yahoo Groups servers (common pool, no retry) | | 195.238.2 | Skynet | | 195.238.3 | Skynet | | 204.107.120 | Ameritrade (no retry) | | 205.206.231 | SecurityFocus.com (unique sender per attempt) | | 207.115.63 | Prodigy - broken software that retries continually (no delay) | | 207.171.168 | Amazon | | 207.171.180 | Amazon | | 207.171.187 | Amazon | | 207.171.190 | Amazon | | 213.136.52.31 | Mysql.com (unique sender) | | 217.158.50.178 | AXKit mailing list (unique sender per attempt) | | 66.94.237 | Yahoo Groups servers (common pool, no retry) | | 205.188.144 | AOL | +----------------+---------------------------------------------------------------+ 28 rows in set (0.00 sec)
November 7, 2004
Great Mac/Postfix site
I came across the site for the ECM Mail Server System a while back, it combines Postfix, MySQL, Courier IMAP, and some other cool utilities into what looks like a pretty solid package, all running under OS X. They've recently revised their code to be up to date with 10.3.5, and the latest Courier IMAP software. Definitely worth a look if you're running Postfix as a mail server and want to add more bells and whistles. When I get around to doing my upgrade here, I'll be following their steps closely.
October 6, 2004
Correction on ipop3d
The other day I mentioned that ipop3d was the pop3 server built into OS X, well, I was mistaken. It turns out that this version of ipop3d was actually installed by Postfix Enabler, this particular version is from the UW-IMAP project, and was in fact not a built in part of OS X.
Sorry for any confusion.
Apple updates Postfix
Yesterday, I read about Apple's latest security update, and noticed that it updates Postfix. After doing some checking on the Postfix mailing list, it appears that this update includes a new version of the Postfix code, and not just a minor config change.
What this means is that anyone (like me) who has updated their version of Postfix from the original Apple code will have their version stomped by intalling this update...
All is not lost, though. In my entry Postfix 2.1 on OS X, I mentioned an article at AFP548 which covers the install steps to get Postfix 2.1.1 running on OS X (article currently 'offline' there but still accessible, there are issues with the SASL implementation mentioned that are keeping the document from being officially sanctioned), and a short bit of code is listed to archive the current Postfix code.
This code was meant to archive the original Apple code so you would have a backup, this same code could be used to backup your current Postfix install, so that you can run the Software Update, then restore your Postfix.
Of course, you also have the choice of ignoring this particular update, but it's likely that when/if Apple released a 10.3.6 update, that the Postfix code will be rolled into that as most updates are, so you may get it down the road without realizing it.
My choice here is I think to just recompile Postfix using the AFP article's steps, and install per that article's instructions. This will also be an opportunity to upgrade to Postfix 2.1.4 (I'm running 2.1.3), and include MySQL support, which I'm sure I didn't do before, to help support the virtual domains that I want to implement.
September 27, 2004
Virtual Domains, almost there
After some strange errors caused by a late-night typo, I've got virtual domains about 50% working. That is, they're working fine in Postfix, mail seems to be getting delivered properly, Postfix is accepting the virtual accounts, etc. But there's one important piece missing... Mail clients can't login to read their mail!
I'm using POP3 for mail delivery, and the POP3 client built into OS X, ipop3d, apparently is only capable of handling local accounts, and not virtual accounts. So, I'm off to find a replacement POP3 client...
At this point I'm leaning towards using Cyrus IMAP, which handles both POP3 and IMAP, and also provides Secure Socket Layer (SSL) support, mainly because Cyrus is what's bundled with Mac OS X Server, but I'm still early in the process on deciding what to run. The other popular choice is Courier-IMAP, this will take some reading to see what seems to run better under OS X.
September 24, 2004
Virtual Domains with Postfix
Since I picked up a few extra domains earlier in the month, I wanted to play with adding one of them to Postfix as a virtual domain. Basically, this lets me set up mail accounts using the other domain names, but does not require that local unix level accounts be created for each user.
I'm still doing some reading on this and probably won't do anything for a month yet, as work has been incredibly busy lately, but I'll definitely post all about it when I get something going.
July 28, 2004
gld upgraded to 1.3
Gld, the greylisting system I use here, seems to have received an update while I was away. Seems the prior version crashed on my system while I was out, it was down about 24 hours before I noticed and brought it back online, so I went in search of an update once I was back.
No problems so far, but it's still a pain to compile on the Mac, and getting it going still involved some editing of the make file to get it working. I still hate the command line, but it's a necessary evil at times...
July 15, 2004
More Postfix setup notes
Well, I learn a little more about Postfix every day, and most recently it was to pay attention to the order you run your restrictions in... I had an odd problem that wasn't getting fixed like I intended, and had to run to the experts on the Postfix mailing list for help, and was set straight in short order. Read along for notes and my latest config...
One of these days, I need to put together a full tutorial on setting up Postfix and then just keep that updated, but for now, here's where I'm at.
First, the proper order of restrictions. If you get this wrong, you'll make mistakes and never know why things aren't working right. I was under the impression that the HELO restrictions happened first (since HELO happens first in SMTP), but that isn't how Postfix handles things. So, here's the actual order for restrictions:
smtpd_client_restrictions
smtpd_helo_restrictions
smtpd_sender_restrictions
smtpd_recipient_restrictions
Did you score 100%? Good for you if you did. ;) My problem was that I was trying to filter a spammer that was proclaiming himself to be sending mail from a mail server with my own DNS name. As if! The blocks I put in place weren't working (it was hitting my other spam filters first and getting bounced), so that's where I started digging.
It turns out that Postfix Enabler, which I run to help manage Postfix on my Mac, set up its own smtpd_client_restrictions with the rbl filters and some other things, and in my other settings I didn't set up my own list of restrictions for that block figuring that PE had it under control, and thinking my HELO restrictions would happen first. So, once I was straightened out on that, I decided to rework my set of restrictions to make things flow a bit better.
One thing that I will eventually go in and change is the access restrictions, and create separate lists for recipient, client, and sender restrictions. Postfix Enabler manages the one list called 'access' and rebuilds it every time I update (using the postmap command in the background, I'm assuming), and for now I'd rather not have to do that manually, but if my lists grow much more I may need to separate them to avoid filter errors.
So, here's my current config, with a few notes:
###Start PostfixEnabler###
alias_maps=hash:/etc/postfix/aliases
alias_database=hash:/etc/postfix/aliases
smtpd_sender_restrictions=hash:/etc/postfix/access
inet_interfaces=all
mynetworks_style=subnet
message_size_limit=10240000
mydomain=wrightthisway.com
myhostname=wrightthisway.com
smtpd_recipient_restrictions=permit_mynetworks,check_recipient_access hash:/etc/postfix/filtered_domains
unknown_local_recipient_reject_code=550
###End PostfixEnabler###
As indicated, this is what Postfix Enabler puts in on its own, most of those settings I'm not overwriting, but the restrictions definitely get changed. Also, I removed all the rbl listings from Postfix Enabler, so it now no longer creates its own client restrictions setting.
###Start Custom Config###
disable_vrfy_command = yes
default_process_limit = 10
smtpd_error_sleep_time = 30
strict_rfc821_envelopes = yes
smtpd_helo_required = yes
Same as what I've had before here, being strict with the protocols, setting some values to better manage bad servers, etc.
smtpd_helo_restrictions=
check_recipient_maps,
check_helo_access hash:/etc/postfix/helo_access,
check_client_access hash:/etc/postfix/access,
check_sender_access hash:/etc/postfix/access,
reject_unknown_hostname,
reject_invalid_hostname,
reject_non_fqdn_hostname,
reject_unknown_client,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
Now, here we have some real changes. I had copied my previous settings for client restrictions (that Postfix Enabler created for me) and brought those into the helo restriction block, and placed these after the checks I wanted in there first.
Specifically, the check_recipient_maps will make sure that the recipient actually has an account on my server, the check_helo_access is what I wrote about here, then the client and sender access filters kick in so that I can specifically allow certain email addresses or server names to pass through without going through further filters (for users on misbehaving servers that their admins won't fix, or some mail lists that happen to show up on a spammer listing, etc.), then my usual checking to make sure that the server I'm talking with checks out and is well configured.
smtpd_recipient_restrictions =
reject_unauth_destination,
check_recipient_access hash:/etc/postfix/access,
check_client_access hash:/etc/postfix/access,
check_sender_access hash:/etc/postfix/access,
check_policy_service inet:127.0.0.1:2525,
reject_rbl_client sbl-xbl.spamhaus.org,
reject_rbl_client list.dsbl.org,
reject_rbl_client relays.ordb.org,
reject_rbl_client dnsbl.njabl.org,
reject_rbl_client bl.spamcop.net,
reject_rbl_client dnsbl.ahbl.org,
reject_rbl_client dnsbl.sorbs.net,
reject_rbl_client relays.visi.com,
reject_rhsbl_client blackhole.securitysage.com,
reject_rhsbl_sender blackhole.securitysage.com,
reject_rhsbl_client rhsbl.ahbl.org,
reject_rhsbl_sender rhsbl.ahbl.org,
reject_rhsbl_client rhsbl.sorbs.net
reject_rhsbl_sender rhsbl.sorbs.net,
reject_rhsbl_client block.rhs.mailpolice.com,
reject_rhsbl_sender block.rhs.mailpolice.com,
reject_rhsbl_client dynamic.rhs.mailpolice.com,
reject_rhsbl_sender dynamic.rhs.mailpolice.com,
reject_rhsbl_client bogusmx.rfc-ignorant.org,
reject_rhsbl_sender bogusmx.rfc-ignorant.org,
reject_rhsbl_client dsn.rfc-ignorant.org,
reject_rhsbl_sender dsn.rfc-ignorant.org
smtpd_data_restrictions =
reject_unauth_pipelining,
permit
Here we can actually check the recipient access list, and I also check client and sender again for good measure, but I think perhaps that's not needed again here. Next is a check using the Gld Greylisting software, this will force a server to try a second time to deliver the message, which most spammers won't, then once the mail comes in again, we run it through the various block lists. If it passes those, it should be pretty clean.
header_checks = regexp:/etc/postfix/maps/header_checks
mime_header_checks = regexp:/etc/postfix/maps/mime_header_checks
body_checks = regexp:/etc/postfix/maps/body_checks
unknown_address_reject_code = 550
unknown_client_reject_code = 550
unknown_hostname_reject_code = 450
default_rbl_reply=$rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason} - see http://$rbl_domain for additional info. If this was actually a legitimate email to a real user, please forward this message to postmaster@wrightthisway.com for assistance.
###End Custom Config###
Here we stop the pipelining that some spammers use to flood a server, then do the header/body checks I've covered previously, set some reject codes (I'm now doing a 450 instead of a 550 on bad hostnames, this has helped some mails come through from larger ISPs that might have one bad server out of several, often redeliveries will come through from a different box and make it in, so the 450 gives them another chance), and lastly a custom rbl_reply, with some added text to help legitimate users reach me in the case of a problem. Spammers won't ever read these even if they do get the bounce.
And to guarantee that my postmaster account gets all mail sent to it, I've assed that account to my access list, so when the recipient checks run, this will pass those mails right on through without filtering.
So, that's this month's round of changes, we'll see how far this gets me. ;)
July 11, 2004
Blocking forged HELO in Postfix
I came across this article on Blocking spammers with Postfix HELO controls after finding a log entry for a (rejected) spam that appeared to be getting sent from my own mail server. My own server never sends email (all that routes through my ISP), so this is all instant spam.
In the case of the mail that appeared on my system, a different spam filter caught it before it was even delivered, but I figured having one more hurdle for mail to pass wouldn't hurt, so this seemed a good one to try out.
I followed the steps in the article to set this up here, pay special attention to the 'Making it so' section about half way through. When this file is created or changed, you need to do a 'postmap helo_access' in the terminal for the changes to take place. If you don't, not only won't your changes take place, but postfix will log a warning during each server connection telling you that the source file is newer than it's database.
July 7, 2004
Anomy's overzealous filtering
I'd completely forgotten that Postfix Enabler also installed a little utility called Anomy for helping to filter email. I'd forgotten until I had to troubleshoot a problem that was effecting some mail my wife was receiving and it brought up a problem I'd forgotten about...
Anomy has the ability to 'defang' parts of HTML code embedded in messages, usually this will keep rogue code from executing on your machine, but in this case was effecting how mail messages were being displayed.
I'd first noticed this when I was forwarding a certain email back to myself, the quoted section had several zeros on there that I couldn't figure where they came from. Eventually I discovered that this was some oddity in the HTML formatting, so I just turned that off in Mail (OS X's mail application) and forgot about it.
Today, my wife received a mail with several numbers at the start from someone who's mailed her before, and she never saw anything like that. After scratching my head then looking at the raw source of the message, things finally started clicking.
I need to dig into this a bit further, but I'm thinking with all the other spam filtering I'm doing, I should be safe to just disable this feature in Anomy, it's a quick fix, just change the following in the anomy.conf file:
feat_html = 1
change to:
feat_html = 0
That should do it! I'll investigate more and see what else I can find out about this problem. I figured it was easier to deactivate this one setting than to try removing Anomy itself.
July 3, 2004
Greylisting working well
The Gld greylisting utility I'm running with Postfix is working pretty well, and it's already saving me some headaches. I'm still getting other headaches from postmasters that don't quite understand how their mail servers should be configured and some ISPs that just don't care, but that's a different issue. One great thing about greylisting is that it can help BIG if your mail server happens to get hit with a dictionary attack, because of greylisting, all attempts, even to valid email account, will get a temporary bounce, and this is usually enough that the spammer won't bother running through their list again.
I'm going to be looking into cutting back on some of the filtering I'm doing now that greylisting is working, I'll post my updated Postfix config in a week or so with some new notes.
June 24, 2004
Gld docs coming, also new version
I'll soon have online my docs on getting Gld, a greylisting utility for Postfix, to compile and run under Mac OS X. It took a bit for me to get this working, but I'm learning as I go. ;)
Also, the author informs me that a new release of gld is in the works, and I hope to be running a beta of that shortly. The 1.0 release was pretty raw, the 1.2 version on the way should be a fair step forward.
June 23, 2004
Gld greylisting working
After some false starts and some tips from the author, I now have gld working to greylist incoming mail. It took a bit to get this all going, I plan to write it all up in a few days and will post the steps involved here.
June 19, 2004
Postfix 2.1.3 on OS X working
Well, after dorking around for a while tonight, I finally got Postfix 2.1.3 up and running, using the basic steps from AFP548.com, but without the problematic SASL code incorporated. So far, it seems to be working fine, but I've learned a few lessons...
I should start out by saying that I'm not a unix geek, I'm a Mac guy. I know enough about unix to be dangerous, which is to say, I don't really know that much, hence the danger. ;)
The box my mail and web server is running on is a dedicated system, I'm not using it for anything else. When I first set it up and was working on getting things up and running, I had to download a number of various routines and bits of code to make everything I wanted work and compile properly, and of course when I downloaded these, everything wound up on the desktop. Not minding much, I left it all there.
So, tonight I decided to clean up some folders, and organize things a bit. In the process of doing this, I created a folder called 'Mail code' and put the new Postfix and some other items in there, and then proceeded to try to get Postfix to compile. All went well until the last step, the 'make upgrade' command. Well, that one kept erroring with a line telling me I didn't have write access to the postfix folder the new code was in, even though I was running as root. After fussing with it for over half an hour, I finally discovered that the problem was that the Make command didn't like the space in the middle of the folder name, after I then changed it to an underscore, and recompiled everything to have the new path set, it worked great.
Second lesson... All those bits of code on my desktop, I have no idea if after compiling the final code was moved someplace more permanent (/etc/bin?) or if the live code is still living in those folders. I don't want to move anything for fear of breaking something, so for now I'm leaving them where they are. But, in the future I'll be a bit more careful about what goes where.
I'm going to leave things running like they are for a bit before messing with anything else. Once I'm sure that Postfix is working properly, I'm going to try out the Gld greylisting utility for Postfix and see how that works. Since it required Postfix 2.1 or later, I'll finally be able to give it a spin.
June 15, 2004
No do-not-spam list
I think this is good news... The FTC will NOT be creating their do-not-spam list. This is one battle that needs to be better fought with technology, and not by legistlation.
June 14, 2004
No SASL for Posfix, yet...
The AFP548.com article I mentioned over the weekend is offlne for now, though that link will still work since it was direct to the article. The author is having some SASL issues that he's still working to resolve. The steps in the article are still good for compiling a newer version of Postfix, if you don't need the SASL support.
Apparently the newer versions of SASL were causing problems with OS X Server, but 2.1.15 'seemed' to work, but still apparently has issues.
June 11, 2004
Postfix 2.1 on OS X
While surfing around for anyone that got Postfix 2.1 running under OS X tonight, I came across this article at AFP548.com. Not only does it give info on getting this running under Mac OS X 10.3, it also includes info on enabling PCRE, or Perl Compatible Regular Expression, a useful feature if you have more involved mail filtering rules set up. Most basic setups will use regex, more involved rules will require pcre, I've seen this most with some user created Spamassassin filters.
Also described are including support for SASL (Simple Authentication and Security Layer), but the article includes a (bad) link to an older version of this software, and isn't clear why the latest version (available when the article was published) wasn't used. I'm trying to contact the author for clarification. Also, the article seems geared towards OS X Server, but I'm assuming it should work well under standard OS X.