setup STARTTLS with sendmail and openSSL

STEP 1: Enable STARTTLS in sendmail: 1. Install openSSL ( as described in the openSSL’s INSTALL
file. After installation, be sure that the openSSL program is in your path
(cp /usr/local/ssl/bin/openssl /usr/bin), otherwise the program does
not work. 2. Create or edit /usr/src/sendmail-8.12.7/devtools/Site/site.config.m4 and
insert the following lines: APPENDDEF(`confINCDIRS’, `-I/usr/local/ssl/include’)
APPENDDEF(`confLIBDIRS’, `-L/usr/local/ssl/lib’)
APPENDDEF(`conf_sendmail_LIBS’, `-lssl -lcrypto’) 3. Rebuild and install sendmail with the -c option (see README in
devtools/Site): cd /usr/src/sendmail-8.12.7
./Build -c
./Build install 4. Check to see if sendmail is compiled with STARTTLS: /usr/sbin/sendmail -d0.1 -bp 5. Edit /usr/src/sendmail-8.12.7/cf/cf/ and insert the following
lines: define(`confCACERT_PATH’, `/etc/mail/certs/’)dnl
define(`confCACERT’, `/etc/mail/certs/cacert.pem’)dnl
define(`confSERVER_CERT’, `/etc/mail/certs/cert.pem’)dnl
define(`confSERVER_KEY’, `/etc/mail/certs/key.pem’)dnl
define(`confCLIENT_CERT’, `/etc/mail/certs/cert.pem’)dnl
define(`confCLIENT_KEY’, `/etc/mail/certs/key.pem’)dnl 6. Backup and regenerate /etc/mail/ cd /etc/mail
cd /usr/src/sendmail-8.12.7/cf/cf
./Build install-cf 7. Now you have to create three files: cacert.pem (CA certificate), cert.pem
(x.509 certificate, signed by CA) and key.pem (x.509 private key). This is
how you do this: cd /usr/local/ssl/certs
../misc/ -newca
../misc/ -newreq-nodes
../misc/ -sign
mkdir /etc/mail/certs
cp demoCA/cacert.pem /etc/mail/certs
cp newreq.pem /etc/mail/certs/key.pem
cp newcert.pem /etc/mail/certs/cert.pem
chmod 400 /etc/mail/certs/key.pem When the command ‘ -newca’ asks for a Common Name, fill in the name of
your organization. When the command ‘ -newreq-nodes’ asks for a Common
Name, you must enter the hostname of your smtp server and it must be the
same name as your smtp-server field on the mailclient, e.g. 8. Restart sendmail : kill `head -1 /var/run/`
/usr/sbin/sendmail -L sm-mta -bd -q30m
telnet localhost 25 9. Check if sendmail supports STARTTLS. Issue a ‘EHLO localhost’ command.
You should see a line 250-STARTTLS: [email protected]:/# telnet localhost 25
Connected to localhost.
Escape character is ‘^]’.
220 ESMTP Sendmail 8.12.7/8.12.7; Wed, 19 Feb 2003
10:59:57 +0100
EHLO localhost Hello [email protected] [], pleased to meet
250 HELP Check your logfiles if you don’t see it (increase LogLevel to 14 in your So far for the server side setup.
STEP 2: Client side setup: The next step is to configure your mailclients for SSL smtp connection and
install the client personal and root certificates on them. If you don’t
install these certificates, the client will complain that it cannot verify
the server certificate. This is normal, because you are using a self signed
servercertificate. You do not have this problem if you are buying a
certificate from a trusted provider. For Outlook Express 6 e.g., you wil see this warning: “The server you are connected to is using a security certificate that could
not be verified. A certificate chain processed, but terminated in a root
certificate which is not trusted by the trust provider.” This is how you create your client certificate: cd /usr/local/ssl/certs
../misc/ -pkcs12 It will create a file newcert.p12 that you can import in your client. You
will have to fill in a password, that the client have to use when he imports
the newcert.p12 file, so don’t use the same as you have used earlier. — For Outlook Express 6 you can follow these steps: 1. Copy the file newcert.p12 to a directory on your Windows client and
rightclick on it. 2. Choose install PFX and follow instructions. It will install a personal
and a root certificate. That’s great. You can check it in Explorer >
Internet Options. 3. In Outlook, turn on the SSL option for your outgoing mailserver. — For Netscape Messenger 4.7 you can follow these steps: 1. Start Messenger, click on the little lock on the bottom left corner. 2. Choose Certificates > Yours > Import a certificate 3. Import the file newcert.p12. 4. Click on Signers, select your CAcert, and edit it to enable all features. 5. Click on Yours, select your personal cert, and click Verify. You
should get “successfuly verified”. 6. Enable Secure SMTP in Messenger config. Now send a mail to yourself and check the message source. It must contain a
Received header with SSL information. You can also check your sendmail log.
If you see Verify=OK, then the server verified the presented client
certificate as OK. If you see Verify=NO, then the client didn’t present a
certificate and you are probably using Outlook Express. This is what I found
on the Internet: Outlook Express as of Internet Explorer 5 will work, but it
will not present any client certificate. So you can encrypt your email
transfer but you cannot authenticate (and relay) with client certificates
(source: … /test.html). Post
a follow-up for comments on this. I get Verify=OK with Netscape Messenger 4.7 and Verify=NO with Outlook
Express 6.
STEP 3: Allow relaying based on client certificate: The last step is to allow relaying based on a trusted client certificate.
This is very useful, because your client can then send mail to you
mailserver, independent of his IP. It only works if the server can verify
the client (Verify must be OK). Do the following: 1. Open your sendmail logfile and search for the “cert-issuer” field that
came from your client. 2. Copy the content of this field to your access database file (probably
/etc/mail/access_map, see also
and insert CERTISSUER: and RELAY, like this:
SomeEmail RELAY 3. Create database map: makemap hash access_map because you want to relay. You can use [email protected]. It will bounce your
mail and you can check all headers. Remember that this does this setup only
provide a secure transmission from sender to mailserver. Your mail will
probably go unsecure from there. Now, that’s it. I hope this document is usefull and correct, don’t mail me
for suggestion/corrections but please follow-up to this post. You can find more information on:]]>

Leave a Reply