# OpenSMTPD Most OpenSMTPD HOWTOs show[^0] and explain some basic config like: ``` pki mail.hypno.cat cert "/etc/ssl/mail.hypno.cat.fullchain.pem" pki mail.hypno.cat key "/etc/ssl/private/mail.hypno.cat.key" table aliases file:/etc/mail/aliases listen on all tls pki mail.hypno.cat listen on all port submission tls-require pki mail.hypno.cat auth filter rspamd action "local_mail" maildir junk alias action "outbound" relay helo mail.hypno.cat match from any for domain "hypno.cat" action "local_mail" match for local action "local_mail" match from any auth for any action "outbound" match for any action "outbound" ``` Which is quite nice, but doesn't show the full power of OpenSMTPD. So lets look how to use OpenSMTPD in combination with FreeIPA, Mailman3, and Request Tracker. ## What do we want So we[^1] administrate a mail server for our users. The mail server stands behind a relay of our ISP. So mails for us will send to our mail server with callout verification. Mails we want to send go through a smarthost. Our users have mail addresses on our domain and the mails are forwarded to there private email account. The addresses and the forward addresses are stored in LDAP. The LDAP also has some mail-groups. So each member of a group shall get mails to the address of the mail-group. We also provide some mailing lists and a RT. So we want to forward mails to mailing lists to the LMTP interface of Mailman. Mails for the rt shall passed to rt-mailgate. ## Tables Most of the magic in our config is based on tables. Tables are everywhere in smtpd and are quite powerful. Depending on the context a table is used to look up if a key exists, look up a value for a key, or to get a list of keys. Also depending on the context different services are used. I.e. a alias lookup can be handled different then a userbase lookup. This allows to use the same table in multiple contexts. Which makes the smtpd.conf more readable. OpenSMTPD per default includes four table types: * static tables defined in the smtpd.conf * file based tables defined in a extra file * db based tables defined in makemap(8) * external tables defined by an external program For external tables there is the opensmtpd-extras package with includes the most common tables. We need the tables for ldap, PostgreSQL and SQLite. Sadly tables are not well tested[^2] so it's possible that you find bugs. So when you want to use them: Test your setup and report bugs before you go live with it. The code itself is quite good, so chances are good that they are fixed in a couple of days. Also at the time of writing the table-ldap of the last release is not working. When you plan to use table-ldap you need to use the latest version out of the repo. [comment]: <> XXX wait till last fix is pushed ## the magic of match OpenSMTPD config has multiple match rules in the config. The first one which matches will be used for this mail. Each rule has a at least a from and to. The from and to are implicit set to `` and `` when not given. There are a bunch of other options which allows all kind of setups. We use rcpt-to because we want to primary match the recipient. So the matching config will look something like this: ``` match from any for domain rcpt-to action action-a match from any for domain rcpt-to action action-b ... match for domain reject ``` There is also the `for rcpt-to` syntax which translate to `for domains rcpt-to`. We explicit set the domains to make the config more clear and avoid unnecessary LDAP lookup. The problem with the rcpt-to option is that it only do lookup for the complete recipient. The static table does "fix" this with a special key-compare function which parses the recipient. But dynamic tables like LDAP can't do this. For some cases a rcpt-user option would be nice. ## LDAP schema Beside the Mail attribute of the organizationalPerson and Mailgroup ObjectClass we want to store some other things in the LDAP. Because we only provide a forwarding service we need to store the forward address in the LDAP. We have multiple forward addresses depending on what route the mail is route to the users. So we have a extra ObjectClass for this: ``` objectClasses: ( 1.3.6.1.4.1.87.8.99.3.1.2.2 NAME 'kitfsmiMailUser' DESC 'Mail-Routing Felder für Nutzer' SUP top AUXILIARY MAY ( kitfsmiPersonalEmail $ kitfsmiOfficeEmail $ kitfsmiListEmail $ kitfsmiEmailAdress ) ) ``` Personal for mails direct to the address of the user, Office for mails to a group which the users is a member of and Lists for mailing lists. The kitfsmiEmailAdress attribute allows to have extra email addresses which are not displayed by the Website. This is useful for name changes. We also want to deliver mails to LDAP groups we need to store the mail address in the LDAP. To have extra Addresses which are not shown by other systems we also have an ObjectClass for this: ``` objectClasses: ( 1.3.6.1.4.1.87.8.99.3.1.2.3 NAME 'kitfsmiMailGroup' DESC 'Mail-Routing Felder für Gruppen' SUP top AUXILIARY MAY ( kitfsmiEmailAdress ) ) ``` Most of the following config is also possible with a public LDAP schema. Only the forwarding might need some extra schema or a different solution (i.e. storing forwards in a SQL database). ## mails per user So our users are stored in IPA and auth is done by PAM. But we want to restrict the sender address. To do this smtpd has the sender option. So we have following config: ``` table users ldaps:/etc/smtpd/ldap.users.conf listen on eth0 auth senders ``` The senders option will request the mailaddrmap service of the table. So the LDAP config file needs the basedn, a filter for the mailaddrmap and the LDAP attribute with the allowed sender: ``` basedn: cn=users,cn=accounts,dc=fsmi,dc=org mailaddrmap_filter: (&(objectclass=person)(uid=%s)) mailaddrmap_attributes: kitfsmiEmailAdress ``` When you use a default schema you can use the mail attribute. We can use this with the normal config: ``` action "relay" relay via $smarthost match from auth for any action relay ``` So now users can send mails. We also want to receive mails. Therefor we need a match rule which match the mail address of the users. So we need a match and action for this: ``` action localusers forwoard-only virtual match for rcpt-to for domain action localusers ``` The rcpt-to option request the mailaddr service and the virtual option uses the alias service. This can simple add ldap.users.conf: ``` alias_filter: (&(objectclass=person)(|(mail=%s)(kitfsmiEmailAdress=%s))) alias_attributes: kitfsmiPersonalEmail mailaddr_filter: (&(objectclass=person)(|(mail=%s)(kitfsmiEmailAdress=%s))) mailaddr_attributes: uid ``` ## groups As mentioned before we also have some mailgroups. Because we want our users to filter them we also want to add some header fields. To do this we deliver our mails to special MDA which adds these fields and pipe them back to smtpd. So first we need to know about which mails are go to the groups: we do this the same way as with the users but with a different basedn and some different filters: ``` basedn: cn=groups,cn=accounts,dc=fsmi,dc=org alias_filter: (&(objectclass=mailgroup)(mail=%s)) alias_attributes: cn mailaddr_filter: (&(objectclass=mailgroup)(mail=%s)) mailaddr_attributes: cn ``` and the corresponding smtpd.conf: ``` table fakeusers sqlite:/etc/smtpd/sqlite.remap-usermap.conf table groups ldap:/etc/smtpd/ldap.groups.conf action "groups" mda "/etc/smtpd/list-id-mda.sh %{dest.user} %{user.username}" user "opensmtpd" userbase virtual match for rcpt-to for domain action "groups" ``` Wait what is this fakeusers table? This is a hack: Because our groups are not users on the mail host[^3]. Because OpenSMTPD only delivers to `users` we need to define this users. The userbase option allows to define these users. Problem is we need a mapping table with values like `$uid:$gid:$homedir`. We can't generate this out of LDAP because they are only mail groups. Because we already know that all ``users'' are ``valid'' we just have a catch all userbase. The config for the sqlite table looks like this: ``` dbpath /dev/null query_userinfo SELECT 110, 117, '/nonexistent' WHERE '' != ? ``` This fakes for each recipient be a user with the uid and gid of OpenSMTPD. The recipient check is done in with rcpt-to in the match, so this doesn't cause problems. So back to groups. The MDA adds some fields and pipes the mail to `sendmail -f postmaster+%{dest.user}@fsmi.org %{user.username}@groups.fsmi.org`. Now we have the mail back in smtpd and must handle them. We have set the rcpt domain to ``groups.fsmi.org'' to match then. Because we set the localpart to the CommonName of the group we check the memberOf attribute to get the users. So table-ldap config looks like following: ``` basedn: cn=users,cn=accounts,dc=fsmi,dc=org alias_filter: (&(objectclass=kitfsmimailuser)(memberof=cn=%s,cn=groups,cn=accounts,dc=fsmi,dc=org)) alias_attributes: kitfsmiOfficeEmail mailaddr_filter: (&(objectclass=kitfsmimailuser)(memberof=cn=%s,cn=groups,cn=accounts,dc=fsmi,dc=org)) mailaddr_attributes: uid ``` and the corresponding smtpd.conf: ``` action "groups_users" forward-only alias match from local for domain "groups.fsmi.org" action "groups_users" ``` Now we have a small problem: what about groups without members? Our old setup had some interesting SQL statements to determine a fallback address. But with table-ldap this is not possible, because I can do only one query per recipient. I mentioned before that I would like to have a `rcpt-user` option. This way a first match could match only groups with members and a second one use a action with fallback addresses. ## sender address When we forward mails to the private mailboxes of our users we want to change the sender. This has two reasons: Quit obvious is that with the naive config SPF will fail. More important for us is that the private address of our users don't leek. As little service we also add some fields for the user to filter the mails. To do this we basically do the same as for groups. first we change alias_attributes to uid and change the action to our MDA. ``` action "users" mda ... ``` We don't need the userbase hack, because the users are normal[^4] users on this system. The MDA sets the recipient to `$uid@users.fsmi.org`. This is matched and forwarded with a forward-only rule and a table-ldap: ``` action "users_out" forward-only alias match from any for domain "users.fsmi.org" action "users_out" ``` and a table-ldap.conf: ``` alias_filter: (&(objectclass=person)(uid=%s)) alias_attributes: kitfsmiPersonalEmail ``` You might wounder the recipient is i.e. `satanist@users.fsmi.org` but the uid is probably only `satanist`. This might cause the same problem as with groups and recipients. But aliases are handled different by OpenSMTPD. For alias and virtual smtpd will make multiple table lookup, first with the full recipient and later with the local part. ## request tracker and mailman Using mailman and request tracker is straightforward. Mailman and request-tracker store all there data in PostgreSQL. So we can use table-postgres to get the recipient addresses. For rt we use virtual to map the addresses to the queue name. We have two tables, one for the psql.rt.conf: ``` query_alias SELECT name FROM queues WHERE correspondaddress LIKE $1; query_mailaddr SELECT name FROM queues WHERE correspondaddress LIKE $1; ``` We also have a similar config for the comment-address. You must be a bit careful with the queue names, because of the encoding. We have decided that we limit the queue name to a-z. For mailing list we only need the addresses. So we need only the mail-addresses. But we also need special handle addresses like -join. These are handled by the SQL query: ``` query_mailaddr SELECT $1 as email FROM mailinglist WHERE $1 SIMILAR TO (list_name || '(-owners?|-requests?|-bounces|-confirm|-join|-leave)?(\+\S*)?@fsmi.org'); ``` These tables are used in separate matches as rcpt-to. So we have a action for each table: ``` action "run_rt_queue_correspond" mda "/usr/bin/rt-mailgate-5 --queue '%{user.username}' --action correspond --url https://rt.fsmi.org/" user "opensmtpd" userbase virtual action "run_rt_queue_comment" mda "/usr/bin/rt-mailgate-5 --queue '%{user.username}' --action comment --url https://rt.fsmi.org/" user "opensmtpd" userbase virtual action "mailman-lmtp" relay host "lmtp://127.0.0.1:8024" match from any for domain rctp-to match from any for domain rctp-to match from any for domain rctp-to ``` Now when we create a new rt-queue all is stored in PostgreSQL. So the mails get routed correctly without touching anything at the mail-server. Same is true for new mailing lists. ## full config Combining all this and add some details we get following config: ``` # TODO move this to a separate file # Add blocked recipients to this table. You can specify individual addresses # by typing "foo@example.com" or entire domains with "@example.com". table blocked_recipients { "somebody-really-bad@example.com" } # Add blocked senders to this table. Syntax as above. table blocked_senders { "somebody-really-bad@example.com" } # Messaged displayed when sending to an blocked recipient. table blocked_rcpt_message { "@" = "error:544 Recipient blocked by an administrator. Please contact rechner@fsmi.org for further information!"} table blocked_sender_message { "@" = "error:544 sender blocked by local policy"} table aliases file:/etc/aliases table domains file:/etc/smtpd/domains table users ldap:/etc/smtpd/ldap.users.conf table users_internal ldap:/etc/smtpd/ldap.users-internal.conf table users_out ldap:/etc/smtpd/ldap.users-out.conf table mailinglists postgres:/etc/smtpd/postgres.mailinglists.conf table rt_queues_correspond postgres:/etc/smtpd/postgres.rt.conf table rt_queues_comment postgres:/etc/smtpd/postgres.rt-comment.conf table tmpusers sqlite:/etc/smtpd/sqlite.remap-usermap.conf table aemter_internal ldap:/etc/smtpd/ldap.aemter-internal.conf table aemter_users ldap:/etc/smtpd/ldap.aemter.conf pki fsmi-mx1.fsmi.org cert "/etc/letsencrypt/live/fsmi-mx1.fsmi.org/fullchain.pem" pki fsmi-mx1.fsmi.org key "/etc/letsencrypt/live/fsmi-mx1.fsmi.org/privkey.pem" # Messages tagged with "internal" can reach internal recipients # (e.g. internal office mails like amt-rechner-list) listen on socket tag internal listen on localhost listen on enp1s0 tls pki fsmi-mx1.fsmi.org listen on enp1s0 port submission mask-src tls-require pki fsmi-mx1.fsmi.org senders auth # actually send out mails to users action "users_out" forward-only alias # fix return-path for users to prevent leaking private addresses and to mitigate SPF issues action "add-user-mail-from" mda "/etc/smtpd/user-mail-from.sh %{dest} %{user.username}" user "opensmtpd" virtual # Adds List-Id header to email action "add-amt-id" mda "/etc/smtpd/list-id-mda.sh %{dest.user} %{user.username}" user "opensmtpd" userbase virtual # Send mails to actual users in office action "aemter_users" forward-only alias action "aliases" expand-only virtual # Mailman3 delivery action "mailman-lmtp" relay host "lmtp://127.0.0.1:8024" # Relay messages through scc action "relay" relay host "smtps://smarthost.kit.edu" # Prevent messages for unwanted recipients action "block_message" expand-only virtual action "block_message_sender" expand-only virtual # Mail-Gateway to https://rt.fsmi.org/ action "run_rt_queue_correspond" mda "/usr/bin/rt-mailgate-5 --queue '%{user.username}' --action correspond --url https://rt.fsmi.org/" user "opensmtpd" userbase virtual action "run_rt_queue_comment" mda "/usr/bin/rt-mailgate-5 --queue '%{user.username}' --action comment --url https://rt.fsmi.org/" user "opensmtpd" userbase virtual # Filter mail to blocked recpients early match from any for rcpt-to action "block_message" match from any mail-from action "block_message_sender" # Mailman3 match from any for domain rcpt-to action "mailman-lmtp" # request-tracker-5 match from any for domain rcpt-to action "run_rt_queue_correspond" match from any for domain rcpt-to action "run_rt_queue_comment" # Offices match from local for domain "aemter.fsmi.org" tag internal action "aemter_users" match from any for domain rcpt-to action "add-amt-id" # Users match from any for domain "users.fsmi.org" tag internal action "users_out" match from any for domain rcpt-to action "add-user-mail-from" # Aliases match from any for domain action "aliases" # Any mail for us not matched should be rejected match from any for domain reject # All messages should leave the server through SCC-Smarthost match from local for any action "relay" match from auth for any action "relay" ``` I would say this config is easy to understand for the complexity it provides. When you have any feedback just send me an email (satanist+smtpd@bureaucracy.de)[mailto:satanist+smtpd@bureaucracy.de]. [^0]: i.e. https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/ [^1]: The student Representative of cs students at KIT [^2]: Tables are currently changing inside OpenSMTPD. This doesn't effect this HOWTO, but means they get better tested. [^3]: They are not even unix groups. [^4] But we have set up a policy in IPA which restrict them to the smtpd PAM service.