From 5b54d1d32ee1f786c6c9a7f6bf4db07ceea871e2 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Wed, 31 Jan 2024 10:24:43 +1300 Subject: [PATCH] refactor: `relay.sh` (#3845) * chore: `relay.sh` helper - Reference user config paths via variables * chore: Better document postfix helper `_vhost_collect_postfix_domains()` The functionality is effectively the same for the two configs for the most part when it comes to parsing out a domain from the target value. Virtual aliases is more flexible in value, which may not have a domain-part present (manual user edit). * chore: `check-for-change.sh` - Support VHOST change visibility - Moves the "handle changes" logic into it's own scoped function, out of the main change detection loop logic. - This will be benefit a future commit change that will rely on `VHOST_UPDATED=1`. * chore: `relay.sh` - Minor revisions to minimize diff noise - Better phrasing of the current logic comments. - Regex patterns assigned to variables (easier to grok intention) - Bulk of the logic for generating `/etc/postfix/relayhost_map` wrapped into a separate function with Postfix config setting handled separately. * refactor: `relay.sh` opt-out logic - Split the two distinct features that configure `/etc/postfix/relayhost_map` into separate functions (_`MATCH_VALID` var no longer needed for legacy support_). - Instead of extracting domains from `postfix-accounts.cf` + `postfix-virtual.cf`, this has already been handled at `/etc/postfix/vhost`, sourcing from there is far less complicated. - Rename loop var `DOMAIN_PART`to `SENDER_DOMAIN` for better context of what it represents when appended to the config file. - Revised maintenance notes + guidance towards a future refactor of this relayhost feature support. * docs: `relay.sh` - Additional comment revisions * feat: `DEFAULT_RELAY_HOST` can now also use relay credentials ENV - Remove comment regarding `smtp_sasl_password_maps = static:${RELAY_USER}:${RELAY_PASSWORD}`, it could be used but `main.cf` presently has `644` permissions vs the `sasl_passwd` file permissions of `600`, less secure at preventing leaking of secrets (ignoring the ENV exposure itself). - Move the `main.cf` settings specific to relayhost credentials support / security into to the relevant function scope instead. This also allows for the configuration to be applied by a change detection event without container restart requirement. - Outer functions for setup and change detection to call have a clearer config dependency guard, as does the `_legacy_support()`. - These changes now support `DEFAULT_RELAY_HOST` to leverage the relay credentials ENV as well. - `DATABASE_RELAYHOSTS` is available in scope to the functions called here that reference it. * docs: Revised ENV docs on relay host config Better quality guidance on configuring relay hosts. * chore: Add entry to `CHANGELOG.md` * fix: `relay.sh` - `grep` regex compatibility with `+` requires `-E` * chore: `postfix.sh` - `FIRST_FIELD` => More descriptive field name --- CHANGELOG.md | 11 ++ docs/content/config/environment.md | 87 +++++++++--- mailserver.env | 16 ++- target/scripts/check-for-changes.sh | 55 ++++---- target/scripts/helpers/postfix.sh | 23 ++-- target/scripts/helpers/relay.sh | 199 ++++++++++++++-------------- 6 files changed, 239 insertions(+), 152 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d16c03ec..1f426a5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,13 @@ The most noteworthy change of this release is the update of the container's base - DMS `main.cf` has renamed `postscreen_dnsbl_whitelist_threshold` to `postscreen_dnsbl_allowlist_threshold` as part of this change. - `smtpd_relay_restrictions` (relay policy) is now evaluated after `smtpd_recipient_restrictions` (spam policy). Previously it was evaluated before `smtpd_recipient_restrictions`. Mail to be relayed via DMS must now pass through the spam policy first. - The TLS fingerprint policy has changed the default from MD5 to SHA256 (_DMS does not modify this Postfix parameter, but may affect any user customizations that do_). +- **Features:** + - The relay host feature was refactored ([#3845](https://github.com/docker-mailserver/docker-mailserver/pull/3845)) + - The only breaking change this should introduce is with the Change Detection service (`check-for-changes.sh`). + - When credentials are configured for relays, change events that trigger the relayhost logic now reapply the relevant Postfix settings: + - `smtp_sasl_auth_enable = yes` (_SASL auth to outbound MTA connections is enabled_) + - `smtp_sasl_security_options = noanonymous` (_credentials are mandatory for outbound mail delivery_) + - `smtp_tls_security_level = encrypt` (_the outbound MTA connection must always be secure due to credentials sent_) - **Environment Variables**: - `SA_SPAM_SUBJECT` has been renamed into `SPAM_SUBJECT` to become anti-spam service agnostic. ([3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820)) - As this functionality is now handled in Dovecot via a Sieve script instead of the respective anti-spam service during Postfix processing, this feature will only apply to mail stored in Dovecot. If you have relied on this feature in a different context, it will no longer be available. @@ -60,6 +67,10 @@ The most noteworthy change of this release is the update of the container's base - DMS config files that are parsed line by line are now more robust to parse by detecting and fixing line-endings ([#3819](https://github.com/docker-mailserver/docker-mailserver/pull/3819)) - Variables related to Rspamd are declared as `readonly`, which would cause warnings in the log when being re-declared; we now guard against this issue ([#3837](https://github.com/docker-mailserver/docker-mailserver/pull/3837)) +- Relay host feature refactored ([#3845](https://github.com/docker-mailserver/docker-mailserver/pull/3845)) + - `DEFAULT_RELAY_HOST` ENV can now also use the `RELAY_USER` + `RELAY_PASSWORD` ENV for supplying credentials. + - `RELAY_HOST` ENV no longer enforces configuring outbound SMTP to require credentials. Like `DEFAULT_RELAY_HOST` it can now configure a relay where credentials are optional. + - Restarting DMS should not be required when configuring relay hosts without these ENV, but solely via `setup relay ...`, as change detection events now apply relevant Postfix setting changes for supporting credentials too. ## [v13.3.1](https://github.com/docker-mailserver/docker-mailserver/releases/tag/v13.3.1) diff --git a/docs/content/config/environment.md b/docs/content/config/environment.md index c98016d1..5db1c2c3 100644 --- a/docs/content/config/environment.md +++ b/docs/content/config/environment.md @@ -1012,36 +1012,90 @@ you to replace both instead of just the envelope sender. - **empty** => Derived from [`OVERRIDE_HOSTNAME`](#override_hostname), `$DOMAINNAME` (internal), or the container's hostname - Set this if auto-detection fails, isn't what you want, or you wish to have a separate container handle DSNs -#### Default Relay Host +#### Relay Host + +!!! tip "`RELAY_HOST` vs `DEFAULT_RELAY_HOST`" + + `DEFAULT_RELAY_HOST` is encouraged, but presently does not support sender domain opt-out (`setup relay exclude-domain`). + +!!! tip "Opt-in for relay host support" + + If you only want to enable relay for specific sender domains, use can use opt-in via `setup relay add-domain`. ##### DEFAULT_RELAY_HOST -- **empty** => don't set default relayhost setting in main.cf -- default host and port to relay all mail through. - Format: `[example.com]:587` (don't forget the brackets if you need this to - be compatible with `$RELAY_USER` and `$RELAY_PASSWORD`, explained below). +Configures a default relay host. -#### Multi-domain Relay Hosts +!!! info + + - All mail sent outbound from DMS will be relayed through the configured host, unless sender-dependent relayhost maps have been configured (_which have precedence_). + - The host value may optionally be wrapped in brackets (_skips DNS query for MX record_): `[mail.example.com]:587` vs `example.com:587` + +!!! abstract "Technical Details" + + Configures the Postfix `main.cf` setting: [`relayhost`][postfix-config::relayhost] ##### RELAY_HOST -- **empty** => don't configure relay host -- default host to relay mail through +Configures a default relay host. + +!!! info + + - This is a legacy ENV. It is however required for the opt-out feature of `postfix-relaymap.cf` to work. + - When configured, all known mail domains managed by DMS will be configured to relay outbound mail, just like `DEFAULT_RELAY_HOST`. + +!!! note + + Expects a value like `mail.example.com`. Internally this will be wrapped to `[mail.example.com]`, so it should resolve to the MTA directly. + + Do not use with `DEFAULT_RELAY_HOST`. `RELAY_HOST` has precedence as it is configured with `sender_dependent_relayhost_maps`. + +!!! abstract "Technical Details" + + This feature is configured internally using the: + + - Postfix setting with config: [`sender_dependent_relayhost_maps = texthash:/etc/postfix/relayhost_map`][postfix-config::relayhost_maps] + - DMS Config volume support via: `postfix-relaymap.cf` (_generates `/etc/postfix/relayhost_map`_) ##### RELAY_PORT -- **empty** => 25 -- default port to relay mail through +Default => 25 + +Support for configuring a different port than 25 for `RELAY_HOST` to use. + +!!! note + + Requires `RELAY_HOST`. + +#### Relay Host Credentials + +!!! warning "Configuring relay host credentials make outbound authentication mandatory" + + Presently when `RELAY_USER` + `RELAY_PASSWORD` or `postfix-sasl-password.cf` are configured, all outbound mail traffic is configured to require a secure connection established and forbids the omission of credentials. + + Additional feature work is required to only enforce these requirements on mail sent through a configured relay host. ##### RELAY_USER -- **empty** => no default -- default relay username (if no specific entry exists in postfix-sasl-password.cf) - ##### RELAY_PASSWORD -- **empty** => no default -- password for default relay user +Provide the credentials to use with `RELAY_HOST` or `DEFAULT_RELAY_HOST`. + +!!! tip "Alternative credentials config" + + You may prefer to use `setup relay add-auth` to avoid exposure of secrets in ENV. + + - With the CLI command you must provide each sender domain relay credentials. + - Alternatively manually edit `postfix-sasl-password.cf` with the correct relayhost entry (_`DEFAULT_RELAY_HOST` value or as defined in `/etc/postfix/relayhost_map`_) to provide credentials per relayhost configured. + +!!! abstract "Technical Details" + + Credentials for relay hosts are configured internally using the: + + - Postfix setting with config: [`smtp_sasl_password_maps = texthash:/etc/postfix/sasl_passwd`][postfix-config::sasl_passwd] + - DMS Config volume support via: `postfix-sasl-password.cf` (_generates `/etc/postfix/sasl_passwd`_) + + This file has relay hosts that must match the `host:port` of `/etc/postfix/relayhost_map` or `main.cf:relayhost`. DMS support handles this for you. [docs-rspamd]: ./security/rspamd.md [docs-tls]: ./security/ssl.md @@ -1050,3 +1104,6 @@ you to replace both instead of just the envelope sender. [docs-tls-selfsigned]: ./security/ssl.md#self-signed-certificates [docs-accounts-quota]: ./user-management.md#quotas [docs::dms-volumes-state]: ./advanced/optional-config.md#volumes-state +[postfix-config::relayhost]: https://www.postfix.org/postconf.5.html#relayhost +[postfix-config::relayhost_maps]: https://www.postfix.org/postconf.5.html#sender_dependent_relayhost_maps +[postfix-config::sasl_passwd]: https://www.postfix.org/postconf.5.html#smtp_sasl_password_maps diff --git a/mailserver.env b/mailserver.env index 38b9adca..6fc9c7c2 100644 --- a/mailserver.env +++ b/mailserver.env @@ -619,8 +619,8 @@ SRS_SECRET= # Setup relaying all mail through a default relay host # -# empty => don't configure default relay host -# default host and optional port to relay all mail through +# Set a default host to relay all mail through (optionally include a port) +# Example: [mail.example.com]:587 DEFAULT_RELAY_HOST= # ----------------------------------------------- @@ -630,18 +630,22 @@ DEFAULT_RELAY_HOST= # Setup relaying for multiple domains based on the domain name of the sender # optionally uses usernames and passwords in postfix-sasl-password.cf and relay host mappings in postfix-relaymap.cf # -# empty => don't configure relay host -# default host to relay mail through +# Set a default host to relay mail through +# Example: mail.example.com RELAY_HOST= # empty => 25 # default port to relay mail RELAY_PORT=25 +# ----------------------------------------------- +# --- Relay Host Credentials Section ------------ +# ----------------------------------------------- + +# Configure a relay user and password to use with RELAY_HOST / DEFAULT_RELAY_HOST + # empty => no default -# default relay username (if no specific entry exists in postfix-sasl-password.cf) RELAY_USER= # empty => no default -# password for default relay user RELAY_PASSWORD= diff --git a/target/scripts/check-for-changes.sh b/target/scripts/check-for-changes.sh index 2fa004b0..3f268710 100755 --- a/target/scripts/check-for-changes.sh +++ b/target/scripts/check-for-changes.sh @@ -49,17 +49,7 @@ function _check_for_changes() { local CHANGED CHANGED=$(_get_changed_files "${CHKSUM_FILE}" "${CHKSUM_FILE}.new") - - # Handle any changes - _ssl_changes - _postfix_dovecot_changes - _rspamd_changes - - _log_with_date 'debug' 'Reloading services due to detected changes' - - [[ ${ENABLE_AMAVIS} -eq 1 ]] && _reload_amavis - _reload_postfix - [[ ${SMTP_ONLY} -ne 1 ]] && dovecot reload + _handle_changes _remove_lock _log_with_date 'debug' 'Completed handling of detected change' @@ -69,6 +59,29 @@ function _check_for_changes() { fi } +function _handle_changes() { + # Variable to identify any config updates dependent upon vhost changes. + local VHOST_UPDATED=0 + # These two configs are the source for /etc/postfix/vhost (managed mail domains) + if [[ ${CHANGED} =~ ${DMS_DIR}/postfix-(accounts|virtual).cf ]]; then + _log_with_date 'trace' 'Regenerating vhosts (Postfix)' + # Regenerate via `helpers/postfix.sh`: + _create_postfix_vhost + + VHOST_UPDATED=1 + fi + + _ssl_changes + _postfix_dovecot_changes + _rspamd_changes + + _log_with_date 'debug' 'Reloading services due to detected changes' + + [[ ${ENABLE_AMAVIS} -eq 1 ]] && _reload_amavis + _reload_postfix + [[ ${SMTP_ONLY} -ne 1 ]] && dovecot reload +} + function _get_changed_files() { local CHKSUM_CURRENT=${1} local CHKSUM_NEW=${2} @@ -85,9 +98,9 @@ function _get_changed_files() { } function _reload_amavis() { - if [[ ${CHANGED} =~ ${DMS_DIR}/postfix-accounts.cf ]] || [[ ${CHANGED} =~ ${DMS_DIR}/postfix-virtual.cf ]]; then - # /etc/postfix/vhost was updated, amavis must refresh it's config by - # reading this file again in case of new domains, otherwise they will be ignored. + # /etc/postfix/vhost was updated, amavis must refresh it's config by + # reading this file again in case of new domains, otherwise they will be ignored. + if [[ ${VHOST_UPDATED} -eq 1 ]]; then amavisd reload fi } @@ -114,14 +127,12 @@ function _postfix_dovecot_changes() { # - postfix-sasl-password.cf used by _relayhost_sasl # - _populate_relayhost_map relies on: # - postfix-relaymap.cf - # - postfix-accounts.cf + postfix-virtual.cf (both will be dropped in future) - if [[ ${CHANGED} =~ ${DMS_DIR}/postfix-accounts.cf ]] \ - || [[ ${CHANGED} =~ ${DMS_DIR}/postfix-virtual.cf ]] \ + if [[ ${VHOST_UPDATED} -eq 1 ]] \ || [[ ${CHANGED} =~ ${DMS_DIR}/postfix-relaymap.cf ]] \ || [[ ${CHANGED} =~ ${DMS_DIR}/postfix-sasl-password.cf ]] then _log_with_date 'trace' 'Regenerating relay config (Postfix)' - _rebuild_relayhost + _process_relayhost_configs fi # Regenerate system + virtual account aliases via `helpers/aliases.sh`: @@ -129,14 +140,6 @@ function _postfix_dovecot_changes() { [[ ${CHANGED} =~ ${DMS_DIR}/postfix-regexp.cf ]] && _handle_postfix_regexp_config [[ ${CHANGED} =~ ${DMS_DIR}/postfix-aliases.cf ]] && _handle_postfix_aliases_config - # Regenerate `/etc/postfix/vhost` (managed mail domains) via `helpers/postfix.sh`: - if [[ ${CHANGED} =~ ${DMS_DIR}/postfix-accounts.cf ]] \ - || [[ ${CHANGED} =~ ${DMS_DIR}/postfix-virtual.cf ]] - then - _log_with_date 'trace' 'Regenerating vhosts (Postfix)' - _create_postfix_vhost - fi - # Legacy workaround handled here, only seems necessary for _create_accounts: # - `helpers/accounts.sh` logic creates folders/files with wrong ownership. _chown_var_mail_if_necessary diff --git a/target/scripts/helpers/postfix.sh b/target/scripts/helpers/postfix.sh index 5fa4fa83..0bb1c158 100644 --- a/target/scripts/helpers/postfix.sh +++ b/target/scripts/helpers/postfix.sh @@ -43,21 +43,28 @@ function _vhost_collect_postfix_domains() { local DATABASE_VIRTUAL='/tmp/docker-mailserver/postfix-virtual.cf' local DOMAIN UNAME - # getting domains FROM mail accounts + # Extract domains from mail accounts: if [[ -f ${DATABASE_ACCOUNTS} ]]; then - while IFS=$'|' read -r LOGIN _; do - DOMAIN=$(echo "${LOGIN}" | cut -d @ -f2) + while IFS=$'|' read -r MAIL_ACCOUNT _; do + # It is expected valid lines have the format local-part@domain-part: + DOMAIN=$(cut -d '@' -f 2 <<< "${MAIL_ACCOUNT}") + echo "${DOMAIN}" >>"${TMP_VHOST}" done < <(_get_valid_lines_from_file "${DATABASE_ACCOUNTS}") fi - # getting domains FROM mail aliases + # TODO: Consider if virtual aliases should be configured to the same vhost file: + # https://github.com/docker-mailserver/docker-mailserver/issues/2813#issuecomment-1272394563 + # Extract domains from virtual alias config: + # Aliases may have the forms: 'local-part@domain-part', only 'local-part', or '@domain-part' (wildcard catch-all) if [[ -f ${DATABASE_VIRTUAL} ]]; then - while read -r FROM _; do - UNAME=$(echo "${FROM}" | cut -d @ -f1) - DOMAIN=$(echo "${FROM}" | cut -d @ -f2) + while read -r ALIAS_FIELD _; do + UNAME=$(cut -d '@' -f 1 <<< "${ALIAS_FIELD}") + DOMAIN=$(cut -d '@' -f 2 <<< "${ALIAS_FIELD}") - # if they are equal it means the line looks like: "user1 other@domain.tld" + # Only add valid domain-parts found: + # The '@' is optional for an alias key (eg: "user1 other@domain.tld"), + # but cut with -f2 would output the same value as it would -f1 when '@' is missing. [[ ${UNAME} != "${DOMAIN}" ]] && echo "${DOMAIN}" >>"${TMP_VHOST}" done < <(_get_valid_lines_from_file "${DATABASE_VIRTUAL}") fi diff --git a/target/scripts/helpers/relay.sh b/target/scripts/helpers/relay.sh index de29dddc..b92da3b4 100644 --- a/target/scripts/helpers/relay.sh +++ b/target/scripts/helpers/relay.sh @@ -4,9 +4,9 @@ # Description: # This helper is responsible for configuring outbound SMTP (delivery) through relay-hosts. # -# When mail is sent from Postfix, it is considered relaying to that destination (or the next hop). -# By default delivery external of the container would be direct to the MTA of the recipient address (destination). -# Alternatively mail can be indirectly delivered to the destination by routing through a different MTA (relay-host service). +# When mail is sent to Postfix and the destination is not a domain DMS manages, this requires relaying to that destination (or the next hop). +# By default outbound mail delivery would be direct to the MTA of the recipient address (destination). +# Alternatively mail can be delivered indirectly to that destination by routing through a different MTA (relay-host service). # # This helper is only concerned with relaying mail from authenticated submission (ports 587 + 465). # Thus it does not deal with `relay_domains` (which routes through `relay_transport` transport, default: `master.cf:relay`), @@ -37,22 +37,23 @@ # `postfix reload` or `supervisorctl restart postfix` should be run to properly apply config (which it is). # Otherwise use another table type such as `hash` and run `postmap` on the table after modification. # -# WARNING: Databases (tables above) are rebuilt during change detection. There is a minor chance of -# a lookup occurring during a rebuild of these files that may affect or delay delivery? +# WARNING: Databases (tables above) are rebuilt during change detection. +# There is a minor chance of a lookup occurring during a rebuild of these files that may affect or delay delivery? # TODO: Should instead perform an atomic operation with a temporary file + `mv` to replace? # Or switch back to using `hash` table type if plaintext access is not needed (unless retaining file for postmap). # Either way, plaintext copy is likely accessible if using our supported configs for providing them to the container. -# NOTE: Present support has enforced wrapping the relay host with `[]` (prevents DNS MX record lookup), -# which restricts what is supported by RELAY_HOST, although you usually do want to provide MX host directly. -# NOTE: Present support expects to always append a port with an implicit default of `25`. -# NOTE: DEFAULT_RELAY_HOST imposes neither restriction. +# NOTE: Present support has enforced wrapping the `RELAY_HOST` value with `[]` (prevents DNS MX record lookup), +# shouldn't be an issue as you typically do want to provide the MX host directly? This was presumably for config convenience. +# NOTE: Present support expects to always append a port (_with an implicit default of `25`_). +# NOTE: The `DEFAULT_RELAY_HOST` ENV imposes neither restriction. # -# TODO: RELAY_PORT should be optional, it will use the transport default port (`postconf smtp_tcp_port`), +# TODO: `RELAY_PORT` should be optional (Postfix would fallback to the transports default port (`postconf smtp_tcp_port`), # That shouldn't be a breaking change, as long as the mapping is maintained correctly. -# TODO: RELAY_HOST should consider dropping `[]` and require the user to include that? -# Future refactor for _populate_relayhost_map may warrant dropping these two ENV in favor of DEFAULT_RELAY_HOST? +# TODO: `RELAY_HOST` should consider dropping the implicit `[]` and require the user to include that? +# +# A future refactor of `_populate_relayhost_map()` may warrant dropping those two ENV in favor of `DEFAULT_RELAY_HOST`? function _env_relay_host() { echo "[${RELAY_HOST}]:${RELAY_PORT:-25}" } @@ -60,10 +61,12 @@ function _env_relay_host() { # Responsible for `postfix-sasl-password.cf` support: # `/etc/postfix/sasl_passwd` example at end of file. function _relayhost_sasl() { - if [[ ! -f /tmp/docker-mailserver/postfix-sasl-password.cf ]] \ - && [[ -z ${RELAY_USER} || -z ${RELAY_PASSWORD} ]] - then - _log 'warn' "Missing relay-host mapped credentials provided via ENV, or from postfix-sasl-password.cf" + local DATABASE_SASL_PASSWD='/tmp/docker-mailserver/postfix-sasl-password.cf' + + # Only relevant when required credential sources are provided: + if [[ ! -f ${DATABASE_SASL_PASSWD} ]] \ + && [[ -z ${RELAY_USER} || -z ${RELAY_PASSWORD} ]]; then + _log 'warn' "Missing relay-host mapped credentials provided via ENV, or from ${DATABASE_SASL_PASSWD}" return 1 fi @@ -74,7 +77,6 @@ function _relayhost_sasl() { chown root:root /etc/postfix/sasl_passwd chmod 0600 /etc/postfix/sasl_passwd - local DATABASE_SASL_PASSWD='/tmp/docker-mailserver/postfix-sasl-password.cf' if [[ -f ${DATABASE_SASL_PASSWD} ]]; then # Add domain-specific auth from config file: _get_valid_lines_from_file "${DATABASE_SASL_PASSWD}" >> /etc/postfix/sasl_passwd @@ -83,90 +85,93 @@ function _relayhost_sasl() { postconf 'smtp_sender_dependent_authentication = yes' fi - # Add an authenticated relay host defined via ENV config: - if [[ -n ${RELAY_USER} ]] && [[ -n ${RELAY_PASSWORD} ]]; then - echo "$(_env_relay_host) ${RELAY_USER}:${RELAY_PASSWORD}" >> /etc/postfix/sasl_passwd + # Support authentication to a primary relayhost (when configured with credentials via ENV): + if [[ -n ${DEFAULT_RELAY_HOST} || -n ${RELAY_HOST} ]] \ + && [[ -n ${RELAY_USER} && -n ${RELAY_PASSWORD} ]]; then + echo "${DEFAULT_RELAY_HOST:-$(_env_relay_host)} ${RELAY_USER}:${RELAY_PASSWORD}" >>/etc/postfix/sasl_passwd fi - # Technically if only a single relay host is configured, a `static` lookup table could be used instead?: - # postconf "smtp_sasl_password_maps = static:${RELAY_USER}:${RELAY_PASSWORD}" - postconf 'smtp_sasl_password_maps = texthash:/etc/postfix/sasl_passwd' + # Enable credential lookup + SASL authentication to relayhost: + # - `noanonymous` enforces authentication requirement + # - `encrypt` enforces requirement for a secure connection (prevents sending credentials over cleartext, aka mandatory TLS) + postconf \ + 'smtp_sasl_password_maps = texthash:/etc/postfix/sasl_passwd' \ + 'smtp_sasl_auth_enable = yes' \ + 'smtp_sasl_security_options = noanonymous' \ + 'smtp_tls_security_level = encrypt' } # Responsible for `postfix-relaymap.cf` support: # `/etc/postfix/relayhost_map` example at end of file. # -# Present support uses a table lookup for sender address or domain mapping to relay-hosts, -# Populated via `postfix-relaymap.cf `, which also features a non-standard way to exclude implicitly added internal domains from the feature. -# It also maps all known sender domains (from configs postfix-accounts + postfix-virtual.cf) to the same ENV configured relay-host. +# `postfix-relaymap.cf` represents table syntax expected for `/etc/postfix/relayhost_map`, except that it adds an opt-out parsing feature. +# All known mail domains managed by DMS (/etc/postfix/vhost) are implicitly configured to use `RELAY_HOST` + `RELAY_PORT` as the default relay. +# This approach is effectively equivalent to using `main.cf:relayhost`, but with an excessive workaround to support the explicit opt-out feature. # -# TODO: The account + virtual config parsing and appending to /etc/postfix/relayhost_map seems to be an excessive `main.cf:relayhost` -# implementation, rather than leveraging that for the same purpose and selectively overriding only when needed with `/etc/postfix/relayhost_map`. -# If the issue was to opt-out select domains, if avoiding a default relay-host was not an option, then mapping those sender domains or addresses -# to a separate transport (which can drop the `relayhost` setting) would be more appropriate. -# TODO: With `sender_dependent_default_transport_maps`, we can extract out the excluded domains and route them through a separate transport. -# while deprecating that support in favor of a transport config, similar to what is offered currently via sasl_passwd and relayhost_map. +# TODO: Refactor this feature support so that in `main.cf`: +# - Relay all outbound mail through an external MTA by default (works without credentials): +# `relayhost = ${DEFAULT_RELAY_HOST}` +# - Opt-in to relaying - Selectively relay outbound mail by sender/domain to an external MTA (relayhost can vary): +# `sender_dependent_relayhost_maps = texthash:/etc/postfix/relayhost_map` +# - Opt-out from relaying - Selectively prevent outbound mail from relaying via separate transport mappings (where relayhost is not configured): +# By sender: `sender_dependent_default_transport_maps = texthash:/etc/postfix/sender_transport_map` (the current opt-out feature could utilize this instead) +# By recipient (has precedence): `transport_maps = texthash:/etc/postfix/recipient_transport_map` +# +# Support for relaying via port 465 or equivalent requires additional config support (as needed for 465 vs 587 transports extending smtpd) +# - Default relay transport is configured by `relay_transport`, with default transport port configured by `smtp_tcp_port`. +# - The `relay` transport itself extends from `smtp` transport. More than one can be configured with separate settings via `master.cf`. + function _populate_relayhost_map() { # Create the relayhost_map config file: : >/etc/postfix/relayhost_map chown root:root /etc/postfix/relayhost_map chmod 0600 /etc/postfix/relayhost_map - # Matches lines that are not comments or only white-space: - local MATCH_VALID='^\s*[^#[:space:]]' - - # This config is mostly compatible with `/etc/postfix/relayhost_map`, but additionally supports - # not providing a relay host for a sender domain to opt-out of RELAY_HOST? (2nd half of function) - if [[ -f /tmp/docker-mailserver/postfix-relaymap.cf ]]; then - _log 'trace' "Adding relay mappings from postfix-relaymap.cf" - - # Match two values with some white-space between them (eg: `@example.test [relay.service.test]:465`): - local MATCH_VALUE_PAIR='\S*\s+\S' - - # Copy over lines which are not a comment *and* have a destination. - sed -n -r "/${MATCH_VALID}${MATCH_VALUE_PAIR}/p" /tmp/docker-mailserver/postfix-relaymap.cf >>/etc/postfix/relayhost_map - fi - - # Everything below here is to parse `postfix-accounts.cf` and `postfix-virtual.cf`, - # extracting out the domain parts (value of email address after `@`), and then - # adding those as mappings to ENV configured RELAY_HOST for lookup in `/etc/postfix/relayhost_map`. - # Provided `postfix-relaymap.cf` didn't exclude any of the domains, - # and they don't already exist within `/etc/postfix/relayhost_map`. - # - # TODO: Breaking change. Replace this lower half and remove the opt-out feature from `postfix-relaymap.cf`. - # Leverage `main.cf:relayhost` for setting a default relayhost as it was prior to this feature addition. - # Any sender domains or addresses that need to opt-out of that default relay-host can either - # map to a different relay-host, or use a separate transport (needs feature support added). - - # Args: - function _list_domain_parts() { - [[ -f $2 ]] && sed -n -r "/${MATCH_VALID}/ ${1}" "${2}" - } - # Matches and outputs (capture group via `/\1/p`) the domain part (value of address after `@`) in the config file. - local PRINT_DOMAIN_PART_ACCOUNTS='s/^[^@|]*@([^\|]+)\|.*$/\1/p' - local PRINT_DOMAIN_PART_VIRTUAL='s/^\s*[^@[:space:]]*@(\S+)\s.*/\1/p' - - { - _list_domain_parts "${PRINT_DOMAIN_PART_ACCOUNTS}" /tmp/docker-mailserver/postfix-accounts.cf - _list_domain_parts "${PRINT_DOMAIN_PART_VIRTUAL}" /tmp/docker-mailserver/postfix-virtual.cf - } | sort -u | while read -r DOMAIN_PART; do - # DOMAIN_PART not already present in `/etc/postfix/relayhost_map`, and not listed as a relay opt-out domain in `postfix-relaymap.cf` - # `^@${DOMAIN_PART}\b` - To check for existing entry, the `\b` avoids accidental partial matches on similar domain parts. - # `^\s*@${DOMAIN_PART}\s*$` - Matches line with only a domain part (eg: @example.test) to avoid including a mapping for those domains to the RELAY_HOST. - if ! grep -q -e "^@${DOMAIN_PART}\b" /etc/postfix/relayhost_map && ! grep -qs -e "^\s*@${DOMAIN_PART}\s*$" /tmp/docker-mailserver/postfix-relaymap.cf; then - _log 'trace' "Adding relay mapping for ${DOMAIN_PART}" - echo "@${DOMAIN_PART} $(_env_relay_host)" >> /etc/postfix/relayhost_map - fi - done + _multiple_relayhosts + _legacy_support postconf 'sender_dependent_relayhost_maps = texthash:/etc/postfix/relayhost_map' } -function _relayhost_configure_postfix() { - postconf \ - 'smtp_sasl_auth_enable = yes' \ - 'smtp_sasl_security_options = noanonymous' \ - 'smtp_tls_security_level = encrypt' +function _multiple_relayhosts() { + if [[ -f ${DATABASE_RELAYHOSTS} ]]; then + _log 'trace' "Adding relay mappings from ${DATABASE_RELAYHOSTS}" + + # Matches lines that are not comments or only white-space: + local MATCH_VALID='^\s*[^#[:space:]]' + # Match two values with some white-space between them (eg: `@example.test [relay.service.test]:465`): + local MATCH_VALUE_PAIR='\S*\s+\S' + + # Copy over lines which are not a comment *and* have a relay destination. + # Extra condition is due to legacy support (due to opt-out feature), otherwise `_get_valid_lines_from_file()` would be valid. + sed -n -r "/${MATCH_VALID}${MATCH_VALUE_PAIR}/p" "${DATABASE_RELAYHOSTS}" >> /etc/postfix/relayhost_map + fi +} + +# Implicitly force configure all domains DMS manages to be relayed that haven't yet been configured or provided an explicit opt-out. +# This would normally be handled via an opt-in approach, or through `main.cf:relayhost` with an opt-out approach (sender_dependent_default_transport_maps) +function _legacy_support() { + local DATABASE_VHOST='/etc/postfix/vhost' + + # Only relevant when `RELAY_HOST` is configured: + [[ -z ${RELAY_HOST} ]] && return 1 + + # Configures each `SENDER_DOMAIN` to send outbound mail through the default `RELAY_HOST` + `RELAY_PORT` + # (by adding an entry in `/etc/postfix/relayhost_map`) provided it: + # - `/etc/postfix/relayhost_map` doesn't already have it as an existing entry. + # - `postfix-relaymap.cf` has no explicit opt-out (SENDER_DOMAIN key exists, but with no relayhost value assigned) + # + # NOTE: /etc/postfix/vhost represents managed mail domains sourced from `postfix-accounts.cf` and `postfix-virtual.cf`. + while read -r SENDER_DOMAIN; do + local MATCH_EXISTING_ENTRY="^@${SENDER_DOMAIN}\s+" + local MATCH_OPT_OUT_LINE="^\s*@${SENDER_DOMAIN}\s*$" + + # NOTE: `-E` is required for `\s+` syntax to avoid escaping `+` + if ! grep -q -E "${MATCH_EXISTING_ENTRY}" /etc/postfix/relayhost_map && ! grep -qs "${MATCH_OPT_OUT_LINE}" "${DATABASE_RELAYHOSTS}"; then + _log 'trace' "Configuring '${SENDER_DOMAIN}' for the default relayhost '${RELAY_HOST}'" + echo "@${SENDER_DOMAIN} $(_env_relay_host)" >> /etc/postfix/relayhost_map + fi + done < <(_get_valid_lines_from_file "${DATABASE_VHOST}") } function _setup_relayhost() { @@ -177,23 +182,23 @@ function _setup_relayhost() { postconf "relayhost = ${DEFAULT_RELAY_HOST}" fi - if [[ -n ${RELAY_HOST} ]]; then - _log 'trace' "Setting up relay hosts (default: ${RELAY_HOST})" - - _relayhost_sasl - _populate_relayhost_map - - _relayhost_configure_postfix - fi + _process_relayhost_configs } -function _rebuild_relayhost() { - if [[ -n ${RELAY_HOST} ]]; then - _relayhost_sasl - _populate_relayhost_map - fi -} +# Called during initial container setup, or by change detection event: +function _process_relayhost_configs() { + local DATABASE_RELAYHOSTS='/tmp/docker-mailserver/postfix-relaymap.cf' + # One of these must configure a relayhost for the feature to relevant: + if [[ ! -f ${DATABASE_RELAYHOSTS} ]] \ + && [[ -z ${DEFAULT_RELAY_HOST} ]] \ + && [[ -z ${RELAY_HOST} ]]; then + return 1 + fi + + _relayhost_sasl + _populate_relayhost_map +} # # Config examples for reference