From 22555347edeccd237fd1327a9cb09426b71e9679 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Tue, 13 Feb 2024 19:42:17 +1300 Subject: [PATCH 01/53] docs: Complete rewrite of PROXY protocol guide (#3882) --- .../tutorials/mailserver-behind-proxy.md | 466 ++++++++++++++---- 1 file changed, 375 insertions(+), 91 deletions(-) diff --git a/docs/content/examples/tutorials/mailserver-behind-proxy.md b/docs/content/examples/tutorials/mailserver-behind-proxy.md index 99939542..47116fde 100644 --- a/docs/content/examples/tutorials/mailserver-behind-proxy.md +++ b/docs/content/examples/tutorials/mailserver-behind-proxy.md @@ -2,126 +2,410 @@ title: 'Tutorials | Mail Server behind a Proxy' --- -## Using DMS behind a Proxy +## Using a Reverse Proxy -### Information +Guidance is provided via a Traefik config example, however if you're only familiar with configuring a reverse proxy for web services there are some differences to keep in mind. -If you are hiding your container behind a proxy service you might have discovered that the proxied requests from now on contain the proxy IP as the request origin. Whilst this behavior is technical correct it produces certain problems on the containers behind the proxy as they cannot distinguish the real origin of the requests anymore. +- A security concern where preserving the client IP is important but needs to be handled at Layer 4 (TCP). +- TLS will be handled differently due protocols like STARTTLS and the need to comply with standards for interoperability with other MTAs. +- The ability to route the same port to different containers by FQDN can be limited. -To solve this problem on TCP connections we can make use of the [proxy protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt). Compared to other workarounds that exist (`X-Forwarded-For` which only works for HTTP requests or `Tproxy` that requires you to recompile your kernel) the proxy protocol: +This reduces many of the benefits for why you might use a reverse proxy, but they can still be useful. -- It is protocol agnostic (can work with any layer 7 protocols, even when encrypted). -- It does not require any infrastructure changes. -- NAT-ing firewalls have no impact it. -- It is scalable. +Some deployments may require a service to route traffic (kubernetes) when deploying, in which case the below advice is important to understand well. -There is only one condition: **both endpoints** of the connection MUST be compatible with proxy protocol. +## What can go wrong? -Luckily `dovecot` and `postfix` are both Proxy-Protocol ready softwares so it depends only on your used reverse-proxy / loadbalancer. +Without a reverse proxy involved, a service is typically aware of the client IP for a connection. -### Configuration of the used Proxy Software +However when a reverse proxy routes the connection this information can be lost, and the proxied service mistakenly treats the client IP as the reverse proxy handling the connection. -The configuration depends on the used proxy system. I will provide the configuration examples of [traefik v2](https://traefik.io/) using IMAP and SMTP with implicit TLS. +- That can be problematic when the client IP is meaningful information for the proxied service to act upon, especially when it [impacts security](#security-concerns). +- The [PROXY protocol][networking::spec:proxy-protocol] is a well established solution to preserve the client IP when both the proxy and service have enabled the support. -Feel free to add your configuration if you achieved the same goal using different proxy software below: +??? abstract "Technical Details - HTTP vs TCP proxying" -??? "Traefik v2" + A key difference for how the network is proxied relates to the [OSI Model][networking::osi-model]: - Truncated configuration of traefik itself: + - Layer 7 (_Application layer protocols: SMTP / IMAP / HTTP / etc_) + - Layer 4 (_Transport layer protocols: TCP / UDP_) - ```yaml + When working with Layer 7 and a protocol like HTTP, it is possible to inspect a protocol header like [`Forwarded`][networking::http-header::forwarded] (_or it's predecessor: [`X-Forwarded-For`][networking::http-header::x-forwarded-for]_). At a lower level with Layer 4, that information is not available and we are routing traffic agnostic to the application protocol being proxied. + + A proxy can prepend the [PROXY protocol][networking::spec:proxy-protocol] header to the TCP/UDP connection as it is routed to the service, which must be configured to be compatible with PROXY protocol (_often this adds a restriction that connections must provide the header, otherwise they're rejected_). + + Beyond your own proxy, traffic may be routed in the network by other means that would also rewrite this information such as Docker's own network management via `iptables` and `userland-proxy` (NAT). The PROXY header ensures the original source and destination IP addresses, along with their ports is preserved across transit. + +## Configuration + +### Reverse Proxy + +The below guidance is focused on configuring [Traefik][traefik-web], but the advice should be roughly applicable elsewhere (_eg: [NGINX][nginx-docs::proxyprotocol], [Caddy][caddy::plugin::l4]_). + +- Support requires the capability to proxy TCP (Layer 4) connections with PROXY protocol enabled for the upstream (DMS). The upstream must also support enabling PROXY protocol (_which for DMS services rejects any connection not using the protocol_). +- TLS should not be terminated at the proxy, that should be delegated to DMS (_which should be configured with the TLS certs_). Reasoning is covered under the [ports section](#ports). + +???+ example "Traefik service" + + The Traefik service config is fairly standard, just define the necessary entrypoints: + + ```yaml title="compose.yaml" services: reverse-proxy: - image: docker.io/traefik:latest # v2.5 - container_name: docker-traefik - restart: always + image: docker.io/traefik:latest # 2.10 / 3.0 + # CAUTION: In production you should configure the Docker API endpoint securely: + # https://doc.traefik.io/traefik/providers/docker/#docker-api-access + volumes: + - /var/run/docker.sock:/var/run/docker.sock command: - - "--providers.docker" - - "--providers.docker.exposedbydefault=false" - - "--providers.docker.network=proxy" - - "--entrypoints.web.address=:80" - - "--entryPoints.websecure.address=:443" - - "--entryPoints.smtp.address=:25" - - "--entryPoints.smtp-ssl.address=:465" - - "--entryPoints.imap-ssl.address=:993" - - "--entryPoints.sieve.address=:4190" + # Docker provider config: + - --providers.docker=true + - --providers.docker.exposedbydefault=false + # DMS ports you want to proxy: + - --entryPoints.mail-smtp.address=:25 + - --entryPoints.mail-submission.address=:587 + - --entryPoints.mail-submissions.address=:465 + - --entryPoints.mail-imap.address=:143 + - --entryPoints.mail-imaps.address=:993 + - --entryPoints.mail-pop3.address=:110 + - --entryPoints.mail-pop3s.address=:995 + - --entryPoints.mail-managesieve.address=:4190 + # Publish external access ports mapped to traefik entrypoint ports: ports: - "25:25" + - "587:587" - "465:465" + - "143:143" - "993:993" + - "110:110" + - "995:995" - "4190:4190" - [...] - ``` - - Truncated list of necessary labels on the DMS container: - - ```yaml - services: - mailserver: - image: ghcr.io/docker-mailserver/docker-mailserver:latest - container_name: mailserver - hostname: mail.example.com - restart: always + # An IP is assigned here for other services (Dovecot) to trust for PROXY protocol: networks: - - proxy - labels: - - "traefik.enable=true" - - "traefik.tcp.routers.smtp.rule=HostSNI(`*`)" - - "traefik.tcp.routers.smtp.entrypoints=smtp" - - "traefik.tcp.routers.smtp.service=smtp" - - "traefik.tcp.services.smtp.loadbalancer.server.port=25" - - "traefik.tcp.services.smtp.loadbalancer.proxyProtocol.version=1" - - "traefik.tcp.routers.smtp-ssl.rule=HostSNI(`*`)" - - "traefik.tcp.routers.smtp-ssl.entrypoints=smtp-ssl" - - "traefik.tcp.routers.smtp-ssl.tls.passthrough=true" - - "traefik.tcp.routers.smtp-ssl.service=smtp-ssl" - - "traefik.tcp.services.smtp-ssl.loadbalancer.server.port=465" - - "traefik.tcp.services.smtp-ssl.loadbalancer.proxyProtocol.version=1" - - "traefik.tcp.routers.imap-ssl.rule=HostSNI(`*`)" - - "traefik.tcp.routers.imap-ssl.entrypoints=imap-ssl" - - "traefik.tcp.routers.imap-ssl.service=imap-ssl" - - "traefik.tcp.routers.imap-ssl.tls.passthrough=true" - - "traefik.tcp.services.imap-ssl.loadbalancer.server.port=10993" - - "traefik.tcp.services.imap-ssl.loadbalancer.proxyProtocol.version=2" - - "traefik.tcp.routers.sieve.rule=HostSNI(`*`)" - - "traefik.tcp.routers.sieve.entrypoints=sieve" - - "traefik.tcp.routers.sieve.service=sieve" - - "traefik.tcp.services.sieve.loadbalancer.server.port=4190" - [...] + default: + ipv4_address: 172.16.42.2 + + # Specifying a subnet to assign a fixed container IP to the reverse proxy: + networks: + default: + name: my-network + ipam: + config: + - subnet: "172.16.42.0/24" ``` - Keep in mind that it is necessary to use port `10993` here. More information below at `dovecot` configuration. + !!! note "Extra considerations" -### Configuration of the Backend (`dovecot` and `postfix`) + - [`--providers.docker.network=my-network`][traefik-docs::provider-docker::network] is useful when there is more than one network to consider. + - If your deployment has any other hops (an edge proxy, load balancer, etc) between the reverse proxy and the client, you'll need PROXY protocol support throughout that chain. For Traefik this additionally requires [enabling PROXY protocol on your entry points][traefik-docs::entrypoint::proxyprotocol]. -The following changes can be achieved completely by adding the content to the appropriate files by using the projects [function to overwrite config files][docs-optionalconfig]. +???+ example "Traefik labels for DMS" -Changes for `postfix` can be applied by adding the following content to `docker-data/dms/config/postfix-main.cf`: + ```yaml title="compose.yaml" + services: + dms: + image: ghcr.io/docker-mailserver/docker-mailserver:latest + hostname: mail.example.com + labels: + - traefik.enable=true -```cf -postscreen_upstream_proxy_protocol = haproxy -``` + # These are examples, configure the equivalent for any additional ports you proxy. + # Explicit TLS (STARTTLS): + - traefik.tcp.routers.mail-smtp.rule=HostSNI(`*`) + - traefik.tcp.routers.mail-smtp.entrypoints=smtp + - traefik.tcp.routers.mail-smtp.service=smtp + - traefik.tcp.services.mail-smtp.loadbalancer.server.port=25 + - traefik.tcp.services.mail-smtp.loadbalancer.proxyProtocol.version=2 -and to `docker-data/dms/config/postfix-master.cf`: + # Implicit TLS is no different, except for optional HostSNI support: + - traefik.tcp.routers.mail-submissions.rule=HostSNI(`*`) + - traefik.tcp.routers.mail-submissions.entrypoints=smtp-submissions + - traefik.tcp.routers.mail-submissions.service=smtp-submissions + - traefik.tcp.services.mail-submissions.loadbalancer.server.port=465 + - traefik.tcp.services.mail-submissions.loadbalancer.proxyProtocol.version=2 + # NOTE: Optionally match by SNI rule, this requires TLS passthrough (not compatible with STARTTLS): + #- traefik.tcp.routers.mail-submissions.rule=HostSNI(`mail.example.com`) + #- traefik.tcp.routers.mail-submissions.tls.passthrough=true + ``` -```cf -submission/inet/smtpd_upstream_proxy_protocol=haproxy -submissions/inet/smtpd_upstream_proxy_protocol=haproxy -``` + !!! note "PROXY protocol compatibility" -Changes for `dovecot` can be applied by adding the following content to `docker-data/dms/config/dovecot.cf`: + Only TCP routers support enabling PROXY Protocol (via [`proxyProtocol.version=2`][traefik-docs::service-tcp::proxyprotocol]) -```cf -haproxy_trusted_networks = , -haproxy_timeout = 3 secs -service imap-login { - inet_listener imaps { - haproxy = yes - ssl = yes - port = 10993 - } -} -``` + Postfix and Dovecot are both compatible with PROXY protocol v1 and v2. -!!! note - Port `10993` is used here to avoid conflicts with internal systems like `postscreen` and `amavis` as they will exchange messages on the default port and obviously have a different origin then compared to the proxy. +??? abstract "Technical Details - Ports (Traefik config)" + + !!! info "Explicit TLS (STARTTLS)" + + **Service Ports:** `mail-smtp` (25), `mail-submission` (587), `mail-imap` (143), `mail-pop3` (110), `mail-managesieve` (4190) + + --- + + - [Traefik expects the TCP router to not enable TLS][traefik-docs::router-tcp::server-first-protocols] (_see "Server First protocols"_) for these connections. They begin in plaintext and potentially upgrade the connection to TLS, Traefik has no involvement in STARTTLS. + - Without an initial TLS connection, the [`HostSNI` router rule is not usable][traefik-docs::router-tcp::host-sni] (_see "HostSNI & TLS"_). This limits routing flexibility for these ports (_eg: routing these ports by the FQDN to different DMS containers_). + + !!! info "Implicit TLS" + + **Service Ports:** `mail-submissions` (465), `mail-imaps` (993), `mail-pop3s` (995) + + --- + + The `HostSNI` router rule could specify the DMS FQDN instead of `*`: + + - This requires the router to have TLS enabled, so that Traefik can inspect the server name sent by the client. + - Traefik can only match the SNI to `*` when the client does not provide a server name. Some clients must explicitly opt-in, such as CLI clients `openssl` (`-servername`) and `swaks` (`--tls-sni`). + - Add [`tls.passthrough=true` to the router][traefik-docs::router-tcp::passthrough] (_this implicitly enables TLS_). + - Traefik should not terminate TLS, decryption should occur within DMS instead when proxying to the same implicit TLS ports. + - Passthrough ignores any certificates configured for Traefik; DMS must be configured with the certificates instead (_[DMS can use `acme.json` from Traefik][docs::tls::traefik]_). + + Unlike proxying HTTPS (port 443) to a container via HTTP (port 80), the equivalent for DMS service ports is not supported: + + - Port 25 must secure the connection via STARTTLS to be reached publicly. + - STARTTLS ports requiring authentication for Postfix (587) and Dovecot (110, 143, 4190) are configured to only permit authentication over an encrypted connection. + - Support would require routing the implicit TLS ports to their explicit TLS equivalent ports with auth restrictions removed. `tls.passthrough.true` would not be required, additionally port 25 would always be unencrypted (_if the proxy exclusively manages TLS/certs_), or unreachable by public MTAs attempting delivery if the proxy enables implicit TLS for this port. + +### DMS (Postfix + Dovecot) + +???+ example "Enable PROXY protocol on existing service ports" + + This can be handled via our config override support. + + --- + + Postfix via [`postfix-master.cf`][docs::overrides::postfix]: + + ```cf title="docker-data/dms/config/postfix-master.cf" + smtp/inet/postscreen_upstream_proxy_protocol=haproxy + submission/inet/smtpd_upstream_proxy_protocol=haproxy + submissions/inet/smtpd_upstream_proxy_protocol=haproxy + ``` + + [`postscreen_upstream_proxy_protocol`][postfix-docs::settings::postscreen_upstream_proxy_protocol] and [`smtpd_upstream_proxy_protocol`][postfix-docs::settings::smtpd_upstream_proxy_protocol] both specify the protocol type used by a proxy. `haproxy` represents the PROXY protocol. + + --- + + Dovecot via [`dovecot.cf`][docs::overrides::dovecot]: + + ```cf title="docker-data/dms/config/dovecot.cf" + haproxy_trusted_networks = 172.16.42.2 + + service imap-login { + inet_listener imap { + haproxy = yes + } + + inet_listener imaps { + haproxy = yes + } + } + + service pop3-login { + inet_listener pop3 { + haproxy = yes + } + + inet_listener pop3s { + haproxy = yes + } + } + + service managesieve-login { + inet_listener sieve { + haproxy = yes + } + } + ``` + + - [`haproxy_trusted_networks`][dovecot-docs::settings::haproxy-trusted-networks] must reference the reverse proxy IP, or a wider subnet using CIDR notation. + - [`haproxy = yes`][dovecot-docs::service-config::haproxy] for the TCP listeners of each login service. + +!!! warning "Internal traffic (_within the network or DMS itself_)" + + - Direct connections to DMS from other containers within the internal network will be rejected when they don't provide the required PROXY header. + - This can also affect services running within the DMS container itself if they attempt to make a connection and aren't PROXY protocol capable. + + --- + + A solution is to configure alternative service ports that offer PROXY protocol support (as shown next). + + Alternatively routing connections to DMS through the local reverse proxy via [DNS query rewriting][gh-dms::dns-rewrite-example] can work too. + +??? example "Configuring services with separate ports for PROXY protocol" + + In this example we'll take the original service ports and add `10000` for the new PROXY protocol service ports. + + Traefik labels will need to update their service ports accordingly (eg: `.loadbalancer.server.port=10465`). + + --- + + Postfix config now requires [our `user-patches.sh` support][docs::overrides::user-patches] to add new services in `/etc/postfix/master.cf`: + + ```bash title="docker-data/dms/config/user-patches.sh" + #!/bin/bash + + # Duplicate the config for the submission(s) service ports (587 / 465) with adjustments for the PROXY ports (10587 / 10465) and `syslog_name` setting: + postconf -Mf submission/inet | sed -e s/^submission/10587/ -e 's/submission/submission-proxyprotocol/' >> /etc/postfix/master.cf + postconf -Mf submissions/inet | sed -e s/^submissions/10465/ -e 's/submissions/submissions-proxyprotocol/' >> /etc/postfix/master.cf + # Enable PROXY Protocol support for these new service variants: + postconf -P 10587/inet/smtpd_upstream_proxy_protocol=haproxy + postconf -P 10465/inet/smtpd_upstream_proxy_protocol=haproxy + + # Create a variant for port 25 too (NOTE: Port 10025 is already assigned in DMS to Amavis): + postconf -Mf smtp/inet | sed -e s/^smtp/12525/ >> /etc/postfix/master.cf + # Enable PROXY Protocol support (different setting as port 25 is handled via postscreen), optionally configure a `syslog_name` to distinguish in logs: + postconf -P 12525/inet/postscreen_upstream_proxy_protocol=haproxy 12525/inet/syslog_name=smtp-proxyprotocol + ``` + + --- + + Dovecot is mostly the same as before: + + - A new service name instead of targeting one to modify. + - Add the new port assignment. + - Set [`ssl = yes`][dovecot-docs::service-config::ssl] when implicit TLS is needed. + + ```cf title="docker-data/dms/config/dovecot.cf" + haproxy_trusted_networks = 172.16.42.2 + + service imap-login { + inet_listener imap-proxied { + haproxy = yes + port = 10143 + } + + inet_listener imaps-proxied { + haproxy = yes + port = 10993 + ssl = yes + } + } + + service pop3-login { + inet_listener pop3-proxied { + haproxy = yes + port = 10110 + } + + inet_listener pop3s-proxied { + haproxy = yes + port = 10995 + ssl = yes + } + } + + service managesieve-login { + inet_listener sieve-proxied { + haproxy = yes + port = 14190 + } + } + ``` + +## Verification + +Send an email through the reverse proxy. If you do not use the DNS query rewriting approach, you'll need to do this from an external client. + +??? example "Sending a generic test mail through `swaks` CLI" + + Run a `swaks` command and then check your DMS logs for the expected client IP, it should no longer be using the reverse proxy IP. + + ```bash + # NOTE: It is common to find port 25 is blocked from outbound connections, you may only be able to test the submission(s) ports. + swaks --helo not-relevant.test --server mail.example.com --port 25 -tls --from hello@not-relevant.test --to user@example.com + ``` + + - You can specify the `--server` as the DMS FQDN or an IP address, where either should connect to the reverse proxy service. + - `not-relevant.test` technically may be subject to some tests, at least for port 25. With the submission(s) ports those should be exempt. + - `-tls` will use STARTTLS on port 25, you can exclude it to send unencrypted, but it would still go through the same port/route being tested. + - To test the submission ports use `--port 587 -tls` or `--port 465 -tlsc` with your credentials `--auth-user user@example.com --auth-password secret` + - Add `--tls-sni mail.example.com` if you have configured `HostSNI` in Traefik router rules (_SNI routing is only valid for implicit TLS ports_). + +??? warning "Do not rely on local testing alone" + + Testing from the Docker host technically works, however the IP is likely subject to more manipulation via `iptables` than an external client. + + The IP will likely appear as from the gateway IP of the Docker network associated to the reverse proxy, where that gateway IP then becomes the client IP when writing the PROXY protocol header. + +## Security concerns + +### Forgery + +Since the PROXY protocol sends a header with the client IP rewritten for software to use instead, this could be abused by bad actors. + +Software on the receiving end of the connection often supports configuring an IP or CIDR range of clients to trust receiving the PROXY protocol header from. + +??? warning "Risk exposure" + + If you trust more than the reverse proxy IP, you must consider the risk exposure: + + - Any container within the network that is compromised could impersonate another IP (_container or external client_) which may have been configured to have additional access/exceptions granted. + - If the reverse proxy is on a separate network/host than DMS, exposure of the PROXY protocol enabled ports outside the network increases the importance of narrowing trust. For example with the [known IPv6 to subnet Gateway IP routing gotcha][docs::ipv6::security-risks] in Docker, trusting the entire subnet DMS belongs to would wrongly trust external clients that have the subnet Gateway IP to impersonate any client IP. + - There is a [known risk with Layer 2 switching][docker::networking::l2-switch-gotcha] (_applicable to VPC networks, impact varies by cloud vendor_): + - Neighbouring hosts can indirectly route to ports published on the interfaces of a separate host system that shouldn't be reachable (_eg: localhost `127.0.0.1`, or a private subnet `172.16.0.0/12`_). + - The scope of this in Docker is limited to published ports only when Docker uses `iptables` with the kernel tunable `sysctl net.ipv4.ip_forward=1` (enabled implicitly). Port access is via `HOST:CONTAINER` ports published to their respective interface(s), that includes the container IP + port. + + While some concerns raised above are rather specific, these type of issues aren't exclusive to Docker and difficult to keep on top of as software is constantly changing. Limit the trusted networks where possible. + +??? warning "Postfix has no concept of trusted proxies" + + Postfix does not appear to have a way to configure trusted proxies like Dovecot does (`haproxy_trusted_networks`). + + [`postscreen_access_list`][postfix-docs::settings::postscreen_access_list] (_or [`smtpd_client_restrictions`][postfix-docs::settings::smtpd_client_restrictions] with [`check_client_access`][postfix-docs::settings::check_client_access] for ports 587/465_) can both restrict access by IP via a [CIDR lookup table][postfix-docs::config-table::cidr], however the client IP is already rewritten at this point via PROXY protocol. + + Thus those settings cannot be used for restricting access to only trusted proxies, only to the actual clients. + + A similar setting [`mynetworks`][postfix-docs::settings::mynetworks] / [`PERMIT_DOCKER`][docs::env::permit_docker] manages elevated trust for bypassing security restrictions. While it is intended for trusted clients, it has no relevance to trusting proxies for the same reasons. + + +### Monitoring + +While PROXY protocol works well with the reverse proxy, you may have some containers internally that interact with DMS on behalf of multiple clients. + +??? example "Roundcube + Fail2Ban" + + You may have other services with functionality like an API to send mail through DMS that likewise delegates credentials through DMS. + + Roundcube is an example of this where authentication is delegated to DMS, which introduces the same concern with loss of client IP. + + - While this service does implement some support for preserving the client IP, it is limited. + - This may be problematic when monitoring services like Fail2Ban are enabled that scan logs for multiple failed authentication attempts which triggers a ban on the shared IP address. + + You should adjust configuration of these monitoring services to monitor for auth failures from those services directly instead, adding an exclusion for that service IP from any DMS logs monitored (_but be mindful of PROXY header forgery risks_). + +[docs::overrides::dovecot]: ../../config/advanced/override-defaults/dovecot.md +[docs::overrides::postfix]: ../../config/advanced/override-defaults/postfix.md +[docs::overrides::user-patches]: ../../config/advanced/override-defaults/user-patches.md +[docs::ipv6::security-risks]: ../../config/advanced/ipv6.md#what-can-go-wrong +[docs::tls::traefik]: ../../config/security/ssl.md#traefik-v2 +[docs::env::permit_docker]: ../../config/environment.md#permit_docker +[gh-dms::dns-rewrite-example]: https://github.com/docker-mailserver/docker-mailserver/issues/3866#issuecomment-1928877236 + +[nginx-docs::proxyprotocol]: https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol +[caddy::plugin::l4]: https://github.com/mholt/caddy-l4 + +[traefik-web]: https://traefik.io +[traefik-docs::entrypoint::proxyprotocol]: https://doc.traefik.io/traefik/routing/entrypoints/#proxyprotocol +[traefik-docs::provider-docker::network]: https://doc.traefik.io/traefik/providers/docker/#network +[traefik-docs::router-tcp::server-first-protocols]: https://doc.traefik.io/traefik/routing/routers/#entrypoints_1 +[traefik-docs::router-tcp::host-sni]: https://doc.traefik.io/traefik/routing/routers/#rule_1 +[traefik-docs::router-tcp::passthrough]: https://doc.traefik.io/traefik/routing/routers/#passthrough +[traefik-docs::service-tcp::proxyprotocol]:https://doc.traefik.io/traefik/routing/services/#proxy-protocol + +[dovecot-docs::settings::haproxy-trusted-networks]: https://doc.dovecot.org/settings/core/#core_setting-haproxy_trusted_networks +[dovecot-docs::service-config::haproxy]: https://doc.dovecot.org/configuration_manual/service_configuration/#haproxy-v2-2-19 +[dovecot-docs::service-config::ssl]: https://doc.dovecot.org/configuration_manual/service_configuration/#ssl + +[postfix-docs::config-table::cidr]: https://www.postfix.org/cidr_table.5.html +[postfix-docs::settings::check_client_access]: https://www.postfix.org/postconf.5.html#check_client_access +[postfix-docs::settings::mynetworks]: https://www.postfix.org/postconf.5.html#mynetworks +[postfix-docs::settings::postscreen_access_list]: https://www.postfix.org/postconf.5.html#postscreen_access_list +[postfix-docs::settings::postscreen_upstream_proxy_protocol]: https://www.postfix.org/postconf.5.html#postscreen_upstream_proxy_protocol +[postfix-docs::settings::smtpd_client_restrictions]: https://www.postfix.org/postconf.5.html#smtpd_client_restrictions +[postfix-docs::settings::smtpd_upstream_proxy_protocol]: https://www.postfix.org/postconf.5.html#smtpd_upstream_proxy_protocol + +[docker::networking::l2-switch-gotcha]: https://github.com/moby/moby/issues/45610 +[networking::spec:proxy-protocol]: https://github.com/haproxy/haproxy/blob/master/doc/proxy-protocol.txt +[networking::http-header::x-forwarded-for]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For +[networking::http-header::forwarded]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded +[networking::osi-model]: https://www.cloudflare.com/learning/ddos/glossary/open-systems-interconnection-model-osi/ From 79a9656f4876e50264060eafbc066287a036a045 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 14 Feb 2024 09:48:35 +0000 Subject: [PATCH 02/53] docs: update `CONTRIBUTORS.md` (#3883) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CONTRIBUTORS.md | 510 +++++++++++++++++++++++++----------------------- 1 file changed, 262 insertions(+), 248 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index a7c4e661..1e252291 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -628,6 +628,13 @@ Thanks goes to these wonderful people ✨ GoliathLabs + + + frugan-dev +
+ frugan-dev + + tbutter @@ -641,15 +648,15 @@ Thanks goes to these wonderful people ✨
yogo1212 - + + willtho89
willtho89 - - + mpanneck @@ -658,10 +665,17 @@ Thanks goes to these wonderful people ✨ - - andrewlow + + ubenmackin
- andrewlow + ubenmackin + + + + + craue +
+ craue @@ -671,6 +685,14 @@ Thanks goes to these wonderful people ✨ abh + + + andrewlow +
+ andrewlow + + + aminvakil @@ -685,21 +707,6 @@ Thanks goes to these wonderful people ✨ elbracht - - - ubenmackin -
- ubenmackin - - - - - - craue -
- craue - - danielpanteleit @@ -727,85 +734,14 @@ Thanks goes to these wonderful people ✨
DuncanvR - + + emazzotta
emazzotta - - - - - fl42 -
- fl42 - - - - - ipernet -
- ipernet - - - - - H4R0 -
- H4R0 - - - - - eltociear -
- eltociear - - - - - jamesfryer -
- jamesfryer - - - - - millaguie -
- millaguie - - - - - - artonge -
- artonge - - - - - jedateach -
- jedateach - - - - - spacecowboy -
- spacecowboy - - - - - martinwepner -
- martinwepner - @@ -815,39 +751,96 @@ Thanks goes to these wonderful people ✨ - - Thiritin + + martinwepner
- Thiritin + martinwepner + + + + + artonge +
+ artonge + + + + + spacecowboy +
+ spacecowboy + + + + + jedateach +
+ jedateach - - thomasschmit + + millaguie
- thomasschmit + millaguie - - TechnicLab + + eltociear
- TechnicLab + eltociear - - sylvaindumont + + H4R0
- sylvaindumont + H4R0 - - syl20bnr + + jamesfryer
- syl20bnr + jamesfryer + + + + + ipernet +
+ ipernet + + + + + fl42 +
+ fl42 + + + + + + simonsystem +
+ simonsystem + + + + + stephan-devop +
+ stephan-devop + + + + + stigok +
+ stigok @@ -858,25 +851,53 @@ Thanks goes to these wonderful people ✨ - - stigok + + syl20bnr
- stigok + syl20bnr + + + + + sylvaindumont +
+ sylvaindumont - - stephan-devop + + TechnicLab
- stephan-devop + TechnicLab - - radicand + + ShiriNmi1520
- radicand + ShiriNmi1520 + + + + + thomasschmit +
+ thomasschmit + + + + + Thiritin +
+ Thiritin + + + + + 42wim +
+ 42wim @@ -885,7 +906,8 @@ Thanks goes to these wonderful people ✨
tweibert - + + torus @@ -906,8 +928,7 @@ Thanks goes to these wonderful people ✨
Twist235 - - + k3it @@ -928,27 +949,6 @@ Thanks goes to these wonderful people ✨
vilisas - - - - 42wim -
- 42wim - - - - - ShiriNmi1520 -
- ShiriNmi1520 - - - - - Zepmann -
- Zepmann - @@ -1015,13 +1015,28 @@ Thanks goes to these wonderful people ✨ piwai + + + rahilarious +
+ rahilarious + + + + + 0xflotus +
+ 0xflotus + + remoe
remoe - + + romansey @@ -1035,8 +1050,7 @@ Thanks goes to these wonderful people ✨
norrs - - + MightySCollins @@ -1064,7 +1078,8 @@ Thanks goes to these wonderful people ✨
svdb0 - + + 3ap @@ -1078,8 +1093,7 @@ Thanks goes to these wonderful people ✨
shyim - - + sjmudd @@ -1088,17 +1102,10 @@ Thanks goes to these wonderful people ✨ - - simonsystem + + mchamplain
- simonsystem - - - - - allddd -
- allddd + mchamplain @@ -1114,15 +1121,15 @@ Thanks goes to these wonderful people ✨
mplx - + + odinis
odinis - - + okamidash @@ -1157,15 +1164,15 @@ Thanks goes to these wonderful people ✨
presocratics - + + rhyst
rhyst - - + rmlhuk @@ -1200,15 +1207,15 @@ Thanks goes to these wonderful people ✨
sportshead - + + squash
squash - - + strarsis @@ -1243,15 +1250,29 @@ Thanks goes to these wonderful people ✨
wolkenschieber - + + worldworm
worldworm - - + + + + Zepmann +
+ Zepmann + + + + + allddd +
+ allddd + + arcaine2 @@ -1272,7 +1293,8 @@ Thanks goes to these wonderful people ✨
brainkiller - + + cternes @@ -1293,8 +1315,7 @@ Thanks goes to these wonderful people ✨
dimalo - - + eleith @@ -1315,7 +1336,8 @@ Thanks goes to these wonderful people ✨
helmutundarnold - + + hnws @@ -1336,8 +1358,7 @@ Thanks goes to these wonderful people ✨
idaadi - - + ixeft @@ -1358,7 +1379,8 @@ Thanks goes to these wonderful people ✨
paralax - + + jpduyx @@ -1379,8 +1401,7 @@ Thanks goes to these wonderful people ✨
callmemagnus - - + marios88 @@ -1395,35 +1416,21 @@ Thanks goes to these wonderful people ✨ matrixes - - - mchamplain -
- mchamplain - - - - - crash7 -
- crash7 - - auchri
auchri - + + arkanovicz
arkanovicz - - + CBeerta @@ -1458,15 +1465,15 @@ Thanks goes to these wonderful people ✨
dbellavista - + + danielvandenberg95
danielvandenberg95 - - + denisix @@ -1501,15 +1508,15 @@ Thanks goes to these wonderful people ✨
vedtam - + + edvorg
edvorg - - + eliroca @@ -1544,20 +1551,20 @@ Thanks goes to these wonderful people ✨
felixn - + + flole
flole - - + - - 0xflotus + + froks
- 0xflotus + froks @@ -1587,15 +1594,15 @@ Thanks goes to these wonderful people ✨
askz - + + aspettl
aspettl - - + acch @@ -1630,15 +1637,15 @@ Thanks goes to these wonderful people ✨
ch3sh1r - + + eglia
eglia - - + groupmsl @@ -1673,15 +1680,15 @@ Thanks goes to these wonderful people ✨
arunvc - + + astrocket
astrocket - - + baxerus @@ -1704,10 +1711,10 @@ Thanks goes to these wonderful people ✨ - - akkumar + + crash7
- akkumar + crash7 @@ -1716,15 +1723,15 @@ Thanks goes to these wonderful people ✨
thechubbypanda - + + KCrawley
KCrawley - - + khuedoan @@ -1759,15 +1766,15 @@ Thanks goes to these wonderful people ✨
luke- - + + LucidityCrash
LucidityCrash - - + MadsRC @@ -1802,15 +1809,15 @@ Thanks goes to these wonderful people ✨
michaeljensen - + + exhuma
exhuma - - + milas @@ -1845,20 +1852,20 @@ Thanks goes to these wonderful people ✨
naveensrinivasan - + + neuralp
neuralp - - + - - froks + + radicand
- froks + radicand @@ -1868,13 +1875,6 @@ Thanks goes to these wonderful people ✨ fkefer - - - frugan-dev -
- frugan-dev - - Marsu31 @@ -1925,21 +1925,28 @@ Thanks goes to these wonderful people ✨ Influencer + + + JacksonZ03 +
+ JacksonZ03 + + JamBalaya56562
JamBalaya56562 - + + jcalfee
jcalfee - - + mivek @@ -1974,15 +1981,15 @@ Thanks goes to these wonderful people ✨
jirislav - + + jmccl
jmccl - - + jurekbarth @@ -2003,6 +2010,13 @@ Thanks goes to these wonderful people ✨
Kaan88 + + + + akkumar +
+ akkumar + From a815bf5ab44388baecb597a0b83f71872a2d34bc Mon Sep 17 00:00:00 2001 From: Robbert Klarenbeek Date: Fri, 16 Feb 2024 08:24:39 +0100 Subject: [PATCH 03/53] fix: Apply SELinux security context after moving to mail-state (#3890) * fix: Apply SELinux security context after moving to mail-state * fix: Ignore failing chcon on non-SELinux systems --- CHANGELOG.md | 1 + target/scripts/startup/setup.d/mail_state.sh | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index c8030f43..2faf36f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -82,6 +82,7 @@ The most noteworthy change of this release is the update of the container's base - `RELAY_HOST` ENV no longer enforces configuring outbound SMTP to require credentials. Like `DEFAULT_RELAY_HOST` it can now configure a relay where credentials are optional. - Restarting DMS should not be required when configuring relay hosts without these ENV, but solely via `setup relay ...`, as change detection events now apply relevant Postfix setting changes for supporting credentials too. - Rspamd configuration: Add a missing comma in `local_networks` so that all internal IP addresses are actually considered as internal ([#3862](https://github.com/docker-mailserver/docker-mailserver/pull/3862)) +- Ensure correct SELinux security context labels for files and directories moved to the mail-state volume during setup ([#3890](https://github.com/docker-mailserver/docker-mailserver/pull/3890)) ## [v13.3.1](https://github.com/docker-mailserver/docker-mailserver/releases/tag/v13.3.1) diff --git a/target/scripts/startup/setup.d/mail_state.sh b/target/scripts/startup/setup.d/mail_state.sh index 9c43fea4..5acf6762 100644 --- a/target/scripts/startup/setup.d/mail_state.sh +++ b/target/scripts/startup/setup.d/mail_state.sh @@ -48,6 +48,9 @@ function _setup_save_states() { _log 'trace' "Moving ${SERVICEFILE} to ${DEST}" # Empty volume was mounted, or new content from enabling a feature ENV: mv "${SERVICEFILE}" "${DEST}" + # Apply SELinux security context to match the state directory, so access + # is not restricted to the current running container: + chcon -R --reference="${STATEDIR}" "${DEST}" 2>/dev/null || true fi # Symlink the original file in the container ($SERVICEFILE) to be @@ -69,6 +72,9 @@ function _setup_save_states() { _log 'trace' "Moving contents of ${SERVICEDIR} to ${DEST}" # Empty volume was mounted, or new content from enabling a feature ENV: mv "${SERVICEDIR}" "${DEST}" + # Apply SELinux security context to match the state directory, so access + # is not restricted to the current running container: + chcon -R --reference="${STATEDIR}" "${DEST}" 2>/dev/null || true fi # Symlink the original path in the container ($SERVICEDIR) to be From d86c3cb1598d44afed1db60d3de8e3d1438dcd7c Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Tue, 20 Feb 2024 21:21:22 +1300 Subject: [PATCH 04/53] chore: `packages.sh` - Remove redundant comment (#3900) --- target/scripts/build/packages.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/target/scripts/build/packages.sh b/target/scripts/build/packages.sh index 1f769739..5360ae95 100644 --- a/target/scripts/build/packages.sh +++ b/target/scripts/build/packages.sh @@ -147,7 +147,6 @@ function _install_dovecot() { _log 'trace' 'Using Dovecot community repository' curl -sSfL https://repo.dovecot.org/DOVECOT-REPO-GPG | gpg --import gpg --export ED409DA1 > /etc/apt/trusted.gpg.d/dovecot.gpg - # VERSION_CODENAME sourced from /etc/os-release echo "deb https://repo.dovecot.org/ce-2.3-latest/debian/${VERSION_CODENAME} ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/dovecot.list _log 'trace' 'Updating Dovecot package signatures' From 67faa95b0b0e355afc84b86cf1b69ab89c674ab1 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Tue, 20 Feb 2024 21:33:04 +1300 Subject: [PATCH 05/53] fix(`setup`): `open-dkim` log for conflicting implementations (#3899) --- target/bin/open-dkim | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/bin/open-dkim b/target/bin/open-dkim index 808ef8cc..9410ecfa 100755 --- a/target/bin/open-dkim +++ b/target/bin/open-dkim @@ -5,7 +5,7 @@ source /usr/local/bin/helpers/index.sh if [[ -f /etc/dms-settings ]] && [[ $(_get_dms_env_value 'ENABLE_RSPAMD') -eq 1 ]]; then if [[ $(_get_dms_env_value 'ENABLE_OPENDKIM') -eq 1 ]]; then - log 'error' "You enabled Rspamd and OpenDKIM - OpenDKIM will be implicitly used for DKIM keys" + _log 'warn' "Conflicting DKIM support, both Rspamd and OpenDKIM enabled - OpenDKIM will manage DKIM keys" else /usr/local/bin/rspamd-dkim "${@}" exit From e232e43d32b4693a43a841250f2bd0c350f48c89 Mon Sep 17 00:00:00 2001 From: Casper Date: Wed, 21 Feb 2024 11:19:41 +0100 Subject: [PATCH 06/53] fix: fetchmail environment variables (#3901) --- CHANGELOG.md | 1 + target/supervisor/conf.d/supervisor-app.conf | 1 + 2 files changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2faf36f7..5b05b5d1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,6 +83,7 @@ The most noteworthy change of this release is the update of the container's base - Restarting DMS should not be required when configuring relay hosts without these ENV, but solely via `setup relay ...`, as change detection events now apply relevant Postfix setting changes for supporting credentials too. - Rspamd configuration: Add a missing comma in `local_networks` so that all internal IP addresses are actually considered as internal ([#3862](https://github.com/docker-mailserver/docker-mailserver/pull/3862)) - Ensure correct SELinux security context labels for files and directories moved to the mail-state volume during setup ([#3890](https://github.com/docker-mailserver/docker-mailserver/pull/3890)) +- Use correct environment variable for fetchmail ([#3901](https://github.com/docker-mailserver/docker-mailserver/pull/3901)) ## [v13.3.1](https://github.com/docker-mailserver/docker-mailserver/releases/tag/v13.3.1) diff --git a/target/supervisor/conf.d/supervisor-app.conf b/target/supervisor/conf.d/supervisor-app.conf index 50a7acc3..7f106456 100644 --- a/target/supervisor/conf.d/supervisor-app.conf +++ b/target/supervisor/conf.d/supervisor-app.conf @@ -122,6 +122,7 @@ autorestart=true stdout_logfile=/var/log/supervisor/%(program_name)s.log stderr_logfile=/var/log/supervisor/%(program_name)s.log user=fetchmail +environment=HOME="/var/lib/fetchmail",USER="fetchmail" command=/usr/bin/fetchmail -f /etc/fetchmailrc --nodetach --daemon "%(ENV_FETCHMAIL_POLL)s" -i /var/lib/fetchmail/.fetchmail-UIDL-cache --pidfile /var/run/fetchmail/fetchmail.pid [program:postfix] From 95dfc71b54a6445190555b2f192cc4e195fe38dc Mon Sep 17 00:00:00 2001 From: Jesse Portnoy Date: Sat, 24 Feb 2024 23:06:58 +0000 Subject: [PATCH 07/53] Fix typo and broken README link (#3906) --- docs/content/config/environment.md | 2 +- docs/content/usage.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/config/environment.md b/docs/content/config/environment.md index 323b6321..a0916aa3 100644 --- a/docs/content/config/environment.md +++ b/docs/content/config/environment.md @@ -212,7 +212,7 @@ Configures the handling of creating mails with forged sender addresses. ##### ENABLE_SRS -Enables the Sender Rewriting Scheme. SRS is needed if DMS acts as forwarder. See [postsrsd](https://github.com/roehling/postsrsd/blob/master/README.md#sender-rewriting-scheme-crash-course) for further explanation. +Enables the Sender Rewriting Scheme. SRS is needed if DMS acts as forwarder. See [postsrsd](https://github.com/roehling/postsrsd/blob/main/README.rst) for further explanation. - **0** => Disabled - 1 => Enabled diff --git a/docs/content/usage.md b/docs/content/usage.md index 30f6e822..76011fe7 100644 --- a/docs/content/usage.md +++ b/docs/content/usage.md @@ -2,7 +2,7 @@ title: Usage --- -This pages explains how to get started with DMS. The guide uses Docker Compose as a reference. In our examples, a volume mounts the host location [`docker-data/dms/config/`][docs::dms-volumes-config] to `/tmp/docker-mailserver/` inside the container. +This page explains how to get started with DMS. The guide uses Docker Compose as a reference. In our examples, a volume mounts the host location [`docker-data/dms/config/`][docs::dms-volumes-config] to `/tmp/docker-mailserver/` inside the container. [docs::dms-volumes-config]: ./config/advanced/optional-config.md#volumes-config From d3ccaddb708dcb427850e0dfe5bbbfee3047f8ec Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sun, 25 Feb 2024 12:54:49 +0100 Subject: [PATCH 08/53] docs: updated `CONTRIBUTORS.md` (#3909) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CONTRIBUTORS.md | 239 +++++++++++++++++++++++++----------------------- 1 file changed, 123 insertions(+), 116 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 1e252291..c387a654 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -291,28 +291,21 @@ Thanks goes to these wonderful people ✨ pyy - - - dennis95stumm -
- dennis95stumm - - arneke
arneke - - + akmet
akmet - + + diiigle @@ -347,15 +340,15 @@ Thanks goes to these wonderful people ✨
lukecyca - - + jsonn
jsonn - + + jamebus @@ -363,13 +356,6 @@ Thanks goes to these wonderful people ✨ jamebus - - - dashohoxha -
- dashohoxha - - mathuin @@ -377,6 +363,13 @@ Thanks goes to these wonderful people ✨ mathuin + + + dashohoxha +
+ dashohoxha + + egavard @@ -390,15 +383,15 @@ Thanks goes to these wonderful people ✨
weo - - + Zehir
Zehir - + + guardiande @@ -433,15 +426,15 @@ Thanks goes to these wonderful people ✨
VanVan - - + mjung
mjung - + + m-schmoock @@ -476,15 +469,15 @@ Thanks goes to these wonderful people ✨
Starbix - - + citec
citec - + + yajo @@ -519,15 +512,15 @@ Thanks goes to these wonderful people ✨
MakerMatrix - - + pbek
pbek - + + keslerm @@ -562,15 +555,15 @@ Thanks goes to these wonderful people ✨
bobbravo2 - - + r-pufky
r-pufky - + + vincentDcmps @@ -605,15 +598,15 @@ Thanks goes to these wonderful people ✨
j-marz - - + lokipo
lokipo - + + msheakoski @@ -648,15 +641,15 @@ Thanks goes to these wonderful people ✨
yogo1212 - - + willtho89
willtho89 - + + mpanneck @@ -691,15 +684,15 @@ Thanks goes to these wonderful people ✨
andrewlow - - + aminvakil
aminvakil - + + elbracht @@ -734,15 +727,15 @@ Thanks goes to these wonderful people ✨
DuncanvR - - + emazzotta
emazzotta - + + nueaf @@ -777,34 +770,20 @@ Thanks goes to these wonderful people ✨
jedateach - - + millaguie
millaguie - + + - - eltociear + + fl42
- eltociear - - - - - H4R0 -
- H4R0 - - - - - jamesfryer -
- jamesfryer + fl42 @@ -815,10 +794,31 @@ Thanks goes to these wonderful people ✨ - - fl42 + + eltociear
- fl42 + eltociear + + + + + jamesfryer +
+ jamesfryer + + + + + H4R0 +
+ H4R0 + + + + + nilshoell +
+ nilshoell @@ -872,13 +872,6 @@ Thanks goes to these wonderful people ✨ TechnicLab - - - ShiriNmi1520 -
- ShiriNmi1520 - - thomasschmit @@ -893,21 +886,13 @@ Thanks goes to these wonderful people ✨ Thiritin - - - 42wim -
- 42wim - - tweibert
tweibert - - + torus @@ -921,7 +906,8 @@ Thanks goes to these wonderful people ✨
VictorKoenders - + + Twist235 @@ -949,15 +935,22 @@ Thanks goes to these wonderful people ✨
vilisas - - + - - nilshoell + + 42wim
- nilshoell + 42wim + + + ShiriNmi1520 +
+ ShiriNmi1520 + + + nknapp @@ -992,15 +985,15 @@ Thanks goes to these wonderful people ✨
mrPjer - - + p3dda
p3dda - + + peter-hartmann @@ -1035,6 +1028,13 @@ Thanks goes to these wonderful people ✨
remoe + + + + robbertkl +
+ robbertkl + @@ -1717,14 +1717,21 @@ Thanks goes to these wonderful people ✨ crash7 + + + fkefer +
+ fkefer + + + thechubbypanda
thechubbypanda - - + KCrawley @@ -1759,15 +1766,15 @@ Thanks goes to these wonderful people ✨
linhandev - + + luke-
luke- - - + LucidityCrash @@ -1802,15 +1809,15 @@ Thanks goes to these wonderful people ✨
dragetd - + + michaeljensen
michaeljensen - - + exhuma @@ -1845,15 +1852,15 @@ Thanks goes to these wonderful people ✨
mpldr - + + naveensrinivasan
naveensrinivasan - - + neuralp @@ -1868,13 +1875,6 @@ Thanks goes to these wonderful people ✨ radicand - - - fkefer -
- fkefer - - Marsu31 @@ -1968,21 +1968,28 @@ Thanks goes to these wonderful people ✨ Jeidnx + + + jessp01 +
+ jessp01 + + JiLleON
JiLleON - + + jirislav
jirislav - - + jmccl From 2c1faa72444a132785362f61a08b9cf659a5d5a9 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 26 Feb 2024 13:45:50 +0100 Subject: [PATCH 09/53] chore(deps): Bump myrotvorets/set-commit-status-action (#3911) --- .github/workflows/docs-preview-deploy.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/docs-preview-deploy.yml b/.github/workflows/docs-preview-deploy.yml index 3b6f3c67..dbe57752 100644 --- a/.github/workflows/docs-preview-deploy.yml +++ b/.github/workflows/docs-preview-deploy.yml @@ -45,7 +45,7 @@ jobs: # but presently does not work correctly via split workflow. It is useful in a split workflow as the 1st stage # no longer indicates if the entire workflow/deployment was successful. - name: 'Commit Status: Set Workflow Status as Pending' - uses: myrotvorets/set-commit-status-action@v2.0.0 + uses: myrotvorets/set-commit-status-action@v2.0.1 with: token: ${{ secrets.GITHUB_TOKEN }} status: pending @@ -105,7 +105,7 @@ jobs: Built with commit: ${{ env.PR_HEADSHA }} - name: 'Commit Status: Update deployment status' - uses: myrotvorets/set-commit-status-action@v2.0.0 + uses: myrotvorets/set-commit-status-action@v2.0.1 # Always run this step regardless of job failing early: if: ${{ always() }} env: From 512f39c7ebc6edce8b5ae5302a2777c4f9173bea Mon Sep 17 00:00:00 2001 From: Dominic Germain Date: Wed, 28 Feb 2024 09:34:30 -0500 Subject: [PATCH 10/53] feat: Configurable number of rotated log files (#3907) --- CHANGELOG.md | 5 +++++ docs/content/config/environment.md | 6 ++++++ mailserver.env | 3 +++ target/scripts/startup/setup.d/log.sh | 8 +++++++- target/scripts/startup/setup.d/security/rspamd.sh | 2 +- target/scripts/startup/variables-stack.sh | 1 + 6 files changed, 23 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5b05b5d1..009e2355 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,11 @@ The most noteworthy change of this release is the update of the container's base - `undef` was previously supported as an opt-out with `SA_SPAM_SUBJECT`. This is no longer valid, the equivalent opt-out value is now an empty value (_or rather the omission of this ENV being configured_). - The feature to include [`_SCORE_` tag](https://spamassassin.apache.org/full/4.0.x/doc/Mail_SpamAssassin_Conf.html#rewrite_header-subject-from-to-STRING) in your value to be replaced by the associated spam score is no longer available. +### Added + +- **Environment Variables:** + - `LOGROTATE_COUNT` defines the number of files kept by logrotate ([#3907](https://github.com/docker-mailserver/docker-mailserver/pull/3907)) + ### Updates - **Environment Variables:** diff --git a/docs/content/config/environment.md b/docs/content/config/environment.md index a0916aa3..83fcdfa6 100644 --- a/docs/content/config/environment.md +++ b/docs/content/config/environment.md @@ -545,6 +545,12 @@ Changes the interval in which log files are rotated. This variable can also determine the interval for Postfix's log summary reports, see [`PFLOGSUMM_TRIGGER`](#pflogsumm_trigger). +##### LOGROTATE_COUNT + +Defines how many files are kept by logrotate. + +- **4** => Number of files + #### SpamAssassin ##### ENABLE_SPAMASSASSIN diff --git a/mailserver.env b/mailserver.env index 44a9a484..0d00de0a 100644 --- a/mailserver.env +++ b/mailserver.env @@ -346,6 +346,9 @@ REPORT_SENDER= # Note: This variable can also determine the interval for Postfix's log summary reports, see [`PFLOGSUMM_TRIGGER`](#pflogsumm_trigger). LOGROTATE_INTERVAL=weekly +# Defines how many log files are kept by logrorate +LOGROTATE_COUNT=4 + # If enabled, employs `reject_unknown_client_hostname` to sender restrictions in Postfix's configuration. # diff --git a/target/scripts/startup/setup.d/log.sh b/target/scripts/startup/setup.d/log.sh index cf282966..06aa679d 100644 --- a/target/scripts/startup/setup.d/log.sh +++ b/target/scripts/startup/setup.d/log.sh @@ -19,13 +19,19 @@ function _setup_logrotate() { _dms_panic__invalid_value 'LOGROTATE_INTERVAL' 'Setup -> Logrotate' fi + if [[ ${LOGROTATE_COUNT} =~ ^[0-9]+$ ]]; then + _log 'trace' "Logrotate count set to ${LOGROTATE_COUNT}" + else + _dms_panic__invalid_value 'LOGROTATE_COUNT' 'Setup -> Logrotate' + fi + cat >/etc/logrotate.d/maillog << EOF /var/log/mail/mail.log { compress copytruncate delaycompress - rotate 4 + rotate ${LOGROTATE_COUNT} ${LOGROTATE_INTERVAL} } EOF diff --git a/target/scripts/startup/setup.d/security/rspamd.sh b/target/scripts/startup/setup.d/security/rspamd.sh index 98f83de6..6b54addd 100644 --- a/target/scripts/startup/setup.d/security/rspamd.sh +++ b/target/scripts/startup/setup.d/security/rspamd.sh @@ -109,7 +109,7 @@ function __rspamd__setup_logfile() { compress copytruncate delaycompress - rotate 4 + rotate ${LOGROTATE_COUNT} ${LOGROTATE_INTERVAL} } EOF diff --git a/target/scripts/startup/variables-stack.sh b/target/scripts/startup/variables-stack.sh index eb5bf149..a3be72b8 100644 --- a/target/scripts/startup/variables-stack.sh +++ b/target/scripts/startup/variables-stack.sh @@ -145,6 +145,7 @@ function __environment_variables_general_setup() { VARS[GETMAIL_POLL]="${GETMAIL_POLL:=5}" VARS[LOG_LEVEL]="${LOG_LEVEL:=info}" VARS[LOGROTATE_INTERVAL]="${LOGROTATE_INTERVAL:=weekly}" + VARS[LOGROTATE_COUNT]="${LOGROTATE_COUNT:=4}" VARS[LOGWATCH_INTERVAL]="${LOGWATCH_INTERVAL:=none}" VARS[LOGWATCH_RECIPIENT]="${LOGWATCH_RECIPIENT:=${REPORT_RECIPIENT}}" VARS[LOGWATCH_SENDER]="${LOGWATCH_SENDER:=${REPORT_SENDER}}" From aa9465773c99a25dea7ca9534f1928ed6301693a Mon Sep 17 00:00:00 2001 From: Casper Date: Wed, 28 Feb 2024 22:08:19 +0100 Subject: [PATCH 11/53] Rename supervisor-app.conf to dms-services.conf (#3908) * rename supervisor-app.conf to dms-services.conf * changelog added --- CHANGELOG.md | 2 ++ .../conf.d/{supervisor-app.conf => dms-services.conf} | 0 .../set3/container_configuration/process_check_restart.bats | 4 ++-- 3 files changed, 4 insertions(+), 2 deletions(-) rename target/supervisor/conf.d/{supervisor-app.conf => dms-services.conf} (100%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 009e2355..62da354f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,8 @@ The most noteworthy change of this release is the update of the container's base - The default has changed to not prepend any prefix to the subject unless configured to do so. If you relied on the implicit prefix, you will now need to provide one explicitly. - `undef` was previously supported as an opt-out with `SA_SPAM_SUBJECT`. This is no longer valid, the equivalent opt-out value is now an empty value (_or rather the omission of this ENV being configured_). - The feature to include [`_SCORE_` tag](https://spamassassin.apache.org/full/4.0.x/doc/Mail_SpamAssassin_Conf.html#rewrite_header-subject-from-to-STRING) in your value to be replaced by the associated spam score is no longer available. +- **Supervisord**: + - `supervisor-app.conf` renamed to `dms-services.conf` ### Added diff --git a/target/supervisor/conf.d/supervisor-app.conf b/target/supervisor/conf.d/dms-services.conf similarity index 100% rename from target/supervisor/conf.d/supervisor-app.conf rename to target/supervisor/conf.d/dms-services.conf diff --git a/test/tests/parallel/set3/container_configuration/process_check_restart.bats b/test/tests/parallel/set3/container_configuration/process_check_restart.bats index 4f0fd90f..78eabd7a 100644 --- a/test/tests/parallel/set3/container_configuration/process_check_restart.bats +++ b/test/tests/parallel/set3/container_configuration/process_check_restart.bats @@ -184,7 +184,7 @@ function _check_if_process_is_running() { # `--list-full` provides information for matching against (full process path) # `--full` allows matching the process against the full path (required if a process is not the exec command, such as started by python3 command without a shebang) - # `--oldest` should select the parent process when there are multiple results, typically the command defined in `supervisor-app.conf` + # `--oldest` should select the parent process when there are multiple results, typically the command defined in `dms-services.conf` local IS_RUNNING=$(_exec_in_container pgrep --full --list-full "${MIN_SECS_RUNNING[@]}" --oldest "${PROCESS}") # When no matches are found, nothing is returned. Provide something we can assert on (helpful for debugging): @@ -199,7 +199,7 @@ function _check_if_process_is_running() { # The process manager (supervisord) should perform a graceful shutdown: # NOTE: Time limit should never be below these configured values: -# - supervisor-app.conf:stopwaitsecs +# - dms-services.conf:stopwaitsecs # - compose.yaml:stop_grace_period function _should_stop_cleanly() { run docker stop -t 60 "${CONTAINER_NAME}" From 736f2e44bcddac31b0bed506e5e60756d8dd032a Mon Sep 17 00:00:00 2001 From: Casper Date: Fri, 1 Mar 2024 01:00:23 +0100 Subject: [PATCH 12/53] Fail2Ban: Align logrotate count & interval (#3915) --- CHANGELOG.md | 1 + target/scripts/startup/setup.d/security/misc.sh | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 62da354f..6b663f46 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,6 +66,7 @@ The most noteworthy change of this release is the update of the container's base - **Environment Variables:** - `LOGROTATE_COUNT` defines the number of files kept by logrotate ([#3907](https://github.com/docker-mailserver/docker-mailserver/pull/3907)) + - The fail2ban log file is now also taken into account by `LOGROTATE_COUNT` and `LOGROTATE_INTERVAL` ([#3915](https://github.com/docker-mailserver/docker-mailserver/pull/3915)) ### Updates diff --git a/target/scripts/startup/setup.d/security/misc.sh b/target/scripts/startup/setup.d/security/misc.sh index aefeba20..df810b2e 100644 --- a/target/scripts/startup/setup.d/security/misc.sh +++ b/target/scripts/startup/setup.d/security/misc.sh @@ -189,14 +189,17 @@ function __setup__security__fail2ban() { _log 'debug' 'Enabling and configuring Fail2Ban' if [[ -e /tmp/docker-mailserver/fail2ban-fail2ban.cf ]]; then + _log 'trace' 'Custom fail2ban-fail2ban.cf found' cp /tmp/docker-mailserver/fail2ban-fail2ban.cf /etc/fail2ban/fail2ban.local fi if [[ -e /tmp/docker-mailserver/fail2ban-jail.cf ]]; then + _log 'trace' 'Custom fail2ban-jail.cf found' cp /tmp/docker-mailserver/fail2ban-jail.cf /etc/fail2ban/jail.d/user-jail.local fi if [[ ${FAIL2BAN_BLOCKTYPE} != 'reject' ]]; then + _log 'trace' "Setting fail2ban blocktype to 'drop'" echo -e '[Init]\nblocktype = drop' >/etc/fail2ban/action.d/nftables-common.local fi @@ -205,6 +208,9 @@ function __setup__security__fail2ban() { _log 'debug' 'Fail2Ban is disabled' rm -f /etc/logrotate.d/fail2ban fi + _log 'trace' 'Configuring fail2ban logrotate rotate count and interval' + sedfile -i "s|rotate 4$|rotate ${LOGROTATE_COUNT}|" /etc/logrotate.d/fail2ban + sedfile -i "s|weekly$|${LOGROTATE_INTERVAL}|" /etc/logrotate.d/fail2ban } function __setup__security__amavis() { From 12f5101d84331e23dea2192b05c502de5e610116 Mon Sep 17 00:00:00 2001 From: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Date: Sat, 2 Mar 2024 02:42:47 +0100 Subject: [PATCH 13/53] Rspamd: improve SPF, DKIM and DMARC Symbol Weights (#3913) --- CHANGELOG.md | 5 +- docs/content/config/security/rspamd.md | 32 ++++---- target/rspamd/scores.d/policies_group.conf | 86 +++++++++++----------- 3 files changed, 66 insertions(+), 57 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b663f46..30c7dd0c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,8 +78,9 @@ The most noteworthy change of this release is the update of the container's base - **Tests:** - Refactored helper methods for sending e-mails with specific `Message-ID` headers and the helpers for retrieving + filtering logs, which together help isolate logs relevant to specific mail when multiple mails have been processed within a single test. ([#3786](https://github.com/docker-mailserver/docker-mailserver/pull/3786)) - **Rspamd**: - - The `rewrite_subject` action, is now disabled by default. It has been replaced with the new `SPAM_SUBJECT` environment variable, which implements the functionality via a Sieve script instead in favor of being anti-spam service agnostic ([3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820)) - - `RSPAMD_NEURAL` was added and is disabled by default. If switched on it wil enable the experimental Rspamd Neural network module to add a layer of analysis to spam detection using neural network technology. ([3833](https://github.com/docker-mailserver/docker-mailserver/pull/3833)) + - The `rewrite_subject` action, is now disabled by default. It has been replaced with the new `SPAM_SUBJECT` environment variable, which implements the functionality via a Sieve script instead which is anti-spam service agnostic ([#3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820)) + - `RSPAMD_NEURAL` was added and is disabled by default. If switched on it will enable the experimental Rspamd "Neural network" module to add a layer of analysis to spam detection ([#3833](https://github.com/docker-mailserver/docker-mailserver/pull/3833)) + - The symbol weights of SPF, DKIM and DMARC have been adjusted again. Fixes a bug and includes more appropriate combinations of symbols ([#3913](https://github.com/docker-mailserver/docker-mailserver/pull/3913)) ### Fixes diff --git a/docs/content/config/security/rspamd.md b/docs/content/config/security/rspamd.md index 6dc5f202..5cb901b7 100644 --- a/docs/content/config/security/rspamd.md +++ b/docs/content/config/security/rspamd.md @@ -4,9 +4,9 @@ title: 'Security | Rspamd' ## About -Rspamd is a ["fast, free and open-source spam filtering system"][www::rspamd-homepage]. DMS integrates Rspamd like any other service. We provide a basic but easy to maintain setup of Rspamd. +Rspamd is a ["fast, free and open-source spam filtering system"][rspamd-web]. DMS integrates Rspamd like any other service. We provide a basic but easy to maintain setup of Rspamd. -If you want to take a look at the default configuration files for Rspamd that DMS packs, navigate to [`target/rspamd/` inside the repository][repo::dms-default-configuration]. Please consult the [section "The Default Configuration"](#the-default-configuration) section down below for a written overview. +If you want to take a look at the default configuration files for Rspamd that DMS packs, navigate to [`target/rspamd/` inside the repository][dms-repo::default-rspamd-configuration]. Please consult the [section "The Default Configuration"](#the-default-configuration) section down below for a written overview. ## Related Environment Variables @@ -56,7 +56,7 @@ And then there is a corresponding `X-Rspamd-Action` header, which shows the over X-Rspamd-Action no action ``` -Since the score is `-2.80`, nothing will happen and the e-mail is not classified as spam. Our custom [`actions.conf`][www::rspamd-actions-config] defines what to do at certain scores: +Since the score is `-2.80`, nothing will happen and the e-mail is not classified as spam. Our custom [`actions.conf`][dms-repo::rspamd-actions-config] defines what to do at certain scores: 1. At a score of 4, the e-mail is to be _greylisted_; 2. At a score of 6, the e-mail is _marked with a header_ (`X-Spam: Yes`); @@ -64,13 +64,13 @@ Since the score is `-2.80`, nothing will happen and the e-mail is not classified --- -There is more to spam analysis than meets the eye: we have not covered the [Bayes training and filters][www::rspamd-docs-bayes] here, nor have we discussed [Sieve rules for e-mails that are marked as spam][docs::spam-to-junk]. +There is more to spam analysis than meets the eye: we have not covered the [Bayes training and filters][rspamd-docs::bayes] here, nor have we discussed [Sieve rules for e-mails that are marked as spam][docs::spam-to-junk]. Even the calculation of the score with the individual symbols has been presented to you in a simplified manner. But with the knowledge from above, you're equipped to read on and use Rspamd confidently. Keep on reading to understand the integration even better - you will want to know about your anti-spam software, not only to keep the bad e-mail out, but also to make sure the good e-mail arrive properly! ### Workers -The proxy worker operates in [self-scan mode][www::rspamd-docs-proxy-self-scan-mode]. This simplifies the setup as we do not require a normal worker. You can easily change this though by [overriding the configuration by DMS](#providing-custom-settings-overriding-settings). +The proxy worker operates in [self-scan mode][rspamd-docs::proxy-self-scan-mode]. This simplifies the setup as we do not require a normal worker. You can easily change this though by [overriding the configuration by DMS](#providing-custom-settings-overriding-settings). DMS does not set a default password for the controller worker. You may want to do that yourself. In setups where you already have an authentication provider in front of the Rspamd webpage, you may want to [set the `secure_ip ` option to `"0.0.0.0/0"` for the controller worker](#with-the-help-of-a-custom-file) to disable password authentication inside Rspamd completely. @@ -88,18 +88,24 @@ Redis uses `/etc/redis/redis.conf` for configuration: ### Web Interface -Rspamd provides a [web interface][www::rspamd-docs-web-interface], which contains statistics and data Rspamd collects. The interface is enabled by default and reachable on port 11334. +Rspamd provides a [web interface][rspamd-docs::web-interface], which contains statistics and data Rspamd collects. The interface is enabled by default and reachable on port 11334. ![Rspamd Web Interface](https://rspamd.com/img/webui.png) ### DNS -DMS does not supply custom values for DNS servers to Rspamd. If you need to use custom DNS servers, which could be required when using [DNS-based black/whitelists](#rbls-realtime-blacklists-dnsbls-dns-based-blacklists), you need to adjust [`options.inc`][www::rspamd-docs-basic-options] yourself. +DMS does not supply custom values for DNS servers to Rspamd. If you need to use custom DNS servers, which could be required when using [DNS-based black/whitelists](#rbls-realtime-blacklists-dnsbls-dns-based-blacklists), you need to adjust [`options.inc`][rspamd-docs::basic-options] yourself. !!! tip "Making DNS Servers Configurable" If you want to see an environment variable (like `RSPAMD_DNS_SERVERS`) to support custom DNS servers for Rspamd being added to DMS, please raise a feature request issue. +!!! warning + + Rspamd heavily relies on a properly working DNS server that it can use to resolve DNS queries. If your DNS server is misconfigured, you will encounter issues when Rspamd queries DNS to assess if mail is spam. Legitimate mail is then unintentionally marked as spam or worse, rejected entirely. + + When Rspamd is deciding if mail is spam, it will check DNS records for SPF, DKIM and DMARC. Each of those has an associated symbol for DNS temporary errors with a non-zero weight assigned. That weight contributes towards the spam score assessed by Rspamd which is normally desirable - provided your network DNS is functioning correctly, otherwise when DNS is broken all mail is biased towards spam due to these failed DNS lookups. + !!! danger While we do not provide values for custom DNS servers by default, we set `soft_reject_on_timeout = true;` by default. This setting will cause a soft reject if a task (presumably a DNS request) timeout takes place. @@ -112,7 +118,7 @@ You can find the Rspamd logs at `/var/log/mail/rspamd.log`, and the correspondin ### Modules -You can find a list of all Rspamd modules [on their website][www::rspamd-docs-modules]. +You can find a list of all Rspamd modules [on their website][rspamd-docs::modules]. #### Disabled By Default @@ -140,9 +146,9 @@ DMS brings sane default settings for Rspamd. They are located at `/etc/rspamd/lo !!! question "What is [`docker-data/dms/config/`][docs::dms-volumes-config]?" -If you want to overwrite the default settings or provide your settings, you can place files at `docker-data/dms/config/rspamd/override.d/`. Files from this directory are copied to `/etc/rspamd/override.d/` during startup. These files [forcibly override][www::rspamd-docs-override-dir] Rspamd and DMS default settings. +If you want to overwrite the default settings or provide your settings, you can place files at `docker-data/dms/config/rspamd/override.d/`. Files from this directory are copied to `/etc/rspamd/override.d/` during startup. These files [forcibly override][rspamd-docs::override-dir] Rspamd and DMS default settings. -!!! question "What is the [`local.d` directory and how does it compare to `override.d`][www::rspamd-docs-config-directories]?" +!!! question "What is the [`local.d` directory and how does it compare to `override.d`][rspamd-docs::config-directories]?" !!! warning "Clashing Overrides" @@ -163,7 +169,7 @@ where `COMMAND` can be: 3. `set-option-for-module`: sets the value for option `ARGUMENT2` to `ARGUMENT3` inside module `ARGUMENT1` 4. `set-option-for-controller`: set the value of option `ARGUMENT1` to `ARGUMENT2` for the controller worker 5. `set-option-for-proxy`: set the value of option `ARGUMENT1` to `ARGUMENT2` for the proxy worker -6. `set-common-option`: set the option `ARGUMENT1` that [defines basic Rspamd behavior][www::rspamd-docs-basic-options] to value `ARGUMENT2` +6. `set-common-option`: set the option `ARGUMENT1` that [defines basic Rspamd behavior][rspamd-docs::basic-options] to value `ARGUMENT2` 7. `add-line`: this will add the complete line after `ARGUMENT1` (with all characters) to the file `/etc/rspamd/override.d/` !!! example "An Example Is [Shown Down Below](#adjusting-and-extending-the-very-basic-configuration)" @@ -231,11 +237,11 @@ There is a dedicated [section for setting up DKIM with Rspamd in our documentati ### _Abusix_ Integration -This subsection provides information about the integration of [Abusix][www::abusix], "a set of blocklists that work as an additional email security layer for your existing mail environment". The setup is straight-forward and well documented: +This subsection provides information about the integration of [Abusix][abusix-web], "a set of blocklists that work as an additional email security layer for your existing mail environment". The setup is straight-forward and well documented: 1. [Create an account](https://app.abusix.com/signup) 2. Retrieve your API key -3. Navigate to the ["Getting Started" documentation for Rspamd][www::abusix-rspamd-integration] and follow the steps described there +3. Navigate to the ["Getting Started" documentation for Rspamd][abusix-docs::rspamd-integration] and follow the steps described there 4. Make sure to change `` to your private API key We recommend mounting the files directly into the container, as they are rather big and not manageable with our [`custom-command.conf` script](#with-the-help-of-a-custom-file). If mounted to the correct location, Rspamd will automatically pick them up. diff --git a/target/rspamd/scores.d/policies_group.conf b/target/rspamd/scores.d/policies_group.conf index 5f9426e9..f2048665 100644 --- a/target/rspamd/scores.d/policies_group.conf +++ b/target/rspamd/scores.d/policies_group.conf @@ -1,72 +1,75 @@ # Please refer to # https://github.com/docker-mailserver/docker-mailserver/issues/3690 # for understanding this file and its scores' values. +# +# This configuration is not 100% compliant with RFC7489. +# This is intentional! Rspamd has additional symbols than those defined in this file. +# 100% compliance is not desirable as those symbols will change the overall spam score. symbols = { # SPF - "R_SPF_ALLOW" { + "R_SPF_ALLOW" { # SPF check succeeded weight = -1; description = "SPF verification allows sending"; groups = ["spf"]; } - "R_SPF_NA" { + "R_SPF_NA" { # SPF is not available for this domain weight = 1.5; description = "Missing SPF record"; one_shot = true; groups = ["spf"]; } - "R_SPF_SOFTFAIL" { - weight = 2.5; - description = "SPF verification soft-failed"; - groups = ["spf"]; - } - "R_SPF_FAIL" { - weight = 4.5; - description = "SPF verification failed"; - groups = ["spf"]; - } - - "R_SPF_NEUTRAL" { # == R_SPF_NA + "R_SPF_NEUTRAL" { # same as R_SPF_NA weight = 1.5; description = "SPF policy is neutral"; groups = ["spf"]; } - "R_SPF_DNSFAIL" { # == R_SPF_SOFTFAIL + "R_SPF_SOFTFAIL" { # there was a temporary DNS issue and SPF could not be checked + weight = 2.5; + description = "SPF verification soft-failed"; + groups = ["spf"]; + } + "R_SPF_DNSFAIL" { # same as R_SPF_SOFTFAIL weight = 2.5; description = "SPF DNS failure"; groups = ["spf"]; } - "R_SPF_PERMFAIL" { # == R_SPF_FAIL + "R_SPF_FAIL" { # SPF check failed + weight = 4.5; + description = "SPF verification failed"; + groups = ["spf"]; + } + "R_SPF_PERMFAIL" { # same as R_SPF_FAIL weight = 4.5; description = "SPF record is malformed or persistent DNS error"; groups = ["spf"]; } # DKIM - "R_DKIM_ALLOW" { + "R_DKIM_ALLOW" { # DKIM check succeeded weight = -1; description = "DKIM verification succeed"; one_shot = true; groups = ["dkim"]; } - "R_DKIM_NA" { - weight = 0; + "R_DKIM_NA" { # DKIM is not available for this domain + weight = 1; description = "Missing DKIM signature"; one_shot = true; groups = ["dkim"]; } - "R_DKIM_TEMPFAIL" { + "R_DKIM_TEMPFAIL" { # there was a temporary DNS issue and DKIM could not be checked weight = 1.5; description = "DKIM verification soft-failed"; groups = ["dkim"]; } - "R_DKIM_PERMFAIL" { + "R_DKIM_PERMFAIL" { # DKIM check failed weight = 4.5; description = "DKIM verification hard-failed (invalid)"; groups = ["dkim"]; } - "R_DKIM_REJECT" { # == R_DKIM_PERMFAIL + "R_DKIM_REJECT" { # same as R_DKIM_PERMFAIL weight = 4.5; description = "DKIM verification failed"; one_shot = true; @@ -74,35 +77,34 @@ symbols = { } # DMARC - "DMARC_NA" { - weight = 1; - description = "No DMARC record"; - groups = ["dmarc"]; - } - "DMARC_POLICY_QUARANTINE" { - weight = 1.5; - description = "DMARC quarantine policy"; - groups = ["dmarc"]; - } - "DMARC_POLICY_REJECT" { - weight = 2; - description = "DMARC reject policy"; - groups = ["dmarc"]; - } - - "DMARC_POLICY_ALLOW" { # no equivalent + "DMARC_POLICY_ALLOW" { # DMARC check succeeded weight = -1; description = "DMARC permit policy"; groups = ["dmarc"]; } - "DMARC_POLICY_ALLOW_WITH_FAILURES" { # no equivalent - weight = -0.5; + "DMARC_POLICY_ALLOW_WITH_FAILURES" { # DMARC check succeeded but either SPF or DKIM was not successful + weight = 0; description = "DMARC permit policy with DKIM/SPF failure"; groups = ["dmarc"]; } - "DMARC_POLICY_SOFTFAIL" { # == DMARC_POLICY_QUARANTINE + "DMARC_NA" { # DMARC is not available for this domain + weight = 0.5; + description = "No DMARC record"; + groups = ["dmarc"]; + } + "DMARC_POLICY_SOFTFAIL" { # there was a temporary DNS issue and DMARC could not be checked weight = 1.5; description = "DMARC soft-failed"; groups = ["dmarc"]; } + "DMARC_POLICY_QUARANTINE" { # DMARC check failed and the policy is to quarantine + weight = 3; + description = "DMARC quarantine policy"; + groups = ["dmarc"]; + } + "DMARC_POLICY_REJECT" { # DMARC check failed and the policy is to reject + weight = 5.5; + description = "DMARC reject policy"; + groups = ["dmarc"]; + } } From 83a48e8958c0fcba9778117f235e6064b401f3a1 Mon Sep 17 00:00:00 2001 From: Casper Date: Sun, 3 Mar 2024 22:48:42 +0100 Subject: [PATCH 14/53] Fail2ban logrotate interval/count: substitute only when necessary (#3919) --- CHANGELOG.md | 2 +- target/scripts/startup/setup.d/security/misc.sh | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 30c7dd0c..b60a7afa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,7 +66,7 @@ The most noteworthy change of this release is the update of the container's base - **Environment Variables:** - `LOGROTATE_COUNT` defines the number of files kept by logrotate ([#3907](https://github.com/docker-mailserver/docker-mailserver/pull/3907)) - - The fail2ban log file is now also taken into account by `LOGROTATE_COUNT` and `LOGROTATE_INTERVAL` ([#3915](https://github.com/docker-mailserver/docker-mailserver/pull/3915)) + - The fail2ban log file is now also taken into account by `LOGROTATE_COUNT` and `LOGROTATE_INTERVAL` ([#3915](https://github.com/docker-mailserver/docker-mailserver/pull/3915), [#3919](https://github.com/docker-mailserver/docker-mailserver/pull/3919)) ### Updates diff --git a/target/scripts/startup/setup.d/security/misc.sh b/target/scripts/startup/setup.d/security/misc.sh index df810b2e..303ae62d 100644 --- a/target/scripts/startup/setup.d/security/misc.sh +++ b/target/scripts/startup/setup.d/security/misc.sh @@ -209,8 +209,8 @@ function __setup__security__fail2ban() { rm -f /etc/logrotate.d/fail2ban fi _log 'trace' 'Configuring fail2ban logrotate rotate count and interval' - sedfile -i "s|rotate 4$|rotate ${LOGROTATE_COUNT}|" /etc/logrotate.d/fail2ban - sedfile -i "s|weekly$|${LOGROTATE_INTERVAL}|" /etc/logrotate.d/fail2ban + [[ ${LOGROTATE_COUNT} -ne 4 ]] && sedfile -i "s|rotate 4$|rotate ${LOGROTATE_COUNT}|" /etc/logrotate.d/fail2ban + [[ ${LOGROTATE_INTERVAL} != "weekly" ]] && sedfile -i "s|weekly$|${LOGROTATE_INTERVAL}|" /etc/logrotate.d/fail2ban } function __setup__security__amavis() { From 0c8d8f26d9323789569cab9ceac33e0b541380e0 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sun, 3 Mar 2024 22:50:06 +0100 Subject: [PATCH 15/53] docs: updated `CONTRIBUTORS.md` (#3916) --- CONTRIBUTORS.md | 356 +++++++++++++++++++++++++----------------------- 1 file changed, 182 insertions(+), 174 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index c387a654..ffe5cecd 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -642,40 +642,19 @@ Thanks goes to these wonderful people ✨ yogo1212 - - - willtho89 -
- willtho89 - - - mpanneck
mpanneck - + + - - ubenmackin + + willtho89
- ubenmackin - - - - - craue -
- craue - - - - - abh -
- abh + willtho89 @@ -685,14 +664,20 @@ Thanks goes to these wonderful people ✨ andrewlow + + + abh +
+ abh + + aminvakil
aminvakil - - + elbracht @@ -700,6 +685,21 @@ Thanks goes to these wonderful people ✨ elbracht + + + craue +
+ craue + + + + + + ubenmackin +
+ ubenmackin + + danielpanteleit @@ -735,49 +735,6 @@ Thanks goes to these wonderful people ✨ emazzotta - - - - nueaf -
- nueaf - - - - - martinwepner -
- martinwepner - - - - - artonge -
- artonge - - - - - spacecowboy -
- spacecowboy - - - - - jedateach -
- jedateach - - - - - millaguie -
- millaguie - - @@ -793,6 +750,13 @@ Thanks goes to these wonderful people ✨ ipernet + + + H4R0 +
+ H4R0 + + eltociear @@ -808,60 +772,53 @@ Thanks goes to these wonderful people ✨ - - H4R0 + + millaguie
- H4R0 - - - - - nilshoell -
- nilshoell + millaguie - - simonsystem + + martinwepner
- simonsystem + martinwepner - - stephan-devop + + artonge
- stephan-devop + artonge - - stigok + + jedateach
- stigok + jedateach - - 5ven + + spacecowboy
- 5ven + spacecowboy - - syl20bnr + + nueaf
- syl20bnr + nueaf - - sylvaindumont + + thomasschmit
- sylvaindumont + thomasschmit @@ -873,10 +830,53 @@ Thanks goes to these wonderful people ✨ - - thomasschmit + + sylvaindumont
- thomasschmit + sylvaindumont + + + + + syl20bnr +
+ syl20bnr + + + + + 5ven +
+ 5ven + + + + + stigok +
+ stigok + + + + + stephan-devop +
+ stephan-devop + + + + + + simonsystem +
+ simonsystem + + + + + radicand +
+ radicand @@ -951,6 +951,13 @@ Thanks goes to these wonderful people ✨ + + + nilshoell +
+ nilshoell + + nknapp @@ -985,15 +992,15 @@ Thanks goes to these wonderful people ✨
mrPjer - + + p3dda
p3dda - - + peter-hartmann @@ -1015,13 +1022,6 @@ Thanks goes to these wonderful people ✨ rahilarious - - - 0xflotus -
- 0xflotus - - remoe @@ -1416,14 +1416,21 @@ Thanks goes to these wonderful people ✨ matrixes + + + 0xflotus +
+ 0xflotus + + + auchri
auchri - - + arkanovicz @@ -1458,15 +1465,15 @@ Thanks goes to these wonderful people ✨
dkarski - + + dbellavista
dbellavista - - + danielvandenberg95 @@ -1495,6 +1502,14 @@ Thanks goes to these wonderful people ✨ mazzz1y + + + doominator42 +
+ doominator42 + + + aydodo @@ -1508,8 +1523,7 @@ Thanks goes to these wonderful people ✨
vedtam - - + edvorg @@ -1537,7 +1551,8 @@ Thanks goes to these wonderful people ✨
ErikEngerd - + + huncode @@ -1551,8 +1566,7 @@ Thanks goes to these wonderful people ✨
felixn - - + flole @@ -1560,13 +1574,6 @@ Thanks goes to these wonderful people ✨ flole - - - froks -
- froks - - ifokeev @@ -1587,15 +1594,15 @@ Thanks goes to these wonderful people ✨
2b - + + askz
askz - - + aspettl @@ -1630,15 +1637,15 @@ Thanks goes to these wonderful people ✨
alexanderneu - + + ch3sh1r
ch3sh1r - - + eglia @@ -1673,15 +1680,15 @@ Thanks goes to these wonderful people ✨
MrFreezeex - + + arunvc
arunvc - - + astrocket @@ -1716,15 +1723,22 @@ Thanks goes to these wonderful people ✨
crash7 - - - - fkefer -
- fkefer - + + + froks +
+ froks + + + + + akkumar +
+ akkumar + + thechubbypanda @@ -1752,7 +1766,8 @@ Thanks goes to these wonderful people ✨
JustAnother1 - + + LeoWinterDE @@ -1766,8 +1781,7 @@ Thanks goes to these wonderful people ✨
linhandev - - + luke- @@ -1795,7 +1809,8 @@ Thanks goes to these wonderful people ✨
madmath03 - + + maxemann96 @@ -1809,8 +1824,7 @@ Thanks goes to these wonderful people ✨
dragetd - - + michaeljensen @@ -1838,7 +1852,8 @@ Thanks goes to these wonderful people ✨
mcchots - + + MohammedNoureldin @@ -1852,8 +1867,7 @@ Thanks goes to these wonderful people ✨
mpldr - - + naveensrinivasan @@ -1869,10 +1883,10 @@ Thanks goes to these wonderful people ✨ - - radicand + + fkefer
- radicand + fkefer @@ -1881,7 +1895,8 @@ Thanks goes to these wonderful people ✨
Marsu31 - + + glandais @@ -1895,8 +1910,7 @@ Thanks goes to these wonderful people ✨
GiovanH - - + harryyoud @@ -1924,7 +1938,8 @@ Thanks goes to these wonderful people ✨
Influencer - + + JacksonZ03 @@ -1938,8 +1953,7 @@ Thanks goes to these wonderful people ✨
JamBalaya56562 - - + jcalfee @@ -1967,7 +1981,8 @@ Thanks goes to these wonderful people ✨
Jeidnx - + + jessp01 @@ -1981,8 +1996,7 @@ Thanks goes to these wonderful people ✨
JiLleON - - + jirislav @@ -2010,20 +2024,14 @@ Thanks goes to these wonderful people ✨
JOduMonT - + + Kaan88
Kaan88 - - - - akkumar -
- akkumar - From 899b644a04cfbfed10d03d67a6864a2ef988ee32 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 4 Mar 2024 18:19:57 +0100 Subject: [PATCH 16/53] chore(deps): Bump docker/setup-buildx-action from 3.0.0 to 3.1.0 (#3924) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/generic_build.yml | 2 +- .github/workflows/generic_publish.yml | 2 +- .github/workflows/generic_test.yml | 2 +- .github/workflows/generic_vulnerability-scan.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/generic_build.yml b/.github/workflows/generic_build.yml index ccef46f5..660b0a2b 100644 --- a/.github/workflows/generic_build.yml +++ b/.github/workflows/generic_build.yml @@ -79,7 +79,7 @@ jobs: platforms: arm64 - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.0.0 + uses: docker/setup-buildx-action@v3.1.0 # NOTE: AMD64 can build within 2 minutes - name: 'Build images' diff --git a/.github/workflows/generic_publish.yml b/.github/workflows/generic_publish.yml index 83357062..bd776cfd 100644 --- a/.github/workflows/generic_publish.yml +++ b/.github/workflows/generic_publish.yml @@ -40,7 +40,7 @@ jobs: platforms: arm64 - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.0.0 + uses: docker/setup-buildx-action@v3.1.0 # Try get the cached build layers from a prior `generic_build.yml` job. # NOTE: Until adopting `type=gha` scoped cache exporter (in `docker/build-push-action`), diff --git a/.github/workflows/generic_test.yml b/.github/workflows/generic_test.yml index 2c1d1045..bc26ef4f 100644 --- a/.github/workflows/generic_test.yml +++ b/.github/workflows/generic_test.yml @@ -38,7 +38,7 @@ jobs: # Ensures consistent BuildKit version (not coupled to Docker Engine), # and increased compatibility of the build cache vs mixing buildx drivers. - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.0.0 + uses: docker/setup-buildx-action@v3.1.0 # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. diff --git a/.github/workflows/generic_vulnerability-scan.yml b/.github/workflows/generic_vulnerability-scan.yml index 78af5ded..f800cfb4 100644 --- a/.github/workflows/generic_vulnerability-scan.yml +++ b/.github/workflows/generic_vulnerability-scan.yml @@ -37,7 +37,7 @@ jobs: # Ensures consistent BuildKit version (not coupled to Docker Engine), # and increased compatibility of the build cache vs mixing buildx drivers. - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.0.0 + uses: docker/setup-buildx-action@v3.1.0 # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. From d227d6dc7368274fa5c2b206e77b33274b4569bc Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Tue, 5 Mar 2024 20:33:04 +1300 Subject: [PATCH 17/53] docs: Reference systemd timer example (`cerbot renew`) (#3921) --- docs/content/config/security/ssl.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/content/config/security/ssl.md b/docs/content/config/security/ssl.md index 1b84c4df..a9174483 100644 --- a/docs/content/config/security/ssl.md +++ b/docs/content/config/security/ssl.md @@ -134,6 +134,8 @@ Certbot provisions certificates to `/etc/letsencrypt`. Add a volume to store the ``` This process can also be [automated via _cron_ or _systemd timers_][certbot::automated-renewal]. + + - [Example with a systemd timer][certbot::automated-renewal::example-systemd-timer] !!! note "Using a different ACME CA" @@ -903,6 +905,7 @@ Despite this, if you must use non-standard DH parameters or you would like to sw [certbot::standalone]: https://certbot.eff.org/docs/using.html#standalone [certbot::renew]: https://certbot.eff.org/docs/using.html#renewing-certificates [certbot::automated-renewal]: https://certbot.eff.org/docs/using.html#automated-renewals +[certbot::automated-renewal::example-systemd-timer]: https://github.com/orgs/docker-mailserver/discussions/3917#discussioncomment-8661690 [certbot::custom-ca]: https://certbot.eff.org/docs/using.htmlchanging-the-acme-server [certbot::webroot]: https://certbot.eff.org/docs/using.html#webroot From b5b193ca4ca114dd14e725ab4980cf011ed58633 Mon Sep 17 00:00:00 2001 From: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Date: Tue, 5 Mar 2024 10:48:49 +0100 Subject: [PATCH 18/53] Rspamd: minor tweaks and follow-up for SPF, DKIM and DMARC symbols (#3923) * move `policies_group.conf` to correct location I originally assumed the file had to be placed into `scores.d`, but I now know that `local.d` is actually correct. * add configuration for composite symbols See updates to #3690: Additional Rspamd Symbols Rspamd has so-called composite symbols that trigger when a condition is met. Especially AUTH_NA and AUTH_NA_OR_FAIL will adjust the scores of various lines in the table above. This needs to be taken into account. * update CHANGELOG --- CHANGELOG.md | 2 +- Dockerfile | 1 - target/rspamd/local.d/composites.conf | 18 ++++++++++++++++++ .../{scores.d => local.d}/policies_group.conf | 0 4 files changed, 19 insertions(+), 2 deletions(-) create mode 100644 target/rspamd/local.d/composites.conf rename target/rspamd/{scores.d => local.d}/policies_group.conf (100%) diff --git a/CHANGELOG.md b/CHANGELOG.md index b60a7afa..1ec71ded 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -80,7 +80,7 @@ The most noteworthy change of this release is the update of the container's base - **Rspamd**: - The `rewrite_subject` action, is now disabled by default. It has been replaced with the new `SPAM_SUBJECT` environment variable, which implements the functionality via a Sieve script instead which is anti-spam service agnostic ([#3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820)) - `RSPAMD_NEURAL` was added and is disabled by default. If switched on it will enable the experimental Rspamd "Neural network" module to add a layer of analysis to spam detection ([#3833](https://github.com/docker-mailserver/docker-mailserver/pull/3833)) - - The symbol weights of SPF, DKIM and DMARC have been adjusted again. Fixes a bug and includes more appropriate combinations of symbols ([#3913](https://github.com/docker-mailserver/docker-mailserver/pull/3913)) + - The symbol weights of SPF, DKIM and DMARC have been adjusted again. Fixes a bug and includes more appropriate combinations of symbols ([#3913](https://github.com/docker-mailserver/docker-mailserver/pull/3913), [#3923](https://github.com/docker-mailserver/docker-mailserver/pull/3923)) ### Fixes diff --git a/Dockerfile b/Dockerfile index 8110cebf..8b2a502c 100644 --- a/Dockerfile +++ b/Dockerfile @@ -104,7 +104,6 @@ EOF # ----------------------------------------------- COPY target/rspamd/local.d/ /etc/rspamd/local.d/ -COPY target/rspamd/scores.d/* /etc/rspamd/scores.d/ # ----------------------------------------------- # --- OAUTH2 ------------------------------------ diff --git a/target/rspamd/local.d/composites.conf b/target/rspamd/local.d/composites.conf new file mode 100644 index 00000000..ab58f45d --- /dev/null +++ b/target/rspamd/local.d/composites.conf @@ -0,0 +1,18 @@ +# In addition to `policies_group.conf`, this file contains +# symbols that are applied when certain other symbols are +# applied (or not applied). +# +# We are especially interested in the `policy` field, because +# there are cases in which `remove_weight` is undesirable. + +# When neither SPF, DKIM, nor DMARC are available, we want +# to increase the base score so we apply at least greylisting. +AUTH_NA { + score = 2.5; + policy = "leave"; +} + +AUTH_NA_OR_FAIL { + score = 1; + policy = "leave"; +} diff --git a/target/rspamd/scores.d/policies_group.conf b/target/rspamd/local.d/policies_group.conf similarity index 100% rename from target/rspamd/scores.d/policies_group.conf rename to target/rspamd/local.d/policies_group.conf From e21e5e04902432079418e96fc5ccba3400dd5582 Mon Sep 17 00:00:00 2001 From: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Date: Wed, 6 Mar 2024 08:44:34 +0100 Subject: [PATCH 19/53] Rspamd: update history key in Redis configuration (#3927) --- CHANGELOG.md | 4 +++- target/scripts/startup/setup.d/security/rspamd.sh | 9 +++++++++ test/tests/parallel/set1/spam_virus/rspamd_full.bats | 12 ++++++++++++ 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ec71ded..e1ba19b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,7 +52,7 @@ The most noteworthy change of this release is the update of the container's base - `smtp_sasl_security_options = noanonymous` (_credentials are mandatory for outbound mail delivery_) - `smtp_tls_security_level = encrypt` (_the outbound MTA connection must always be secure due to credentials sent_) - **Environment Variables**: - - `SA_SPAM_SUBJECT` has been renamed into `SPAM_SUBJECT` to become anti-spam service agnostic. ([3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820)) + - `SA_SPAM_SUBJECT` has been renamed into `SPAM_SUBJECT` to become anti-spam service agnostic. ([#3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820)) - As this functionality is now handled in Dovecot via a Sieve script instead of the respective anti-spam service during Postfix processing, this feature will only apply to mail stored in Dovecot. If you have relied on this feature in a different context, it will no longer be available. - Rspamd previously handled this functionality via the `rewrite_subject` action which as now been disabled by default in favor of the new approach with `SPAM_SUBJECT`. - `SA_SPAM_SUBJECT` is now deprecated and will log a warning if used. The value is copied as a fallback to `SPAM_SUBJECT`. @@ -61,6 +61,8 @@ The most noteworthy change of this release is the update of the container's base - The feature to include [`_SCORE_` tag](https://spamassassin.apache.org/full/4.0.x/doc/Mail_SpamAssassin_Conf.html#rewrite_header-subject-from-to-STRING) in your value to be replaced by the associated spam score is no longer available. - **Supervisord**: - `supervisor-app.conf` renamed to `dms-services.conf` +- **Rspamd**: + - the Redis history key has been changed in order to not incorporate the hostname of the container (which is desirable in Kubernetes environments) ([#3927](https://github.com/docker-mailserver/docker-mailserver/pull/3927)) ### Added diff --git a/target/scripts/startup/setup.d/security/rspamd.sh b/target/scripts/startup/setup.d/security/rspamd.sh index 6b54addd..702a27ee 100644 --- a/target/scripts/startup/setup.d/security/rspamd.sh +++ b/target/scripts/startup/setup.d/security/rspamd.sh @@ -126,6 +126,15 @@ function __rspamd__setup_redis() { servers = "127.0.0.1:6379"; expand_keys = true; +EOF + + # We do not use `{{HOSTNAME}}` but only `{{COMPRES}}` to better support + # Kubernetes, see https://github.com/orgs/docker-mailserver/discussions/3922 + cat >"${RSPAMD_LOCAL_D}/history_redis.conf" << "EOF" +# documentation: https://rspamd.com/doc/modules/history_redis.html + +key_prefix = "rs_history{{COMPRESS}}"; + EOF # Here we adjust the Redis default configuration that we supply to Redis when starting it. diff --git a/test/tests/parallel/set1/spam_virus/rspamd_full.bats b/test/tests/parallel/set1/spam_virus/rspamd_full.bats index 9b14860b..32891a27 100644 --- a/test/tests/parallel/set1/spam_virus/rspamd_full.bats +++ b/test/tests/parallel/set1/spam_virus/rspamd_full.bats @@ -86,6 +86,18 @@ function teardown_file() { _default_teardown ; } refute_line --regexp 'rewrite_subject = [0-9]+;' } +@test 'Rspamd Redis configuration is correct' { + _run_in_container rspamadm configdump redis + assert_success + assert_line 'expand_keys = true;' + assert_line 'servers = "127.0.0.1:6379";' + + _run_in_container rspamadm configdump history_redis + assert_success + assert_line 'compress = true;' + assert_line 'key_prefix = "rs_history{{COMPRESS}}";' +} + @test "contents of '/etc/rspamd/override.d/' are copied" { local OVERRIDE_D='/etc/rspamd/override.d' _file_exists_in_container "${OVERRIDE_D}/testmodule_complicated.conf" From 364969919757216f97a205cf05e98dfae0c7c924 Mon Sep 17 00:00:00 2001 From: Kirill Kirilenko Date: Thu, 7 Mar 2024 01:13:22 +0300 Subject: [PATCH 20/53] fix: Move spam to mailbox associated to the `\Junk` special-use attribute (#3925) Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com> --- CHANGELOG.md | 4 ++++ docs/content/config/environment.md | 8 ++++--- target/dovecot/90-sieve.conf | 2 +- .../scripts/startup/setup.d/security/misc.sh | 4 ++-- test/config/junk-mailbox/user-patches.sh | 21 +++++++++++++++++++ .../parallel/set1/spam_virus/rspamd_full.bats | 2 +- .../set1/spam_virus/spam_junk_folder.bats | 13 ++++++++---- 7 files changed, 43 insertions(+), 11 deletions(-) create mode 100755 test/config/junk-mailbox/user-patches.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index e1ba19b4..de5642bf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,6 +36,10 @@ The most noteworthy change of this release is the update of the container's base - DMS `main.cf` has renamed `postscreen_dnsbl_whitelist_threshold` to `postscreen_dnsbl_allowlist_threshold` as part of this change. - `smtpd_relay_restrictions` (relay policy) is now evaluated after `smtpd_recipient_restrictions` (spam policy). Previously it was evaluated before `smtpd_recipient_restrictions`. Mail to be relayed via DMS must now pass through the spam policy first. - The TLS fingerprint policy has changed the default from MD5 to SHA256 (_DMS does not modify this Postfix parameter, but may affect any user customizations that do_). +- **Dovecot** + - The "Junk" mailbox (folder) is now referenced by it's [special-use flag `\Junk`](https://docker-mailserver.github.io/docker-mailserver/v13.3/examples/use-cases/imap-folders/) instead of an explicit mailbox. ([#3925](https://github.com/docker-mailserver/docker-mailserver/pull/3925)) + - This provides compatibility for the Junk mailbox when it's folder name differs (_eg: Renamed to "Spam"_). + - Potential breakage if your deployment modifies our `spam_to_junk.sieve` sieve script (_which is created during container startup when ENV `MOVE_SPAM_TO_JUNK=1`_) that handles storing spam mail into a users "Junk" mailbox folder. - **rsyslog:** - Debian 12 adjusted the `rsyslog` configuration for the default file template from `RSYSLOG_TraditionalFileFormat` to `RSYSLOG_FileFormat` (_upstream default since 2012_). This change may affect you if you have any monitoring / analysis of log output (_eg: `mail.log` / `docker logs`_). - The two formats are roughly equivalent to [RFC 3164](https://www.rfc-editor.org/rfc/rfc3164)) and [RFC 5424](https://datatracker.ietf.org/doc/html/rfc5424#section-1) respectively. diff --git a/docs/content/config/environment.md b/docs/content/config/environment.md index 83fcdfa6..9adfd5d9 100644 --- a/docs/content/config/environment.md +++ b/docs/content/config/environment.md @@ -328,10 +328,12 @@ Note: More information at ##### MOVE_SPAM_TO_JUNK -- 0 => Spam messages will be delivered in the mailbox. -- **1** => Spam messages will be delivered in the `Junk` folder. +- 0 => Spam messages will be delivered to the inbox. +- **1** => Spam messages will be delivered to the Junk mailbox. -Routes mail identified as spam into the recipient(s) Junk folder (_via a Dovecot Sieve script_). +Routes mail identified as spam into the recipient(s) Junk mailbox (_a specialized folder for junk associated to the [special-use flag `\Junk`][docs::dovecot::special-use-flag], handled via a Dovecot sieve script internally_). + +[docs::dovecot::special-use-flag]: ../examples/use-cases/imap-folders.md !!! info diff --git a/target/dovecot/90-sieve.conf b/target/dovecot/90-sieve.conf index 8b559be2..f388009a 100644 --- a/target/dovecot/90-sieve.conf +++ b/target/dovecot/90-sieve.conf @@ -51,7 +51,7 @@ plugin { # deprecated imapflags extension in addition to all extensions were already # enabled by default. #sieve_extensions = +notify +imapflags - sieve_extensions = +notify +imapflags +vnd.dovecot.pipe +vnd.dovecot.filter + sieve_extensions = +notify +imapflags +special-use +vnd.dovecot.pipe +vnd.dovecot.filter # Which Sieve language extensions are ONLY available in global scripts. This # can be used to restrict the use of certain Sieve extensions to administrator diff --git a/target/scripts/startup/setup.d/security/misc.sh b/target/scripts/startup/setup.d/security/misc.sh index 303ae62d..bb460716 100644 --- a/target/scripts/startup/setup.d/security/misc.sh +++ b/target/scripts/startup/setup.d/security/misc.sh @@ -304,11 +304,11 @@ function _setup_spam_to_junk() { _log 'debug' 'Spam emails will be moved to the Junk folder' mkdir -p /usr/lib/dovecot/sieve-global/after/ cat >/usr/lib/dovecot/sieve-global/after/spam_to_junk.sieve << EOF -require ["fileinto","mailbox"]; +require ["fileinto","special-use"]; if anyof (header :contains "X-Spam-Flag" "YES", header :contains "X-Spam" "Yes") { - fileinto "Junk"; + fileinto :specialuse "\\\\Junk" "Junk"; } EOF sievec /usr/lib/dovecot/sieve-global/after/spam_to_junk.sieve diff --git a/test/config/junk-mailbox/user-patches.sh b/test/config/junk-mailbox/user-patches.sh new file mode 100755 index 00000000..0a16a21f --- /dev/null +++ b/test/config/junk-mailbox/user-patches.sh @@ -0,0 +1,21 @@ +#!/bin/bash +## +# This user script will be executed between configuration and starting daemons +# To enable it you must save it in your config directory as "user-patches.sh" +## + +echo "[user-patches.sh] Adjusting 'Junk' mailbox name to verify delivery to Junk mailbox based on special-use flag instead of mailbox's name" + +sed -i -e 's/mailbox Junk/mailbox Spam/' /etc/dovecot/conf.d/15-mailboxes.conf + +### Before / After ### + +# mailbox Junk { +# auto = subscribe +# special_use = \Junk +# } + +# mailbox Spam { +# auto = subscribe +# special_use = \Junk +# } diff --git a/test/tests/parallel/set1/spam_virus/rspamd_full.bats b/test/tests/parallel/set1/spam_virus/rspamd_full.bats index 32891a27..3bc9fb10 100644 --- a/test/tests/parallel/set1/spam_virus/rspamd_full.bats +++ b/test/tests/parallel/set1/spam_virus/rspamd_full.bats @@ -244,7 +244,7 @@ function teardown_file() { _default_teardown ; } _service_log_should_contain_string 'rspamd' 'add header "Gtube pattern"' _print_mail_log_for_msgid 'rspamd-test-email-header' - assert_output --partial "fileinto action: stored mail into mailbox 'Junk'" + assert_output --partial "fileinto action: stored mail into mailbox [SPECIAL-USE \\Junk]" _count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 2 _count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/.Junk/new/ 1 diff --git a/test/tests/parallel/set1/spam_virus/spam_junk_folder.bats b/test/tests/parallel/set1/spam_virus/spam_junk_folder.bats index 0c873382..5590d6f7 100644 --- a/test/tests/parallel/set1/spam_virus/spam_junk_folder.bats +++ b/test/tests/parallel/set1/spam_virus/spam_junk_folder.bats @@ -49,9 +49,11 @@ function teardown() { _default_teardown ; } _should_receive_spam_at '/var/mail/localhost.localdomain/user1/new/' } -@test "(enabled + MOVE_SPAM_TO_JUNK=1) should deliver spam message into Junk folder" { +@test "(enabled + MOVE_SPAM_TO_JUNK=1) should deliver spam message into Junk mailbox" { export CONTAINER_NAME=${CONTAINER3_NAME} + _init_with_defaults + local CUSTOM_SETUP_ARGUMENTS=( --env ENABLE_AMAVIS=1 --env ENABLE_SPAMASSASSIN=1 @@ -60,14 +62,17 @@ function teardown() { _default_teardown ; } --env MOVE_SPAM_TO_JUNK=1 --env PERMIT_DOCKER=container ) - _init_with_defaults + + # Adjust 'Junk' mailbox name to verify delivery to Junk mailbox based on special-use flag instead of mailbox's name + mv "${TEST_TMP_CONFIG}/junk-mailbox/user-patches.sh" "${TEST_TMP_CONFIG}/" + _common_container_setup 'CUSTOM_SETUP_ARGUMENTS' _should_send_spam_message _should_be_received_by_amavis 'Passed SPAM {RelayedTaggedInbound,Quarantined}' - # Should move delivered spam to Junk folder - _should_receive_spam_at '/var/mail/localhost.localdomain/user1/.Junk/new/' + # Should move delivered spam to the Junk mailbox (adjusted to be located at '.Spam/') + _should_receive_spam_at '/var/mail/localhost.localdomain/user1/.Spam/new/' } # NOTE: Same as test for `CONTAINER3_NAME`, only differing by ENV `MARK_SPAM_AS_READ=1` + `_should_receive_spam_at` location From 267fc552d220414be9f8ba7dbb2f0ed2431f3228 Mon Sep 17 00:00:00 2001 From: Casper Date: Sat, 9 Mar 2024 14:21:02 +0100 Subject: [PATCH 21/53] getmail: remove temp file usage (#3920) --- target/scripts/startup/setup.d/getmail.sh | 8 ++++---- target/scripts/startup/setup.d/security/rspamd.sh | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/target/scripts/startup/setup.d/getmail.sh b/target/scripts/startup/setup.d/getmail.sh index a6c304c6..6f280118 100644 --- a/target/scripts/startup/setup.d/getmail.sh +++ b/target/scripts/startup/setup.d/getmail.sh @@ -18,10 +18,10 @@ function _setup_getmail() { CONFIGS=1 ID=$(cut -d '-' -f 3 <<< "${FILE}" | cut -d '.' -f 1) local GETMAIL_CONFIG="${GETMAILRC}/getmailrc-${ID}" - cat /etc/getmailrc_general >"${GETMAIL_CONFIG}.tmp" - echo -e "message_log = /var/log/mail/getmail-${ID}.log\n" >>"${GETMAIL_CONFIG}.tmp" - cat "${GETMAIL_CONFIG}.tmp" "${FILE}" >"${GETMAIL_CONFIG}" - rm "${GETMAIL_CONFIG}.tmp" + + cat /etc/getmailrc_general >"${GETMAIL_CONFIG}" + echo -e "message_log = /var/log/mail/getmail-${ID}.log\n" >>"${GETMAIL_CONFIG}" + cat "${FILE}" >>"${GETMAIL_CONFIG}" fi done diff --git a/target/scripts/startup/setup.d/security/rspamd.sh b/target/scripts/startup/setup.d/security/rspamd.sh index 702a27ee..37916282 100644 --- a/target/scripts/startup/setup.d/security/rspamd.sh +++ b/target/scripts/startup/setup.d/security/rspamd.sh @@ -128,7 +128,7 @@ expand_keys = true; EOF - # We do not use `{{HOSTNAME}}` but only `{{COMPRES}}` to better support + # We do not use `{{HOSTNAME}}` but only `{{COMPRESS}}` to better support # Kubernetes, see https://github.com/orgs/docker-mailserver/discussions/3922 cat >"${RSPAMD_LOCAL_D}/history_redis.conf" << "EOF" # documentation: https://rspamd.com/doc/modules/history_redis.html From 8bdda5f43325c826f5e0bd42bf74d19bb83e0a8e Mon Sep 17 00:00:00 2001 From: Ikko Eltociear Ashimine Date: Mon, 11 Mar 2024 20:02:22 +0900 Subject: [PATCH 22/53] Update user-patches.sh (#3932) --- test/config/rspamd_full/user-patches.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/config/rspamd_full/user-patches.sh b/test/config/rspamd_full/user-patches.sh index 9f4dc9fb..f731ff36 100644 --- a/test/config/rspamd_full/user-patches.sh +++ b/test/config/rspamd_full/user-patches.sh @@ -8,7 +8,7 @@ echo "enable_test_patterns = true;" >>/etc/rspamd/local.d/options.inc # We want Dovecot to be very detailed about what it is doing, -# specificially for Sieve because we need to check whether the +# specifically for Sieve because we need to check whether the # Sieve scripts are executed so Rspamd is trained when using # `RSPAMD_LEARN=1`. echo 'mail_debug = yes' >>/etc/dovecot/dovecot.conf From 9bc8869715f1d431274067c8b830d72cf861924e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 11 Mar 2024 19:40:48 +0100 Subject: [PATCH 23/53] chore(deps): Bump docker/build-push-action from 5.1.0 to 5.2.0 (#3934) --- .github/workflows/generic_build.yml | 2 +- .github/workflows/generic_publish.yml | 2 +- .github/workflows/generic_test.yml | 2 +- .github/workflows/generic_vulnerability-scan.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/generic_build.yml b/.github/workflows/generic_build.yml index 660b0a2b..aa3051c8 100644 --- a/.github/workflows/generic_build.yml +++ b/.github/workflows/generic_build.yml @@ -83,7 +83,7 @@ jobs: # NOTE: AMD64 can build within 2 minutes - name: 'Build images' - uses: docker/build-push-action@v5.1.0 + uses: docker/build-push-action@v5.2.0 with: context: . # Build at least the AMD64 image (which runs against the test suite). diff --git a/.github/workflows/generic_publish.yml b/.github/workflows/generic_publish.yml index bd776cfd..a729e5b8 100644 --- a/.github/workflows/generic_publish.yml +++ b/.github/workflows/generic_publish.yml @@ -67,7 +67,7 @@ jobs: password: ${{ secrets.GITHUB_TOKEN }} - name: 'Build and publish images' - uses: docker/build-push-action@v5.1.0 + uses: docker/build-push-action@v5.2.0 with: context: . build-args: | diff --git a/.github/workflows/generic_test.yml b/.github/workflows/generic_test.yml index bc26ef4f..73db6f6c 100644 --- a/.github/workflows/generic_test.yml +++ b/.github/workflows/generic_test.yml @@ -43,7 +43,7 @@ jobs: # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. - name: 'Build AMD64 image from cache' - uses: docker/build-push-action@v5.1.0 + uses: docker/build-push-action@v5.2.0 with: context: . tags: mailserver-testing:ci diff --git a/.github/workflows/generic_vulnerability-scan.yml b/.github/workflows/generic_vulnerability-scan.yml index f800cfb4..8ca0b8b2 100644 --- a/.github/workflows/generic_vulnerability-scan.yml +++ b/.github/workflows/generic_vulnerability-scan.yml @@ -42,7 +42,7 @@ jobs: # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. - name: 'Build AMD64 image from cache' - uses: docker/build-push-action@v5.1.0 + uses: docker/build-push-action@v5.2.0 with: context: . tags: mailserver-testing:ci From a04b53f4f8ba39217739043059a5272e2ba84fd6 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 12 Mar 2024 11:46:34 +1300 Subject: [PATCH 24/53] chore(deps): Bump nwtgck/actions-netlify from 2.1 to 3.0 (#3933) Bumps [nwtgck/actions-netlify](https://github.com/nwtgck/actions-netlify) from 2.1 to 3.0. - [Release notes](https://github.com/nwtgck/actions-netlify/releases) - [Changelog](https://github.com/nwtgck/actions-netlify/blob/develop/CHANGELOG.md) - [Commits](https://github.com/nwtgck/actions-netlify/compare/v2.1...v3.0) --- updated-dependencies: - dependency-name: nwtgck/actions-netlify dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/docs-preview-deploy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/docs-preview-deploy.yml b/.github/workflows/docs-preview-deploy.yml index dbe57752..7c924f86 100644 --- a/.github/workflows/docs-preview-deploy.yml +++ b/.github/workflows/docs-preview-deploy.yml @@ -55,7 +55,7 @@ jobs: context: 'Deploy Preview (pull_request => workflow_run)' - name: 'Send preview build to Netlify' - uses: nwtgck/actions-netlify@v2.1 + uses: nwtgck/actions-netlify@v3.0 id: preview timeout-minutes: 1 env: From 2133b51e782ae207475788a9b168b0008a5b450b Mon Sep 17 00:00:00 2001 From: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Date: Tue, 12 Mar 2024 09:31:44 +0100 Subject: [PATCH 25/53] docs: rewrite Kubernetes page (#3928) --- docs/content/assets/css/customizations.css | 34 + docs/content/config/advanced/kubernetes.md | 1192 ++++++++++------- .../tutorials/mailserver-behind-proxy.md | 7 +- docs/mkdocs.yml | 5 + 4 files changed, 778 insertions(+), 460 deletions(-) diff --git a/docs/content/assets/css/customizations.css b/docs/content/assets/css/customizations.css index 25cb0274..7255a837 100644 --- a/docs/content/assets/css/customizations.css +++ b/docs/content/assets/css/customizations.css @@ -107,3 +107,37 @@ div.md-content article.md-content__inner a.toclink code { .md-nav__item--nested > .md-nav__link { font-weight: 700; } + +/* ============================================================================================================= */ + +/* + TaskList style for a pro/con list. Presently only used for this type of list in the kubernetes docs. + Uses a custom icon for the unchecked (con) state: :octicons-x-circle-fill-24: + https://github.com/squidfunk/mkdocs-material/discussions/6811#discussioncomment-8700795 + + TODO: Can better scope the style under a class name when migrating to block extension syntax: + https://github.com/facelessuser/pymdown-extensions/discussions/1973 +*/ + +:root { + --md-tasklist-icon--failed: url('data:image/svg+xml;charset=utf-8,'); +} + +.md-typeset [type="checkbox"] + .task-list-indicator::before { + background-color: rgb(216, 87, 48); + -webkit-mask-image: var(--md-tasklist-icon--failed); + mask-image: var(--md-tasklist-icon--failed); +} + +/* More suitable shade of green */ +.md-typeset [type=checkbox]:checked+.task-list-indicator:before { + background-color: rgb(97, 216, 42); +} + +/* Tiny layout shift */ +[dir=ltr] .md-typeset .task-list-indicator:before { + left: -1.6em; + top: 1px; +} + +/* ============================================================================================================= */ diff --git a/docs/content/config/advanced/kubernetes.md b/docs/content/config/advanced/kubernetes.md index c277852a..97a7e414 100644 --- a/docs/content/config/advanced/kubernetes.md +++ b/docs/content/config/advanced/kubernetes.md @@ -4,518 +4,794 @@ title: 'Advanced | Kubernetes' ## Introduction -This article describes how to deploy DMS to Kubernetes. Please note that there is also a [Helm chart] available. +This article describes how to deploy DMS to Kubernetes. We highly recommend everyone to use our community [DMS Helm chart][github-web::docker-mailserver-helm]. -!!! attention "Requirements" +!!! note "Requirements" - We assume basic knowledge about Kubernetes from the reader. Moreover, we assume the reader to have a basic understanding of mail servers. Ideally, the reader has deployed DMS before in an easier setup with Docker (Compose). + 1. Basic knowledge about Kubernetes from the reader. + 2. A basic understanding of mail servers. + 3. Ideally, the reader has already deployed DMS before with a simpler setup (_`docker run` or Docker Compose_). -!!! warning "About Support for Kubernetes" +!!! warning "Limited Support" - Please note that Kubernetes **is not** officially supported and we do not build images specifically designed for it. When opening an issue, please remember that only Docker & Docker Compose are officially supported. + DMS **does not officially support Kubernetes**. This content is entirely community-supported. If you find errors, please open an issue and raise a PR. - This content is entirely community-supported. If you find errors, please open an issue and provide a PR. +## Manually Writing Manifests -## Manifests +If using our Helm chart is not viable for you, here is some guidance to start with your own manifests. -### Configuration + +!!! quote "" -We want to provide the basic configuration in the form of environment variables with a `ConfigMap`. Note that this is just an example configuration; tune the `ConfigMap` to your needs. + === "`ConfigMap`" -```yaml ---- -apiVersion: v1 -kind: ConfigMap + Provide the basic configuration via environment variables with a `ConfigMap`. + + !!! example -metadata: - name: mailserver.environment + Below is only an example configuration, adjust the `ConfigMap` to your own needs. -immutable: false + ```yaml + --- + apiVersion: v1 + kind: ConfigMap -data: - TLS_LEVEL: modern - POSTSCREEN_ACTION: drop - OVERRIDE_HOSTNAME: mail.example.com - FAIL2BAN_BLOCKTYPE: drop - POSTMASTER_ADDRESS: postmaster@example.com - UPDATE_CHECK_INTERVAL: 10d - POSTFIX_INET_PROTOCOLS: ipv4 - ENABLE_CLAMAV: '1' - ENABLE_POSTGREY: '0' - ENABLE_FAIL2BAN: '1' - AMAVIS_LOGLEVEL: '-1' - SPOOF_PROTECTION: '1' - MOVE_SPAM_TO_JUNK: '1' - ENABLE_UPDATE_CHECK: '1' - ENABLE_SPAMASSASSIN: '1' - SUPERVISOR_LOGLEVEL: warn - SPAMASSASSIN_SPAM_TO_INBOX: '1' + metadata: + name: mailserver.environment - # here, we provide an example for the SSL configuration - SSL_TYPE: manual - SSL_CERT_PATH: /secrets/ssl/rsa/tls.crt - SSL_KEY_PATH: /secrets/ssl/rsa/tls.key -``` + immutable: false -We can also make use of user-provided configuration files, e.g. `user-patches.sh`, `postfix-accounts.cf` and more, to adjust DMS to our likings. We encourage you to have a look at [Kustomize][kustomize] for creating `ConfigMap`s from multiple files, but for now, we will provide a simple, hand-written example. This example is absolutely minimal and only goes to show what can be done. + data: + TLS_LEVEL: modern + POSTSCREEN_ACTION: drop + OVERRIDE_HOSTNAME: mail.example.com + FAIL2BAN_BLOCKTYPE: drop + POSTMASTER_ADDRESS: postmaster@example.com + UPDATE_CHECK_INTERVAL: 10d + POSTFIX_INET_PROTOCOLS: ipv4 + ENABLE_CLAMAV: '1' + ENABLE_POSTGREY: '0' + ENABLE_FAIL2BAN: '1' + AMAVIS_LOGLEVEL: '-1' + SPOOF_PROTECTION: '1' + MOVE_SPAM_TO_JUNK: '1' + ENABLE_UPDATE_CHECK: '1' + ENABLE_SPAMASSASSIN: '1' + SUPERVISOR_LOGLEVEL: warn + SPAMASSASSIN_SPAM_TO_INBOX: '1' -```yaml ---- -apiVersion: v1 -kind: ConfigMap - -metadata: - name: mailserver.files + # here, we provide an example for the SSL configuration + SSL_TYPE: manual + SSL_CERT_PATH: /secrets/ssl/rsa/tls.crt + SSL_KEY_PATH: /secrets/ssl/rsa/tls.key + ``` -data: - postfix-accounts.cf: | - test@example.com|{SHA512-CRYPT}$6$someHashValueHere - other@example.com|{SHA512-CRYPT}$6$someOtherHashValueHere -``` - -!!! attention "Static Configuration" - - With the configuration shown above, you can **not** dynamically add accounts as the configuration file mounted into the mail server can not be written to. - - Use persistent volumes for production deployments. - -### Persistence - -Thereafter, we need persistence for our data. Make sure you have a storage provisioner and that you choose the correct `storageClassName`. - -```yaml ---- -apiVersion: v1 -kind: PersistentVolumeClaim - -metadata: - name: data - -spec: - storageClassName: local-path - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 25Gi -``` - -### Service - -A `Service` is required for getting the traffic to the pod itself. The service is somewhat crucial. Its configuration determines whether the original IP from the sender will be kept. [More about this further down below](#exposing-your-mail-server-to-the-outside-world). - -The configuration you're seeing does keep the original IP, but you will not be able to scale this way. We have chosen to go this route in this case because we think most Kubernetes users will only want to have one instance. - -```yaml ---- -apiVersion: v1 -kind: Service - -metadata: - name: mailserver - labels: - app: mailserver - -spec: - type: LoadBalancer - - selector: - app: mailserver - - ports: - # Transfer - - name: transfer - port: 25 - targetPort: transfer - protocol: TCP - # ESMTP with implicit TLS - - name: esmtp-implicit - port: 465 - targetPort: esmtp-implicit - protocol: TCP - # ESMTP with explicit TLS (STARTTLS) - - name: esmtp-explicit - port: 587 - targetPort: esmtp-explicit - protocol: TCP - # IMAPS with implicit TLS - - name: imap-implicit - port: 993 - targetPort: imap-implicit - protocol: TCP - -``` - -### Deployments - -Last but not least, the `Deployment` becomes the most complex component. It instructs Kubernetes how to run the DMS container and how to apply your `ConfigMaps`, persisted storage, etc. Additionally, we can set options to enforce runtime security here. - -```yaml ---- -apiVersion: apps/v1 -kind: Deployment - -metadata: - name: mailserver - - annotations: - ignore-check.kube-linter.io/run-as-non-root: >- - 'mailserver' needs to run as root - ignore-check.kube-linter.io/privileged-ports: >- - 'mailserver' needs privileged ports - ignore-check.kube-linter.io/no-read-only-root-fs: >- - There are too many files written to make The - root FS read-only - -spec: - replicas: 1 - selector: - matchLabels: - app: mailserver - - template: - metadata: - labels: - app: mailserver - - annotations: - container.apparmor.security.beta.kubernetes.io/mailserver: runtime/default - - spec: - hostname: mail - containers: - - name: mailserver - image: ghcr.io/docker-mailserver/docker-mailserver:latest - imagePullPolicy: IfNotPresent - - securityContext: - # Required to support SGID via `postdrop` executable - # in `/var/mail-state` for Postfix (maildrop + public dirs): - # https://github.com/docker-mailserver/docker-mailserver/pull/3625 - allowPrivilegeEscalation: true - readOnlyRootFilesystem: false - runAsUser: 0 - runAsGroup: 0 - runAsNonRoot: false - privileged: false - capabilities: - add: - # file permission capabilities - - CHOWN - - FOWNER - - MKNOD - - SETGID - - SETUID - - DAC_OVERRIDE - # network capabilities - - NET_ADMIN # needed for F2B - - NET_RAW # needed for F2B - - NET_BIND_SERVICE - # miscellaneous capabilities - - SYS_CHROOT - - KILL - drop: [ALL] - seccompProfile: - type: RuntimeDefault - - # You want to tune this to your needs. If you disable ClamAV, - # you can use less RAM and CPU. This becomes important in - # case you're low on resources and Kubernetes refuses to - # schedule new pods. - resources: - limits: - memory: 4Gi - cpu: 1500m - requests: - memory: 2Gi - cpu: 600m - - volumeMounts: - - name: files - subPath: postfix-accounts.cf - mountPath: /tmp/docker-mailserver/postfix-accounts.cf - readOnly: true - - # PVCs - - name: data - mountPath: /var/mail - subPath: data - readOnly: false - - name: data - mountPath: /var/mail-state - subPath: state - readOnly: false - - name: data - mountPath: /var/log/mail - subPath: log - readOnly: false - - # certificates - - name: certificates-rsa - mountPath: /secrets/ssl/rsa/ - readOnly: true - - # other - - name: tmp-files - mountPath: /tmp - readOnly: false - - ports: - - name: transfer - containerPort: 25 - protocol: TCP - - name: esmtp-implicit - containerPort: 465 - protocol: TCP - - name: esmtp-explicit - containerPort: 587 - - name: imap-implicit - containerPort: 993 - protocol: TCP - - envFrom: - - configMapRef: - name: mailserver.environment - - restartPolicy: Always - - volumes: - # configuration files - - name: files - configMap: - name: mailserver.files - - # PVCs - - name: data - persistentVolumeClaim: - claimName: data - - # certificates - - name: certificates-rsa - secret: - secretName: mail-tls-certificate-rsa - items: - - key: tls.key - path: tls.key - - key: tls.crt - path: tls.crt - - # other - - name: tmp-files - emptyDir: {} -``` - -### Certificates - An Example - -In this example, we use [`cert-manager`][cert-manager] to supply RSA certificates. You can also supply RSA certificates as fallback certificates, which DMS supports out of the box with `SSL_ALT_CERT_PATH` and `SSL_ALT_KEY_PATH`, and provide ECDSA as the proper certificates. - -```yaml ---- -apiVersion: cert-manager.io/v1 -kind: Certificate - -metadata: - name: mail-tls-certificate-rsa - -spec: - secretName: mail-tls-certificate-rsa - isCA: false - privateKey: - algorithm: RSA - encoding: PKCS1 - size: 2048 - dnsNames: [mail.example.com] - issuerRef: - name: mail-issuer - kind: Issuer -``` - -!!! attention - - You will need to have [`cert-manager`][cert-manager] configured. Especially the issue will need to be configured. Since we do not know how you want or need your certificates to be supplied, we do not provide more configuration here. The documentation for [`cert-manager`][cert-manager] is excellent. - -### Sensitive Data - -!!! attention "Sensitive Data" - - For storing OpenDKIM keys, TLS certificates or any sort of sensitive data, you should be using `Secret`s. You can mount secrets like `ConfigMap`s and use them the same way. - -The [TLS docs page][docs-tls] provides guidance when it comes to certificates and transport layer security. Always provide sensitive information vai `Secrets`. + You can also make use of user-provided configuration files (_e.g. `user-patches.sh`, `postfix-accounts.cf`, etc_), to customize DMS to your needs. + + ??? example "Providing config files" + + Here is a minimal example that supplies a `postfix-accounts.cf` file inline with two users: + + ```yaml + --- + apiVersion: v1 + kind: ConfigMap + + metadata: + name: mailserver.files + + data: + postfix-accounts.cf: | + test@example.com|{SHA512-CRYPT}$6$someHashValueHere + other@example.com|{SHA512-CRYPT}$6$someOtherHashValueHere + ``` + + !!! warning "Static Configuration" + + The inline `postfix-accounts.cf` config example above provides file content that is static. It is mounted as read-only at runtime, thus cannot support modifications. + + For production deployments, use persistent volumes instead (via `PersistentVolumeClaim`). That will enable files like `postfix-account.cf` to add and remove accounts, while also persisting those changes externally from the container. + + !!! tip "Modularize your `ConfigMap`" + + [Kustomize][kustomize] can be a useful tool as it supports creating a `ConfigMap` from multiple files. + + === "`PersistentVolumeClaim`" + + To persist data externally from the DMS container, configure a `PersistentVolumeClaim` (PVC). + + Make sure you have a storage system (like Longhorn, Rook, etc.) and that you choose the correct `storageClassName` (according to your storage system). + + !!! example + + ```yaml + --- + apiVersion: v1 + kind: PersistentVolumeClaim + + metadata: + name: data + + spec: + storageClassName: local-path + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 25Gi + ``` + + === "`Service`" + + A [`Service`][k8s-docs::config::service] is required for getting the traffic to the pod itself. It configures a load balancer with the ports you'll need. + + The configuration for a `Service` affects if the original IP from a connecting client is preserved (_this is important_). [More about this further down below](#exposing-your-mail-server-to-the-outside-world). + + !!! example + + ```yaml + --- + apiVersion: v1 + kind: Service + + metadata: + name: mailserver + labels: + app: mailserver + + spec: + type: LoadBalancer + + selector: + app: mailserver + + ports: + # smtp + - name: smtp + port: 25 + targetPort: smtp + protocol: TCP + # submissions (ESMTP with implicit TLS) + - name: submission + port: 465 + targetPort: submissions + protocol: TCP + # submission (ESMTP with explicit TLS) + - name: submission + port: 587 + targetPort: submission + protocol: TCP + # imaps (implicit TLS) + - name: imaps + port: 993 + targetPort: imaps + protocol: TCP + ``` + + === "`Certificate`" + + !!! example "Using [`cert-manager`][cert-manager] to supply TLS certificates" + + ```yaml + --- + apiVersion: cert-manager.io/v1 + kind: Certificate + + metadata: + name: mail-tls-certificate-rsa + + spec: + secretName: mail-tls-certificate-rsa + isCA: false + privateKey: + algorithm: RSA + encoding: PKCS1 + size: 2048 + dnsNames: [mail.example.com] + issuerRef: + name: mail-issuer + kind: Issuer + ``` + + The [TLS docs page][docs-tls] provides guidance when it comes to certificates and transport layer security. + + !!! tip "ECDSA + RSA (fallback)" + + You could supply RSA certificates as fallback certificates instead, with ECDSA as the primary. DMS supports dual certificates via the ENV `SSL_ALT_CERT_PATH` and `SSL_ALT_KEY_PATH`. + + !!! warning "Always provide sensitive information via a `Secret`" + + For storing OpenDKIM keys, TLS certificates, or any sort of sensitive data - you should be using `Secret`s. + + A `Secret` is similar to `ConfigMap`, it can be used and mounted as a volume as demonstrated in the [`Deployment` manifest][docs::k8s::config-deployment] tab. + + === "`Deployment`" + + The [`Deployment`][k8s-docs::config::deployment] config is the most complex component. + + - It instructs Kubernetes how to run the DMS container and how to apply your `ConfigMap`s, persisted storage, etc. + - Additional options can be set to enforce runtime security. + + ???+ example + + ```yaml + --- + apiVersion: apps/v1 + kind: Deployment + + metadata: + name: mailserver + + annotations: + ignore-check.kube-linter.io/run-as-non-root: >- + 'mailserver' needs to run as root + ignore-check.kube-linter.io/privileged-ports: >- + 'mailserver' needs privileged ports + ignore-check.kube-linter.io/no-read-only-root-fs: >- + There are too many files written to make the root FS read-only + + spec: + replicas: 1 + selector: + matchLabels: + app: mailserver + + template: + metadata: + labels: + app: mailserver + + annotations: + container.apparmor.security.beta.kubernetes.io/mailserver: runtime/default + + spec: + hostname: mail + containers: + - name: mailserver + image: ghcr.io/docker-mailserver/docker-mailserver:latest + imagePullPolicy: IfNotPresent + + securityContext: + # `allowPrivilegeEscalation: true` is required to support SGID via the `postdrop` + # executable in `/var/mail-state` for Postfix (maildrop + public dirs): + # https://github.com/docker-mailserver/docker-mailserver/pull/3625 + allowPrivilegeEscalation: true + readOnlyRootFilesystem: false + runAsUser: 0 + runAsGroup: 0 + runAsNonRoot: false + privileged: false + capabilities: + add: + # file permission capabilities + - CHOWN + - FOWNER + - MKNOD + - SETGID + - SETUID + - DAC_OVERRIDE + # network capabilities + - NET_ADMIN # needed for F2B + - NET_RAW # needed for F2B + - NET_BIND_SERVICE + # miscellaneous capabilities + - SYS_CHROOT + - KILL + drop: [ALL] + seccompProfile: + type: RuntimeDefault + + # Tune this to your needs. + # If you disable ClamAV, you can use less RAM and CPU. + # This becomes important in case you're low on resources + # and Kubernetes refuses to schedule new pods. + resources: + limits: + memory: 4Gi + cpu: 1500m + requests: + memory: 2Gi + cpu: 600m + + volumeMounts: + - name: files + subPath: postfix-accounts.cf + mountPath: /tmp/docker-mailserver/postfix-accounts.cf + readOnly: true + + # PVCs + - name: data + mountPath: /var/mail + subPath: data + readOnly: false + - name: data + mountPath: /var/mail-state + subPath: state + readOnly: false + - name: data + mountPath: /var/log/mail + subPath: log + readOnly: false + + # certificates + - name: certificates-rsa + mountPath: /secrets/ssl/rsa/ + readOnly: true + + ports: + - name: smtp + containerPort: 25 + protocol: TCP + - name: submissions + containerPort: 465 + protocol: TCP + - name: submission + containerPort: 587 + - name: imaps + containerPort: 993 + protocol: TCP + + envFrom: + - configMapRef: + name: mailserver.environment + + restartPolicy: Always + + volumes: + # configuration files + - name: files + configMap: + name: mailserver.files + + # PVCs + - name: data + persistentVolumeClaim: + claimName: data + + # certificates + - name: certificates-rsa + secret: + secretName: mail-tls-certificate-rsa + items: + - key: tls.key + path: tls.key + - key: tls.crt + path: tls.crt + ``` ## Exposing your Mail Server to the Outside World -The more difficult part with Kubernetes is to expose a deployed DMS to the outside world. Kubernetes provides multiple ways for doing that; each has downsides and complexity. The major problem with exposing DMS to outside world in Kubernetes is to [preserve the real client IP][Kubernetes-service-source-ip]. The real client IP is required by DMS for performing IP-based SPF checks and spam checks. If you do not require SPF checks for incoming mails, you may disable them in your [Postfix configuration][docs-postfix] by dropping the line that states: `check_policy_service unix:private/policyd-spf`. +The more difficult part with Kubernetes is to expose a deployed DMS instance to the outside world. -The easiest approach was covered above, using `#!yaml externalTrafficPolicy: Local`, which disables the service proxy, but makes the service local as well (which does not scale). This approach only works when you are given the correct (that is, a public and routable) IP address by a load balancer (like MetalLB). In this sense, the approach above is similar to the next example below. We want to provide you with a few alternatives too. **But** we also want to communicate the idea of another simple method: you could use a load-balancer without an external IP and DNAT the network traffic to the mail server. After all, this does not interfere with SPF checks because it keeps the origin IP address. If no dedicated external IP address is available, you could try the latter approach, if one is available, use the former. +The major problem with exposing DMS to the outside world in Kubernetes is to [preserve the real client IP][k8s-docs::service-source-ip]. The real client IP is required by DMS for performing IP-based DNS and spam checks. -### External IPs Service +Kubernetes provides multiple ways to address this; each has its upsides and downsides. -The simplest way is to expose DMS as a [Service][Kubernetes-network-service] with [external IPs][Kubernetes-network-external-ip]. This is very similar to the approach taken above. Here, an external IP is given to the service directly by you. With the approach above, you tell your load-balancer to do this. + +!!! quote "" -```yaml ---- -apiVersion: v1 -kind: Service + === "Configure IP Manually" -metadata: - name: mailserver - labels: - app: mailserver + ???+ abstract "Advantages / Disadvantages" -spec: - selector: - app: mailserver - ports: - - name: smtp - port: 25 - targetPort: smtp - # ... + - [x] Simple + - [ ] Requires the node to have a dedicated, publicly routable IP address + - [ ] Limited to a single node (_associated to the dedicated IP address_) + - [ ] Your deployment requires an explicit IP in your configuration (_or an entire Load Balancer_). - externalIPs: - - 80.11.12.10 -``` + !!! info "Requirements" -This approach + 1. You can dedicate a **publicly routable IP** address for the DMS configured `Service`. + 2. A dedicated IP is required to allow your mail server to have matching `A` and `PTR` records (_which other mail servers will use to verify trust when they receive mail sent from your DMS instance_). -- does not preserve the real client IP, so SPF check of incoming mail will fail. -- requires you to specify the exposed IPs explicitly. + !!! example -### Proxy port to Service + Assign the DMS `Service` an external IP directly, or delegate an LB to assign the IP on your behalf. -The [proxy pod][Kubernetes-proxy-service] helps to avoid the necessity of specifying external IPs explicitly. This comes at the cost of complexity; you must deploy a proxy pod on each [Node][Kubernetes-nodes] you want to expose DMS on. + === "External-IP Service" -This approach + The DMS `Service` is configured with an "[external IP][k8s-docs::network-external-ip]" manually. Append your externally reachable IP address to `spec.externalIPs`. -- does not preserve the real client IP, so SPF check of incoming mail will fail. + ```yaml + --- + apiVersion: v1 + kind: Service -### Bind to concrete Node and use host network + metadata: + name: mailserver + labels: + app: mailserver -One way to preserve the real client IP is to use `hostPort` and `hostNetwork: true`. This comes at the cost of availability; you can reach DMS from the outside world only via IPs of [Node][Kubernetes-nodes] where DMS is deployed. + spec: + selector: + app: mailserver + ports: + - name: smtp + port: 25 + targetPort: smtp + # ... -```yaml ---- -apiVersion: extensions/v1beta1 -kind: Deployment + externalIPs: + - 10.20.30.40 + ``` -metadata: - name: mailserver + === "Load-Balancer" -# ... - spec: - hostNetwork: true + The config differs depending on your choice of load balancer. This example uses [MetalLB][metallb-web]. - # ... - containers: - # ... - ports: - - name: smtp - containerPort: 25 - hostPort: 25 - - name: smtp-auth - containerPort: 587 - hostPort: 587 - - name: imap-secure - containerPort: 993 - hostPort: 993 - # ... -``` + ```yaml + --- + apiVersion: v1 + kind: Service -With this approach, + metadata: + name: mailserver + labels: + app: mailserver + annotations: + metallb.universe.tf/address-pool: mailserver -- it is not possible to access DMS via other cluster Nodes, only via the Node DMS was deployed at. -- every Port within the Container is exposed on the Host side. + # ... -### Proxy Port to Service via PROXY Protocol + --- + apiVersion: metallb.io/v1beta1 + kind: IPAddressPool -This way is ideologically the same as [using a proxy pod](#proxy-port-to-service), but instead of a separate proxy pod, you configure your ingress to proxy TCP traffic to the DMS pod using the PROXY protocol, which preserves the real client IP. + metadata: + name: mail + namespace: metallb-system -#### Configure your Ingress + spec: + addresses: [ ] + autoAssign: true -With an [NGINX ingress controller][Kubernetes-nginx], set `externalTrafficPolicy: Local` for its service, and add the following to the TCP services config map (as described [here][Kubernetes-nginx-expose]): + --- + apiVersion: metallb.io/v1beta1 + kind: L2Advertisement -```yaml -25: "mailserver/mailserver:25::PROXY" -465: "mailserver/mailserver:465::PROXY" -587: "mailserver/mailserver:587::PROXY" -993: "mailserver/mailserver:993::PROXY" -``` + metadata: + name: mail + namespace: metallb-system -!!! help "HAProxy" - With [HAProxy][dockerhub-haproxy], the configuration should look similar to the above. If you know what it actually looks like, add an example here. :smiley: + spec: + ipAddressPools: [ mailserver ] + ``` -#### Configure the Mailserver + === "Host network" -Then, configure both [Postfix][docs-postfix] and [Dovecot][docs-dovecot] to expect the PROXY protocol: + ???+ abstract "Advantages / Disadvantages" -??? example "HAProxy Example" + - [x] Simple + - [ ] Requires the node to have a dedicated, publicly routable IP address + - [ ] Limited to a single node (_associated to the dedicated IP address_) + - [ ] It is not possible to access DMS via other cluster nodes, only via the node that DMS was deployed on + - [ ] Every port within the container is exposed on the host side - ```yaml - kind: ConfigMap - apiVersion: v1 - metadata: - name: mailserver.config - labels: - app: mailserver - data: - postfix-main.cf: | - postscreen_upstream_proxy_protocol = haproxy - postfix-master.cf: | - smtp/inet/postscreen_upstream_proxy_protocol=haproxy - submission/inet/smtpd_upstream_proxy_protocol=haproxy - submissions/inet/smtpd_upstream_proxy_protocol=haproxy - dovecot.cf: | - # Assuming your ingress controller is bound to 10.0.0.0/8 - haproxy_trusted_networks = 10.0.0.0/8, 127.0.0.0/8 - service imap-login { - inet_listener imap { - haproxy = yes - } - inet_listener imaps { - haproxy = yes - } - } - # ... - --- + !!! example - kind: Deployment - apiVersion: extensions/v1beta1 - metadata: - name: mailserver - spec: - template: - spec: - containers: - - name: docker-mailserver - volumeMounts: - - name: config - subPath: postfix-main.cf - mountPath: /tmp/docker-mailserver/postfix-main.cf - readOnly: true - - name: config - subPath: postfix-master.cf - mountPath: /tmp/docker-mailserver/postfix-master.cf - readOnly: true - - name: config - subPath: dovecot.cf - mountPath: /tmp/docker-mailserver/dovecot.cf - readOnly: true - ``` + Using `hostPort` and `hostNetwork: true` is a similar approach to [`network_mode: host` with Docker Compose][docker-docs::compose::network_mode]. -With this approach, + ```yaml + --- + apiVersion: apps/v1 + kind: Deployment -- it is not possible to access DMS via cluster-DNS, as the PROXY protocol is required for incoming connections. + metadata: + name: mailserver -[Helm chart]: https://github.com/docker-mailserver/docker-mailserver-helm -[kustomize]: https://kustomize.io/ -[cert-manager]: https://cert-manager.io/docs/ + # ... + spec: + hostNetwork: true + # ... + containers: + # ... + ports: + - name: smtp + containerPort: 25 + hostPort: 25 + - name: submissions + containerPort: 465 + hostPort: 465 + - name: submission + containerPort: 587 + hostPort: 587 + - name: imaps + containerPort: 993 + hostPort: 993 + ``` + + === "Using the PROXY Protocol" + + ???+ abstract "Advantages / Disadvantages" + + - [x] Preserves the origin IP address of clients (_which is crucial for DNS related checks_) + - [x] Aligns with a best practice for Kubernetes by using a dedicated ingress, routing external traffic to the k8s cluster (_with the benefits of flexible routing rules_) + - [x] Avoids the restraint of a single [node][k8s-docs::nodes] (_as a workaround to preserve the original client IP_) + - [ ] Introduces complexity by requiring: + - A reverse-proxy / ingress controller (_potentially extra setup_) + - Kubernetes manifest changes for the DMS configured `Service` + - DMS configuration changes for Postfix and Dovecot + - [ ] To keep support for direct connections to DMS services internally within cluster, service ports must be "duplicated" to offer an alternative port for connections using PROXY protocol + + ??? question "What is the PROXY protocol?" + + PROXY protocol is a network protocol for preserving a client’s IP address when the client’s TCP connection passes through a proxy. + + It is a common feature supported among reverse-proxy services (_NGINX, HAProxy, Traefik_), which you may already have handling ingress traffic for your cluster. + + ```mermaid + flowchart LR + A(External Mail Server) -->|Incoming connection| B + subgraph cluster + B("Ingress Acting as a Proxy") -->|PROXY protocol connection| C(DMS) + end + ``` + + For more information on the PROXY protocol, refer to [our dedicated docs page][docs-mailserver-behind-proxy] on the topic. + + ???+ example "Configure the Ingress Controller" + + === "Traefik" + + On Traefik's side, the configuration is very simple. + + - Create an entrypoint for each port that you want to expose (_probably 25, 465, 587 and 993_). + - Each entrypoint should configure an [`IngressRouteTCP`][traefik-docs::k8s::ingress-route-tcp] that routes to the equivalent internal DMS `Service` port which supports PROXY protocol connections. + + The below snippet demonstrates an example for two entrypoints, `submissions` (port 465) and `imaps` (port 993). + + ```yaml + --- + apiVersion: v1 + kind: Service + + metadata: + name: mailserver + + spec: + # This an optimization to get rid of additional routing steps. + # Previously "type: LoadBalancer" + type: ClusterIP + + --- + apiVersion: traefik.io/v1alpha1 + kind: IngressRouteTCP + + metadata: + name: smtp + + spec: + entryPoints: [ submissions ] + routes: + - match: HostSNI(`*`) + services: + - name: mailserver + namespace: mail + port: subs-proxy # note the 15 character limit here + proxyProtocol: + version: 2 + + --- + apiVersion: traefik.io/v1alpha1 + kind: IngressRouteTCP + + metadata: + name: imaps + + spec: + entryPoints: [ imaps ] + routes: + - match: HostSNI(`*`) + services: + - name: mailserver + namespace: mail + port: imaps-proxy + proxyProtocol: + version: 2 + ``` + + !!! info "`*-proxy` port name suffix" + + The `IngressRouteTCP` example configs above reference ports with a `*-proxy` suffix. + + - These port variants will be defined in the [`Deployment` manifest][docs::k8s::config-deployment], and are scoped to the `mailserver` service (via `spec.routes.services.name`). + - The suffix is used to distinguish that these ports are only compatible with connections using the PROXY protocol, which is what your ingress controller should be managing for you by adding the correct PROXY protocol headers to TCP connections it routes to DMS. + + === "NGINX" + + With an [NGINX ingress controller][k8s-docs::nginx], add the following to the TCP services config map (_as described [here][k8s-docs::nginx-expose]_): + + ```yaml + 25: "mailserver/mailserver:25::PROXY" + 465: "mailserver/mailserver:465::PROXY" + 587: "mailserver/mailserver:587::PROXY" + 993: "mailserver/mailserver:993::PROXY" + ``` + + ???+ example "Adjust DMS config for Dovecot + Postfix" + + ??? warning "Only ingress should connect to DMS with PROXY protocol" + + While Dovecot will restrict connections via PROXY protocol to only clients trusted configured via `haproxy_trusted_networks`, Postfix does not have an equivalent setting. Public clients should always route through ingress to establish a PROXY protocol connection. + + You are responsible for properly managing traffic inside your cluster and to **ensure that only trustworthy entities** can connect to the designated PROXY protocol ports. + + With Kubernetes, this is usually the task of the CNI (_container network interface_). + + !!! tip "Advised approach" + + The _"Separate PROXY protocol ports"_ tab below introduces a little more complexity, but provides better compatibility for internal connections to DMS. + + === "Only accept connections with PROXY protocol" + + !!! warning "Connections to DMS within the internal cluster will be rejected" + + The services for these ports can only enable PROXY protocol support by mandating the protocol on all connections for these ports. + + This can be problematic when you also need to support internal cluster traffic directly to DMS (_instead of routing indirectly through the ingress controller_). + + Here is an example configuration for [Postfix][docs-postfix], [Dovecot][docs-dovecot], and the required adjustments for the [`Deployment` manifest][docs::k8s::config-deployment]. The port names are adjusted here only to convey the additional context described earlier. + + ```yaml + kind: ConfigMap + apiVersion: v1 + metadata: + name: mailserver-extra-config + labels: + app: mailserver + data: + postfix-main.cf: | + postscreen_upstream_proxy_protocol = haproxy + postfix-master.cf: | + smtp/inet/postscreen_upstream_proxy_protocol=haproxy + submission/inet/smtpd_upstream_proxy_protocol=haproxy + submissions/inet/smtpd_upstream_proxy_protocol=haproxy + dovecot.cf: | + haproxy_trusted_networks = + service imap-login { + inet_listener imap { + haproxy = yes + } + inet_listener imaps { + haproxy = yes + } + } + # ... + + --- + kind: Deployment + apiVersion: apps/v1 + metadata: + name: mailserver + spec: + template: + spec: + containers: + - name: docker-mailserver + # ... + ports: + - name: smtp-proxy + containerPort: 25 + protocol: TCP + - name: imap-proxy + containerPort: 143 + protocol: TCP + - name: subs-proxy + containerPort: 465 + protocol: TCP + - name: sub-proxy + containerPort: 587 + protocol: TCP + - name: imaps-proxy + containerPort: 993 + protocol: TCP + # ... + volumeMounts: + - name: config + subPath: postfix-main.cf + mountPath: /tmp/docker-mailserver/postfix-main.cf + readOnly: true + - name: config + subPath: postfix-master.cf + mountPath: /tmp/docker-mailserver/postfix-master.cf + readOnly: true + - name: config + subPath: dovecot.cf + mountPath: /tmp/docker-mailserver/dovecot.cf + readOnly: true + ``` + + === "Separate PROXY protocol ports for ingress" + + !!! info + + Supporting internal cluster connections to DMS without using PROXY protocol requires both Postfix and Dovecot to be configured with alternative ports for each service port (_which only differ by enforcing PROXY protocol connections_). + + - The ingress controller will route public connections to the internal alternative ports for DMS (`*-proxy` variants). + - Internal cluster connections will instead use the original ports configured for the DMS container directly (_which are private to the cluster network_). + + In this example we'll create a copy of the original service ports with PROXY protocol enabled, and increment the port number assigned by `10000`. + + Create a `user-patches.sh` file to apply these config changes during container startup: + + ```bash + #!/bin/bash + + # Duplicate the config for the submission(s) service ports (587 / 465) with adjustments for the PROXY ports (10587 / 10465) and `syslog_name` setting: + postconf -Mf submission/inet | sed -e s/^submission/10587/ -e 's/submission/submission-proxyprotocol/' >> /etc/postfix/master.cf + postconf -Mf submissions/inet | sed -e s/^submissions/10465/ -e 's/submissions/submissions-proxyprotocol/' >> /etc/postfix/master.cf + # Enable PROXY Protocol support for these new service variants: + postconf -P 10587/inet/smtpd_upstream_proxy_protocol=haproxy + postconf -P 10465/inet/smtpd_upstream_proxy_protocol=haproxy + + # Create a variant for port 25 too (NOTE: Port 10025 is already assigned in DMS to Amavis): + postconf -Mf smtp/inet | sed -e s/^smtp/12525/ >> /etc/postfix/master.cf + # Enable PROXY Protocol support (different setting as port 25 is handled via postscreen), optionally configure a `syslog_name` to distinguish in logs: + postconf -P 12525/inet/postscreen_upstream_proxy_protocol=haproxy 12525/inet/syslog_name=smtp-proxyprotocol + ``` + + For Dovecot, you can configure [`dovecot.cf`][docs-dovecot] to look like this: + + ```cf + haproxy_trusted_networks = + + service imap-login { + inet_listener imap-proxied { + haproxy = yes + port = 10143 + } + + inet_listener imaps-proxied { + haproxy = yes + port = 10993 + ssl = yes + } + } + ``` + + Update the [`Deployment` manifest][docs::k8s::config-deployment] `ports` section by appending these new ports: + + ```yaml + - name: smtp-proxy + # not 10025 in this example due to a possible clash with Amavis + containerPort: 12525 + protocol: TCP + - name: imap-proxy + containerPort: 10143 + protocol: TCP + - name: subs-proxy + containerPort: 10465 + protocol: TCP + - name: sub-proxy + containerPort: 10587 + protocol: TCP + - name: imaps-proxy + containerPort: 10993 + protocol: TCP + ``` + + !!! note + + If you use other Dovecot ports (110, 995, 4190), you may want to configure those similar to above. The `dovecot.cf` config for these ports is [documented here][docs-mailserver-behind-proxy] (_in the equivalent section of that page_). + +[docs::k8s::config-deployment]: #deployment [docs-tls]: ../security/ssl.md [docs-dovecot]: ./override-defaults/dovecot.md [docs-postfix]: ./override-defaults/postfix.md -[dockerhub-haproxy]: https://hub.docker.com/_/haproxy -[Kubernetes-nginx]: https://kubernetes.github.io/ingress-nginx -[Kubernetes-nginx-expose]: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services -[Kubernetes-network-service]: https://kubernetes.io/docs/concepts/services-networking/service -[Kubernetes-network-external-ip]: https://kubernetes.io/docs/concepts/services-networking/service/#external-ips -[Kubernetes-nodes]: https://kubernetes.io/docs/concepts/architecture/nodes -[Kubernetes-proxy-service]: https://github.com/kubernetes/contrib/tree/master/for-demos/proxy-to-service -[Kubernetes-service-source-ip]: https://kubernetes.io/docs/tutorials/services/source-ip +[docs-mailserver-behind-proxy]: ../../examples/tutorials/mailserver-behind-proxy.md + +[github-web::docker-mailserver-helm]: https://github.com/docker-mailserver/docker-mailserver-helm +[docker-docs::compose::network_mode]: https://docs.docker.com/compose/compose-file/compose-file-v3/#network_mode +[kustomize]: https://kustomize.io/ +[cert-manager]: https://cert-manager.io/docs/ +[metallb-web]: https://metallb.universe.tf/ + +[k8s-docs::config::service]: https://kubernetes.io/docs/concepts/services-networking/service +[k8s-docs::config::deployment]: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#creating-a-deployment +[k8s-docs::nodes]: https://kubernetes.io/docs/concepts/architecture/nodes +[k8s-docs::nginx]: https://kubernetes.github.io/ingress-nginx +[k8s-docs::nginx-expose]: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services +[k8s-docs::service-source-ip]: https://kubernetes.io/docs/tutorials/services/source-ip +[k8s-docs::network-external-ip]: https://kubernetes.io/docs/concepts/services-networking/service/#external-ips + +[traefik-docs::k8s::ingress-route-tcp]: https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-ingressroutetcp diff --git a/docs/content/examples/tutorials/mailserver-behind-proxy.md b/docs/content/examples/tutorials/mailserver-behind-proxy.md index 47116fde..60e4c539 100644 --- a/docs/content/examples/tutorials/mailserver-behind-proxy.md +++ b/docs/content/examples/tutorials/mailserver-behind-proxy.md @@ -14,6 +14,8 @@ This reduces many of the benefits for why you might use a reverse proxy, but the Some deployments may require a service to route traffic (kubernetes) when deploying, in which case the below advice is important to understand well. +The guide here has also been adapted for [our Kubernetes docs][docs::kubernetes]. + ## What can go wrong? Without a reverse proxy involved, a service is typically aware of the client IP for a connection. @@ -354,9 +356,8 @@ Software on the receiving end of the connection often supports configuring an IP [`postscreen_access_list`][postfix-docs::settings::postscreen_access_list] (_or [`smtpd_client_restrictions`][postfix-docs::settings::smtpd_client_restrictions] with [`check_client_access`][postfix-docs::settings::check_client_access] for ports 587/465_) can both restrict access by IP via a [CIDR lookup table][postfix-docs::config-table::cidr], however the client IP is already rewritten at this point via PROXY protocol. Thus those settings cannot be used for restricting access to only trusted proxies, only to the actual clients. - - A similar setting [`mynetworks`][postfix-docs::settings::mynetworks] / [`PERMIT_DOCKER`][docs::env::permit_docker] manages elevated trust for bypassing security restrictions. While it is intended for trusted clients, it has no relevance to trusting proxies for the same reasons. + A similar setting [`mynetworks`][postfix-docs::settings::mynetworks] / [`PERMIT_DOCKER`][docs::env::permit_docker] manages elevated trust for bypassing security restrictions. While it is intended for trusted clients, it has no relevance to trusting proxies for the same reasons. ### Monitoring @@ -373,6 +374,8 @@ While PROXY protocol works well with the reverse proxy, you may have some contai You should adjust configuration of these monitoring services to monitor for auth failures from those services directly instead, adding an exclusion for that service IP from any DMS logs monitored (_but be mindful of PROXY header forgery risks_). +[docs::kubernetes]: ../../config/advanced/kubernetes.md#using-the-proxy-protocol + [docs::overrides::dovecot]: ../../config/advanced/override-defaults/dovecot.md [docs::overrides::postfix]: ../../config/advanced/override-defaults/postfix.md [docs::overrides::user-patches]: ../../config/advanced/override-defaults/user-patches.md diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 84366423..e0ff50c9 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -82,6 +82,11 @@ markdown_extensions: format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.tabbed: alternate_style: true + slugify: !!python/object/apply:pymdownx.slugs.slugify + kwds: + case: lower + - pymdownx.tasklist: + custom_checkbox: true - pymdownx.magiclink - pymdownx.inlinehilite - pymdownx.tilde From ede95e6f7fca0b02b756bec62d9f6cf81b64dfb1 Mon Sep 17 00:00:00 2001 From: Rahil Bhimjiani Date: Thu, 14 Mar 2024 02:44:14 +0530 Subject: [PATCH 26/53] docs: Update links for account management in `README.md` (#3937) --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index cfd453ff..e34aeb12 100644 --- a/README.md +++ b/README.md @@ -38,8 +38,8 @@ A production-ready fullstack but simple containerized mail server (SMTP, IMAP, L ## :package: Included Services -- [Postfix](http://www.postfix.org) with SMTP or LDAP authentication and support for [extension delimiters](https://docker-mailserver.github.io/docker-mailserver/latest/config/user-management/aliases/#address-tags-extension-delimiters-an-alternative-to-aliases) -- [Dovecot](https://www.dovecot.org) with SASL, IMAP, POP3, LDAP, [basic Sieve support](https://docker-mailserver.github.io/docker-mailserver/latest/config/advanced/mail-sieve) and [quotas](https://docker-mailserver.github.io/docker-mailserver/latest/config/user-management/accounts#notes) +- [Postfix](http://www.postfix.org) with SMTP or LDAP authentication and support for [extension delimiters](https://docker-mailserver.github.io/docker-mailserver/v13.3/config/user-management/#address-tags-extension-delimiters-as-an-alternative-to-aliases) +- [Dovecot](https://www.dovecot.org) with SASL, IMAP, POP3, LDAP, [basic Sieve support](https://docker-mailserver.github.io/docker-mailserver/latest/config/advanced/mail-sieve) and [quotas](https://docker-mailserver.github.io/docker-mailserver/v13.3/config/user-management/#quotas) - [Rspamd](https://rspamd.com/) - [Amavis](https://www.amavis.org/) - [SpamAssassin](http://spamassassin.apache.org/) supporting custom rules From cdcd86420e83984228dbb027d5f148f3cea9e0c6 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Thu, 14 Mar 2024 15:24:33 +1300 Subject: [PATCH 27/53] docs: Add IPv6 troubleshooting tip (#3938) Sometimes a user may have a configuration error and get halfway there. This should help point them in the right direction. --- docs/content/config/advanced/ipv6.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/content/config/advanced/ipv6.md b/docs/content/config/advanced/ipv6.md index 77dfb0c9..7e9c904c 100644 --- a/docs/content/config/advanced/ipv6.md +++ b/docs/content/config/advanced/ipv6.md @@ -177,6 +177,12 @@ docker run --rm -d --network dms-ipv6 -p 80:80 traefik/whoami curl --max-time 5 http://[2001:db8::1]:80 ``` +!!! warning "IPv6 gateway IP" + + If instead of the remote IPv6 address, you may notice the gateway IP for the IPv6 subnet your DMS container belongs to. + + This will happen when DMS has an IPv6 IP address assigned, for the same reason as with IPv4, `userland-proxy: true`. It indicates that your `daemon.json` has not been configured correctly or had the updated config applied for `ip6tables :true` + `experimental: true`. Make sure you used `systemctl restart docker` after updating `daemon.json`. + !!! info "IPv6 ULA address priority" DNS lookups that have records for both IPv4 and IPv6 addresses (_eg: `localhost`_) may prefer IPv4 over IPv6 (ULA) for private addresses, whereas for public addresses IPv6 has priority. This shouldn't be anything to worry about, but can come across as a surprise when testing your IPv6 setup on the same host instead of from a remote client. From 910667d5866cfdb2e04e893960bb009b626882fd Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 14 Mar 2024 13:22:43 +0100 Subject: [PATCH 28/53] docs: updated `CONTRIBUTORS.md` (#3930) --- CONTRIBUTORS.md | 419 ++++++++++++++++++++++++------------------------ 1 file changed, 213 insertions(+), 206 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index ffe5cecd..9b8e4043 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -658,10 +658,17 @@ Thanks goes to these wonderful people ✨ - - andrewlow + + ubenmackin
- andrewlow + ubenmackin + + + + + craue +
+ craue @@ -671,13 +678,21 @@ Thanks goes to these wonderful people ✨ abh + + + andrewlow +
+ andrewlow + + aminvakil
aminvakil - + + elbracht @@ -685,21 +700,6 @@ Thanks goes to these wonderful people ✨ elbracht - - - craue -
- craue - - - - - - ubenmackin -
- ubenmackin - - danielpanteleit @@ -737,48 +737,12 @@ Thanks goes to these wonderful people ✨ - - fl42 + + nueaf
- fl42 + nueaf - - - ipernet -
- ipernet - - - - - H4R0 -
- H4R0 - - - - - eltociear -
- eltociear - - - - - jamesfryer -
- jamesfryer - - - - - millaguie -
- millaguie - - - martinwepner @@ -793,13 +757,6 @@ Thanks goes to these wonderful people ✨ artonge - - - jedateach -
- jedateach - - spacecowboy @@ -808,46 +765,82 @@ Thanks goes to these wonderful people ✨ - - nueaf + + jedateach
- nueaf + jedateach - - thomasschmit + + millaguie
- thomasschmit + millaguie - - TechnicLab + + eltociear
- TechnicLab + eltociear - - sylvaindumont + + H4R0
- sylvaindumont + H4R0 - - syl20bnr + + jamesfryer
- syl20bnr + jamesfryer - - 5ven + + ipernet
- 5ven + ipernet + + + + + fl42 +
+ fl42 + + + + + radicand +
+ radicand + + + + + + sjmudd +
+ sjmudd + + + + + simonsystem +
+ simonsystem + + + + + stephan-devop +
+ stephan-devop @@ -858,25 +851,53 @@ Thanks goes to these wonderful people ✨ - - stephan-devop + + 5ven
- stephan-devop + 5ven + + + + + syl20bnr +
+ syl20bnr - - simonsystem + + sylvaindumont
- simonsystem + sylvaindumont - - radicand + + TechnicLab
- radicand + TechnicLab + + + + + 42wim +
+ 42wim + + + + + vilisas +
+ vilisas + + + + + thomasschmit +
+ thomasschmit @@ -885,7 +906,8 @@ Thanks goes to these wonderful people ✨
Thiritin - + + tweibert @@ -906,8 +928,7 @@ Thanks goes to these wonderful people ✨
VictorKoenders - - + Twist235 @@ -928,27 +949,6 @@ Thanks goes to these wonderful people ✨
Drakulix - - - - vilisas -
- vilisas - - - - - 42wim -
- 42wim - - - - - ShiriNmi1520 -
- ShiriNmi1520 - @@ -1015,6 +1015,13 @@ Thanks goes to these wonderful people ✨ piwai + + + auchri +
+ auchri + + rahilarious @@ -1028,15 +1035,15 @@ Thanks goes to these wonderful people ✨
remoe - + + robbertkl
robbertkl - - + romansey @@ -1071,15 +1078,15 @@ Thanks goes to these wonderful people ✨
klamann - + + svdb0
svdb0 - - + 3ap @@ -1095,10 +1102,10 @@ Thanks goes to these wonderful people ✨ - - sjmudd + + ShiriNmi1520
- sjmudd + ShiriNmi1520 @@ -1114,15 +1121,15 @@ Thanks goes to these wonderful people ✨
millerjason - + + mplx
mplx - - + odinis @@ -1157,15 +1164,15 @@ Thanks goes to these wonderful people ✨
pravynandas - + + presocratics
presocratics - - + rhyst @@ -1200,15 +1207,15 @@ Thanks goes to these wonderful people ✨
smargold476 - + + sportshead
sportshead - - + squash @@ -1243,15 +1250,15 @@ Thanks goes to these wonderful people ✨
wligtenberg - + + wolkenschieber
wolkenschieber - - + worldworm @@ -1286,15 +1293,15 @@ Thanks goes to these wonderful people ✨
awb99 - + + brainkiller
brainkiller - - + cternes @@ -1329,15 +1336,15 @@ Thanks goes to these wonderful people ✨
ghnp5 - + + helmutundarnold
helmutundarnold - - + hnws @@ -1372,15 +1379,15 @@ Thanks goes to these wonderful people ✨
jjtt - + + paralax
paralax - - + jpduyx @@ -1415,22 +1422,8 @@ Thanks goes to these wonderful people ✨
matrixes - - - - 0xflotus -
- 0xflotus - - - - auchri -
- auchri - - arkanovicz @@ -1465,15 +1458,15 @@ Thanks goes to these wonderful people ✨
dkarski - - + dbellavista
dbellavista - + + danielvandenberg95 @@ -1508,15 +1501,15 @@ Thanks goes to these wonderful people ✨
doominator42 - - + aydodo
aydodo - + + vedtam @@ -1551,15 +1544,15 @@ Thanks goes to these wonderful people ✨
ErikEngerd - - + huncode
huncode - + + felixn @@ -1574,6 +1567,20 @@ Thanks goes to these wonderful people ✨ flole + + + akkumar +
+ akkumar + + + + + 0xflotus +
+ 0xflotus + + ifokeev @@ -1587,15 +1594,15 @@ Thanks goes to these wonderful people ✨
20th - + + 2b
2b - - + askz @@ -1630,15 +1637,15 @@ Thanks goes to these wonderful people ✨
kachkaev - + + alexanderneu
alexanderneu - - + ch3sh1r @@ -1673,15 +1680,15 @@ Thanks goes to these wonderful people ✨
iRhonin - + + MrFreezeex
MrFreezeex - - + arunvc @@ -1716,28 +1723,14 @@ Thanks goes to these wonderful people ✨
erdos4d - + + crash7
crash7 - - - - - froks -
- froks - - - - - akkumar -
- akkumar - @@ -1760,6 +1753,13 @@ Thanks goes to these wonderful people ✨ khuedoan + + + UltraCoderRU +
+ UltraCoderRU + + JustAnother1 @@ -1882,21 +1882,28 @@ Thanks goes to these wonderful people ✨ neuralp + + + froks +
+ froks + + fkefer
fkefer - + + Marsu31
Marsu31 - - + glandais @@ -1931,15 +1938,15 @@ Thanks goes to these wonderful people ✨
sirgantrithon - + + Influencer
Influencer - - + JacksonZ03 @@ -1974,15 +1981,15 @@ Thanks goes to these wonderful people ✨
init-js - + + Jeidnx
Jeidnx - - + jessp01 @@ -2017,15 +2024,15 @@ Thanks goes to these wonderful people ✨
jurekbarth - + + JOduMonT
JOduMonT - - + Kaan88 From 066773e79fe8e49d7f9b4ae4ff85f1da75abab96 Mon Sep 17 00:00:00 2001 From: Casper Date: Sun, 17 Mar 2024 16:31:55 +0100 Subject: [PATCH 29/53] Better support regular container restarts (#3929) Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> --- CHANGELOG.md | 3 +++ target/scripts/start-mailserver.sh | 36 +++++++++++++++++---------- target/scripts/startup/check-stack.sh | 9 ------- target/scripts/startup/setup-stack.sh | 2 +- 4 files changed, 27 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index de5642bf..31f906d5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,9 @@ The most noteworthy change of this release is the update of the container's base - `LOGROTATE_COUNT` defines the number of files kept by logrotate ([#3907](https://github.com/docker-mailserver/docker-mailserver/pull/3907)) - The fail2ban log file is now also taken into account by `LOGROTATE_COUNT` and `LOGROTATE_INTERVAL` ([#3915](https://github.com/docker-mailserver/docker-mailserver/pull/3915), [#3919](https://github.com/docker-mailserver/docker-mailserver/pull/3919)) +- **Internal:** + - Regular container restarts are now better supported. Setup scripts that ran previously will now be skipped ([#3929](https://github.com/docker-mailserver/docker-mailserver/pull/3929)) + ### Updates - **Environment Variables:** diff --git a/target/scripts/start-mailserver.sh b/target/scripts/start-mailserver.sh index f18abdf6..cbe38da9 100755 --- a/target/scripts/start-mailserver.sh +++ b/target/scripts/start-mailserver.sh @@ -33,7 +33,6 @@ function _register_functions() { # ? >> Checks - _register_check_function '_check_improper_restart' _register_check_function '_check_hostname' _register_check_function '_check_log_level' _register_check_function '_check_spam_prefix' @@ -170,24 +169,35 @@ function _register_functions() { # ? >> Executing all stacks / actual start of DMS # ------------------------------------------------------------ -_early_supervisor_setup -_early_variables_setup +# Ensure DMS only adjusts config files for a new container. +# Container restarts should skip as they retain the modified config. +if [[ ! -f /CONTAINER_START ]]; then + _early_supervisor_setup + _early_variables_setup -_log 'info' "Welcome to docker-mailserver ${DMS_RELEASE}" + _log 'info' "Welcome to docker-mailserver ${DMS_RELEASE}" -_register_functions -_check -_setup -[[ ${LOG_LEVEL} =~ (debug|trace) ]] && print-environment -_run_user_patches -_start_daemons + _register_functions + _check + _setup + _run_user_patches +else + # container was restarted + _early_variables_setup + + _log 'info' 'Container was restarted. Skipping setup routines.' + _log 'info' "Welcome to docker-mailserver ${DMS_RELEASE}" + + _register_functions +fi # marker to check if container was restarted date >/CONTAINER_START +[[ ${LOG_LEVEL} =~ (debug|trace) ]] && print-environment +_start_daemons + _log 'info' "${HOSTNAME} is up and running" touch /var/log/mail/mail.log -tail -Fn 0 /var/log/mail/mail.log - -exit 0 +exec tail -Fn 0 /var/log/mail/mail.log diff --git a/target/scripts/startup/check-stack.sh b/target/scripts/startup/check-stack.sh index cc98cfe2..766fcccf 100644 --- a/target/scripts/startup/check-stack.sh +++ b/target/scripts/startup/check-stack.sh @@ -14,15 +14,6 @@ function _check() { done } -function _check_improper_restart() { - _log 'debug' 'Checking for improper restart' - - if [[ -f /CONTAINER_START ]]; then - _log 'warn' 'This container was (likely) improperly restarted which can result in undefined behavior' - _log 'warn' "Please use 'docker compose up --force-recreate' or equivalent (view our troubleshooting docs)" - fi -} - function _check_hostname() { _log 'debug' 'Checking that hostname/domainname is provided or overridden' diff --git a/target/scripts/startup/setup-stack.sh b/target/scripts/startup/setup-stack.sh index 81e1a98e..8c8e6461 100644 --- a/target/scripts/startup/setup-stack.sh +++ b/target/scripts/startup/setup-stack.sh @@ -40,7 +40,7 @@ function _early_supervisor_setup() { if ! grep -q "loglevel = ${SUPERVISOR_LOGLEVEL}" /etc/supervisor/supervisord.conf; then case "${SUPERVISOR_LOGLEVEL}" in ( 'critical' | 'error' | 'info' | 'debug' ) - sed -i -E \ + sedfile -i -E \ "s|(loglevel).*|\1 = ${SUPERVISOR_LOGLEVEL}|g" \ /etc/supervisor/supervisord.conf From 7017f4c0817efbfe810b8f61e343d026adff7d26 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 20 Mar 2024 09:46:14 +1300 Subject: [PATCH 30/53] chore(deps): Bump docker/build-push-action from 5.2.0 to 5.3.0 (#3947) Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5.2.0 to 5.3.0. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/v5.2.0...v5.3.0) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/generic_build.yml | 2 +- .github/workflows/generic_publish.yml | 2 +- .github/workflows/generic_test.yml | 2 +- .github/workflows/generic_vulnerability-scan.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/generic_build.yml b/.github/workflows/generic_build.yml index aa3051c8..6a29126d 100644 --- a/.github/workflows/generic_build.yml +++ b/.github/workflows/generic_build.yml @@ -83,7 +83,7 @@ jobs: # NOTE: AMD64 can build within 2 minutes - name: 'Build images' - uses: docker/build-push-action@v5.2.0 + uses: docker/build-push-action@v5.3.0 with: context: . # Build at least the AMD64 image (which runs against the test suite). diff --git a/.github/workflows/generic_publish.yml b/.github/workflows/generic_publish.yml index a729e5b8..7d9b6166 100644 --- a/.github/workflows/generic_publish.yml +++ b/.github/workflows/generic_publish.yml @@ -67,7 +67,7 @@ jobs: password: ${{ secrets.GITHUB_TOKEN }} - name: 'Build and publish images' - uses: docker/build-push-action@v5.2.0 + uses: docker/build-push-action@v5.3.0 with: context: . build-args: | diff --git a/.github/workflows/generic_test.yml b/.github/workflows/generic_test.yml index 73db6f6c..7b60f319 100644 --- a/.github/workflows/generic_test.yml +++ b/.github/workflows/generic_test.yml @@ -43,7 +43,7 @@ jobs: # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. - name: 'Build AMD64 image from cache' - uses: docker/build-push-action@v5.2.0 + uses: docker/build-push-action@v5.3.0 with: context: . tags: mailserver-testing:ci diff --git a/.github/workflows/generic_vulnerability-scan.yml b/.github/workflows/generic_vulnerability-scan.yml index 8ca0b8b2..d9d5639b 100644 --- a/.github/workflows/generic_vulnerability-scan.yml +++ b/.github/workflows/generic_vulnerability-scan.yml @@ -42,7 +42,7 @@ jobs: # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. - name: 'Build AMD64 image from cache' - uses: docker/build-push-action@v5.2.0 + uses: docker/build-push-action@v5.3.0 with: context: . tags: mailserver-testing:ci From 849293f88c8a5faebbe2c704badae92b923b2ac5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 19 Mar 2024 20:48:35 +0000 Subject: [PATCH 31/53] chore(deps): Bump docker/setup-buildx-action from 3.1.0 to 3.2.0 (#3946) Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.1.0 to 3.2.0. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](https://github.com/docker/setup-buildx-action/compare/v3.1.0...v3.2.0) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com> --- .github/workflows/generic_build.yml | 2 +- .github/workflows/generic_publish.yml | 2 +- .github/workflows/generic_test.yml | 2 +- .github/workflows/generic_vulnerability-scan.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/generic_build.yml b/.github/workflows/generic_build.yml index 6a29126d..f0bd2111 100644 --- a/.github/workflows/generic_build.yml +++ b/.github/workflows/generic_build.yml @@ -79,7 +79,7 @@ jobs: platforms: arm64 - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.1.0 + uses: docker/setup-buildx-action@v3.2.0 # NOTE: AMD64 can build within 2 minutes - name: 'Build images' diff --git a/.github/workflows/generic_publish.yml b/.github/workflows/generic_publish.yml index 7d9b6166..266090b6 100644 --- a/.github/workflows/generic_publish.yml +++ b/.github/workflows/generic_publish.yml @@ -40,7 +40,7 @@ jobs: platforms: arm64 - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.1.0 + uses: docker/setup-buildx-action@v3.2.0 # Try get the cached build layers from a prior `generic_build.yml` job. # NOTE: Until adopting `type=gha` scoped cache exporter (in `docker/build-push-action`), diff --git a/.github/workflows/generic_test.yml b/.github/workflows/generic_test.yml index 7b60f319..4c49ca3e 100644 --- a/.github/workflows/generic_test.yml +++ b/.github/workflows/generic_test.yml @@ -38,7 +38,7 @@ jobs: # Ensures consistent BuildKit version (not coupled to Docker Engine), # and increased compatibility of the build cache vs mixing buildx drivers. - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.1.0 + uses: docker/setup-buildx-action@v3.2.0 # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. diff --git a/.github/workflows/generic_vulnerability-scan.yml b/.github/workflows/generic_vulnerability-scan.yml index d9d5639b..23544820 100644 --- a/.github/workflows/generic_vulnerability-scan.yml +++ b/.github/workflows/generic_vulnerability-scan.yml @@ -37,7 +37,7 @@ jobs: # Ensures consistent BuildKit version (not coupled to Docker Engine), # and increased compatibility of the build cache vs mixing buildx drivers. - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.1.0 + uses: docker/setup-buildx-action@v3.2.0 # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. From 3125cad45a7895e93e4779aed1a7c86ad8cf3ee8 Mon Sep 17 00:00:00 2001 From: Casper Date: Thu, 21 Mar 2024 00:53:04 +0100 Subject: [PATCH 32/53] Enable spamassassin only, when amavis is enabled too. (#3943) --- CHANGELOG.md | 1 + target/scripts/startup/setup.d/security/misc.sh | 9 +++++---- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 31f906d5..89efb030 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -84,6 +84,7 @@ The most noteworthy change of this release is the update of the container's base - It's only functionality remaining was to opt-out of run-time state consolidation with `ONE_DIR=0` (_when a volume was already mounted to `/var/mail-state`_). - **Internal:** - Changed the Postgrey whitelist retrieved during build to [source directly from Github](https://github.com/schweikert/postgrey/blob/master/postgrey_whitelist_clients) as the list is updated more frequently than the [author publishes at their website](https://postgrey.schweikert.ch) ([#3879](https://github.com/docker-mailserver/docker-mailserver/pull/3879)) + - Enable spamassassin only, when amavis is enabled too. ([#3943](https://github.com/docker-mailserver/docker-mailserver/pull/3943)) - **Tests:** - Refactored helper methods for sending e-mails with specific `Message-ID` headers and the helpers for retrieving + filtering logs, which together help isolate logs relevant to specific mail when multiple mails have been processed within a single test. ([#3786](https://github.com/docker-mailserver/docker-mailserver/pull/3786)) - **Rspamd**: diff --git a/target/scripts/startup/setup.d/security/misc.sh b/target/scripts/startup/setup.d/security/misc.sh index bb460716..444589df 100644 --- a/target/scripts/startup/setup.d/security/misc.sh +++ b/target/scripts/startup/setup.d/security/misc.sh @@ -67,6 +67,11 @@ function __setup__security__postscreen() { } function __setup__security__spamassassin() { + if [[ ${ENABLE_AMAVIS} -ne 1 && ${ENABLE_SPAMASSASSIN} -eq 1 ]]; then + _log 'warn' 'Spamassassin does not work when Amavis is disabled. Enable Amavis to fix it.' + ENABLE_SPAMASSASSIN=0 + fi + if [[ ${ENABLE_SPAMASSASSIN} -eq 1 ]]; then _log 'debug' 'Enabling and configuring SpamAssassin' @@ -240,10 +245,6 @@ function __setup__security__amavis() { if [[ ${ENABLE_CLAMAV} -eq 1 ]] && [[ ${ENABLE_RSPAMD} -eq 0 ]]; then _log 'warn' 'ClamAV will not work when Amavis & rspamd are disabled. Enable either Amavis or rspamd to fix it.' fi - - if [[ ${ENABLE_SPAMASSASSIN} -eq 1 ]]; then - _log 'warn' 'Spamassassin will not work when Amavis is disabled. Enable Amavis to fix it.' - fi fi } From 0dad7c49a4f4e6880c091ef0cdbed828cfd86068 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 21 Mar 2024 20:53:00 +0100 Subject: [PATCH 33/53] docs: updated `CONTRIBUTORS.md` (#3944) --- CONTRIBUTORS.md | 472 ++++++++++++++++++++++++------------------------ 1 file changed, 236 insertions(+), 236 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 9b8e4043..7c81c968 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -320,49 +320,6 @@ Thanks goes to these wonderful people ✨ kiliant - - - robertdolca -
- robertdolca - - - - - okainov -
- okainov - - - - - lukecyca -
- lukecyca - - - - - jsonn -
- jsonn - - - - - - jamebus -
- jamebus - - - - - mathuin -
- mathuin - - dashohoxha @@ -378,53 +335,53 @@ Thanks goes to these wonderful people ✨ - - weo + + mathuin
- weo + mathuin - - Zehir + + jamebus
- Zehir + jamebus - - guardiande + + robertdolca
- guardiande + robertdolca - - kamuri + + okainov
- kamuri + okainov - - davidszp + + jsonn
- davidszp + jsonn - - andreasgerstmayr + + lukecyca
- andreasgerstmayr + lukecyca - - VanVan + + m-schmoock
- VanVan + m-schmoock @@ -436,10 +393,17 @@ Thanks goes to these wonderful people ✨ - - m-schmoock + + eltociear
- m-schmoock + eltociear + + + + + VanVan +
+ VanVan @@ -449,6 +413,42 @@ Thanks goes to these wonderful people ✨ voordev + + + andreasgerstmayr +
+ andreasgerstmayr + + + + + davidszp +
+ davidszp + + + + + kamuri +
+ kamuri + + + + + + guardiande +
+ guardiande + + + + + Zehir +
+ Zehir + + Birkenstab @@ -485,6 +485,20 @@ Thanks goes to these wonderful people ✨ yajo + + + MakerMatrix +
+ MakerMatrix + + + + + weo +
+ weo + + analogue @@ -505,27 +519,20 @@ Thanks goes to these wonderful people ✨
reneploetz - - - - MakerMatrix -
- MakerMatrix - - + + pbek
pbek - - + - - keslerm + + yogo1212
- keslerm + yogo1212 @@ -542,6 +549,21 @@ Thanks goes to these wonderful people ✨ p-fruck + + + tbutter +
+ tbutter + + + + + rahilarious +
+ rahilarious + + + Rillke @@ -562,8 +584,7 @@ Thanks goes to these wonderful people ✨
r-pufky - - + vincentDcmps @@ -571,6 +592,21 @@ Thanks goes to these wonderful people ✨ vincentDcmps + + + frugan-dev +
+ frugan-dev + + + + + mpanneck +
+ mpanneck + + + andymel123 @@ -605,49 +641,13 @@ Thanks goes to these wonderful people ✨
lokipo - - + msheakoski
msheakoski - - - - GoliathLabs -
- GoliathLabs - - - - - frugan-dev -
- frugan-dev - - - - - tbutter -
- tbutter - - - - - yogo1212 -
- yogo1212 - - - - - mpanneck -
- mpanneck - @@ -657,6 +657,13 @@ Thanks goes to these wonderful people ✨ willtho89 + + + GoliathLabs +
+ GoliathLabs + + ubenmackin @@ -684,15 +691,15 @@ Thanks goes to these wonderful people ✨
andrewlow - + + aminvakil
aminvakil - - + elbracht @@ -727,21 +734,14 @@ Thanks goes to these wonderful people ✨
DuncanvR - + + emazzotta
emazzotta - - - - - nueaf -
- nueaf - @@ -772,25 +772,25 @@ Thanks goes to these wonderful people ✨ - - millaguie + + nueaf
- millaguie + nueaf - - eltociear + + keslerm
- eltociear + keslerm - - H4R0 + + millaguie
- H4R0 + millaguie @@ -800,13 +800,6 @@ Thanks goes to these wonderful people ✨ jamesfryer - - - ipernet -
- ipernet - - fl42 @@ -815,18 +808,25 @@ Thanks goes to these wonderful people ✨ - - radicand + + H4R0
- radicand + H4R0 + + + + + ipernet +
+ ipernet - - sjmudd + + neuralp
- sjmudd + neuralp @@ -879,20 +879,6 @@ Thanks goes to these wonderful people ✨ TechnicLab - - - 42wim -
- 42wim - - - - - vilisas -
- vilisas - - thomasschmit @@ -906,8 +892,7 @@ Thanks goes to these wonderful people ✨
Thiritin - - + tweibert @@ -921,7 +906,8 @@ Thanks goes to these wonderful people ✨
torus - + + VictorKoenders @@ -949,8 +935,36 @@ Thanks goes to these wonderful people ✨
Drakulix + + + + vilisas +
+ vilisas + + + + + 42wim +
+ 42wim + + + + ShiriNmi1520 +
+ ShiriNmi1520 + + + + + radicand +
+ radicand + + nilshoell @@ -978,7 +992,8 @@ Thanks goes to these wonderful people ✨
OrvilleQ - + + ovidiucp @@ -992,8 +1007,7 @@ Thanks goes to these wonderful people ✨
mrPjer - - + p3dda @@ -1015,20 +1029,6 @@ Thanks goes to these wonderful people ✨ piwai - - - auchri -
- auchri - - - - - rahilarious -
- rahilarious - - remoe @@ -1102,10 +1102,10 @@ Thanks goes to these wonderful people ✨ - - ShiriNmi1520 + + sjmudd
- ShiriNmi1520 + sjmudd @@ -1424,6 +1424,20 @@ Thanks goes to these wonderful people ✨ + + + 0xflotus +
+ 0xflotus + + + + + auchri +
+ auchri + + arkanovicz @@ -1451,7 +1465,8 @@ Thanks goes to these wonderful people ✨
espitall - + + dkarski @@ -1465,8 +1480,7 @@ Thanks goes to these wonderful people ✨
dbellavista - - + danielvandenberg95 @@ -1494,7 +1508,8 @@ Thanks goes to these wonderful people ✨
mazzz1y - + + doominator42 @@ -1508,8 +1523,7 @@ Thanks goes to these wonderful people ✨
aydodo - - + vedtam @@ -1537,7 +1551,8 @@ Thanks goes to these wonderful people ✨
ekkis - + + ErikEngerd @@ -1551,8 +1566,7 @@ Thanks goes to these wonderful people ✨
huncode - - + felixn @@ -1567,20 +1581,6 @@ Thanks goes to these wonderful people ✨ flole - - - akkumar -
- akkumar - - - - - 0xflotus -
- 0xflotus - - ifokeev @@ -1732,6 +1732,20 @@ Thanks goes to these wonderful people ✨ crash7 + + + froks +
+ froks + + + + + akkumar +
+ akkumar + + thechubbypanda @@ -1752,7 +1766,8 @@ Thanks goes to these wonderful people ✨
khuedoan - + + UltraCoderRU @@ -1766,8 +1781,7 @@ Thanks goes to these wonderful people ✨
JustAnother1 - - + LeoWinterDE @@ -1795,7 +1809,8 @@ Thanks goes to these wonderful people ✨
LucidityCrash - + + MadsRC @@ -1809,8 +1824,7 @@ Thanks goes to these wonderful people ✨
madmath03 - - + maxemann96 @@ -1838,7 +1852,8 @@ Thanks goes to these wonderful people ✨
exhuma - + + milas @@ -1852,8 +1867,7 @@ Thanks goes to these wonderful people ✨
mcchots - - + MohammedNoureldin @@ -1875,20 +1889,6 @@ Thanks goes to these wonderful people ✨ naveensrinivasan - - - neuralp -
- neuralp - - - - - froks -
- froks - - fkefer From 082e076377b48378761890a6a39acfda2a0db68d Mon Sep 17 00:00:00 2001 From: Inseo Song Date: Thu, 28 Mar 2024 13:02:11 +0900 Subject: [PATCH 34/53] docs: Add relay host config guide for Gmail (#3958) Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com> --- .../advanced/mail-forwarding/gmail-smtp.md | 50 +++++++++++++++++++ docs/mkdocs.yml | 1 + 2 files changed, 51 insertions(+) create mode 100644 docs/content/config/advanced/mail-forwarding/gmail-smtp.md diff --git a/docs/content/config/advanced/mail-forwarding/gmail-smtp.md b/docs/content/config/advanced/mail-forwarding/gmail-smtp.md new file mode 100644 index 00000000..a581acd1 --- /dev/null +++ b/docs/content/config/advanced/mail-forwarding/gmail-smtp.md @@ -0,0 +1,50 @@ +--- +title: 'Mail Forwarding | Configure Gmail as a relay host' +--- + +This page provides a guide for configuring DMS to use [GMAIL as an SMTP relay host][gmail-smtp]. + +!!! example "Configuration via ENV" + + [Configure a relay host in DMS][docs::relay]. This example shows how the related ENV settings map to the Gmail service config: + + - `RELAY_HOST` should be configured as [advised by Gmail][gmail-smtp::relay-host], there are two SMTP endpoints to choose: + - `smtp.gmail.com` (_for a personal Gmail account_) + - `smtp-relay.gmail.com` (_when using Google Workspace_) + - `RELAY_PORT` should be set to [one of the supported Gmail SMTP ports][gmail-smtp::relay-port] (_eg: 587 for STARTTLS_). + - `RELAY_USER` should be your gmail address (`user@gmail.com`). + - `RELAY_PASSWORD` should be your [App Password][gmail-smtp::app-password], **not** your personal gmail account password. + + ```env + RELAY_HOST=smtp.gmail.com + RELAY_PORT=587 + # Alternative to RELAY_HOST + RELAY_PORT which is compatible with LDAP: + DEFAULT_RELAY_HOST=[smtp.gmail.com]:587 + + RELAY_USER=username@gmail.com + RELAY_PASSWORD=secret + ``` + +!!! tip + + - As per our main [relay host docs page][docs::relay], you may prefer to configure your credentials via `setup relay add-auth` instead of the `RELAY_USER` + `RELAY_PASSWORD` ENV. + - If you configure for `smtp-relay.gmail.com`, the `DEFAULT_RELAY_HOST` ENV should be all you need as shown in the above example. Credentials can be optional when using Google Workspace (`smtp-relay.gmail.com`), which supports restricting connections to trusted IP addresses. + +!!! note "Verify the relay host is configured correctly" + + To verify proper operation, send an email to an external account of yours and inspect the mail headers. + + You will also see the connection to the Gmail relay host (`smtp.gmail.com`) in the mail logs: + + ```log + postfix/smtp[910]: Trusted TLS connection established to smtp.gmail.com[64.233.188.109]:587: + TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) + postfix/smtp[910]: 4BCB547D9D: to=, relay=smtp.gmail.com[64.233.188.109]:587, + delay=2.9, delays=0.01/0.02/1.7/1.2, dsn=2.0.0, status=sent (250 2.0.0 OK 17... - gsmtp) + ``` + +[docs::relay]: ./relay-hosts.md +[gmail-smtp]: https://support.google.com/a/answer/2956491 +[gmail-smtp::relay-host]: https://support.google.com/a/answer/176600 +[gmail-smtp::relay-port]: https://support.google.com/a/answer/2956491 +[gmail-smtp::app-password]: https://support.google.com/accounts/answer/185833 diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index e0ff50c9..9d7fc81d 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -161,6 +161,7 @@ nav: - 'Email Forwarding': - 'Relay Hosts': config/advanced/mail-forwarding/relay-hosts.md - 'AWS SES': config/advanced/mail-forwarding/aws-ses.md + - 'Configure Gmail as a relay host': config/advanced/mail-forwarding/gmail-smtp.md - 'Full-Text Search': config/advanced/full-text-search.md - 'Kubernetes': config/advanced/kubernetes.md - 'IPv6': config/advanced/ipv6.md From 4f10089c90625eb4ccb8bff1cf6e40630e0d8b6c Mon Sep 17 00:00:00 2001 From: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Date: Fri, 29 Mar 2024 02:07:13 +0100 Subject: [PATCH 35/53] docs: add note about custom F2B setup with PROXY protocol (#3964) --- docs/content/config/advanced/kubernetes.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/config/advanced/kubernetes.md b/docs/content/config/advanced/kubernetes.md index 97a7e414..8f5ac901 100644 --- a/docs/content/config/advanced/kubernetes.md +++ b/docs/content/config/advanced/kubernetes.md @@ -26,7 +26,7 @@ If using our Helm chart is not viable for you, here is some guidance to start wi === "`ConfigMap`" Provide the basic configuration via environment variables with a `ConfigMap`. - + !!! example Below is only an example configuration, adjust the `ConfigMap` to your own needs. @@ -512,6 +512,7 @@ Kubernetes provides multiple ways to address this; each has its upsides and down - Kubernetes manifest changes for the DMS configured `Service` - DMS configuration changes for Postfix and Dovecot - [ ] To keep support for direct connections to DMS services internally within cluster, service ports must be "duplicated" to offer an alternative port for connections using PROXY protocol + - [ ] Custom Fail2Ban required: Because the traffic to DMS is now coming from the proxy, banning the origin IP address will have no effect; you'll need to implement a [custom solution for your setup][github-web::docker-mailserver::proxy-protocol-fail2ban]. ??? question "What is the PROXY protocol?" @@ -795,3 +796,4 @@ Kubernetes provides multiple ways to address this; each has its upsides and down [k8s-docs::network-external-ip]: https://kubernetes.io/docs/concepts/services-networking/service/#external-ips [traefik-docs::k8s::ingress-route-tcp]: https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-ingressroutetcp +[github-web::docker-mailserver::proxy-protocol-fail2ban]: https://github.com/docker-mailserver/docker-mailserver/issues/1761#issuecomment-2016879319 From 6733a172d77480e1cbc919125c7d9b7b7fd467ac Mon Sep 17 00:00:00 2001 From: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Date: Sun, 31 Mar 2024 04:14:02 +0200 Subject: [PATCH 36/53] docs: add FAQ entry about DNS servers and drop feature request on custom DNS servers for Rspamd (#3966) * add FAQ entry about DNS servers I also opted for including a quote from @polarthene which illustrates how DNS servers are a difficult topic and should not be DMS' responsibility. * link to DNS FAQ from Rspamd page & drop feature request The feature request annotation has been removed because we decided it's not DMS responsibility to ensure correctly working DNS servers. --- docs/content/config/security/rspamd.md | 8 +++----- docs/content/faq.md | 13 ++++++++++++- 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/docs/content/config/security/rspamd.md b/docs/content/config/security/rspamd.md index 5cb901b7..fd0fe25e 100644 --- a/docs/content/config/security/rspamd.md +++ b/docs/content/config/security/rspamd.md @@ -94,11 +94,7 @@ Rspamd provides a [web interface][rspamd-docs::web-interface], which contains st ### DNS -DMS does not supply custom values for DNS servers to Rspamd. If you need to use custom DNS servers, which could be required when using [DNS-based black/whitelists](#rbls-realtime-blacklists-dnsbls-dns-based-blacklists), you need to adjust [`options.inc`][rspamd-docs::basic-options] yourself. - -!!! tip "Making DNS Servers Configurable" - - If you want to see an environment variable (like `RSPAMD_DNS_SERVERS`) to support custom DNS servers for Rspamd being added to DMS, please raise a feature request issue. +DMS does not supply custom values for DNS servers (to Rspamd). If you need to use custom DNS servers, which could be required when using [DNS-based deny/allowlists](#rbls-real-time-blacklists-dnsbls-dns-based-blacklists), you need to adjust [`options.inc`][rspamd-docs::basic-options] yourself. Make sure to also read our [FAQ page on DNS servers][docs::faq::dns-servers]. !!! warning @@ -270,3 +266,5 @@ While _Abusix_ can be integrated into Postfix, Postscreen and a multitude of oth [docs::dms-volumes-config]: ../advanced/optional-config.md#volumes-config [docs::dms-volumes-state]: ../advanced/optional-config.md#volumes-state + +[docs::faq::dns-servers]: ../../faq.md#what-about-dns-servers diff --git a/docs/content/faq.md b/docs/content/faq.md index 6b1782e3..4add0589 100644 --- a/docs/content/faq.md +++ b/docs/content/faq.md @@ -79,6 +79,14 @@ volumes: Optionally, you can set the `TZ` ENV variable; e.g. `TZ=Europe/Berlin`. Check [this list](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for which values are allowed. +### What About DNS Servers? + +Properly working DNS servers are crucial for differentiating spam from legitimate e-mails. Records like `SPF`, `DKIM` and `DMARC` records, as well as working name (resolving `A` records) and reverse name (resolving `PTR` records) resolution ensures legitimate e-mails arrive while e-mails that are more likely phishing and spam do not. + +Anti-spam measures (like SpamAssassin or Rspamd) make use of DNS block lists. To learn more check out our [Rspamd documentation on this topic][docs::rspamd-rbl-dnsbl]. In case you want to utilize RBL/DNSBLs, you need a recursive DNS resolver (_not big custom resolvers like Cloudflare, Quad9, Google, etc._). + +DMS does not integrate support for an internal DNS service as this is a [responsibility that is sensitive to the host environment][gh-discussion::dms-avoid-maintaining-internal-dns]. You can configure internal services within DMS to use your own managed DNS server, or configure for such at the host or container level (_such as with [`compose.yaml`][docker-compose::docs::config-dns]_). + ### What is the file format? All files are using the Unix format with `LF` line endings. Please do not use `CRLF`. @@ -376,7 +384,7 @@ The default setup `@local_domains_acl = ( ".$mydomain" );` does not match subdom Put received spams in `.Junk/` imap folder using `SPAMASSASSIN_SPAM_TO_INBOX=1` and `MOVE_SPAM_TO_JUNK=1` and add a _user_ cron like the following: -!!! example +!!! example **NOTE:** This example assumes you have a [`/var/mail-state` volume][docs::dms-volumes-state] mounted. @@ -482,6 +490,7 @@ $spam_quarantine_to = "quarantine\@example.com"; [fail2ban-customize]: ./config/security/fail2ban.md [docs::dms-volumes-state]: ./config/advanced/optional-config.md#volumes-state +[docs::rspamd-rbl-dnsbl]: ./config/security/rspamd.md#rbls-real-time-blacklists-dnsbls-dns-based-blacklists [docs-maintenance]: ./config/advanced/maintenance/update-and-cleanup.md [docs-override-postfix]: ./config/advanced/override-defaults/postfix.md [docs-userpatches]: ./config/advanced/override-defaults/user-patches.md @@ -495,4 +504,6 @@ $spam_quarantine_to = "quarantine\@example.com"; [github-issue-1405-comment]: https://github.com/docker-mailserver/docker-mailserver/issues/1405#issuecomment-590106498 [github-issue-1639]: https://github.com/docker-mailserver/docker-mailserver/issues/1639 [github-issue-1792]: https://github.com/docker-mailserver/docker-mailserver/pull/1792 +[gh-discussion::dms-avoid-maintaining-internal-dns]: https://github.com/orgs/docker-mailserver/discussions/3959#discussioncomment-8956322 +[docker-compose::docs::config-dns]: https://docs.docker.com/compose/compose-file/compose-file-v3/#dns [hanscees-userpatches]: https://github.com/hanscees/dockerscripts/blob/master/scripts/tomav-user-patches.sh From d502dae0687b25c81aa3f010d88ad5552bc9ab41 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 1 Apr 2024 17:20:35 +0200 Subject: [PATCH 37/53] docs: updated `CONTRIBUTORS.md` (#3967) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CONTRIBUTORS.md | 329 ++++++++++++++++++++++++------------------------ 1 file changed, 168 insertions(+), 161 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 7c81c968..ea523c94 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -19,6 +19,13 @@ Thanks goes to these wonderful people ✨ fbartels + + + polarathene +
+ polarathene + + NorseGaud @@ -39,15 +46,15 @@ Thanks goes to these wonderful people ✨
wernerfred - + + georglauterbach
georglauterbach - - + tomav @@ -55,13 +62,6 @@ Thanks goes to these wonderful people ✨ tomav - - - polarathene -
- polarathene - - erik-wramner @@ -743,6 +743,20 @@ Thanks goes to these wonderful people ✨ emazzotta + + + keslerm +
+ keslerm + + + + + nueaf +
+ nueaf + + martinwepner @@ -763,7 +777,8 @@ Thanks goes to these wonderful people ✨
spacecowboy - + + jedateach @@ -771,28 +786,6 @@ Thanks goes to these wonderful people ✨ jedateach - - - nueaf -
- nueaf - - - - - - keslerm -
- keslerm - - - - - millaguie -
- millaguie - - jamesfryer @@ -800,13 +793,6 @@ Thanks goes to these wonderful people ✨ jamesfryer - - - fl42 -
- fl42 - - H4R0 @@ -814,12 +800,26 @@ Thanks goes to these wonderful people ✨ H4R0 + + + millaguie +
+ millaguie + + ipernet
ipernet + + + + fl42 +
+ fl42 + @@ -829,6 +829,13 @@ Thanks goes to these wonderful people ✨ neuralp + + + sjmudd +
+ sjmudd + + simonsystem @@ -856,15 +863,15 @@ Thanks goes to these wonderful people ✨
5ven - + + syl20bnr
syl20bnr - - + sylvaindumont @@ -879,13 +886,28 @@ Thanks goes to these wonderful people ✨ TechnicLab + + + 42wim +
+ 42wim + + + + + vilisas +
+ vilisas + + thomasschmit
thomasschmit - + + Thiritin @@ -906,8 +928,7 @@ Thanks goes to these wonderful people ✨
torus - - + VictorKoenders @@ -928,7 +949,8 @@ Thanks goes to these wonderful people ✨
k3it - + + Drakulix @@ -936,28 +958,6 @@ Thanks goes to these wonderful people ✨ Drakulix - - - vilisas -
- vilisas - - - - - 42wim -
- 42wim - - - - - - ShiriNmi1520 -
- ShiriNmi1520 - - radicand @@ -1015,6 +1015,13 @@ Thanks goes to these wonderful people ✨ p3dda + + + auchri +
+ auchri + + peter-hartmann @@ -1028,15 +1035,15 @@ Thanks goes to these wonderful people ✨
piwai - + + remoe
remoe - - + robbertkl @@ -1071,15 +1078,15 @@ Thanks goes to these wonderful people ✨
501st-alpha1 - + + klamann
klamann - - + svdb0 @@ -1102,10 +1109,10 @@ Thanks goes to these wonderful people ✨ - - sjmudd + + ShiriNmi1520
- sjmudd + ShiriNmi1520 @@ -1114,15 +1121,15 @@ Thanks goes to these wonderful people ✨
mchamplain - + + millerjason
millerjason - - + mplx @@ -1157,15 +1164,15 @@ Thanks goes to these wonderful people ✨
ontheair81 - + + pravynandas
pravynandas - - + presocratics @@ -1200,15 +1207,15 @@ Thanks goes to these wonderful people ✨
schnippl0r - + + smargold476
smargold476 - - + sportshead @@ -1243,15 +1250,15 @@ Thanks goes to these wonderful people ✨
vivacarvajalito - + + wligtenberg
wligtenberg - - + wolkenschieber @@ -1286,15 +1293,15 @@ Thanks goes to these wonderful people ✨
arcaine2 - + + awb99
awb99 - - + brainkiller @@ -1329,15 +1336,15 @@ Thanks goes to these wonderful people ✨
eleith - + + ghnp5
ghnp5 - - + helmutundarnold @@ -1372,15 +1379,15 @@ Thanks goes to these wonderful people ✨
ixeft - + + jjtt
jjtt - - + paralax @@ -1415,28 +1422,14 @@ Thanks goes to these wonderful people ✨
marios88 - + + matrixes
matrixes - - - - - 0xflotus -
- 0xflotus - - - - - auchri -
- auchri - @@ -1465,15 +1458,15 @@ Thanks goes to these wonderful people ✨
espitall - - + dkarski
dkarski - + + dbellavista @@ -1508,15 +1501,15 @@ Thanks goes to these wonderful people ✨
mazzz1y - - + doominator42
doominator42 - + + aydodo @@ -1551,15 +1544,15 @@ Thanks goes to these wonderful people ✨
ekkis - - + ErikEngerd
ErikEngerd - + + huncode @@ -1581,21 +1574,35 @@ Thanks goes to these wonderful people ✨ flole + + + Kaan88 +
+ Kaan88 + + + + + 0xflotus +
+ 0xflotus + + ifokeev
ifokeev - + + 20th
20th - - + 2b @@ -1630,15 +1637,15 @@ Thanks goes to these wonderful people ✨
vifino - + + kachkaev
kachkaev - - + alexanderneu @@ -1673,15 +1680,15 @@ Thanks goes to these wonderful people ✨
green-anger - + + iRhonin
iRhonin - - + MrFreezeex @@ -1716,15 +1723,15 @@ Thanks goes to these wonderful people ✨
spock - + + erdos4d
erdos4d - - + crash7 @@ -1732,13 +1739,6 @@ Thanks goes to these wonderful people ✨ crash7 - - - froks -
- froks - - akkumar @@ -1889,14 +1889,21 @@ Thanks goes to these wonderful people ✨ naveensrinivasan + + + froks +
+ froks + + + fkefer
fkefer - - + Marsu31 @@ -1931,15 +1938,15 @@ Thanks goes to these wonderful people ✨
HeySora - + + sirgantrithon
sirgantrithon - - + Influencer @@ -1947,6 +1954,13 @@ Thanks goes to these wonderful people ✨ Influencer + + + in-seo +
+ in-seo + + JacksonZ03 @@ -1967,7 +1981,8 @@ Thanks goes to these wonderful people ✨
jcalfee - + + mivek @@ -1981,8 +1996,7 @@ Thanks goes to these wonderful people ✨
init-js - - + Jeidnx @@ -2010,7 +2024,8 @@ Thanks goes to these wonderful people ✨
jirislav - + + jmccl @@ -2024,21 +2039,13 @@ Thanks goes to these wonderful people ✨
jurekbarth - - + JOduMonT
JOduMonT - - - - Kaan88 -
- Kaan88 - From 8c5cf03203ff164a89f45848e542256f2725bc21 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 8 Apr 2024 17:56:12 +0200 Subject: [PATCH 38/53] chore(deps): Bump docker/setup-buildx-action from 3.2.0 to 3.3.0 (#3972) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/generic_build.yml | 2 +- .github/workflows/generic_publish.yml | 2 +- .github/workflows/generic_test.yml | 2 +- .github/workflows/generic_vulnerability-scan.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/generic_build.yml b/.github/workflows/generic_build.yml index f0bd2111..da5d51c1 100644 --- a/.github/workflows/generic_build.yml +++ b/.github/workflows/generic_build.yml @@ -79,7 +79,7 @@ jobs: platforms: arm64 - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.2.0 + uses: docker/setup-buildx-action@v3.3.0 # NOTE: AMD64 can build within 2 minutes - name: 'Build images' diff --git a/.github/workflows/generic_publish.yml b/.github/workflows/generic_publish.yml index 266090b6..ead38a3f 100644 --- a/.github/workflows/generic_publish.yml +++ b/.github/workflows/generic_publish.yml @@ -40,7 +40,7 @@ jobs: platforms: arm64 - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.2.0 + uses: docker/setup-buildx-action@v3.3.0 # Try get the cached build layers from a prior `generic_build.yml` job. # NOTE: Until adopting `type=gha` scoped cache exporter (in `docker/build-push-action`), diff --git a/.github/workflows/generic_test.yml b/.github/workflows/generic_test.yml index 4c49ca3e..4e648e32 100644 --- a/.github/workflows/generic_test.yml +++ b/.github/workflows/generic_test.yml @@ -38,7 +38,7 @@ jobs: # Ensures consistent BuildKit version (not coupled to Docker Engine), # and increased compatibility of the build cache vs mixing buildx drivers. - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.2.0 + uses: docker/setup-buildx-action@v3.3.0 # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. diff --git a/.github/workflows/generic_vulnerability-scan.yml b/.github/workflows/generic_vulnerability-scan.yml index 23544820..2f1a3803 100644 --- a/.github/workflows/generic_vulnerability-scan.yml +++ b/.github/workflows/generic_vulnerability-scan.yml @@ -37,7 +37,7 @@ jobs: # Ensures consistent BuildKit version (not coupled to Docker Engine), # and increased compatibility of the build cache vs mixing buildx drivers. - name: 'Set up Docker Buildx' - uses: docker/setup-buildx-action@v3.2.0 + uses: docker/setup-buildx-action@v3.3.0 # Importing from the cache should create the image within approx 30 seconds: # NOTE: `qemu` step is not needed as we only test for AMD64. From ad5d1011f82401415421b0738209d855bcb12443 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 11 Apr 2024 10:18:17 +0200 Subject: [PATCH 39/53] docs: updated `CONTRIBUTORS.md` (#3971) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CONTRIBUTORS.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index ea523c94..01c7897f 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -1783,10 +1783,10 @@ Thanks goes to these wonderful people ✨ - - LeoWinterDE + + leowinterde
- LeoWinterDE + leowinterde From f2314259829d170984eacd5790406a6f745ae032 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 15 Apr 2024 18:05:07 +0200 Subject: [PATCH 40/53] chore(deps): Bump peaceiris/actions-gh-pages from 3.9.3 to 4.0.0 (#3978) --- .github/workflows/docs-production-deploy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/docs-production-deploy.yml b/.github/workflows/docs-production-deploy.yml index cb8bdbed..2c6c1e2c 100644 --- a/.github/workflows/docs-production-deploy.yml +++ b/.github/workflows/docs-production-deploy.yml @@ -59,7 +59,7 @@ jobs: {} + - name: 'Deploy to Github Pages' - uses: peaceiris/actions-gh-pages@v3.9.3 + uses: peaceiris/actions-gh-pages@v4.0.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} # Build directory contents to publish to the `gh-pages` branch: From dc5185003058dbdb1d18947a870a09c43a275d85 Mon Sep 17 00:00:00 2001 From: fanqiaojun <166898955+fanqiaojun@users.noreply.github.com> Date: Tue, 16 Apr 2024 03:48:55 +0800 Subject: [PATCH 41/53] chore: remove repetitive words (#3977) --- docs/content/config/advanced/auth-ldap.md | 2 +- docs/content/examples/use-cases/auth-lua.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/config/advanced/auth-ldap.md b/docs/content/config/advanced/auth-ldap.md index ea24526d..397e42eb 100644 --- a/docs/content/config/advanced/auth-ldap.md +++ b/docs/content/config/advanced/auth-ldap.md @@ -26,7 +26,7 @@ Those variables contain the LDAP lookup filters for postfix, using `%s` as the p - Technically, there is no difference between `ALIAS` and `GROUP`, but ideally you should use `ALIAS` for personal aliases for a singular person (like `ceo@example.org`) and `GROUP` for multiple people (like `hr@example.org`). - ...for outgoing email, the sender address is put through the `SENDERS` filter, and only if the authenticated user is one of the returned entries, the email can be sent. - This only applies if `SPOOF_PROTECTION=1`. - - If the `SENDERS` filter is missing, the `USER`, `ALIAS` and `GROUP` filters will be used in in a disjunction (OR). + - If the `SENDERS` filter is missing, the `USER`, `ALIAS` and `GROUP` filters will be used in a disjunction (OR). - To for example allow users from the `admin` group to spoof any sender email address, and to force everyone else to only use their personal mailbox address for outgoing email, you can use something like this: `(|(memberOf=cn=admin,*)(mail=%s))` ???+ example diff --git a/docs/content/examples/use-cases/auth-lua.md b/docs/content/examples/use-cases/auth-lua.md index 82586885..e34ed3b0 100644 --- a/docs/content/examples/use-cases/auth-lua.md +++ b/docs/content/examples/use-cases/auth-lua.md @@ -34,7 +34,7 @@ A drawback of this method is that any (compromised) Nextcloud application passwo To answer the questions asked earlier for this specific scenario: -1. Do I want to use Lua to identify mailboxes and verify that users are are authorized to use mail services? **No. Provisioning is done through LDAP.** +1. Do I want to use Lua to identify mailboxes and verify that users are authorized to use mail services? **No. Provisioning is done through LDAP.** 1. Do I want to use Lua to verify passwords that users authenticate with for IMAP/POP3/SMTP in their mail clients? **Yes. Password authentication is done through Lua against Nextcloud.** 1. If the answer is 'yes' to question 1 or 2: are there other methods that better facilitate my use case instead of custom scripts which rely on me being a developer and not just a user? **No. Only HTTP can be used to authenticate against Nextcloud, which is not supported natively by Dovecot or DMS.** From d87e4d3bfd7780def3c187b0f31c9034954cdfe7 Mon Sep 17 00:00:00 2001 From: Iztok Fister Jr Date: Tue, 16 Apr 2024 22:25:45 +0200 Subject: [PATCH 42/53] docs: Fix typos (#3979) --- docs/content/config/environment.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/config/environment.md b/docs/content/config/environment.md index 9adfd5d9..867c7459 100644 --- a/docs/content/config/environment.md +++ b/docs/content/config/environment.md @@ -456,8 +456,8 @@ Default: 6 (which corresponds to the `add_header` action) ##### RSPAMD_NEURAL -Can be used to enable or disable the [Neural network module][rspamd-docs-neural-network]. This is an experimental anti-spam weigh method using three neuaral networks in the configuration added here. As far as we can tell it trains itsself by using other modules to find out what spam is. It will take a while (a week or more) to train its first neural network. The config trains new networks all the time and discards of old networks. -Since it is experimental it is switched of by default. +Can be used to enable or disable the [Neural network module][rspamd-docs-neural-network]. This is an experimental anti-spam weigh method using three neural networks in the configuration added here. As far as we can tell it trains itself by using other modules to find out what spam is. It will take a while (a week or more) to train its first neural network. The config trains new networks all the time and discards old networks. +Since it is experimental, it is switched off by default. - **0** => Disabled - 1 => Enabled From 942920615c0ab3d9946d6ff1db6a83f9034351cc Mon Sep 17 00:00:00 2001 From: Tobia Bocchi <29007647+tobiabocchi@users.noreply.github.com> Date: Thu, 18 Apr 2024 03:08:26 +0200 Subject: [PATCH 43/53] docs: Fix typo on usage page (#3980) --- docs/content/usage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/usage.md b/docs/content/usage.md index 76011fe7..087547ea 100644 --- a/docs/content/usage.md +++ b/docs/content/usage.md @@ -11,7 +11,7 @@ This page explains how to get started with DMS. The guide uses Docker Compose as Before you can get started with deploying your own mail server, there are some requirements to be met: 1. You need to have a host that you can manage. -2. You need to own a domain, and you need to able to manage DNS for this domain. +2. You need to own a domain, and you need to be able to manage DNS for this domain. ### Host Setup From ac22caf74eff031ad8086ac147949c0b15c24398 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Sat, 20 Apr 2024 11:25:02 +1200 Subject: [PATCH 44/53] docs: Updates to TLS page (Caddy, testing, etc) (#3981) --- docs/content/config/security/ssl.md | 227 +++++++++++++++------------- 1 file changed, 124 insertions(+), 103 deletions(-) diff --git a/docs/content/config/security/ssl.md b/docs/content/config/security/ssl.md index a9174483..a5ba005a 100644 --- a/docs/content/config/security/ssl.md +++ b/docs/content/config/security/ssl.md @@ -481,113 +481,108 @@ DSM-generated letsencrypt certificates get auto-renewed every three months. ### Caddy -For Caddy v2 you can specify the `key_type` in your server's global settings, which would end up looking something like this if you're using a `Caddyfile`: +[Caddy][web::caddy] is an open-source web server with built-in TLS certificate generation. You can use the [official Docker image][dockerhub::caddy] and write your own `Caddyfile`. -```caddyfile -{ - debug - admin localhost:2019 - http_port 80 - https_port 443 - default_sni example.com - key_type rsa2048 -} -``` +!!! example -If you are instead using a json config for Caddy v2, you can set it in your site's TLS automation policies: + ```yaml title="compose.yaml" + services: + # Basic Caddy service to provision certs: + reverse-proxy: + image: caddy:2.7 + ports: + - 80:80 + - 443:443 + volumes: + - ./Caddyfile:/etc/caddy/Caddyfile:ro + - ${CADDY_DATA_DIR}:/data -??? example "Caddy v2 JSON example snippet" + # Share the Caddy data volume for certs and configure SSL_TYPE to `letsencrypt` + mailserver: + image: ghcr.io/docker-mailserver/docker-mailserver:latest + hostname: mail.example.com + environment: + SSL_TYPE: letsencrypt + # While you could use a named data volume instead of a bind mount volume, it would require the long-syntax to rename cert files: + # https://docs.docker.com/compose/compose-file/05-services/#volumes + volumes: + - ${CADDY_DATA_DIR}/certificates/acme-v02.api.letsencrypt.org-directory/mail.example.com/mail.example.com.crt:/etc/letsencrypt/live/mail.example.com/fullchain.pem + - ${CADDY_DATA_DIR}/certificates/acme-v02.api.letsencrypt.org-directory/mail.example.com/mail.example.com.key:/etc/letsencrypt/live/mail.example.com/privkey.pem + ``` - ```json - { - "apps": { - "http": { - "servers": { - "srv0": { - "listen": [ - ":443" - ], - "routes": [ - { - "match": [ - { - "host": [ - "mail.example.com", - ] - } - ], - "handle": [ - { - "handler": "subroute", - "routes": [ - { - "handle": [ - { - "body": "", - "handler": "static_response" - } - ] - } - ] - } - ], - "terminal": true - }, - ] - } - } - }, - "tls": { - "automation": { - "policies": [ - { - "subjects": [ - "mail.example.com", - ], - "key_type": "rsa2048", - "issuer": { - "email": "admin@example.com", - "module": "acme" - } - }, - { - "issuer": { - "email": "admin@example.com", - "module": "acme" - } - } - ] - } - } + ```caddyfile title="Caddyfile" + mail.example.com { + tls internal { + key_type rsa2048 } + + # Optional, can be useful for troubleshooting + # connection to Caddy with correct certificate: + respond "Hello DMS" } ``` -The generated certificates can then be mounted: + While DMS does not need a webserver to work, this workaround will provision a TLS certificate for DMS to use. -```yaml -volumes: - - ${CADDY_DATA_DIR}/certificates/acme-v02.api.letsencrypt.org-directory/mail.example.com/mail.example.com.crt:/etc/letsencrypt/live/mail.example.com/fullchain.pem - - ${CADDY_DATA_DIR}/certificates/acme-v02.api.letsencrypt.org-directory/mail.example.com/mail.example.com.key:/etc/letsencrypt/live/mail.example.com/privkey.pem -``` + - [`tls internal`][caddy-docs::tls-internal] will create a local self-signed cert for testing. This targets only the site-address, unlike the global `local_certs` option. + - [`key_type`][caddy-docs::key-type] can be used in the `tls` block if you need to enforce RSA as the key type for certificates provisioned. The default is currently ECDSA (P-256). -### Traefik v2 +??? example "With `caddy-docker-proxy`" -[Traefik][traefik::github] is an open-source application proxy using the [ACME protocol][ietf::rfc::acme]. [Traefik][traefik::github] can request certificates for domains and subdomains, and it will take care of renewals, challenge negotiations, etc. We strongly recommend to use [Traefik][traefik::github]'s major version 2. + Using [`lucaslorentz/caddy-docker-proxy`][github::caddy-docker-proxy] allows you to generate a `Caddyfile` by adding labels to your services in `compose.yaml`: -[Traefik][traefik::github]'s storage format is natively supported if the `acme.json` store is mounted into the container at `/etc/letsencrypt/acme.json`. The file is also monitored for changes and will trigger a reload of the mail services (Postfix and Dovecot). + ```yaml title="compose.yaml" + services: + reverse-proxy: + image: lucaslorentz/caddy-docker-proxy:2.8 + ports: + - 80:80 + - 443:443 + volumes: + - /var/run/docker.sock:/var/run/docker.sock + - ${CADDY_DATA_DIR}:/data + labels: + # Set global config here, this option has an empty value to enable self-signed certs for local testing: + # NOTE: Remove this label when going to production. + caddy.local_certs: "" -Wildcard certificates are supported. If your FQDN is `mail.example.com` and your wildcard certificate is `*.example.com`, add the ENV: `#!bash SSL_DOMAIN=example.com`. + # Use labels to configure Caddy to provision DMS certs + mailserver: + image: ghcr.io/docker-mailserver/docker-mailserver:latest + hostname: mail.example.com + environment: + SSL_TYPE: letsencrypt + volumes: + - ${CADDY_DATA_DIR}/certificates/acme-v02.api.letsencrypt.org-directory/mail.example.com/mail.example.com.crt:/etc/letsencrypt/live/mail.example.com/fullchain.pem + - ${CADDY_DATA_DIR}/certificates/acme-v02.api.letsencrypt.org-directory/mail.example.com/mail.example.com.key:/etc/letsencrypt/live/mail.example.com/privkey.pem + labels: + # Set your DMS FQDN here to add the site-address into the generated Caddyfile: + caddy_0: mail.example.com + # Add a dummy directive is required: + caddy_0.respond: "Hello DMS" + # Uncomment to make a proxy for Rspamd + # caddy_1: rspamd.example.com + # caddy_1.reverse_proxy: "{{upstreams 11334}}" + ``` -DMS will select it's certificate from `acme.json` checking these ENV for a matching FQDN (_in order of priority_): +!!! warning "Caddy certificate location varies" -1. `#!bash ${SSL_DOMAIN}` -2. `#!bash ${HOSTNAME}` -3. `#!bash ${DOMAINNAME}` + The path contains the certificate provisioner used. This path may be different from the example above for you and may change over time when multiple provisioner services are used][dms-pr-feedback::caddy-provisioning-gotcha]. -This setup only comes with one caveat: The domain has to be configured on another service for [Traefik][traefik::github] to actually request it from _Let's Encrypt_, i.e. [Traefik][traefik::github] will not issue a certificate without a service / router demanding it. + This can make the volume mounting for DMS to find the certificates non-deterministic, but you can [restrict provisioning to single service via the `acme_ca` setting][caddy::restrict-acme-provisioner]. + +### Traefik + +[Traefik][traefik::github] is an open-source application proxy using the [ACME protocol][ietf::rfc::acme]. Traefik can request certificates for domains and subdomains, and it will take care of renewals, challenge negotiations, etc. + +Traefik's storage format is natively supported if the `acme.json` store is mounted into the container at `/etc/letsencrypt/acme.json`. The file is also monitored for changes and will trigger a reload of the mail services (Postfix and Dovecot). + +DMS will select it's certificate from `acme.json` prioritizing a match for the DMS FQDN (hostname), while also checking one DNS level up (_eg: `mail.example.com` => `example.com`_). Wildcard certificates are supported. + +This setup only comes with one caveat - The domain has to be configured on another service for Traefik to actually request it from _Let's Encrypt_ (_i.e. Traefik will not issue a certificate without a service / router demanding it_). ???+ example "Example Code" + Here is an example setup for [`docker-compose`](https://docs.docker.com/compose/): ```yaml @@ -716,7 +711,7 @@ The local and internal paths may be whatever you prefer, so long as both `SSL_CE ## Testing a Certificate is Valid -- From your host: +!!! example "Connect to DMS on port 25" ```sh docker exec mailserver openssl s_client \ @@ -725,26 +720,42 @@ The local and internal paths may be whatever you prefer, so long as both `SSL_CE -CApath /etc/ssl/certs/ ``` -- Or: + The response should show the certificate chain with a line further down: `Verify return code: 0 (ok)` + + --- + + This example runs within the DMS container itself to verify the cert is working locally. + + - Adjust the `-connect` IP if testing externally from another system. Additionally testing for port 143 (Dovecot IMAP) is encouraged (_change the protocol for `-starttls` from `smtp` to `imap`_). + - `-CApath` will help verify the certificate chain, provided the location contains the root CA that signed your TLS cert for DMS. + +??? example "Verify certificate dates" ```sh docker exec mailserver openssl s_client \ - -connect 0.0.0.0:143 \ - -starttls imap \ - -CApath /etc/ssl/certs/ + -connect 0.0.0.0:25 \ + -starttls smtp \ + -CApath /etc/ssl/certs/ \ + 2>/dev/null | openssl x509 -noout -dates ``` -And you should see the certificate chain, the server certificate and: `Verify return code: 0 (ok)` +!!! tip "Testing and troubleshooting" -In addition, to verify certificate dates: + If you need to test a connection without resolving DNS, `curl` can connect with `--resolve` option to map an FQDN + Port to an IP address, instead of the request address provided. -```sh -docker exec mailserver openssl s_client \ - -connect 0.0.0.0:25 \ - -starttls smtp \ - -CApath /etc/ssl/certs/ \ - 2>/dev/null | openssl x509 -noout -dates -``` + ```bash + # NOTE: You may want to use `--insecure` if the cert was provisioned with a private CA not present on the curl client: + # Use `--verbose` for additional insights on the connection. + curl --resolve mail.example.com:443:127.0.0.1 https://mail.example.com + ``` + + Similarly with `openssl` you can connect to an IP as shown previously, but provide an explicit SNI if necessary with `-servername mail.example.com`. + + --- + + Both `curl` and `openssl` also support `-4` and `-6` for enforcing IPv4 or IPv6 lookup. + + This can be useful, such as when [DNS resolves the IP to different servers leading to different certificates returned][dms-discussion::gotcha-fqdn-bad-dns]. As shown in that link, `step certificate inspect` is also handy for viewing details of the cert returned or on disk. ## Plain-Text Access @@ -919,3 +930,13 @@ Despite this, if you must use non-standard DH parameters or you would like to sw [acme-companion::standalone]: https://github.com/nginx-proxy/acme-companion/blob/main/docs/Standalone-certificates.md [acme-companion::standalone-changes]: https://github.com/nginx-proxy/acme-companion/blob/main/docs/Standalone-certificates.md#picking-up-changes-to-letsencrypt_user_data [acme-companion::service-loop]: https://github.com/nginx-proxy/acme-companion/blob/main/docs/Container-utilities.md + +[web::caddy]: https://caddyserver.com +[dockerhub::caddy]: https://hub.docker.com/_/caddy +[github::caddy-docker-proxy]: https://github.com/lucaslorentz/caddy-docker-proxy +[dms-pr-feedback::caddy-provisioning-gotcha]: https://github.com/docker-mailserver/docker-mailserver/pull/3485/files#r1297512818 +[caddy-docs::tls-internal]: https://caddyserver.com/docs/caddyfile/directives/tls#syntax +[caddy-docs::key-type]: https://caddyserver.com/docs/caddyfile/options#key-type +[caddy::restrict-acme-provisioner]: https://caddy.community/t/is-there-a-way-on-a-caddyfile-to-force-a-specific-acme-ca/14506 + +[dms-discussion::gotcha-fqdn-bad-dns]: https://github.com/docker-mailserver/docker-mailserver/issues/3955#issuecomment-2027882633 From d739fe3785ec16dcda1dd2213f64298e2269ce72 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Mon, 22 Apr 2024 11:28:11 +1200 Subject: [PATCH 45/53] chore: Remove base-60 port quote warning from example `compose.yaml` (#3982) This should not be relevant to users of `docker compose` which is the primary demographic. --- compose.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/compose.yaml b/compose.yaml index e9c49596..8f5bfdb2 100644 --- a/compose.yaml +++ b/compose.yaml @@ -7,7 +7,6 @@ services: env_file: mailserver.env # More information about the mail-server ports: # https://docker-mailserver.github.io/docker-mailserver/latest/config/security/understanding-the-ports/ - # To avoid conflicts with yaml base-60 float, DO NOT remove the quotation marks. ports: - "25:25" # SMTP (explicit TLS => STARTTLS, Authentication is DISABLED => use port 465/587 instead) - "143:143" # IMAP4 (explicit TLS => STARTTLS) From df360516ff221e6c5c6380d62000100178038be8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?F=C3=BCrst?= <7149167+furstblumier@users.noreply.github.com> Date: Mon, 22 Apr 2024 01:50:02 +0200 Subject: [PATCH 46/53] docs: Add config guide for relaying to and from a private DMS instance (#3973) Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Co-authored-by: Casper --- CHANGELOG.md | 2 + .../external-relay-only-mailserver.md | 233 ++++++++++++++++++ docs/mkdocs.yml | 1 + 3 files changed, 236 insertions(+) create mode 100644 docs/content/examples/use-cases/external-relay-only-mailserver.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 89efb030..10d30016 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,8 @@ The most noteworthy change of this release is the update of the container's base ### Added +- **Docs:** + - A guide for configuring a public server to relay inbound and outbound mail from DMS on a private server ([#3973](https://github.com/docker-mailserver/docker-mailserver/pull/3973)) - **Environment Variables:** - `LOGROTATE_COUNT` defines the number of files kept by logrotate ([#3907](https://github.com/docker-mailserver/docker-mailserver/pull/3907)) - The fail2ban log file is now also taken into account by `LOGROTATE_COUNT` and `LOGROTATE_INTERVAL` ([#3915](https://github.com/docker-mailserver/docker-mailserver/pull/3915), [#3919](https://github.com/docker-mailserver/docker-mailserver/pull/3919)) diff --git a/docs/content/examples/use-cases/external-relay-only-mailserver.md b/docs/content/examples/use-cases/external-relay-only-mailserver.md new file mode 100644 index 00000000..44e83c91 --- /dev/null +++ b/docs/content/examples/use-cases/external-relay-only-mailserver.md @@ -0,0 +1,233 @@ +--- +title: 'Use Cases | Relay inbound and outbound mail for an internal DMS' +hide: + - toc +--- + +## Introduction + +!!! info "Community contributed guide" + + Adapted into a guide from [this discussion](https://github.com/orgs/docker-mailserver/discussions/3965). + + **Requirements:** + + - A _public server_ with a static IP, like many VPS providers offer. It will only relay mail to DMS, no mail is stored on this system. + - A _private server_ (e.g.: a local system at home) that will run DMS. + - Both servers are connected to the same network via a VPN (_optional convenience for trust via the `mynetworks` setting_). + + --- + + The guide below will assume the VPN is setup on `192.168.2.0/24` with: + + - The _public server_ is using `192.168.2.2` + - The _private server_ is using `192.168.2.3` + +The goal of this guide is to configure a _public server_ that can receive inbound mail and relay that over to DMS on a _private server_, which can likewise submit mail outbound through a _public server_ or service. + +The primary motivation is to keep your mail storage private instead of storing it to disk unencrypted on a VPS host. + +## DNS setup + +Follow our [standard guidance][docs::usage-dns-setup] for DNS setup. + +Set your A, MX and PTR records for the _public server_ as if it were running DMS. + +!!! example "DNS Zone file example" + + For this guide, we assume DNS is configured with: + + - A public reachable IP address of `11.22.33.44` + - Mail for `@example.com` addresses must have an MX record pointing to `mail.example.com`. + - An A record for `mail.example.com` pointing to the IP address of your _public server_. + + ```txt + $ORIGIN example.com + @ IN A 11.22.33.44 + mail IN A 11.22.33.44 + + ; mail server for example.com + @ IN MX 10 mail.example.com. + ``` + + SPF records should also be set up as you normally would for `mail.example.com`. + +## Public Server (Basic Postfix setup) + +You will need to install Postfix on your _public server_. The functionality that is needed for this setup is not yet implemented in DMS, so a vanilla Postfix will probably be easier to work with, especially since this server will only be used as an inbound and outbound relay. + +It's necessary to adjust some settings afterwards. + + +!!! quote "" + + === "Postfix main config" + + ??? example "Create or replace `/etc/postfix/main.cf`" + + ```cf + # See /usr/share/postfix/main.cf.dist for a commented, more complete version + + smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) + biff = no + + # appending .domain is the MUA's job. + append_dot_mydomain = no + + # Uncomment the next line to generate "delayed mail" warnings + #delay_warning_time = 4h + + # See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on + # fresh installs. + compatibility_level = 3.6 + + # TLS parameters + smtpd_tls_cert_file=/etc/postfix/certificates/mail.example.com.crt + smtpd_tls_key_file=/etc/postfix/certificates/mail.example.com.key + smtpd_tls_security_level=may + smtp_tls_CApath=/etc/ssl/certs + smtp_tls_security_level=may + smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache + + alias_database = hash:/etc/aliases + alias_maps = hash:/etc/aliases + maillog_file = /var/log/postfix.log + mailbox_size_limit = 0 + inet_interfaces = all + inet_protocols = ipv4 + readme_directory = no + recipient_delimiter = + + + # Customizations relevant to this guide: + myhostname = mail.example.com + myorigin = example.com + mydestination = localhost + mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.2.0/24 + smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination + transport_maps = hash:/etc/postfix/transport + relay_domains = $mydestination, hash:/etc/postfix/relay + + # Disable local system accounts and delivery: + local_recipient_maps = + local_transport = error:local mail delivery is disabled + ``` + + Let's highlight some of the important parts: + + - Avoid including `mail.example.com` in `mydestination`, in fact you can just set `localhost` or nothing at all here as we want all mail to be relayed to our _private server_ (DMS). + - `mynetworks` should contain your VPN network (_eg: `192.168.2.0/24` subnet_). + - Important are `transport_maps = hash:/etc/postfix/transport` and `relay_domains = $mydestination, hash:/etc/postfix/relay`, with their file contents covered below. + - For good measure, also disable `local_recipient_maps`. + - You should have a valid certificate configured for `mail.example.com`. + + !!! warning "Open relay" + + Please be aware that setting `mynetworks` to a public CIDR will leave you with an open relay. **Only** set it to the CIDR of your VPN beyond the localhost ranges. + + === "Route outbound mail through a separate transport" + + When mail arrives to the _public server_ for an `@example.com` address, we want to send it via the `relay` transport to our _private server_ over port 25 for delivery to DMS. + + [`transport_maps`][postfix-docs::transport_maps] is configured with a [`transport` table][postfix-docs::transport_table] file that matches recipient addresses and assigns a non-default transport. This setting has priority over [`relay_transport`][postfix-docs::relay_transport]. + + !!! example "Create `/etc/postfix/transport`" + + ```txt + example.com relay:[192.168.2.3]:25 + ``` + + **Other considerations:** + + - If you have multiple domains, you can add them here too (on separate lines). + - If you use a smarthost add `* relay:[X.X.X.X]:port` to the bottom (eg: `* relay:[relay1.org]:587`), which will relay everything outbound via this relay host. + + !!! tip + + Instead of a file, you could alternatively configure `main.cf` with `transport_maps = inline:{ example.com=relay:[192.168.2.3]:25 }` + + === "Configure recipient domains to relay mail" + + We want `example.com` to be relayed inbound and everything else relayed outbound. + + [`relay_domains`][postfix-docs::relay_domains] is configured with a file with a list of domains that should be relayed (one per line), the 2nd value is required but can be anything. + + !!! example "Create `/etc/postfix/relay`" + + ```txt + example.com OK + ``` + + !!! tip + + Instead of a file, you could alternatively configure `main.cf` with `relay_domains = example.com`. + +!!! note "Files configured with `hash:` table type must run `postmap` to apply changes" + + Run `postmap /etc/postfix/transport` and `postmap /etc/postfix/relay` after creating or updating either of these files, this processes them into a separate file for Postfix to use. + +## Private Server (Running DMS) + +You can set up your DMS instance as you normally would. + +- Be careful not to give it a hostname of `mail.example.com`. Instead, use `internal-mail.example.com` or something similar. +- DKIM can be setup as usual since it considers checks whether the message body has been tampered with, which our public relay doesn't do. Set DKIM up for `mail.example.com`. + +Next, we need to configure our _private server_ to relay all outbound mail through the _public server_ (or a separate smarthost service). The setup is [similar to the default relay setup][docs::relay-host-details]. + + +!!! quote "" + + === "Configure the relay host" + + !!! example "Create `postfix-relaymap.cf`" + + ```txt + @example.com [192.168.2.2]:25 + ``` + + Meaning all mail sent outbound from `@example.com` addresses will be relayed through the _public server_ at that VPN IP. + + The _public server_ `mynetworks` setting from earlier trusts any mail received on port 25 from the VPN network, which is what allows the mail to be sent outbound when it'd otherwise be denied. + + === "Trust the _public server_" + + !!! example "Create `postfix-main.cf`" + + ```txt + mynetworks = 192.168.2.0/24 + ``` + + This will trust any connection from the VPN network to DMS, such as from the _public server_ when relaying mail over to DMS at the _private server_. + + This step is necessary to skip some security measures that DMS normally checks for, like verifying DNS records like SPF are valid. As the mail is being relayed, those checks would fail otherwise as the IP of your _public server_ would not be authorized to send mail on behalf of the sender address in mail being relayed. + + ??? tip "Alternative to `mynetworks` setting" + + Instead of trusting connections by their IP with the `mynetworks` setting, those same security measures can be skipped for any authenticated deliveries to DMS over port 587 instead. + + This is a bit more work. `mynetworks` on the _public server_ `main.cf` Postfix config is for trusting DMS when it sends mail from the _private server_, thus you'll need to have that public Postfix service configured with a login account that DMS can use. + + On the _private server_, DMS needs to know the credentials for that login account, that is handled with `postfix-sasl-password.cf`: + + ```txt + @example.com user:secret + ``` + + You could also relay mail through SendGrid, AWS SES or similar instead of the _public server_ you're running to receive mail from. Login credentials for those relay services are provided via the same `postfix-sasl-password.cf` file. + + --- + + Likewise for the _public server_ to send mail to DMS, it would need to be configured to relay mail with credentials too, removing the need for `mynetworks` on the DMS `postfix-main.cf` config. + + The extra effort to require authentication instead of blind trust of your private subnet can be beneficial at reducing the impact of a compromised system or service on that network that wasn't expected to be permitted to send mail. + +## IMAP / POP3 + +IMAP and POP3 need to point towards your _private server_, since that is where the mailboxes are located, which means you need to have a way for your MUA to connect to it. + +[docs::usage-dns-setup]: ../../usage.md#minimal-dns-setup +[docs::relay-host-details]: ../../config/advanced/mail-forwarding/relay-hosts.md#technical-details +[postfix-docs::relay_domains]: https://www.postfix.org/postconf.5.html#relay_domains +[postfix-docs::relay_transport]: https://www.postfix.org/postconf.5.html#relay_transport +[postfix-docs::transport_maps]: https://www.postfix.org/postconf.5.html#transport_maps +[postfix-docs::transport_table]: https://www.postfix.org/transport.5.html diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 9d7fc81d..59263443 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -180,6 +180,7 @@ nav: - 'iOS Mail Push Support': examples/use-cases/ios-mail-push-support.md - 'Lua Authentication': examples/use-cases/auth-lua.md - 'Bind outbound SMTP to a specific network': examples/use-cases/bind-smtp-network-interface.md + - 'Relay inbound and outbound mail for an internal DMS': examples/use-cases/external-relay-only-mailserver.md - 'FAQ' : faq.md - 'Contributing': - 'General Information': contributing/general.md From 1051a5d921a07bae4cfe2f43e981d058baa05fb4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 22 Apr 2024 15:46:34 +0200 Subject: [PATCH 47/53] chore(deps): Bump akhilmhdh/contributors-readme-action (#3987) Bumps [akhilmhdh/contributors-readme-action](https://github.com/akhilmhdh/contributors-readme-action) from 2.3.6 to 2.3.8. - [Release notes](https://github.com/akhilmhdh/contributors-readme-action/releases) - [Commits](https://github.com/akhilmhdh/contributors-readme-action/compare/v2.3.6...v2.3.8) --- updated-dependencies: - dependency-name: akhilmhdh/contributors-readme-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/contributors.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/contributors.yml b/.github/workflows/contributors.yml index 6f8476f2..9a71b648 100644 --- a/.github/workflows/contributors.yml +++ b/.github/workflows/contributors.yml @@ -19,7 +19,7 @@ jobs: uses: actions/checkout@v4 - name: 'Update CONTRIBUTORS.md' - uses: akhilmhdh/contributors-readme-action@v2.3.6 + uses: akhilmhdh/contributors-readme-action@v2.3.8 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: From 162e66276a4d4c836d6eb4434fdfb3cfb35e2fad Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 25 Apr 2024 08:38:38 +0200 Subject: [PATCH 48/53] docs: updated `CONTRIBUTORS.md` (#3984) --- CONTRIBUTORS.md | 287 ++++++++++++++++++++++++++---------------------- 1 file changed, 154 insertions(+), 133 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 01c7897f..e1027231 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -787,10 +787,17 @@ Thanks goes to these wonderful people ✨ - - jamesfryer + + fl42
- jamesfryer + fl42 + + + + + ipernet +
+ ipernet @@ -808,34 +815,13 @@ Thanks goes to these wonderful people ✨ - - ipernet + + jamesfryer
- ipernet - - - - - fl42 -
- fl42 + jamesfryer - - - neuralp -
- neuralp - - - - - sjmudd -
- sjmudd - - simonsystem @@ -863,8 +849,7 @@ Thanks goes to these wonderful people ✨
5ven - - + syl20bnr @@ -878,7 +863,8 @@ Thanks goes to these wonderful people ✨
sylvaindumont - + + TechnicLab @@ -886,28 +872,13 @@ Thanks goes to these wonderful people ✨ TechnicLab - - - 42wim -
- 42wim - - - - - vilisas -
- vilisas - - thomasschmit
thomasschmit - - + Thiritin @@ -915,6 +886,13 @@ Thanks goes to these wonderful people ✨ Thiritin + + + tobiabocchi +
+ tobiabocchi + + tweibert @@ -928,7 +906,8 @@ Thanks goes to these wonderful people ✨
torus - + + VictorKoenders @@ -949,8 +928,7 @@ Thanks goes to these wonderful people ✨
k3it - - + Drakulix @@ -958,6 +936,28 @@ Thanks goes to these wonderful people ✨ Drakulix + + + vilisas +
+ vilisas + + + + + 42wim +
+ 42wim + + + + + + ShiriNmi1520 +
+ ShiriNmi1520 + + radicand @@ -1015,13 +1015,6 @@ Thanks goes to these wonderful people ✨ p3dda - - - auchri -
- auchri - - peter-hartmann @@ -1029,6 +1022,13 @@ Thanks goes to these wonderful people ✨ peter-hartmann + + + neuralp +
+ neuralp + + piwai @@ -1109,20 +1109,27 @@ Thanks goes to these wonderful people ✨ - - ShiriNmi1520 + + sjmudd
- ShiriNmi1520 + sjmudd + + + Zepmann +
+ Zepmann + + + mchamplain
mchamplain - - + millerjason @@ -1157,15 +1164,15 @@ Thanks goes to these wonderful people ✨
olaf-mandel - + + ontheair81
ontheair81 - - + pravynandas @@ -1200,15 +1207,15 @@ Thanks goes to these wonderful people ✨
rriski - + + schnippl0r
schnippl0r - - + smargold476 @@ -1243,15 +1250,15 @@ Thanks goes to these wonderful people ✨
tamueller - + + vivacarvajalito
vivacarvajalito - - + wligtenberg @@ -1273,13 +1280,6 @@ Thanks goes to these wonderful people ✨ worldworm - - - Zepmann -
- Zepmann - - allddd @@ -1338,6 +1338,13 @@ Thanks goes to these wonderful people ✨ + + + fanqiaojun +
+ fanqiaojun + + ghnp5 @@ -1372,15 +1379,15 @@ Thanks goes to these wonderful people ✨
idaadi - + + ixeft
ixeft - - + jjtt @@ -1415,15 +1422,15 @@ Thanks goes to these wonderful people ✨
callmemagnus - + + marios88
marios88 - - + matrixes @@ -1431,6 +1438,13 @@ Thanks goes to these wonderful people ✨ matrixes + + + auchri +
+ auchri + + arkanovicz @@ -1451,7 +1465,8 @@ Thanks goes to these wonderful people ✨
damianmoore - + + espitall @@ -1465,8 +1480,7 @@ Thanks goes to these wonderful people ✨
dkarski - - + dbellavista @@ -1494,7 +1508,8 @@ Thanks goes to these wonderful people ✨
mlatorre31 - + + mazzz1y @@ -1508,8 +1523,7 @@ Thanks goes to these wonderful people ✨
doominator42 - - + aydodo @@ -1537,7 +1551,8 @@ Thanks goes to these wonderful people ✨
eliroca - + + ekkis @@ -1551,8 +1566,7 @@ Thanks goes to these wonderful people ✨
ErikEngerd - - + huncode @@ -1575,12 +1589,13 @@ Thanks goes to these wonderful people ✨ - - Kaan88 + + froks
- Kaan88 + froks - + + 0xflotus @@ -1594,8 +1609,7 @@ Thanks goes to these wonderful people ✨
ifokeev - - + 20th @@ -1623,7 +1637,8 @@ Thanks goes to these wonderful people ✨
aspettl - + + acch @@ -1637,8 +1652,7 @@ Thanks goes to these wonderful people ✨
vifino - - + kachkaev @@ -1666,7 +1680,8 @@ Thanks goes to these wonderful people ✨
eglia - + + groupmsl @@ -1680,8 +1695,7 @@ Thanks goes to these wonderful people ✨
green-anger - - + iRhonin @@ -1709,7 +1723,8 @@ Thanks goes to these wonderful people ✨
astrocket - + + baxerus @@ -1723,8 +1738,7 @@ Thanks goes to these wonderful people ✨
spock - - + erdos4d @@ -1739,13 +1753,21 @@ Thanks goes to these wonderful people ✨ crash7 + + + Kaan88 +
+ Kaan88 + + akkumar
akkumar - + + thechubbypanda @@ -1766,8 +1788,7 @@ Thanks goes to these wonderful people ✨
khuedoan - - + UltraCoderRU @@ -1788,7 +1809,8 @@ Thanks goes to these wonderful people ✨
leowinterde - + + linhandev @@ -1809,8 +1831,7 @@ Thanks goes to these wonderful people ✨
LucidityCrash - - + MadsRC @@ -1831,7 +1852,8 @@ Thanks goes to these wonderful people ✨
maxemann96 - + + dragetd @@ -1852,8 +1874,7 @@ Thanks goes to these wonderful people ✨
exhuma - - + milas @@ -1874,7 +1895,8 @@ Thanks goes to these wonderful people ✨
MohammedNoureldin - + + mpldr @@ -1889,14 +1911,6 @@ Thanks goes to these wonderful people ✨ naveensrinivasan - - - froks -
- froks - - - fkefer @@ -1924,7 +1938,8 @@ Thanks goes to these wonderful people ✨
GiovanH - + + harryyoud @@ -1938,8 +1953,7 @@ Thanks goes to these wonderful people ✨
HeySora - - + sirgantrithon @@ -1961,6 +1975,14 @@ Thanks goes to these wonderful people ✨ in-seo + + + firefly-cpp +
+ firefly-cpp + + + JacksonZ03 @@ -1981,8 +2003,7 @@ Thanks goes to these wonderful people ✨
jcalfee - - + mivek @@ -2003,7 +2024,8 @@ Thanks goes to these wonderful people ✨
Jeidnx - + + jessp01 @@ -2024,8 +2046,7 @@ Thanks goes to these wonderful people ✨
jirislav - - + jmccl From be8615f129b4616ea1d4f318da878e7fc01673e2 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sun, 28 Apr 2024 19:06:50 +0200 Subject: [PATCH 49/53] docs: updated `CONTRIBUTORS.md` (#3992) --- CONTRIBUTORS.md | 268 +++++++++++++++++++++++++----------------------- 1 file changed, 138 insertions(+), 130 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index e1027231..a96b2659 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -665,17 +665,17 @@ Thanks goes to these wonderful people ✨ - - ubenmackin + + andrewlow
- ubenmackin + andrewlow - - craue + + aminvakil
- craue + aminvakil @@ -685,27 +685,27 @@ Thanks goes to these wonderful people ✨ abh - - - andrewlow -
- andrewlow - - - - - - aminvakil -
- aminvakil - - elbracht
elbracht + + + + + craue +
+ craue + + + + + ubenmackin +
+ ubenmackin + @@ -786,27 +786,6 @@ Thanks goes to these wonderful people ✨ jedateach - - - fl42 -
- fl42 - - - - - ipernet -
- ipernet - - - - - H4R0 -
- H4R0 - - millaguie @@ -820,8 +799,36 @@ Thanks goes to these wonderful people ✨
jamesfryer + + + + H4R0 +
+ H4R0 + + + + + ipernet +
+ ipernet + + + + + fl42 +
+ fl42 + + + + 0xflotus +
+ 0xflotus + + simonsystem @@ -856,15 +863,15 @@ Thanks goes to these wonderful people ✨
syl20bnr - + + sylvaindumont
sylvaindumont - - + TechnicLab @@ -899,15 +906,15 @@ Thanks goes to these wonderful people ✨
tweibert - + + torus
torus - - + VictorKoenders @@ -942,20 +949,27 @@ Thanks goes to these wonderful people ✨
vilisas - + + 42wim
42wim - - + - - ShiriNmi1520 + + matrixes
- ShiriNmi1520 + matrixes + + + + + neuralp +
+ neuralp @@ -978,7 +992,8 @@ Thanks goes to these wonderful people ✨
nknapp - + + pcqnt @@ -992,8 +1007,7 @@ Thanks goes to these wonderful people ✨
OrvilleQ - - + ovidiucp @@ -1021,22 +1035,15 @@ Thanks goes to these wonderful people ✨
peter-hartmann - - - - neuralp -
- neuralp - - + + piwai
piwai - - + remoe @@ -1071,15 +1078,15 @@ Thanks goes to these wonderful people ✨
MightySCollins - + + 501st-alpha1
501st-alpha1 - - + klamann @@ -1114,13 +1121,6 @@ Thanks goes to these wonderful people ✨
sjmudd - - - - Zepmann -
- Zepmann - @@ -1280,6 +1280,21 @@ Thanks goes to these wonderful people ✨ worldworm + + + ShiriNmi1520 +
+ ShiriNmi1520 + + + + + Zepmann +
+ Zepmann + + + allddd @@ -1293,8 +1308,7 @@ Thanks goes to these wonderful people ✨
arcaine2 - - + awb99 @@ -1322,7 +1336,8 @@ Thanks goes to these wonderful people ✨
dborowy - + + dimalo @@ -1336,8 +1351,7 @@ Thanks goes to these wonderful people ✨
eleith - - + fanqiaojun @@ -1365,7 +1379,8 @@ Thanks goes to these wonderful people ✨
hnws - + + i-C-o-d-e-r @@ -1379,8 +1394,7 @@ Thanks goes to these wonderful people ✨
idaadi - - + ixeft @@ -1408,7 +1422,8 @@ Thanks goes to these wonderful people ✨
jpduyx - + + landergate @@ -1422,8 +1437,7 @@ Thanks goes to these wonderful people ✨
callmemagnus - - + marios88 @@ -1431,20 +1445,6 @@ Thanks goes to these wonderful people ✨ marios88 - - - matrixes -
- matrixes - - - - - auchri -
- auchri - - arkanovicz @@ -1597,10 +1597,10 @@ Thanks goes to these wonderful people ✨ - - 0xflotus + + fkefer
- 0xflotus + fkefer @@ -1753,21 +1753,28 @@ Thanks goes to these wonderful people ✨ crash7 + + + auchri +
+ auchri + + Kaan88
Kaan88 - + + akkumar
akkumar - - + thechubbypanda @@ -1802,15 +1809,15 @@ Thanks goes to these wonderful people ✨
JustAnother1 - + + leowinterde
leowinterde - - + linhandev @@ -1845,15 +1852,15 @@ Thanks goes to these wonderful people ✨
madmath03 - + + maxemann96
maxemann96 - - + dragetd @@ -1888,15 +1895,15 @@ Thanks goes to these wonderful people ✨
mcchots - + + MohammedNoureldin
MohammedNoureldin - - + mpldr @@ -1912,10 +1919,10 @@ Thanks goes to these wonderful people ✨ - - fkefer + + furstblumier
- fkefer + furstblumier @@ -1931,15 +1938,15 @@ Thanks goes to these wonderful people ✨
glandais - + + GiovanH
GiovanH - - + harryyoud @@ -1974,15 +1981,15 @@ Thanks goes to these wonderful people ✨
in-seo - + + firefly-cpp
firefly-cpp - - + JacksonZ03 @@ -2017,15 +2024,15 @@ Thanks goes to these wonderful people ✨
init-js - + + Jeidnx
Jeidnx - - + jessp01 @@ -2060,7 +2067,8 @@ Thanks goes to these wonderful people ✨
jurekbarth - + + JOduMonT From 83da191f3a4cbe1d8ee458860de6e93ea152b08f Mon Sep 17 00:00:00 2001 From: Wael <8544289+forzagreen@users.noreply.github.com> Date: Thu, 2 May 2024 02:08:29 +0200 Subject: [PATCH 50/53] docs: Fix link for `getmail6` (#3996) --- docs/content/config/advanced/mail-getmail.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/config/advanced/mail-getmail.md b/docs/content/config/advanced/mail-getmail.md index d08f5d07..f62388b0 100644 --- a/docs/content/config/advanced/mail-getmail.md +++ b/docs/content/config/advanced/mail-getmail.md @@ -95,7 +95,7 @@ environment: It is possible to utilize the `getmail-gmail-xoauth-tokens` helper to provide authentication using `xoauth2` for [gmail (example 12)][getmail-docs-xoauth-12] or [Microsoft Office 365 (example 13)][getmail-docs-xoauth-13] -[getmail-website]: https://www.getmail.org +[getmail-website]: https://www.getmail6.org [getmail-docs]: https://getmail6.org/configuration.html [getmail-docs-xoauth-12]: https://github.com/getmail6/getmail6/blob/1f95606156231f1e074ba62a9baa64f892a92ef8/docs/getmailrc-examples#L286 [getmail-docs-xoauth-13]: https://github.com/getmail6/getmail6/blob/1f95606156231f1e074ba62a9baa64f892a92ef8/docs/getmailrc-examples#L351 From 7dcbbd7173a8fd1713a77d55ed224d14c8970aa5 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Thu, 2 May 2024 19:41:25 +1200 Subject: [PATCH 51/53] fix(`accounts.sh`): Sync user home location for alias workaround (#3997) --- CHANGELOG.md | 1 + target/scripts/helpers/accounts.sh | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 10d30016..20ce0ed4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -105,6 +105,7 @@ The most noteworthy change of this release is the update of the container's base - Rspamd configuration: Add a missing comma in `local_networks` so that all internal IP addresses are actually considered as internal ([#3862](https://github.com/docker-mailserver/docker-mailserver/pull/3862)) - Ensure correct SELinux security context labels for files and directories moved to the mail-state volume during setup ([#3890](https://github.com/docker-mailserver/docker-mailserver/pull/3890)) - Use correct environment variable for fetchmail ([#3901](https://github.com/docker-mailserver/docker-mailserver/pull/3901)) +- Dovecot dummy accounts (_virtual alias workaround for dovecot feature `ENABLE_QUOTAS=1`_) now correctly matches the home location of the user for that alias ([#3997](https://github.com/docker-mailserver/docker-mailserver/pull/3997)) ## [v13.3.1](https://github.com/docker-mailserver/docker-mailserver/releases/tag/v13.3.1) diff --git a/target/scripts/helpers/accounts.sh b/target/scripts/helpers/accounts.sh index 31ded04a..78464b88 100644 --- a/target/scripts/helpers/accounts.sh +++ b/target/scripts/helpers/accounts.sh @@ -134,7 +134,7 @@ function _create_dovecot_alias_dummy_accounts() { fi fi - DOVECOT_USERDB_LINE="${ALIAS}:${REAL_ACC[1]}:${DMS_VMAIL_UID}:${DMS_VMAIL_GID}::/var/mail/${REAL_DOMAINNAME}/${REAL_USERNAME}::${REAL_ACC[2]:-}" + DOVECOT_USERDB_LINE="${ALIAS}:${REAL_ACC[1]}:${DMS_VMAIL_UID}:${DMS_VMAIL_GID}::/var/mail/${REAL_DOMAINNAME}/${REAL_USERNAME}/home::${REAL_ACC[2]:-}" if grep -qi "^${ALIAS}:" "${DOVECOT_USERDB_FILE}"; then _log 'warn' "Alias '${ALIAS}' will not be added to '${DOVECOT_USERDB_FILE}' twice" else From d00edd7209d80f8f9b45bf9274fcb6cf858e6867 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Thu, 2 May 2024 19:44:54 +1200 Subject: [PATCH 52/53] docs: Revise fetchmail page (#3998) --- .../content/config/advanced/mail-fetchmail.md | 191 ++++++++++-------- 1 file changed, 109 insertions(+), 82 deletions(-) diff --git a/docs/content/config/advanced/mail-fetchmail.md b/docs/content/config/advanced/mail-fetchmail.md index 254196a1..936d5188 100644 --- a/docs/content/config/advanced/mail-fetchmail.md +++ b/docs/content/config/advanced/mail-fetchmail.md @@ -2,7 +2,7 @@ title: 'Advanced | Email Gathering with Fetchmail' --- -To enable the [fetchmail][fetchmail-website] service to retrieve e-mails set the environment variable `ENABLE_FETCHMAIL` to `1`. Your `compose.yaml` file should look like following snippet: +To enable the [fetchmail][fetchmail-website] service to retrieve e-mails, set the environment variable `ENABLE_FETCHMAIL` to `1`. Your `compose.yaml` file should look like following snippet: ```yaml environment: @@ -18,108 +18,135 @@ Generate a file called `fetchmail.cf` and place it in the `docker-data/dms/confi │   ├── fetchmail.cf │   ├── postfix-accounts.cf │   └── postfix-virtual.cf -├── compose.yaml -└── README.md +└── compose.yaml ``` ## Configuration -A detailed description of the configuration options can be found in the [online version of the manual page][fetchmail-docs]. +Configuration options for `fetchmail.cf` are covered at the [official fetchmail docs][fetchmail-docs-config] (_see the section "The run control file" and the table with "keyword" column for all settings_). -### IMAP Configuration +!!! example "Basic `fetchmail.cf` configuration" -!!! example + Retrieve mail from `remote-user@somewhere.com` and deliver it to `dms-user@example.com`: ```fetchmailrc - poll 'imap.gmail.com' proto imap - user 'username' - pass 'secret' - is 'user1@example.com' - ssl + poll 'mail.somewhere.com' + proto imap + user 'remote-user' + pass 'secret' + is 'dms-user@example.com' ``` -### POP3 Configuration + - `poll` sets the remote mail server to connect to retrieve mail from. + - `proto` lets you connect via IMAP or POP3. + - `user` and `pass` provide the login credentials for the remote mail service account to access. + - `is` configures where the fetched mail will be sent to (_eg: your local DMS account in `docker-data/dms/config/postfix-accounts.cf`_). -!!! example + --- - ```fetchmailrc - poll 'pop3.gmail.com' proto pop3 - user 'username' - pass 'secret' - is 'user2@example.com' - ssl + ??? warning "`proto imap` will still delete remote mail once fetched" + + This is due to a separate default setting `no keep`. Adding the setting `keep` to your config on a new line will prevent deleting the remote copy. + +??? example "Multiple users or remote servers" + + The official docs [config examples][fetchmail-config-examples] show a common convention to indent settings on subsequent lines for visually grouping per server. + + === "Minimal syntax" + + ```fetchmailrc + poll 'mail.somewhere.com' proto imap + user 'john.doe' pass 'secret' is 'johnny@example.com' + user 'jane.doe' pass 'secret' is 'jane@example.com' + + poll 'mail.somewhere-else.com' proto pop3 + user 'john.doe@somewhere-else.com' pass 'secret' is 'johnny@example.com' + ``` + + === "With optional syntax" + + - `#` for adding comments. + - The config file may include "noise" keywords to improve readability. + + ```fetchmailrc + # Retrieve mail for users `john.doe` and `jane.doe` via IMAP at this remote mail server: + poll 'mail.somewhere.com' with proto imap wants: + user 'john.doe' with pass 'secret', is 'johnny@example.com' here + user 'jane.doe' with pass 'secret', is 'jane@example.com' here + + # Also retrieve mail from this mail server (but via POP3). + # NOTE: This could also be all on a single line, or with each key + value as a separate line. + # Notice how the remote username includes a full email address, + # Some mail servers like DMS use the full email address as the username: + poll 'mail.somewhere-else.com' with proto pop3 wants: + user 'john.doe@somewhere-else.com' with pass 'secret', is 'johnny@example.com' here + ``` + +!!! tip "`FETCHMAIL_POLL` ENV: Override default polling interval" + + By default the fetchmail service will check every 5 minutes for new mail at the configured mail accounts. + + ```yaml + environment: + # The fetchmail polling interval in seconds: + FETCHMAIL_POLL: 60 ``` -!!! caution - - Don’t forget the last line! (_eg: `is 'user1@example.com'`_). After `is`, you have to specify an email address from the configuration file: `docker-data/dms/config/postfix-accounts.cf`. - -More details how to configure fetchmail can be found in the [fetchmail man page in the chapter “The run control file”][fetchmail-docs-run]. - -### Polling Interval - -By default the fetchmail service searches every 5 minutes for new mails on your external mail accounts. You can override this default value by changing the ENV variable `FETCHMAIL_POLL`: - -```yaml -environment: - - FETCHMAIL_POLL=60 -``` - -You must specify a numeric argument which is a polling interval in seconds. The example above polls every minute for new mails. - ## Debugging -To debug your `fetchmail.cf` configuration run this command: +To debug your `fetchmail.cf` configuration run this `setup debug` command: ```sh -./setup.sh debug fetchmail +docker exec -it dms-container-name setup debug fetchmail ``` -For more information about the configuration script `setup.sh` [read the corresponding docs][docs-setup]. +??? example "Sample output of `setup debug fetchmail`" -Here a sample output of `./setup.sh debug fetchmail`: + ```log + fetchmail: 6.3.26 querying outlook.office365.com (protocol POP3) at Mon Aug 29 22:11:09 2016: poll started + Trying to connect to 132.245.48.18/995...connected. + fetchmail: Server certificate: + fetchmail: Issuer Organization: Microsoft Corporation + fetchmail: Issuer CommonName: Microsoft IT SSL SHA2 + fetchmail: Subject CommonName: outlook.com + fetchmail: Subject Alternative Name: outlook.com + fetchmail: Subject Alternative Name: *.outlook.com + fetchmail: Subject Alternative Name: office365.com + fetchmail: Subject Alternative Name: *.office365.com + fetchmail: Subject Alternative Name: *.live.com + fetchmail: Subject Alternative Name: *.internal.outlook.com + fetchmail: Subject Alternative Name: *.outlook.office365.com + fetchmail: Subject Alternative Name: outlook.office.com + fetchmail: Subject Alternative Name: attachment.outlook.office.net + fetchmail: Subject Alternative Name: attachment.outlook.officeppe.net + fetchmail: Subject Alternative Name: *.office.com + fetchmail: outlook.office365.com key fingerprint: 3A:A4:58:42:56:CD:BD:11:19:5B:CF:1E:85:16:8E:4D + fetchmail: POP3< +OK The Microsoft Exchange POP3 service is ready. [SABFADEAUABSADAAMQBDAEEAMAAwADAANwAuAGUAdQByAHAAcgBkADAAMQAuAHAAcgBvAGQALgBlAHgAYwBoAGEAbgBnAGUAbABhAGIAcwAuAGMAbwBtAA==] + fetchmail: POP3> CAPA + fetchmail: POP3< +OK + fetchmail: POP3< TOP + fetchmail: POP3< UIDL + fetchmail: POP3< SASL PLAIN + fetchmail: POP3< USER + fetchmail: POP3< . + fetchmail: POP3> USER user1@outlook.com + fetchmail: POP3< +OK + fetchmail: POP3> PASS * + fetchmail: POP3< +OK User successfully logged on. + fetchmail: POP3> STAT + fetchmail: POP3< +OK 0 0 + fetchmail: No mail for user1@outlook.com at outlook.office365.com + fetchmail: POP3> QUIT + fetchmail: POP3< +OK Microsoft Exchange Server 2016 POP3 server signing off. + fetchmail: 6.3.26 querying outlook.office365.com (protocol POP3) at Mon Aug 29 22:11:11 2016: poll completed + fetchmail: normal termination, status 1 + ``` -```log -fetchmail: 6.3.26 querying outlook.office365.com (protocol POP3) at Mon Aug 29 22:11:09 2016: poll started -Trying to connect to 132.245.48.18/995...connected. -fetchmail: Server certificate: -fetchmail: Issuer Organization: Microsoft Corporation -fetchmail: Issuer CommonName: Microsoft IT SSL SHA2 -fetchmail: Subject CommonName: outlook.com -fetchmail: Subject Alternative Name: outlook.com -fetchmail: Subject Alternative Name: *.outlook.com -fetchmail: Subject Alternative Name: office365.com -fetchmail: Subject Alternative Name: *.office365.com -fetchmail: Subject Alternative Name: *.live.com -fetchmail: Subject Alternative Name: *.internal.outlook.com -fetchmail: Subject Alternative Name: *.outlook.office365.com -fetchmail: Subject Alternative Name: outlook.office.com -fetchmail: Subject Alternative Name: attachment.outlook.office.net -fetchmail: Subject Alternative Name: attachment.outlook.officeppe.net -fetchmail: Subject Alternative Name: *.office.com -fetchmail: outlook.office365.com key fingerprint: 3A:A4:58:42:56:CD:BD:11:19:5B:CF:1E:85:16:8E:4D -fetchmail: POP3< +OK The Microsoft Exchange POP3 service is ready. [SABFADEAUABSADAAMQBDAEEAMAAwADAANwAuAGUAdQByAHAAcgBkADAAMQAuAHAAcgBvAGQALgBlAHgAYwBoAGEAbgBnAGUAbABhAGIAcwAuAGMAbwBtAA==] -fetchmail: POP3> CAPA -fetchmail: POP3< +OK -fetchmail: POP3< TOP -fetchmail: POP3< UIDL -fetchmail: POP3< SASL PLAIN -fetchmail: POP3< USER -fetchmail: POP3< . -fetchmail: POP3> USER user1@outlook.com -fetchmail: POP3< +OK -fetchmail: POP3> PASS * -fetchmail: POP3< +OK User successfully logged on. -fetchmail: POP3> STAT -fetchmail: POP3< +OK 0 0 -fetchmail: No mail for user1@outlook.com at outlook.office365.com -fetchmail: POP3> QUIT -fetchmail: POP3< +OK Microsoft Exchange Server 2016 POP3 server signing off. -fetchmail: 6.3.26 querying outlook.office365.com (protocol POP3) at Mon Aug 29 22:11:11 2016: poll completed -fetchmail: normal termination, status 1 -``` +!!! tip "Troubleshoot with this reference `compose.yaml`" + + [A minimal `compose.yaml` example][fetchmail-compose-example] demonstrates how to run two instances of DMS locally, with one instance configured with `fetchmail.cf` and the other to simulate a remote mail server to fetch from. -[docs-setup]: ../../config/setup.sh.md [fetchmail-website]: https://www.fetchmail.info -[fetchmail-docs]: https://www.fetchmail.info/fetchmail-man.html -[fetchmail-docs-run]: https://www.fetchmail.info/fetchmail-man.html#31 +[fetchmail-docs-config]: https://www.fetchmail.info/fetchmail-man.html#the-run-control-file +[fetchmail-config-examples]: https://www.fetchmail.info/fetchmail-man.html#configuration-examples +[fetchmail-compose-example]: https://github.com/orgs/docker-mailserver/discussions/3994#discussioncomment-9290570 From 7822a9743034c486cf2ceeebde64eed6c3ace8c2 Mon Sep 17 00:00:00 2001 From: Brennan Kinney <5098581+polarathene@users.noreply.github.com> Date: Thu, 2 May 2024 19:48:05 +1200 Subject: [PATCH 53/53] docs(FAQ): Add advice for restricting login by IP (#3999) --- docs/content/faq.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/content/faq.md b/docs/content/faq.md index 4add0589..a350f937 100644 --- a/docs/content/faq.md +++ b/docs/content/faq.md @@ -291,6 +291,12 @@ mydestination = localhost.$mydomain, localhost proxy_interfaces = X.X.X.X (your public IP) ``` +For reverse proxy support you will want to view [our dedicated guide][docs::examples::reverse-proxy]. + +### How to restrict login by IP? + +There are a few ways you could approach this, see [this discussion answer][gh-discussion::restrict-login-by-ip] for advice. + ### How to adjust settings with the `user-patches.sh` script Suppose you want to change a number of settings that are not listed as variables or add things to the server that are not included? @@ -496,6 +502,7 @@ $spam_quarantine_to = "quarantine\@example.com"; [docs-userpatches]: ./config/advanced/override-defaults/user-patches.md [docs::env::sa_env]: ./config/environment.md#spamassassin [docs::env::sa_kill]: ./config/environment.md#sa_kill +[docs::examples::reverse-proxy]: ./examples/tutorials/mailserver-behind-proxy.md [github-comment-baredomain]: https://github.com/docker-mailserver/docker-mailserver/issues/3048#issuecomment-1432358353 [github-comment-override-hostname]: https://github.com/docker-mailserver/docker-mailserver/issues/1731#issuecomment-753968425 [github-issue-95]: https://github.com/docker-mailserver/docker-mailserver/issues/95 @@ -505,5 +512,6 @@ $spam_quarantine_to = "quarantine\@example.com"; [github-issue-1639]: https://github.com/docker-mailserver/docker-mailserver/issues/1639 [github-issue-1792]: https://github.com/docker-mailserver/docker-mailserver/pull/1792 [gh-discussion::dms-avoid-maintaining-internal-dns]: https://github.com/orgs/docker-mailserver/discussions/3959#discussioncomment-8956322 +[gh-discussion::restrict-login-by-ip]: https://github.com/orgs/docker-mailserver/discussions/3847 [docker-compose::docs::config-dns]: https://docs.docker.com/compose/compose-file/compose-file-v3/#dns [hanscees-userpatches]: https://github.com/hanscees/dockerscripts/blob/master/scripts/tomav-user-patches.sh