diff --git a/docs/content/assets/css/customizations.css b/docs/content/assets/css/customizations.css index 25cb0274..7255a837 100644 --- a/docs/content/assets/css/customizations.css +++ b/docs/content/assets/css/customizations.css @@ -107,3 +107,37 @@ div.md-content article.md-content__inner a.toclink code { .md-nav__item--nested > .md-nav__link { font-weight: 700; } + +/* ============================================================================================================= */ + +/* + TaskList style for a pro/con list. Presently only used for this type of list in the kubernetes docs. + Uses a custom icon for the unchecked (con) state: :octicons-x-circle-fill-24: + https://github.com/squidfunk/mkdocs-material/discussions/6811#discussioncomment-8700795 + + TODO: Can better scope the style under a class name when migrating to block extension syntax: + https://github.com/facelessuser/pymdown-extensions/discussions/1973 +*/ + +:root { + --md-tasklist-icon--failed: url('data:image/svg+xml;charset=utf-8,'); +} + +.md-typeset [type="checkbox"] + .task-list-indicator::before { + background-color: rgb(216, 87, 48); + -webkit-mask-image: var(--md-tasklist-icon--failed); + mask-image: var(--md-tasklist-icon--failed); +} + +/* More suitable shade of green */ +.md-typeset [type=checkbox]:checked+.task-list-indicator:before { + background-color: rgb(97, 216, 42); +} + +/* Tiny layout shift */ +[dir=ltr] .md-typeset .task-list-indicator:before { + left: -1.6em; + top: 1px; +} + +/* ============================================================================================================= */ diff --git a/docs/content/config/advanced/kubernetes.md b/docs/content/config/advanced/kubernetes.md index c277852a..97a7e414 100644 --- a/docs/content/config/advanced/kubernetes.md +++ b/docs/content/config/advanced/kubernetes.md @@ -4,518 +4,794 @@ title: 'Advanced | Kubernetes' ## Introduction -This article describes how to deploy DMS to Kubernetes. Please note that there is also a [Helm chart] available. +This article describes how to deploy DMS to Kubernetes. We highly recommend everyone to use our community [DMS Helm chart][github-web::docker-mailserver-helm]. -!!! attention "Requirements" +!!! note "Requirements" - We assume basic knowledge about Kubernetes from the reader. Moreover, we assume the reader to have a basic understanding of mail servers. Ideally, the reader has deployed DMS before in an easier setup with Docker (Compose). + 1. Basic knowledge about Kubernetes from the reader. + 2. A basic understanding of mail servers. + 3. Ideally, the reader has already deployed DMS before with a simpler setup (_`docker run` or Docker Compose_). -!!! warning "About Support for Kubernetes" +!!! warning "Limited Support" - Please note that Kubernetes **is not** officially supported and we do not build images specifically designed for it. When opening an issue, please remember that only Docker & Docker Compose are officially supported. + DMS **does not officially support Kubernetes**. This content is entirely community-supported. If you find errors, please open an issue and raise a PR. - This content is entirely community-supported. If you find errors, please open an issue and provide a PR. +## Manually Writing Manifests -## Manifests +If using our Helm chart is not viable for you, here is some guidance to start with your own manifests. -### Configuration + +!!! quote "" -We want to provide the basic configuration in the form of environment variables with a `ConfigMap`. Note that this is just an example configuration; tune the `ConfigMap` to your needs. + === "`ConfigMap`" -```yaml ---- -apiVersion: v1 -kind: ConfigMap + Provide the basic configuration via environment variables with a `ConfigMap`. + + !!! example -metadata: - name: mailserver.environment + Below is only an example configuration, adjust the `ConfigMap` to your own needs. -immutable: false + ```yaml + --- + apiVersion: v1 + kind: ConfigMap -data: - TLS_LEVEL: modern - POSTSCREEN_ACTION: drop - OVERRIDE_HOSTNAME: mail.example.com - FAIL2BAN_BLOCKTYPE: drop - POSTMASTER_ADDRESS: postmaster@example.com - UPDATE_CHECK_INTERVAL: 10d - POSTFIX_INET_PROTOCOLS: ipv4 - ENABLE_CLAMAV: '1' - ENABLE_POSTGREY: '0' - ENABLE_FAIL2BAN: '1' - AMAVIS_LOGLEVEL: '-1' - SPOOF_PROTECTION: '1' - MOVE_SPAM_TO_JUNK: '1' - ENABLE_UPDATE_CHECK: '1' - ENABLE_SPAMASSASSIN: '1' - SUPERVISOR_LOGLEVEL: warn - SPAMASSASSIN_SPAM_TO_INBOX: '1' + metadata: + name: mailserver.environment - # here, we provide an example for the SSL configuration - SSL_TYPE: manual - SSL_CERT_PATH: /secrets/ssl/rsa/tls.crt - SSL_KEY_PATH: /secrets/ssl/rsa/tls.key -``` + immutable: false -We can also make use of user-provided configuration files, e.g. `user-patches.sh`, `postfix-accounts.cf` and more, to adjust DMS to our likings. We encourage you to have a look at [Kustomize][kustomize] for creating `ConfigMap`s from multiple files, but for now, we will provide a simple, hand-written example. This example is absolutely minimal and only goes to show what can be done. + data: + TLS_LEVEL: modern + POSTSCREEN_ACTION: drop + OVERRIDE_HOSTNAME: mail.example.com + FAIL2BAN_BLOCKTYPE: drop + POSTMASTER_ADDRESS: postmaster@example.com + UPDATE_CHECK_INTERVAL: 10d + POSTFIX_INET_PROTOCOLS: ipv4 + ENABLE_CLAMAV: '1' + ENABLE_POSTGREY: '0' + ENABLE_FAIL2BAN: '1' + AMAVIS_LOGLEVEL: '-1' + SPOOF_PROTECTION: '1' + MOVE_SPAM_TO_JUNK: '1' + ENABLE_UPDATE_CHECK: '1' + ENABLE_SPAMASSASSIN: '1' + SUPERVISOR_LOGLEVEL: warn + SPAMASSASSIN_SPAM_TO_INBOX: '1' -```yaml ---- -apiVersion: v1 -kind: ConfigMap - -metadata: - name: mailserver.files + # here, we provide an example for the SSL configuration + SSL_TYPE: manual + SSL_CERT_PATH: /secrets/ssl/rsa/tls.crt + SSL_KEY_PATH: /secrets/ssl/rsa/tls.key + ``` -data: - postfix-accounts.cf: | - test@example.com|{SHA512-CRYPT}$6$someHashValueHere - other@example.com|{SHA512-CRYPT}$6$someOtherHashValueHere -``` - -!!! attention "Static Configuration" - - With the configuration shown above, you can **not** dynamically add accounts as the configuration file mounted into the mail server can not be written to. - - Use persistent volumes for production deployments. - -### Persistence - -Thereafter, we need persistence for our data. Make sure you have a storage provisioner and that you choose the correct `storageClassName`. - -```yaml ---- -apiVersion: v1 -kind: PersistentVolumeClaim - -metadata: - name: data - -spec: - storageClassName: local-path - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 25Gi -``` - -### Service - -A `Service` is required for getting the traffic to the pod itself. The service is somewhat crucial. Its configuration determines whether the original IP from the sender will be kept. [More about this further down below](#exposing-your-mail-server-to-the-outside-world). - -The configuration you're seeing does keep the original IP, but you will not be able to scale this way. We have chosen to go this route in this case because we think most Kubernetes users will only want to have one instance. - -```yaml ---- -apiVersion: v1 -kind: Service - -metadata: - name: mailserver - labels: - app: mailserver - -spec: - type: LoadBalancer - - selector: - app: mailserver - - ports: - # Transfer - - name: transfer - port: 25 - targetPort: transfer - protocol: TCP - # ESMTP with implicit TLS - - name: esmtp-implicit - port: 465 - targetPort: esmtp-implicit - protocol: TCP - # ESMTP with explicit TLS (STARTTLS) - - name: esmtp-explicit - port: 587 - targetPort: esmtp-explicit - protocol: TCP - # IMAPS with implicit TLS - - name: imap-implicit - port: 993 - targetPort: imap-implicit - protocol: TCP - -``` - -### Deployments - -Last but not least, the `Deployment` becomes the most complex component. It instructs Kubernetes how to run the DMS container and how to apply your `ConfigMaps`, persisted storage, etc. Additionally, we can set options to enforce runtime security here. - -```yaml ---- -apiVersion: apps/v1 -kind: Deployment - -metadata: - name: mailserver - - annotations: - ignore-check.kube-linter.io/run-as-non-root: >- - 'mailserver' needs to run as root - ignore-check.kube-linter.io/privileged-ports: >- - 'mailserver' needs privileged ports - ignore-check.kube-linter.io/no-read-only-root-fs: >- - There are too many files written to make The - root FS read-only - -spec: - replicas: 1 - selector: - matchLabels: - app: mailserver - - template: - metadata: - labels: - app: mailserver - - annotations: - container.apparmor.security.beta.kubernetes.io/mailserver: runtime/default - - spec: - hostname: mail - containers: - - name: mailserver - image: ghcr.io/docker-mailserver/docker-mailserver:latest - imagePullPolicy: IfNotPresent - - securityContext: - # Required to support SGID via `postdrop` executable - # in `/var/mail-state` for Postfix (maildrop + public dirs): - # https://github.com/docker-mailserver/docker-mailserver/pull/3625 - allowPrivilegeEscalation: true - readOnlyRootFilesystem: false - runAsUser: 0 - runAsGroup: 0 - runAsNonRoot: false - privileged: false - capabilities: - add: - # file permission capabilities - - CHOWN - - FOWNER - - MKNOD - - SETGID - - SETUID - - DAC_OVERRIDE - # network capabilities - - NET_ADMIN # needed for F2B - - NET_RAW # needed for F2B - - NET_BIND_SERVICE - # miscellaneous capabilities - - SYS_CHROOT - - KILL - drop: [ALL] - seccompProfile: - type: RuntimeDefault - - # You want to tune this to your needs. If you disable ClamAV, - # you can use less RAM and CPU. This becomes important in - # case you're low on resources and Kubernetes refuses to - # schedule new pods. - resources: - limits: - memory: 4Gi - cpu: 1500m - requests: - memory: 2Gi - cpu: 600m - - volumeMounts: - - name: files - subPath: postfix-accounts.cf - mountPath: /tmp/docker-mailserver/postfix-accounts.cf - readOnly: true - - # PVCs - - name: data - mountPath: /var/mail - subPath: data - readOnly: false - - name: data - mountPath: /var/mail-state - subPath: state - readOnly: false - - name: data - mountPath: /var/log/mail - subPath: log - readOnly: false - - # certificates - - name: certificates-rsa - mountPath: /secrets/ssl/rsa/ - readOnly: true - - # other - - name: tmp-files - mountPath: /tmp - readOnly: false - - ports: - - name: transfer - containerPort: 25 - protocol: TCP - - name: esmtp-implicit - containerPort: 465 - protocol: TCP - - name: esmtp-explicit - containerPort: 587 - - name: imap-implicit - containerPort: 993 - protocol: TCP - - envFrom: - - configMapRef: - name: mailserver.environment - - restartPolicy: Always - - volumes: - # configuration files - - name: files - configMap: - name: mailserver.files - - # PVCs - - name: data - persistentVolumeClaim: - claimName: data - - # certificates - - name: certificates-rsa - secret: - secretName: mail-tls-certificate-rsa - items: - - key: tls.key - path: tls.key - - key: tls.crt - path: tls.crt - - # other - - name: tmp-files - emptyDir: {} -``` - -### Certificates - An Example - -In this example, we use [`cert-manager`][cert-manager] to supply RSA certificates. You can also supply RSA certificates as fallback certificates, which DMS supports out of the box with `SSL_ALT_CERT_PATH` and `SSL_ALT_KEY_PATH`, and provide ECDSA as the proper certificates. - -```yaml ---- -apiVersion: cert-manager.io/v1 -kind: Certificate - -metadata: - name: mail-tls-certificate-rsa - -spec: - secretName: mail-tls-certificate-rsa - isCA: false - privateKey: - algorithm: RSA - encoding: PKCS1 - size: 2048 - dnsNames: [mail.example.com] - issuerRef: - name: mail-issuer - kind: Issuer -``` - -!!! attention - - You will need to have [`cert-manager`][cert-manager] configured. Especially the issue will need to be configured. Since we do not know how you want or need your certificates to be supplied, we do not provide more configuration here. The documentation for [`cert-manager`][cert-manager] is excellent. - -### Sensitive Data - -!!! attention "Sensitive Data" - - For storing OpenDKIM keys, TLS certificates or any sort of sensitive data, you should be using `Secret`s. You can mount secrets like `ConfigMap`s and use them the same way. - -The [TLS docs page][docs-tls] provides guidance when it comes to certificates and transport layer security. Always provide sensitive information vai `Secrets`. + You can also make use of user-provided configuration files (_e.g. `user-patches.sh`, `postfix-accounts.cf`, etc_), to customize DMS to your needs. + + ??? example "Providing config files" + + Here is a minimal example that supplies a `postfix-accounts.cf` file inline with two users: + + ```yaml + --- + apiVersion: v1 + kind: ConfigMap + + metadata: + name: mailserver.files + + data: + postfix-accounts.cf: | + test@example.com|{SHA512-CRYPT}$6$someHashValueHere + other@example.com|{SHA512-CRYPT}$6$someOtherHashValueHere + ``` + + !!! warning "Static Configuration" + + The inline `postfix-accounts.cf` config example above provides file content that is static. It is mounted as read-only at runtime, thus cannot support modifications. + + For production deployments, use persistent volumes instead (via `PersistentVolumeClaim`). That will enable files like `postfix-account.cf` to add and remove accounts, while also persisting those changes externally from the container. + + !!! tip "Modularize your `ConfigMap`" + + [Kustomize][kustomize] can be a useful tool as it supports creating a `ConfigMap` from multiple files. + + === "`PersistentVolumeClaim`" + + To persist data externally from the DMS container, configure a `PersistentVolumeClaim` (PVC). + + Make sure you have a storage system (like Longhorn, Rook, etc.) and that you choose the correct `storageClassName` (according to your storage system). + + !!! example + + ```yaml + --- + apiVersion: v1 + kind: PersistentVolumeClaim + + metadata: + name: data + + spec: + storageClassName: local-path + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 25Gi + ``` + + === "`Service`" + + A [`Service`][k8s-docs::config::service] is required for getting the traffic to the pod itself. It configures a load balancer with the ports you'll need. + + The configuration for a `Service` affects if the original IP from a connecting client is preserved (_this is important_). [More about this further down below](#exposing-your-mail-server-to-the-outside-world). + + !!! example + + ```yaml + --- + apiVersion: v1 + kind: Service + + metadata: + name: mailserver + labels: + app: mailserver + + spec: + type: LoadBalancer + + selector: + app: mailserver + + ports: + # smtp + - name: smtp + port: 25 + targetPort: smtp + protocol: TCP + # submissions (ESMTP with implicit TLS) + - name: submission + port: 465 + targetPort: submissions + protocol: TCP + # submission (ESMTP with explicit TLS) + - name: submission + port: 587 + targetPort: submission + protocol: TCP + # imaps (implicit TLS) + - name: imaps + port: 993 + targetPort: imaps + protocol: TCP + ``` + + === "`Certificate`" + + !!! example "Using [`cert-manager`][cert-manager] to supply TLS certificates" + + ```yaml + --- + apiVersion: cert-manager.io/v1 + kind: Certificate + + metadata: + name: mail-tls-certificate-rsa + + spec: + secretName: mail-tls-certificate-rsa + isCA: false + privateKey: + algorithm: RSA + encoding: PKCS1 + size: 2048 + dnsNames: [mail.example.com] + issuerRef: + name: mail-issuer + kind: Issuer + ``` + + The [TLS docs page][docs-tls] provides guidance when it comes to certificates and transport layer security. + + !!! tip "ECDSA + RSA (fallback)" + + You could supply RSA certificates as fallback certificates instead, with ECDSA as the primary. DMS supports dual certificates via the ENV `SSL_ALT_CERT_PATH` and `SSL_ALT_KEY_PATH`. + + !!! warning "Always provide sensitive information via a `Secret`" + + For storing OpenDKIM keys, TLS certificates, or any sort of sensitive data - you should be using `Secret`s. + + A `Secret` is similar to `ConfigMap`, it can be used and mounted as a volume as demonstrated in the [`Deployment` manifest][docs::k8s::config-deployment] tab. + + === "`Deployment`" + + The [`Deployment`][k8s-docs::config::deployment] config is the most complex component. + + - It instructs Kubernetes how to run the DMS container and how to apply your `ConfigMap`s, persisted storage, etc. + - Additional options can be set to enforce runtime security. + + ???+ example + + ```yaml + --- + apiVersion: apps/v1 + kind: Deployment + + metadata: + name: mailserver + + annotations: + ignore-check.kube-linter.io/run-as-non-root: >- + 'mailserver' needs to run as root + ignore-check.kube-linter.io/privileged-ports: >- + 'mailserver' needs privileged ports + ignore-check.kube-linter.io/no-read-only-root-fs: >- + There are too many files written to make the root FS read-only + + spec: + replicas: 1 + selector: + matchLabels: + app: mailserver + + template: + metadata: + labels: + app: mailserver + + annotations: + container.apparmor.security.beta.kubernetes.io/mailserver: runtime/default + + spec: + hostname: mail + containers: + - name: mailserver + image: ghcr.io/docker-mailserver/docker-mailserver:latest + imagePullPolicy: IfNotPresent + + securityContext: + # `allowPrivilegeEscalation: true` is required to support SGID via the `postdrop` + # executable in `/var/mail-state` for Postfix (maildrop + public dirs): + # https://github.com/docker-mailserver/docker-mailserver/pull/3625 + allowPrivilegeEscalation: true + readOnlyRootFilesystem: false + runAsUser: 0 + runAsGroup: 0 + runAsNonRoot: false + privileged: false + capabilities: + add: + # file permission capabilities + - CHOWN + - FOWNER + - MKNOD + - SETGID + - SETUID + - DAC_OVERRIDE + # network capabilities + - NET_ADMIN # needed for F2B + - NET_RAW # needed for F2B + - NET_BIND_SERVICE + # miscellaneous capabilities + - SYS_CHROOT + - KILL + drop: [ALL] + seccompProfile: + type: RuntimeDefault + + # Tune this to your needs. + # If you disable ClamAV, you can use less RAM and CPU. + # This becomes important in case you're low on resources + # and Kubernetes refuses to schedule new pods. + resources: + limits: + memory: 4Gi + cpu: 1500m + requests: + memory: 2Gi + cpu: 600m + + volumeMounts: + - name: files + subPath: postfix-accounts.cf + mountPath: /tmp/docker-mailserver/postfix-accounts.cf + readOnly: true + + # PVCs + - name: data + mountPath: /var/mail + subPath: data + readOnly: false + - name: data + mountPath: /var/mail-state + subPath: state + readOnly: false + - name: data + mountPath: /var/log/mail + subPath: log + readOnly: false + + # certificates + - name: certificates-rsa + mountPath: /secrets/ssl/rsa/ + readOnly: true + + ports: + - name: smtp + containerPort: 25 + protocol: TCP + - name: submissions + containerPort: 465 + protocol: TCP + - name: submission + containerPort: 587 + - name: imaps + containerPort: 993 + protocol: TCP + + envFrom: + - configMapRef: + name: mailserver.environment + + restartPolicy: Always + + volumes: + # configuration files + - name: files + configMap: + name: mailserver.files + + # PVCs + - name: data + persistentVolumeClaim: + claimName: data + + # certificates + - name: certificates-rsa + secret: + secretName: mail-tls-certificate-rsa + items: + - key: tls.key + path: tls.key + - key: tls.crt + path: tls.crt + ``` ## Exposing your Mail Server to the Outside World -The more difficult part with Kubernetes is to expose a deployed DMS to the outside world. Kubernetes provides multiple ways for doing that; each has downsides and complexity. The major problem with exposing DMS to outside world in Kubernetes is to [preserve the real client IP][Kubernetes-service-source-ip]. The real client IP is required by DMS for performing IP-based SPF checks and spam checks. If you do not require SPF checks for incoming mails, you may disable them in your [Postfix configuration][docs-postfix] by dropping the line that states: `check_policy_service unix:private/policyd-spf`. +The more difficult part with Kubernetes is to expose a deployed DMS instance to the outside world. -The easiest approach was covered above, using `#!yaml externalTrafficPolicy: Local`, which disables the service proxy, but makes the service local as well (which does not scale). This approach only works when you are given the correct (that is, a public and routable) IP address by a load balancer (like MetalLB). In this sense, the approach above is similar to the next example below. We want to provide you with a few alternatives too. **But** we also want to communicate the idea of another simple method: you could use a load-balancer without an external IP and DNAT the network traffic to the mail server. After all, this does not interfere with SPF checks because it keeps the origin IP address. If no dedicated external IP address is available, you could try the latter approach, if one is available, use the former. +The major problem with exposing DMS to the outside world in Kubernetes is to [preserve the real client IP][k8s-docs::service-source-ip]. The real client IP is required by DMS for performing IP-based DNS and spam checks. -### External IPs Service +Kubernetes provides multiple ways to address this; each has its upsides and downsides. -The simplest way is to expose DMS as a [Service][Kubernetes-network-service] with [external IPs][Kubernetes-network-external-ip]. This is very similar to the approach taken above. Here, an external IP is given to the service directly by you. With the approach above, you tell your load-balancer to do this. + +!!! quote "" -```yaml ---- -apiVersion: v1 -kind: Service + === "Configure IP Manually" -metadata: - name: mailserver - labels: - app: mailserver + ???+ abstract "Advantages / Disadvantages" -spec: - selector: - app: mailserver - ports: - - name: smtp - port: 25 - targetPort: smtp - # ... + - [x] Simple + - [ ] Requires the node to have a dedicated, publicly routable IP address + - [ ] Limited to a single node (_associated to the dedicated IP address_) + - [ ] Your deployment requires an explicit IP in your configuration (_or an entire Load Balancer_). - externalIPs: - - 80.11.12.10 -``` + !!! info "Requirements" -This approach + 1. You can dedicate a **publicly routable IP** address for the DMS configured `Service`. + 2. A dedicated IP is required to allow your mail server to have matching `A` and `PTR` records (_which other mail servers will use to verify trust when they receive mail sent from your DMS instance_). -- does not preserve the real client IP, so SPF check of incoming mail will fail. -- requires you to specify the exposed IPs explicitly. + !!! example -### Proxy port to Service + Assign the DMS `Service` an external IP directly, or delegate an LB to assign the IP on your behalf. -The [proxy pod][Kubernetes-proxy-service] helps to avoid the necessity of specifying external IPs explicitly. This comes at the cost of complexity; you must deploy a proxy pod on each [Node][Kubernetes-nodes] you want to expose DMS on. + === "External-IP Service" -This approach + The DMS `Service` is configured with an "[external IP][k8s-docs::network-external-ip]" manually. Append your externally reachable IP address to `spec.externalIPs`. -- does not preserve the real client IP, so SPF check of incoming mail will fail. + ```yaml + --- + apiVersion: v1 + kind: Service -### Bind to concrete Node and use host network + metadata: + name: mailserver + labels: + app: mailserver -One way to preserve the real client IP is to use `hostPort` and `hostNetwork: true`. This comes at the cost of availability; you can reach DMS from the outside world only via IPs of [Node][Kubernetes-nodes] where DMS is deployed. + spec: + selector: + app: mailserver + ports: + - name: smtp + port: 25 + targetPort: smtp + # ... -```yaml ---- -apiVersion: extensions/v1beta1 -kind: Deployment + externalIPs: + - 10.20.30.40 + ``` -metadata: - name: mailserver + === "Load-Balancer" -# ... - spec: - hostNetwork: true + The config differs depending on your choice of load balancer. This example uses [MetalLB][metallb-web]. - # ... - containers: - # ... - ports: - - name: smtp - containerPort: 25 - hostPort: 25 - - name: smtp-auth - containerPort: 587 - hostPort: 587 - - name: imap-secure - containerPort: 993 - hostPort: 993 - # ... -``` + ```yaml + --- + apiVersion: v1 + kind: Service -With this approach, + metadata: + name: mailserver + labels: + app: mailserver + annotations: + metallb.universe.tf/address-pool: mailserver -- it is not possible to access DMS via other cluster Nodes, only via the Node DMS was deployed at. -- every Port within the Container is exposed on the Host side. + # ... -### Proxy Port to Service via PROXY Protocol + --- + apiVersion: metallb.io/v1beta1 + kind: IPAddressPool -This way is ideologically the same as [using a proxy pod](#proxy-port-to-service), but instead of a separate proxy pod, you configure your ingress to proxy TCP traffic to the DMS pod using the PROXY protocol, which preserves the real client IP. + metadata: + name: mail + namespace: metallb-system -#### Configure your Ingress + spec: + addresses: [ ] + autoAssign: true -With an [NGINX ingress controller][Kubernetes-nginx], set `externalTrafficPolicy: Local` for its service, and add the following to the TCP services config map (as described [here][Kubernetes-nginx-expose]): + --- + apiVersion: metallb.io/v1beta1 + kind: L2Advertisement -```yaml -25: "mailserver/mailserver:25::PROXY" -465: "mailserver/mailserver:465::PROXY" -587: "mailserver/mailserver:587::PROXY" -993: "mailserver/mailserver:993::PROXY" -``` + metadata: + name: mail + namespace: metallb-system -!!! help "HAProxy" - With [HAProxy][dockerhub-haproxy], the configuration should look similar to the above. If you know what it actually looks like, add an example here. :smiley: + spec: + ipAddressPools: [ mailserver ] + ``` -#### Configure the Mailserver + === "Host network" -Then, configure both [Postfix][docs-postfix] and [Dovecot][docs-dovecot] to expect the PROXY protocol: + ???+ abstract "Advantages / Disadvantages" -??? example "HAProxy Example" + - [x] Simple + - [ ] Requires the node to have a dedicated, publicly routable IP address + - [ ] Limited to a single node (_associated to the dedicated IP address_) + - [ ] It is not possible to access DMS via other cluster nodes, only via the node that DMS was deployed on + - [ ] Every port within the container is exposed on the host side - ```yaml - kind: ConfigMap - apiVersion: v1 - metadata: - name: mailserver.config - labels: - app: mailserver - data: - postfix-main.cf: | - postscreen_upstream_proxy_protocol = haproxy - postfix-master.cf: | - smtp/inet/postscreen_upstream_proxy_protocol=haproxy - submission/inet/smtpd_upstream_proxy_protocol=haproxy - submissions/inet/smtpd_upstream_proxy_protocol=haproxy - dovecot.cf: | - # Assuming your ingress controller is bound to 10.0.0.0/8 - haproxy_trusted_networks = 10.0.0.0/8, 127.0.0.0/8 - service imap-login { - inet_listener imap { - haproxy = yes - } - inet_listener imaps { - haproxy = yes - } - } - # ... - --- + !!! example - kind: Deployment - apiVersion: extensions/v1beta1 - metadata: - name: mailserver - spec: - template: - spec: - containers: - - name: docker-mailserver - volumeMounts: - - name: config - subPath: postfix-main.cf - mountPath: /tmp/docker-mailserver/postfix-main.cf - readOnly: true - - name: config - subPath: postfix-master.cf - mountPath: /tmp/docker-mailserver/postfix-master.cf - readOnly: true - - name: config - subPath: dovecot.cf - mountPath: /tmp/docker-mailserver/dovecot.cf - readOnly: true - ``` + Using `hostPort` and `hostNetwork: true` is a similar approach to [`network_mode: host` with Docker Compose][docker-docs::compose::network_mode]. -With this approach, + ```yaml + --- + apiVersion: apps/v1 + kind: Deployment -- it is not possible to access DMS via cluster-DNS, as the PROXY protocol is required for incoming connections. + metadata: + name: mailserver -[Helm chart]: https://github.com/docker-mailserver/docker-mailserver-helm -[kustomize]: https://kustomize.io/ -[cert-manager]: https://cert-manager.io/docs/ + # ... + spec: + hostNetwork: true + # ... + containers: + # ... + ports: + - name: smtp + containerPort: 25 + hostPort: 25 + - name: submissions + containerPort: 465 + hostPort: 465 + - name: submission + containerPort: 587 + hostPort: 587 + - name: imaps + containerPort: 993 + hostPort: 993 + ``` + + === "Using the PROXY Protocol" + + ???+ abstract "Advantages / Disadvantages" + + - [x] Preserves the origin IP address of clients (_which is crucial for DNS related checks_) + - [x] Aligns with a best practice for Kubernetes by using a dedicated ingress, routing external traffic to the k8s cluster (_with the benefits of flexible routing rules_) + - [x] Avoids the restraint of a single [node][k8s-docs::nodes] (_as a workaround to preserve the original client IP_) + - [ ] Introduces complexity by requiring: + - A reverse-proxy / ingress controller (_potentially extra setup_) + - Kubernetes manifest changes for the DMS configured `Service` + - DMS configuration changes for Postfix and Dovecot + - [ ] To keep support for direct connections to DMS services internally within cluster, service ports must be "duplicated" to offer an alternative port for connections using PROXY protocol + + ??? question "What is the PROXY protocol?" + + PROXY protocol is a network protocol for preserving a client’s IP address when the client’s TCP connection passes through a proxy. + + It is a common feature supported among reverse-proxy services (_NGINX, HAProxy, Traefik_), which you may already have handling ingress traffic for your cluster. + + ```mermaid + flowchart LR + A(External Mail Server) -->|Incoming connection| B + subgraph cluster + B("Ingress Acting as a Proxy") -->|PROXY protocol connection| C(DMS) + end + ``` + + For more information on the PROXY protocol, refer to [our dedicated docs page][docs-mailserver-behind-proxy] on the topic. + + ???+ example "Configure the Ingress Controller" + + === "Traefik" + + On Traefik's side, the configuration is very simple. + + - Create an entrypoint for each port that you want to expose (_probably 25, 465, 587 and 993_). + - Each entrypoint should configure an [`IngressRouteTCP`][traefik-docs::k8s::ingress-route-tcp] that routes to the equivalent internal DMS `Service` port which supports PROXY protocol connections. + + The below snippet demonstrates an example for two entrypoints, `submissions` (port 465) and `imaps` (port 993). + + ```yaml + --- + apiVersion: v1 + kind: Service + + metadata: + name: mailserver + + spec: + # This an optimization to get rid of additional routing steps. + # Previously "type: LoadBalancer" + type: ClusterIP + + --- + apiVersion: traefik.io/v1alpha1 + kind: IngressRouteTCP + + metadata: + name: smtp + + spec: + entryPoints: [ submissions ] + routes: + - match: HostSNI(`*`) + services: + - name: mailserver + namespace: mail + port: subs-proxy # note the 15 character limit here + proxyProtocol: + version: 2 + + --- + apiVersion: traefik.io/v1alpha1 + kind: IngressRouteTCP + + metadata: + name: imaps + + spec: + entryPoints: [ imaps ] + routes: + - match: HostSNI(`*`) + services: + - name: mailserver + namespace: mail + port: imaps-proxy + proxyProtocol: + version: 2 + ``` + + !!! info "`*-proxy` port name suffix" + + The `IngressRouteTCP` example configs above reference ports with a `*-proxy` suffix. + + - These port variants will be defined in the [`Deployment` manifest][docs::k8s::config-deployment], and are scoped to the `mailserver` service (via `spec.routes.services.name`). + - The suffix is used to distinguish that these ports are only compatible with connections using the PROXY protocol, which is what your ingress controller should be managing for you by adding the correct PROXY protocol headers to TCP connections it routes to DMS. + + === "NGINX" + + With an [NGINX ingress controller][k8s-docs::nginx], add the following to the TCP services config map (_as described [here][k8s-docs::nginx-expose]_): + + ```yaml + 25: "mailserver/mailserver:25::PROXY" + 465: "mailserver/mailserver:465::PROXY" + 587: "mailserver/mailserver:587::PROXY" + 993: "mailserver/mailserver:993::PROXY" + ``` + + ???+ example "Adjust DMS config for Dovecot + Postfix" + + ??? warning "Only ingress should connect to DMS with PROXY protocol" + + While Dovecot will restrict connections via PROXY protocol to only clients trusted configured via `haproxy_trusted_networks`, Postfix does not have an equivalent setting. Public clients should always route through ingress to establish a PROXY protocol connection. + + You are responsible for properly managing traffic inside your cluster and to **ensure that only trustworthy entities** can connect to the designated PROXY protocol ports. + + With Kubernetes, this is usually the task of the CNI (_container network interface_). + + !!! tip "Advised approach" + + The _"Separate PROXY protocol ports"_ tab below introduces a little more complexity, but provides better compatibility for internal connections to DMS. + + === "Only accept connections with PROXY protocol" + + !!! warning "Connections to DMS within the internal cluster will be rejected" + + The services for these ports can only enable PROXY protocol support by mandating the protocol on all connections for these ports. + + This can be problematic when you also need to support internal cluster traffic directly to DMS (_instead of routing indirectly through the ingress controller_). + + Here is an example configuration for [Postfix][docs-postfix], [Dovecot][docs-dovecot], and the required adjustments for the [`Deployment` manifest][docs::k8s::config-deployment]. The port names are adjusted here only to convey the additional context described earlier. + + ```yaml + kind: ConfigMap + apiVersion: v1 + metadata: + name: mailserver-extra-config + labels: + app: mailserver + data: + postfix-main.cf: | + postscreen_upstream_proxy_protocol = haproxy + postfix-master.cf: | + smtp/inet/postscreen_upstream_proxy_protocol=haproxy + submission/inet/smtpd_upstream_proxy_protocol=haproxy + submissions/inet/smtpd_upstream_proxy_protocol=haproxy + dovecot.cf: | + haproxy_trusted_networks = + service imap-login { + inet_listener imap { + haproxy = yes + } + inet_listener imaps { + haproxy = yes + } + } + # ... + + --- + kind: Deployment + apiVersion: apps/v1 + metadata: + name: mailserver + spec: + template: + spec: + containers: + - name: docker-mailserver + # ... + ports: + - name: smtp-proxy + containerPort: 25 + protocol: TCP + - name: imap-proxy + containerPort: 143 + protocol: TCP + - name: subs-proxy + containerPort: 465 + protocol: TCP + - name: sub-proxy + containerPort: 587 + protocol: TCP + - name: imaps-proxy + containerPort: 993 + protocol: TCP + # ... + volumeMounts: + - name: config + subPath: postfix-main.cf + mountPath: /tmp/docker-mailserver/postfix-main.cf + readOnly: true + - name: config + subPath: postfix-master.cf + mountPath: /tmp/docker-mailserver/postfix-master.cf + readOnly: true + - name: config + subPath: dovecot.cf + mountPath: /tmp/docker-mailserver/dovecot.cf + readOnly: true + ``` + + === "Separate PROXY protocol ports for ingress" + + !!! info + + Supporting internal cluster connections to DMS without using PROXY protocol requires both Postfix and Dovecot to be configured with alternative ports for each service port (_which only differ by enforcing PROXY protocol connections_). + + - The ingress controller will route public connections to the internal alternative ports for DMS (`*-proxy` variants). + - Internal cluster connections will instead use the original ports configured for the DMS container directly (_which are private to the cluster network_). + + In this example we'll create a copy of the original service ports with PROXY protocol enabled, and increment the port number assigned by `10000`. + + Create a `user-patches.sh` file to apply these config changes during container startup: + + ```bash + #!/bin/bash + + # Duplicate the config for the submission(s) service ports (587 / 465) with adjustments for the PROXY ports (10587 / 10465) and `syslog_name` setting: + postconf -Mf submission/inet | sed -e s/^submission/10587/ -e 's/submission/submission-proxyprotocol/' >> /etc/postfix/master.cf + postconf -Mf submissions/inet | sed -e s/^submissions/10465/ -e 's/submissions/submissions-proxyprotocol/' >> /etc/postfix/master.cf + # Enable PROXY Protocol support for these new service variants: + postconf -P 10587/inet/smtpd_upstream_proxy_protocol=haproxy + postconf -P 10465/inet/smtpd_upstream_proxy_protocol=haproxy + + # Create a variant for port 25 too (NOTE: Port 10025 is already assigned in DMS to Amavis): + postconf -Mf smtp/inet | sed -e s/^smtp/12525/ >> /etc/postfix/master.cf + # Enable PROXY Protocol support (different setting as port 25 is handled via postscreen), optionally configure a `syslog_name` to distinguish in logs: + postconf -P 12525/inet/postscreen_upstream_proxy_protocol=haproxy 12525/inet/syslog_name=smtp-proxyprotocol + ``` + + For Dovecot, you can configure [`dovecot.cf`][docs-dovecot] to look like this: + + ```cf + haproxy_trusted_networks = + + service imap-login { + inet_listener imap-proxied { + haproxy = yes + port = 10143 + } + + inet_listener imaps-proxied { + haproxy = yes + port = 10993 + ssl = yes + } + } + ``` + + Update the [`Deployment` manifest][docs::k8s::config-deployment] `ports` section by appending these new ports: + + ```yaml + - name: smtp-proxy + # not 10025 in this example due to a possible clash with Amavis + containerPort: 12525 + protocol: TCP + - name: imap-proxy + containerPort: 10143 + protocol: TCP + - name: subs-proxy + containerPort: 10465 + protocol: TCP + - name: sub-proxy + containerPort: 10587 + protocol: TCP + - name: imaps-proxy + containerPort: 10993 + protocol: TCP + ``` + + !!! note + + If you use other Dovecot ports (110, 995, 4190), you may want to configure those similar to above. The `dovecot.cf` config for these ports is [documented here][docs-mailserver-behind-proxy] (_in the equivalent section of that page_). + +[docs::k8s::config-deployment]: #deployment [docs-tls]: ../security/ssl.md [docs-dovecot]: ./override-defaults/dovecot.md [docs-postfix]: ./override-defaults/postfix.md -[dockerhub-haproxy]: https://hub.docker.com/_/haproxy -[Kubernetes-nginx]: https://kubernetes.github.io/ingress-nginx -[Kubernetes-nginx-expose]: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services -[Kubernetes-network-service]: https://kubernetes.io/docs/concepts/services-networking/service -[Kubernetes-network-external-ip]: https://kubernetes.io/docs/concepts/services-networking/service/#external-ips -[Kubernetes-nodes]: https://kubernetes.io/docs/concepts/architecture/nodes -[Kubernetes-proxy-service]: https://github.com/kubernetes/contrib/tree/master/for-demos/proxy-to-service -[Kubernetes-service-source-ip]: https://kubernetes.io/docs/tutorials/services/source-ip +[docs-mailserver-behind-proxy]: ../../examples/tutorials/mailserver-behind-proxy.md + +[github-web::docker-mailserver-helm]: https://github.com/docker-mailserver/docker-mailserver-helm +[docker-docs::compose::network_mode]: https://docs.docker.com/compose/compose-file/compose-file-v3/#network_mode +[kustomize]: https://kustomize.io/ +[cert-manager]: https://cert-manager.io/docs/ +[metallb-web]: https://metallb.universe.tf/ + +[k8s-docs::config::service]: https://kubernetes.io/docs/concepts/services-networking/service +[k8s-docs::config::deployment]: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#creating-a-deployment +[k8s-docs::nodes]: https://kubernetes.io/docs/concepts/architecture/nodes +[k8s-docs::nginx]: https://kubernetes.github.io/ingress-nginx +[k8s-docs::nginx-expose]: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services +[k8s-docs::service-source-ip]: https://kubernetes.io/docs/tutorials/services/source-ip +[k8s-docs::network-external-ip]: https://kubernetes.io/docs/concepts/services-networking/service/#external-ips + +[traefik-docs::k8s::ingress-route-tcp]: https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-ingressroutetcp diff --git a/docs/content/examples/tutorials/mailserver-behind-proxy.md b/docs/content/examples/tutorials/mailserver-behind-proxy.md index 47116fde..60e4c539 100644 --- a/docs/content/examples/tutorials/mailserver-behind-proxy.md +++ b/docs/content/examples/tutorials/mailserver-behind-proxy.md @@ -14,6 +14,8 @@ This reduces many of the benefits for why you might use a reverse proxy, but the Some deployments may require a service to route traffic (kubernetes) when deploying, in which case the below advice is important to understand well. +The guide here has also been adapted for [our Kubernetes docs][docs::kubernetes]. + ## What can go wrong? Without a reverse proxy involved, a service is typically aware of the client IP for a connection. @@ -354,9 +356,8 @@ Software on the receiving end of the connection often supports configuring an IP [`postscreen_access_list`][postfix-docs::settings::postscreen_access_list] (_or [`smtpd_client_restrictions`][postfix-docs::settings::smtpd_client_restrictions] with [`check_client_access`][postfix-docs::settings::check_client_access] for ports 587/465_) can both restrict access by IP via a [CIDR lookup table][postfix-docs::config-table::cidr], however the client IP is already rewritten at this point via PROXY protocol. Thus those settings cannot be used for restricting access to only trusted proxies, only to the actual clients. - - A similar setting [`mynetworks`][postfix-docs::settings::mynetworks] / [`PERMIT_DOCKER`][docs::env::permit_docker] manages elevated trust for bypassing security restrictions. While it is intended for trusted clients, it has no relevance to trusting proxies for the same reasons. + A similar setting [`mynetworks`][postfix-docs::settings::mynetworks] / [`PERMIT_DOCKER`][docs::env::permit_docker] manages elevated trust for bypassing security restrictions. While it is intended for trusted clients, it has no relevance to trusting proxies for the same reasons. ### Monitoring @@ -373,6 +374,8 @@ While PROXY protocol works well with the reverse proxy, you may have some contai You should adjust configuration of these monitoring services to monitor for auth failures from those services directly instead, adding an exclusion for that service IP from any DMS logs monitored (_but be mindful of PROXY header forgery risks_). +[docs::kubernetes]: ../../config/advanced/kubernetes.md#using-the-proxy-protocol + [docs::overrides::dovecot]: ../../config/advanced/override-defaults/dovecot.md [docs::overrides::postfix]: ../../config/advanced/override-defaults/postfix.md [docs::overrides::user-patches]: ../../config/advanced/override-defaults/user-patches.md diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 84366423..e0ff50c9 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -82,6 +82,11 @@ markdown_extensions: format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.tabbed: alternate_style: true + slugify: !!python/object/apply:pymdownx.slugs.slugify + kwds: + case: lower + - pymdownx.tasklist: + custom_checkbox: true - pymdownx.magiclink - pymdownx.inlinehilite - pymdownx.tilde