* chore(refactor): DRY up the `_setup_ssl` method
- `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`.
- The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead.
- This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result.
- Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths.
- Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor.
- `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key.
- That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files.
- I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion).
---
* chore: Remove mkcert logic and dovecot cert
- No longer serving a purpose.
- Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`.
- To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image.
- Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition.
---
* Error handling for SSL_TYPE
- Added a panic utility to exit early when SSL_TYPE conditions are misconfigured.
- Some info text had order of key/cert occurrence swapped to be consistent with key then cert.
- Some existing comments moved and rephrased.
- Additional comments added.
- `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks).
- _notify messages lifted out of conditionals so that they always output when the case is hit.
- ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~
---
* Use sedfile + improve sed expressions + update case style
- Uses sedfile when appropriate (file change intentional, not optional match/check).
- sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)).
- sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement.
- case statements adopt the `(` prefix, adopting recent shell style for consistency.
---
* Refactor SSL_TYPE=disabled
- Postfix is also disabled now.
- Included heavy inline documentation reference for maintainers.
- Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'.
---
* I'm a friggin' sed wizard now
- The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`.
- I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines).
- Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better.
---
* tests(fix): Tests that require SSL/TLS to pass
- SSL_TYPE=snakeoil added as temporary workaround.
- nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests.
- Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past.
- The removed SSL test is already handled in mail_ssl_manual.bats
ldap test:
- Replace `--link` alias option with `--network` and alias assignment.
- Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure.
privacy test:
- Also needed `SSL_TYPE` to pass the starttls test.
`tests.bats` had another starttls test for imap:
- Workaround for now is to give the main test container `SSL_TYPE=snakeoil`.
---
* Remove the expired lets-encrypt cert
This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores.
No longer the case today.
---
* chore: Housekeeping
Not required for this PR branch, little bit of tidying up while working on these two test files.
- privacy test copied over content when extracted from `tests.bats` that isn't relevant.
- ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find.
- Additionally converted the two localhost FQDN to vars.
---
* Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case)
- This is to prevent other tests from failing by hitting the panic catchall case.
- More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value.
---
* Add inline documentation for `dms_panic`
- This could later be better formatted and placed into contributor docs.
Panic with kill (shutdown) not exit (errex):
- `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process).
- `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't.
The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
This feature was originally introduced by the PR: https://github.com/docker-mailserver/docker-mailserver/pull/1463
- Assign default DH params to use via Dockerfile build instead of copy and update at runtime.
- Parameterized service names and paths.
- Refactor postfix and dovecot dh methods to wrap shared dh logic
- I don't see any value in checking the alternative service for dh params file to copy over, so that's now dropped too.
- Another conditional check is dropped and the default fallback message for existing DH params file is no longer relevant.
- Improved the remaining `_notify` messages. Collapsing the warning into a single logged message also seemed relevant.
- There is no apparent need for special handling with `ONE_DIR=1`. Dropped it.
- Refactor DH params tests
- Combine custom and default DH param tests into single test file
- docs: Add instructions to use custom DH params
There is no official documented support for custom DH parameters. As no guarantee is provided, this is considered an internal change, not a breaking one.
* changed the locking function to better support multiple servers running at once and sharing the same config
* helper function testing now runs inside of container
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Decoupling setup process from `setup.sh` script by introducing a setup script _inside_ the container that coordinates the setup process.
**This is not a breaking change**. This way, we do not have to keep track of versions of `setup.sh`.
This change brings the additional benefit for Kubernetes users to be able to make use of `setup` now, without the need for `setup.sh`.
---
* move setup process into container; setup.sh versioning not needed anymore
* add tilde functionality to docs
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Centralize the collection of the HOSTNAME and DOMAINAME so that it's predictable and uniform across the various scripts (using the helper). Ensure it supports the various configurations users can have (both subdomain and without subdomain, override and no override).
---
* using _obtain_hostname_and_domainname helper + covers when not a subdomain
doc: OVERRIDE_HOSTNAME takes priority
* added tests for non-subdomain hostname + further improvements
* moved SRS DOMAINANME tests into hostname test file + Allowing DOMAINNAME ENV to override what would be automatically set
---
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* docker_container first, then fall back to docker_image
+ test changes to support
+ test change to wait for smtp port to fix flakey tests since https://github.com/docker-mailserver/docker-mailserver/pull/2104
* quick fix
* Update setup.sh
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
* fix: make sure the SASLAUTHD_LDAP_HOST/PROTO logic makes sense and use LDAP_SERVER_HOST as a fallback (#1983)
* chore(docs): document changes to LDAP/SASLAUTHD as of #1983
* fix!: apply default value modifications suggested in #1983https://github.com/docker-mailserver/docker-mailserver/issues/1983#issuecomment-844848224
* chore(test): Test SASLAUTHD_LDAP_SERVER with protocol and ..._SSL=0, as well as with default bind credentials
Note that there are currently no regression tests for this as there's only one setup_file, so that would require big changes to the testing methodology.
* refactor!: completely remove SASLAUTHD_LDAP_SSL and SASLAUTHD_LDAP_PROTO
Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
* Use dovecot's LDAP uris option instead of hosts (fixes#1510)
* Clean up variables & environment documentation for #1901
Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
* splitting start-mailserver.sh
* refactoring part 2
* refactored setup-stack.sh
* stzarted adjusting target/bin/*.sh to use new usage format
* corrected lowercase-uppercase test error
* better handling of .bashrc variable export
* linting tests and fix for default assignements
* last stylistic changes and rebase
* feat: Change Postfix smtpd_tls key and cert files to chain_files
Since Postfix 3.4, `smtpd_tls_cert_file` and `smtpd_tls_key_file` have been deprecated in favor of `smtpd_tls_chain_files` which supports a list of values where a single or sequence of file paths provide a private key followed by it's certificate chain.
* feat: Dual certificate support
`smtpd_tls_chain_files` allows for multiple key+cert bundles so that you can provide different key types, such as ECDSA and RSA.
To maintain compatibility with the current CERT/KEY ENV vars only a 2nd certificate is supported.
Since Dovecot 2.2.31 a related feature is also available, but it is limited to only providing one alternative certificate via separate cert and key settings.
---
This feature enables support for multiple certificates, eg for serving modern ECDSA certs with RSA as fallback.
* chore: Refactor variable names to meet style guide
Improved some comments too.
* chore: Have function definitions respect style guide
* chore: Minor edits to comments
* chore: Expand on comments for maintenance, alert of insecure config
When `SSL_TYPE` isn't properly setup, we're still offering SSL connections but not warning in logs about the insecurity of such, or why a misconfiguration may have occurred.
This commit more clearly communicates to the user that they should look into the issue before considering deploying to production.
The `TODO` comments communicate to any future maintainer to consider treating these improper configs as disabling TLS instead.
* fix: Use `snakeoil` cert
I mistakenly thought this was placeholder text, which broke some tests. This adds the two files in the correct order (private key followed by cert/chain), to fix that issue.
* fix: Disable alt cert for Dovecot if necessary
Certain scenarios may persist state of previously configured alt cert via ENV vars that are removed from a future run. If the config is not reset to original immutable state, this will correctly disable the config from using alt cert unintentionally.
* fix: Satisfy ShellCheck lint
By switching from string var to array / list expansion, this better stores the extracted result and applies it in a manner that ShellCheck linting approves, removing the need to disable the rule.
* feat: Support dual cert test
Few tweaks to the test script allows re-purposing it for covering dual cert support as well.
* chore: Rearranged cert and key lines
A little reorganization, mostly placing private key ahead of related cert lines.
* chore: Refactor `_set_certificate`
This should make the parameters a little less confusing.
Previously was 3 parameters, but the Postfix parameter (1st) may look like two variables if you don't pay attention to the surrounding quotes; while the Dovecot parameters (2nd + 3rd) would have an opposing order. There was also a variant where the `FULLKEYCHAIN` var was passed in three times.
Now it's two params, with the 2nd param as an optional one. If the 2nd param is provided, then the two params are in the order of private key then certificate, otherwise if only a single parameter it's a single PEM file with the full cert chain and private key bundled.
This avoids implying that Postfix and Dovecot might use different files.
* chore: Document current state of `SSL_TYPE` logic better
Inlined for the benefit of anyone else maintaining this section if I'm unable to address the concerns within my own time.
* docs: ENV vars
`TLS_LEVEL=old` isn't in the codebase anymore, not likely to be relevant to retain.
No point in documenting what is considered invalid / unsupported config value in the first place for `SSL_TYPE`.
`SSL_TYPE=manual` was missing documentation for both related file path ENV vars, they've been added along with their alt fallback variants.
* chore: Update Dovecot LMTP SSL test config
Not sure how relevant this is, the file isn't complete sync with the main dovecot `10-ssl.conf` config, adding the support just in case.
* chore: Rename `FULLKEYCHAIN` to avoid confusion
There doesn't appear to be a standardized name for this type of file bundle, and `keychain` may be misleading (fullkeychain often provides macOS keychain results on search engines).
Opting for a more explicit `KEY_WITH_FULLCHAIN` name instead.
* fix: Invalid var name
`_set_certificate` refactor commit accidentally changed a var name and committed that breaking the dual cert support (thanks tests!).
* test: Refactor `mail_ssl_manual.bats`
Proper test return values instead of `wc -l` based checking.
Tests with dual cert support active, tests that feature (to better detect failure case.
Third test case was unable to verify new self-signed certificate, added new certs signed with self-signed root CA.
Adjusted openssl `CApath` parameter to use `CAfile` instead as `letsencrypt` cert was replaced thus CA cert is missing from the system trust store.
* test: Properly check for files in `mail_ssl_manual.bats`
Fixes lint error.
Also realized I was accidentally asserting a file exists in the test environment, not within the container.
Resolved that and also added an additional test case to ensure the ENV var files are valid when passed in, in the event a change misconfigures them and that the issue is identified earlier.
* chore: Apply PR review feedback
Better format some strings that had mixed quotes when they weren't necessary.
Additionally DRYed up the config path for Postfix and Dovecot within the `_setup_ssl` method.
Co-authored-by: Georg Lauterbach <infrastructure@itbsd.com>
* let dkim generator accept selector as parameter
* test dkim-generator with selector parameter
* fix: correct name of domain argument in usage
* fix: adapt command to new syntax
* tests: use different quotes
* tests: use different quotes
* tests: remove domains that were never added
* style: change test name
* refactor: dkim setup
* style: remove trailing whitespace
* tests: remove test of removed dummy file
Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
* provide complete refactoring of openDKIM usage and tests
* fix leftover linting errors
* correct defualt key size and README usage
* provide independent order for arguments
* added `config` and adjusted usage information
* fixing shift in setup.sh
* adjust usage information to use new style and rename script
* use updated argument keysize instead of size
* Migrate PR#1730 from tomav/docker-mailserver repo to new
docker-mailserver/docker-mailserver repo
* Resolved review comments
* Moved counter increment to have consistency between fetchmail process
and fetchmail config files
* Added tests for new fetchmail option
Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
* first migration steps
* altered issue templates
* altered README
* removed .travis.yml
* adjusting registry & repository, Dockerfile and compose.env
* Close stale issues automatically
* Integrated CI with Github Actions (#3)
* feat: integrated ci with github actions
* fix: use secrets for docker org and update image
* docs: clarify why we use -t if no tty exists
* fix: correct remaining references to old repo
chore: prettier automatically updated markdown as well
* fix: hardcode docker org
* change testing image to just testing
* ci: add armv7 as a supported platform
* finished migration steps
* corrected linting in build-push action
* corrected linting in build-push action (2)
* minor preps for PR
* correcting push on pull request and minor details
* adjusted workflows to adhere closer to @wernerfred's diagram
* minor patches
* adjusting Dockerfile's installation of base packages
* adjusting schedule for stale issue action
* reverting license text
* improving CONTRIBUTING.md PR text
* Update CONTRIBUTING.md
* a bigger patch at the end
* moved all scripts into one directory under target/scripts/
* moved the quota-warning.sh script into target/scripts/ and removed empty directory /target/dovecot/scripts
* minor fixes here and there
* adjusted workflows for use a fully qualified name (i.e. docker.io/...)
* improved on the Dockerfile layer count
* corrected local tests - now they (actually) work (fine)!
* corrected start-mailserver.sh to make use of defaults consistently
* removed very old, deprecated variables (actually only one)
* various smaller improvements in the end
* last commit before merging #6
* rearranging variables to use alphabetic order
Co-authored-by: casperklein <casperklein@users.noreply.github.com>
Co-authored-by: Nick Pappas <radicand@users.noreply.github.com>
Co-authored-by: William Desportes <williamdes@wdes.fr>
* documentation and script updates trying to fix#1647
* preparations for refactoring target/bin/
* complete refactor for target/bin/
* changing script output slightly
* outsourcing functions in `bin-helper.sh`
* re-wrote linting to allow for proper shellcheck -x execution
* show explanation for shellcheck ignore
* adding some more information
this is needed to make test
"checking accounts: user3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf even when that file does not exist"
pass
* outsourced linting tests into its own file
* trigger rebuild
* added SCRIPT variable to setup.sh
* trigger rebuild again
* major test rewrite
* outsourced `hadolint` too
* rewrote some parts of the linting logic due to a logic bug
* adjusted TravisCI
* corrected .bats test line
* corrected logging in linting tests
* updated `hadolint`
Previously, only postfix-relaymap.cf and postfix-accounts.cf would be
used to populate the relayhost_map file.
Now, also use postfix-virtual.cf when present. To me, there is nothing
absurd about sending mail "From:" a virtual account (or more
specifically its domain) so it makes sense that when a $RELAY_HOST is
defined it should be used for virtual accounts as well.
check-for-changes.sh did not have a special case to handle lines in
postfix-relaymap.cf consisting of only a domain (indicating that said
domain should never be relayed). This case is handled by
start-mailserver.sh so when such a line existed, things would work well
until a config file update was detected by check-for-changes.sh. After
that, the generated relayhost_map file would be corrupted.
Fixed by factoring a 'populate_relayhost_map' function out of
start-mailserver.sh and into helper_functions.sh and reusing it in
check-for-changes.sh.
Note: There are certainly quite a few more pieces of code that could be
refactored in a similar fashion.
Note2: check-for-changes.sh would previously never update the
relayhost_map file when $ENABLE_LDAP was set to 1. I don't think this
was intended —there is after all no such condition in
start-mailserver.sh— and so this condition no longer applies.
If a change to one of the tracked files happened soon after (<1 second?)
a previously detected change, it could end up going undetected. In
particular, this could cause integration tests to fail (see next
commits).
Fixed by computing the new checksum file _before_ checking for changes.
Will extract certificates from acme.json as written by traefik for usage in dovecot and postfix.
Also watches acme.json for changes. For this to work the file has to be mounted/present at `/etc/letsencrypt/acme.json`
on hosts that belong to wildcard domains pop3.example.com might
actually resolve to pop3.example.com.[mydomain.com] and give a valid ip
the return code of fetchmail then no longer is 11 (dns failure) but
something else (2 for socket error in our case)
to make sure we always get return code 11 we use the domain name
pop3.example.com. that is not allowed to be resolved to a subdomain.
* Provide version information through labels
Can be retrieved by calling e.g. `docker inspect -f {{.Config.Labels}} tvial/docker-mailserver:testing`
Signed-off-by: Felix Bartels <felix@host-consultants.de>
* add build hook so that Docker Hub can work with the build args
Signed-off-by: Felix Bartels <felix@host-consultants.de>
* remove obsolete build-no-cache
build args invalidate build cache already
Signed-off-by: Felix Bartels <felix@host-consultants.de>
* adapt travis file
Signed-off-by: Felix Bartels <felix@host-consultants.de>
* shellcheck
Signed-off-by: Felix Bartels <felix@host-consultants.de>
