* splitting start-mailserver.sh
* refactoring part 2
* refactored setup-stack.sh
* stzarted adjusting target/bin/*.sh to use new usage format
* corrected lowercase-uppercase test error
* better handling of .bashrc variable export
* linting tests and fix for default assignements
* last stylistic changes and rebase
My `~` substitution and any usage of `/` within `start-mailserver.sh` has been replaced with the `|` delimiter instead as advised for matching style guide preference. Note there are other `sed` substitution delimiters still in use such as `+`.
Also added warning for empty `SSL_TYPE` ENV var that may result in an internal state config persist bug when changing `SSL_TYPE` depending on how a container is restarted.
* feat: Change Postfix smtpd_tls key and cert files to chain_files
Since Postfix 3.4, `smtpd_tls_cert_file` and `smtpd_tls_key_file` have been deprecated in favor of `smtpd_tls_chain_files` which supports a list of values where a single or sequence of file paths provide a private key followed by it's certificate chain.
* feat: Dual certificate support
`smtpd_tls_chain_files` allows for multiple key+cert bundles so that you can provide different key types, such as ECDSA and RSA.
To maintain compatibility with the current CERT/KEY ENV vars only a 2nd certificate is supported.
Since Dovecot 2.2.31 a related feature is also available, but it is limited to only providing one alternative certificate via separate cert and key settings.
---
This feature enables support for multiple certificates, eg for serving modern ECDSA certs with RSA as fallback.
* chore: Refactor variable names to meet style guide
Improved some comments too.
* chore: Have function definitions respect style guide
* chore: Minor edits to comments
* chore: Expand on comments for maintenance, alert of insecure config
When `SSL_TYPE` isn't properly setup, we're still offering SSL connections but not warning in logs about the insecurity of such, or why a misconfiguration may have occurred.
This commit more clearly communicates to the user that they should look into the issue before considering deploying to production.
The `TODO` comments communicate to any future maintainer to consider treating these improper configs as disabling TLS instead.
* fix: Use `snakeoil` cert
I mistakenly thought this was placeholder text, which broke some tests. This adds the two files in the correct order (private key followed by cert/chain), to fix that issue.
* fix: Disable alt cert for Dovecot if necessary
Certain scenarios may persist state of previously configured alt cert via ENV vars that are removed from a future run. If the config is not reset to original immutable state, this will correctly disable the config from using alt cert unintentionally.
* fix: Satisfy ShellCheck lint
By switching from string var to array / list expansion, this better stores the extracted result and applies it in a manner that ShellCheck linting approves, removing the need to disable the rule.
* feat: Support dual cert test
Few tweaks to the test script allows re-purposing it for covering dual cert support as well.
* chore: Rearranged cert and key lines
A little reorganization, mostly placing private key ahead of related cert lines.
* chore: Refactor `_set_certificate`
This should make the parameters a little less confusing.
Previously was 3 parameters, but the Postfix parameter (1st) may look like two variables if you don't pay attention to the surrounding quotes; while the Dovecot parameters (2nd + 3rd) would have an opposing order. There was also a variant where the `FULLKEYCHAIN` var was passed in three times.
Now it's two params, with the 2nd param as an optional one. If the 2nd param is provided, then the two params are in the order of private key then certificate, otherwise if only a single parameter it's a single PEM file with the full cert chain and private key bundled.
This avoids implying that Postfix and Dovecot might use different files.
* chore: Document current state of `SSL_TYPE` logic better
Inlined for the benefit of anyone else maintaining this section if I'm unable to address the concerns within my own time.
* docs: ENV vars
`TLS_LEVEL=old` isn't in the codebase anymore, not likely to be relevant to retain.
No point in documenting what is considered invalid / unsupported config value in the first place for `SSL_TYPE`.
`SSL_TYPE=manual` was missing documentation for both related file path ENV vars, they've been added along with their alt fallback variants.
* chore: Update Dovecot LMTP SSL test config
Not sure how relevant this is, the file isn't complete sync with the main dovecot `10-ssl.conf` config, adding the support just in case.
* chore: Rename `FULLKEYCHAIN` to avoid confusion
There doesn't appear to be a standardized name for this type of file bundle, and `keychain` may be misleading (fullkeychain often provides macOS keychain results on search engines).
Opting for a more explicit `KEY_WITH_FULLCHAIN` name instead.
* fix: Invalid var name
`_set_certificate` refactor commit accidentally changed a var name and committed that breaking the dual cert support (thanks tests!).
* test: Refactor `mail_ssl_manual.bats`
Proper test return values instead of `wc -l` based checking.
Tests with dual cert support active, tests that feature (to better detect failure case.
Third test case was unable to verify new self-signed certificate, added new certs signed with self-signed root CA.
Adjusted openssl `CApath` parameter to use `CAfile` instead as `letsencrypt` cert was replaced thus CA cert is missing from the system trust store.
* test: Properly check for files in `mail_ssl_manual.bats`
Fixes lint error.
Also realized I was accidentally asserting a file exists in the test environment, not within the container.
Resolved that and also added an additional test case to ensure the ENV var files are valid when passed in, in the event a change misconfigures them and that the issue is identified earlier.
* chore: Apply PR review feedback
Better format some strings that had mixed quotes when they weren't necessary.
Additionally DRYed up the config path for Postfix and Dovecot within the `_setup_ssl` method.
Co-authored-by: Georg Lauterbach <infrastructure@itbsd.com>
* let dkim generator accept selector as parameter
* test dkim-generator with selector parameter
* fix: correct name of domain argument in usage
* fix: adapt command to new syntax
* tests: use different quotes
* tests: use different quotes
* tests: remove domains that were never added
* style: change test name
* refactor: dkim setup
* style: remove trailing whitespace
* tests: remove test of removed dummy file
Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
* provide complete refactoring of openDKIM usage and tests
* fix leftover linting errors
* correct defualt key size and README usage
* provide independent order for arguments
* added `config` and adjusted usage information
* fixing shift in setup.sh
* adjust usage information to use new style and rename script
* use updated argument keysize instead of size
* rewrite to fix docker-mailserver#1808 (again)
* exiting script correctly now
* over-engineered usage information
the usage is now displayed like a man page and the paging mechanism (i.e. the display of the information) is borrowed from batcat
* fix typos
* Migrate PR#1730 from tomav/docker-mailserver repo to new
docker-mailserver/docker-mailserver repo
* Resolved review comments
* Moved counter increment to have consistency between fetchmail process
and fetchmail config files
* Added tests for new fetchmail option
Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
* first migration steps
* altered issue templates
* altered README
* removed .travis.yml
* adjusting registry & repository, Dockerfile and compose.env
* Close stale issues automatically
* Integrated CI with Github Actions (#3)
* feat: integrated ci with github actions
* fix: use secrets for docker org and update image
* docs: clarify why we use -t if no tty exists
* fix: correct remaining references to old repo
chore: prettier automatically updated markdown as well
* fix: hardcode docker org
* change testing image to just testing
* ci: add armv7 as a supported platform
* finished migration steps
* corrected linting in build-push action
* corrected linting in build-push action (2)
* minor preps for PR
* correcting push on pull request and minor details
* adjusted workflows to adhere closer to @wernerfred's diagram
* minor patches
* adjusting Dockerfile's installation of base packages
* adjusting schedule for stale issue action
* reverting license text
* improving CONTRIBUTING.md PR text
* Update CONTRIBUTING.md
* a bigger patch at the end
* moved all scripts into one directory under target/scripts/
* moved the quota-warning.sh script into target/scripts/ and removed empty directory /target/dovecot/scripts
* minor fixes here and there
* adjusted workflows for use a fully qualified name (i.e. docker.io/...)
* improved on the Dockerfile layer count
* corrected local tests - now they (actually) work (fine)!
* corrected start-mailserver.sh to make use of defaults consistently
* removed very old, deprecated variables (actually only one)
* various smaller improvements in the end
* last commit before merging #6
* rearranging variables to use alphabetic order
Co-authored-by: casperklein <casperklein@users.noreply.github.com>
Co-authored-by: Nick Pappas <radicand@users.noreply.github.com>
Co-authored-by: William Desportes <williamdes@wdes.fr>
* silence errorneous output when not generating reports
* remove incorrect variable assignment
* change error messages and logic when reports turned off
* changing warn -> inf
Co-authored-by: Charles Harris
Co-authored-by: Georg Lauterbach
* documentation and script updates trying to fix#1647
* preparations for refactoring target/bin/
* complete refactor for target/bin/
* changing script output slightly
* outsourcing functions in `bin-helper.sh`
* re-wrote linting to allow for proper shellcheck -x execution
* show explanation for shellcheck ignore
* adding some more information
* added option to use non-default network-interface (#1227)
* minor (stylistic) changes
* properly working with Bash arrays for CONTAINER_NETWORKS
* cleanup to trigger rebuild
* added CODE_OF_CONDUCT to trigger rebuild
Previously, only postfix-relaymap.cf and postfix-accounts.cf would be
used to populate the relayhost_map file.
Now, also use postfix-virtual.cf when present. To me, there is nothing
absurd about sending mail "From:" a virtual account (or more
specifically its domain) so it makes sense that when a $RELAY_HOST is
defined it should be used for virtual accounts as well.
check-for-changes.sh did not have a special case to handle lines in
postfix-relaymap.cf consisting of only a domain (indicating that said
domain should never be relayed). This case is handled by
start-mailserver.sh so when such a line existed, things would work well
until a config file update was detected by check-for-changes.sh. After
that, the generated relayhost_map file would be corrupted.
Fixed by factoring a 'populate_relayhost_map' function out of
start-mailserver.sh and into helper_functions.sh and reusing it in
check-for-changes.sh.
Note: There are certainly quite a few more pieces of code that could be
refactored in a similar fashion.
Note2: check-for-changes.sh would previously never update the
relayhost_map file when $ENABLE_LDAP was set to 1. I don't think this
was intended —there is after all no such condition in
start-mailserver.sh— and so this condition no longer applies.
If a change to one of the tracked files happened soon after (<1 second?)
a previously detected change, it could end up going undetected. In
particular, this could cause integration tests to fail (see next
commits).
Fixed by computing the new checksum file _before_ checking for changes.
Will extract certificates from acme.json as written by traefik for usage in dovecot and postfix.
Also watches acme.json for changes. For this to work the file has to be mounted/present at `/etc/letsencrypt/acme.json`
To prevent announcing software or version to malicious people or scripts, it is advised to hide such information.
This information is provided as part of the Lynis community project. It is related to Lynis control MAIL-8818 and should be considered as-is and without guarantees.
https://cisofy.com/lynis/controls/MAIL-8818/
[Postfix docs](http://www.postfix.org/postconf.5.html#tls_ssl_options):
> Disable SSL compression even if supported by the OpenSSL library. Compression is CPU-intensive, and compression before encryption does not always improve security.
[Postfix mailing list discussion](http://postfix.1071664.n5.nabble.com/patch-mitigate-CRIME-attack-td57978.html):
> The CRIME attack does not apply to SMTP, because unlike SMTP, there is no javascript in SMTP clients that makes them send thousands of email messages with chosen plaintext compressed together in the same packet with SASL credentials or other sensitive data.
> The auditor completely failed to take the context into account.
[Mailing list discussion of potential compression CRIME-like attack](https://lists.cert.at/pipermail/ach/2014-December/001660.html)
> keeping compression disabled is a good idea.
If you need a good test score, PCI compliance will likely flag compression despite not having any known risk with non-HTTP TLS.
This will allow to forward safely any email from any host, no matter how strict their SPF policy is, by setting `SRS_SENDER_CLASSES=envelope_sender,header_sender`.
@Tecnativa TT20505
Added optional FQDN arguement to setup.sh script which avoids using temporary container hostname for cert names. Also fixed issue with certs being saved outside config volume
* Added DEFAULT_RELAY_HOST setting
* If set this value will be used as the relayhost in /etc/postfix/maincf causing all mail to be delivered using this relay host
* Test for default relay host setting
* Modified start-mailserver.sh with two new options for SSL certificate Configuration ():
+ ‘’ (empty string) modifies dovecot configs to allow plain text access
+ * (default) does nothing but warn with message ‘SSL configured by default’
* Updated README.md:
SSL_TYPE environment variable with unknown value will set SSL by default
* Describe format for .env in README
* Display used domain and hostname even when they are not acceptable
This should be clearer for the user when the hostname was set incorrectly.
The message size limit was reduced in c8728eab from the postfix
default [1] of 10,240,000B = 10,000kiB = ~10MiB to only
1,048,576B = 1MiB. And the documentation claims that this would be 10MiB
instead of 1MiB.
Restore the old behaviour as default and fix the documentation as well.
[1]: http://www.postfix.org/postconf.5.html
* DOMAINNAME can fail to be set in postsrsd-wrapper.sh
if the container doesn’t have a proper hostname, postsrsd will fail to start
because SRS_DOMAIN is empty. Make a best effort to figure out the domain name
and provide a way to set one if needed.
* Ensure that the provided username actually contains a domain
* Update README.md to be consistent with addmailuser script
* Add a test to check if the username includes the domain
* Add new configuration for multi-domain relay hosts (#922)
* Creates new environment variables (replacing existing AWS_SES variables)
* Optionally allows more advanced setups using config files
* Update relay hosts during change detection (#922)
* Add helper scripts for adding relay hosts and per-domain auth
* Allow the possibility to deliver some mail directly
* adding a domain with no destination will exclude it from the
relayhost_map and so Postfix will attempt to deliver the mail directly
* tests for setup.sh script
* tests for relay host configuration
* these tests cover the code in `start-mailserver.sh` dealing with both
the env vars and the configuration files
* Added dependencies, binary, startup configuration
* Added env variable to dist files/readme
* send summary after each logrotate, added env variable for mail/logrotate interval
* remove mail.log from rsyslogs logrotate
* rotate mail.log when no email is set
* Added documentation for POSTFIX_LOGROTATE_INTERVAL
* Removed interval option, since its not being tested for.
* changed test to force logrotate to rotate fixed logrotate config
* readded setup_environment, made logrotate_setup being called everytime
* changed documentation for new variable names - again
* Did Documentation, added a default recipient, added test for default config.
* layout fix
* changed variable names apposite the documentation
Added `/var/lib/dovecot/` to the list of folders that get persisted in `mailstate`. So the creation of `ssl-parameters.dat` has not to be done every restart again. This may only take some dozen seconds, but can be very long on systems with high load.
* update postmaster_address in dovecot config according to POSTMASTER_ADDRESS env var
* tests: add another test for postmaster_address with default settings
* remove two ciphers according to https://www.htbridge.com/ssl/ (NIST, HIPAA)
* added a switch via an environment variable to choose between modern and intermediate ciphers
* new setup.sh function, new tests, new script and some minor updates to main.cf
* fix for missing files
* removed obsolete test-files
* restart postfix if neccessary.
* see pr #845
* fixed typo
* fixed branchmixup
* changed postfix reload command & changed to operate on container instead of image
* reload postfix only on adding new restriction
* main.cf is only changed when user is added.
- Postfix reload changed
- working on container instead of image now in setup.sh
- added cleanup after tests
* moved cleanup to makefile
* Introduced Postscreen
cheaper, earlier and simpler blocking of zombies/spambots.
From http://postfix.cs.utah.edu/POSTSCREEN_README.html :
As a first layer, postscreen(8) blocks connections from zombies and other spambots that are responsible for about 90% of all spam. It is implemented as a single process to make this defense as cheap as possible.
Things we need to consider:
- Do we need a whitelist/backlist file? (http://postfix.cs.utah.edu/postconf.5.html#postscreen_access_list)
- Via introducing an optional config/postfix-access.cidr
- The only permanent whitelisting I could imagine are monitoring services(which might (still?) behave weird/hastely) or blacklisting backup servers(since no traffic should be coming from them anyway)
- Do we need deep inspections? They are desireable, but these tests are expensive: a good client must disconnect after it passes the test, before it can talk to a real Postfix SMTP server. Considered tests are:
- postscreen_bare_newline_enable (http://postfix.cs.utah.edu/postconf.5.html#postscreen_bare_newline_action)
- postscreen_non_smtp_command_enable (http://postfix.cs.utah.edu/postconf.5.html#postscreen_non_smtp_command_action)
- postscreen_pipelining_enable (http://postfix.cs.utah.edu/postconf.5.html#postscreen_pipelining_action)
- Do we need to make the blacklisting via dnsblocking configurable? It's currently set and weighted as follows, where a score of 3 results in blocking, a score of -1 results in whitelisting:
(*: adds the specified weight to the SMTP client's DNSBL score. Specify a negative number for whitelisting.)
(http://postfix.cs.utah.edu/postconf.5.html#postscreen_dnsbl_sites)
- zen.spamhaus.org*3
- bl.mailspike.net
- b.barracudacentral.org*2
- bl.spameatingmonkey.net
- bl.spamcop.net
- dnsbl.sorbs.net
- psbl.surriel.com
- list.dnswl.org=127.0.[0..255].0*-2
- list.dnswl.org=127.0.[0..255].1*-3
- list.dnswl.org=127.0.[0..255].[2..3]*-4
- What to do when blacklisting? I currently set it to drop. We could
- ignore: Ignore the failure of this test. Allow other tests to complete. Repeat this test the next time the client connects. This option is useful for testing and collecting statistics without blocking mail.
- enforce: Allow other tests to complete. Reject attempts to deliver mail with a 550 SMTP reply, and log the helo/sender/recipient information. Repeat this test the next time the client connects.
- drop: Drop the connection immediately with a 521 SMTP reply. Repeat this test the next time the client connects.
In the end I think we could drop postgrey support. Postscreen replaces postgrey in its entirety, while being more selective and not delaying mail. Especially if we consider using the deep inspection options of postscreen.
Hope that wasn't too much to read! ;)
* main.cf got misformatted..
Don't know how, should be ok now.
* fixed malformatted main.cf & repaired master.cf
* reenabled rbl stuff.. It's cached, therefore doesn't hurt
* fixed tests
* added tests, repaired tests, added info, introduced new Variable POSTSCREEN_ACTION, fixes
* install dovecot from backports
* dovecot 2.2.33 has a slightly different TLS-configuration than 2.2.27
* want to have both images a the same time
* make use of the /etc/dovecot/ssl as mkcert.sh (2.2.33) is using that folder for certs.
* Allow setup of LDAP STARTTLS for Dovecot and Postfix
* Added tests for TLS config override
* Add missing Postfix TLS options
* Added missing new line at the end of the file
* Added STARTTLS tests for Postfix config
* Switch to stretch-slim as base image.
- first step correct the testdata, as newer packages are more strict
about the mail-structure.
* Switch to stretch-slim: correcting the test-environment and the build
- add missing build-step to make
- clean the userdb aswell
- use timeout of netcat, as postgrey would not close the connection
- there is 2 extra mail-logs -> assert_output 5
- cosmetic: use "" instead of ''
* Switch to stretch-slim:
new image:
- smaller size
- 0 CVEs compared to 11 CVEs in ubuntu 16.04 Image
better backport situation
- postfix 3.1.6 vs 3.1.0
- fail2ban 0.9.6 vs 0.9.3
...
changes needed because of stretch-slim:
- add missing gnupg and iproute2 package
- remove non-free rar, unrar-free should do
- rsyslog does not add syslog user and has different conf-structure
- pyzor command discover was deprecated and is missing in the new
stretch package
- dovecot does not know SSLv2 anymore. removed because of warnings in
log
- iptables does not know imap3, IMAP working group chose imap2 in favor
of imap3
* Switch to debian stretch slim:
SSLv2 seems to be a not known protocol anymore - good!
* switch to debian stretch slim:
make this test more stable. there might be more than only one mail.log
(mail.info, mail.warn, ...)
* switch to debian stretc slim:
new openssl 1.1.0 needs stronger ciphers, removed some weekers ones.
Please, look through the new list of cipher! this needs to be done in
another commit for all other SSL/TLS-Endpoints aswell.
* Switch to debian stretch slim:
let our server pre-empt the cipher list.
Did a read through, wwwDOTpostfixDOTorg/FORWARD_SECRECY_READMEDOThtml
and
wwwDOTpostfixDOTorg/TLS_READMEDOThtml
* Switch to debian stretch slim: lets give this openssl-based test a new and independent but identical container. many other test on the main 'mail' container might interfere here.
* Switch to debian stretch slim: remove unused lines
* fix for issue #747 (NOT TESTED YET)
Need to test this. Dont merge it yet.
* Small Cleanup and bug fixes Not done Yet!
Not done yet. Dont merge. See comment in issue #747
* Add --ignore-missing. Now is fixed.
