* chore: Detect missing final newline in configs read
These lines will be not be processed by `read`, emit a warning to raise awareness.
* fix: Ensure parsed config has final newline appended (when possible)
This functionality was handled in `accounts.sh` via a similar sed command (that the linked references also offer).
`printf` is better for this, no shellcheck comment required either.
We additionally don't attempt to modify files that are read-only.
* fix: Ensure parsed configs have CRLF to LF corrected (where possible)
Likewise, this runtime fix was only covering two config files. It now applies to all callers of this method.
* fix: Sanitize `postfix-master.cf` via helper
This feature should have been using the helper to avoid user error from their config updates accidentally introducing subtle breakage implicitly (due to CRLF or missing final newline).
* tests: Add test cases for new helpers
* tests: `rm` is redundant when using `BATS_TEST_TMPDIR`
This temporary directory is created and removed implicitly. Even after a test failure.
* chore: Remove old `postfix-virtual.cf` migration logic
This was introduced in 2018, there should be no one needing to rely on this anymore?
* tests: Remove comment on sed failure concern
* chore: Add entry to `CHANGELOG.md`
* Apply suggestions from code review
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
---------
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
The UID / GID shifted during a new release. Until DKIM handling is refactored in a new major release, this fix ensures the content maintains the expected `_rspamd` ownership.
This is a more explicit reminder for any future contributors that get thrown off by the usage of `sed` here and may be inclined to change it.
Add a link to reference a comment where it's already been explored what the alternative `sed` invocations available are.
* feat: add support for MTA-STS for outgoing mails
* Hook-up mta-sts-daemon into basic process handling test
* fix: Call python script directly
The python3 shebang will run it, which will now meet the expectations of the process testing via pgrep. fail2ban has the same approach.
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* chore: Disable `smtputf8` support in config
This was always configured disabled at runtime, better to just set explicitly in `main.cf` unless config diverges when Dovecot is enabled to opt-out of this feature.
* added check for Rspamd DKIM on startup
The newly added function `__rspamd__check_dkim_permissions` performs a
check on DKIM private key files. This is useful to prevent issues
like #3621 in the future. The function is deliberately kept simple and
may not catch every single misconfiguration in terms of permissions and
ownership, but it should be quite accurate.
Please note that the Rspamd setup does NOT change at all, and the checks
will not abort the setup in case they fail. A simple warning is emmited.
* add more documentation to Rspamd functions
* Apply suggestions from code review
* improve `__do_as_rspamd_user`
* rework check similar to review suggestion
see https://github.com/docker-mailserver/docker-mailserver/pull/3627#discussion_r1388697547
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* update K8s deployment
Because `allowPrivilegeEscalation` controls SUID/SGID, we require it
when postdrop is invoked.
* correct permissions for maildrop/public
The reason our permissions previously worked out as that in setups where
SUID/SGID worked, the binaries used to place files in these directories
already have SGID set; the current set of permissions makes less sense
(as explained in this comment:
https://github.com/docker-mailserver/docker-mailserver/issues/3619#issuecomment-1793816412)
Since the binaries used to place files inside these directories alredy
have SUID/SGID set, we do not require these bits (or the sticky bit) to
be set on the directories.
* Apply suggestions from code review
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* outsource Rspamd ENVs into explicit helper
This will allow us to uniformly source the helper and get the values
from everywhere consistently. This is more than desirable since we will
be using these values not only for the Rspamd setup, but also for DKIM
management and during change-detection.
* integrate Rspamd into changedetection
We outsource one more function to reside in the helper script for Rspamd
so that we can call this function from the Rspamd setup and from the
changedetection functionality too.
* realize deprecation of old commands file for Rspamd
THIS IS A BREAKING CHANGE!
This change realizes the log message: "Using old file location now
(deprecated) - this will prevent startup in v13.0.0" Startup will now
fail.
* added '--force' option to Rspamd DKIM script
* use new helper to get ENVs for Rspamd in DKIM script
* remove the need for linking directories
This was unnecessary, as explained in
https://github.com/docker-mailserver/docker-mailserver/pull/3597#discussion_r1369413599
* Apply suggestions from code review
review by @polarathene
* apply more review feedback from @polarathene
- <https://github.com/docker-mailserver/docker-mailserver/pull/3599#discussion_r1370885519>
- <https://github.com/docker-mailserver/docker-mailserver/pull/3599#discussion_r1370904201>
* update documentation
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Instead of using `etc/rspamd/override.d/dkim_signing.conf`, we will now
be using `/tmp/docker-mailserver/rspamd/override.d/dkim_signing.conf`.
The new location is persisted (and linked again during startup) and
hence better suited.
* simplify `_setup_logrotate`
* adjust Rspamd's log file and improve it's management
* add information to docs about Rspamd log
* update log query helper to allow another file location
* bail in case `LOGROTATE_INTERVAL` is invalid
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Some deployment scenarios are not compatible with `5000:5000` static vmail user with `/var/mail`. This feature allows adjusting the defaults to a UID / GID that is compatible.
Signed-off-by: vincent <vincent@ducamps.win>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* chore: Drop management of `SASLAUTHD_*` ENV
- `variables-stack.sh` does not need to manage all these extra ENV or store them. They're not used anywhere else.
- `saslauthd.sh` is the only consumer of these ENV which are effectively direct key/value mappings, with some defaults provided / inherited.
Instead of trying to conditionally support key/value pairs when ENV is set, we could instead use `sed` to delete lines with empty values.
* chore: Drop fallbacks + update configs to match docs
- Drop deprecated support:
- `DOVECOT_HOSTS` is an ENV deprecated since v10.
- Fallback for missing URI scheme introduced for Dovecot and SASLAuthd in v10.
- Adding error log message when no LDAP URI scheme is detected for the supported ENV (when set).
- Docs updated for ENV to reflect the mandatory requirement. `mailserver.env` partially synced equivalent sections.
- Provided base LDAP configs (for overriding) likewise updated from `domain.com` to `example.com`.
- LDAP test updated for required `ldap://` URI scheme. Common ENV shared across LDAP configs hoisted out of the Postfix group.
* chore: Remove unset lines in generated `saslauthd.conf`
- Deprecation startup script check is kept for `ENABLE_LDAP=1` but adjusted to emit an error instead. It can be dropped in a future release. Just a precaution for those who mistakenly update (_possibly via automation_) without checking the release notes, an error log is somewhat helpful, although it could alternatively panic?
- Docs updated to remove the `ENABLE_LDAP=1` usage
- ENV docs updated to reference a maintained LDAP image.
- Changelog includes the breaking change, and slight revision to prior release mention of deprecation.
The new function can
1. update/append
2. update/prepend
3. initialize if non-existent
options in `/etc/postfix/main.cf` in a safe and secure manner. When the
container is improperly restarted, the option is not applied twice.
---
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* Fix issue with concatenating $dmarc_milter and $dkim_milter in main.cf
Upon each start the `smtpd_milters` and `non_smtpd_milters` would be extended with the following:
```
smtpd_milters = $dmarc_milter $dkim_milter
non_smtpd_milters = $dkim_milter
```
In my case they became long enough that mail delivery stopped. I think this was because of the extra headers that are added by these steps. (which in turn would cause the mail to be dropped)
* fix sed to work when the variables are there and when they are not.
---------
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* add new home dir for Dovecot
I tried changing the mail dir, but this is a _very_ disruptive change,
so I took approach 3 on
<https://doc.dovecot.org/configuration_manual/home_directories_for_virtual_users/>,
whereby the home directory is now inside the mail directory.
The MDBOX/SDBOX formats are not touched by this change. The change
itself could be considered breaking though.
* adjust Sieve tests accordingly
* Update target/dovecot/10-mail.conf
* Update target/dovecot/auth-passwdfile.inc
---------
Co-authored-by: Casper <casperklein@users.noreply.github.com>
* adjust learning of ham
See #3333
When moving a mail from the Junk folder to the Trash folder, the mail
previously classified as ham due to the wildcard match of `*`. Because
the syntax does not allow for negation, we can only change the behavior
in a way that mails are learned as ham when they are moved into `INBOX`
from `Junk`. This is reasonable though.
* adjust tests accordingly
* adjust docs accordingly
* add sanity check for Clam size & adjusted MaxScanSize
The second part is of special importance! See
<https://askubuntu.com/a/1448525>, which explains that the maximum scan
size is important as well. We previously just set the maximum file size,
which actually is pretty insecure as we silently not scan mile bigger
than `MaxScanSize`. This is corrected now.
* add SlamAV size configuration to Rspamd
* move modules adjustment file to new location
Because we link `/tmp/docker-mailserver/rspamd/override.d` to
`/etc/rspamd/override.d`, I think it makes sense to move the modules
adjustment file into `/tmp/docker-mailserver/rspamd/` as well.
I write the code in a way that it is backwards compatible for now, so
this is NOT a breaking change.
* minor improvement to `__rspamd__handle_user_modules_adjustments`
The expansion of `ARGUMENT3` is now done in a way that only adds the
whitespace in case the variable is set and not null.
* move test file structure to respect latest changes
Because we're now linking `rspamd/override.d/`, we can simplify the
setup a bit. But this requires a change in directory structure.
The current Rspamd test will be renamed to `rspamd_full.bats`, because I
plan on adding more tests in different files for different feature sets.
This is done to make this feature well-tested!
* improved and added tests to Rspamd-full
FYI: The line
```bats
_run_in_container grep 'sieve_global_extensions.*\+vnd\.dovecot\.pipe'
"${SIEVE_CONFIG_FILE}"
```
was testing a condition that should actually not be met, but when I
started working on this feature, I thought this was the correct
configuration. Adding the `assert_success` statements revealed this
wrong line.
I also added tests to check whether `override.d` is linked correctly.
* renamed: `rspamd.bats` => `rspamd_full.bats`
* added new tests for incomplete Rspamd feature set
We now test that warnings are emitted & features are disabled correctly.
* update documentation
* added checks whether OpenDKIM/OpenDMARC/policyd-spf are enabled
* added functions to check if VAR is 0/0 or an int
and also added tests.
I also adjusted the test file to not run in a container, because there
is no need. This also decreases test time, which, in turn, increases
maintainers' happiness.
* added more checks to Rspamd setup
I added the helpers from the previous commit to the Rspamd setup to make
the whole setup more robust, and indicate to the user that an ENV
variable's value is incorrect.
While we did not issues for this in the past, I believe it to be
worthwhile for the future.
* added canonical directory for users to place files in
This dir is canonical with DMS's optional configuration dirs, as it
lives in well-known volume mounts. Hence, users will not need to adjust
`/etc/rspamd/override.d` manually anymore, or mount a volume to this
place.
The docs explain this now, but the DKIM page needs a slight update on
this too I guess. I will follow-up here.
* misc minor improvements
* use variables for common directories
