* chore: `relay.sh` helper - Reference user config paths via variables
* chore: Better document postfix helper `_vhost_collect_postfix_domains()`
The functionality is effectively the same for the two configs for the most part when it comes to parsing out a domain from the target value.
Virtual aliases is more flexible in value, which may not have a domain-part present (manual user edit).
* chore: `check-for-change.sh` - Support VHOST change visibility
- Moves the "handle changes" logic into it's own scoped function, out of the main change detection loop logic.
- This will be benefit a future commit change that will rely on `VHOST_UPDATED=1`.
* chore: `relay.sh` - Minor revisions to minimize diff noise
- Better phrasing of the current logic comments.
- Regex patterns assigned to variables (easier to grok intention)
- Bulk of the logic for generating `/etc/postfix/relayhost_map` wrapped into a separate function with Postfix config setting handled separately.
* refactor: `relay.sh` opt-out logic
- Split the two distinct features that configure `/etc/postfix/relayhost_map` into separate functions (_`MATCH_VALID` var no longer needed for legacy support_).
- Instead of extracting domains from `postfix-accounts.cf` + `postfix-virtual.cf`, this has already been handled at `/etc/postfix/vhost`, sourcing from there is far less complicated.
- Rename loop var `DOMAIN_PART`to `SENDER_DOMAIN` for better context of what it represents when appended to the config file.
- Revised maintenance notes + guidance towards a future refactor of this relayhost feature support.
* docs: `relay.sh` - Additional comment revisions
* feat: `DEFAULT_RELAY_HOST` can now also use relay credentials ENV
- Remove comment regarding `smtp_sasl_password_maps = static:${RELAY_USER}:${RELAY_PASSWORD}`, it could be used but `main.cf` presently has `644` permissions vs the `sasl_passwd` file permissions of `600`, less secure at preventing leaking of secrets (ignoring the ENV exposure itself).
- Move the `main.cf` settings specific to relayhost credentials support / security into to the relevant function scope instead. This also allows for the configuration to be applied by a change detection event without container restart requirement.
- Outer functions for setup and change detection to call have a clearer config dependency guard, as does the `_legacy_support()`.
- These changes now support `DEFAULT_RELAY_HOST` to leverage the relay credentials ENV as well.
- `DATABASE_RELAYHOSTS` is available in scope to the functions called here that reference it.
* docs: Revised ENV docs on relay host config
Better quality guidance on configuring relay hosts.
* chore: Add entry to `CHANGELOG.md`
* fix: `relay.sh` - `grep` regex compatibility with `+` requires `-E`
* chore: `postfix.sh` - `FIRST_FIELD` => More descriptive field name
* docs: Better document DMS volumes
* docs: Remove any mention of `ONE_DIR` ENV
* chore: Remove `ONE_DIR` ENV from scripts
Only `ONE_DIR=0` has any effect. As the actual feature is now dependent upon the `/var/mail-state` location existing.
It is advised not mounting anything there instead if wanting to avoid runtime state consolidation.
* docs: Adjust link ref convention
This is more search friendly / organized to find references to all DMS volumes.
* lint: Ensure final newline is present
VSCode by default excludes this if the last line rendered is removed (rendered as a separate blank line).
A separate setting can enforce adding the final newline upon save regardless.
* correct misc typos
We also seem to be favoring `behavior` over `behaviour`.
* bump MkDocs version
* resolve errors shown when buildg docs
* improve the Rspamd page
* behaviour -> behavior
Streamline the usage of this word. The majority used behavior, so I
opted to go with this way of spelling it.
* Apply suggestions from code review
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* feat: add support for MTA-STS for outgoing mails
* Hook-up mta-sts-daemon into basic process handling test
* fix: Call python script directly
The python3 shebang will run it, which will now meet the expectations of the process testing via pgrep. fail2ban has the same approach.
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* chore: Extract out Dovecot Quota test cases into new test file
Test cases are just cut + paste, no logic changed there yet.
* chore: Rename test case descriptions
* chore: Use `setup ...` methods instead of direct calls
* chore: Adjust `_run_in_container_bash` to `_run_in_container`
Plus some additional bug fixes in the disabled test case
* tests(refactor): Revise ENV test cases for max mailbox and message sizes
* tests(refactor): Revise ENV test cases for mailbox and message limits v2
Removes the extra variables and filtering in favour of explicit values instead of matching for comparison.
- Easier at a glance to know what is actually expected.
- Additionally reworks the quota limit checks in other test cases. Using a different formatter for `doveadm` is easier to match the desired value (`Limit`).
* chore: Sync improvement from `tests.bats` master
---
NOTE: This PR has been merged to avoid additional maintenance burden without losing the improvements. It was not considered complete, but remaining tasks were not documented in the PR.
- Make this easier to find when browsing the example environment file.
- Adjust ENV documentation to properly mark the actual default value.
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Some deployment scenarios are not compatible with `5000:5000` static vmail user with `/var/mail`. This feature allows adjusting the defaults to a UID / GID that is compatible.
Signed-off-by: vincent <vincent@ducamps.win>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* chore: Drop management of `SASLAUTHD_*` ENV
- `variables-stack.sh` does not need to manage all these extra ENV or store them. They're not used anywhere else.
- `saslauthd.sh` is the only consumer of these ENV which are effectively direct key/value mappings, with some defaults provided / inherited.
Instead of trying to conditionally support key/value pairs when ENV is set, we could instead use `sed` to delete lines with empty values.
* chore: Drop fallbacks + update configs to match docs
- Drop deprecated support:
- `DOVECOT_HOSTS` is an ENV deprecated since v10.
- Fallback for missing URI scheme introduced for Dovecot and SASLAuthd in v10.
- Adding error log message when no LDAP URI scheme is detected for the supported ENV (when set).
- Docs updated for ENV to reflect the mandatory requirement. `mailserver.env` partially synced equivalent sections.
- Provided base LDAP configs (for overriding) likewise updated from `domain.com` to `example.com`.
- LDAP test updated for required `ldap://` URI scheme. Common ENV shared across LDAP configs hoisted out of the Postfix group.
* chore: Remove unset lines in generated `saslauthd.conf`
- Deprecation startup script check is kept for `ENABLE_LDAP=1` but adjusted to emit an error instead. It can be dropped in a future release. Just a precaution for those who mistakenly update (_possibly via automation_) without checking the release notes, an error log is somewhat helpful, although it could alternatively panic?
- Docs updated to remove the `ENABLE_LDAP=1` usage
- ENV docs updated to reference a maintained LDAP image.
- Changelog includes the breaking change, and slight revision to prior release mention of deprecation.
* adjust learning of ham
See #3333
When moving a mail from the Junk folder to the Trash folder, the mail
previously classified as ham due to the wildcard match of `*`. Because
the syntax does not allow for negation, we can only change the behavior
in a way that mails are learned as ham when they are moved into `INBOX`
from `Junk`. This is reasonable though.
* adjust tests accordingly
* adjust docs accordingly
The user management docs are now one page, because the division between
accounts and aliases is useless because there simply isn't enough
content to justify the split. I improved and updated the text a bit.
* added options to toggle OpenDKIM & OpenDMARC
rspamd can provide DKIM signing and DMARC checking itself, so users
should be able to disable OpenDKIM & OpenDMARC. The default is left at
1, so users have to to opt-in when the want to disable the features.
* misc small enhancements
* adjusted start of rspamd
The order of starting redis + rspamd was reversed (now correct) and
rspamd now starts with the correct user.
* adjusted rspamd core configuration
The main configuration was revised. This includes AV configuration as
well as worker/proxy/controller configuration used to control the main
rspamd processes.
The configuration is not tested extensively, but well enough that I am
confident to go forward with it until we declare rspamd support as
stable.
* update & improve the documentation
* add tests
These are some initial tests which test the most basic functionality.
* tests(refactor): Improve consistency and documentation for test helpers (#3012)
* added `ALWAYS_RUN` target `Makefile` recipies (#3013)
This ensures the recipies are always run.
Co-authored-by: georglauterbach <44545919+georglauterbach@users.noreply.github.com>
* adjusted rspamd test to refactored test helper functions
* improve documentation
* apply suggestions from code review (no. 1 by @polarthene)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* streamline heredoc (EOM -> EOF)
* adjust rspamd test (remove unnecessary run arguments)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* chore: Drop ENV `ENABLE_POSTFIX_VIRTUAL_TRANSPORT`
* tests(chore): Remove redundant `dovecot-lmtp` config
None of this is needed. Only relevant change is changing the LMTP service listener for Dovecot and that can be delegated to `user-patches.sh`.
* tests(refactor): Use `user-patches.sh` instead of replacing config file
The only relevant changes in `test/config/dovecot-lmtp` regarding LMTP was:
- `/etc/dovecot/dovecot.conf` (`protocols = imap lmtp`) and `/etc/dovecot/protocols.d/` (`protocols = $protocols lmtp`).
- `conf.d/10-master.conf` only changed the LMTP service listener from a unix socket to TCP on port 24 (_this was the only change required for the test to pass_).
None of those configs are required as:
- `protocols = imap pop3 lmtp` [is the upstream default](https://doc.dovecot.org/settings/core/#core_setting-protocols), no need to add `lmtp`.
- The LMTP service listener is now configured for the test with `user-patches.sh`.
* tests(refactor): `mail_lmtp_ip.bats`
- Converted to new testing conventions and common container helpers.
- `ENABLE_POSTFIX_VIRTUAL_TRANSPORT` was not relevant, dropped.
- Revised test cases, logic remains the same.
- Large custom config used was not documented and doesn't appear to serve any purpose. Simplified by replacing with a single modification with `user-patches.sh`.
- Added some additional comments for context of test and improvements that could be made.
* tests(chore): Adjust comments
The comment from `mail_hostname` provides no valid context, it was likely copied over from `tests.bats` in Oct 2020 by accident.
The email sent is just for testing, nothing relevant to LMTP.
---
Added additional comment for test to reference extra information from.
* tests(chore): Update similar log line matching
Extracts out the match pattern and formatting commands into separate vars (reduces horizontal scrolling), and includes extra docs about what the matched line should be expected to look like.
These two links have remained broken for over 6 months. Removing them.
* chore(housekeeping): Broken links
* chore: Remove broken links from `mailserver.env`
As per deprecation notice from v11.3 release notes, and a related prior PR; this ENV is to be removed.
It's no longer considered useful, and none of the tests that configured it were actually using it for relaying anything.
* outsourcing env variable setup
This commit contains major parts of the work of refactoring the setup
and usage of environment variables. It outsources the setup into its own
script and provides dedicated functions to be executed at a later point in time.
A **new** env variable was added: `USER_PROVISIONG` which provides a
better way of defining which method / protocol to use when it comes to
setting up users. This way, the `ENABLE_LDAP` variable is deprecated,
but all of this is backwards compatible due to a "compatibility layer", a function provided by the new variables script.
This is not a breaking change. It mostly refators internal scripts. The
only change facing the user-side is the deprecation of `ENABLE_LDAP`. We
can prolong the period of deprecation for this variable as long as we
want, because the new function that ensures backwards compatibility
provides a clean interface for the future.
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
* first adjustments to use Fail2Ban with nftables
* replace `iptables` -> `nftables` and adjust tests
nftables lists IPs a bit differently , so the order was adjusted for the
tests to be more flexible.
* line correction in mailserver.env
* change from `.conf` -> `.local` and remove redundant config
* revert HEREDOC to `echo`
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Dominic Germain
Jesse Portnoy
Hans-Cees Speel
Brennan Kinney
Georg Lauterbach
Joerg Sonnenberger
Keval Kapdee
Casper
Vincent Ducamps
H4R0
rriski
LucidityCrash
JiLleON