docker
/
docker-mailserver
mirror of https://github.com/tomav/docker-mailserver.git
1
0
Fork 0
Code Issues Releases Wiki Activity

spam: use Sieve for rewriting subject with Rspamd & SA/Amavis (#3820)

This commit is contained in:
Georg Lauterbach 2024-01-29 13:38:01 +01:00 committed by GitHub
parent 3b11a8305e
commit afb0093939
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
16 changed files with 262 additions and 179 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
CHANGELOG.md
View File

@ -37,6 +37,14 @@ The most noteworthy change of this release is the update of the container's base
- DMS `main.cf` has renamed `postscreen_dnsbl_whitelist_threshold` to `postscreen_dnsbl_allowlist_threshold` as part of this change.
- `smtpd_relay_restrictions` (relay policy) is now evaluated after `smtpd_recipient_restrictions` (spam policy). Previously it was evaluated before `smtpd_recipient_restrictions`. Mail to be relayed via DMS must now pass through the spam policy first.
- The TLS fingerprint policy has changed the default from MD5 to SHA256 (_DMS does not modify this Postfix parameter, but may affect any user customizations that do_).
- **Environment Variables**:
- `SA_SPAM_SUBJECT` has been renamed into `SPAM_SUBJECT` to become anti-spam service agnostic. ([3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820))
- As this functionality is now handled in Dovecot via a Sieve script instead of the respective anti-spam service during Postfix processing, this feature will only apply to mail stored in Dovecot. If you have relied on this feature in a different context, it will no longer be available.
- Rspamd previously handled this functionality via the `rewrite_subject` action which as now been disabled by default in favor of the new approach with `SPAM_SUBJECT`.
- `SA_SPAM_SUBJECT` is now deprecated and will log a warning if used. The value is copied as a fallback to `SPAM_SUBJECT`.
- The default has changed to not prepend any prefix to the subject unless configured to do so. If you relied on the implicit prefix, you will now need to provide one explicitly.
- `undef` was previously supported as an opt-out with `SA_SPAM_SUBJECT`. This is no longer valid, the equivalent opt-out value is now an empty value (_or rather the omission of this ENV being configured_).
- The feature to include [`_SCORE_` tag](https://spamassassin.apache.org/full/4.0.x/doc/Mail_SpamAssassin_Conf.html#rewrite_header-subject-from-to-STRING) in your value to be replaced by the associated spam score is no longer available.
### Updates
@ -45,6 +53,8 @@ The most noteworthy change of this release is the update of the container's base
- It's only functionality remaining was to opt-out of run-time state consolidation with `ONE_DIR=0` (_when a volume was already mounted to `/var/mail-state`_).
- **Tests:**
- Refactored helper methods for sending e-mails with specific `Message-ID` headers and the helpers for retrieving + filtering logs, which together help isolate logs relevant to specific mail when multiple mails have been processed within a single test. ([#3786](https://github.com/docker-mailserver/docker-mailserver/pull/3786))
- **Rspamd**:
- The `rewrite_subject` action, is now disabled by default. It has been replaced with the new `SPAM_SUBJECT` environment variable, which implements the functionality via a Sieve script instead in favor of being anti-spam service agnostic ([3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820))
### Fixes

35
docs/content/config/environment.md
View File

@ -51,7 +51,7 @@ Configures the provisioning source of user accounts (including aliases) for user
!!! tip "OAuth2 Support"
Presently DMS supports OAuth2 only as an supplementary authentication method.
Presently DMS supports OAuth2 only as an supplementary authentication method.
- A third-party service must provide a valid token for the user which Dovecot validates with the authentication service provider. To enable this feature reference the [OAuth2 configuration example guide][docs::auth::oauth2-config-guide].
- User accounts must be provisioned to receive mail via one of the supported `ACCOUNT_PROVISIONER` providers.
@ -354,6 +354,16 @@ Enable to treat received spam as "read" (_avoids notification to MUA client of n
- `X-Spam: Yes` (_added by Rspamd_)
- `X-Spam-Flag: YES` (_added by SpamAssassin - requires [`SPAMASSASSIN_SPAM_TO_INBOX=1`](#spamassassin_spam_to_inbox)_)
##### SPAM_SUBJECT
This variable defines a prefix for e-mails tagged with the `X-Spam: Yes` (Rspamd) or `X-Spam-Flag: YES` (SpamAssassin/Amavis) header.
Default: empty (no prefix will be added to e-mails)
??? example "Including trailing white-space"
Add trailing white-space by quote wrapping the value: `SPAM_SUBJECT='[SPAM] '`
#### Rspamd
##### ENABLE_RSPAMD
@ -562,7 +572,7 @@ Changes the interval in which log files are rotated.
- [`MOVE_SPAM_TO_JUNK=1`](#move_spam_to_junk)
- [`MARK_SPAM_AS_READ=1`](#mark_spam_as_read)
- [`SA_SPAM_SUBJECT`](#sa_spam_subject)
- [`SPAM_SUBJECT`](#spam_subject)
##### SA_TAG
@ -602,8 +612,8 @@ When a spam score is high enough, mark mail as spam (_Appends the mail header: `
!!! info "Interaction with other ENV"
- [`SA_SPAM_SUBJECT`](#sa_spam_subject) modifies the mail subject to better communicate spam mail to the user.
- [`MOVE_SPAM_TO_JUNK=1`](#move_spam_to_junk): The mail is still delivered, but to the recipient(s) junk folder instead. This feature reduces the usefulness of `SA_SPAM_SUBJECT`.
- [`SPAM_SUBJECT`](#spam_subject) modifies the mail subject to better communicate spam mail to the user.
- [`MOVE_SPAM_TO_JUNK=1`](#move_spam_to_junk): The mail is still delivered, but to the recipient(s) junk folder instead. This feature reduces the usefulness of `SPAM_SUBJECT`.
##### SA_KILL
@ -659,23 +669,6 @@ Controls the spam score threshold for triggering an action on mail that has a hi
[amavis-docs::actions]: https://www.ijs.si/software/amavisd/amavisd-new-docs.html#actions
[amavis-docs::quarantine]: https://www.ijs.si/software/amavisd/amavisd-new-docs.html#quarantine
##### SA_SPAM_SUBJECT
Adds a prefix to the subject header when mail is marked as spam (_via [`SA_TAG2`](#sa_tag2)_).
- **`'***SPAM*** '`** => A string value to use as a mail subject prefix.
- `undef` => Opt-out of modifying the subject for mail marked as spam.
??? example "Including trailing white-space"
Add trailing white-space by quote wrapping the value: `SA_SPAM_SUBJECT='[SPAM] '`
??? example "Including the associated spam score"
The [`_SCORE_` tag][sa-docs::score-tag] will be substituted with the SpamAssassin score: `SA_SPAM_SUBJECT=***SPAM(_SCORE_)***`.
[sa-docs::score-tag]: https://spamassassin.apache.org/full/4.0.x/doc/Mail_SpamAssassin_Conf.html#rewrite_header-subject-from-to-STRING
##### SA_SHORTCIRCUIT_BAYES_SPAM
- **1** => will activate SpamAssassin short circuiting for bayes spam detection.

5
docs/content/config/security/rspamd.md
View File

@ -22,8 +22,9 @@ The following environment variables are related to Rspamd:
5. [`RSPAMD_HFILTER`](../environment.md#rspamd_hfilter)
6. [`RSPAMD_HFILTER_HOSTNAME_UNKNOWN_SCORE`](../environment.md#rspamd_hfilter_hostname_unknown_score)
7. [`RSPAMD_LEARN`](../environment.md#rspamd_learn)
8. [`MOVE_SPAM_TO_JUNK`][docs-spam-to-junk]
9. [`MARK_SPAM_AS_READ`](../environment.md#mark_spam_as_read)
8. [`SPAM_SUBJECT`](../environment.md#spam_subject)
9. [`MOVE_SPAM_TO_JUNK`][docs-spam-to-junk]
10. [`MARK_SPAM_AS_READ`](../environment.md#mark_spam_as_read)
With these variables, you can enable Rspamd itself, and you can enable / disable certain features related to Rspamd.

13
mailserver.env
View File

@ -130,6 +130,11 @@ ENABLE_IMAP=1
# **0** => Disabled
ENABLE_CLAMAV=0
# Add the value as a prefix to the mail subject when spam is detected.
# NOTE: By default spam is delivered to a junk folder, reducing the value of a subject prefix for spam.
# NOTE: When not using Docker Compose, other CRI may not support quote-wrapping the value here to preserve any trailing white-space.
SPAM_SUBJECT=
# Enables Rspamd
# **0** => Disabled
# 1 => Enabled
@ -378,7 +383,7 @@ ENABLE_SPAMASSASSIN=0
# Note: only has an effect if `ENABLE_SPAMASSASSIN=1`
ENABLE_SPAMASSASSIN_KAM=0
# deliver spam messages to the inbox (tagged using SA_SPAM_SUBJECT)
# deliver spam messages to the inbox (tagged using SPAM_SUBJECT)
SPAMASSASSIN_SPAM_TO_INBOX=1
# spam messages will be moved in the Junk folder (SPAMASSASSIN_SPAM_TO_INBOX=1 required)
@ -396,12 +401,6 @@ SA_TAG2=6.31
# triggers spam evasive actions
SA_KILL=10.0
# add tag to subject if spam detected
# The value `undef` opts-out of this feature. The value shown below is the default.
# NOTE: By default spam is delivered to a junk folder, reducing the value of adding a subject prefix.
# NOTE: If not using Docker Compose, other CRI may require the single quotes removed.
#SA_SPAM_SUBJECT='***SPAM*** '
# -----------------------------------------------
# --- Fetchmail Section -------------------------
# -----------------------------------------------

8
target/rspamd/local.d/actions.conf
View File

@ -6,7 +6,11 @@
# and to be able to explain the impact on the whole system.
greylist = 4;
add_header = 6;
rewrite_subject = 7;
reject = 11;
subject = "***SPAM*** %s"
# The value `null` disabled the action. A subject rewrite is handled by `SPAM_SUBJECT`:
# https://docker-mailserver.github.io/docker-mailserver/latest/config/environment/#spam_subject
#
# The reasoning for this can be found in
# https://github.com/docker-mailserver/docker-mailserver/issues/3804
rewrite_subject = null;

2
target/scripts/start-mailserver.sh
View File

@ -36,6 +36,7 @@ function _register_functions() {
_register_check_function '_check_improper_restart'
_register_check_function '_check_hostname'
_register_check_function '_check_log_level'
_register_check_function '_check_spam_prefix'
# ? >> Setup
@ -48,6 +49,7 @@ function _register_functions() {
_register_setup_function '_setup_dovecot_sieve'
_register_setup_function '_setup_dovecot_dhparam'
_register_setup_function '_setup_dovecot_quota'
_register_setup_function '_setup_spam_subject'
_register_setup_function '_setup_spam_to_junk'
_register_setup_function '_setup_spam_mark_as_read'
fi

8
target/scripts/startup/check-stack.sh
View File

@ -52,3 +52,11 @@ function _check_log_level() {
LOG_LEVEL="${DEFAULT_LOG_LEVEL}"
fi
}
function _check_spam_prefix() {
# This check should be independent of ENABLE_POP3 and ENABLE_IMAP
if [[ ${MOVE_SPAM_TO_JUNK} -eq 0 ]] \
&& [[ -z ${SPAM_SUBJECT} ]]; then
_log 'warn' "'MOVE_SPAM_TO_JUNK=0' and 'SPAM_SUBJECT' is empty - make sure this is intended: spam e-mails might not be immediately recognizable in this configuration"
fi
}

61
target/scripts/startup/setup.d/security/misc.sh
View File

@ -81,13 +81,9 @@ function __setup__security__spamassassin() {
# shellcheck disable=SC2016
sed -i -r 's|^\$sa_kill_level_deflt (.*);|\$sa_kill_level_deflt = '"${SA_KILL}"';|g' /etc/amavis/conf.d/20-debian_defaults
if [[ ${SA_SPAM_SUBJECT} == 'undef' ]]; then
# shellcheck disable=SC2016
sed -i -r 's|^\$sa_spam_subject_tag (.*);|\$sa_spam_subject_tag = undef;|g' /etc/amavis/conf.d/20-debian_defaults
else
# shellcheck disable=SC2016
sed -i -r 's|^\$sa_spam_subject_tag (.*);|\$sa_spam_subject_tag = '"'${SA_SPAM_SUBJECT}'"';|g' /etc/amavis/conf.d/20-debian_defaults
fi
# disable rewriting the subject as this is handles by _setup_spam_subject (which uses Dovecot Sieve)
# shellcheck disable=SC2016
sed -i -r 's|^\$sa_spam_subject_tag (.*);|\$sa_spam_subject_tag = undef;|g' /etc/amavis/conf.d/20-debian_defaults
# activate short circuits when SA BAYES is certain it has spam or ham.
if [[ ${SA_SHORTCIRCUIT_BAYES_SPAM} -eq 1 ]]; then
@ -245,6 +241,57 @@ function __setup__security__amavis() {
fi
}
# If `SPAM_SUBJECT` is not empty, we create a Sieve script that alters the `Subject`
# header, in order to prepend a user-defined string.
function _setup_spam_subject() {
if [[ -z ${SPAM_SUBJECT} ]]
then
_log 'debug' 'Spam subject is not set - no prefix will be added to spam e-mails'
else
_log 'debug' "Spam subject is set - the prefix '${SPAM_SUBJECT}' will be added to spam e-mails"
_log 'trace' "Enabling '+editheader' Sieve extension"
# check whether sieve_global_extensions is disabled (and enabled it if so)
sed -i -E 's|#(sieve_global_extensions.*)|\1|' /etc/dovecot/conf.d/90-sieve.conf
# then append the extension
sedfile -i -E 's|(sieve_global_extensions.*)|\1 +editheader|' /etc/dovecot/conf.d/90-sieve.conf
_log 'trace' "Adding global (before) Sieve script for subject rewrite"
# This directory contains Sieve scripts that are executed before user-defined Sieve
# scripts run.
local DOVECOT_SIEVE_GLOBAL_BEFORE_DIR='/usr/lib/dovecot/sieve-global/before'
local DOVECOT_SIEVE_FILE='spam_subject'
readonly DOVECOT_SIEVE_GLOBAL_BEFORE_DIR DOVECOT_SIEVE_FILE
mkdir -p "${DOVECOT_SIEVE_GLOBAL_BEFORE_DIR}"
# ref: https://superuser.com/a/1502589
cat >"${DOVECOT_SIEVE_GLOBAL_BEFORE_DIR}/${DOVECOT_SIEVE_FILE}.sieve" << EOF
require ["editheader","variables"];
if anyof (header :contains "X-Spam-Flag" "YES",
header :contains "X-Spam" "Yes")
{
# Match the entire subject ...
if header :matches "Subject" "*" {
# ... to get it in a match group that can then be stored in a variable:
set "subject" "\${1}";
}
# We can't "replace" a header, but we can delete (all instances of) it and
# re-add (a single instance of) it:
deleteheader "Subject";
# Note that the header is added ":last" (so it won't appear before possible
# "Received" headers).
addheader :last "Subject" "${SPAM_SUBJECT}\${subject}";
}
EOF
sievec "${DOVECOT_SIEVE_GLOBAL_BEFORE_DIR}/${DOVECOT_SIEVE_FILE}.sieve"
chown dovecot:root "${DOVECOT_SIEVE_GLOBAL_BEFORE_DIR}/${DOVECOT_SIEVE_FILE}."{sieve,svbin}
fi
}
# We can use Sieve to move spam emails to the "Junk" folder.
function _setup_spam_to_junk() {
if [[ ${MOVE_SPAM_TO_JUNK} -eq 1 ]]; then

8
target/scripts/startup/variables-stack.sh
View File

@ -25,6 +25,12 @@ function __environment_variables_backwards_compatibility() {
_log 'error' "The ENV for which LDAP host to connect to must include the URI scheme ('ldap://', 'ldaps://', 'ldapi://')"
fi
if [[ -n ${SA_SPAM_SUBJECT:-} ]]; then
_log 'warn' "'SA_SPAM_SUBJECT' has been renamed to 'SPAM_SUBJECT' - this warning will block startup on v15.0.0"
_log 'info' "Copying value of 'SA_SPAM_SUBJECT' into 'SPAM_SUBJECT' if 'SPAM_SUBJECT' has not been set explicitly"
SPAM_SUBJECT=${SPAM_SUBJECT:-${SA_SPAM_SUBJECT}}
fi
# TODO this can be uncommented in a PR handling the HOSTNAME/DOMAINNAME issue
# TODO see check_for_changes.sh and dns.sh
# if [[ -n ${OVERRIDE_HOSTNAME:-} ]]
@ -67,7 +73,7 @@ function __environment_variables_general_setup() {
VARS[RSPAMD_HFILTER_HOSTNAME_UNKNOWN_SCORE]="${RSPAMD_HFILTER_HOSTNAME_UNKNOWN_SCORE:=6}"
VARS[RSPAMD_LEARN]="${RSPAMD_LEARN:=0}"
VARS[SA_KILL]=${SA_KILL:="10.0"}
VARS[SA_SPAM_SUBJECT]=${SA_SPAM_SUBJECT:="***SPAM*** "}
VARS[SPAM_SUBJECT]=${SPAM_SUBJECT:=}
VARS[SA_TAG]=${SA_TAG:="2.0"}
VARS[SA_TAG2]=${SA_TAG2:="6.31"}
VARS[SPAMASSASSIN_SPAM_TO_INBOX]="${SPAMASSASSIN_SPAM_TO_INBOX:=1}"

16
test/helper/common.bash
View File

@ -417,5 +417,21 @@ function _nc_wrapper() {
_run_in_container_bash "nc ${NC_PARAMETERS} < /tmp/docker-mailserver-test/${FILE}"
}
# A simple wrapper for a test that checks whether a file exists.
#
# @param ${1} = the path to the file inside the container
function _file_exists_in_container() {
_run_in_container_bash "[[ -f ${1} ]]"
assert_success
}
# A simple wrapper for a test that checks whether a file does not exist.
#
# @param ${1} = the path to the file (that should not exists) inside the container
function _file_does_not_exist_in_container() {
_run_in_container_bash "[[ -f ${1} ]]"
assert_failure
}
# ? << Miscellaneous helper functions
# ! -------------------------------------------------------------------

101
test/tests/parallel/set1/spam_virus/amavis.bats
View File

@ -2,8 +2,9 @@ load "${REPOSITORY_ROOT}/test/helper/common"
load "${REPOSITORY_ROOT}/test/helper/setup"
BATS_TEST_NAME_PREFIX='[Amavis + SA] '
CONTAINER1_NAME='dms-test_amavis_enabled'
CONTAINER2_NAME='dms-test_amavis_disabled'
CONTAINER1_NAME='dms-test_amavis-enabled-default'
CONTAINER2_NAME='dms-test_amavis-enabled-custom'
CONTAINER3_NAME='dms-test_amavis-disabled'
function setup_file() {
export CONTAINER_NAME
@ -12,13 +13,25 @@ function setup_file() {
_init_with_defaults
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_AMAVIS=1
--env AMAVIS_LOGLEVEL=2
--env ENABLE_SPAMASSASSIN=1
)
_common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
CONTAINER_NAME=${CONTAINER2_NAME}
_init_with_defaults
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_AMAVIS=1
--env AMAVIS_LOGLEVEL=2
--env ENABLE_SPAMASSASSIN=1
--env SA_TAG=-5.0
--env SA_TAG2=2.0
--env SA_KILL=3.0
--env SPAM_SUBJECT='***SPAM*** '
)
_common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
CONTAINER_NAME=${CONTAINER3_NAME}
_init_with_defaults
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_AMAVIS=0
--env ENABLE_SPAMASSASSIN=0
@ -27,11 +40,37 @@ function setup_file() {
}
function teardown_file() {
docker rm -f "${CONTAINER1_NAME}" "${CONTAINER2_NAME}"
docker rm -f "${CONTAINER1_NAME}" "${CONTAINER2_NAME}" "${CONTAINER3_NAME}"
}
@test '(Amavis enabled) configuration should be correct' {
@test '(Amavis enabled - defaults) default Amavis config is correct' {
export CONTAINER_NAME=${CONTAINER1_NAME}
local AMAVIS_DEFAULTS_FILE='/etc/amavis/conf.d/20-debian_defaults'
_run_in_container grep 'sa_tag_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
assert_success
assert_output --partial 'sa_tag_level_deflt = 2.0;'
_run_in_container grep 'sa_tag2_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
assert_success
# shellcheck disable=SC2016
assert_output --partial '$sa_tag2_level_deflt = 6.31;'
_run_in_container grep 'sa_kill_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
assert_success
# shellcheck disable=SC2016
assert_output --partial '$sa_kill_level_deflt = 10.0;'
# This feature is handled by our SPAM_SUBJECT ENV through a sieve script instead.
# Thus the feature here should always be disabled via the 'undef' value.
_run_in_container grep 'sa_spam_subject_tag' "${AMAVIS_DEFAULTS_FILE}"
assert_success
# shellcheck disable=SC2016
assert_output --partial '$sa_spam_subject_tag = undef;'
}
@test '(Amavis enabled - custom) configuration should be correct' {
export CONTAINER_NAME=${CONTAINER2_NAME}
_run_in_container postconf -h content_filter
assert_success
@ -41,47 +80,47 @@ function teardown_file() {
_run_in_container grep -F '127.0.0.1:10025' /etc/postfix/master.cf
assert_success
_run_in_container_bash '[[ -f /etc/cron.d/amavisd-new.disabled ]]'
assert_failure
_run_in_container_bash '[[ -f /etc/cron.d/amavisd-new ]]'
assert_success
_file_does_not_exist_in_container /etc/cron.d/amavisd-new.disabled
_file_exists_in_container /etc/cron.d/amavisd-new
}
@test '(Amavis enabled) SA integration should be active' {
export CONTAINER_NAME=${CONTAINER1_NAME}
@test '(Amavis enabled - custom) SA integration should be active' {
export CONTAINER_NAME=${CONTAINER2_NAME}
# give Amavis just a bit of time to print out its full debug log
run _repeat_in_container_until_success_or_timeout 20 "${CONTAINER_NAME}" grep 'SpamControl: init_pre_fork on SpamAssassin done' /var/log/mail/mail.log
assert_success
}
@test '(Amavis enabled) SA ENV should update Amavis config' {
export CONTAINER_NAME=${CONTAINER1_NAME}
@test '(Amavis enabled - custom) ENV should update Amavis config' {
export CONTAINER_NAME=${CONTAINER2_NAME}
local AMAVIS_DEFAULTS_FILE='/etc/amavis/conf.d/20-debian_defaults'
# shellcheck disable=SC2016
_run_in_container grep '\$sa_tag_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
assert_success
assert_output --partial '= 2.0'
# shellcheck disable=SC2016
_run_in_container grep '\$sa_tag2_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
_run_in_container grep 'sa_tag_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
assert_success
assert_output --partial '= 6.31'
# shellcheck disable=SC2016
assert_output --partial '$sa_tag_level_deflt = -5.0;'
# shellcheck disable=SC2016
_run_in_container grep '\$sa_kill_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
_run_in_container grep 'sa_tag2_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
assert_success
assert_output --partial '= 10.0'
# shellcheck disable=SC2016
assert_output --partial '$sa_tag2_level_deflt = 2.0;'
# shellcheck disable=SC2016
_run_in_container grep '\$sa_spam_subject_tag' "${AMAVIS_DEFAULTS_FILE}"
_run_in_container grep 'sa_kill_level_deflt' "${AMAVIS_DEFAULTS_FILE}"
assert_success
assert_output --partial "= '***SPAM*** ';"
# shellcheck disable=SC2016
assert_output --partial '$sa_kill_level_deflt = 3.0;'
# This feature is handled by our SPAM_SUBJECT ENV through a sieve script instead.
# Thus the feature here should always be disabled via the 'undef' value.
_run_in_container grep 'sa_spam_subject_tag' "${AMAVIS_DEFAULTS_FILE}"
assert_success
# shellcheck disable=SC2016
assert_output --partial '$sa_spam_subject_tag = undef;'
}
@test '(Amavis disabled) configuration should be correct' {
export CONTAINER_NAME=${CONTAINER2_NAME}
export CONTAINER_NAME=${CONTAINER3_NAME}
_run_in_container postconf -h content_filter
assert_success
@ -91,8 +130,6 @@ function teardown_file() {
_run_in_container grep -F '127.0.0.1:10025' /etc/postfix/master.cf
assert_failure
_run_in_container_bash '[[ -f /etc/cron.d/amavisd-new.disabled ]]'
assert_success
_run_in_container_bash '[[ -f /etc/cron.d/amavisd-new ]]'
assert_failure
_file_exists_in_container /etc/cron.d/amavisd-new.disabled
_file_does_not_exist_in_container /etc/cron.d/amavisd-new
}

9
test/tests/parallel/set1/spam_virus/rspamd_dkim.bats
View File

@ -62,8 +62,7 @@ function teardown_file() { _default_teardown ; }
assert_output --partial "Supplying a default configuration (to '${SIGNING_CONF_FILE}')"
refute_output --partial "'${SIGNING_CONF_FILE}' exists, not supplying a default"
assert_output --partial "Finished DKIM key creation"
_run_in_container_bash "[[ -f ${SIGNING_CONF_FILE} ]]"
assert_success
_file_exists_in_container "${SIGNING_CONF_FILE}"
_exec_in_container_bash "echo 'blabla' >${SIGNING_CONF_FILE}"
local INITIAL_SHA512_SUM=$(_exec_in_container sha512sum "${SIGNING_CONF_FILE}")
@ -87,8 +86,7 @@ function teardown_file() { _default_teardown ; }
assert_success
_count_files_in_directory_in_container /tmp/docker-mailserver/rspamd/dkim/ 3
_run_in_container_bash "[[ -f ${SIGNING_CONF_FILE} ]]"
assert_success
_file_exists_in_container "${SIGNING_CONF_FILE}"
__check_path_in_signing_config "/tmp/docker-mailserver/rspamd/dkim/rsa-2048-mail-${DOMAIN_NAME}.private.txt"
__check_selector_in_signing_config 'mail'
@ -241,8 +239,7 @@ function __check_rsa_keys() {
function __check_key_files_are_present() {
local BASE_FILE_NAME="${1:?Base file name must be supplied to __check_key_files_are_present}"
for FILE in ${BASE_FILE_NAME}.{public.txt,public.dns.txt,private.txt}; do
_run_in_container_bash "[[ -f ${FILE} ]]"
assert_success
_file_exists_in_container "${FILE}"
done
}

89
test/tests/parallel/set1/spam_virus/rspamd_full.bats
View File

@ -29,6 +29,7 @@ function setup_file() {
--env RSPAMD_GREYLISTING=1
--env RSPAMD_HFILTER=1
--env RSPAMD_HFILTER_HOSTNAME_UNKNOWN_SCORE=7
--env SPAM_SUBJECT='[POTENTIAL SPAM] '
)
cp -r "${TEST_TMP_CONFIG}"/rspamd_full/* "${TEST_TMP_CONFIG}/"
@ -43,7 +44,7 @@ function setup_file() {
_wait_for_service postfix
_wait_for_smtp_port_in_container
# We will send 4 emails:
# We will send 5 emails:
# 1. The first one should pass just fine
_send_email_with_msgid 'rspamd-test-email-pass'
# 2. The second one should be rejected (Rspamd-specific GTUBE pattern for rejection)
@ -56,6 +57,9 @@ function setup_file() {
# ref: https://rspamd.com/doc/gtube_patterns.html
_send_email_with_msgid 'rspamd-test-email-header' \
--body "YJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X"
# 5. The fifth one will have its subject rewritten, but now spam header is applied.
_send_email_with_msgid 'rspamd-test-email-rewrite_subject' \
--body "ZJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X"
_run_in_container cat /var/log/mail.log
assert_success
@ -73,11 +77,18 @@ function teardown_file() { _default_teardown ; }
assert_output 'rspamd_milter = inet:localhost:11332'
}
@test 'Rspamd base configuration is correct' {
_run_in_container rspamadm configdump actions
assert_success
assert_line 'greylist = 4;'
assert_line 'reject = 11;'
assert_line 'add_header = 6;'
refute_line --regexp 'rewrite_subject = [0-9]+;'
}
@test "contents of '/etc/rspamd/override.d/' are copied" {
local OVERRIDE_D='/etc/rspamd/override.d'
_run_in_container_bash "[[ -f ${OVERRIDE_D}/testmodule_complicated.conf ]]"
assert_success
_file_exists_in_container "${OVERRIDE_D}/testmodule_complicated.conf"
}
@test 'startup log shows all features as properly enabled' {
@ -89,6 +100,7 @@ function teardown_file() { _default_teardown ; }
assert_line --partial 'Enabling greylisting'
assert_line --partial 'Hfilter (group) module is enabled'
assert_line --partial "Adjusting score for 'HFILTER_HOSTNAME_UNKNOWN' in Hfilter group module to"
assert_line --partial "Spam subject is set - the prefix '[POTENTIAL SPAM] ' will be added to spam e-mails"
assert_line --partial "Found file '/tmp/docker-mailserver/rspamd/custom-commands.conf' - parsing and applying it"
}
@ -114,7 +126,7 @@ function teardown_file() { _default_teardown ; }
_print_mail_log_for_msgid 'rspamd-test-email-pass'
assert_output --partial "stored mail into mailbox 'INBOX'"
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 1
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 2
}
@test 'detects and rejects spam' {
@ -129,7 +141,7 @@ function teardown_file() { _default_teardown ; }
refute_output --partial "stored mail into mailbox 'INBOX'"
assert_failure
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 1
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 2
}
@test 'detects and rejects virus' {
@ -144,14 +156,13 @@ function teardown_file() { _default_teardown ; }
refute_output --partial "stored mail into mailbox 'INBOX'"
assert_failure
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 1
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 2
}
@test 'custom commands work correctly' {
# check `testmodule1` which should be disabled
local MODULE_PATH='/etc/rspamd/override.d/testmodule1.conf'
_run_in_container_bash "[[ -f ${MODULE_PATH} ]]"
assert_success
_file_exists_in_container "${MODULE_PATH}"
_run_in_container grep -F '# documentation: https://rspamd.com/doc/modules/testmodule1.html' "${MODULE_PATH}"
assert_success
_run_in_container grep -F 'enabled = false;' "${MODULE_PATH}"
@ -161,8 +172,7 @@ function teardown_file() { _default_teardown ; }
# check `testmodule2` which should be enabled and it should have extra options set
MODULE_PATH='/etc/rspamd/override.d/testmodule2.conf'
_run_in_container_bash "[[ -f ${MODULE_PATH} ]]"
assert_success
_file_exists_in_container "${MODULE_PATH}"
_run_in_container grep -F '# documentation: https://rspamd.com/doc/modules/testmodule2.html' "${MODULE_PATH}"
assert_success
_run_in_container grep -F 'enabled = true;' "${MODULE_PATH}"
@ -181,8 +191,7 @@ function teardown_file() { _default_teardown ; }
# check whether adding a single line writes the line properly in `testmodule4.something`
MODULE_PATH='/etc/rspamd/override.d/testmodule4.something'
_run_in_container_bash "[[ -f ${MODULE_PATH} ]]"
assert_success
_file_exists_in_container "${MODULE_PATH}"
# shellcheck disable=SC2016
_run_in_container grep -F 'some very long line with "weird $charact"ers' "${MODULE_PATH}"
assert_success
@ -193,37 +202,31 @@ function teardown_file() { _default_teardown ; }
# check whether spaces in front of options are handles properly in `testmodule_complicated`
MODULE_PATH='/etc/rspamd/override.d/testmodule_complicated.conf'
_run_in_container_bash "[[ -f ${MODULE_PATH} ]]"
assert_success
_file_exists_in_container "${MODULE_PATH}"
_run_in_container grep -F ' anOption = anotherValue;' "${MODULE_PATH}"
# check whether controller option was written properly
MODULE_PATH='/etc/rspamd/override.d/worker-controller.inc'
_run_in_container_bash "[[ -f ${MODULE_PATH} ]]"
assert_success
_file_exists_in_container "${MODULE_PATH}"
_run_in_container grep -F 'someOption = someValue42;' "${MODULE_PATH}"
assert_success
# check whether controller option was written properly
MODULE_PATH='/etc/rspamd/override.d/worker-proxy.inc'
_run_in_container_bash "[[ -f ${MODULE_PATH} ]]"
assert_success
_file_exists_in_container "${MODULE_PATH}"
_run_in_container grep -F 'abcdefg71 = RAAAANdooM;' "${MODULE_PATH}"
assert_success
# check whether basic options are written properly
MODULE_PATH='/etc/rspamd/override.d/options.inc'
_run_in_container_bash "[[ -f ${MODULE_PATH} ]]"
assert_success
_file_exists_in_container "${MODULE_PATH}"
_run_in_container grep -F 'OhMy = "PraiseBeLinters !";' "${MODULE_PATH}"
assert_success
}
@test 'MOVE_SPAM_TO_JUNK works for Rspamd' {
_run_in_container_bash '[[ -f /usr/lib/dovecot/sieve-global/after/spam_to_junk.sieve ]]'
assert_success
_run_in_container_bash '[[ -f /usr/lib/dovecot/sieve-global/after/spam_to_junk.svbin ]]'
assert_success
_file_exists_in_container /usr/lib/dovecot/sieve-global/after/spam_to_junk.sieve
_file_exists_in_container /usr/lib/dovecot/sieve-global/after/spam_to_junk.svbin
_service_log_should_contain_string 'rspamd' 'S (add header)'
_service_log_should_contain_string 'rspamd' 'add header "Gtube pattern"'
@ -231,14 +234,38 @@ function teardown_file() { _default_teardown ; }
_print_mail_log_for_msgid 'rspamd-test-email-header'
assert_output --partial "fileinto action: stored mail into mailbox 'Junk'"
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 1
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 2
_count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/.Junk/new/ 1
}
@test 'Rewriting subject works when enforcing it via GTUBE' {
_service_log_should_contain_string 'rspamd' 'S (rewrite subject)'
_service_log_should_contain_string 'rspamd' 'rewrite subject "Gtube pattern"'
_print_mail_log_for_msgid 'rspamd-test-email-rewrite_subject'
assert_output --partial "stored mail into mailbox 'INBOX'"
# check that the inbox contains the subject-rewritten e-mail
_run_in_container_bash "grep --fixed-strings 'Subject: *** SPAM ***' /var/mail/localhost.localdomain/user1/new/*"
assert_success
# check that the inbox contains the normal e-mail (that passes just fine)
_run_in_container_bash "grep --fixed-strings 'Subject: test' /var/mail/localhost.localdomain/user1/new/*"
assert_success
}
@test 'SPAM_SUBJECT works' {
_file_exists_in_container /usr/lib/dovecot/sieve-global/before/spam_subject.sieve
_file_exists_in_container /usr/lib/dovecot/sieve-global/before/spam_subject.svbin
# we only have one e-mail in the junk folder, hence using '*' is fine
_run_in_container_bash "grep --fixed-strings 'Subject: [POTENTIAL SPAM]' /var/mail/localhost.localdomain/user1/.Junk/new/*"
assert_success
}
@test 'RSPAMD_LEARN works' {
for FILE in learn-{ham,spam}.{sieve,svbin}; do
_run_in_container_bash "[[ -f /usr/lib/dovecot/sieve-pipe/${FILE} ]]"
assert_success
_file_exists_in_container "/usr/lib/dovecot/sieve-pipe/${FILE}"
done
_run_in_container grep 'mail_plugins.*imap_sieve' /etc/dovecot/conf.d/20-imap.conf
@ -305,8 +332,7 @@ function teardown_file() { _default_teardown ; }
@test 'hfilter group module is configured correctly' {
local MODULE_FILE='/etc/rspamd/local.d/hfilter_group.conf'
_run_in_container_bash "[[ -f ${MODULE_FILE} ]]"
assert_success
_file_exists_in_container "${MODULE_FILE}"
_run_in_container grep '__TAG__HFILTER_HOSTNAME_UNKNOWN' "${MODULE_FILE}"
assert_success
@ -315,8 +341,7 @@ function teardown_file() { _default_teardown ; }
@test 'checks on authenticated users are disabled' {
local MODULE_FILE='/etc/rspamd/local.d/settings.conf'
_run_in_container_bash "[[ -f ${MODULE_FILE} ]]"
assert_success
_file_exists_in_container "${MODULE_FILE}"
_run_in_container grep -E -A 6 'authenticated \{' "${MODULE_FILE}"
assert_success

9
test/tests/parallel/set1/spam_virus/rspamd_partly.bats
View File

@ -55,6 +55,7 @@ function teardown_file() { _default_teardown ; }
assert_line --partial 'Intelligent learning of spam and ham is disabled'
assert_line --partial 'Greylisting is disabled'
assert_line --partial 'Disabling Hfilter (group) module'
assert_line --partial 'Spam subject is not set'
}
@test 'antivirus maximum size was not adjusted unnecessarily' {
@ -64,8 +65,7 @@ function teardown_file() { _default_teardown ; }
@test 'learning is properly disabled' {
for FILE in learn-{ham,spam}.{sieve,svbin}; do
_run_in_container_bash "[[ -f /usr/lib/dovecot/sieve-pipe/${FILE} ]]"
assert_failure
_file_does_not_exist_in_container "/usr/lib/dovecot/sieve-pipe/${FILE}"
done
_run_in_container grep 'mail_plugins.*imap_sieve' /etc/dovecot/conf.d/20-imap.conf
@ -83,14 +83,13 @@ function teardown_file() { _default_teardown ; }
}
@test 'hfilter group module configuration is deleted' {
_run_in_container_bash '[[ -f /etc/rspamd/local.d/hfilter_group.conf ]]'
_file_does_not_exist_in_container /etc/rspamd/local.d/hfilter_group.conf
assert_failure
}
@test 'checks on authenticated users are enabled' {
local MODULE_FILE='/etc/rspamd/local.d/settings.conf'
_run_in_container_bash "[[ -f ${MODULE_FILE} ]]"
assert_success
_file_exists_in_container "${MODULE_FILE}"
_run_in_container grep -E 'authenticated \{' "${MODULE_FILE}"
assert_failure

6
test/tests/parallel/set1/spam_virus/spam_junk_folder.bats
View File

@ -34,7 +34,7 @@ function teardown() { _default_teardown ; }
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_AMAVIS=1
--env ENABLE_SPAMASSASSIN=1
--env SA_SPAM_SUBJECT="SPAM: "
--env SPAM_SUBJECT="SPAM: "
--env SPAMASSASSIN_SPAM_TO_INBOX=1
--env MOVE_SPAM_TO_JUNK=0
--env PERMIT_DOCKER=container
@ -55,7 +55,7 @@ function teardown() { _default_teardown ; }
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_AMAVIS=1
--env ENABLE_SPAMASSASSIN=1
--env SA_SPAM_SUBJECT="SPAM: "
--env SPAM_SUBJECT="SPAM: "
--env SPAMASSASSIN_SPAM_TO_INBOX=1
--env MOVE_SPAM_TO_JUNK=1
--env PERMIT_DOCKER=container
@ -77,7 +77,7 @@ function teardown() { _default_teardown ; }
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_AMAVIS=1
--env ENABLE_SPAMASSASSIN=1
--env SA_SPAM_SUBJECT="SPAM: "
--env SPAM_SUBJECT="SPAM: "
--env SPAMASSASSIN_SPAM_TO_INBOX=1
--env MARK_SPAM_AS_READ=1
--env PERMIT_DOCKER=container

61
test/tests/parallel/set1/spam_virus/undef_spam_subject.bats
View File

@ -1,61 +0,0 @@
load "${REPOSITORY_ROOT}/test/helper/setup"
load "${REPOSITORY_ROOT}/test/helper/common"
BATS_TEST_NAME_PREFIX='[Spam] (undefined subject) '
CONTAINER1_NAME='dms-test_spam-undef-subject_1'
CONTAINER2_NAME='dms-test_spam-undef-subject_2'
CONTAINER_NAME=${CONTAINER2_NAME}
function teardown() { _default_teardown ; }
@test "'SA_SPAM_SUBJECT=undef' should update Amavis config" {
export CONTAINER_NAME=${CONTAINER1_NAME}
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_AMAVIS=1
--env ENABLE_SPAMASSASSIN=1
--env SA_SPAM_SUBJECT='undef'
)
_init_with_defaults
_common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
_run_in_container_bash "grep '\$sa_spam_subject_tag' /etc/amavis/conf.d/20-debian_defaults | grep '= undef'"
assert_success
}
# TODO: Unclear why some of these ENV are relevant for the test?
@test "Docker env variables are set correctly (custom)" {
export CONTAINER_NAME=${CONTAINER2_NAME}
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_CLAMAV=1
--env SPOOF_PROTECTION=1
--env ENABLE_SPAMASSASSIN=1
--env REPORT_RECIPIENT=user1@localhost.localdomain
--env REPORT_SENDER=report1@mail.my-domain.com
--env SA_TAG=-5.0
--env SA_TAG2=2.0
--env SA_KILL=3.0
--env SA_SPAM_SUBJECT="SPAM: "
--env VIRUSMAILS_DELETE_DELAY=7
--env ENABLE_SRS=1
--env ENABLE_MANAGESIEVE=1
--env PERMIT_DOCKER=host
# NOTE: ulimit required for `ENABLE_SRS=1` until running a newer `postsrsd`
--ulimit "nofile=$(ulimit -Sn):$(ulimit -Hn)"
)
_init_with_defaults
_common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
_run_in_container_bash "grep '\$sa_tag_level_deflt' /etc/amavis/conf.d/20-debian_defaults | grep '= -5.0'"
assert_success
_run_in_container_bash "grep '\$sa_tag2_level_deflt' /etc/amavis/conf.d/20-debian_defaults | grep '= 2.0'"
assert_success
_run_in_container_bash "grep '\$sa_kill_level_deflt' /etc/amavis/conf.d/20-debian_defaults | grep '= 3.0'"
assert_success
_run_in_container_bash "grep '\$sa_spam_subject_tag' /etc/amavis/conf.d/20-debian_defaults | grep '= .SPAM: .'"
assert_success
}