1
0
mirror of https://github.com/tomav/docker-mailserver.git synced 2024-06-28 12:11:32 +02:00
Commit Graph

197 Commits

Author SHA1 Message Date
Georg Lauterbach
7453bc096b
Dovecot: make home dir distinct from mail dir (#3335)
* add new home dir for Dovecot

I tried changing the mail dir, but this is a _very_ disruptive change,
so I took approach 3 on
<https://doc.dovecot.org/configuration_manual/home_directories_for_virtual_users/>,
whereby the home directory is now inside the mail directory.

The MDBOX/SDBOX formats are not touched by this change. The change
itself could be considered breaking though.

* adjust Sieve tests accordingly

* Update target/dovecot/10-mail.conf

* Update target/dovecot/auth-passwdfile.inc

---------

Co-authored-by: Casper <casperklein@users.noreply.github.com>
2023-05-15 20:10:29 +02:00
Casper
a72adc2731
Fix typos (#3344) 2023-05-15 19:11:36 +02:00
Andreas Perhab
ec330a35a1
ClamAV: add a warning for the internal message size limit (#3341) 2023-05-15 15:46:13 +02:00
Georg Lauterbach
9fd00bd6ad
Rspamd: adjust learning of ham (#3334)
* adjust learning of ham

See #3333

When moving a mail from the Junk folder to the Trash folder, the mail
previously classified as ham due to the wildcard match of `*`. Because
the syntax does not allow for negation, we can only change the behavior
in a way that mails are learned as ham when they are moved into `INBOX`
from `Junk`. This is reasonable though.

* adjust tests accordingly

* adjust docs accordingly
2023-05-13 13:59:16 +02:00
Georg Lauterbach
78b7f0cbea
scripts: improve CLAMAV_MESSAGE_SIZE_LIMIT usage (#3332)
* add sanity check for Clam size & adjusted MaxScanSize

The second part is of special importance! See
<https://askubuntu.com/a/1448525>, which explains that the maximum scan
size is important as well. We previously just set the maximum file size,
which actually is pretty insecure as we silently not scan mile bigger
than `MaxScanSize`. This is corrected now.

* add SlamAV size configuration to Rspamd
2023-05-12 16:04:41 +02:00
ghnp5
823ef33a92
fix: typo about OpenDMARC (#3330)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-05-11 18:10:51 +02:00
Georg Lauterbach
e4274ef113
docs: improve Rspamd docs about DKIM signing of multiple domains (#3329)
* improve Rspamd docs

See #3326 & #3328

* improve warning message

See #3328
2023-05-11 18:08:54 +02:00
Georg Lauterbach
bba72daedf
scripts: add DKIM helper script for Rspamd (#3286)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2023-05-03 08:30:49 +02:00
Georg Lauterbach
449d53fc3f
docs/scripts: remove WIP warnings for Rspamd (#3283) 2023-04-23 15:14:36 +02:00
Georg Lauterbach
cd1721334c
scripts: Rspamd stabilization pt. 2 (#3282)
* move modules adjustment file to new location

Because we link `/tmp/docker-mailserver/rspamd/override.d` to
`/etc/rspamd/override.d`, I think it makes sense to move the modules
adjustment file into `/tmp/docker-mailserver/rspamd/` as well.

I write the code in a way that it is backwards compatible for now, so
this is NOT a breaking change.

* minor improvement to `__rspamd__handle_user_modules_adjustments`

The expansion of `ARGUMENT3` is now done in a way that only adds the
whitespace in case the variable is set and not null.

* move test file structure to respect latest changes

Because we're now linking `rspamd/override.d/`, we can simplify the
setup a bit. But this requires a change in directory structure.

The current Rspamd test will be renamed to `rspamd_full.bats`, because I
plan on adding more tests in different files for different feature sets.
This is done to make this feature well-tested!

* improved and added tests to Rspamd-full

FYI: The line

```bats
_run_in_container grep 'sieve_global_extensions.*\+vnd\.dovecot\.pipe'
"${SIEVE_CONFIG_FILE}"
```

was testing a condition that should actually not be met, but when I
started working on this feature, I thought this was the correct
configuration. Adding the `assert_success` statements revealed this
wrong line.

I also added tests to check whether `override.d` is linked correctly.

* renamed: `rspamd.bats` => `rspamd_full.bats`

* added new tests for incomplete Rspamd feature set

We now test that warnings are emitted & features are disabled correctly.

* update documentation
2023-04-23 14:02:56 +02:00
Georg Lauterbach
638975922e
scripts: Rspamd stabilization pt. 1 (#3261)
* added checks whether OpenDKIM/OpenDMARC/policyd-spf are enabled
* added functions to check if VAR is 0/0 or an int

and also added tests.

I also adjusted the test file to not run in a container, because there
is no need. This also decreases test time, which, in turn, increases
maintainers' happiness.

* added more checks to Rspamd setup

I added the helpers from the previous commit to the Rspamd setup to make
the whole setup more robust, and indicate to the user that an ENV
variable's value is incorrect.

While we did not issues for this in the past, I believe it to be
worthwhile for the future.

* added canonical directory for users to place files in

This dir is canonical with DMS's optional configuration dirs, as it
lives in well-known volume mounts. Hence, users will not need to adjust
`/etc/rspamd/override.d` manually anymore, or mount a volume to this
place.

The docs explain this now, but the DKIM page needs a slight update on
this too I guess. I will follow-up here.

* misc minor improvements
* use variables for common directories
2023-04-23 12:22:54 +02:00
Georg Lauterbach
88cd244e47
scripts: misc improvements (#3281)
* corrected typo
* corrected indentation
2023-04-23 12:16:53 +02:00
Andreas Perhab
2b330fdc49
scripts: remove superfluous EOF in dmarc_dkim_spf.sh (#3266)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-04-20 09:52:07 +02:00
Casper
ea07bcdb4c
scripts: improve shutdown function by making PANIC_STRATEGY obsolete (#3265) 2023-04-18 23:38:46 +02:00
James
a735dddc52
scripts: fix setting SRS_EXCLUDE_DOMAINS during startup (#3271) 2023-04-18 17:07:08 +02:00
Andreas Perhab
2f33f44f4a
postfix.sh: add missing -E for extended regexes in smtpd_sender_restrictions (#3272) 2023-04-18 11:08:19 +02:00
Georg Lauterbach
c8dfb9ac76
Posfix: add option to re-enable reject_unknown_client_hostname after #3248 (#3255) 2023-04-16 14:09:00 +02:00
Georg Lauterbach
03772f612a
scripts: get all policyd-spf setup in one place (#3263) 2023-04-15 00:40:42 +02:00
Georg Lauterbach
9a284150b2
Rspamd: replace reject_unknown_client_hostname with Rspamd HFILTER_HOSTNAME_UNKNOWN and make it configurable (#3248) 2023-04-11 18:51:23 +02:00
Georg Lauterbach
806d3efef9
Rspamd: add greylisting option & code refactoring (#3206) 2023-04-11 09:16:57 +02:00
Georg Lauterbach
9ee33a81b7
scripts: make policyd-spf configurable (#3246) 2023-04-11 08:52:43 +02:00
Casper
f4fe5bf527
Update SA_KILL values; follow up to #3058 (#3204) 2023-03-27 01:59:43 +02:00
Casper
6fa06f4986
Fix: only chmod when there are files (#3203) 2023-03-26 20:30:34 +08:00
Georg Lauterbach
e58dd1b95b
Rspamd: more features (#3159) 2023-03-18 23:32:48 +08:00
Casper
2da3e1b022
fix: SRS setup (#3158) 2023-03-12 12:10:45 +01:00
Georg Lauterbach
dab70709d9
scripts: improve panic helpers (#3155) 2023-03-06 10:06:50 +01:00
Georg Lauterbach
b5fc40eb7a
fix regression introduced in #3153 (#3157) 2023-03-05 22:23:25 +01:00
Casper
a8f6fa6181
DRY: Replace path with variable in mail_state.sh (#3153) 2023-03-05 16:59:01 +01:00
Georg Lauterbach
4e9ffbf224
fix(Postfix): special bits for maildrop and public directory (#3149)
* fixed special bits for maildrop and public dir

After changing the group, special bits are lost, but they should be set for the directories `/var/spool/postfix/{maildrop,public}`, otherwise you see the following error:

```
postfix/postdrop[17400]: warning: mail_queue_enter: create file maildrop/729504.17400: Permission denied
```

* fix: Match octal permissions originally provided

Officially Postfix source seems to imply:
- `730` for `maildrop/` (_but has mentioned a sticky bit in the past, set-gid bit only for the postdrop binary involved_)
- `710` for `public/`

Both folders are assigned the same group that `postdrop` belongs to which has the SGID permission for it's executable. SGID special bit on`public/` doesn't seem necessary, but left as-is to match the default from Debian.

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-03-05 20:57:40 +13:00
Georg Lauterbach
8ec5dbec74
scripts: touchups for v12.0.0 (#3144) 2023-03-04 10:57:42 +01:00
Georg Lauterbach
5e9849d94f
rspamd: rename ENABLE_REDIS & add persistence for Redis (#3143) 2023-03-04 10:45:43 +01:00
Georg Lauterbach
f0edcc28d9
config: ensure SASL socket file is not inside a volume mount (#3131) 2023-03-03 23:42:55 +01:00
Georg Lauterbach
5ec6845c96
config/ENV: improve Postfix config for spoof protection (#3127) 2023-03-03 15:55:32 +01:00
Georg Lauterbach
9e2f96448a
scripts: remove PostSRSD wrapper (#3128)
* remove PostSRSD wrapper

The setup is now completely done during _actual_ setup phase. The
wrapper did not even catch signals (SIGINT, etc.), which I think is
strange.

I also added all the ENVs the wrapper relied on (which previously could
have been unset) to the variables script.

* forgot adjusting the `Dockerfile`
2023-03-02 17:44:01 +01:00
Georg Lauterbach
f3edcf9cd5
issue warning in case of improper restart (#3129) 2023-03-01 16:33:54 +01:00
Georg Lauterbach
0949f16344
scripts/ENV: make disabling Redis possible (#3132)
* make disabling Redis possible

* add documentation

* Apply suggestions from code review

* Update docs/content/config/environment.md

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-03-01 16:24:59 +01:00
Georg Lauterbach
dfc2b39308
scripts: housekeeping & cleanup setup (2/2) (#3123) 2023-02-28 10:25:23 +01:00
Georg Lauterbach
9ead9a54ac
scripts: follow up of #3115 (feedback) (#3124) 2023-02-27 23:37:35 +01:00
Georg Lauterbach
4b04c3e31c
scripts: housekeeping & cleanup setup (1/2) (#3121) 2023-02-27 20:21:45 +01:00
Georg Lauterbach
f35b60042f
scripts: split setup-stack.sh (#3115) 2023-02-26 11:42:14 +01:00
Georg Lauterbach
bee9e3627d
rspamd: add feature for adjusting options with a file parsed by DMS (#3059)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-02-19 12:36:43 +01:00
Georg Lauterbach
40e10d755d
setup: improve Amavis setup routine (#3079)
* improve Amavis setup routine

see https://github.com/docker-mailserver/docker-mailserver/pull/3046#issuecomment-1423718811
2023-02-18 19:22:20 +01:00
Georg Lauterbach
632012aead
add information so Envelope From is properly set (#3081) 2023-02-18 16:21:51 +01:00
Casper
26861ddf92
fix order (#3082) 2023-02-13 00:05:52 +01:00
Georg Lauterbach
ac1df91181
chore(Postfix): disable DNSBLs (#3069) 2023-02-11 00:24:09 +01:00
Casper
021a4a3c09
Fix: Make logrotate state persistant (#3077)
Fixes https://github.com/docker-mailserver/docker-mailserver/issues/3075
2023-02-09 22:34:26 +00:00
Brennan Kinney
88767f7cc8
tests(refactor): open_dkim.bats (#3060)
* tests(refactor): Make test cases for opendkim keysizes DRY

- These all do roughly the same logic that can be split into two separate methods.
- `_should_generate_dkim_key()` covers a bit more logic as it can be leveraged to handle other test cases that also perform the same logic.
- The `config/opendkim/` doesn't seem necessary for tests. Only the first few test cases here are testing against it, so we can conditionally make that available. `process_check_restart.bats` also depended on it to run OpenDKIM successfully, but this was due to the `setup-stack.sh` config defaults failing to find an "empty" file forcing `supervisord` to constantly restart the process..
- With this, there we inverse the default opendkim config, so we don't have to mount unique / empty subfolders for each test case, followed by copying over the two extra configs.

* tests(refactor): DRY up more test cases

All the remaining test cases but the last one were refactored here for a clean commit diff. The last test case will be refactored in the following commit.

Plenty of repeated logic spread across these test cases, now condensed into shared methods.

* tests(refactor): Make final test case DRY

* chore: Migrate to new testing helpers

* chore: Revise test case descriptions

* tests(refactor): Improve and simplify assertions

* tests(refactor): Use common container setup instead of `docker run`

- As the majority of test cases are only running `open-dkim` helper, we don't actually have to wait for a full container setup. So an alternative container start is called.
- Also improves assertions a bit more instead of just counting lines.
- Some test cases don't bind mount all of `/tmp/docker-mailserver` contents, thus don't raise permission errors on subsequent test runs.
- Instead of `rm -f` on some config files, have opted to mount them read-only instead, or alternatively mount an anonymous empty volume instead.
- Collapsed the first three test cases into one, thus no `setup_file()` necessary.
- Shift the `_wait_for_finished_setup_in_container()` method into `_common_container_setup()` instead since nothing else is using `_common_container_start()` yet, this allows for avoiding the wait.

* tests(refactor): Collapse dkim key size test cases into single test case

This makes these tests a bit more DRY, and enhances the raised quality issue with these tests. Now not only is the domain checked in the generated DNS dkim record, but we also verify the key size is corrected in the public and private keys via openssl.

* chore: Revise container names

* chore: Swap order of test case 1 and 2

* tests(refactor): Assert generated log output

- `__should_have_tables_trustedhosts_for_domain` shifted in each test case to just after generating the domains keys.
- Asserts `open-dkim` logs instead of just counting them.
- Added checks for domains that should not be present in a test case.
- Additional coverage and notes about the alias from vhost `@localdomain.com`
- Single assert statement with switch statement as all are using common args.

* chore: Minor changes

* tests(refactor):  Share `find` logic in helpers and tests

* tests(fix): Listing file content does not need to match line order

The order printed from local system vs CI differed causing the CI to fail. The order of lines is irrelevant so `--index` is not required.

Additionally correct the prefix of the called method to be only one `_` now that it's a `common.bash` helper method.

* chore: Collapse custom DKIM selector test into custom DKIM domain test

These cover the same test logic for the most part, the first domain could also be testing the custom selector.

`special_use_folders.bats` + `mailbox_format_dbox` can assert lines instead, removing the need for `--partial`.

* Apply suggestions from code review

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>

* chore: Split switch statement method into wrapper methods

---------

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-02-10 00:18:06 +13:00
Georg Lauterbach
00b1d88ed7
removal: configomat (submodule) (#3045) 2023-02-05 12:39:05 +01:00
Georg Lauterbach
774a340d76
chore(Amavis): only add configuration to Postfix when enabled (#3046)
* only add Amavis configuration to Postfix when enabled

Since I am running Rspamd nowadays, I noticed there still are ports open
that belong to Amavis. This is because the Amavis configuration is a
fixed part of Postfix's `master.cf`. I changed that. Now, the Amavis
section is added when Amavis really is enabled.

I took the chance and added proper indentation to `master.cf`; hence the
diff is a bit fuzzy. **But**, only the Amavis part was adjusted, the
rest is just styling.
2023-02-03 17:32:07 +01:00
Georg Lauterbach
1a0c2a351a
rspamd: follow-up of #3016 (#3039) 2023-01-30 08:23:58 +01:00
Brennan Kinney
3d8cfc5b5a
chore: Remove wrapper script for fail2ban service (#3032)
* chore: Remove the wrapper script for `fail2ban`

- This does not appear necessary. The server can be run with foreground mode.
- `daemons-stack.sh` removal of the socket can be handled by the fail2ban server when using the `-x` option.

* chore: Remove `touch /var/log/auth.log`

These were both added as supposed fixes in 2016 for the then Ubuntu 2014 base image.

Removing them causes no failures in tests.

* fix: Install optional python packages for `fail2ban`

These have barely any overhead in layer weight. The DNS package may provide some QoL improvements, while the `pyinotify` is a better alternative than polling logs to check for updates.

We have `gamin` package installed but `fail2ban` would complain in the log that it was not able to initialize the module for it. There only appears to be a `python-gamin` dependent on EOL python 2, no longer available from Debian Bullseye.
2023-01-29 12:35:57 +00:00
Georg Lauterbach
555fbb78c4
feature: provide better rspamd suppport (#3016)
* added options to toggle OpenDKIM & OpenDMARC

rspamd can provide DKIM signing and DMARC checking itself, so users
should be able to disable OpenDKIM & OpenDMARC. The default is left at
1, so users have to to opt-in when the want to disable the features.

* misc small enhancements

* adjusted start of rspamd

The order of starting redis + rspamd was reversed (now correct) and
rspamd now starts with the correct user.

* adjusted rspamd core configuration

The main configuration was revised. This includes AV configuration as
well as worker/proxy/controller configuration used to control the main
rspamd processes.

The configuration is not tested extensively, but well enough that I am
confident to go forward with it until we declare rspamd support as
stable.

* update & improve the documentation

* add tests

These are some initial tests which test the most basic functionality.

* tests(refactor): Improve consistency and documentation for test helpers (#3012)

* added `ALWAYS_RUN` target `Makefile` recipies (#3013)

This ensures the recipies are always run.

Co-authored-by: georglauterbach <44545919+georglauterbach@users.noreply.github.com>

* adjusted rspamd test to refactored test helper functions

* improve documentation

* apply suggestions from code review (no. 1 by @polarthene)

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>

* streamline heredoc (EOM -> EOF)

* adjust rspamd test (remove unnecessary run arguments)

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-01-25 10:28:59 +01:00
Brennan Kinney
cb8e336d25
fix: Ensure state persisted to /var/mail-state retains correct group (#3011)
* fix: RSPAM ENV should only add to array if ENV enabled

* fix: Correctly match ownership for Postfix content

- `/var/lib/postfix` dir and content is `postfix:postfix`, not `postfix:root`.
- `/var/spool/postfix` is `root:root` not `postfix:root` like it's content.
- Add additional comments, including ownership changes by Postfix to `/var/spool/postfix` when process starts / restarts.

* fix: Ensure correct `chown -R` user and groups applied

These were all fine except for clamav not using the correct clamav group. `fetchmail` group is `nogroup` as per the group set by the debian package.

Additionally formatted the `-eq 1 ]]` content to align on the same columns, and added additional comment about the purpose of this `chown -R` usage so that it's clear what bug / breakage it's attempting to prevent / fix.

* refactor: `misc-stack.sh` conditional handling

The last condition doesn't get triggered at all AFAIK.  Nor does it make sense to make a folder path with `mkdir -p` to symlink to when the container does not have anything to copy over?

- If that was for files, the `mkdir -p` approach seems invalid?
- If it was for a directory that could come up later, it should instead be created in advance? None of the current values for `FILES` seem to hit this path.

Removing as it doesn't seem relevant to current support.

Symlinking was done for each case, I've opted to just perform that after the conditional instead.

Additional inline docs added for additional context.

* chore: Move amavis `chown -R` fix into `misc-stack.sh`

This was handled separately for some reason. It belongs with the other services handling this fix in `misc-stack.sh`.

The `-h` option isn't relevant, when paired with `-R` it has no effect.

* fix: Dockerfile should preserve `clamav` ownership with `COPY --link`

The UID and GID were copied over but would not match `clamav` user and group due to numeric ID mismatch between containers. `--chown=clamav` fixes that.

* chore: Workaround `buildx` bug with separate `chown -R`

Avoids increasing the image weight from this change by leveraging `COPY` in the final stage.

* chore: `COPY --link` from a separate stage instead of relying on scratch

The `scratch` approach wasn't great. A single layer invalidation in the previous stage would result in a new 600MB layer to store.

`make build` with this change seems to barely be affected by such if a change came before copying over the linked stage, although with `buildx` and the `docker-container` driver with `--load` it would take much longer to import and seemed to keep adding storage. Possibly because I was testing with a minimal `buildx` command, that wasn't leveraging proper cache options?

* lint: Appease the linting gods

* chore: Align `misc-stack.sh` paths for `chown -R` operations

Review feedback

Co-authored-by: Casper <casperklein@users.noreply.github.com>

* fix: Reduce one extra cache layer copy

No apparent advantage of a `COPY --link` initially in separate stage.

Just `COPY --chown` in the separate stage and `COPY --link` the stage content. 230MB less in build cache used.

* fix: Remove separate ClamAV stage by adding `clamav` user explicitly

Creating the user before the package is installed allows to ensure a fixed numeric ID that we can provide to `--chown` that is compatible with `--link`.

This keeps the build cache minimal for CI, without being anymore complex as a workaround than the separate stage was for the most part.

* chore: Add reference link regarding users to `misc-stack.sh`
2023-01-25 12:53:47 +13:00
Brennan Kinney
133eb9bc2e
tests(refactor): mail_lmtp_ip.bats (#3004)
* chore: Drop ENV `ENABLE_POSTFIX_VIRTUAL_TRANSPORT`

* tests(chore): Remove redundant `dovecot-lmtp` config

None of this is needed. Only relevant change is changing the LMTP service listener for Dovecot and that can be delegated to `user-patches.sh`.

* tests(refactor): Use `user-patches.sh` instead of replacing config file

The only relevant changes in `test/config/dovecot-lmtp` regarding LMTP was:
- `/etc/dovecot/dovecot.conf` (`protocols = imap lmtp`) and `/etc/dovecot/protocols.d/` (`protocols = $protocols lmtp`).
- `conf.d/10-master.conf` only changed the LMTP service listener from a unix socket to TCP on port 24 (_this was the only change required for the test to pass_).

None of those configs are required as:
- `protocols = imap pop3 lmtp` [is the upstream default](https://doc.dovecot.org/settings/core/#core_setting-protocols), no need to add `lmtp`.
- The LMTP service listener is now configured for the test with `user-patches.sh`.

* tests(refactor): `mail_lmtp_ip.bats`

- Converted to new testing conventions and common container helpers.
- `ENABLE_POSTFIX_VIRTUAL_TRANSPORT` was not relevant, dropped.
- Revised test cases, logic remains the same.
- Large custom config used was not documented and doesn't appear to serve any purpose. Simplified by replacing with a single modification with `user-patches.sh`.
- Added some additional comments for context of test and improvements that could be made.

* tests(chore): Adjust comments

The comment from `mail_hostname` provides no valid context, it was likely copied over from `tests.bats` in Oct 2020 by accident.

The email sent is just for testing, nothing relevant to LMTP.

---

Added additional comment for test to reference extra information from.

* tests(chore): Update similar log line matching

Extracts out the match pattern and formatting commands into separate vars (reduces horizontal scrolling), and includes extra docs about what the matched line should be expected to look like.
2023-01-15 18:33:31 +13:00
Brennan Kinney
a7e6439a39
fix: Workaround postconf write settling logic (#2998)
* fix: Workaround `postconf` write settle logic

After updating `main.cf`, to avoid an enforced delay from reading the config by postfix tools, we can ensure the modified time is at least 2 seconds in the past as a workaround. This should be ok with our usage AFAIK.

Shaves off 2+ seconds roughly off each container startup, reduces roughly 2+ minutes off tests.

* chore: Only modify `mtime` if less than 2 seconds ago

- Slight improvement by avoiding unnecessary writes with a conditional check on the util method.
- Can more comfortably call this during `postfix reload` in the change detection cycle now.
- Identified other tests that'd benefit from this, created a helper method to call instead of copy/paste.
- The `setup email restrict` command also did a modification and reload. Added util method here too.

* tests(fix): `mail_smtponly.bats` should wait for Postfix

- `postfix reload` fails if the service is not ready yet.
- `service postfix reload` and `/etc/init.d/postfix reload` presumably wait until it is ready? (as these work regardless)

* chore: Review feedback - Move reload method into utilities
2023-01-13 10:10:58 +13:00
Georg Lauterbach
26ac48f34a
feature: provide initial Rspamd support (#2902) 2023-01-05 08:39:00 +01:00
Gabriel Euzet
a00cdcdee9
fix regex in quota activation code (#2958) 2023-01-04 18:37:00 +01:00
Georg Lauterbach
0ef8d5ae2b
fix: adjust DNSBL return code interpretation (#2890) 2022-11-07 09:28:47 +01:00
Casper
474eb503ae
Fix: Make fetchmail data persistant (#2851) 2022-10-23 03:12:27 +02:00
Casper
a930aeb18a
Remove unusual space from shebang line (#2834) 2022-10-17 10:40:09 +02:00
Casper
ac795a5976
fix: run Amavis cron job only when Amavis is enabled (#2831) 2022-10-16 11:52:53 +02:00
Casper
efed7d9e44
Remove unnecessary postconf switch '-e' and use single quotes where possible (#2746) 2022-08-29 13:26:44 +02:00
Georg Lauterbach
ab55343d8e
scripts: rework environment variables setup (#2716)
* outsourcing env variable setup

This commit contains major parts of the work of refactoring the setup
and usage of environment variables. It outsources the setup into its own
script and provides dedicated functions to be executed at a later point in time.

A **new** env variable was added: `USER_PROVISIONG` which provides a
better way of defining which method / protocol to use when it comes to
setting up users. This way, the `ENABLE_LDAP` variable is deprecated,
but all of this is backwards compatible due to a "compatibility layer", a function provided by the new variables script.

This is not a breaking change. It mostly refators internal scripts. The
only change facing the user-side is the deprecation of `ENABLE_LDAP`. We
can prolong the period of deprecation for this variable as long as we
want, because the new function that ensures backwards compatibility
provides a clean interface for the future.

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-08-22 08:31:32 +02:00
Brennan Kinney
26053c22bd
fix: _create_accounts() should run after waiting (#2731)
When waiting on an account to be added to `postfix-accounts.cf`, Dovecot account creation during the startup process had already run.

Startup continued without properly creating the mail account for Dovecot. Methods like `setup email list` (with `ENABLE_QUOTAS=1`) would fail. `changedetector` service was required to be triggered to re-create Dovecot users.

- Wrapped the logic for wait + shutdown into a function call.
- Moved `_create_accounts()` to bottom of the setup function.
2022-08-22 10:44:23 +12:00
Casper
428477a878
Set configomat output to loglevel debug (#2701) 2022-07-28 20:55:21 +02:00
Brennan Kinney
e3cc627e18
refactor: Share a common helper (vhost builder) for sourcing domains (#2620)
* chore: Split vhost helper method and use filepath vars

- Helpers `accounts.sh` and `aliases.sh` can move their vhost code into this helper.
- They share duplicate code with `bin/open-dkim` which will also leverage this vhost helper going forward.

* chore: Sync vhost generation logic into helper

- Chunky commit, but mostly copy/paste of logic into a common method.
- `bin/open-dkim` additionally wrapped relevant logic in a function call and revised inline docs.

* chore: Include LDAP vhost support

- Revises notes for LDAP vhost support.
- This now ensures LDAP users get vhost rebuilt to match the startup script for when change detection support is enabled.
- `bin/open-dkim` will additionally be able to support the default `DOMAINNAME` var (set via `helpers/dns.sh`) for LDAP users instead of requiring them to provide one explicitly.

* chore(`bin/open-dkim`): Ensure `DOMAINNAME` is properly set

- This will ensure LDAP users insert the same `DOMAINNAME` value as used during container startup.
- The container itself should panic at startup (during `helpers/dns.sh`) if this isn't configured correctly already, thus it should not introduce any breaking change to users of this utility?

* chore: Set the 2nd value as blank `_`

Line is split by a delimiter such as white-space (or via IFS: `|`), the blank `_` var is to indicate we're not interested in that value, but still leverage how `read -r` works, instead of splitting the var ourselves first thing.

* chore: Remove shellcheck disable lines

No longer applicable with the switch to `_`
2022-06-10 10:57:10 +12:00
Brennan Kinney
0a722276a8
chore: Extract out /var/mail ownership workaround (#2628)
Keep it in sync between the two locations via shared helper method.
2022-06-08 10:09:19 +12:00
Brennan Kinney
54904aa02c
chore(housekeeping): Normalize how config files filter out unwanted lines (#2619)
* chore(`aliases.sh`): Filepath to local var `DATABASE_VIRTUAL`

* chore(`accounts.sh`): Filepath to local var `DATABASE_ACCOUNTS`

* chore(`accounts.sh`): Filepath to local var `DATABASE_VIRTUAL`

* chore(`accounts.sh`): Filepath to local var `DATABASE_DOVECOT_MASTERS`

* chore(`bin/open-dkim`): Filepaths to local vars (accounts,virtual,vhost)

* chore(`relay.sh`): Filepath to local var `DATABASE_SASL_PASSWD`

* chore: Rename method

Prior PR feedback suggested a better helper method name.

* chore: Normalize filtering config lines as input for iterating

* chore: Remove `_is_comment` helper method

No longer serving a purpose with more appropriate filter method for pre-processing the entire config file.
2022-06-07 01:07:30 +12:00
Georg Lauterbach
0010786d18
scripts: improve custom user-supplied Postfix configuration (#2598)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-06-06 09:30:10 +02:00
Brennan Kinney
40e2d88482
chore: Merge helpers/sasl.sh into helpers/relay.sh (#2605)
This helper was to support an earlier ENV for SASL auth support. When extracting logic into individual helpers, it was assumed this was separate from relay support, which it appears was not the case.

---

The `SASL_PASSWD` ENV is specified in tests but no longer used. There is no `external-domain.com` relay configured or tested against anywhere in the project.

The ENV was likely used in tests prior to improved relay support that allowed for adding more than a single set of relay credentials.

---

It likewise has no real relevance anywhere else outside of `relay.sh` as it's the only portion of code to operate with it.

It's only relevant for SASL auth as an SMTP client, not the SMTP server (`smtpd`) SASL support that is delegated to Dovecot. Functionality has been completely migrated into `relay.sh` as a result.

Documentation is poor for this ENV, it is unlikely in wide use? Should consider for removal.

---

The ENV has been dependent upon `RELAY_HOST` to actually enable postfix to use `/etc/postfix/sasl_passwd`, thus not likely relevant in existing setups?

---

Migrate `/etc/postfix/sasl_passwd` check from `tests.bats` as it belongs to relay tests.
2022-06-06 10:59:42 +12:00
Brennan Kinney
3d6e7a7bb8
service(postfix): Better handling of the compatibility_level setting (#2597)
* chore: Fix typo

* chore: Apply explicit chroot default for `sender-cleanup`

The implicit default is set to `y` as a compatibility fallback, but otherwise it is [advised to set to `n` going forward](http://www.postfix.org/COMPATIBILITY_README.html#chroot).

Test was changed to catch any backwards-compatibility logs, not just those for `chroot=y`. `using` added as a prefix to avoid catching log message whenever a setting is changed that the default compatibility level is active.

* chore: Set `compatibility_level` in `main.cf`

We retain the level`2` value previously set via scripts. This avoids log noise that isn't helpful.

Applied review feedback to give maintainers some context with this setting and why we have it presently set to `2`.
2022-06-05 12:10:20 +12:00
Brennan Kinney
d04aceccae
fix: Only add state of enabled services for volume mount (#2608)
* fix: Conditionally add service state

These services will no longer copy over state unless they are enabled.

The biggest offender here was ClamAV as it's database that is baked into the docker image is over 200MB and would copy over to every container instance with a volume mounted state directory.

* chore: Add Dovecot to conditional support
2022-06-05 12:02:25 +12:00
Brennan Kinney
c862e1451d
chore(housekeeping): Create helpers/change-detection.sh (#2610)
* chore: Extract change-detection method to it's own helper

This doesn't really belong in `helpers/ssl.sh`. Moving to it's own helper script.

* chore: Co-locate related change-detection method from container startup

It seems relevant to migrate the related support during startup for the change detection feature into this helper.

I opted to move the call from `start-mailserver.sh` into the `_setup` call at the end for a more explicit/visible location.

* chore: Move `CHKSUM_FILE` into `helpers/change-detection.sh`

It belongs there, not in `helpers/index.sh`.

* chore: Revise inline documentation

* tests(fix): Ensure correct functionality

Presently `test/test_helper.bats` is using it's own  `CHKSUM_FILE` instead of sourcing the var for the filepath.

`test_helper/common.bash` was calling a method to check for changes, but this helper may not correctly detect letsencrypt related changes as these are not ENV rely on, but global vars handled by `helpers/dns.sh`, so that should be run first like it is for `check-for-changes.sh`.

* tests(chore): Use `CHKSUM_FILE` var from helper

* chore: `addmailuser` should use `CHKSUM_FILE` var

* chore: Update `check-for-changes.sh` log message with correct path
2022-06-05 11:59:54 +12:00
Casper
9a739113f4
chore: Migrate SASLAUTHD_* vars into start-mailserver.sh (#2562) 2022-06-04 19:28:13 +00:00
Georg Lauterbach
05e45c349a
scripts: fix error message caused by grep on first starts (#2591)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-05-28 20:59:31 +02:00
Georg Lauterbach
b7e5d42e09
scripts: more consistency in start-mailserver.sh (#2587)
* consistently name functions (starting with `_`) in `start-mailserver.sh`

Most of the functions that execute the different stacks during startup
were not prefixed with `_`, but all our other functions are. This has
now been fixed.

* cleanup in `start-mailserver.sh`

I adjusted the comments for all sections in the start script so they are
properly displayed again.
2022-05-15 23:37:21 +02:00
Maximilian Hippler
777d4a8cb7
scripts: loglevel warning message fix (#2588)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2022-05-13 10:50:45 +02:00
Casper
2977cb6962
Streamline 'printf' usage and consequently use a format string (#2564) 2022-05-05 17:48:21 +02:00
Casper
628e902233
Remove unnecessary quotes from command substitutions (#2561) 2022-05-05 10:28:38 +02:00
Casper
32fba30bbc
Don't issue warning when DKIM is not used (#2563) 2022-04-29 19:03:16 +02:00
Casper
de61d42e68
Add ban feature to fail2ban script (#2538) 2022-04-19 08:44:51 +00:00
Brennan Kinney
9aaf15b38f
chore: (setup-stack.sh) Fix a small typo (#2552) 2022-04-19 12:53:45 +12:00
Georg Lauterbach
412f675bfe
setup: added grace period for account creation (#2531) 2022-04-18 10:48:48 +02:00
Georg Lauterbach
a1726dc45a
scripts: added TZ environment variable to set timezone (#2530) 2022-04-06 16:48:41 +02:00
Georg Lauterbach
c7b16a599c
log: adjust level and message(s) slightly for four messages (#2532)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-04-05 16:21:07 +02:00
Georg Lauterbach
a9305a073f
firewall: replace iptables with nftables (#2505)
* first adjustments to use Fail2Ban with nftables

* replace `iptables` -> `nftables` and adjust tests

nftables lists IPs a bit differently , so the order was adjusted for the
tests to be more flexible.

* line correction in mailserver.env

* change from `.conf` -> `.local` and remove redundant config

* revert HEREDOC to `echo`

Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-04-05 15:13:59 +02:00
Ikko Ashimine
05f680b472
fix: typo in setup-stack.sh (#2521) 2022-04-02 10:41:46 +02:00
Georg Lauterbach
a54d774587
scripts: refactored daemon-stack.sh (#2496)
* refactored `daemon-stack.sh`

A new method was introduced to uniformaly start daemons and log output
accordingly. The methods for daemon start were renamed (plural ->
singular), therefore the adjustments in `start-mailserver.sh`.

* cleaned Fetchmail setup from `daemon-stack.sh`

Not sure why, but the Fetchmail setup was somehow happening in
`daemon-stack.sh` - this is not supposed to be the case. I relocated the
setup into `setup-stack.sh`, where it belong.

* delete old, unnecessary script in `target/bin/`

These are unused leftovers from the last commit, that relocated the
setup of Fetchmail into `setup.stack.sh`.

* corrected changedetector function name

* Apply suggestions from code review

* adjusted `debug-fetchmail` script

It is absolutely fine to source `setup-stack.sh` because sourcing the
script does not execute a single function (by desing of the script).
This way, we retain functionality.

* praise be ShellCheck

* added `log.sh` to `debug-fetchmail` as a dependency

* final cleanup

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-03-27 09:43:39 +02:00
Georg Lauterbach
7721a48b9b
scripts: refactoring & miscellaneous small changes (#2499)
* `update-check.sh` now uses the new log

* refactored `setup-stack.sh`

The changes are:

1. Replaced `""` wiht `''` where possible (reasoning: Bash is very
   implicit and I'd like to use `''` where possible to indicate no
   variables are expanded here)
2. `> /file` -> `>/file` according to our style guide
3. Some log adjustments for messages where I deemed it appropriate
4. Then, an error message from a Dovecot setup was also prevented (by
   adding a check whether the directory is present before a `: >...`
   command would create a file in this directory).

These are all small, miscellaneous changes that I wanted to combine into
one commit and ultimately one PR because I see no point in opening a PR
for every small change here. I hope this is fine.

* added a small `sleep` to the `_shutdown` function

This ensure the last log message is actually logged before Supervisor
logs the message that it received a SIGTERM. This makes reading the log
easier because now the causal relationship is shown (we are terminating
Supervisor, and not someone else and we're just logging it).

I forgot to replace `""` with `''` in `update-check.sh`, so I included
it here because this is the last commit before PR review.

* re-add exit on successful update (only)

* re-added date information to update-check log messages

* added `_log_with_date` function

The new function will log a message with a proper timestamp. This is all
handled in `log.sh`, we therefore not need to source other files too.

This will be used in the future by `check-for-changes.sh` as well :)

Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-03-26 10:17:08 +01:00
Georg Lauterbach
24031ae365
scripts: new log (#2493)
* added new `_log` function

With `_log`, the `_notify` method wa rendered obsolete. `_notify` was
not completely removed due to test failures in `check-for-changes.sh`.

The new `_log` function properly uses log levels such as `trace`,
`debug`, `info`, `warn` and `error`. It provides a cleaner solution
and renders `DMS_DEBUG` obsolete too (as only `_notify` depends on it).

* converted all helper script to new `_log` function

* converted all startup stacks to new `log` function

* `start-mailserver.sh` now uses new `_log` function

* final test and misc small script adjustments

* updated documentation
2022-03-21 07:07:52 +01:00
Georg Lauterbach
321ae744fa
scripts: small refactorings (#2485)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-03-17 11:24:30 +01:00
Georg Lauterbach
d101d0aa0a
fix: remove SA reload for KAM (#2456)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-03-04 23:38:29 +01:00
Casper
b9fdb1e7b5
Introduce CLAMAV_MESSAGE_SIZE_LIMIT env (#2453) 2022-03-03 16:17:01 +01:00
Georg Lauterbach
e6af5a118f
Adjust envrionment variables - more sensible defaults (#2428)
The new setup will now set env variables on one place and on one place
only. The old setup used two separate places wich is not DRY and
confusing.

Some default values changed:

1. PFLOGSUMM_TRIGGER: logrotate => none
2. REPORT_SENDER: mailserver-report@HOSTNAME => mailserver-report@DOMAIN
3. REPORT_RECIPIENT: "0" => POSTMASTER_ADDRESS

One env variable was renamed: REPORT_INTERVAL => LOGROTATE_INTERVAL

I believe these defaults to be more sensible, especially the REPORT_RECIPIENT
address. The PFLOGSUMM_TRIGGER value was changed to `none` because otherwise
people would start getting daily Postfix log summary reports automatically.
Now, this is opt-in, and reports are sent only when enabled properly.

Some of the variables changed were marked as deprecated. I removed the note,
as the variables now bear some (sane) defaults again for other variables
(i.e.) REPORT_RECIPIENT is now default for other recipient addresses.

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-03-03 10:22:17 +13:00
Casper
57c52d7b5b
PERMIT_DOCKER=none as new default value (#2424)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-03-02 16:03:40 +01:00
Georg Lauterbach
399284c092
fix: do not add accounts that already exists to account files (#2419)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-02-21 13:31:03 +01:00
Georg Lauterbach
b61dfe1e24
refactoring: split helper functions into smaller scripts (#2420) 2022-02-21 11:56:57 +01:00
Georg Lauterbach
2927cc47c7
Add SpamAssassin KAM (#2418) 2022-02-21 10:48:28 +01:00
Georg Lauterbach
ec8b99335e
Add changedetector functionality for ${SSL_TYPE} == manual (#2404)
Now, setups that use `SSL_TYPE=manual` will profit from the changedetector as well. Certificate changes are picked up and properly propagated.
2022-02-18 11:29:51 +01:00
Christian Raue
908e95fa74
consistently make 1 the default value for SPAMASSASSIN_SPAM_TO_INBOX (#2361)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-02-17 09:31:45 +01:00