Commit Graph

1013 Commits

Author SHA1 Message Date
Dirk-Jan C. Binnema 317fe53ff7 tests: update test helpers and users
Move test-mu-common to mu-test-utils. Use mu_test_init as a wrapper for
g_test_init. Update users.
2022-08-11 22:55:10 +03:00
Dirk-Jan C. Binnema 9aabe74df6 mu4e: add more tests for related/skipdups queries 2022-08-07 12:01:18 +03:00
Dirk-Jan C. Binnema 9fdcee8c22 test-mu-store-query: add more contact-matching tests 2022-07-14 08:10:15 +03:00
Dirk-Jan C. Binnema ed93ff4968 message: sanitize maildir
Remove trailing '/' in maildirs, since people have that (like "/foo/"),
and earlier version didn't complain about that.

Fixes #2298
2022-07-13 23:27:54 +03:00
Dirk-Jan C. Binnema d8bbeb0218 mu-server: more informative error message 2022-07-05 00:11:56 +03:00
Dirk-Jan C. Binnema ca8836b631 document: cosmetic 2022-06-29 22:20:34 +03:00
Dirk-Jan C. Binnema 810b9643f4 query: exclude some test code from coverage 2022-06-29 22:20:09 +03:00
Dirk-Jan C. Binnema 97459beed9 maildir: improve testing coverage
Remove some dead/unused code. Update docs. Add test cases.
2022-06-29 22:19:26 +03:00
Dirk-Jan C. Binnema fc25bb2866 server: commit after indexing
So external users see the changes.
2022-06-29 22:18:15 +03:00
Dirk-Jan C. Binnema c4ed3e6ba7 server: fix read-mark propagation
Whenever a message is flagged as 'read', do the same for all the duplicates as well.
This used to work, make it work again.

Fixes #2277
2022-06-29 08:59:40 +03:00
Dirk-Jan C. Binnema df80935c2e document: index some sub-parts as well
1. Also add 'normal' terms for some indexable fields
2. Add terms for e-mail address components

And add some tests.

This helps for some corner-case queries (see tests).

Fixes #2278
Fixes #2281
2022-06-29 08:00:43 +03:00
Dirk-Jan C. Binnema 23ac71e6a7 maildir: add tests for maildir_link / maildir_clear_links 2022-06-29 07:50:16 +03:00
Dirk-Jan C. Binnema 0708033e48 build: update autotools build 2022-06-18 15:21:26 +03:00
Dirk-Jan C. Binnema cade7493fd build: fix some compiler warnings 2022-06-18 15:21:26 +03:00
Dirk-Jan C. Binnema 1f9172a008 message: remove x-ms-has-attach check
Apparently, it's not very useful (marks sigs as attachments).
2022-06-16 22:50:36 +03:00
Dirk-Jan C. Binnema f69ad37e7a utils: add regex-split 2022-06-16 22:49:46 +03:00
Dirk-Jan C. Binnema 040d74a326 lib/test-mu-store-query: add aujourd'hui tests
We match aujourd'hui now; add a test.

Fixes #1150.
2022-06-14 23:48:59 +03:00
Dirk-Jan C. Binnema 48695a1981 query-parser: tidy up
Remove "Data", and use Field directly.
2022-06-14 23:15:27 +03:00
Dirk-Jan C. Binnema 412adf3400 query-match-deciders: cosmetics 2022-06-10 23:28:43 +03:00
Dirk-Jan C. Binnema 40669aae49 message/mime-object: add MimeObject::headers
To get a list of all header values.
2022-06-10 23:28:43 +03:00
Dirk-Jan C. Binnema 1ec03ce07c message-file: filter out non-file flags
Only take flags from file names that are actually file-flags, not random other
things
2022-06-10 23:28:43 +03:00
Dirk-Jan C. Binnema 9bf580de3d message: refactor/improve attachment heuristic a bit
Also check for X-MS-Has-Attach
2022-06-10 23:28:43 +03:00
Dirk-Jan C. Binnema 39dcd08fbe message: add calendar flag
For message that contain calendar invitations
2022-06-10 00:47:31 +03:00
Dirk-Jan C. Binnema 646a672ae0 test-utils: add error test 2022-06-09 00:39:34 +03:00
Dirk-Jan C. Binnema 2f3d1bb522 lib: add tokenizer test tool 2022-06-09 00:39:34 +03:00
Dirk-Jan C. Binnema c1f66059a8 store: fix typo 2022-06-09 00:39:34 +03:00
Dirk-Jan C. Binnema ce28c86803 tests: add store/query test
Add some tests for storing/indexing message and then run queries on them.
2022-06-09 00:39:34 +03:00
Dirk-Jan C. Binnema d38d32c404 query-threads/results: cosmetics 2022-06-09 00:39:34 +03:00
Dirk-Jan C. Binnema 49b97e1b6b message/fields: explicit mark contact fields as IndexableTerm
They were so *implicitly*, but the query parser needs the info so contact fields
can use phrase-searces, too.
2022-06-09 00:04:35 +03:00
Dirk-Jan C. Binnema 4379e66b9f bench-store-index: make a bit more realistic
```
             .',;::::;,'.                djcb@evergrey
         .';:cccccccccccc:;,.            -------------
      .;cccccccccccccccccccccc;.         OS: Fedora release 36 (Thirty Six) x86_64
    .:cccccccccccccccccccccccccc:.       Host: MS-7B09 2.0
  .;ccccccccccccc;.:dddl:.;ccccccc;.     Kernel: 5.17.11-300.fc36.x86_64
 .:ccccccccccccc;OWMKOOXMWd;ccccccc:.    Uptime: 6 hours, 16 mins
.:ccccccccccccc;KMMc;cc;xMMc:ccccccc:.   Packages: 4340 (rpm), 70 (flatpak)
,cccccccccccccc;MMM.;cc;;WW::cccccccc,   Shell: zsh 5.8.1
:cccccccccccccc;MMM.;cccccccccccccccc:   Resolution: 3840x2160
:ccccccc;oxOOOo;MMM0OOk.;cccccccccccc:   WM: sway
cccccc:0MMKxdd:;MMMkddc.;cccccccccccc;   Theme: Adwaita [GTK2]
ccccc:XM0';cccc;MMM.;cccccccccccccccc'   Icons: Adwaita [GTK2]
ccccc;MMo;ccccc;MMW.;ccccccccccccccc;    Terminal: tilix
ccccc;0MNc.ccc.xMMd:ccccccccccccccc;     CPU: AMD Ryzen Threadripper 1950X (32) @ 3.399GHz
cccccc;dNMWXXXWM0::cccccccccccccc:,      GPU: AMD ATI Radeon RX 470/480/570/570X/580/580X/590
cccccccc;.:odl:.;cccccccccccccc:,.       Memory: 7101MiB / 15881MiB
:cccccccccccccccccccccccccccc:'.
.:cccccccccccccccccccccc:;,..
  '::cccccccccccccc::;,.

```

% make benchmark
ninja -C /home/djcb/Sources/mu/build benchmark
ninja: Entering directory `/home/djcb/Sources/mu/build'
[0/1] Running benchmark suite.
1/1 bench-store-index        OK             17.40s
2022-06-07 23:06:25 +03:00
Dirk-Jan C. Binnema 65e60a6ed5 indexer: make faster
Remove an (apparently) unneeded lock which help back much of the multi-core
handing.
2022-06-07 23:06:25 +03:00
Dirk-Jan C. Binnema 30c46fba61 message: fix 'changed' test 2022-06-05 15:22:35 +03:00
Dirk-Jan C. Binnema 3d3fd1aa7a server: add some more logging for read-marking
See what happens with multiple messages.
2022-06-04 00:41:28 +03:00
Dirk-Jan C. Binnema d89820b26b lib/tests: move tests to parent
Otherwise, the coverage checker won't notice them!
2022-06-04 00:41:28 +03:00
Dirk-Jan C. Binnema 015a7ee0cb utils: improve locale_workaround
Fixes: #2270.
2022-06-04 00:41:28 +03:00
Dirk-Jan C. Binnema 0fe8f9a613 mu-util-play: simplify
Remove some unused options
2022-06-04 00:41:28 +03:00
Dirk-Jan C. Binnema 172ff25bb6 mu-maildir: refactor some duplicate code
mu-flags & mu-maildir were having some duplicate code; refactor it into mu-message-file.{cc,hh}
2022-06-04 00:41:28 +03:00
Dirk-Jan C. Binnema 13f0e24241 lib: improve test coverage
Add a bunch of tests
2022-06-02 21:07:41 +03:00
Dirk-Jan C. Binnema a66360eda5 maildir: revamp unit tests
Re-enable many of the tests that were turned off, and update them for the new impl.
2022-05-31 23:43:13 +03:00
Dirk-Jan C. Binnema 26d64ec78f mu-maildir: fix message_file_parts / empty flags
Remove the suffix part even with no flags.

Fixes: issue #2268
2022-05-31 23:41:50 +03:00
Dirk-Jan C. Binnema 022ae9c2e3 result: make assert_valid_result a bit more informative
For unit tests.
2022-05-31 23:39:50 +03:00
Dirk-Jan C. Binnema ce1fea6302 store: log whether a message is 'personal' 2022-05-30 20:31:23 +03:00
Dirk-Jan C. Binnema 734445f78d mu-server: add a bit more debugging info 2022-05-30 20:30:33 +03:00
Dirk-Jan C. Binnema 5df56896fb lib/meson.build: explicitly add thread_dep
For many setups (local, CI), things work fine without this, but apparently not
for all users. So, add thread_dep explicitly.

Fixes: #2266.
2022-05-29 11:10:58 +03:00
Dirk-Jan C. Binnema d5632b9ac9 script: avoid compiler warning 2022-05-29 00:17:41 +03:00
Dirk-Jan C. Binnema ffcb59bfd3 utils: add fallback for time_to_string
Still don't understand issue #2230
2022-05-28 00:51:08 +03:00
Dirk-Jan C. Binnema ad6f8f1190 thirdparty: include in EXTRA_DIST
For autotools.
2022-05-28 00:51:08 +03:00
Dirk-Jan C. Binnema 5c161cfdbb lib/tests: add indexing benchmark 2022-05-28 00:51:08 +03:00
Dirk-Jan C. Binnema 0adf91f821 fields: include references in sexp
We need them for setting up replies. This requires a db update.
2022-05-28 00:51:08 +03:00
Dirk-Jan C. Binnema 91dcd19dad tests: improve coverage a bit 2022-05-24 19:36:39 +03:00
Dirk-Jan C. Binnema 046398b1ae utils: avoid compiler warning 2022-05-24 19:36:21 +03:00
Dirk-Jan C. Binnema 277a7e0e10 guile: don't extern-c'ify libguile
Already has it... seems to break compilation on some machines?
2022-05-24 19:35:29 +03:00
Dirk-Jan C. Binnema 9c9f9ecae3 lib: improve coverage a bit 2022-05-23 23:55:11 +03:00
Dirk-Jan C. Binnema 8eac392221 guile: re-enable the guile support
Update for the internal (Mu::Message) APIs.

Update build.
2022-05-21 17:44:09 +03:00
Dirk-Jan C. Binnema ce46c44ea9 query-results: ease getting Xapian::Document for iter
This makes it easier to construct guile-usable Messages.
2022-05-21 17:42:25 +03:00
Dirk-Jan C. Binnema a3ad04f12f message: make it easier to create heap-allocated messages
We need this for guile to coöperate with its garbage collector.
2022-05-21 17:41:21 +03:00
Dirk-Jan C. Binnema cc63eca8ad autotools: improve build
*almost* works (except for guile/, which does work yet anyway).
2022-05-19 23:38:49 +03:00
Dirk-Jan C. Binnema cea3167f3e lib: remove some dead code (mu-str/utils)
Remove unused library code; recycle some electrons.
2022-05-19 23:38:49 +03:00
Dirk-Jan C. Binnema 9af9d0fa94 indexer/store: avoid completed callback
This cause havoc / race conditions.
2022-05-18 20:16:48 +03:00
Dirk-Jan C. Binnema c3778cd467 store: reset indexer in dtor
Clear it up as soon as possible.
2022-05-18 18:24:30 +03:00
Dirk-Jan C. Binnema 0be54119d8 store: stop indexer in dtor 2022-05-18 18:07:19 +03:00
Dirk-Jan C. Binnema 2fd133761f utils: fix unsetting timezone 2022-05-18 01:12:20 +03:00
Dirk-Jan C. Binnema b03590cca8 tests: skip missing timezones in queries, too 2022-05-18 01:08:40 +03:00
Dirk-Jan C. Binnema 33d30775ee test-utils: Skip test if timezone is not available 2022-05-18 00:52:28 +03:00
Dirk-Jan C. Binnema dbd15cf704 mu-maildir: don't allow empty path for new maildir
be a bit stricter.
2022-05-18 00:35:02 +03:00
Dirk-Jan C. Binnema 493549a896 mime-object: fix some pedantic compiler warnings 2022-05-18 00:14:01 +03:00
Dirk-Jan C. Binnema c4cc9795dc utils: add locale_workaround
Attempt to work around some locale handling throwing in some systems.

   "locale::facet::_S_create_c_locale name not valid"

Ugly, but maybe it helps.
2022-05-17 22:31:03 +03:00
Dirk-Jan C. Binnema 6e37790012 mu: improve error reporting 2022-05-15 13:40:44 +03:00
Dirk-Jan C. Binnema e13dd0da98 server: Implement forwarding attachments
Using the new infra in Message.
2022-05-15 11:33:14 +03:00
Dirk-Jan C. Binnema 5d9bb72c0b message: augment cache-path() so it supports subparts
Take an optional index parameter for a *subpart*.

This is for the case where we save attachments from a message (in particular,
when forwarding). We can't save them in the same directory for the (rare) case
when there are multiple attachments with the same name. And we don't want to
uniquify the name, since that shows up in e.g. the forwarded file name.

This can be solved by saving each in their own indexed subdir.
2022-05-15 11:28:48 +03:00
Dirk-Jan C. Binnema 9e78d3d13f utils: insist on semicolon after MU_ENABLE_BITOPS 2022-05-14 12:47:26 +03:00
Dirk-Jan C. Binnema bf9e083ed4 message: avoid shadowing variable 2022-05-14 12:47:01 +03:00
Dirk-Jan C. Binnema 907751bc7b test: enable some more 2022-05-13 23:01:33 +03:00
Dirk-Jan C. Binnema 4edbe01aa0 indexer: add extra fsm state "finishing" to fix some races
As reported by helgrind
2022-05-13 23:00:20 +03:00
Dirk-Jan C. Binnema 201acfbbd3 store: add lock to index_complete
It must be protected.
2022-05-12 23:23:47 +03:00
Dirk-Jan C. Binnema eb22ec450c message: Don't assume there's a sub-message
Also add unit-test.

Fixes: issue #2256.
2022-05-12 19:59:10 +03:00
Dirk-Jan C. Binnema 5b5dbf3aae fields: some small tweaks 2022-05-12 08:56:13 +03:00
Dirk-Jan C. Binnema d79d770958 store: improve error handling
Detect store-lock
2022-05-12 08:56:13 +03:00
Dirk-Jan C. Binnema 34c5ca1627 mu-error: encode retval/flags in enum 2022-05-12 08:56:13 +03:00
Dirk-Jan C. Binnema 2e9666af0b store/info: Gather some usage statistics
Keep track of the latest-change/latest-index.
2022-05-10 08:16:47 +03:00
Dirk-Jan C. Binnema 81689f0af3 contacts-cache: return most relevant contacts
Return in the contacts in *reverse* rank order, i.e. the most relevant come first.
This is useful since we only want the first maxnum contacts, and those should of
course be the most relevant.

Update mu cfind/server as well.

cfind
2022-05-09 22:25:28 +03:00
Dirk-Jan C. Binnema 80d84bf635 store: use Result<Store> builder, add auto upgrade
Make it a Result type, and add auto-upgrade (not enabled yet)
Update dependents.
2022-05-09 22:25:28 +03:00
Dirk-Jan C. Binnema 7470a5f8ea store/fields: Use one MaxTermLength everywhere
One should be enough!
2022-05-09 22:25:28 +03:00
Dirk-Jan C. Binnema dc69505580 mu-maildir: No need for namespaced function names
Mu::mu_maildir_... -> Mu::maildir_...
2022-05-09 22:25:28 +03:00
Dirk-Jan C. Binnema 48311ea18d mu/mu4e: tweak :list vs :mailing list 2022-05-07 20:48:21 +03:00
Dirk-Jan C. Binnema 80c9ffae1e mu/mu4e: use :mailing-list as the mailing-list property
In queries you can use list: as before

This helps for issue #2246.
2022-05-07 16:10:52 +03:00
Dirk-Jan C. Binnema d79488efdc avoid warnings, update tests 2022-05-06 22:54:41 +03:00
Dirk-Jan C. Binnema f83d1b3ed0 update tests: avoid helgrind warning
Log to the black_hole or helgrind gets unhappy
2022-05-06 22:17:53 +03:00
Dirk-Jan C. Binnema 8c3d1ae90a message: cosmetics 2022-05-06 22:17:53 +03:00
Dirk-Jan C. Binnema 4fc05261ce message: implement rfc2047-encoding addresses
For address names with commas, quotes etc.
2022-05-06 22:17:53 +03:00
Dirk-Jan C. Binnema da8489d0f6 sexp: allow for some prettified string output
Allow for adding newlines between list items
2022-05-06 22:17:53 +03:00
Dirk-Jan C. Binnema 2a5c1e239c message: support embed:text in attached mail
As before; scan attached messages as well.
2022-05-06 22:17:53 +03:00
Dirk-Jan C. Binnema f9550f3cb6 mu: merge flags into fields command
One is enough
2022-05-06 22:17:53 +03:00
Dirk-Jan C. Binnema 836bfc9062 mu/mu4e: implement mu4e-compose-complete-max
Allow limiting number of contacts for auto-completion to a specific number;
defaulting to 2000.
2022-05-06 21:58:51 +03:00
Dirk-Jan C. Binnema ffce3dda21 contacts: support encoding names with '"', ',' etc. 2022-05-06 09:03:11 +03:00
Dirk-Jan C. Binnema e0d047105b server/message-part: restore forwarding attachments 2022-05-05 23:45:11 +03:00
Dirk-Jan C. Binnema 62e546a4e2 autotools: fix build 2022-05-05 23:40:56 +03:00
Dirk-Jan C. Binnema 8d80bedf38 maildir: add some useful defaults for mu_maildir_mkdir 2022-05-05 01:41:32 +03:00
Dirk-Jan C. Binnema 2ccf703769 server: update for dealing with message/sexps 2022-05-05 01:40:17 +03:00
Dirk-Jan C. Binnema 4b56b8779d mime-object: ensure init_gmime is called
Was missing in some paths
2022-05-05 01:40:17 +03:00