emacs/mu
1
0
Fork 0
mirror of https://github.com/djcb/mu.git synced 2024-06-28 07:41:04 +02:00
Code Issues Releases Wiki Activity
6,904 Commits 6 Branches 120 Tags 27 MiB
67c8ff75a7
Commit Graph

1177 Commits

Author SHA1 Message Date
Dirk-Jan C. Binnema
843c086b2c indexer: fix build 2023-08-07 19:09:19 +03:00
Dirk-Jan C. Binnema
fabeb4a89a index: fix lazy indexing 
After the previous changes
 2023-08-07 08:49:44 +03:00
Dirk-Jan C. Binnema
253b44043b indexer: disable lazy check for "full" scan 
Lazy would actually do _more_ work a full scan.
 2023-08-06 16:19:43 +03:00
Dirk-Jan C. Binnema
4ecf386cda utils-file: don't use regexp in join_paths 
It's slow.
 2023-08-06 16:19:43 +03:00
Dirk-Jan C. Binnema
01a516f0d3 server: tweak sexp generation 2023-08-06 16:19:43 +03:00
Dirk-Jan C. Binnema
4945e699c8 lib/mu: use fmt-based time/date formatting 
For a small speedup
 2023-08-06 16:19:43 +03:00
Dirk-Jan C. Binnema
27c07280b1 utils: replace time_to_string with fmt-based formatting 
It's faster; make "mu find" ~5-10% faster, and removes some code we no
longer need.
 2023-08-06 16:19:43 +03:00
Dirk-Jan C. Binnema
6dfb2aae7b server: don't use structured-bindings / lambda for contacts_handler 
Older clang doesn't like that.
 2023-08-04 22:00:51 +03:00
Dirk-Jan C. Binnema
75c37a506b server: don't use structured-bindings / lambda 
Older clang doesn't like that.
 2023-08-04 21:44:58 +03:00
Dirk-Jan C. Binnema
f89e4c26d7 server: attempt to appease clang (pair/tuple) 
https://stackoverflow.com/questions/46114214/lambda-implicit-capture-fails-with-variable-declared-from-structured-binding
 2023-08-04 21:21:49 +03:00
Dirk-Jan C. Binnema
051fdb4ccf lib/config: set default batch-size to 50000 
The default was 250000 but that led to problems on some systems with
limited memory, esp. since mu's indexing does quite a bit more than in
the olden days (e.g. html mail).

Fixes #2529.
 2023-08-04 00:09:02 +03:00
Dirk-Jan C. Binnema
25151aad00 mu-query: small optimization tweaks 2023-08-04 00:09:02 +03:00
Dirk-Jan C. Binnema
aea95b5be0 mu-server: use strings, not sexps object (optimization) 
When passing messages to mu, often we got a (parsed from string)
message-sexp from the message document; then appended some more
properties ("build_message_sexp").

Instead, we can do it in terms of the strings; this is _a little_
inelegant, but also much faster; compare:

(base)
[mu4e] Found 500 matching messages; 0 hidden; search: 1298.0 ms (2.60 ms/msg); render: 642.1 ms (1.28 ms/msg)

(with temp-file optimization (earlier commit)
[mu4e] Found 500 matching messages; 0 hidden; search: 1152.7 ms (2.31 ms/msg); render: 270.1 ms (0.54 ms/msg)

(with temp file optimize _and_ the string opt (this commit)
[mu4e] Found 500 matching messages; 0 hidden; search: 266.0 ms (0.53 ms/msg); render: 199.7 ms (0.40 ms/msg)
 2023-08-04 00:09:02 +03:00
Dirk-Jan C. Binnema
1018f0f0a1 mu-document: Make sexp() lazy (optimization) 
This makes queries where we don't need the sexp much faster; e.g.

before:
   mu find "a" --include-related  47,51s user 2,68s system 99% cpu 50,651 total
after:
  mu find "a" --include-related  7,12s user 1,97s system 87% cpu 10,363 total
 2023-08-04 00:09:02 +03:00
Dirk-Jan C. Binnema
924bb2145e mu-server: implement temp-file optimization 
It can be faster to feed big mu -> mu4e data, such as contacts are
message headers through a temp-file instead directly though stdout;
implement this, and add the server parameter --allow-temp-file.

Implement this the "contacts" and "find" commands.
 2023-08-04 00:09:02 +03:00
Dirk-Jan C. Binnema
111e48efa3 utils: add expand_path (wordexp wrapper) 
For expanding command-line options for shells that don't do that by themselves.
 2023-08-03 22:47:27 +03:00
Dirk-Jan C. Binnema
33fd79a9f0 mu-regex: add multiline test 2023-07-30 00:50:45 +03:00
Dirk-Jan C. Binnema
3a38d6366a mu-view: test locale to C for tests 2023-07-29 17:25:07 +03:00
Dirk-Jan C. Binnema
766d1849ff test-utils: add TempTz, RAII temporary timezone 2023-07-29 16:39:08 +03:00
Dirk-Jan C. Binnema
1f0342a91f mu-view: add unit-test 2023-07-28 19:43:46 +03:00
Dirk-Jan C. Binnema
dc29dc8395 html-to-text: add missing include <array> 2023-07-26 23:30:54 +03:00
Dirk-Jan C. Binnema
c06e765d13 html-to-text: be explicit with array type 
clang in CI fails to deduce it, so let's help it a bit.
 2023-07-26 23:24:29 +03:00
Dirk-Jan C. Binnema
455119f695 Merge branch 'wip/djcb/html-to-text' 2023-07-26 19:11:41 +03:00
Dirk-Jan C. Binnema
da290c21a9 benchmark: improve setup 
Add some useful make targets, and separate (optimized) build.
 2023-07-25 23:56:19 +03:00
Dirk-Jan C. Binnema
4c0b7db3d8 store: add 'add_document' optimization, use it 
*Usually* we need Xapian's replace_document() API, but when we know a
document (message) is completely new, we can use the faster
add_document(). That is the case with the initial (re)indexing, when
start with an empty database.

Also a few smaller cleanups.
 2023-07-25 23:56:19 +03:00
Dirk-Jan C. Binnema
4d8ba5f579 index/scanner: implement i-node sorting 
On rotational devices (HDD) processing direntries is much faster when
doing so sorted by i-node for the dir-entries. This is an old
optimization (perhaps mu <= 1.6 or so?) that was implemented yet after
indexing changed, likely because my systems use SDDs instead!

But, let's restore that optimization; the sorting is fast enough that we
don't care for SDDs; on HDD it should be quite a bit faster.
 2023-07-25 22:39:12 +03:00
Dirk-Jan C. Binnema
b795242d5a message: use html-to-text scraper for html parts 
We were dumping the HTML-parts as-is in the Xapian indexer; however,
it's better to remove the html decoration first, and just pass the text.

We use the new built-in html->text scraper for that.
 2023-07-25 21:26:36 +03:00
Dirk-Jan C. Binnema
56b8fad89e utils: implement html-to-text 
Implement a crude html-to-text scraper function, to extract plain text
from html messages, so we can use it for indexing.
 2023-07-25 21:26:36 +03:00
Dirk-Jan C. Binnema
11c807f955 utils/readline: use fmt-based apis 2023-07-25 21:26:01 +03:00
Dirk-Jan C. Binnema
9580d11fef utils/result: add std::move version of Err 
Avoid a copy in some situations
 2023-07-25 21:26:01 +03:00
Dirk-Jan C. Binnema
72f43f11df lib: improve store error messages 
Use xapian_try_result
 2023-07-23 21:04:26 +03:00
Dirk-Jan C. Binnema
d374d94031 clang: avoid some build warnings 2023-07-23 21:04:26 +03:00
Dirk-Jan C. Binnema
7b38f094c4 migrate some more code to mu_format / join_paths 
Let's modernize a bit.
 2023-07-20 23:14:29 +03:00
Dirk-Jan C. Binnema
6ad5cccc53 store/index: and unit test for circular symlink 
Check that we bail out early
 2023-07-18 23:18:21 +03:00
Dirk-Jan C. Binnema
885903c496 index: limit length of maildir path to MaxTermLength 
This limit was already in place, but now we detect it a bit earlier (in
the indexer). We _could_ increase it (by using hashes for dirstamps), but
right now it's a good catch for circular symlinks.
 2023-07-18 23:18:21 +03:00
Dirk-Jan C. Binnema
cf6c5a36d7 utils: rework running system commands 
Use g_spawn and pass arguments, so we don't involve a shell that needs
escaping etc.

Improve error handling.
 2023-07-18 20:19:27 +03:00
Dirk-Jan C. Binnema
e8462e0204 lib/index: add rudimentary scanner test 
Make the defunct existing one a working test.
 2023-07-18 19:08:16 +03:00
Dirk-Jan C. Binnema
99a0eaaa76 lib/store: improve dirstamp / set_dirstamp code 
Modernize.
 2023-07-11 22:54:01 +03:00
Dirk-Jan C. Binnema
545494225a lib/contacts-cache: improve code 2023-07-11 22:54:01 +03:00
Dirk-Jan C. Binnema
6f69f5d482 utils/mu-regex: add move constructor 2023-07-11 22:54:01 +03:00
Dirk-Jan C. Binnema
f3bfdf5add lib/maildir: use mv for moving to avoid warnings 
using gio gives some (false, we assume) valgrind warnings, so for now
use 'mv' instead.

Also slightly update the code with some mu_format overtaking format.
 2023-07-10 23:17:06 +03:00
Dirk-Jan C. Binnema
18490a818d store/server: centralize docids-for-msgid 
No need for two near-identical impls

Remove some dead declarations.
 2023-07-10 23:17:06 +03:00
Dirk-Jan C. Binnema
0b4f7c4cbe lib: xapian-db/store: simplify 
No need for "pimpl" in xapian-db; keep it simple.
 2023-07-10 23:15:40 +03:00
Dirk-Jan C. Binnema
cc65b8b401 utils: add some more helpers for test code 
Creating and removing (temp) dirs, running mu commands.
 2023-07-10 23:15:40 +03:00
Dirk-Jan C. Binnema
904f64aa03 utils/result: add "unwrap" convenience function 2023-07-10 23:15:40 +03:00
Dirk-Jan C. Binnema
31f0c40893 migrate to fmt-based logging in some more places 
and improve logging.
 2023-07-08 10:30:36 +03:00
Dirk-Jan C. Binnema
4171fe14c3 store: removing trailing / for root_maildir 
Other code depends on that.
 2023-07-06 21:49:50 +03:00
Dirk-Jan C. Binnema
75a2cab3f1 message: don't check language without cld2 support 
No point in checking if we do not have the support.
 2023-07-06 21:49:50 +03:00
Dirk-Jan C. Binnema
82235b9d49 fmt: more update to use new fmt-based APIs 2023-07-06 21:49:50 +03:00
Dirk-Jan C. Binnema
91c5a9bac5 store: only write last-change _once_ 
xapian-db does it, no longer needed in store.cc
 2023-07-06 21:49:50 +03:00
Dirk-Jan C. Binnema
e48485cb58 sexp: use fmt for parsing_error 
Should help with Apple clang build too.
 2023-07-05 23:40:36 +03:00
Dirk-Jan C. Binnema
aed1395c53 lib/utils: update library paths 2023-07-05 23:14:34 +03:00
Dirk-Jan C. Binnema
4920b56671 update to use fmt-based apis 
Not complete, but a first big stab converting users of Mu::Error and
various g_warning & friends, format to the new libfmt-based APIs.
 2023-07-05 23:10:13 +03:00
Dirk-Jan C. Binnema
742ca33740 utils: rework Mu::Error and g_ logging macros with fmt 
A bit more C++ template magic to remove a lot of code.
 2023-07-05 23:07:11 +03:00
Dirk-Jan C. Binnema
e6ab61d233 thirdparty: add fmt 
Add libfmt for formatting strings. With C++23, perhaps we can remove
this. Let's see.
 2023-07-05 23:07:11 +03:00
Dirk-Jan C. Binnema
5c878558e0 thirdparty: move {optional,expected}.hpp to tl/ 
Follow upstream naming.
 2023-07-05 23:07:11 +03:00
Dirk-Jan C. Binnema
c5f3894374 message: double-check part-type in Multipart 
Avoid exception for some messages (unsigned messages with signed
attachment).
 2023-07-05 20:39:47 +03:00
Dirk-Jan C. Binnema
867f3205e2 lib/xapian-db: add missing include unordered_map 2023-07-02 10:08:58 +03:00
Dirk-Jan C. Binnema
c4b5795328 cli: support --ignored-address for init command 
Allow for skipping unwanted addresses (such as 'noreply') from the
contacts cache.
 2023-07-02 10:00:40 +03:00
Dirk-Jan C. Binnema
5dc41ed811 lib: support ignoring addresses for contacts-cache 
Skip annoying 'noreply' & friends.
 2023-07-02 00:06:56 +03:00
Dirk-Jan C. Binnema
c6fff6a281 all: update for API changes (config etc.) 
Use the new & improved APIs.
 2023-07-02 00:06:56 +03:00
Dirk-Jan C. Binnema
2acc1c2271 lib/contacts-cache: rework using config 
Use the new config class to pass information and serialization.
 2023-07-02 00:06:56 +03:00
Dirk-Jan C. Binnema
3791d0c375 lib/store: rework to use xapian-db / config 
Simplifies the implementation.
 2023-07-02 00:06:56 +03:00
Dirk-Jan C. Binnema
c28fde9155 lib: create mu-xapian-db, mu-config 
XapianDb is a fairly thing wrapper around Xapian, which handles locking,
exception handling and some tracking.

On top of that, Config add a configuration database for type /
introspectable configuration info.
 2023-07-02 00:06:56 +03:00
Dirk-Jan C. Binnema
426c1d35d0 thirdparty: bump tl::expected version 2023-06-27 22:43:29 +03:00
Dirk-Jan C. Binnema
fd7c011195 lib: update for api changes, update tests 
With the new Sexp. And expand unit tests.
 2023-06-27 00:53:23 +03:00
Dirk-Jan C. Binnema
ae9607530f utils/sexp: extend and improve tests 
Extend functionality for parsing; update documentation, tests.
 2023-06-27 00:52:07 +03:00
Dirk-Jan C. Binnema
0bdab5ae06 contact/display-name: quote "names" with '@' 
Apparently, some mail servers want this, and gnus doesn't handle it
automatically.
 2023-06-12 23:37:39 +03:00
Dirk-Jan C. Binnema
b9026c5042 message/fields: add some helpers for combination-fields 
such as "recip" and "contacts".
 2023-06-04 14:56:40 +03:00
Dirk-Jan C. Binnema
9bab2a4050 utils/sexp: add some convenience 
Add some convenience function for dealing the head/tail
Some whitespace / cosmetics
Add operators ==
Add operator !=
Update command handler for this.
 2023-06-04 14:56:40 +03:00
Dirk-Jan C. Binnema
eafc81f168 utils: fix non-cld2 build 2023-05-12 21:52:32 +03:00
Dirk-Jan C. Binnema
7f2eeb1010 message: try to detect body text language 
Try to detect the language of the e-mail body and make it searchable.
 2023-05-11 23:23:52 +03:00
Dirk-Jan C. Binnema
ad64093183 utils: implement language-detector 
A thing wrapper around CLD2 with which we can detect the language of an
e-mail message.
 2023-05-11 23:23:52 +03:00
Dirk-Jan C. Binnema
3a05dd8725 lib/utils: implement read_from_stdin 2023-04-29 23:01:12 +03:00
Dirk-Jan C. Binnema
e4af910d04 utils: implement g_cancellable_new_with_timeout 2023-04-29 23:01:12 +03:00
Dirk-Jan C. Binnema
36d89f400f message: remove control characters from subject 
And add a unit-test.

Fixes #2477
 2023-04-24 21:52:13 +03:00
Dirk-Jan C. Binnema
a3538e5ef1 store: small cleanups 
Remove readonly/inmemory from properties. Add read_only() accessor.
 2023-04-23 20:26:16 +03:00
Dirk-Jan C. Binnema
3c578d7a3b utils: add thread_dep for logger test 
Needed on some systems

Fixes #2473
 2023-04-14 12:48:03 +03:00
Dirk-Jan C. Binnema
b41955b82a mu-server: improve error message 2023-04-11 21:31:13 +03:00
Dirk-Jan C. Binnema
8762ac27dc mu: improve error numbers 
Ensure they match what mu expects.
 2023-04-11 20:47:48 +03:00
Dirk-Jan C. Binnema
ec942a988d tests: always log to file during tests 
And set the cachedir (for logging and other things) to a random tmpdir.
 2023-04-09 14:43:36 +03:00
Dirk-Jan C. Binnema
9b8a0146e0 logger: create logdir if non-existent 2023-04-09 14:43:17 +03:00
Dirk-Jan C. Binnema
e43452fa18 test: don't "uniqify" link names during test 2023-04-09 11:02:58 +03:00
Dirk-Jan C. Binnema
2aa0d20f39 mu-logger: Add lock for file-logger and stress test 
Add a lock to the file logger and add a unit-test to stress test it.
(It does fail without the lock, and passed with it).

Should help for: #2469.
 2023-04-08 10:26:15 +03:00
Dirk-Jan C. Binnema
026a19bcfa message: allow extracting message parts to file 
And add unit-test.

Fixes #2467
 2023-04-06 07:19:26 +03:00
Dirk-Jan C. Binnema
cd23e6015d mu: allow relative message paths in mu commands 2023-04-06 07:19:26 +03:00
Dirk-Jan C. Binnema
ff1e06c7c8 mu-find: restore JSON output 
Fixes #2453
 2023-03-30 18:12:33 +03:00
Dirk-Jan C. Binnema
39101cf583 mu-find: brush off --format=links unit test 
Enable it by default, and modernize it a little bit
 2023-03-28 22:26:56 +03:00
Dirk-Jan C. Binnema
f3c3811818 message: fix user-after-scope in test 2023-03-24 23:24:19 +02:00
Dirk-Jan C. Binnema
96e16034c7 contact: fix quoting 
Ensure that addresses with commas are quoted. Don't 2047-encode though.

Update tests.
 2023-03-14 06:18:56 +02:00
Dirk-Jan C. Binnema
617c8129ae lib/thirdparty: update tabulate to 1.5.0 2023-03-03 18:47:48 +02:00
Dirk-Jan C. Binnema
6f0e295326 lib/thirdparty: update CLI11 to 2.3.2 2023-03-03 18:46:48 +02:00
Dirk-Jan C. Binnema
9d0fe0e5d4 lib/thirdparty: update expected/optional 2023-03-03 18:41:45 +02:00
Dirk-Jan C. Binnema
413a2564b6 server: don't encode addresses in contacts_handlers 
Seems it doesn't always play nice with composer
 2023-02-28 23:23:52 +02:00
Dirk-Jan C. Binnema
a50357966d mu: add müller test-case 2023-02-28 22:42:55 +02:00
Dirk-Jan C. Binnema
bd5ae78c49 mime-object: add missing include 2023-02-23 18:38:18 +02:00
Dirk-Jan C. Binnema
6da223fcb6 mu-message-part: rework cooked_filename a bit 
Improve working, add a 'minimal' mode, and add some unit tests.
 2023-02-19 10:47:50 +02:00
Dirk-Jan C. Binnema
3d4805de0c server: small output improvement 
For human users.
 2023-02-17 10:20:14 +02:00
Dirk-Jan C. Binnema
76fedf4d64 mu-server: make :update for mkdir optional 
It's useful to do "opportunistic" mkdir form mu4e; don't send updates in
that case (i.e. with :update nil)
 2023-02-12 11:16:47 +02:00
Dirk-Jan C. Binnema
2255769de7 build: bump schema version, update contacts-cache 
Bump schema version to trigger update after we added html processing.

Since we're updating the database schema version, let's use the
opportunity to remove outdated data from the contacts-cache.
 2023-01-31 23:41:57 +02:00