Ensure the metadata (dirstamps) for messages are only written / committed _after_
the accompanying message have been written / committed.
This avoids missing updates when indexing gets terminated unexpectedly.
The 'processed' statistic for indexing was more-or-less synonymous for
'updated'; let's change to something more useful, 'checked' which roughly means
the number of messages checked for updates (typically a cheap timestamp check).
Update all cc code using .clang-format; please do so as well for future PRs
etc.; emacs has a handy 'clang-format' mode to make this automatic.
For comparing old changes with git blame, we can disregard this one using
--ignore-rev
(see https://www.moxio.com/blog/43/ignoring-bulk-change-commits-with-git-blame )
Let's use the _current time_ (time(NULL)) instead of the dir-tstamp for a maildir;
this avoids re-indexing mail where the mails have a newer time, but their
directory hasn't (e.g. 'touch')
Experimental, let see how this works.
We got many reports where the 'lazy check' didn't work too well for
people... so make it a bit less lazy, so it'll just work for more
people.
In practice, never skip _directories_ unless they're leaf directories;
this avoids the mtime-does-not-bubble-up problem.
Some #includes were missing for the latter (but only noticeable on some
systems - e.g., build breaks on Cygwin).
So let's replace with something that works equally everywhere.
Fixes: #2060
The scanner had a timeout for scanning, which doesn't work too well with
machine with rel. fast disks / rel. slow machines. Which I don't happen
to have!
Let's remove the timeout, should help with getting unwanted timeouts
which would cut short the indexing.
- Move the lib/query/ stuff up a level into lib/
- Associate directly with the Query object
- Rework the Query object to be C++ rather than mixed with C
- Update all dependencies, tests
If the user has wants to postpone clean-up we shouldn't lock the
indexer waiting for something that will never happen. Clear the flag
event though we are actually skipping cleanup.
Implement a new message indexer consisting of a single-threaded scanner
and a multi-threaded indexer.
This allows for a number of optimizations as well as background
indexing, though this initial version should be behave similar to the
old indexer.
Dirk-Jan C. Binnema
Jonas Bernoulli
damon-kwok
Alex Bennée