We got some errors when some of the key values exceeded the Xapian
maximum; in particular the message-id.
So make all the key-methods check, and truncate the message-id if
necessary.
We were transforming wild-card searches into regular-expression
searches; while that works, it's also significantly slower.
So, instead, special-case wildcards, and use the Xapian machinery for
wildcard queries.
In the olden days, we stored dates like e.g. 20180131121234, and do a
lexicographical check. With that, we could use e.g. upper-limits
201802312359 for "all dates in Feb 2018", even if Feb doesn't have 31
days.
However, nowadays we use time_t values, and g_date_time_new_local raises
errors for non-existent days; easiest fix is to massage things a bit; so
let's do that.
Fixes issue #1197.
For now, don't treat "and not" specially; this gets us back into a
somewhat working state. At some point, we probably _do_ want to
special-case and_not though (since Xapian supports it).
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
When you have multiple mu home directories, e.g. for the use case
detailed in my "Changing mu4e-{maildir,mu-home} from a context hook"
post to the mailing list it's quite inconvenient to have to hammer out
"mu --muhome=.. find .." every time you want to run some ad-hoc
command.
This allows me to set up a screen session where I do searches in mu
directory A in some screen panes, and searches in directory B in
others.
I initially called this MU_MUHOME but then I noticed that the perl
plugin has MUP_MU_HOME for analogous functionality, so I'm just
following its example.
The code I'm adding in mu-util.c is just a copy/paste & adjustment of
the same sort of already tested functionality in
mu_util_guess_maildir() just a few lines earlier.
djcb
Marcelo Henrique Cerri
Yuri D'Elia
Sean 'Shaleh' Perry
Ævar Arnfjörð Bjarmason