Commit Graph

10 Commits

Author SHA1 Message Date
djcb ebbe3ea023 mu: _XOPEN_SOURCE: fix typo 2018-03-10 13:05:44 +02:00
djcb 6fe67b354d lib/parser: fix month days
In the olden days, we stored dates like e.g. 20180131121234, and do a
lexicographical check. With that, we could use e.g. upper-limits
201802312359 for "all dates in Feb 2018", even if Feb doesn't have 31
days.

However, nowadays we use time_t values, and g_date_time_new_local raises
errors for non-existent days; easiest fix is to massage things a bit; so
let's do that.

Fixes issue #1197.
2018-02-17 18:07:13 +02:00
djcb 15ba4699ab lib/parser: use g_vasprintf, _XOPEN_SOURCE
Attempt to restore building on Cygwin.
2018-02-11 12:02:53 +02:00
djcb 6a0654c91b parser/utils: enforce 64-bit times on 32-bit platforms
don't assume a 64-bit platform.
2017-11-04 11:30:23 +00:00
djcb 57b5fe6156 mu: some optimizations
add fast-path for (common) plain-ascii. fix silly static misuse.

should improve indexing with some single-digit percentage.
2017-10-29 13:34:57 +02:00
djcb 55ffb524db tokenizer: clean unicode-aware 2017-10-28 14:13:09 +03:00
djcb fe18603843 parser: fix some post-c++14 code
don't require anything post c++14
2017-10-27 18:40:37 +03:00
djcb 7cd7d118e2 query-parser: support phrase queries 2017-10-26 21:31:22 +03:00
djcb 5e9cafea59 integrate new query parser 2017-10-25 23:50:17 +03:00
djcb b75f9f508b lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.

Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).

Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.

The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.

From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.

From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
      subject:/h.ll?o/
  will find subjects with hallo, hello, halo,  philosophy, ...

  As you can imagine, this can be a _heavy_ operation on the database,
  and might take quite a bit longer than a normal query; but it can be
  quite useful.
2017-10-24 22:55:35 +03:00