lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/*
|
2022-02-22 21:58:31 +01:00
|
|
|
** Copyright (C) 2020-2022 Dirk-Jan C. Binnema <djcb@djcbsoftware.nl>
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
**
|
|
|
|
** This library is free software; you can redistribute it and/or
|
|
|
|
** modify it under the terms of the GNU Lesser General Public License
|
|
|
|
** as published by the Free Software Foundation; either version 2.1
|
|
|
|
** of the License, or (at your option) any later version.
|
|
|
|
**
|
|
|
|
** This library is distributed in the hope that it will be useful,
|
|
|
|
** but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
** Lesser General Public License for more details.
|
|
|
|
**
|
|
|
|
** You should have received a copy of the GNU Lesser General Public
|
|
|
|
** License along with this library; if not, write to the Free
|
|
|
|
** Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
|
|
|
|
** 02110-1301, USA.
|
|
|
|
*/
|
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
#ifndef __MU_UTILS_HH__
|
|
|
|
#define __MU_UTILS_HH__
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
#include <string>
|
2022-02-26 08:45:16 +01:00
|
|
|
#include <string_view>
|
2020-01-18 12:38:41 +01:00
|
|
|
#include <sstream>
|
2017-10-26 20:31:22 +02:00
|
|
|
#include <vector>
|
2020-06-26 18:21:04 +02:00
|
|
|
#include <chrono>
|
2022-04-16 12:07:46 +02:00
|
|
|
#include <memory>
|
2019-12-16 21:41:17 +01:00
|
|
|
#include <cstdarg>
|
2020-01-05 00:15:07 +01:00
|
|
|
#include <glib.h>
|
2020-01-23 23:21:53 +01:00
|
|
|
#include <ostream>
|
2020-06-26 18:21:04 +02:00
|
|
|
#include <iostream>
|
2021-10-18 11:22:26 +02:00
|
|
|
#include <type_traits>
|
2022-03-26 15:14:23 +01:00
|
|
|
#include <algorithm>
|
|
|
|
#include <numeric>
|
2022-06-16 21:49:46 +02:00
|
|
|
#include <regex>
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2022-04-28 21:49:45 +02:00
|
|
|
#include "mu-utils-format.hh"
|
|
|
|
#include "mu-option.hh"
|
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
namespace Mu {
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2020-01-05 00:15:07 +01:00
|
|
|
using StringVec = std::vector<std::string>;
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/**
|
|
|
|
* Flatten a string -- downcase and fold diacritics etc.
|
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
*
|
|
|
|
* @return a flattened string
|
|
|
|
*/
|
2021-10-20 11:18:15 +02:00
|
|
|
std::string utf8_flatten(const char* str);
|
|
|
|
inline std::string
|
|
|
|
utf8_flatten(const std::string& s)
|
|
|
|
{
|
|
|
|
return utf8_flatten(s.c_str());
|
|
|
|
}
|
2019-03-23 16:00:25 +01:00
|
|
|
|
2017-10-28 13:13:09 +02:00
|
|
|
/**
|
|
|
|
* Replace all control characters with spaces, and remove leading and trailing space.
|
|
|
|
*
|
|
|
|
* @param dirty an unclean string
|
|
|
|
*
|
|
|
|
* @return a cleaned-up string.
|
|
|
|
*/
|
2021-10-20 11:18:15 +02:00
|
|
|
std::string utf8_clean(const std::string& dirty);
|
2017-10-28 13:13:09 +02:00
|
|
|
|
2021-03-16 16:07:39 +01:00
|
|
|
/**
|
|
|
|
* Remove ctrl characters, replacing them with ' '; subsequent
|
|
|
|
* ctrl characters are replaced by a single ' '
|
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
*
|
|
|
|
* @return the string without control characters
|
|
|
|
*/
|
2021-10-20 11:18:15 +02:00
|
|
|
std::string remove_ctrl(const std::string& str);
|
2021-03-16 16:07:39 +01:00
|
|
|
|
2017-10-26 20:31:22 +02:00
|
|
|
/**
|
2022-02-22 21:58:31 +01:00
|
|
|
* Split a string in parts. As a special case, splitting an empty string
|
|
|
|
* yields an empty vector (not a vector with a single empty element)
|
2017-10-26 20:31:22 +02:00
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
* @param sepa the separator
|
|
|
|
*
|
|
|
|
* @return the parts.
|
|
|
|
*/
|
2021-10-20 11:18:15 +02:00
|
|
|
std::vector<std::string> split(const std::string& str, const std::string& sepa);
|
2017-10-26 20:31:22 +02:00
|
|
|
|
2022-03-19 09:58:13 +01:00
|
|
|
/**
|
|
|
|
* Split a string in parts. As a special case, splitting an empty string
|
|
|
|
* yields an empty vector (not a vector with a single empty element)
|
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
* @param sepa the separator
|
|
|
|
*
|
|
|
|
* @return the parts.
|
|
|
|
*/
|
|
|
|
std::vector<std::string> split(const std::string& str, char sepa);
|
|
|
|
|
2022-06-16 21:49:46 +02:00
|
|
|
/**
|
|
|
|
* Split a string in parts
|
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
* @param sepa the separator regex
|
|
|
|
*
|
|
|
|
* @return the parts.
|
|
|
|
*/
|
|
|
|
std::vector<std::string> split(const std::string& str, const std::regex& sepa_rx);
|
2022-03-19 09:58:13 +01:00
|
|
|
|
|
|
|
/**
|
|
|
|
* Join the strings in svec into a string, separated by sepa
|
|
|
|
*
|
|
|
|
* @param svec a string vector
|
|
|
|
* @param sepa separator
|
|
|
|
*
|
|
|
|
* @return string
|
|
|
|
*/
|
|
|
|
std::string join(const std::vector<std::string>& svec, const std::string& sepa);
|
|
|
|
static inline std::string join(const std::vector<std::string>& svec, char sepa) {
|
|
|
|
return join(svec, std::string(1, sepa));
|
|
|
|
}
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/**
|
2022-04-28 21:49:45 +02:00
|
|
|
* Parse a date string to the corresponding time_t
|
|
|
|
* *
|
2020-01-25 18:31:20 +01:00
|
|
|
* @param date the date expressed a YYYYMMDDHHMMSS or any n... of the first
|
2022-04-28 21:49:45 +02:00
|
|
|
* characters, using the local timezone.
|
2020-01-25 18:31:20 +01:00
|
|
|
* @param first whether to fill out incomplete dates to the start or the end;
|
|
|
|
* ie. either 1972 -> 197201010000 or 1972 -> 197212312359
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
*
|
2022-04-28 21:49:45 +02:00
|
|
|
* @return the corresponding time_t or Nothing if parsing failed.
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
*/
|
2022-04-28 21:49:45 +02:00
|
|
|
Option<int64_t> parse_date_time(const std::string& date, bool first);
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
|
|
|
/**
|
2017-11-04 12:30:23 +01:00
|
|
|
* 64-bit incarnation of time_t expressed as a 10-digit string. Uses 64-bit for the time-value,
|
|
|
|
* regardless of the size of time_t.
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
*
|
2017-11-04 12:30:23 +01:00
|
|
|
* @param t some time value
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
*
|
|
|
|
* @return
|
|
|
|
*/
|
2021-10-20 11:18:15 +02:00
|
|
|
std::string date_to_time_t_string(int64_t t);
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2021-11-10 20:32:46 +01:00
|
|
|
/**
|
|
|
|
* Get a string for a given time_t and format
|
|
|
|
* memory that must be freed after use.
|
|
|
|
*
|
|
|
|
* @param frm the format of the string (in strftime(3) format)
|
|
|
|
* @param t the time as time_t
|
|
|
|
* @param utc whether to display as UTC(if true) or local time
|
|
|
|
*
|
|
|
|
* @return a string representation of the time in UTF8-format, or empty in case
|
|
|
|
* of error.
|
|
|
|
*/
|
|
|
|
std::string time_to_string(const std::string& frm, time_t t, bool utc = false) G_GNUC_CONST;
|
|
|
|
|
2022-03-26 15:14:23 +01:00
|
|
|
|
2022-05-17 21:31:03 +02:00
|
|
|
/**
|
|
|
|
* Hack to avoid locale crashes
|
|
|
|
*
|
|
|
|
* @return true if setting locale worked; false otherwise
|
|
|
|
*/
|
|
|
|
bool locale_workaround();
|
|
|
|
|
2021-11-02 21:24:17 +01:00
|
|
|
|
2022-05-18 00:08:40 +02:00
|
|
|
/**
|
|
|
|
* Is the given timezone available? For tests
|
|
|
|
*
|
|
|
|
* @param tz a timezone, such as Europe/Helsinki
|
|
|
|
*
|
|
|
|
* @return true or false
|
|
|
|
*/
|
|
|
|
bool timezone_available(const std::string& tz);
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-16 12:07:46 +02:00
|
|
|
// https://stackoverflow.com/questions/19053351/how-do-i-use-a-custom-deleter-with-a-stdunique-ptr-member
|
|
|
|
template <auto fn>
|
|
|
|
struct deleter_from_fn {
|
|
|
|
template <typename T>
|
|
|
|
constexpr void operator()(T* arg) const {
|
|
|
|
fn(arg);
|
|
|
|
}
|
|
|
|
};
|
|
|
|
template <typename T, auto fn>
|
|
|
|
using deletable_unique_ptr = std::unique_ptr<T, deleter_from_fn<fn>>;
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-06-27 10:51:34 +02:00
|
|
|
using Clock = std::chrono::steady_clock;
|
|
|
|
using Duration = Clock::duration;
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2021-10-20 11:18:15 +02:00
|
|
|
template <typename Unit>
|
|
|
|
constexpr int64_t
|
|
|
|
to_unit(Duration d)
|
|
|
|
{
|
|
|
|
using namespace std::chrono;
|
|
|
|
return duration_cast<Unit>(d).count();
|
2020-06-26 18:21:04 +02:00
|
|
|
}
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2021-10-20 11:18:15 +02:00
|
|
|
constexpr int64_t
|
|
|
|
to_s(Duration d)
|
|
|
|
{
|
|
|
|
return to_unit<std::chrono::seconds>(d);
|
|
|
|
}
|
|
|
|
constexpr int64_t
|
|
|
|
to_ms(Duration d)
|
|
|
|
{
|
|
|
|
return to_unit<std::chrono::milliseconds>(d);
|
|
|
|
}
|
|
|
|
constexpr int64_t
|
|
|
|
to_us(Duration d)
|
|
|
|
{
|
|
|
|
return to_unit<std::chrono::microseconds>(d);
|
|
|
|
}
|
2020-06-27 10:51:34 +02:00
|
|
|
|
2020-11-26 08:23:52 +01:00
|
|
|
struct StopWatch {
|
2021-10-20 11:18:15 +02:00
|
|
|
using Clock = std::chrono::steady_clock;
|
|
|
|
StopWatch(const std::string name) : start_{Clock::now()}, name_{name} {}
|
|
|
|
~StopWatch()
|
|
|
|
{
|
2022-02-18 09:49:56 +01:00
|
|
|
const auto us{static_cast<double>(to_us(Clock::now() - start_))};
|
2021-10-20 11:18:15 +02:00
|
|
|
if (us > 2000000)
|
2022-02-18 09:49:56 +01:00
|
|
|
g_debug("%s: finished after %0.1f s", name_.c_str(), us / 1000000);
|
2021-10-20 11:18:15 +02:00
|
|
|
else if (us > 2000)
|
2022-02-18 09:49:56 +01:00
|
|
|
g_debug("%s: finished after %0.1f ms", name_.c_str(), us / 1000);
|
2021-10-20 11:18:15 +02:00
|
|
|
else
|
2022-02-18 09:49:56 +01:00
|
|
|
g_debug("%s: finished after %g us", name_.c_str(), us);
|
2021-10-20 11:18:15 +02:00
|
|
|
}
|
|
|
|
|
2021-11-02 21:24:17 +01:00
|
|
|
private:
|
2021-10-20 11:18:15 +02:00
|
|
|
Clock::time_point start_;
|
|
|
|
std::string name_;
|
2020-11-26 08:23:52 +01:00
|
|
|
};
|
|
|
|
|
2020-06-27 16:00:57 +02:00
|
|
|
/**
|
|
|
|
* See g_canonicalize_filename
|
|
|
|
*
|
|
|
|
* @param filename
|
|
|
|
* @param relative_to
|
|
|
|
*
|
|
|
|
* @return
|
|
|
|
*/
|
|
|
|
std::string canonicalize_filename(const std::string& path, const std::string& relative_to);
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/**
|
|
|
|
* Convert a size string to a size in bytes
|
|
|
|
*
|
|
|
|
* @param sizestr the size string
|
|
|
|
* @param first
|
|
|
|
*
|
2022-04-28 21:49:45 +02:00
|
|
|
* @return the size or Nothing if parsing failed
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
*/
|
2022-04-28 21:49:45 +02:00
|
|
|
Option<int64_t> parse_size(const std::string& sizestr, bool first);
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
|
|
|
/**
|
|
|
|
* Convert a size into a size in bytes string
|
|
|
|
*
|
|
|
|
* @param size the size
|
|
|
|
* @param first
|
|
|
|
*
|
|
|
|
* @return the size expressed as a string with the decimal number of bytes
|
|
|
|
*/
|
2021-10-20 11:18:15 +02:00
|
|
|
std::string size_to_string(int64_t size);
|
2020-01-05 00:15:07 +01:00
|
|
|
|
2020-01-18 12:38:41 +01:00
|
|
|
/**
|
|
|
|
* Convert any ostreamable<< value to a string
|
|
|
|
*
|
|
|
|
* @param t the value
|
|
|
|
*
|
|
|
|
* @return a std::string
|
|
|
|
*/
|
|
|
|
template <typename T>
|
2021-10-20 11:18:15 +02:00
|
|
|
static inline std::string
|
|
|
|
to_string(const T& val)
|
2020-01-18 12:38:41 +01:00
|
|
|
{
|
2021-10-20 11:18:15 +02:00
|
|
|
std::stringstream sstr;
|
|
|
|
sstr << val;
|
2020-01-18 12:38:41 +01:00
|
|
|
|
2021-10-20 11:18:15 +02:00
|
|
|
return sstr.str();
|
2020-01-18 12:38:41 +01:00
|
|
|
}
|
|
|
|
|
2022-04-28 21:49:45 +02:00
|
|
|
/**
|
|
|
|
* Consume a gchar and return a std::string
|
|
|
|
*
|
|
|
|
* @param str a gchar* (consumed/freed)
|
|
|
|
*
|
|
|
|
* @return a std::string, empty if gchar was {}
|
|
|
|
*/
|
|
|
|
static inline std::string
|
|
|
|
to_string_gchar(gchar*&& str)
|
|
|
|
{
|
|
|
|
std::string s(str?str:"");
|
|
|
|
g_free(str);
|
|
|
|
return s;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Lexicals Number are lexicographically sortable string representations
|
|
|
|
* of numbers. Start with 'g' + length of number in hex, followed by
|
|
|
|
* the ascii for the hex represntation. So,
|
|
|
|
*
|
|
|
|
* 0 -> 'g0'
|
|
|
|
* 1 -> 'g1'
|
|
|
|
* 10 -> 'ga'
|
|
|
|
* 16 -> 'h10'
|
|
|
|
*
|
|
|
|
* etc.
|
|
|
|
*/
|
|
|
|
std::string to_lexnum(int64_t val);
|
|
|
|
int64_t from_lexnum(const std::string& str);
|
|
|
|
|
2022-03-26 15:14:23 +01:00
|
|
|
/**
|
|
|
|
* Like std::find_if, but using sequence instead of a range.
|
|
|
|
*
|
|
|
|
* @param seq some std::find_if compatible sequence
|
|
|
|
* @param pred a predicate
|
|
|
|
*
|
|
|
|
* @return an iterator
|
|
|
|
*/
|
|
|
|
template<typename Sequence, typename UnaryPredicate>
|
|
|
|
typename Sequence::const_iterator seq_find_if(const Sequence& seq, UnaryPredicate pred) {
|
|
|
|
return std::find_if(seq.cbegin(), seq.cend(), pred);
|
|
|
|
}
|
|
|
|
|
2022-03-28 21:39:19 +02:00
|
|
|
/**
|
|
|
|
* Is at least pred(element) true for at least one element of sequence
|
|
|
|
*
|
|
|
|
* @param seq sequence
|
|
|
|
* @param pred a predicate
|
|
|
|
*
|
|
|
|
* @return true or false
|
|
|
|
*/
|
|
|
|
template<typename Sequence, typename UnaryPredicate>
|
|
|
|
bool seq_some(const Sequence& seq, UnaryPredicate pred) {
|
|
|
|
return seq_find_if(seq, pred) != seq.cend();
|
|
|
|
}
|
|
|
|
|
2022-03-26 15:14:23 +01:00
|
|
|
/**
|
|
|
|
* Create a sequence that has all element of seq for which pred is true
|
|
|
|
*
|
|
|
|
* @param seq sequence
|
|
|
|
* @param pred false
|
|
|
|
*
|
|
|
|
* @return sequence
|
|
|
|
*/
|
|
|
|
template<typename Sequence, typename UnaryPredicate>
|
|
|
|
Sequence seq_filter(const Sequence& seq, UnaryPredicate pred) {
|
|
|
|
Sequence res;
|
|
|
|
std::copy_if(seq.begin(), seq.end(), std::back_inserter(res), pred);
|
|
|
|
return res;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Create a sequence that has all element of seq for which pred is false
|
|
|
|
*
|
|
|
|
* @param seq sequence
|
|
|
|
* @param pred false
|
|
|
|
*
|
|
|
|
* @return sequence
|
|
|
|
*/
|
|
|
|
template<typename Sequence, typename UnaryPredicate>
|
|
|
|
Sequence seq_remove(const Sequence& seq, UnaryPredicate pred) {
|
|
|
|
Sequence res;
|
|
|
|
std::remove_copy_if(seq.begin(), seq.end(), std::back_inserter(res), pred);
|
|
|
|
return res;
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename Sequence, typename Compare>
|
|
|
|
void seq_sort(Sequence& seq, Compare cmp) { std::sort(seq.begin(), seq.end(), cmp); }
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Like std::accumulate, but using a sequence instead of a range.
|
|
|
|
*
|
|
|
|
* @param seq some std::accumulate compatible sequence
|
|
|
|
* @param init the initial value
|
|
|
|
* @param op binary operation to calculate the next element
|
|
|
|
*
|
|
|
|
* @return the result value.
|
|
|
|
*/
|
|
|
|
template<typename Sequence, typename ResultType, typename BinaryOp>
|
|
|
|
ResultType seq_fold(const Sequence& seq, ResultType init, BinaryOp op) {
|
|
|
|
return std::accumulate(seq.cbegin(), seq.cend(), init, op);
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename Sequence, typename UnaryOp>
|
|
|
|
void seq_for_each(const Sequence& seq, UnaryOp op) {
|
|
|
|
std::for_each(seq.cbegin(), seq.cend(), op);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2022-02-26 08:45:16 +01:00
|
|
|
/**
|
|
|
|
* Convert string view in something printable with %*s
|
|
|
|
*/
|
|
|
|
#define STR_V(sv__) static_cast<int>((sv__).size()), (sv__).data()
|
|
|
|
|
2020-06-26 18:21:04 +02:00
|
|
|
struct MaybeAnsi {
|
2021-10-20 11:18:15 +02:00
|
|
|
explicit MaybeAnsi(bool use_color) : color_{use_color} {}
|
|
|
|
|
|
|
|
enum struct Color {
|
|
|
|
Black = 30,
|
|
|
|
Red = 31,
|
|
|
|
Green = 32,
|
|
|
|
Yellow = 33,
|
|
|
|
Blue = 34,
|
|
|
|
Magenta = 35,
|
|
|
|
Cyan = 36,
|
|
|
|
White = 37,
|
|
|
|
|
|
|
|
BrightBlack = 90,
|
|
|
|
BrightRed = 91,
|
|
|
|
BrightGreen = 92,
|
|
|
|
BrightYellow = 93,
|
|
|
|
BrightBlue = 94,
|
|
|
|
BrightMagenta = 95,
|
|
|
|
BrightCyan = 96,
|
|
|
|
BrightWhite = 97,
|
|
|
|
};
|
|
|
|
|
|
|
|
std::string fg(Color c) const { return ansi(c, true); }
|
|
|
|
std::string bg(Color c) const { return ansi(c, false); }
|
|
|
|
|
|
|
|
std::string reset() const { return color_ ? "\x1b[0m" : ""; }
|
|
|
|
|
2021-11-02 21:24:17 +01:00
|
|
|
private:
|
2021-10-20 11:18:15 +02:00
|
|
|
std::string ansi(Color c, bool fg = true) const
|
|
|
|
{
|
|
|
|
return color_ ? format("\x1b[%dm", static_cast<int>(c) + (fg ? 0 : 10)) : "";
|
|
|
|
}
|
|
|
|
|
|
|
|
const bool color_;
|
2020-06-26 18:21:04 +02:00
|
|
|
};
|
|
|
|
|
2020-01-05 00:15:07 +01:00
|
|
|
/// Allow using enum structs as bitflags
|
2021-10-20 11:18:15 +02:00
|
|
|
#define MU_TO_NUM(ET, ELM) std::underlying_type_t<ET>(ELM)
|
|
|
|
#define MU_TO_ENUM(ET, NUM) static_cast<ET>(NUM)
|
2022-02-26 08:46:06 +01:00
|
|
|
#define MU_ENABLE_BITOPS(ET) \
|
|
|
|
constexpr ET operator&(ET e1, ET e2) \
|
|
|
|
{ \
|
|
|
|
return MU_TO_ENUM(ET, MU_TO_NUM(ET, e1) & MU_TO_NUM(ET, e2)); \
|
|
|
|
} \
|
|
|
|
constexpr ET operator|(ET e1, ET e2) \
|
|
|
|
{ \
|
|
|
|
return MU_TO_ENUM(ET, MU_TO_NUM(ET, e1) | MU_TO_NUM(ET, e2)); \
|
|
|
|
} \
|
|
|
|
constexpr ET operator~(ET e) { return MU_TO_ENUM(ET, ~(MU_TO_NUM(ET, e))); } \
|
|
|
|
constexpr bool any_of(ET e) { return MU_TO_NUM(ET, e) != 0; } \
|
|
|
|
constexpr bool none_of(ET e) { return MU_TO_NUM(ET, e) == 0; } \
|
2022-03-04 23:38:59 +01:00
|
|
|
constexpr bool one_of(ET e1, ET e2) { return (e1 & e2) == e2; } \
|
2022-02-26 08:46:06 +01:00
|
|
|
constexpr ET& operator&=(ET& e1, ET e2) { return e1 = e1 & e2; } \
|
2022-05-14 11:47:26 +02:00
|
|
|
constexpr ET& operator|=(ET& e1, ET e2) { return e1 = e1 | e2; } \
|
|
|
|
static_assert(1==1) // require a semicolon
|
2020-01-05 00:15:07 +01:00
|
|
|
|
|
|
|
/**
|
|
|
|
* For unit tests, assert two std::string's are equal.
|
|
|
|
*
|
|
|
|
* @param s1 string1
|
|
|
|
* @param s2 string2
|
|
|
|
*/
|
2022-02-21 22:21:04 +01:00
|
|
|
#define assert_equal(s1__,s2__) do { \
|
2022-02-22 21:58:31 +01:00
|
|
|
std::string s1s__(s1__), s2s__(s2__); \
|
2022-02-21 22:21:04 +01:00
|
|
|
g_assert_cmpstr(s1s__.c_str(), ==, s2s__.c_str()); \
|
|
|
|
} while(0)
|
|
|
|
|
|
|
|
|
2022-04-22 07:06:12 +02:00
|
|
|
#define assert_equal_seq(seq1__, seq2__) do { \
|
|
|
|
g_assert_cmpuint(seq1__.size(), ==, seq2__.size()); \
|
|
|
|
size_t n__{}; \
|
|
|
|
for (auto&& item__: seq1__) { \
|
|
|
|
g_assert_true(item__ == seq2__.at(n__)); \
|
|
|
|
++n__; \
|
|
|
|
} \
|
|
|
|
} while(0)
|
|
|
|
|
|
|
|
#define assert_equal_seq_str(seq1__, seq2__) do { \
|
|
|
|
g_assert_cmpuint(seq1__.size(), ==, seq2__.size()); \
|
|
|
|
size_t n__{}; \
|
|
|
|
for (auto&& item__: seq1__) { \
|
|
|
|
assert_equal(item__, seq2__.at(n__)); \
|
|
|
|
++n__; \
|
|
|
|
} \
|
|
|
|
} while(0)
|
|
|
|
|
2020-01-05 00:15:07 +01:00
|
|
|
/**
|
|
|
|
* For unit-tests, allow warnings in the current function.
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
void allow_warnings();
|
|
|
|
|
2022-04-09 18:14:48 +02:00
|
|
|
|
|
|
|
/**
|
|
|
|
* For unit-tests, a RAII tempdir.
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
struct TempDir {
|
|
|
|
/**
|
|
|
|
* Construct a temporary directory
|
|
|
|
*/
|
2022-04-28 21:49:45 +02:00
|
|
|
TempDir(bool autodelete=true);
|
2022-04-09 18:14:48 +02:00
|
|
|
|
|
|
|
/**
|
|
|
|
* DTOR; removes the temporary directory
|
|
|
|
*
|
|
|
|
*
|
|
|
|
* @return
|
|
|
|
*/
|
|
|
|
~TempDir();
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Path to the temporary directory
|
|
|
|
*
|
|
|
|
* @return the path.
|
|
|
|
*
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
const std::string& path() {return path_; }
|
|
|
|
private:
|
|
|
|
std::string path_;
|
2022-04-28 21:49:45 +02:00
|
|
|
const bool autodelete_;
|
2022-04-09 18:14:48 +02:00
|
|
|
};
|
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
} // namespace Mu
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
#endif /* __MU_UTILS_HH__ */
|