lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/*
|
|
|
|
** Copyright (C) 2017 Dirk-Jan C. Binnema <djcb@djcbsoftware.nl>
|
|
|
|
**
|
|
|
|
** This library is free software; you can redistribute it and/or
|
|
|
|
** modify it under the terms of the GNU Lesser General Public License
|
|
|
|
** as published by the Free Software Foundation; either version 2.1
|
|
|
|
** of the License, or (at your option) any later version.
|
|
|
|
**
|
|
|
|
** This library is distributed in the hope that it will be useful,
|
|
|
|
** but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
** Lesser General Public License for more details.
|
|
|
|
**
|
|
|
|
** You should have received a copy of the GNU Lesser General Public
|
|
|
|
** License along with this library; if not, write to the Free
|
|
|
|
** Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
|
|
|
|
** 02110-1301, USA.
|
|
|
|
*/
|
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
#ifndef __MU_UTILS_HH__
|
|
|
|
#define __MU_UTILS_HH__
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
#include <string>
|
2020-01-18 12:38:41 +01:00
|
|
|
#include <sstream>
|
2017-10-26 20:31:22 +02:00
|
|
|
#include <vector>
|
2019-12-16 21:41:17 +01:00
|
|
|
#include <cstdarg>
|
2020-01-05 00:15:07 +01:00
|
|
|
#include <glib.h>
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
namespace Mu {
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2020-01-05 00:15:07 +01:00
|
|
|
using StringVec = std::vector<std::string>;
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/**
|
|
|
|
* Flatten a string -- downcase and fold diacritics etc.
|
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
*
|
|
|
|
* @return a flattened string
|
|
|
|
*/
|
2019-03-23 16:00:25 +01:00
|
|
|
std::string utf8_flatten (const char *str);
|
|
|
|
inline std::string utf8_flatten (const std::string& s) { return utf8_flatten(s.c_str()); }
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2017-10-28 13:13:09 +02:00
|
|
|
|
|
|
|
/**
|
|
|
|
* Replace all control characters with spaces, and remove leading and trailing space.
|
|
|
|
*
|
|
|
|
* @param dirty an unclean string
|
|
|
|
*
|
|
|
|
* @return a cleaned-up string.
|
|
|
|
*/
|
|
|
|
std::string utf8_clean (const std::string& dirty);
|
|
|
|
|
|
|
|
|
2017-10-26 20:31:22 +02:00
|
|
|
/**
|
|
|
|
* Split a string in parts
|
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
* @param sepa the separator
|
|
|
|
*
|
|
|
|
* @return the parts.
|
|
|
|
*/
|
|
|
|
std::vector<std::string> split (const std::string& str,
|
|
|
|
const std::string& sepa);
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/**
|
|
|
|
* Quote & escape a string
|
|
|
|
*
|
|
|
|
* @param str a string
|
|
|
|
*
|
|
|
|
* @return quoted string
|
|
|
|
*/
|
|
|
|
std::string quote (const std::string& str);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Format a string, printf style
|
|
|
|
*
|
|
|
|
* @param frm format string
|
|
|
|
* @param ... parameters
|
|
|
|
*
|
|
|
|
* @return a formatted string
|
|
|
|
*/
|
2019-12-16 21:41:17 +01:00
|
|
|
std::string format (const char *frm, ...) __attribute__((format(printf, 1, 2)));
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Format a string, printf style
|
|
|
|
*
|
|
|
|
* @param frm format string
|
|
|
|
* @param ... parameters
|
|
|
|
*
|
|
|
|
* @return a formatted string
|
|
|
|
*/
|
|
|
|
std::string format (const char *frm, va_list args) __attribute__((format(printf, 1, 0)));
|
|
|
|
|
|
|
|
|
2020-01-19 16:23:24 +01:00
|
|
|
/**
|
|
|
|
* Quote a string -- put in "" and escape any special characters by putting '\'
|
|
|
|
* in front of them.
|
|
|
|
*
|
|
|
|
* @param str
|
|
|
|
*
|
|
|
|
* @return
|
|
|
|
*/
|
|
|
|
std::string quoted (const std::string& str);
|
|
|
|
|
|
|
|
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
/**
|
|
|
|
* Convert an ISO date to the corresponding time expressed as a string
|
|
|
|
* with a 10-digit time_t
|
|
|
|
*
|
|
|
|
* @param date
|
|
|
|
* @param first
|
|
|
|
*
|
|
|
|
* @return
|
|
|
|
*/
|
|
|
|
std::string date_to_time_t_string (const std::string& date, bool first);
|
|
|
|
|
|
|
|
/**
|
2017-11-04 12:30:23 +01:00
|
|
|
* 64-bit incarnation of time_t expressed as a 10-digit string. Uses 64-bit for the time-value,
|
|
|
|
* regardless of the size of time_t.
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
*
|
2017-11-04 12:30:23 +01:00
|
|
|
* @param t some time value
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
*
|
|
|
|
* @return
|
|
|
|
*/
|
2017-11-04 12:30:23 +01:00
|
|
|
std::string date_to_time_t_string (int64_t t);
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Convert a size string to a size in bytes
|
|
|
|
*
|
|
|
|
* @param sizestr the size string
|
|
|
|
* @param first
|
|
|
|
*
|
|
|
|
* @return the size expressed as a string with the decimal number of bytes
|
|
|
|
*/
|
|
|
|
std::string size_to_string (const std::string& sizestr, bool first);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Convert a size into a size in bytes string
|
|
|
|
*
|
|
|
|
* @param size the size
|
|
|
|
* @param first
|
|
|
|
*
|
|
|
|
* @return the size expressed as a string with the decimal number of bytes
|
|
|
|
*/
|
|
|
|
std::string size_to_string (int64_t size);
|
|
|
|
|
2020-01-05 00:15:07 +01:00
|
|
|
|
2020-01-18 12:38:41 +01:00
|
|
|
/**
|
|
|
|
* Convert any ostreamable<< value to a string
|
|
|
|
*
|
|
|
|
* @param t the value
|
|
|
|
*
|
|
|
|
* @return a std::string
|
|
|
|
*/
|
|
|
|
template <typename T>
|
|
|
|
static inline std::string to_string (const T& val)
|
|
|
|
{
|
|
|
|
std::stringstream sstr;
|
|
|
|
sstr << val;
|
|
|
|
|
|
|
|
return sstr.str();
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-01-05 00:15:07 +01:00
|
|
|
/**
|
|
|
|
*
|
|
|
|
* don't repeat these catch blocks everywhere...
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
|
|
|
|
#define MU_STORE_CATCH_BLOCK_RETURN(GE,R) \
|
|
|
|
catch (const MuStoreError& merr) { \
|
|
|
|
mu_util_g_set_error ((GE), \
|
|
|
|
merr.mu_error(), "%s", \
|
|
|
|
merr.what().c_str()); \
|
|
|
|
return (R); \
|
|
|
|
} \
|
|
|
|
|
|
|
|
|
|
|
|
#define MU_XAPIAN_CATCH_BLOCK \
|
|
|
|
catch (const Xapian::Error &xerr) { \
|
|
|
|
g_critical ("%s: xapian error '%s'", \
|
|
|
|
__func__, xerr.get_msg().c_str()); \
|
|
|
|
} catch (const std::runtime_error& re) { \
|
|
|
|
g_critical ("%s: error: %s", __func__, re.what()); \
|
|
|
|
} catch (...) { \
|
|
|
|
g_critical ("%s: caught exception", __func__); \
|
|
|
|
}
|
|
|
|
|
|
|
|
#define MU_XAPIAN_CATCH_BLOCK_G_ERROR(GE,E) \
|
|
|
|
catch (const Xapian::DatabaseLockError &xerr) { \
|
|
|
|
mu_util_g_set_error ((GE), \
|
|
|
|
MU_ERROR_XAPIAN_CANNOT_GET_WRITELOCK, \
|
|
|
|
"%s: xapian error '%s'", \
|
|
|
|
__func__, xerr.get_msg().c_str()); \
|
|
|
|
} catch (const Xapian::DatabaseError &xerr) { \
|
|
|
|
mu_util_g_set_error ((GE),MU_ERROR_XAPIAN, \
|
|
|
|
"%s: xapian error '%s'", \
|
|
|
|
__func__, xerr.get_msg().c_str()); \
|
|
|
|
} catch (const Xapian::Error &xerr) { \
|
|
|
|
mu_util_g_set_error ((GE),(E), \
|
|
|
|
"%s: xapian error '%s'", \
|
|
|
|
__func__, xerr.get_msg().c_str()); \
|
|
|
|
} catch (const std::runtime_error& ex) { \
|
|
|
|
mu_util_g_set_error ((GE),(MU_ERROR_INTERNAL), \
|
|
|
|
"%s: error: %s", __func__, ex.what()); \
|
|
|
|
\
|
|
|
|
} catch (...) { \
|
|
|
|
mu_util_g_set_error ((GE),(MU_ERROR_INTERNAL), \
|
|
|
|
"%s: caught exception", __func__); \
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
#define MU_XAPIAN_CATCH_BLOCK_RETURN(R) \
|
|
|
|
catch (const Xapian::Error &xerr) { \
|
|
|
|
g_critical ("%s: xapian error '%s'", \
|
|
|
|
__func__, xerr.get_msg().c_str()); \
|
|
|
|
return (R); \
|
|
|
|
} catch (const std::runtime_error& ex) { \
|
|
|
|
g_critical("%s: error: %s", __func__, ex.what()); \
|
|
|
|
return (R); \
|
|
|
|
} catch (...) { \
|
|
|
|
g_critical ("%s: caught exception", __func__); \
|
|
|
|
return (R); \
|
|
|
|
}
|
|
|
|
|
|
|
|
#define MU_XAPIAN_CATCH_BLOCK_G_ERROR_RETURN(GE,E,R) \
|
|
|
|
catch (const Xapian::Error &xerr) { \
|
|
|
|
mu_util_g_set_error ((GE),(E), \
|
|
|
|
"%s: xapian error '%s'", \
|
|
|
|
__func__, xerr.get_msg().c_str()); \
|
|
|
|
return (R); \
|
|
|
|
} catch (const std::runtime_error& ex) { \
|
|
|
|
mu_util_g_set_error ((GE),(MU_ERROR_INTERNAL), \
|
|
|
|
"%s: error: %s", __func__, ex.what()); \
|
|
|
|
return (R); \
|
|
|
|
} catch (...) { \
|
|
|
|
if ((GE)&&!(*(GE))) \
|
|
|
|
mu_util_g_set_error ((GE), \
|
|
|
|
(MU_ERROR_INTERNAL), \
|
|
|
|
"%s: caught exception", __func__); \
|
|
|
|
return (R); \
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/// Allow using enum structs as bitflags
|
|
|
|
#define MU_TO_NUM(ET,ELM) std::underlying_type_t<ET>(ELM)
|
|
|
|
#define MU_TO_ENUM(ET,NUM) static_cast<ET>(NUM)
|
|
|
|
#define MU_ENABLE_BITOPS(ET) \
|
|
|
|
constexpr ET operator& (ET e1, ET e2) { return MU_TO_ENUM(ET,MU_TO_NUM(ET,e1)&MU_TO_NUM(ET,e2)); } \
|
|
|
|
constexpr ET operator| (ET e1, ET e2) { return MU_TO_ENUM(ET,MU_TO_NUM(ET,e1)|MU_TO_NUM(ET,e2)); } \
|
|
|
|
constexpr ET operator~ (ET e) { return MU_TO_ENUM(ET,~(MU_TO_NUM(ET, e))); } \
|
|
|
|
constexpr bool any_of(ET e) { return MU_TO_NUM(ET,e) != 0; } \
|
|
|
|
constexpr bool none_of(ET e) { return MU_TO_NUM(ET,e) == 0; } \
|
|
|
|
static inline ET& operator&=(ET& e1, ET e2) { return e1 = e1 & e2;} \
|
|
|
|
static inline ET& operator|=(ET& e1, ET e2) { return e1 = e1 | e2;}
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* For unit tests, assert two std::string's are equal.
|
|
|
|
*
|
|
|
|
* @param s1 string1
|
|
|
|
* @param s2 string2
|
|
|
|
*/
|
|
|
|
void assert_equal(const std::string& s1, const std::string& s2);
|
|
|
|
/**
|
|
|
|
* For unit tests, assert that to containers are the same.
|
|
|
|
*
|
|
|
|
* @param c1 container1
|
|
|
|
* @param c2 container2
|
|
|
|
*/
|
|
|
|
void assert_equal (const StringVec& v1, const StringVec& v2);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* For unit-tests, allow warnings in the current function.
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
void allow_warnings();
|
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
} // namespace Mu
|
lib: implement new query parser
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
2017-10-24 21:55:35 +02:00
|
|
|
|
2020-01-05 00:15:07 +01:00
|
|
|
|
2019-12-16 21:41:17 +01:00
|
|
|
#endif /* __MU_UTILS_HH__ */
|