Make external API of class/module to be smaller, explicitely mark
all internal functions. Also annotate methods that are implemented
as the part of the parent class interface.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
This will allow our callers who are capable of dealing with
readonly folders to properly detect this condition and act
accordingly.
One example is Gmail's "Chats" folder that is read-only,
but contains logs of the quick chats.
Minor Changelog improvements.
Tested-by: Abdó Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
As reported by James Cook, we would not check the fingerprint of the SSL
server, as we were looking for the 'ssl' module in locals() rather than
globals(). Ooops!
Rather than using globals() though, I simply remove the by-now
superfluous check. We now rely on python2.6 and we unconditionally
import the SSL module in any case, so it needs to be there.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
'set' is builtin since python2.6, so remove the imports. Also 'ssl' exists
since 2.6 and has everything we need, so no need for conditional import
tests here anymore.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Pass through the 'force' argument from selectro() to select() so that it
can also enforce a new SELECT even if we already are on that folder.
Also change the default parameter from '0' to 'False' to make clear that
this is a Bool.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The default parameter value was "None", and we were comparing that
directly to the imaplib2 value of is_readonly which is False or True, so
the comparison always returned "False".
Fix this by setting the default parameter to "False" and not
"None". Also convert all users of that function to use False/True.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Python 2.5 has no ssl module, and we can therefor not get the server
certificate for fingerprint verification. Add a check that disables
fingerprint verification for python 2.5.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
imaplib2 has changed internally to use self.sock for its ssl socket when
it used to be sslobj. Reflect that change.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
If we connect to a SSL server (not STARTTLS) and no CA cert has been
specified for verification, we check the configured SSL fingerprint and
bail out in case it has not been set yet, or it does not match.
This means one more mandatory option for SSL configuration, but it
improves security a lot.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Imapserver.acquireconnection will succeed even whent the server connection
has been terminated and the first IMAP operation will throw an exception.
Often this is the folder SELECT operation (e.g. after an idle timeout), as
has been reported by John Wiegley. Catch this case and throw an
OfflineImapError with severity FOLDER_RETRY to notify consumers that they
are supposed to retry.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
imaplib renamed self.sslobj to self.sock and our overriden open()
functions were failing for that reason when updating imaplib2 to
v2.28. It turns out that all of our custom initializations are being
done by stock imaplib2 now anyway, so there is no need to override them
anymore. This lets us simplify the code we have to worry about.
Move the verifycert() function to the imapserver.py file, it is now a
callback function that is being handed to imaplib from there, so it
makes sense to also define it in our imapserver function...
(this also lets us easily make use of the verifycert function in the
starttls case in the future)
TODO: we need to examine if and why we still need to override the
select() function, it is the only reason why we still wrap the IMAP4
classes.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We set an imapobj.mustquote which apparently was used in previous
incarnations of imaplib or imaplib2, however, nothing in our codebase
makes use of that. So let us remove it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
getselectedfolder was using a cached variable that we were setting in
select(), but sometimes the IMAP4 instance got into the SELECTED state
without explicitely select()ing, it seems, and our variable was unset.
Let us just use the self.mailbox variable that imaplib2 is setting when
select()ing rather than doing our own caching. Also remove the part
where we were setting the cache.
Just access self.state rather than looking up self.state via
self.getstate() every time, it is just an unnecessary layer of
redirection.
Original-patch-by: Arnaud Fontaine <arnau@debian.org>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We currently don't care about expiration dates of the servers SSL
certificate. This patch adds a check that fails Cert verification when
it is past its due date. There is no way or option to override this
check.
Unfortunately we only seem to be able to get SSL certificate data when
we passed in a CA cert file? How do we get that date when we don't have
a ca cert file?
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Raise OfflineImapError with severity REPO explaining that the connection failed.
Before, no valuable information was given to the user.
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Throw an OfflineImapError when SELECTing a folder is unsuccessful and
bail out with a FOLDER serverity. In accounts.py catch all
OfflineImapErrors and either just log the error and skip the folder or
bubble it up if it's severe.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
* IMAP4_Tunnel constructor should support base-class arguments, in
order to support the timeout argument.
* IMAP4_Tunnel needs to store the member IMAP4.host, which is normally
done in IMAP4.open().
* Update IMAP4_Tunnel.read() and IMAP4_Tunnel.send(). We turn on
nonblocking mode for these sockets, so we can return immediately
with whatever data is available.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
The three classes with names starting with UsefulIMAP4 were used for two
purposes, to include the UsefulIMAPMixIn class and to implement various
system-specific kludges. None of these kludges remain, so it is cleaner
to include UsefulIMAPMixIn directly in imaplibutil and forget about them
for good.
Signed-off-by: Vincent Beffara <vbeffara@ens-lyon.fr>
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Now that we do not need any system-specific hack, the three UsefulIMAP4
classes have become empty and should be removed. This implies importing
UsefulIMAPMixIn directly from the classes defined in imaplibutil.
Prepare this change by moving the code into imaplibutil.py. Functionally
this is a no-op.
Signed-off-by: Vincent Beffara <vbeffara@ens-lyon.fr>
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
For read(), the imaplib2 version seems to work perfectly well. The
others aren't used any more, either by imaplib2, nor by us, so we may
as well get rid of them.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
The biggest change here is that imapobj.untagged_responses is no
longer a dictionary, but a list. To access it, I use the semi-private
_get_untagged_response method.
* offlineimap/folder/IMAP.py (IMAPFolder.quickchanged,
IMAPFolder.cachemessagelist): imaplib2 now explicitly removes its
EXISTS response on select(), so instead we use the return values from
select() to get the number of messages.
* offlineimap/imapserver.py (UsefulIMAPMixIn.select): imaplib2 now
stores untagged_responses for different mailboxes, which confuses us
because it seems like our mailboxes are "still" in read-only mode when
we just re-opened them. Additionally, we have to return the value
from imaplib2's select() so that the above thing works.
* offlineimap/imapserver.py (UsefulIMAPMixIn._mesg): imaplib2 now
calls _mesg with the name of a thread, so we display this
information in debug output. This requires a corresponding change to
imaplibutil.new_mesg.
* offlineimap/imaplibutil.py: We override IMAP4_SSL.open, whose
default arguments have changed, so update the default arguments. We
also subclass imaplib.IMAP4 in a few different places, which now
relies on having a read_fd file descriptor to poll on.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
imaplib2 has slightly different semantics than standard imaplib, so
this patch will break the build, but I thought it was helpful to have it as
a separate commit.
Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Read() should return empty string when EOF happen, instead of looping
forever. This is the right semantics of read(), and a wrapped version
should not change it.
If you read the read(2) system call manpage, it tells you that when EOF
is seen, return value is 0; it does not say
``loop forever when EOF happen''.
After the EOF detection is patched you can see the
following exception:
WARNING: ERROR attempting to copy message 344 for account Gmail:Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line 282, in copymessageto
message = self.getmessage(uid)
File "/usr/lib/pymodules/python2.6/offlineimap/folder/IMAP.py", line 216, in getmessage
initialresult = imapobj.uid('fetch', '%d' % uid, '(BODY.PEEK[])')
File "/usr/lib/python2.6/imaplib.py", line 753, in uid
typ, dat = self._simple_command(name, command, *args)
File "/usr/lib/python2.6/imaplib.py", line 1060, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "/usr/lib/python2.6/imaplib.py", line 890, in _command_complete
raise self.abort('command: %s => %s' % (name, val))
abort: command: UID => socket error: EOF
Signed-off-by: Bao Haojun <baohaojun@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This seems to prevent mysterious hangs with SSL imap servers (especially
gmail?) and does not harm in any case. So let us enable keep-alive
messages for ssl connections.
Our thread pool should be made more robust against closed SSL
connections (which do not always seem to raise Exceptions), and not
deadlock while waiting for resources or data that will never arrive.
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Gábor Melis <mega@retes.hu>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Signed-off-by: Thomas Jost <schnouki@schnouki.net>
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
The current code path checked the CA cert host name, even if we did not
specify a CA cert file to use. Make the host name check dependent on a
CA cert file.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
The latter is shorter and looks nicer. UIBase was a very weird class
name for something that is "user visible". We don't need to use (or
see) it from higher level code for most of the code now.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Previously, we did not check at all the authenticy and validity of
the SSL server we connected to. This is bad as it allows
man-in-the-middle attacks etc. This patch remedies the situation
somewhat.
If we specify a sslcacertfile= setting in the Repository section,
validate the server cert (on python>=2.6 or abort with python<=2.5).
As before, no certificate check is performed without that option.
In the future, the hostname check should be made optional and also
a mutt-lick "accept this certificate forever" thing should be
implemented.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Move them into the correct classes, overriding the open() function.
This is what we intent to do anyway, so do it in a clean way.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Frankly, the original code doesn't even make much sense, and
moreover it's not forward compatible with python 2.6, and
furthermore:
DeprecationWarning: socket.ssl() is deprecated.
So, this commit is just a temporary fix.
Signed-off-by: Michael Witten <mfwitten@mit.edu>
Added WrappedIMAP4_SSL class to help fix up performance of SSL
Standard imaplib.py is really bad with this, since it reads one
character at a time.
Reported by Aaron Kaplan at
http://lists.complete.org/offlineimap@complete.org/2008/01/msg00012.html.gz
He wrote:
I just noticed that the version of offlineimap I've been using
(3.99.17) is well over four years old. How time flies. I haven't
had any problems with it, but out of curiosity I decided to pull in
5.99.2 from the fedora repository. It turns out to take
consistently over twice as long as the old version to sync the same
account. Is this expected?
He tracked it down at
http://lists.complete.org/offlineimap@complete.org/2008/02/msg00012.html.gz
The following changeset is the one responsible for the difference in
speed I was noticing between the imaplib.py that was packaged with
older versions of offlineimap and the one that comes with python:
* /offlineimap/head: changeset 169
More optimizations -- this time fix readline() to not work
character-by-character!
Eygene Ryabinkin
Sebastian Spaeth
Nicolas Sebrecht
Ethan Glasser-Camp
Vincent Beffara
Haojun Bao
Gábor Melis
Thomas Jost
Toke Høiland-Jørgensen
John Goerzen
James Bunton
Michael Witten