\input texinfo.tex @c -*-texinfo-*- @c %**start of header @setfilename mu-guile.info @settitle mu-guile user manual @documentencoding utf-8 @c %**end of header @dircategory The Algorithmic Language Scheme @direntry * mu-guile manual: (mu-guile). Guile bindings for the @t{mu} e-mail indexer/searcher. @end direntry @copying Copyright @copyright{} 2012 Dirk-Jan C. Binnema @quotation Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License.'' @end quotation @end copying @node Top @top mu4e Manual Welcome to @t{mu-guile}! @t{mu-guile} is a binding of the @t{mu} email search engine for the @t{guile} programming language. That means that you can write simple (and not so simple) programs to data-mine your e-mail database. @menu * Introduction:: * Getting started:: * Initializing mu-guile:: * Messages:: * Contacts:: * Attachments and other parts:: * Statistics:: * Plotting data:: Appendices * GNU Free Documentation License:: The license of this manual. @end menu @node Introduction @chapter Introduction @t{mu} is a program for indexing / searching e-mails stored in Maildirs. @t{guile} is the @emph{GNU Ubiquitous Intelligent Language for Extensions} - a version of the @emph{Scheme} programming language and the official GNU extension language. @t{mu-guile} connects @t{mu} and @t{guile}, and allows you to easily write programs for your specific needs. @node Getting started @chapter Getting started @menu * Installation:: * First steps:: @end menu This chapter walks you through the installation and some basic steps to ensure things work correctly. @node Installation @section Installation @t{mu-guile} is part of @t{mu} - by installing the latter, the former will be installed as well, provided that you have @t{guile} version 2.0 installed. At the time of writing, there are no distribution packages for @t{mu-guile}, so we are assuming installation from source packages. Installation follows the normal sequence of: @example $ tar xvfz mu-.tar.gz # use the specific version $ cd mu- $./configure @end example The output of @t{./configure} should end with a little text describing the detected versions of various libraries @t{mu} depends on. In particular, it should mention the @t{guile} version, e.g. @example Guile version : 2.0.3.82-a2c66 @end example If you don't see any line referring to @t{guile}, please install it, and run @t{configure} again. Note once more, @t{mu-guile} requires @t{guile} version 2.0. After a succesfull @t{./configure}, we can make and install the package: @example $ make && sudo make install @end example After this, @t{mu} and @t{mu-guile} should be installed. Note that the above instructions will normally install things under @t{/usr/local}; you may need to update @t{guile}'s @t{%load-path} to find it there. You can check the current load-path with the following: @example guile -c '(display %load-path)(newline)' @end example If necessary, you can add the @t{%load-path} by adding something like the following to your @file{~/.guile}: @lisp (set! %load-path (cons "/usr/local/share/guile/site/2.0" %load-path)) @end lisp Or, alternatively, you can set @t{GUILE_LOAD_PATH}: @example export GUILE_LOAD_PATH="/usr/local/share/guile/site/2.0" @end example Note, in both cases the directory should be the directory that contains the installed @t{mu.scm}; so if you installed @t{mu} under a different prefix, you must change the load-path accordingly. After this, you should be ready to go. @node First steps @section First steps Assuming @t{mu-guile} has been installed correctly (@ref{Installation}), and also assuming that you have already indexed your e-mail messages (if necessary, see the @t{mu-index} man-page), we are ready to start @t{mu-guile}; a session may look something like this: @verbatim $ guile GNU Guile 2.0.3.82-a2c66 Copyright (C) 1995-2011 Free Software Foundation, Inc. Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. This program is free software, and you are welcome to redistribute it under certain conditions; type `,show c' for details. Enter `,help' for help. scheme@(guile-user)> @end verbatim Now, we need to load some of the @t{mu-guile} modules: @verbatim scheme@(guile-user)> (use-modules (mu) (mu message)) @end verbatim This will load the basic modules for dealing with messages. After we have loaded the modules, we need to initialize the @t{mu-guile} system: @verbatim scheme@(guile-user)> (mu:initialize) @end verbatim When this is done, we can start querying the database. We discuss various methods and functions later in this manual, but just to give an example, let's get a list of the subjects of all messages that mention @emph{hello}: @verbatim scheme@(guile-user)> (for-each (lambda(msg) (format #t "Subject: ~a\n" (mu:subject msg))) (mu:message-list "hello")) @end verbatim Note, the multi-lines in the example are only for readability; since it can be a bit uncomfortable to type long sequences at the 'REPL' (the Guile command-line), we recommend using a tool like Geiser@footnote{@url{http://www.nongnu.org/geiser/}}. @node Initializing mu-guile @chapter Initializing mu-guile It is of course possible to write separate programs with @t{mu-guile}, but for now we'll do things @emph{interactively}, i.e., from the Guile-prompt (``@abbr{REPL}''). We start our @t{mu-guile} session by starting @t{guile}: @verbatim $ guile GNU Guile 2.0.3.82-a2c66 Copyright (C) 1995-2011 Free Software Foundation, Inc. Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. This program is free software, and you are welcome to redistribute it under certain conditions; type `,show c' for details. Enter `,help' for help. scheme@(guile-user)> @end verbatim Now, the first thing we need to do is load the @t{mu-guile} modules; currently, there are six available: @itemize @item @code{mu} - initialization, functions to get messages, contacts @item @code{mu message} - functions to deal with messages @item @code{mu contact} - functions to deal with contacts @item @code{mu part} - functions to deal with message-parts / attachments @item @code{mu stats} - some functions for doing statistics on your messages @item @code{mu plot} - functions to draw graphs from the statistics (requires @t{gnuplot} @end itemize Let's simply load all of them: @verbatim scheme@(guile-user)> (use-modules (mu) (mu message) (mu contact) (mu part) (mu stats) (mu plot)) @end verbatim Assuming you have installed everything correctly, the first time you do this, @t{guile} will probably respond by showing some message about compiling the modules, and then return to you with another prompt. Before we can do anything with @t{mu guile}, we need to initialize the system. This goes like this: @verbatim scheme@(guile-user)> (mu:initialize) @end verbatim Which will use the default location of @file{~/.mu}. Or, instead, if you keep your @t{mu} data in a non-standard place: @verbatim scheme@(guile-user)> (mu:initialize "/path/to/my/mu/") @end verbatim If all worked up until here, we're ready to go with @t{mu-guile} - hurray! In the next chapters we'll walk through all the modules. @node Messages @chapter Messages In this chapter, we discuss how to find messages and how to do various things with them. @menu * Finding messages:: * Message methods:: * Example - the longest subject:: @end menu @node Finding messages @section Finding messages Now we are ready to retrieve some messages from the system. There are two main functions to do this: @itemize @item @code{(mu:message-list [])} @item @code{(mu:for-each-message [])} @end itemize The first function, @code{mu:message-list} returns a list of all messages matching @t{}; if you leave @t{} out, it returns @emph{all} messages. For example, to get all messages with @emph{coffee} in the subject line, you could do: @verbatim scheme@(guile-user)> (mu:message-list "subject:coffee") $1 = (#< 9040640> #< 9040630> #< 9040570>) @end verbatim So, since apparently we have three messages matching @t{subject:coffee}, we get a list of three @t{} objects. Let's just use the @code{mu:subject} function ('method') provided by @t{} objects to retrieve the subject-field (more about methods in the next section). For your convenience, @t{guile} has saved the result of our last query in a variable called @t{$1}, so to get the subject of the first message in the list, we can do: @verbatim scheme@(guile-user)> (mu:subject (car $1)) $2 = "Re: best coffee ever!" @end verbatim The second function we mentioned, @code{mu:for-each-message}, executes some function for each message matched by the search expression (or @emph{all} messages if the search expression is omitted). @verbatim scheme@(guile-user)> (mu:for-each-message (lambda(msg) (display (subject msg)) (newline)) "subject:coffee") Re: best coffee ever! best coffee ever! Coffee beans scheme@(guile-user)> @end verbatim Using @code{mu:message-list} and/or @code{mu:for-each-message}@footnote{Implementation node: @code{mu:message-list} is implemented in terms of @code{mu:for-each-message}, not the other way around. Due to the way @t{mu} works, @code{mu:for-each-message} is rather more efficient than a combination of @code{for-each} and @code{mu:message-list}} and a couple of @t{} methods, together with what Guile/Scheme provides, should allow for many interesting programs. @node Message methods @section Message methods Now that we've seen how to retrieve lists of message objects (@code{}), let's see what we can do with such an object. @code{} defines the following methods that all take a single @code{} object as a parameter. We won't go into the exact meanings for all of these functions here - for the details about various flags / properties, please refer to the @t{mu-find} man-page. @itemize @item @code{(mu:bcc msg)}: the @t{Bcc} field of the message, or @t{#f} if there is none @item @code{(mu:body-html msg)}: : the html body of the message, or @t{#f} if there is none @item @code{(mu:body-txt msg)}: the plain-text body of the message, or @t{#f} if there is none @item @code{(mu:cc msg)}: the @t{Bcc} field of the message, or @t{#f} if there is none @item @code{(mu:date msg)}: the @t{Date} field of the message, or 0 if there is none @item @code{(mu:flags msg)}: list of message-flags for this message @item @code{(mu:from msg)}: the @t{From} field of the message, or @t{#f} if there is none @item @code{(mu:maildir msg)}: the maildir this message lives in, or @t{#f} if there is none @item @code{(mu:message-id msg)}: the @t{Message-Id} field of the message, or @t{#f} if there is none @item @code{(mu:path msg)}: the file system path for this message @item @code{(mu:priority msg)}: the priority of this message (either @t{mu:prio:low}, @t{mu:prio:normal} or @t{mu:prio:high} @item @code{(mu:references msg)}: the list of messages (message-ids) this message refers to in(mu: the @t{References:} header @item @code{(mu:size msg)}: size of the message in bytes @item @code{(mu:subject msg)}: the @t{Subject} field of the message, or @t{#f} if there is none. @item @code{(mu:tags msg)}: list of tags for this message @item @code{(mu:to msg)}: the sender of the message, or @t{#f} if there is none. @end itemize With these methods, we can query messages for their properties; for example: @verbatim scheme@(guile-user)> (define msg (car (mu:message-list "snow"))) scheme@(guile-user)> (mu:subject msg) $1 = "Re: Running in the snow is beautiful" scheme@(guile-user)> (mu:flags msg) $2 = (mu:flag:replied mu:flag:seen) scheme@(guile-user)> (strftime "%F" (localtime (mu:date msg))) $3 = "2011-01-15" @end verbatim There are a couple more methods: @itemize @item @code{(mu:header msg "")} returns an arbitrary message header (or @t{#f} if not found) -- e.g. @code{(header msg "User-Agent")} @item If you include the @t{mu contact} module, the @code{(mu:contacts msg [contact-type])} method (to get a list of contacts) is added. @xref{Contacts}. @item If you include the @t{mu part} module, the @code{((mu:parts msg)} and @code{(mu:attachments msg)} methods are added. @xref{Attachments and other parts}. @end itemize @node Example - the longest subject @section Example - the longest subject Now, let's write a little example -- let's find out what is the @emph{longest subject} of any e-mail messages we received in the year 2011. You can try this if you put the following in a separate file, make it executable, and run it like any program. @verbatim #!/bin/sh exec guile -s $0 $@ !# (use-modules (mu) (mu message)) (use-modules (srfi srfi-1)) (mu:initialize) ;; note: (subject msg) => #f if there is no subject (define list-of-subjects (map (lambda (msg) (or (mu:subject msg) "")) (mu:message-list "date:2011..2011"))) ;; see the mu-find manpage for the date syntax (define longest-subject (fold (lambda (subj1 subj2) (if (> (string-length subj1) (string-length subj2)) subj1 subj2)) "" list-of-subjects)) (format #t "Longest subject: ~s" longest-subject) (newline) @end verbatim There are many other ways to solve the same problem, for example by using an iterative approach with @code{mu:for-each-message}, but it should show how one can easily write little programs to answer specific questions about an e-mail corpus. @node Contacts @chapter Contacts We can retrieve the sender and recipients of an e-mail message using methods like @code{mu:from}, @code{mu:to} etc.; @xref{Message methods}. These functions return the list of recipients as a single string; however, often it is more useful to deal with recipients as separate objects. @t{mu-guile} offers some functionality for this in the @code{(mu contact)} module. Also, it adds some contact-related methods for @code{} objects. @menu * Contact functions and objects:: * All contacts:: * Utility functions:: * Example - mutt export:: @end menu @node Contact functions and objects @section Contact functions and objects @verbatim (use-modules (mu contact)) @end verbatim After loading the @code{(mu contact)}, message objects (@pxref{Messages}) gain the the @t{contacts}-methods: @code{(mu:contacts [])} The @t{} is a symbol, one of @code{mu:to}, @code{mu:from}, @code{mu:cc} or @code{mu:bcc}; this will then get the contact objects for the contacts of the corresponding type. If you leave out the contact-type (or specify @t{#t} for it, you will get a list of @emph{all} contact objects for the message. A contact object (@code{}) has two methods: @itemize @item @code{mu:name} returns the name of the contact, or #f if there is none @item @code{mu:email} returns the e-mail address of the contact, or #f if there is none @end itemize Let's get a list of all names and e-mail addresses in the 'To:' field, of messages matching 'book': @lisp (use-modules (mu) (mu message) (mu contact)) (mu:initialize) (mu:for-each-message (lambda (msg) (for-each (lambda (contact) (format #t "~a => ~a\n" (or (mu:email contact) "") (or (mu:name contact) "no-name"))) (mu:contacts msg mu:field:to))) "book") @end lisp This shows what the methods do, but for many uses, it would be more useful to have each of the contacts only show up @emph{once} - for that, please refer to @xref{All contacts}. @node All contacts @section All contacts Sometimes it may also be useful to look at @emph{all} the different contacts in the @t{mu} database -- that is, all the different contacts. This is useful, for example, when exporting contacts to some external format that can then be important in an e-mail program. To enable this, there is the function @code{mu:for-each-contact}, defined as @code{(mu:for-each-contact function [search-expression])}. This will aggregate the unique contacts from @emph{all} messages matching @t{} (when it is left empty, it will match all messages in the database), and execute @t{function} for each of these contacts. The @t{function} receives an object of the type @t{}, which is a @emph{subclass} of the @t{} class discussed in @xref{Contact functions and objects}. @t{} objects expose the following additional methods: @itemize @item @code{(mu:frequency )}: returns the @emph{number of times} this contact occured in one of the address fields @item @code{(mu:last-seen )}: returns the @emph{most recent time} the contact was seen in one of the address fields, as a @t{time_t} value @end itemize The method assumes an e-mail address is unique for a certain contact; if a certain e-mail address occurs with different names, it uses the most recent non-empty name. @node Utility functions @section Utility functions To make dealing with contacts even easier, there are a number of utility functions that can save you a bit of typing. For converting contacts to some textual form, there is @code{(mu:contact->string format)}, which takes a contact and returns a text string with the given format. Currently supported formats are @t{"org-contact}, @t{"mutt-alias"}, @t{"mutt-ab"}, @t{"wanderlust"} and @t{"plain"}. @node Example - mutt export @section Example - mutt export Let's see how we could export the addresses in the @t{mu} database to the addressbook format of the venerable @t{mutt}@footnote{@url{http://www.mutt.org/}} e-mail client. The addressbook format that @t{mutt} uses is a sequence of lines that look something like: @verbatim alias [] "<" ">" @end verbatim Anyway, there is the function @code{(mu:contact->string format)} that we can use to do the conversion. We may want to focus on people with whom we have frequent correspondence; so we may want to limit ourselves to people we have seen at least 10 times in the last year. It is a bit hard to @emph{guess} the nick name for e-mail contacts, but @code{mu:contact->string} tries something based on the name. You can always adjust them later by hand, obviously. @lisp #!/bin/sh exec guile -s $0 $@ !# (use-modules (mu) (mu message) (mu contact)) (mu:initialize) ;; Get a list of contacts that were seen at least 20 times since 2010 (define (selected-contacts) (let ((addrs '()) (start (car (mktime (car (strptime "%F" "2010-01-01"))))) (minfreq 20)) (mu:for-each-contact (lambda (contact) (if (and (mu:email contact) (>= (mu:frequency contact) minfreq) (>= (mu:last-seen contact) start)) (set! addrs (cons contact addrs))))) addrs)) (for-each (lambda (contact) (format #t "~a\n" (mu:contact->string contact "mutt-alias"))) (selected-contacts)) @end lisp This simple program could be improved in many ways; this is left as an excercise to the reader. @node Attachments and other parts @chapter Attachments and other parts To deal with @emph{attachments}, or, more in general @emph{MIME-parts}, there is the @t{mu part} module. @menu * Parts and their methods:: * Attachment example:: @end menu @node Parts and their methods @section Parts and their methods The module defines the @code{} class, and adds two methods to @code{} objects: @itemize @item @code{(mu:parts msg)} - returns a list @code{} objects, one for each MIME-parts in the message. @item @code{(mu:attachments msg)} - like @code{parts}, but only list those MIME-parts that look like proper attachments. @end itemize A @code{} object exposes a few methods to get information about the part: @itemize @item @code{(mu:name )} - returns the file name of the mime-part, or @code{#f} if there is none. @item @code{(mu:mime-type )} - returns the mime-type of the mime-part, or @code{#f} if there is none. @item @code{(mu:size )} - returns the size in bytes of the mime-part @end itemize Then, we may want to save the part to a file; this can be done using either: @itemize @item @code{(mu:save part )} - save a part to a temporary file, return the file name@footnote{the temporary filename is a predictable function of (user-id, msg-path, part-index)} @item @code{(mu:save-as )} - save part to file at path @end itemize @node Attachment example @section Attachment example Let's look at some small example. Let's get a list of the biggest attachments in messages about Luxemburg: @lisp #!/bin/sh exec guile -s $0 $@ !# (use-modules (mu) (mu message) (mu part)) (mu:initialize) (define (all-attachments expr) "Return a list of (name . size) for all attachments in messages matching EXPR." (let ((pairs '())) (mu:for-each-message (lambda (msg) (for-each (lambda (att) ;; add (filename . size) to the list (set! pairs (cons (cons (mu:name att) (or (mu:size att) 0)) pairs))) (mu:attachments msg))) expr) pairs)) (for-each (lambda (att) (format #t "~a: ~,1fKb\n" (car att) (exact->inexact (/ (cdr att) 1024)))) (sort (all-attachments "Luxemburg") (lambda (att1 att2) (< (cdr att1) (cdr att2))))) @end lisp As an exercise for the reader, you might want to re-rewrite the @code{all-attachments} in terms of @code{mu:message-list}, which would probably be a bit more elegant. @node Statistics @chapter Statistics @t{mu-guile} offers some convenience functions to determine various statistics about the messages in the database. @code{(mu:tabulate-messages [])} applies @t{} to each message matching @t{} (leave empty to match @emph{all} messages), and returns a associative list (a list of pairs) with each of the different results of @t{} and their frequencies. This can best be demonstrated with a little example. Suppose we want to know how many messages we receive per weekday: @lisp #!/bin/sh exec guile -s $0 $@ !# (use-modules (mu) (mu message) (mu stats) (mu plot)) (mu:initialize) ;; create a list like (("Sun" . 13) ("Mon" . 23) ...) (define weekday-table (mu:weekday-numbers->names (sort (mu:tabulate-messages (lambda (msg) (tm:wday (localtime (mu:date msg))))) (lambda (a b) (< (car a) (car b)))))) (for-each (lambda (elm) (format #t "~a: ~a\n" (car elm) (cdr elm))) weekday-table) @end lisp The function @code{weekday-table} uses @code{mu:tabulate-message} to get the frequencies per hour -- this returns a list of pairs: @verbatim ((5 . 2339) (0 . 2278) (4 . 2800) (2 . 3184) (6 . 1856) (3 . 2833) (1 . 2993)) @end verbatim We sort these pairs by the day number, and then apply @code{mu:weekday-numbers->names}, which takes the list, and returns a list where the day numbers are replace by there abbreviated name (in the current locale). Note, there is also @code{mu:month-numbers->names}. The script then outputs these numbers in the following form: @verbatim Sun: 2278 Mon: 2993 Tue: 3184 Wed: 2833 Thu: 2800 Fri: 2339 Sat: 1856 @end verbatim Clearly, Saturday is a slow day for e-mail... @node Plotting data @chapter Plotting data You can plot the results in the format produced by @code{mu:tabulate} with the @t{(mu plot)} module, an experimental module that requires the @t{gnuplot}@footnote{@url{http://www.gnuplot.info/}} program to be installed on your system. The @code{mu:plot} function takes the following arguments: @code{(mu:plot <x-label> <y-label> [<want-ascii>])} Here, @code{<data>} is a table of data in the format that @code{mu:tabulate} produces. @code{<title>}, @code{<x-label>} and @code{<y-lablel>} are, respectively, the title of the graph, and the labels for X- and Y-axis. Finally, if you pass @t{#t} for the final @code{<want-ascii>} parameter, a plain-text rendering of the graph will be produced; otherwise, a graphical window will be shown. An example should clarify how this works in practice; let's plot the number of message per hour: @lisp #!/bin/sh exec guile -s $0 $@ !# (use-modules (mu) (mu message) (mu contact) (mu stats) (mu plot)) (mu:initialize) (define (mail-per-hour-table) (sort (mu:tabulate-messages (lambda (msg) (tm:hour (localtime (mu:date msg))))) (lambda (x y) (< (car x) (car y))))) (mu:plot (mail-per-hour-table) "Mail per hour" "Hour" "Frequency" #t) @end lisp @verbatim Mail per hour Frequency 1200 ++--+--+--+--+-+--+--+--+--+-+--+--+--+-+--+--+--+--+-+--+--+--+--++ |+ + + + + + + "/tmp/fileHz7D2u" using 2:xticlabels(1) ******** 1100 ++ *** +* **** * * * 1000 *+ * **** * +* * * ****** **** * ** * * 900 *+ * * ** **** * **** ** * +* * * * ** * * ********* * ** ** * * 800 *+ * **** ** * * * * ** * * ** ** * +* 700 *+ *** **** * ** * * * * ** **** * ** ** * +* * * * **** * * ** * * * * ** * **** ** ** * * 600 *+ * **** * * * * ** * * * * ** * * * ** ** * +* * * ** * * * * * ** * * * * ** * * * ** ** * * 500 *+ * ** * * * * * ** * * * * ** * * * ** ** * +* * * ** **** *** * * * ** * * * * ** * * * ** ** * * 400 *+ * ** ** **** * * * * * ** * * * * ** * * * ** ** * +* *+ *+**+**+* +*******+* +* +*+ *+**+* +*+ *+ *+**+* +*+ *+**+**+* +* 300 ******************************************************************** 0 1 2 3 4 5 6 7 8 910 11 12 1314 15 16 17 1819 20 21 22 23 Hour @end verbatim @node GNU Free Documentation License @appendix GNU Free Documentation License @include fdl.texi @bye