* mm: add org-contacts support and some documentation

toys/mm/TODO

@@ -0,0 +1,62 @@
+* TODO
+
+** Bugs
+
+*** forward should take the attachments from the original
+*** database locks
+
+
+** Features i
+
+*** mark thread
+*** version check at startup
+*** documentation
+*** customizable bookmarks
+
+
+** Features ii
+
+*** bounce support
+*** sorting
+*** recursive ask-maildir
+*** display unread info at startup
+*** tool bars
+*** make links clickable
+*** colorize cited parts in view
+*** refiling-by-pattern
+*** window management
+*** inspect message (muile)
+*** message statistics
+*** include exchange handling / org integration
+*** integrate with org-contacts, bbdb
+
+
+
+* DONE
+** don't put updated messages at the end
+** mark region
+** editing drafts
+** combine download / re-index
+** toggleable message line wrapping in view
+** threading problems
+** missing? message
+** double messages (different docids though)
+** sometimes commands are not executed immediately
+** menus
+** flag information for content flags (encrypted, signed, has-attach)
+** use top bar
+** add footer to search results
+** q while adding headers
+** handle start/stopping server
+** hostname in draftname
+** make quick folders work for both jump and move
+** synchronize headers / view buffer point
+** make date format customizable
+** add :from-or-to
+** fix to:/cc: etc.
+** give info about target for move
+** --maildir=~/Maildir does not work anymore
+** message threading
+** view raw message
+** Sent Items
+** raw-mode / quit

toys/mm/mm-tech.org

@@ -0,0 +1,119 @@
+* mm
+
+I haven't written many =emacs-fu= posts recently, but that doesn't mean I
+haven't used emacs a lot. In fact, over the last few months I've been working on
+a bigger emacs-related project; the working title is =mm=, and it's an
+emacs-based e-mail client based on my [[http://www.djcbsoftware.nl/code/mu][mu]] maildir searcher/indexer that I
+discussed before. Even though I've been using =mm= myself for about two months,
+it's not really ready from prime-time yet - but I'm planning to have something
+ready this year still. 
+
+In this post, let me discuss some of the technical background, which may be
+interesting for others planning emacs-based front-ends to other tools.
+
+* How to make an emacs-based e-mail client
+
+  Emacs does not (as of today) support threads; but one way to do asynchronous
+  processing is to start another process, and let emacs deal with its
+  output. Let us see how...
+  
+** Getting output from =mu=
+  
+   One way to implement this (for =mu=), is to call the =mu= command-line tool
+   with some parameters and then parse its output. In fact, that is how some
+   tools do it, and it was my first approach - so I would invoke =mu find= and
+   then process the output in emacs (more about that in a minute).
+
+   However, then I realized that I'd need to load the entire e-mail Xapian
+   database for each invocation. Wouldn't it be nicer to keep a running =mu=
+   instance around?  Indeed, it would - so I implemented the =mu server=
+   sub-command. Now, when you run =mu server=, you get a shell, in which you can
+   give commands to =mu=, and which will then spit out the results. =mu server=
+   is not really meant for humans, but still I can use it manually, which is
+   great for debugging.
+
+   The next question was what format mu should use for its output for emacs to
+   process. Some other programs use =JSON= here, but I figured that it would be
+   easier (and possibly, more efficient) just to use emacs' native
+ =s-expressions= (=plists= to be precise). So that is what I did - and I can
+   easily evaluate them using =read-from-string=.
+  
+** Processing the output in emacs
+   
+   So, now let's look how we process the data from =mu server= in emacs.
+
+   First you create a process with, for example, =start-process=, and then
+   register a filter function for it, which will be invoked whenever the process
+   has some chunk of output. Something like:
+
+#+BEGIN_SRC emacs-lisp
+  (let ((proc (start-process <arguments>)))
+    (set-process-filter proc 'my-process-filter)
+    (set-process-sentinel proc 'my-process-sentinel))    
+#+END_SRC
+  
+   Note, the process sentinel is invoked when the process is terminated -- so there
+   you can clean things up.
+   
+   The function =my-process-filter= is a user-defined function that takes the
+   process and the chunk of output as arguments; in =mm= it looks something like
+   (pseudo-lisp):
+#+begin_SRC emacs-lisp
+(defun my-process-filter (proc str)
+  (setq mm/buf (concat mm/buf str)) ;; a global var updated with the new chunk
+  (when <we-have-received-a-full-expression>
+      <eat-expression-from mm/buf> 
+      <evaluate-expression>))
+#+end_src
+
+   The =<evaluate-expression>= de-multiplexes the s-expression we got. For example,
+   if the s-expression looks like an e-mail message header, it will be processed by
+   the header-handling function, which will append it to the header list. If the
+   s-expression looks like an error message, it will be reported to the user. And
+   so on.
+
+   Finally, let me try to answer some anticipated questions:
+   
+* Why does the world need yet another e-mail client?
+  
+  I don't the world needs another client, but I spend a *lot* of time
+  (professionally and privately) with my e-mail client, so I'd like it to behave
+  exactly like I want it to. An even more important goal for me was to write
+  some bigger program in emacs lisp, to better understand the language and its
+  idioms.
+
+  Specifically, when it comes to emacs-based clients, I have tried a few of
+  them. I never really got into =gnus=; I think it is by far the most popular
+  emacs-based mail client, but I found it hard to make behave the way I like it;
+  and in particular, I do not like its indirect approach to Maildirs.
+
+  Then, for some years I've been using Wanderlust; a fine, very feature-rich
+  client, but it shows its age - and especially with emacs-24, its cache file
+  got corrupted very often, requiring me to delete them etc. Still, you will
+  recognize some Wanderlust features in =mm/mu=.
+
+* Why not use [[http://notmuchmail.org/][notmuch]]? It seems similar.
+
+  There are certainly similarities with =notmuch= (and to some lesser extent,
+  with [[https://github.com/nicferrier/md][md]]) -- the overall architecture is similar: both are scanning maildirs,
+  using [[http://spruce.sourceforge.net/gmime/][GMime]] and [[http://xapian.org/][Xapian]]. (=mu= precedes =notmuch= by a year or so; but
+ =notmuch= was the first to add an emacs front-end).
+  
+  There are some differences as well. The main thing is that in =notmuch='s
+  philosophy, messages are usually not moved or deleted, but instead uses tags
+  in the database. While tags are nice, I like the 'state' to be in the messages
+  and the folders they are in, which it easy to synchronize with other email
+  clients (or synchronize with IMAP-folders through [[http://offlineimap.org/][OfflineIMAP]]). I'd like to be
+  able to move messages around, delete messages and so on. This is in fact one
+  of the things I liked in [[http://www.gohome.org/wl/][Wanderlust]], and wouldn't want to live without - so
+ =mu=/=mm= make this really easy.
+  
+  Clearly, the emacs-interface to =notmuch= is more mature, and the development
+  team is bigger, so I'd give it a try. On the other hand, if you happen to like
+  e-mail the way I like it, =mm= may be something for you.
+
+
+
+
+
+

toys/mm/mm-view.el

@@ -470,6 +470,36 @@ removing '^M' etc."
     (switch-to-buffer mm/view-buffer)
     (kill-buffer)))
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+;; functions for org-contacts
+
+(defun mm/org-contacts-from (name-or-email)
+  "Get a message field if we are in view mode; NAME-OR-EMAIL should
+be either 'name or 'email to get the corresponding field. If the
+field is not found, \"\" is returned. Use this with org-contact
+with a template like:
+
+  (\"c\" \"Contacts\" entry (file \"~/Org/contacts.org\")
+          \"* %(mm/org-contacts-from 'name)
+  :PROPERTIES:
+  :EMAIL: %(mm/org-contacts-from 'email)
+  :END:\")))
+
+See the `org-contacts' documentation for more details."
+  (with-current-buffer mm/view-buffer-name ;; hackish...
+    (unless (eq major-mode 'mm/view-mode)
+      (error "Not in mm/view mode."))
+    (unless mm/current-msg
+      (error "No current message."))
+    (let ((from (car-safe (plist-get mm/current-msg :from))))
+      (cond
+	((not from) "") ;; nothing found
+	((eq name-or-email 'name)
+	  (or (car-safe from) ""))
+	((eq name-or-email 'email)
+	  (or (cdr-safe from) ""))
+	(t (error "Not supported: %S" name-or-email))))))
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
 
 

toys/mm/mm.org

@@ -0,0 +1,38 @@
+* mm
+
+** Introduction
+
+   Welcome to *mu mail* - an emacs client for the [[http://www.djcbsoftware.nl/code/mu][mu]] maildir indexing/searching
+   tool. It turns mu into an e-mail-client.
+
+   Mu Mail has things in common with programs such as 'notmuch' and 'md', but -
+   in the opinion of it's author - it offers some unique features as
+   well. Basically, the mail handling (deleting, moving etc.) is inspired by
+ *Wanderlust* (another emacs-based e-mail client) and *dired*, while it takes
+   some cues from GMail with respect to being search-based. In practice this
+   means that mu mail provides a 'traditional' folder-based e-mail client, on
+   top of a search based back-end.
+
+** How does it work
+
+   While not necessarily interesting for all users of mu mail, for some it may
+   be interesting to know how mu mail does its job.
+
+   Since version 0.9.8, mu has a special =server= command, which drops you into
+   a command line where you can give certain commands to mu (see the =mu-server=
+   man page). While it would certainly be possible to have specific commands to
+   get lists of messages, move them, delete them etc., having a running instance
+   around gets rid of the startup time of mu and especially the message
+   database.
+
+   So, when running mu mail inside emacs, it fires up an instance of 'mu
+   server', and communicates with it as long as it runs.
+
+   mu mail shows its results only after the mu server reports their
+   completion. Still, the execution is asynchronous, so you do not need wait for
+   anything. We found that, for example, deleting messages is fast enough to
+   allow us to wait for the results. An alternative design would be to update
+   the user-interface already; so far we are quite content with the performance.
+
+   Regarding performance, showing large numbers (thousands) of message can still
+   be a bit slower than is desirable. This is an area for improvement still.