e-mail sync options?

[Joshua Judson Rosen]

This is going to start-off silly-sounding, but bear with me:

My wife called and told me, a few months back, `Wikipedia is down!'.
It took me a minute to figure out that she actually did mean that
*Wikipedia* was down--not her laptop's WiFi, not our WAP, not our
residential-grade DSL (*again*...), but Wikipedia.

Of course, part of the reason that I was so befuddled by this was that
*I* hadn't noticed. I'd consulted Wikipedia several times while it was
down. Well, I'd consulted the Wikipedia *content*--not the web-site.

When I bought my WikiReader, I had expected it to give me access
to Wikipedia when *I* was offline, but it turned out that it also
served me well when *Wikipedia* went offline. So, now I'm thinking:
what other parts of the Internet can I download and carry around
in my (offline) pocket?

I know what you're thinking, and--for many aspects of the Internet--
you're right: that's just silly. But, for some things, is *does* make sense:
e-mail, for example. Especially if I take the idea back to when *I'm*
the one offline (like when I'm on a trip to Maine, working on one of my
FOSS projects, and I realise that I need to dig something out of an
e-mail thread to get past a block).

Now, obviously I can't *send* or *receive* *new* e-mail unless I'm online,
but I can *read* and *write* e-mail while offline. So, that's what
I wanted to ask you all about (what better place to ask about e-mail
than on an e-mail list? And I guess I'll take the answer *online*,
for the time being...).

The platforms where I care about this are on my laptop (running Debian),
my desktop (idem), my server (idem), and on my FreeRunner (which is similar
enough, inside).


_What I want_:

I'd like to be able to offline have access to my entire e-mail store
from all devices--be able to search through and read messages, write
messages and queue them for sending, move messages between folders,
set/change flags on messages--and then have everything sync-up every
which way when I go online: I want messages that have been queued for
sending to be sent, I want messages that were already in the store(s)
to have changes to their flags propagated, and I want new inbound
messages to be retrieved and cached locally. Basically, I want
everything mirrored everywhere, and I don't want to have to think
about *where* something last changed.

I could just write a suite of code to implement this myself,
but maybe someone can save me the effort by pointing me to a canned
solution :)


_What I've tried_:

Right now, I read my mail with Gnus in a Screen session on my server,
talking to an IMAP server on the same host, which is also my MX and
also services all of my SMTP needs. Mail is in a Maildir tree.
Most of the flags are stored in the Maildir message status-bits
(the trailing part of the message-files' names), but some flags
are stored however the IMAP server does that.

I occasionally run Claws Mail on the FreeRunner to read mail via IMAP,
but all mail-reading/-writing is done through the Gnus/Screen session
in the sky when I'm using either my laptop or desktop; though,
if I can get a general N-way mail-sync solution in place, then I'd
like to change that--and always read and write mail locally).

The reason that Gnus is talking to a local IMAP server instead of just
using the Maildir directly is partly that Gnus' Maildir support is
basically terrible (and I should make an apology for the langauge,
at this point, by paraphrasing Michael Elkins: all mail clients suck,
but some suck less): the Gnus people invented their own method of storing
message-flags *outside* of the Maildir flagspace, and Gnus assumes
that it is the only thing reading the Maildir that cares about flags.
To be fair, Gnus had to maintain its own list of flags when using Maildir,
because it supports more flags than Maildir does (in other words,
the other part of why I'm using a local IMAP server is that
Maildir is also, in some aspects, terrible--though).

Gnus appears to still suck (though less) even going through IMAP,
because it doesn't quite have the right mapping between Gnus-internal
and IMAP flags--Gnus' `expirable' and IMAP's `Deleted' seem analogous
to me, but Gnus effectively doesn't map either flag in one space to
*anything* in the other space, so all of my `expirable' messages just
show up as `Seen' outside of Gnus, and all of my `Deleted' messages
show up as `old' inside Gnus. I have a small hack in place that does
equate `Deleted' and `expirable', but if anyone has another suggestion
for working around this, or has an explanation of how I'm mistaken,
or how VM-mode or something is better than Gnus, then I'd be appreciative.

I've tried Thunderbird, and it doesn't look like a viable option--
partly because I can't stand the lousy built-in text-editor
(though I guess there are extensions that'll kluge-in support
for spawning an external editor), and because it doesn't have
any functionality for cleaning-up the broken formatting in some
of the messages that I receive (Gnus does). Its Maildir support
is also even worse than Gnus' (there is none!) and it's now `made up'
for that by using its own opaque databases..., but I can work around
*that* by using a local IMAP server if I need to--and maybe being able
to share mail-texts between different applications on the same computer
(without using an IMAP server-process) isn't really as important as it
has seemed to me.

I use Unison to sync most of my home-directory between my computers,
and I've considered using that to sync Maildirs, but I'm pretty sure
that it won't handle the Maildir message-flags properly, because
Maildir implements flags as parts of the message's file-name--
Unison will see that one end has `deleted a file and added another',
which is OK... unless that same message has also been re-flagged
another way by another host, in which case Unison will see that
`1 file was deleted, 2 added'.


_What I've surmised thus far_:

Git's heuristic rename-detection sounds like it *might* work for this,
but I'm dubious after having seen it fail in similar situations.

Bazaar's support for explicit rename-tracking won't work, because
e-mail clients don't know how to invoke `bzr mv' instead of rename()
when changing the flags on a Maildir message.

So the `allowing interaction with the message-store on *multiple* hosts
between syncs' basically seems intractable unless someone can recommend
something like a (lightweight) IMAP-server that does replication.

It really seems like IMAP is the only thing available that's
at all like a `portable, ubiquitous mail-handling API'..., so
maybe what I want is to just use OfflineIMAP in IMAP-to-IMAP mode?
If so, what sync-*topology* should I be using? Do I have to
use a star topology, where everything syncs with each other
through a central node? Or can should the IMAP UUID support
be sufficient for me to do an N-way distributed sync with
data flowing every which way?

Or...?

-- 
"Don't be afraid to ask (λf.((λx.xx) (λr.f(rr))))."