bomail - Homepage


A homemade, hobbyist, hacky system for organizing and interacting with email.

Author: Bo Waggoner
Page last modified: 2020-05-14.

About

bomail is: a mail user agent like Thunderbird or Outlook (but more like mutt). It is not an email hosting domain like gmail.com or hotmail.com. Its purpose is to organize an already-existing email account using local copies on your computer.

It has three layers:

It was made for my own benefit and enjoyment and I do not expect anyone else to actually use it. I do hope some of the things I've learned will be interesting reading (see Lessons & Writeups). It is written in python and has only been tested on Linux systems.

Source code available on bitbucket.


Interface and Concepts

Here is a (zoomed) example of the text-based interface, which uses the ncurses programming library. (Image without annotations.)

the text interface

Concept 1: tabs. The user can create any number of tabs (the first ten are hotkeyed to 1-9 and 0). Each tab displays a list of messages, the result of a custom search query. So the same approach is used to search emails and to organize them: a tab listing all emails matching a query.

Example tabs you might set up and their queries (often tags are useful for organization):

Notes: The UI is built on command-line tools, so you'd see the same list of emails on the command line by running e.g. "bomail search -open -t work". Also, replies are auto-tagged to match their parent emails, and you can define custom filters to e.g. tag all mail from facebook.com as "social", etc. So the idea is you can get most email to automatically sort into relevant tabs.


Here is a more realistic-size interface (for modern screens anyway) with different spacing and color settings. You can "mark" multiple messages (the red Xs at right) and apply the same operation to all of them, like tagging or moving to trash.

zoomed out text interface

Concept 2: "open/closed/scheduled" states. Each email is in one of 3 states. Open means "in the inbox". Closed means archived. Scheduled YYYY-MM-DD hh:mm means scheduled to be opened at that point. You can quickly schedule an email for plus 3 days from now by typing "sp3d<enter>".

Since your daily use tabs normally include the search term "-open", they only show open emails, not closed or scheduled emails. When the date passes, the email's state is changed to open and it appears in your tab again. Of course all emails show up if you search without "-open".


Concept 3: "dirlike" tags. bomail has no folders: all email is stored by date, and organized by "tagging" messages with tags like "social", "work", "money", etc.

Tags can have forward slashes in them, like "social/facebook/messages" (that's one long tag). This allows tags to be organized in directory-like hierarchies. In particular, a tag will match any search for a "higher-level" tag (a prefix up to a "/" character). For example, a search for tag "social/facebook" will return messages tagged social/facebook/messages, social/facebook/events/barbecues, social/facebook, etc. It will NOT return the tags: social, social/facebookstuff, facebook, twitter, etc.

In bomail you can also have totally normal tags by just never using "/", but I've found dirlike tags very useful, for example, work/research, work/teaching, work/travel, etc. The search tag "work" matches them all, but I also want each of these to sort into different tabs normally.


Concept 4: keyboard interface. The idea on the UI end is to make common operations -- trash, archive ("close"), read, tag -- as quick as possible, ideally single-keypress. Here is an example gif of using the interface to navigate email (description below) (500KB version, 1.5MB version.).

doing some common operations

Here are the steps taken (keys pressed). I'm using vim-style navigation 'hjkl', but these can be re-bound to the arrows.

  1. Close (archive) the first thread ('c').
  2. Navigate down two threads ('jj').
  3. Schedule the current thread to reappear in plus 2 days from now ('sp2d<enter>').
  4. Create a new tab for threads tagged "social" and containing "Facebook" ('+-t social Facebook<enter>').
  5. Navigate into the email thread ('h', as "right" goes into threads and "left" goes out).
  6. Tag email as "social/facebook" ('tsoc<tab>fac<enter>')
  7. Mark all threads and trash them ('Xt').
  8. Undo each previous change: trash, add tag, new tab, schedule, close. ('uuuuu').

Concept 5: (separate) plain-text files and transparency. Rather than dealing with the "raw" emails themselves, bomail processes them to get only a few headers it cares about along with a plain-text version of the body of the email (attachments are stripped into a separate folder). These emails are much smaller than the originals (maybe 1/10th on average in my dataset), so they're much easier to load and edit. New metadata information such as tags are stored directly in these files and bomail doesn't interact with the originals ever again.

This works pretty well with manually editing the files and the small tools for interacting with emails ("open/close", search, add/remove tags, draft a response, etc). Also, the UI is just a transparent interface on top of these commands. In particular, each call to something like "add tags" opens up the plain text files and rewrites them with the new tags.

For example, threads (i.e. conversations) are re-constructed from the email data each time they are needed (though I tried to make this efficient with a bit of extra metadata in the emails themselves, see Lessons & Writeups).

The behavior of threads in the interface is supposed to be very transparent: If any email in a thread matches a tab's search term, the entire thread is displayed (along with a count of how many of its emails match), and threads are sorted by their most recent email. Applying an operation like 'tag' or 'close' to a thread applies the operation to all of its emails. (You can still navigate into the thread and apply operations to the emails individualy.)