---+ BRIEF:
using Outlook 2016 as my email client, connecting to Gmail account
Q: what features allow orthogonal properties to be specified independently for messages?
AFAICT the only orthogonal features are read/unread, and only a single type of Outlook/flag mapping to single type of Gmail star.
Which is leading me to consider
Flagged/starred=not-handled-yet
have a (Gmail) rule set the Gmail star as messages are received
clear the flag/star when you have handled it
which is slightly safer than
Flagged/starred=deferred
new messages in inbox will be unflagged/unstarred and hence not deferred yet)
must take care to set the flag/star whenever deferring
must take care to set the flag/star when moving to a topic folder/label(and out of Inbox) or else unhandled messages may be lost
and must take care to clear the flag/star whenever archiving
and of course both of these policies conflict with historical/legacy uses of flags and stars
---+ LESS BRIEF:
I am (perhaps unfortunately) impelled[*] to use Microsoft Outlook as my email client on my PC.
More precisely Microsoft Outlook for Microsoft 365 MSO … which as far as I can tell is essentially Outlook 2016 with slight updates. NOT the web version.
I am connecting it to my Gmail account of many years.
This connection is IMAP/SMTP. I am individual, personal, no exchange servers in the mix etc.
My email workflow and organization is nothing very special. But I'll describe briefly, for motivation. My email workflow for many years has wanted to be something like the following [**]:
• Scan my Inbox
• if I can handle an email right away, do so
• if not, mark it as "Deferred"
I typically have several subfolders or labels under "Deferred", such as work/personal, urgent/soon/later. Sometimes I use GTD like classifications or lists. Sometimes I want to move or mark or label or tag emails as related to particular projects or topics. Sometimes I have rules that apply such marks or labels or tags or folders as email is received. Plus I tend to use Gmail labels or Outlook folders for topics, projects, etc. I really like using saved searches or search folders. I really liked being able to sort email, e.g. by subject, sender, thread, etc..
Basically, I want to be able to be able to mark or classify my email with several ORTHOGONAL or INDEPENDENT properties. and I want to be able to query combinations of these orthogonal/independent properties.
For example:
• email that is associated with a topic
• read/unread
• deferred/not
• handled
○ I'm finished with the email, etc.
And I want to be able to make queries such as
• "show me all email that I have not completely handled yet"
○ whether or not it is in my Inbox, or in a Deferred folder, etc.
Email systems have various ways of indicating such properties, orthogonal or not
• e.g. Outlook folders and subfolders
○ strictly speaking not truly orthogonal
• e.g. Gmail labels - which are truly orthogonal since you can have multiple labels per email
• Gmail stars
• Outlook flags
• Outlook categories
The kluge that maps Gmail labels to IMAP folders Give the Outlook users some control over Gmail's truly orthogonal labels. But AFAIK the Outlook user has no way to detect whether an email has more than one Gmail label, i.e. no way to look for the same message being in more than one Outlook folder. So I can't have Outlook do the equivalent of a Gmail search "in:Inbox AND label:Deferred AND label:Someday". Not by mapping Outlook folders to Gmail labels.
(If there were some way of having Outlook detect that a message has been copied to more than one folder, I think I would be done. Some cheap way. Email deduplication tools accomplish this in some sense, but they really want to what they think are duplicates, which are really Gmail messages with multiple labels.)
Outlook in its natural state has limited orthogonal properties in the form of flags and categories. Good enough for stuff like
• show me all the email in folder Topic-X that has category "Needs-Handling"
• show me all the email in folder Topic-X that is flagged …
other you quickly run out of categories and flags in a way that you do not with Gmail labels.
But when I'm attaching my Outlook email client to the Gmail message store (across IMAP, if it matters)
Outlook categories are not supported by Gmail. if you set a category in Outlook it does not appear on the Gmail message. it appears that the Outlook category may survive for a while, perhaps as long as it's in the Outlook cache of Gmail, but if you uninstall and then reinstall Outlook on a different machine, the categories are lost. AFAICT.
As far as I can tell the only orthogonal properties that are common to the intersection of Outlook on Gmail are
• the klugey Gmail label / Outlook folder mapping
• Gmail star mapped to Outlook flag
and only one flavor of star/flag. Gmail supports multiple colors of star, but Outlook only sees them as a single flag. and vice versa.
Q: are there any other orthogonal message properties that can be set and queried by both Outlook and Gmail?
I highly suspect not, but I just wanted to check.
---+ WORKFLOW given these limitations
messages received by Gmail
○ optionally labeled automatically by Gmail rules
○ and/or left in:Inbox
I scan Inbox (using either Outlook or Gmail clients)
○ handle immediately, and archive (remove the Inbox label)
○ or defer
○ or leave in Inbox as read/unread.
while I might defer by moving out of Inbox into the Defer folder-label or a subfolder, doing this does not allow me to query "Topic1 && deferred/…" in Outlook.
If Outlook-flag/Gmail-star is really the only orthogonal property, I think that I might have to use it to indicate deferred.
Two choices:
Flagged/starred=deferred
• new messages in inbox will be un-flagged unstarred and hence not deferred yet)
• must take care to set the flag/star whenever deferring
• must take care to set the flag/star when moving to a topic folder/label (and out of Inbox)
○ or else unhandled messages may be lost
• and must take care to clear the flag/star whenever archiving
or
Flagged/starred=not-handled-yet
• have a (Gmail) rule set the Gmail star as messages are received
• clear the flag/star when you have handled it
I think the latter choice, Flagged/starred=not-handled-yet, is "safer", in the sense that the most common user error would be to not clear the flag/star when it is handled or archived. this could be annoying, clutter up your searches for " work I haven't finished yet", but it doesn't lose anything. whereas the other polarity can quite easily result in work falling through the cracks.
Note: deferred is a subset of not-handled-yet.
The biggest problem with flagged/starred=not-handled-yet is that I, and probably many others, have historically used flags/stars to indicate things like " this is interesting", or "this is important". i.e. there is a semantic conflict between these historic uses and using flag/start to indicate deferred or not handled yet
Q: has anybody else used the flag/star=not-handled-yet approach? problems?
---+ MORE BACKGROUND
Note**: "My email workflow for many years has wanted to be something like the following [**]:…"
I say "wanted to be" because I've tried to get as close to this is possible, not always successfully.
I was most successful back in the the 20th century when using GNUS email on emacs. Procmail sorting into folders. I would only mark email as "read" when I finished processing it.
Unfortunately, more and more people need to know the distinction between the following three states:
"I have not seen/read your email yet"
"I have seen your email, read it but not yet handled it"
"I have both read and handle your email"
Also, way back in these GNUS/Procmail days, I was free to add my own headers to messages in my saved email tree. But this flexibility is not available nowadays in Gmail/Outlook. I think if I could add my own headers I would be logically happy enough, although the UI might be a bit unsatisfactory.
Note*: "I am (perhaps unfortunately) impelled[] to use Microsoft Outlook as my email client on my PC*."
First, because of what my physical therapist calls computeritis I am trying to use speech recognition to do as much as possible, to avoid typing. Speech recognition just works better for Microsoft Outlook than it does with Gmail.
Second, Gmail does not sort messages in folders or queries. IMHO sorting in other email clients like Outlook and Thunderbird essentially gives you the answer to queries that you do not yet know that you should ask. e.g. you might notice that you have suddenly received 100 messages from a particular sender on different topics. It would be better if you had groups sorts the could return message counts, but eyeballing is often good enough.
... Why use Gmail as the mail storage/server underneath the Outlook user client? why not move my email to a system where all of the Outlook features are supported? First, Gmail free is a good price, but Microsoft has similar. Second, mainly because I have used Gmail for a very long time, and fear the potential work or hassle of moving all of my historic email. and I do need to move all my historic email. Plus, with my luck I would move all of my email from Gmail to Microsoft, and then Microsoft would start charging. But third, I really do like the concept of Gmail labels. I am reluctant to move to a system that does not support such orthogonality. It is unfortunate that Gmail does not support sorting, and I don't know of any other email client that is speech friendly which accesses labels natively.
I spent almost 2 years trying to use Thunderbird rather than Microsoft Outlook as my email client, but I've recently stopped doing so. PRO: Thunderbird sorts like Outlook. CON: Thunderbird is not as speech friendly as Outlook, but I was able to work around much of its problems, so in this respect Thunderbird was slightly more speech friendly than Gmail. CON: Thunderbird interfaces with Gmail labels by this includes that Outlook does. PRO: there is a Thunderbird add-on that can at least show you all of the labels associated with message, although AFAICT no queries. PRO: Thunderbird produces HTML email that more of my correspondence in the open source/hardware community can understand, Whereas Outlook produces HTML email that many people find unreadable on non-Outlook email readers. CON: Thunderbird does not have highlighting (I kluged this, but it was not pretty). PRO: Thunderbird is open source so in theory you can fix things, write your own code, preferably as a add-on. CON: Thunderbird fairly recently underwent a major reorganization of its add-on architecture, and most of the historical add-ons that I want to use no longer work. And it seems that many of them cannot be made to work because of Thunderbird's newish security model. OVERALL, while Thunderbird might be a place to return to, I think I have spent enough time doing so for the moment.
I was and am still most hopeful about being able to return to an emacs based email reader like Gnus. PRO: although not really designed for speech, I have found emacs's command orientation very amenable to speech recognition. In some ways preferable to Outlook. CON: Gnus IMAP Is online only, not acceptable for somebody who needs to read email on planes when disconnected etc. CON: nearly every time I connected Gnus/IMAP/Gmail my emacs would be locked out for 15 minutes, half an hour, etc. CON: I spent some time working with offlineimap (flakey), and mbsync (syncs Gmail-IMAP-Maildir bidirectionally, and then Gnus/Emacs accesses using nnmaildir. but I kept running into Gmail account problems, bandwidth limits, and just plain flakiness. Overall, I get the impression that Google/Gmail are really pushing against non-Oauth authentication. and authentication in open source seems to be a bit of an issue; if an open source app has all of the keys inserts necessary to connect to Gmail Oauth style, AFAICT but might be a violation of the Gmail terms and conditions.
Bottom line: I was and am still hopeful about being able to return to an emacs based mail reader like Gnus accessing Gmail, weather across IMAP or Gmail's native API. off-line. Oauth. and in my dreams with support for Gmail style labels,
If the Gmail labels were exposed as headers, X-GM-LABELS, then I think that my need for orthogonality would be satisfied. I know that I can query such headers in Gnus, and I think they can be queried in Outlook. It remains my hope that I can get headers for Gnus/Emacs. But as far as I know Outlook does not have a way of obtaining such headers. and as far as I know MailDir is hostile the things that modify message content in this way.
All of this last section is just background.
For the moment, all I really want to know is if there are any other orthogonal features when Outlook accesses Gmail. Other than Outlook-flag/Gmail-star.
I will answer my own question, not because I want to do the Q&A, but to clearly state my present understanding, even though it is already mentioned my question.
As far as I can tell
Gmail by itself has the following user settable and query able message properties:
labels: text strings, hierarchical, otherwise fully orthogonal, or close
read/unread
important/not
stars: I believe 12 standard, yellow/Orange/red/purple/blue/green, red-bang!, orange-guillemet, yellow-bang, green-check, blue-info, purple question
categories: Primary, social, promotions, updates, forms, reservation, purchases
I would not be at all surprised if all of these are implemented as labels under the hood, mapped to different icons and operators like labels/category:
Outlook natively, i.e. with exchange, has the following user settable and query able message properties:
folders... not exactly mutually orthogonal, since can only be in a single folder at a time, but folders are orthogonal with the other properties
read/unread
flags - In different flavors ranging from simple flag/unflag, followup flags with a text string, date flags, ...
categories: a limited number of named & colored categories
importance: normal/high/low
sensitivity
And probably some user definable fields.
Although it would seem that Outlook has more orthogonal properties than Gmail, and it does, doesn't mean it's more powerful in terms of querying and setting. it mainly means that some of them have predefined behaviors and GUI features like icons.
However, the intersection of the Gmail and Outlook features is quite narrow:
Gmail does not support Outlook categories, Importance, sensitivity...
AFAICT there is only one orthogonal property apart from the folder/label: Outlook flagged / Gmail starred
There is a simple mapping between Gmail star and Outlook flag.
All of the at least twelve possible Gmail flags maps to a single Flagged value in Outlook - I.e. Outlook cannot tell what the Gmail star is.
If Outlook flagsthe message, it displays as Gmail yellow star. Outlook cannot control what Gmail star is used.
There seem to be synchronization issues between Outlook flagged and Gmail starred. e.g. I have not yet managed to have Outlook clear a flag that Gmail has set, and have that reflected on Gmail's web interface.
Outlook, IOS and Android clients, as well as the built-in Web-Interface client of this Installation all seem to use different IMAP folder naming schemes.
The current bulk of users on that system use Outlook 2010. As well as the above, I also need to support WP, Thunderbird(Cross-Platform).
Some clients have easy ways to change the folder assignment for (sometimes only some of) the special folders, some I haven't found easy ways yet, and for some I'm unsure of even the possibility.
Outlook here is a special case, as it creates localized folder names (.&AMk-l&AOk-ments envoy&AOk-s means .Éléments envoyés) server-side.
IOS (i think) also does this, but has options to change the foldernames used)
This leads to folder mess in Outlook, if the users have used several mailbox access clients.
Is there some kind of Server-Protocol, or Option I could enable on the client or server (dovecot here) to force usage of "International, short names" server-side (Sent, Trash, Drafts, Junk, Inbox, Infected), and let the client bother with translation to the users regional settings? (Like the "C:\Users\ folder in windows already does it).
Or any other way to force Outlook to not create/use non-english folders?
EDIT: Outlook<2013 has options to move some of the imap folders. Later version support some subset of RFC6154 (XLIST).
But I still have not found, how to change the outlook defaults (policy, reg-key, whatever) for Outlook<2013 and Servers without XLIST.
There is no way in Outlook 2013+ to change the IMAP folders except using RFC 6154 on the server. On Servers without XLIST Outlook 2013 seems to do some "guessing" which often fails. The documentation on how to do this in dovecot is here: http://wiki.dovecot.org/MailboxSettings
This will not help you with existing clients though, since outlook checks this only on the initial connection. You might try outlook.exe /resetfoldernames, but probably removing and re-adding the account is necessary for this to take effect.
In my company, we use exchange server, so I setup a davmail server to act as a proxy. Now I can use mu4e in emacs to get/write emails through smtp and imap protocol.
But I want to see the calendar in emacs. davmail supports Caldav, but I am new to both emacs calendar and caldav.
Any idea?
There is org-caldav but I haven't used it personally.
I typically keep around Thunderbird instead of other solutions to ensure that all the calendar updates are synced properly.
I've tried looking up old versions of the standalone calendar application Sunbird, but I run into library issues that I didn't have time to solve under Linux.
I also don't know how convenient/performant caldav would be if you've got a huge number of meetings.
At this time my recommendation would be to rely on an external application.
Is there a way to edit the raw contents of a message with some API in Perl, Python, or even Ruby? A quick Google search showed me little. In short: someone sent me a message a long time ago with the wrong timestamp, and it shows up at the top of folder in Thunderbird. This has annoyed me for a while, and I want it to be sent from the proper year (2010), not from the future (2020). To be clear, headers show right date and times. The other's client clearly did this, so it should require minimal effort? Right?
The IMAP rfc - rfc3501 - doesn't seem to establish a way to change the date/time of a message.
What is you email client ? According to http://kb.mozillazine.org/Invalid_date_in_IMAP_messages, outlook uses the date of reception of the message by the mail server while thunderbird uses the date in the email header (set by the first smtp server, iirc).
If you use thunderbird, there is a bug report: https://bugzilla.mozilla.org/show_bug.cgi?id=570355
mutt can do it. Anywhere you can get with it (and certainly IMAP support in mutt is not great, but you can open GMail, Exchange, dovecot, or any other widely used email store with it), you can run edit command on a message (per default bound to e key). The result of editing will be original message marked deleted and new edited message added to the current folder.
for my group at the university I'd like to set up a server-sided email-to-rss service.
It should work like that, that different people can send emails to a certain address (nothing proprietary like gmail but a certain imap or pop server) which will the be translated into an rss feed. One main and important feature has to be that one can see the sender of the email in the feed. Furthermore it would be nice (to take the load off the server) if the emails get translated to a feed only once a day or so.
Does anyone has some input on this subject? Are there any scripts/services which will allow that?
Thanks a bunch.
Instead of "reinventing the wheel", you could use a mailing list that supports RSS. Your people can then write the mails to the mailing list and you can then use the mailing list's RSS feed however you intend to.
This should help you find a solution: https://encrypted.google.com/search?q=mailing%20list%20rss
Pick a programming language you're familiar with, then use either an imap library to fetch the E-mails (through cron, every hour or something like that), or if you have access to procmail on your mail server, launch your script as an email arrives (this shouldn't be too much work to handle for a server, unless you're talking a vast amount of E-mail).
The script would just insert the E-mails into a database, before extracting them and outputting the RSS-feed directly from that (this shouldn't be more than a handful of lines of code).
There's a couple of providers that does this for you, although it seems that the most popular ones have disappeared. Advanced Email2RSS seems to be an option, although I have no idea how good they are or if it'll even solve your issue.