Hey Vlad,
On Tue, May 14, 2013 at 12:08:07AM +0200, Vladimir Marek wrote:
Since I'm usually using several mail clients (from phone, old mutt, mutt-kz) it happens quite often that mutt complains.
rename: No such file or directory (errno = 2)
and the only thing I can do about it is to kill whole mutt. What I believe is happening is that if I view unread mail, mutt remembers that it has to move the mail once it leaves current inbox. But before that, someone else touches the mail (probably moving the message from 'new' directory to 'cur'. After that mutt-kz can't find the original file and complains with the error message.
I think this is a problem with mutt-kz not syncing the notmuch database as it updates the maildir flags for the email. I have noticed this while using emacs-notmuch, notmuch cli, and mutt-kz.
Now consider the following scenario. Mutt sees new mail in a maildir, then you read it. Now until the next notmuch sync (say with `notmuch new' from a cron job or OfflineIMAP hook) notmuch thinks the file is <maildir>/new/<some_name>,N. Whereas mutt has renamed it to <maildir>/cur/<some_name>,S! This can happen everytime mutt updates a maildir flag (IOW renames a message file). IMO, this is an inherrent issue with the maildir format. So you will see this out-of-sync behaviour when mutt flags an email as "old", read, or adds the replied flag.
Right, that's also what I think is happening. Although when I try to provocate the change deliberately, I'm not successful.
Since I read and send emails primarily with mutt, I usually encounter the problem from other clients. Given that, I would think you can replicate it from mutt with the following steps:
1. Open a virtual folder and do something that modifies the maildir flags, like reading a message. Do not write the changes back.
2. Open the same message from another client: emacs-notmuch, alot[1], bower[2], notmuch-vim-ruby[3], notmuch-web[4], etc (that is some list!). This time write the changes back to disk.
3. If you try to write the change back to disk from mutt now, you should be able trigger the error.
Actually, I just successfully trigger the error with the above recipe with mutt and alot. On second thought, you should be replicate it with two different instances of mutt :-p!
This seems to be problem in the mutt maildir handling rather than in the notmuch addon, still I wonder if you saw that or even better have some plans on tackling that :)
I thought one possible solution might be introducing a hook to what ever function that does the file renaming when maildir flags are updated. I'm not sure how one can introduce a hook though. Another option would be to add a inotify watch on the maildir currently mutt is viewing. And follow-up on any inotify events.
That seems to be complicated and no matter what you do you will IMO just decrease the likehood of such issue. You can't stop it completely, there will always be a window when you can get two concurrent MUAs out of sync.
This I don't agree with completely but for a very different reason :). IMO if mutt-kz opens a virtual/real folder, modifies message flags, and writes it to disk. It is mutt-kz's responsibility to update the notmuch database as well. After all that is the claim, mutt-kz is mutt + notmuch support! Mutt-kz should not be breaking other tools using the database by leaving it in an inconsistent state; specially breakage of the notmuch cli should not be acceptable.
What I would like to try is to make mutt ignore any local changes to the message when it has been changed underneath it. Just accept the new state which has been force upon it.
Yes, I think that is the right approach given the motivation behind maildir design. Quoting from the Wikipedia article on maildir:
[...] The local filesystem handles file locking as messages are added, moved and deleted. A major design goal of Maildir is to eliminate program code having to handle locking, which is often difficult.
I really appreciate that you are looking into these things :). Hopefully my comments help.
Cheers,
Footnotes:
[1] https://github.com/pazz/alot
[2] https://github.com/wangp/bower
[3] https://github.com/felipec/notmuch-vim-ruby
[4] https://bitbucket.org/wuzzeb/notmuch-web