Is it supposed to be possible to deliver mail while mutt has the mbox open?

[Chris Green]

What is supposed to happen in the following scenario:-

I'm viewing my incoming mail (inbox), looking at the index view in mutt.

Some new mail arrives, delivered by procmail/python script/whatever
to the inbox.

Is there supposed to be some mechanism whereby mutt recognises that all
that has happened is that new mail has been appended to the mbox?

At present I *always* get the "Mailbox was externally modified. Flags
may be wrong." message.

[Erik Christiansen]

Post by Chris Green
What is supposed to happen in the following scenario:-
I'm viewing my incoming mail (inbox), looking at the index view in mutt.
Some new mail arrives, delivered by procmail/python script/whatever
to the inbox.
Is there supposed to be some mechanism whereby mutt recognises that all
that has happened is that new mail has been appended to the mbox?

The manual has a brief description, in section: "10. New Mail Detection"
(If mutt is well set up, that's on <F1>, but I figure you know that.)

Post by Chris Green
At present I *always* get the "Mailbox was externally modified. Flags
may be wrong." message.

That happens here, if I have two mutts open concurrently. My guess is
that your python isn't satisfying mutt's access/modification time
requirements.

Erik

[Chris Green]

Post by Erik Christiansen

Post by Chris Green
What is supposed to happen in the following scenario:-
I'm viewing my incoming mail (inbox), looking at the index view in mutt.
Some new mail arrives, delivered by procmail/python script/whatever
to the inbox.
Is there supposed to be some mechanism whereby mutt recognises that all
that has happened is that new mail has been appended to the mbox?

The manual has a brief description, in section: "10. New Mail Detection"
(If mutt is well set up, that's on <F1>, but I figure you know that.)

Yes, but it doesn't tell me what provokes the "Mailbox was externally
modified" message.

Post by Erik Christiansen

Post by Chris Green
At present I *always* get the "Mailbox was externally modified. Flags
may be wrong." message.

That happens here, if I have two mutts open concurrently. My guess is
that your python isn't satisfying mutt's access/modification time
requirements.

It's satisfying them in that mutt recognises new mail arriving in all my
(separate) mailboxes, that's working perfectly. It's just that if I
happen to be *reading* the mbox where the new mail arrives I get the
error message.

I'm sure this didn't used to happen and I've not changed the delivery
mechanism to my knowledge.

What I want to know is:-

Is it possible for a message to be delivered into an mbox that mutt
is looking at without provoking the ""Mailbox was externally
modified" message?

If the above is possible then what does the delivering MTA have to
do so that the delivered message is 'N' and all others are left
unchanged?

[Chris Green]

Post by Chris Green
What I want to know is:-
Is it possible for a message to be delivered into an mbox that mutt
is looking at without provoking the ""Mailbox was externally
modified" message?
If the above is possible then what does the delivering MTA have to
do so that the delivered message is 'N' and all others are left
unchanged?

I just took a look at the code (desperate measures!) and it gives me the
answer.

If the mbox in question has a message *appended* to it then mutt regards
this as OK and will not output the error. It detects an append by
looking to see if the 'message separator' before the appended message is
exactly where the end-of-file was previously.

I suspect my MTA doesn't agree exactly with mutt about where the
'message separator' is.

I need to play some more! :-)

[Chris Green]

Post by Chris Green

Post by Chris Green
What I want to know is:-
Is it possible for a message to be delivered into an mbox that mutt
is looking at without provoking the ""Mailbox was externally
modified" message?
If the above is possible then what does the delivering MTA have to
do so that the delivered message is 'N' and all others are left
unchanged?

I just took a look at the code (desperate measures!) and it gives me the
answer.
If the mbox in question has a message *appended* to it then mutt regards
this as OK and will not output the error. It detects an append by
looking to see if the 'message separator' before the appended message is
exactly where the end-of-file was previously.
I suspect my MTA doesn't agree exactly with mutt about where the
'message separator' is.
I need to play some more! :-)

I have found my problem!

My (python) MTA puts a blank line at the start of each message before
the "From MAILER-DAEMON Wed Mar 20 14:27:05 2013" line which is the mbox
start line. As described above mutt sees the start as being at the
"From " so it thinks the mbox has been modified otherwise than by a
simple append.

I guess something must have changed in the python libraries (or maybe in
mutt) to have changed this logic because, as I said, it did used to work
OK.

I'll just have to fiddle something somewhere. I can probably remove the
empty line in my python code.

[David Champion]

Post by Chris Green
I suspect my MTA doesn't agree exactly with mutt about where the
'message separator' is.

When your script delivers a message, does it append a message and then
a blank line, or does it append a blank line and then a message?

[Chris Green]

Post by David Champion

Post by Chris Green
I suspect my MTA doesn't agree exactly with mutt about where the
'message separator' is.

When your script delivers a message, does it append a message and then
a blank line, or does it append a blank line and then a message?

It appears to add a blank line and then the message.

Has the mutt handling of this changed in the last few versions?

The python way of doing this is correct according to the RFC as far as I
can tell, there *should* be a blank line between the end of a message
and the 'From ' starting the next message. Thus the python library I'm
using does add this blank line.

However mutt detects this as an error and outputs the messsage about the
mailbox being modified.

If, on the other hand, I append a message to my mbox file by hand and I
*don't* put a blank line in then mutt is quite happy. Thus mutt is quite
happy with the following in an mbox:-

...
...
User-Agent: Mutt/1.5.21 (2010-09-15)
Status: O
Content-Length: 21
Lines: 2

Hello
--
Chris Green
From ***@zbmc.eu Wed Mar 20 15:16:33 2013
Return-Path: <***@zbmc.eu>
X-Original-To: ***@chris.zbmc.eu
Delivered-To: ***@chris.zbmc.eu
Received: by chris.zbmc.eu (Postfix, from userid 1000)
id 88E8C381B20; Wed, 20 Mar 2013 15:16:33 +0000 (GMT)
Date: Wed, 20 Mar 2013 15:16:33 +0000
From: Chris Green <***@isbd.co.uk>
...
...

but strictly this is wrong as there should be an empty line there.

[David Champion]

Post by Chris Green
Has the mutt handling of this changed in the last few versions?

Not that I know of, and I doubt it.

Post by Chris Green
The python way of doing this is correct according to the RFC as far as I
can tell, there *should* be a blank line between the end of a message
and the 'From ' starting the next message. Thus the python library I'm
using does add this blank line.
However mutt detects this as an error and outputs the messsage about the
mailbox being modified.

There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.

[James Griffin]

[--------- Wed 20.Mar'13 at 18:08:49 -0500 David Champion :---------]

Post by David Champion

Post by Chris Green
Has the mutt handling of this changed in the last few versions?

Not that I know of, and I doubt it.

Post by Chris Green
The python way of doing this is correct according to the RFC as far as I
can tell, there *should* be a blank line between the end of a message
and the 'From ' starting the next message. Thus the python library I'm
using does add this blank line.
However mutt detects this as an error and outputs the messsage about the
mailbox being modified.

There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.
--

Chris, do you have a site where this python script is viewable? just so
we can see how it works. It might help clarify the issue and of course
it's interesting to see how people have come with ideas and used
languages/scripts to handle mail.

Jamie

[Chris Green]

Post by James Griffin
[--------- Wed 20.Mar'13 at 18:08:49 -0500 David Champion :---------]

Post by David Champion

Post by Chris Green
Has the mutt handling of this changed in the last few versions?

Not that I know of, and I doubt it.

Post by Chris Green
The python way of doing this is correct according to the RFC as far as I
can tell, there *should* be a blank line between the end of a message
and the 'From ' starting the next message. Thus the python library I'm
using does add this blank line.
However mutt detects this as an error and outputs the messsage about the
mailbox being modified.

There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.
--

Chris, do you have a site where this python script is viewable? just so
we can see how it works. It might help clarify the issue and of course
it's interesting to see how people have come with ideas and used
languages/scripts to handle mail.

It uses the standard Python libraries, I have attached the code (there's
nothing sensitive in there).

The relevant bit that appends the message to the mbox is:-


#
#
# Deliver a message to a local mbox
#
def deliverMboxMsg(dest, msg, log):
#
#
# Create the destination mbox instance
#
mbx = mailbox.mbox(dest, factory=None)

log.info("From: " + msg.get("From", "unknown"))
log.info("Destination is: " + dest)
#
#
# Lock the mbox while we append to it
#

for tries in xrange(3):
try:
mbx.lock()
#
#
# Append the incoming message to the appropriate mbox
#
mbx.add(msg)
#
#
# now set the modified time later than the access time (which is 'now') so
# that mutt will see new mail in the mbox.
#
os.utime(dest, ((time.time()), (time.time() + 5)))
mbx.unlock()
break

except mailbox.ExternalClashError:
log.info("Destination locked, try " + str(tries))
time.sleep(1)

else: # get here if we ran out of tries
log.warn("Failed to lock destination after 3 attempts, giving up")



As you can see I'm using the 'off the shelf' standard Python library
class mailbox.mbox to deliver the message. If this is doing it wrong
then I'm very surprised.

[Derek Martin]

Post by David Champion
There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.

This matches my expectation also.

[Chris Green]

Post by Derek Martin

Post by David Champion
There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.

This matches my expectation also.

That's all very well but it does have some issues:-

What should an MTA do if there *isn't* a blank line at the end of
the current mbox where it is going to append a new message? It
seems to me that what the Python libraries do guarantees that there
will always be a blank line before the 'From ' line, if there's one
already then it doesn't matter too much.

Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

[James Griffin]

[--------- Fri 22.Mar'13 at 9:04:21 +0000 Chris Green :---------]

Post by Chris Green

Post by Derek Martin

Post by David Champion
There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.

This matches my expectation also.

That's all very well but it does have some issues:-
What should an MTA do if there *isn't* a blank line at the end of
the current mbox where it is going to append a new message? It
seems to me that what the Python libraries do guarantees that there
will always be a blank line before the 'From ' line, if there's one
already then it doesn't matter too much.
Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

Sorry Chris, I believe you're confusing MTA with MDA/LDA, although
that's not really relevent. Could you perhaps alter your python script
as David suggested but start over with new mboxes; i.e. move the current
mboxes into an archive folder inside the ~/Mail directory so as to avoid
the confusion with how your messages are currently laid-out in the mbox
file. Do you think that might help?

[James Griffin]

[--------- Fri 22.Mar'13 at 7:38:54 -0400 Patrick Shanahan :---------]

[...]

Post by James Griffin
Sorry Chris, I believe you're confusing MTA with MDA/LDA, although
that's not really relevent. Could you perhaps alter your python script
as David suggested but start over with new mboxes; i.e. move the current
mboxes into an archive folder inside the ~/Mail directory so as to avoid
the confusion with how your messages are currently laid-out in the mbox
file. Do you think that might help?

As mbox is actually *one* contiguous file, only the last appended msg will
be relevant and it's trailing or lack of a trailing blank line. Archiving
would provide a definitive separation but is really not necessary.

Ah, well that's ok. I don't know a lot about mbox - although I do use
them - I just thought the files may have been left in a state that could
still be showing the issues described. Having a clean file with the
adjustments to the python script -- starting afresh so to speak - might
have helped.

If that's not the case then I'm glad you pointed that out.

[Chris Green]

Post by James Griffin
[--------- Fri 22.Mar'13 at 9:04:21 +0000 Chris Green :---------]

Post by Chris Green

Post by Derek Martin

Post by David Champion
There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.

This matches my expectation also.

That's all very well but it does have some issues:-
What should an MTA do if there *isn't* a blank line at the end of
the current mbox where it is going to append a new message? It
seems to me that what the Python libraries do guarantees that there
will always be a blank line before the 'From ' line, if there's one
already then it doesn't matter too much.
Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

Sorry Chris, I believe you're confusing MTA with MDA/LDA, although

Yes, I am, though by default both are postfix on my system.

Post by James Griffin
that's not really relevent. Could you perhaps alter your python script
as David suggested but start over with new mboxes; i.e. move the current
mboxes into an archive folder inside the ~/Mail directory so as to avoid
the confusion with how your messages are currently laid-out in the mbox
file. Do you think that might help?

I've already fixed the problem for myself, as follows:-

Mutt accepts an mbox with no blank lines (not surprising as it's
what it creates itself).

I have created a derived class of my own from the Python library
class mailbox.mbox and I have overriden the _pre_message_hook(self,
f) method so that it doesn't append a blank line before appending a
new message to an mbox. I use this derived class in my filter
program which delivers mail to my incoming mbox files.

So I have fixed my particular problem.

It does seem to me though that mutt doesn't necessarily do the right
thing to detect whether an mbox has been 'modified' or just appended to.

[David Champion]

Post by Chris Green
What should an MTA do if there *isn't* a blank line at the end of
the current mbox where it is going to append a new message? It
seems to me that what the Python libraries do guarantees that there
will always be a blank line before the 'From ' line, if there's one
already then it doesn't matter too much.

I don't want to butt heads with Python library people, but I would argue
that the Python library's approach is backwards. This has historically
been mildly problematic with Mailman archives, for example, which
begin with a blank line instead of with 'From ', and are unreadable
without modification using some mail applications. (Mutt was patched to
accomodate this quirk only relatively recently.)

Post by Chris Green
Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

Mutt instead writes Lines: and Content-Length: headers, which are a good
alternative to using '\n\nFrom ' as a message delimiter. If you have
Lines: and/or Content-Length: in your delivery message, your LDA/filter
does not need to place a blank line at all. But unless you're adding
those yourself, you can't depend on having them. It is therefore best
to append a blank line to messages during local delivery.

[Chris Green]

Post by David Champion

Post by Chris Green
What should an MTA do if there *isn't* a blank line at the end of
the current mbox where it is going to append a new message? It
seems to me that what the Python libraries do guarantees that there
will always be a blank line before the 'From ' line, if there's one
already then it doesn't matter too much.

I don't want to butt heads with Python library people, but I would argue
that the Python library's approach is backwards. This has historically
been mildly problematic with Mailman archives, for example, which
begin with a blank line instead of with 'From ', and are unreadable
without modification using some mail applications. (Mutt was patched to
accomodate this quirk only relatively recently.)

When I looked through a number of my mbox files it seemed to me that
there must be quite a few 'deliverers' out there appending or prepending
blank lines as there just about always seem to be several between each
message.

Post by David Champion

Post by Chris Green
Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

Mutt instead writes Lines: and Content-Length: headers, which are a good
alternative to using '\n\nFrom ' as a message delimiter. If you have
Lines: and/or Content-Length: in your delivery message, your LDA/filter
does not need to place a blank line at all. But unless you're adding
those yourself, you can't depend on having them. It is therefore best
to append a blank line to messages during local delivery.

However not all software will *use* the Lines: and Content-Length:
headers so the '\n\nFrom ' should be there as well surely. Apart from
anything else someone/something may have modified the message on the way
who/which doesn't know about Lines: and Content-Length: so they could
well be wrong.

[Derek Martin]

Post by Chris Green

Post by Derek Martin

Post by David Champion
There absolutely should be a blank line. I think though that the order
is wrong: mutt expects that a message (i.e. a From_ line) appears at
the old EOF marker, and that the EOF marker is on/after a blank line.
I think that if you adjust your filter to write the message and then a
blank line instead of a blank line and then a message, the error will go
away.

This matches my expectation also.

That's all very well but it does have some issues:-
What should an MTA do if there *isn't* a blank line at the end of
the current mbox where it is going to append a new message?

That's a reasonable question (technically MTA does not come into play
here, this is an MDA or MUA behavior). To be 100% correct, most
likely the mailer should CHECK what the state of the mailbox is, add a
blank line if required, and NOT add a blank line if one is there.
That's more work than making assumptions, but it's not so horrible.
But it does seem that M*A software in general expects the trailing
blank line, so it should always ensure it's there on messages it
writes itself. This makes sense: the blank line is often called the
"message separator", but what it really is is the end of message
indicator. Thinking about it as a separator leads to thinking that
it's optional if there's not a subsequent message, which leads to the
problem you're having.

Post by Chris Green
It seems to me that what the Python libraries do guarantees that
there will always be a blank line before the 'From ' line, if
there's one already then it doesn't matter too much.

It does change the content of the last message though, which is
incorrect behavior. You say it doesn't matter much, but it might, if
you have some automated process that expects a particular mailbox to
be in a particular state. What does it do if you subsequently delete
the last message, and then add another one? My suspicion is that it
might add yet another blank line, which clearly is wrong. Though in
any case, adding even a single extra blank line is wrong.

Post by Chris Green
Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

Yet it works with every MDA I've ever used... which seems to suggest
that this is the expected behavior, and that the Python library got it
wrong. And to be honest it's not really *that* surprising... I've
seen other cases where Python libraries were doing it wrong (whatever
"it" might be).

[Erik Christiansen]

Post by Derek Martin

Post by Chris Green
Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

When I save a message to another mailbox, mutt always leaves a blank
line between messages. (Sometimes there are two blank lines - one is
then presumably part of the message.) I have 1092 mailboxes, many with
thousands of messages. Most of them receive mail only by save or copy.
A quick awk script showed one apparent exception to this mutt behaviour
in the 1596 messages in one of those saved-by-mutt-only mailboxes, but
it was a /^From / occurring within an attachment.

Post by Derek Martin
Yet it works with every MDA I've ever used... which seems to suggest
that this is the expected behavior, and that the Python library got it
wrong. And to be honest it's not really *that* surprising... I've
seen other cases where Python libraries were doing it wrong (whatever
"it" might be).

Having only used fetchmail, these last couple of decades, I can only
confirm that it inserts a blank line, thus mimicking mutt's behaviour.
There are several hundred thousand messages in my archive, and I do
occasionally go in with vim, but have never seen one message butt up
against the previous. Checking a delivery mailbox with the awk script
finds no exceptions in 3180 messages.

Erik

[Chris Green]

Post by Erik Christiansen

Post by Chris Green
Mutt itself *doesn't* put a blank line there, if you S[ave] or
C[opy] messages to a new mbox the messages have no blank lines
before the 'From '.

When I save a message to another mailbox, mutt always leaves a blank
line between messages. (Sometimes there are two blank lines - one is
then presumably part of the message.) I have 1092 mailboxes, many with
thousands of messages. Most of them receive mail only by save or copy.
A quick awk script showed one apparent exception to this mutt behaviour
in the 1596 messages in one of those saved-by-mutt-only mailboxes, but
it was a /^From / occurring within an attachment.

Well I first actually tried it and saw no blank line.

I've now looked throught my archive (1845 mailboxes) and most of them,
saved with mutt, using S[ave], seem *not* to have a blank line either.
There are some with blank lines between but I suspect that's because of
migrating back and forth between various formats quite a lot over the
years.

Certainly my current mutt 1.5.21 doesn't put a blank line in there and
it's quite standard from the Ubuntu repositories (though I suppose they
*might* have done something funny to it).

[Erik Christiansen]

Post by Chris Green
Well I first actually tried it and saw no blank line.
I've now looked throught my archive (1845 mailboxes) and most of them,
saved with mutt, using S[ave], seem *not* to have a blank line either.
There are some with blank lines between but I suspect that's because of
migrating back and forth between various formats quite a lot over the
years.

That is weird, because I'm using:
Mutt 1.5.21+145 (2a1c5d3dd72e) (2012-12-30)
but have used a whole string of older mutts over the years. And I've
never made (or heard of) a related config setting which could explain
the behavioural difference.

Post by Chris Green
Certainly my current mutt 1.5.21 doesn't put a blank line in there and
it's quite standard from the Ubuntu repositories (though I suppose they
*might* have done something funny to it).

So your mailboxes, viewed in vim or similar, have the /^From / line
immediately following the last non-blank line of the previous message!!?
That's something I've never seen ... in decades, mostly with mutt.

I've just run the script on a set of 387 mailboxes exclusively written
by mutt (in the last few years, right up to tonight). Absolutely all of
the 11542 messages are separated by blank lines.

Maybe your mutt has indeed been ubuntued.

Erik

[Chris Green]

Post by Erik Christiansen

Post by Chris Green
Well I first actually tried it and saw no blank line.
I've now looked throught my archive (1845 mailboxes) and most of them,
saved with mutt, using S[ave], seem *not* to have a blank line either.
There are some with blank lines between but I suspect that's because of
migrating back and forth between various formats quite a lot over the
years.

Mutt 1.5.21+145 (2a1c5d3dd72e) (2012-12-30)
but have used a whole string of older mutts over the years. And I've
never made (or heard of) a related config setting which could explain
the behavioural difference.

Post by Chris Green
Certainly my current mutt 1.5.21 doesn't put a blank line in there and
it's quite standard from the Ubuntu repositories (though I suppose they
*might* have done something funny to it).

So your mailboxes, viewed in vim or similar, have the /^From / line
immediately following the last non-blank line of the previous message!!?
That's something I've never seen ... in decades, mostly with mutt.

Here, I've just sent myself three test E-Mails and have saved them to a
mailbox called test, here is the result:-

From MAILER-DAEMON Sat Mar 23 15:19:20 2013
Return-Path: <***@zbmc.eu>
X-Original-To: chris
Delivered-To: ***@zbmc.eu
Received: by chris.zbmc.eu (Postfix, from userid 1000)
id 94F62380255; Sat, 23 Mar 2013 15:19:20 +0000 (GMT)
Date: Sat, 23 Mar 2013 15:19:20 +0000
From: Chris Green <***@isbd.co.uk>
To: Chris Green <***@zbmc.eu>
Subject: Test
Message-ID: <***@chris>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
Content-Length: 31
Lines: 3

This is test 3

--
Chris Green
From MAILER-DAEMON Sat Mar 23 15:18:53 2013
Return-Path: <***@zbmc.eu>
X-Original-To: chris
Delivered-To: ***@zbmc.eu
Received: by chris.zbmc.eu (Postfix, from userid 1000)
id B4267380255; Sat, 23 Mar 2013 15:18:53 +0000 (GMT)
Date: Sat, 23 Mar 2013 15:18:53 +0000
From: Chris Green <***@isbd.co.uk>
To: Chris Green <***@zbmc.eu>
Subject: Test1
Message-ID: <***@chris>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
Status: O
Content-Length: 31
Lines: 3

This is test 1

--
Chris Green
From MAILER-DAEMON Sat Mar 23 15:19:06 2013
Return-Path: <***@zbmc.eu>
X-Original-To: chris
Delivered-To: ***@zbmc.eu
Received: by chris.zbmc.eu (Postfix, from userid 1000)
id 12196380255; Sat, 23 Mar 2013 15:19:06 +0000 (GMT)
Date: Sat, 23 Mar 2013 15:19:06 +0000
From: Chris Green <***@isbd.co.uk>
To: Chris Green <***@zbmc.eu>
Subject: Test2
Message-ID: <***@chris>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
Status: O
Content-Length: 31
Lines: 3

This is test 2

--
Chris Green


I saved them separately (i.e. I didn't do a tag-save), nothing else
special at all, as you can see I didn't save them in the order I
sent them.

'mutt -v' returns:-

chris$ mutt -v
Mutt 1.5.21 (2010-09-15)
Copyright (C) 1996-2009 Michael R. Elkins and others.
Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'.
Mutt is free software, and you are welcome to redistribute it
under certain conditions; type `mutt -vv' for details.

System: Linux 3.5.0-26-generic (x86_64)
ncurses: ncurses 5.9.20110404 (compiled with 5.9)
libidn: 1.25 (compiled with 1.25)
hcache backend: tokyocabinet 1.4.47
Compile options:
-DOMAIN
+DEBUG
-HOMESPOOL +USE_SETGID +USE_DOTLOCK +DL_STANDALONE +USE_FCNTL
-USE_FLOCK
+USE_POP +USE_IMAP +USE_SMTP
-USE_SSL_OPENSSL +USE_SSL_GNUTLS +USE_SASL +USE_GSS
+HAVE_GETADDRINFO
+HAVE_REGCOMP -USE_GNU_REGEX
+HAVE_COLOR +HAVE_START_COLOR +HAVE_TYPEAHEAD +HAVE_BKGDSET
+HAVE_CURS_SET +HAVE_META +HAVE_RESIZETERM
+CRYPT_BACKEND_CLASSIC_PGP +CRYPT_BACKEND_CLASSIC_SMIME
+CRYPT_BACKEND_GPGME
-EXACT_ADDRESS -SUN_ATTACHMENT
+ENABLE_NLS -LOCALES_HACK +COMPRESSED +HAVE_WC_FUNCS
+HAVE_LANGINFO_CODESET +HAVE_LANGINFO_YESEXPR
+HAVE_ICONV -ICONV_NONTRANS +HAVE_LIBIDN +HAVE_GETSID +USE_HCACHE
-ISPELL
SENDMAIL="/usr/sbin/sendmail"
MAILPATH="/var/mail"
PKGDATADIR="/usr/share/mutt"
SYSCONFDIR="/etc"
EXECSHELL="/bin/sh"
MIXMASTER="mixmaster"
To contact the developers, please mail to <mutt-***@mutt.org>.
To report a bug, please visit http://bugs.mutt.org/.

misc/am-maintainer-mode
features/ifdef
features/xtitles
features/trash-folder
features/purge-message
features/imap_fast_trash
features/sensible_browser_position
features-old/patch-1.5.4.vk.pgp_verbose_mime
features/compressed-folders
features/compressed-folders.debian
debian-specific/Muttrc
debian-specific/Md.etc_mailname_gethostbyname.diff
debian-specific/use_usr_bin_editor.diff
debian-specific/correct_docdir_in_man_page.diff
debian-specific/dont_document_not_present_features.diff
debian-specific/document_debian_defaults
debian-specific/assumed_charset-compat
debian-specific/467432-write_bcc.patch
debian-specific/566076-build_doc_adjustments.patch
misc/define-pgp_getkeys_command.diff
misc/gpg.rc-paths
misc/smime.rc
upstream/531430-imapuser.patch
upstream/537818-emptycharset.patch
upstream/543467-thread-segfault.patch
upstream/542817-smimekeys-tmpdir.patch
upstream/548577-gpgme-1.2.patch
upstream/553321-ansi-escape-segfault.patch
upstream/568295-references.patch
upstream/547980-smime_keys-chaining.patch
upstream/528233-readonly-open.patch
upstream/228671-pipe-mime.patch
upstream/383769-score-match.patch
upstream/578087-header-strchr.patch
upstream/603288-split-fetches.patch
upstream/537061-dont-recode-saved-attachments.patch
upstream/608706-fix-spelling-errors.patch
upstream/620854-pop3-segfault.patch
upstream/611412-bts-regexp.patch
upstream/624058-gnutls-deprecated-set-priority.patch
upstream/624085-gnutls-deprecated-verify-peers.patch
upstream/584138-mx_update_context-segfault.patch
upstream/619216-gnutls-CN-validation.patch
upstream/611410-no-implicit_autoview-for-text-html.patch
upstream/path_max
mutt.org

Are all those patches listed Ubuntu/debian additions?

[Brian Salter-Duke]

I looked at a mbox that was entirely created under Ubuntu and there are
blank lines at the end of each message before the From line.

Post by Chris Green

Post by Erik Christiansen

Post by Chris Green
Well I first actually tried it and saw no blank line.
I've now looked throught my archive (1845 mailboxes) and most of them,
saved with mutt, using S[ave], seem *not* to have a blank line either.
There are some with blank lines between but I suspect that's because of
migrating back and forth between various formats quite a lot over the
years.

Mutt 1.5.21+145 (2a1c5d3dd72e) (2012-12-30)
but have used a whole string of older mutts over the years. And I've
never made (or heard of) a related config setting which could explain
the behavioural difference.

Post by Chris Green
Certainly my current mutt 1.5.21 doesn't put a blank line in there and
it's quite standard from the Ubuntu repositories (though I suppose they
*might* have done something funny to it).

So your mailboxes, viewed in vim or similar, have the /^From / line
immediately following the last non-blank line of the previous message!!?
That's something I've never seen ... in decades, mostly with mutt.

Here, I've just sent myself three test E-Mails and have saved them to a
mailbox called test, here is the result:-
From MAILER-DAEMON Sat Mar 23 15:19:20 2013
X-Original-To: chris
Received: by chris.zbmc.eu (Postfix, from userid 1000)
id 94F62380255; Sat, 23 Mar 2013 15:19:20 +0000 (GMT)
Date: Sat, 23 Mar 2013 15:19:20 +0000
Subject: Test
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
Content-Length: 31
Lines: 3
This is test 3
--
Chris Green
From MAILER-DAEMON Sat Mar 23 15:18:53 2013
X-Original-To: chris
Received: by chris.zbmc.eu (Postfix, from userid 1000)
id B4267380255; Sat, 23 Mar 2013 15:18:53 +0000 (GMT)
Date: Sat, 23 Mar 2013 15:18:53 +0000
Subject: Test1
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
Status: O
Content-Length: 31
Lines: 3
This is test 1
--
Chris Green
From MAILER-DAEMON Sat Mar 23 15:19:06 2013
X-Original-To: chris
Received: by chris.zbmc.eu (Postfix, from userid 1000)
id 12196380255; Sat, 23 Mar 2013 15:19:06 +0000 (GMT)
Date: Sat, 23 Mar 2013 15:19:06 +0000
Subject: Test2
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
Status: O
Content-Length: 31
Lines: 3
This is test 2
--
Chris Green
I saved them separately (i.e. I didn't do a tag-save), nothing else
special at all, as you can see I didn't save them in the order I
sent them.
'mutt -v' returns:-
chris$ mutt -v
Mutt 1.5.21 (2010-09-15)
Copyright (C) 1996-2009 Michael R. Elkins and others.
Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'.
Mutt is free software, and you are welcome to redistribute it
under certain conditions; type `mutt -vv' for details.
System: Linux 3.5.0-26-generic (x86_64)
ncurses: ncurses 5.9.20110404 (compiled with 5.9)
libidn: 1.25 (compiled with 1.25)
hcache backend: tokyocabinet 1.4.47
-DOMAIN
+DEBUG
-HOMESPOOL +USE_SETGID +USE_DOTLOCK +DL_STANDALONE +USE_FCNTL
-USE_FLOCK
+USE_POP +USE_IMAP +USE_SMTP
-USE_SSL_OPENSSL +USE_SSL_GNUTLS +USE_SASL +USE_GSS
+HAVE_GETADDRINFO
+HAVE_REGCOMP -USE_GNU_REGEX
+HAVE_COLOR +HAVE_START_COLOR +HAVE_TYPEAHEAD +HAVE_BKGDSET
+HAVE_CURS_SET +HAVE_META +HAVE_RESIZETERM
+CRYPT_BACKEND_CLASSIC_PGP +CRYPT_BACKEND_CLASSIC_SMIME
+CRYPT_BACKEND_GPGME
-EXACT_ADDRESS -SUN_ATTACHMENT
+ENABLE_NLS -LOCALES_HACK +COMPRESSED +HAVE_WC_FUNCS
+HAVE_LANGINFO_CODESET +HAVE_LANGINFO_YESEXPR
+HAVE_ICONV -ICONV_NONTRANS +HAVE_LIBIDN +HAVE_GETSID +USE_HCACHE
-ISPELL
SENDMAIL="/usr/sbin/sendmail"
MAILPATH="/var/mail"
PKGDATADIR="/usr/share/mutt"
SYSCONFDIR="/etc"
EXECSHELL="/bin/sh"
MIXMASTER="mixmaster"
To report a bug, please visit http://bugs.mutt.org/.
misc/am-maintainer-mode
features/ifdef
features/xtitles
features/trash-folder
features/purge-message
features/imap_fast_trash
features/sensible_browser_position
features-old/patch-1.5.4.vk.pgp_verbose_mime
features/compressed-folders
features/compressed-folders.debian
debian-specific/Muttrc
debian-specific/Md.etc_mailname_gethostbyname.diff
debian-specific/use_usr_bin_editor.diff
debian-specific/correct_docdir_in_man_page.diff
debian-specific/dont_document_not_present_features.diff
debian-specific/document_debian_defaults
debian-specific/assumed_charset-compat
debian-specific/467432-write_bcc.patch
debian-specific/566076-build_doc_adjustments.patch
misc/define-pgp_getkeys_command.diff
misc/gpg.rc-paths
misc/smime.rc
upstream/531430-imapuser.patch
upstream/537818-emptycharset.patch
upstream/543467-thread-segfault.patch
upstream/542817-smimekeys-tmpdir.patch
upstream/548577-gpgme-1.2.patch
upstream/553321-ansi-escape-segfault.patch
upstream/568295-references.patch
upstream/547980-smime_keys-chaining.patch
upstream/528233-readonly-open.patch
upstream/228671-pipe-mime.patch
upstream/383769-score-match.patch
upstream/578087-header-strchr.patch
upstream/603288-split-fetches.patch
upstream/537061-dont-recode-saved-attachments.patch
upstream/608706-fix-spelling-errors.patch
upstream/620854-pop3-segfault.patch
upstream/611412-bts-regexp.patch
upstream/624058-gnutls-deprecated-set-priority.patch
upstream/624085-gnutls-deprecated-verify-peers.patch
upstream/584138-mx_update_context-segfault.patch
upstream/619216-gnutls-CN-validation.patch
upstream/611410-no-implicit_autoview-for-text-html.patch
upstream/path_max
mutt.org
Are all those patches listed Ubuntu/debian additions?
--
Chris Green

--

[James Griffin]

[--------- Sun 24.Mar'13 at 7:05:44 +0000 Brian Salter-Duke :---------]

Post by Brian Salter-Duke
I looked at a mbox that was entirely created under Ubuntu and there are
blank lines at the end of each message before the From line.

I can confirm that my sent mbox, created by mutt (current version), also
has a blank line at the end of each message. This is not Ubuntu OS, it's
OpenBSD, but the OS shouldn't be relevent. So I don't think it's a
"mutt" issue.

The other mboxes, created by procmail, also show the correct format:
with a blank line at the end of each message before the From: line of
the proceeding message.

It looks as though your python delivery program might be doing this?
Have you tried taking the python program out of the process to test, to
see if the messages are being appended to the mbox correctly? I know
you've looked at it again and maybe made some changes to it.

Jamie

[Chris Green]

Post by James Griffin
It looks as though your python delivery program might be doing this?
Have you tried taking the python program out of the process to test, to
see if the messages are being appended to the mbox correctly? I know
you've looked at it again and maybe made some changes to it.

How could the python delivery program have anything to do with it?

I simply s[aved] three messages using the mutt save command, no other
program is involved, mutt is simply writing three messages to the same
mbox.

Are you all sure that mutt isn't simply saving what it's provided with
by the incoming MDA?

If the *delivered* message ends with a blank line (i.e. what's in the
inbox) then what mutt saves will end with a blank line.

To do exactly what I did:-

Deliver some messages to a mbox file.

Check that there is *no* blank line at the end of these messages.

Use mutt to s[ave] (or c[opy]) the messages to another mbox.

Look at the new mbox file.

I think you'll find there's no added blank line.

[Will Fiveash]

Post by Chris Green

Post by James Griffin
It looks as though your python delivery program might be doing this?
Have you tried taking the python program out of the process to test, to
see if the messages are being appended to the mbox correctly? I know
you've looked at it again and maybe made some changes to it.

How could the python delivery program have anything to do with it?
I simply s[aved] three messages using the mutt save command, no other
program is involved, mutt is simply writing three messages to the same
mbox.
Are you all sure that mutt isn't simply saving what it's provided with
by the incoming MDA?
If the *delivered* message ends with a blank line (i.e. what's in the
inbox) then what mutt saves will end with a blank line.
To do exactly what I did:-
Deliver some messages to a mbox file.
Check that there is *no* blank line at the end of these messages.
Use mutt to s[ave] (or c[opy]) the messages to another mbox.
Look at the new mbox file.
I think you'll find there's no added blank line.

When I use the mutt (on Solaris 11) copy function to copy three messages
in an existing mbox file to an new mbox file I see a blank line before
each new message's "From " line except for the first message. Maybe
it's time to break out the debugging tools on your end?

[Chris Green]

Post by Will Fiveash

Post by Chris Green

Post by James Griffin
It looks as though your python delivery program might be doing this?
Have you tried taking the python program out of the process to test, to
see if the messages are being appended to the mbox correctly? I know
you've looked at it again and maybe made some changes to it.

How could the python delivery program have anything to do with it?
I simply s[aved] three messages using the mutt save command, no other
program is involved, mutt is simply writing three messages to the same
mbox.
Are you all sure that mutt isn't simply saving what it's provided with
by the incoming MDA?
If the *delivered* message ends with a blank line (i.e. what's in the
inbox) then what mutt saves will end with a blank line.
To do exactly what I did:-
Deliver some messages to a mbox file.
Check that there is *no* blank line at the end of these messages.
Use mutt to s[ave] (or c[opy]) the messages to another mbox.
Look at the new mbox file.
I think you'll find there's no added blank line.

When I use the mutt (on Solaris 11) copy function to copy three messages
in an existing mbox file to an new mbox file I see a blank line before
each new message's "From " line except for the first message. Maybe
it's time to break out the debugging tools on your end?

Yes, but did you look in the "existing mbox file" to see if there were
blank lines betweeen the messages there?

[Will Fiveash]

Post by Chris Green

Post by Will Fiveash

Post by Chris Green

Post by James Griffin
It looks as though your python delivery program might be doing this?
Have you tried taking the python program out of the process to test, to
see if the messages are being appended to the mbox correctly? I know
you've looked at it again and maybe made some changes to it.

How could the python delivery program have anything to do with it?
I simply s[aved] three messages using the mutt save command, no other
program is involved, mutt is simply writing three messages to the same
mbox.
Are you all sure that mutt isn't simply saving what it's provided with
by the incoming MDA?
If the *delivered* message ends with a blank line (i.e. what's in the
inbox) then what mutt saves will end with a blank line.
To do exactly what I did:-
Deliver some messages to a mbox file.
Check that there is *no* blank line at the end of these messages.
Use mutt to s[ave] (or c[opy]) the messages to another mbox.
Look at the new mbox file.
I think you'll find there's no added blank line.

When I use the mutt (on Solaris 11) copy function to copy three messages
in an existing mbox file to an new mbox file I see a blank line before
each new message's "From " line except for the first message. Maybe
it's time to break out the debugging tools on your end?

Yes, but did you look in the "existing mbox file" to see if there were
blank lines betweeen the messages there?

If I edit an mbox file and remove the blank line preceding each "From ",
run mutt on that mbox and have it save all the messages to an mbox it
creates I see no blank lines preceding "From " in the new mbox. So you
are correct in that mutt does not add a blank line. This mutt behavior
hasn't been a problem for me because all my incoming mail is put into
mbox files via fetchmail->procmail which appears to add the blank line.

[Chris Green]

Post by Will Fiveash

Post by Chris Green

Post by Will Fiveash
When I use the mutt (on Solaris 11) copy function to copy three messages
in an existing mbox file to an new mbox file I see a blank line before
each new message's "From " line except for the first message. Maybe
it's time to break out the debugging tools on your end?

Yes, but did you look in the "existing mbox file" to see if there were
blank lines betweeen the messages there?

If I edit an mbox file and remove the blank line preceding each "From ",
run mutt on that mbox and have it save all the messages to an mbox it
creates I see no blank lines preceding "From " in the new mbox. So you
are correct in that mutt does not add a blank line. This mutt behavior
hasn't been a problem for me because all my incoming mail is put into
mbox files via fetchmail->procmail which appears to add the blank line.

Exactly! So mutt *doesn't* add a blank line before the 'File ' if there
wasn't one there already, in fact it doesn't do anything except copy
what it's given by the MDA.

This may be correct in that mutt isn't an MDA but it isn't what everyone
here has being trying to tell me! :-)

[Chris Green]

Post by Chris Green
Exactly! So mutt *doesn't* add a blank line before the 'File ' if there
wasn't one there already, in fact it doesn't do anything except copy
what it's given by the MDA.
This may be correct in that mutt isn't an MDA but it isn't what everyone
here has being trying to tell me! :-)

I don't know why my last four messages haven't made it through the
mailing list, but actually I have been trying without success to tell
you exactly that. :) You're spot on.
Mutt adds headers to indicate the message length when it saves a
message. Apart from headers, it copies the message content exactly.
The blank line is considered part of the message content, so if present
it also gets copied.
I'm sending this mainly for Chris's benefit, but since I'm sending
I might as well try the list again. BCC to Chris since he used a
Mail-Followup-To.

Thanks David, it does seem to have got to the list.

I think I might try and deliver some messages 'direct' using postfix, I
can set up a dummy user here with no filter/procmail and take a look at
what arrives in their inbox.

If postfix *does* append a blank line to all messages I can then,
maybe, go to the python library developers and say I think what
they're doing is wrong.

[David Champion]

Post by Chris Green
Exactly! So mutt *doesn't* add a blank line before the 'File ' if there
wasn't one there already, in fact it doesn't do anything except copy
what it's given by the MDA.
This may be correct in that mutt isn't an MDA but it isn't what everyone
here has being trying to tell me! :-)

I don't know why my last four messages haven't made it through the
mailing list, but actually I have been trying without success to tell
you exactly that. :) You're spot on.

Mutt adds headers to indicate the message length when it saves a
message. Apart from headers, it copies the message content exactly.
The blank line is considered part of the message content, so if present
it also gets copied.

I'm sending this mainly for Chris's benefit, but since I'm sending
I might as well try the list again. BCC to Chris since he used a
Mail-Followup-To.

[David Champion]

I don't know why my last four messages haven't made it through the
mailing list,

Looks like someone (Steve?) fixed gbnet/flirble's smtp? I've been
getting smtp rejections because my dns domain was ostensibly invalid,
but my mail just went through at last. If anyone else has tried to send
and failed, you might try again.

[Nathan Stratton Treadway]

Post by Chris Green
Yes, but did you look in the "existing mbox file" to see if there were
blank lines betweeen the messages there?

For what it's worth, I took a simple test message, saved it into a new
mailbox, and then used vi to make four copies of the message. I
removed the Content-Length: and Lines: header lines from all four
copies, and the trailing blank line from two of the copies.

When I opened up this mbox file using "mutt -f" and saved each message
to a new mailbox (in various orders), I found that the presence-or-not
of a trailing blank line was preserved in the copies.

I got the same results from these tests with both with a "current"
version of mutt, v1.5.21 from the Ubuntu Precise package, and a "very
old" one, v1.5.9i from the Debian Sarge package.


So... it does appear that Mutt will preserve whatever trailing-blank-
line situation was originally created by the MDA, even as it saves the
messages into new mailboxes (and that it doesn't actually care what
convention was used by other messages previously found in the mailbox
being appended to).

Nathan

[Chris Green]

Post by Nathan Stratton Treadway
So... it does appear that Mutt will preserve whatever trailing-blank-
line situation was originally created by the MDA, even as it saves the
messages into new mailboxes (and that it doesn't actually care what
convention was used by other messages previously found in the mailbox
being appended to).

Yes, exactly right, mutt just keeps what it's given but it *doesn't*
add the "blank line before 'From '" itself.

[raf]

Post by Chris Green

Post by David Champion

Post by Chris Green
I suspect my MTA doesn't agree exactly with mutt about where the
'message separator' is.

When your script delivers a message, does it append a message and then
a blank line, or does it append a blank line and then a message?

It appears to add a blank line and then the message.
Has the mutt handling of this changed in the last few versions?
The python way of doing this is correct according to the RFC as far as I
can tell, there *should* be a blank line between the end of a message
and the 'From ' starting the next message. Thus the python library I'm
using does add this blank line.

it is not correct. the blank line should be at the *end* of each message,
not at the *start*. think about it. does an mbox file start with a blank line?
no. it ends with a blank line.

Post by Chris Green
However mutt detects this as an error and outputs the messsage about the
mailbox being modified.

not an error, just a modification.

presumably because there is an extra blank line, mutt sensibly concludes
that the message that was previously the last message now has an extra
blank line in it (i.e. the line that used to be a message separator but
which is now a new last line of the message followed by the new message
separator).

Post by Chris Green
If, on the other hand, I append a message to my mbox file by hand and I
*don't* put a blank line in then mutt is quite happy. Thus mutt is quite
happy with the following in an mbox:-

because that is correct.

the python code needs to be changed to write the "From " header, then the
message, then the blank line.

cheers,
raf

[Chris Green]

Post by raf
the python code needs to be changed to write the "From " header, then the
message, then the blank line.

Then it appears to be a bug in the Python mailbox library code. I have
attached it here. If you look through you will see, in the mbox class,
the following:-

def _pre_message_hook(self, f):
"""Called before writing each message to file f."""
if f.tell() != 0:
f.write(os.linesep)

I.e. it writes a line separator to the mbox file *before* appending a
new message. I suppose it could be said that this guarantees the mbox
is correctly formatted even if the *previous* writer to the mbox hasn't
put a blank line there.

By the way I tested mutt itself and it *doesn't* put a blank line
between messages, the 'From xxxxx' line is immediately after the last
line of text of the previous message.



I have fixed the problem for myself by creating a class derived from
mailbox.mbox and overriding _pre_message_hook().

[Erik Christiansen]

Post by Chris Green
What I want to know is:-
Is it possible for a message to be delivered into an mbox that mutt
is looking at without provoking the ""Mailbox was externally
modified" message?

Yes. Here the message is "New mail in this mailbox."
(Confirmed by sending myself an email while in !)

Post by Chris Green
If the above is possible then what does the delivering MTA have to
do so that the delivered message is 'N' and all others are left
unchanged?

I'd grab the source. The message is output by curs_main.c, on detecting
(check == M_REOPENED). There are assignments to "check" in mx.c and
curs_main.c, and mbox_check_mailbox() in mbox.c tells (in comments) that
our guess at the meaning of M_REOPENED is correct. It's 2 a.m. here,
so I'll leave it to you to figure out how to make the python behave so
that it detects M_NEW_MAIL instead. (Is it respecting the file locking
in mx_lock_file(0 in mx.c?) I see that function does a sleep(). I should
do that too.

Erik