Discussion:
Composing in utf8 from latin1 terminal
Nuno Silva
2017-10-12 17:02:20 UTC
Permalink
Hello,

Recently, I have tried to use mutt on a non-utf8 terminal. Everything
works as expected in an utf8 environment, but when I compose new e-mails
in a latin1/ISO-8859-1 terminal, mutt will expect the file to be in the
same encoding as the terminal, while my text editor will save the file
in utf8. The result is that non-ASCII characters get misinterpreted,
which can affect the message headers as well (e.g. real names in To: and
Cc:).

When replying to messages, this has not been an issue so far, as mutt
includes the quoted copy of the message I'm replying to and the editor
can use that to guess the encoding.

Is there some way to configure mutt so that it always uses utf8 to read
the new message after I exit the editor? Or a way to enable some
encoding autodetection that can tell utf8 apart from latin1?
--
Thanks in advance,
Nuno Silva
Nuno Silva
2018-10-23 21:31:45 UTC
Permalink
Post by Nuno Silva
Recently, I have tried to use mutt on a non-utf8 terminal. Everything
works as expected in an utf8 environment, but when I compose new e-mails
in a latin1/ISO-8859-1 terminal, mutt will expect the file to be in the
same encoding as the terminal, while my text editor will save the file
in utf8. The result is that non-ASCII characters get misinterpreted,
which can affect the message headers as well (e.g. real names in To: and
Cc:).
[...]
Post by Nuno Silva
Is there some way to configure mutt so that it always uses utf8 to read
the new message after I exit the editor? Or a way to enable some
encoding autodetection that can tell utf8 apart from latin1?
So far I did not find a way to change this on the mutt side, but I made
a new major mode for mutt messages in Emacs (the editor I use with
mutt), with a hook that changes the file encoding to latin1 if the file
was opened in a latin1 terminal and Emacs cannot detect a non-ASCII file
encoding.

It appears to work here. I'm sure someone who is more versed in Emacs
than me would be able to come up with a more elegant solution, but I'm
sharing mine here just in case it is useful to somebody else someday:


(define-derived-mode my-mutt-message-mode message-mode "MuttMSG")
(add-hook
'my-mutt-message-mode-hook
(lambda ()
(when (equal (terminal-coding-system) 'iso-latin-1-unix)
(let ((encoding (detect-coding-region (point-min) (point-max))))
(when (or
(equal encoding '(undecided-unix))
(equal encoding '(undecided)))
(setq buffer-file-coding-system 'iso-latin-1-unix))))))
(add-to-list 'auto-mode-alist '("/mutt" . my-mutt-message-mode))
--
Nuno Silva
Ian Zimmerman
2018-10-24 20:44:24 UTC
Permalink
Post by Nuno Silva
So far I did not find a way to change this on the mutt side, but I made
a new major mode for mutt messages in Emacs (the editor I use with
mutt), with a hook that changes the file encoding to latin1 if the file
was opened in a latin1 terminal and Emacs cannot detect a non-ASCII file
encoding.
It appears to work here. I'm sure someone who is more versed in Emacs
than me would be able to come up with a more elegant solution, but I'm
(define-derived-mode my-mutt-message-mode message-mode "MuttMSG")
(add-hook
'my-mutt-message-mode-hook
(lambda ()
(when (equal (terminal-coding-system) 'iso-latin-1-unix)
(let ((encoding (detect-coding-region (point-min) (point-max))))
(when (or
(equal encoding '(undecided-unix))
(equal encoding '(undecided)))
(setq buffer-file-coding-system 'iso-latin-1-unix))))))
(add-to-list 'auto-mode-alist '("/mutt" . my-mutt-message-mode))
You could just hook message-mode-hook with a function that checks
buffer-file-name, I think that would be a bit more straightforward than
adding a new mode.

Other possibilities: you could handle this still in Emacs, but after you
finish writing, at the point you save the temporary file (with one of
the write hooks). Or you can write a script that runs Emacs and then
recodes the file outside of Emacs, using something like iconv(1).
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Nuno Silva
2018-10-25 11:24:06 UTC
Permalink
Post by Ian Zimmerman
Post by Nuno Silva
So far I did not find a way to change this on the mutt side, but I made
a new major mode for mutt messages in Emacs (the editor I use with
mutt), with a hook that changes the file encoding to latin1 if the file
was opened in a latin1 terminal and Emacs cannot detect a non-ASCII file
encoding.
It appears to work here. I'm sure someone who is more versed in Emacs
than me would be able to come up with a more elegant solution, but I'm
(define-derived-mode my-mutt-message-mode message-mode "MuttMSG")
(add-hook
'my-mutt-message-mode-hook
(lambda ()
(when (equal (terminal-coding-system) 'iso-latin-1-unix)
(let ((encoding (detect-coding-region (point-min) (point-max))))
(when (or
(equal encoding '(undecided-unix))
(equal encoding '(undecided)))
(setq buffer-file-coding-system 'iso-latin-1-unix))))))
(add-to-list 'auto-mode-alist '("/mutt" . my-mutt-message-mode))
You could just hook message-mode-hook with a function that checks
buffer-file-name, I think that would be a bit more straightforward than
adding a new mode.
Yes, it would be. In my case, I am also redefining a keybinding (C-c
C-c, and possibly more in the future), so the new mode is a way to keep
these mutt-related changes together.
Post by Ian Zimmerman
Other possibilities: you could handle this still in Emacs, but after you
finish writing, at the point you save the temporary file (with one of
the write hooks). Or you can write a script that runs Emacs and then
recodes the file outside of Emacs, using something like iconv(1).
--
Nuno Silva
Derek Martin
2018-10-24 22:30:39 UTC
Permalink
Post by Nuno Silva
Recently, I have tried to use mutt on a non-utf8 terminal. Everything
works as expected in an utf8 environment, but when I compose new e-mails
in a latin1/ISO-8859-1 terminal, mutt will expect the file to be in the
same encoding as the terminal, while my text editor will save the file
in utf8. The result is that non-ASCII characters get misinterpreted,
which can affect the message headers as well (e.g. real names in To: and
Cc:).
[...]
Post by Nuno Silva
Is there some way to configure mutt so that it always uses utf8 to read
the new message after I exit the editor? Or a way to enable some
encoding autodetection that can tell utf8 apart from latin1?
The bottom line is that your environment is misconfigured. If you
want this to work, you need to have LANG set properly at every point
along the execution path. Your terminal, terminal font, editor, and
Mutt all need to know that you're using latin1 instead of Unicode, by
having been started with a latin1 LANG setting. You may need to
configure your terminal to use the correct font, although with many
modern terminals (like gnome-term, kterm, etc.) this should be
unnecessary.

If you are launching the latin1 terminal from a shell that has its
LANG set to UTF-8, it could break (an example of this is starting
hanterm, a terminal program expressly for Korean input with EUC-KR,
with a UTF-8 locale--won't work). If the shell running inside the
terminal has LANG set to UTF-8, both Mutt and your editor could break.
If you have manual settings on any of these programs to override the
locale defined by the environment, it could break. If you don't have
all of these things set the same way, it could, and almost certainly
will break. Sometimes the breakage is subtle, e.g. if you dump the
right characters to a terminal (say, with the cat command) tht has the
right font, it will generally display them correctly, even if the
locale is wrong. But using them with programs that need to know the
locale will still break.

If you're using a Mutt setting to connect to an existing emacs
instance (via emacsclient or similar) that's already running in a
UTF-8 locale, that's broken. You need to start a new instance of
emacs whose locale is latin1.

Lastly, you may need to adjust send_charset in Mutt. It can have
multiple locales, and Mutt will pick the first one that your document
can be displayed in. For example, mine is:

set send_charset="iso-8859-1:utf-8"

If my e-mail contains no characters that need UTF-8, Mutt will choose
to send the message as iso-8859-1, but otherwise as UTF-8.

If you do those things, it should "just work" and if you don't it
won't, at least without jumping through pointless hoops to force it,
which will most likely just break other things.
--
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
Nuno Silva
2018-10-25 11:53:43 UTC
Permalink
Post by Derek Martin
Post by Nuno Silva
Recently, I have tried to use mutt on a non-utf8 terminal. Everything
works as expected in an utf8 environment, but when I compose new e-mails
in a latin1/ISO-8859-1 terminal, mutt will expect the file to be in the
same encoding as the terminal, while my text editor will save the file
in utf8. The result is that non-ASCII characters get misinterpreted,
which can affect the message headers as well (e.g. real names in To: and
Cc:).
[...]
Post by Nuno Silva
Is there some way to configure mutt so that it always uses utf8 to read
the new message after I exit the editor? Or a way to enable some
encoding autodetection that can tell utf8 apart from latin1?
The bottom line is that your environment is misconfigured. If you
want this to work, you need to have LANG set properly at every point
along the execution path. Your terminal, terminal font, editor, and
Mutt all need to know that you're using latin1 instead of Unicode, by
having been started with a latin1 LANG setting. You may need to
configure your terminal to use the correct font, although with many
modern terminals (like gnome-term, kterm, etc.) this should be
unnecessary.
If you are launching the latin1 terminal from a shell that has its
LANG set to UTF-8, it could break (an example of this is starting
hanterm, a terminal program expressly for Korean input with EUC-KR,
with a UTF-8 locale--won't work). If the shell running inside the
terminal has LANG set to UTF-8, both Mutt and your editor could break.
If you have manual settings on any of these programs to override the
locale defined by the environment, it could break. If you don't have
all of these things set the same way, it could, and almost certainly
will break. Sometimes the breakage is subtle, e.g. if you dump the
right characters to a terminal (say, with the cat command) tht has the
right font, it will generally display them correctly, even if the
locale is wrong. But using them with programs that need to know the
locale will still break.
If you're using a Mutt setting to connect to an existing emacs
instance (via emacsclient or similar) that's already running in a
UTF-8 locale, that's broken. You need to start a new instance of
emacs whose locale is latin1.
I haven't noticed this before, but there *is* indeed a difference when
starting a fresh new Emacs instance instead of connecting to an existing
one using emacsclient: the new instance does use latin1 to read/write
files. (That is, the behaviour expected by mutt.)

When I use emacsclient, the interface locale is not broken: the terminal
I/O encoding is correctly set from the locale. The only difference (that
I know of) is that Emacs will use utf8 to read/write files. If this
should match the terminal encoding, then it *is* broken.

I might be happy with the way things are now (as my files are usually in
utf8, and mutt is the only context where I need the file encoding to
match the terminal), but I won't claim it isn't broken if it is.
Post by Derek Martin
Lastly, you may need to adjust send_charset in Mutt. It can have
multiple locales, and Mutt will pick the first one that your document
set send_charset="iso-8859-1:utf-8"
If my e-mail contains no characters that need UTF-8, Mutt will choose
to send the message as iso-8859-1, but otherwise as UTF-8.
If you do those things, it should "just work" and if you don't it
won't, at least without jumping through pointless hoops to force it,
which will most likely just break other things.
send_charset appears to be working correctly here, I've checked it a
couple days ago. It isn't even set in any configuration file, so I
suppose it is using the default setting.

For now, I will leave the Emacs hack in place, as I prefer to use the
Emacs "server instance" instead of creating a new one. Everything else
is hopefully correctly set, as this has been the only encoding problem
I've had in the past months. (Now that I've said this, I will probably
discover a new one tomorrow...)
--
Nuno Silva
Derek Martin
2018-10-25 16:11:52 UTC
Permalink
Post by Nuno Silva
When I use emacsclient, the interface locale is not broken: the terminal
I/O encoding is correctly set from the locale. The only difference (that
I know of) is that Emacs will use utf8 to read/write files. If this
should match the terminal encoding, then it *is* broken.
I suspected as much. The issue, as far as I can tell based on your
description of things, is that the emacs *server* is running with a
UTF-8 locale, which is why it is editing files in that locale, and
you're connecting to it from a client that's running a different
locale. That's definitely broken, as would be any such locale
mismatch. Your hack may well work around it... but you should be
very clear that you're doing something unexpected (and generally
undesirable) which may well break in other ways later. That's what's
called a gross hack.

A likely better solution would be to run two instances of emacs
server, one in each locale, and make mutt connect to the right one.
But either way it comes down to the choice of running two different
instances of emacs, or using a gross hack to avoid that.
--
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
Derek Martin
2018-10-25 16:33:09 UTC
Permalink
Post by Nuno Silva
When I use emacsclient, the interface locale is not broken: the terminal
I/O encoding is correctly set from the locale. The only difference (that
I know of) is that Emacs will use utf8 to read/write files. If this
should match the terminal encoding, then it *is* broken.
Also, depending on exactly why you're doing this multi-locale stuff,
an even better solution may be to let Mutt's send_charset handle it
for you. If set properly, you should be able to compose your messages
in UTF-8, and as long as you don't use any non-latin1 characters,
send_charset *should* make sure the message goes out encoded as
latin1.

The biggest drawback to this approach is you have to be very careful
to not use any non-latin1 characters, or else the message will be sent
as unicode. Otherwise, you'd need to check every message before you
send it to make sure Mutt will send it as the desired encoding. I
have a vague notion that certain other message transforms, like PGP,
may also interfere with this, but I'm not 100% sure.
--
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
Nuno Silva
2018-10-25 17:14:18 UTC
Permalink
Post by Derek Martin
Post by Nuno Silva
When I use emacsclient, the interface locale is not broken: the terminal
I/O encoding is correctly set from the locale. The only difference (that
I know of) is that Emacs will use utf8 to read/write files. If this
should match the terminal encoding, then it *is* broken.
Also, depending on exactly why you're doing this multi-locale stuff,
an even better solution may be to let Mutt's send_charset handle it
for you. If set properly, you should be able to compose your messages
in UTF-8, and as long as you don't use any non-latin1 characters,
send_charset *should* make sure the message goes out encoded as
latin1.
The biggest drawback to this approach is you have to be very careful
to not use any non-latin1 characters, or else the message will be sent
as unicode. Otherwise, you'd need to check every message before you
send it to make sure Mutt will send it as the desired encoding. I
have a vague notion that certain other message transforms, like PGP,
may also interfere with this, but I'm not 100% sure.
But, if I understood the purpose of send_charset correctly, it only
affects the encoding of the outgoing message. It won't change the
encoding with which mutt reads the temporary file after I close the text
editor, will it?
--
Nuno Silva
Derek Martin
2018-10-25 18:06:07 UTC
Permalink
Post by Nuno Silva
Post by Derek Martin
Also, depending on exactly why you're doing this multi-locale stuff,
an even better solution may be to let Mutt's send_charset handle it
for you.
[...]
Post by Nuno Silva
But, if I understood the purpose of send_charset correctly, it only
affects the encoding of the outgoing message. It won't change the
encoding with which mutt reads the temporary file after I close the text
editor, will it?
No, but why would you care? It's just an intermediate data encoding.
What I'm saying is, forget about your latin1 terminal, do EVERYTHING
in Unicode, and let Mutt send the data out as latin1 if appropriate.
This may not be appropriate for your use case--it comes down to your
reason for using a latin1 terminal to run Mutt in. I can't think of a
good reason to do that any longer... so my assumption is your end goal
is to produce an outgoing message in latin1 so some recipients who
can't yet handle Unicode properly can read it.

If your purpose is something else, that may not work, but like I said
I can't think of a reason why you'd need to do that. A unicode
terminal should have no trouble letting you enter Portuguese (or
whatever other language) so long as your locale is correct (and
consistent across all programs) and your IME is configured properly.
Then Mutt can convert to latin1 to send.
--
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
Ian Zimmerman
2018-10-27 17:25:47 UTC
Permalink
Post by Derek Martin
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
Oh, I feel the itch again. Ow-ow, it's unbearable! I must scratch,

Has anyone tried to verify Derek's GPG signature on his message?
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Ian Zimmerman
2018-10-27 23:53:32 UTC
Permalink
Post by Ian Zimmerman
Oh, I feel the itch again. Ow-ow, it's unbearable! I must scratch,
Has anyone tried to verify Derek's GPG signature on his message?
Checking, checking, checking gently,
Why it fails, that beats me.
Bad sigs rub so unpleasantly.
Let us check my own sweet key.
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Ian Zimmerman
2018-10-28 00:19:23 UTC
Permalink
Post by Ian Zimmerman
Post by Ian Zimmerman
Oh, I feel the itch again. Ow-ow, it's unbearable! I must scratch,
Has anyone tried to verify Derek's GPG signature on his message?
Checking, checking, checking gently,
Why it fails, that beats me.
Bad sigs rub so unpleasantly.
Let us check my own sweet key.
Ok, that worked. Now let's try this:
This line ends with a couple of dangling spaces.

Now _that_ would be really stupid!
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Ken Moffat
2018-10-28 00:39:49 UTC
Permalink
Post by Ian Zimmerman
Post by Ian Zimmerman
Post by Ian Zimmerman
Oh, I feel the itch again. Ow-ow, it's unbearable! I must scratch,
Has anyone tried to verify Derek's GPG signature on his message?
Checking, checking, checking gently,
Why it fails, that beats me.
Bad sigs rub so unpleasantly.
Let us check my own sweet key.
This line ends with a couple of dangling spaces.
Now _that_ would be really stupid!
For the *very* little it is worth:

1. On lkml I typically get successful and failed verifications from
the same people.

2. On the few occasions I have bothered to sign posts, the list copy
that comes back to me fails to verify.

3. Neither the post you were asking about, nor either of your tests,
passed verification here.

/me wonders why I ever bothered to set up a key, since it wastes
time when reading lists, and only randomly works. ;-)

ĸen
--
Is it about a bicycle ?
Claus Assmann
2018-10-28 01:23:50 UTC
Permalink
Post by Ken Moffat
3. Neither the post you were asking about, nor either of your tests,
passed verification here.
Just FYI: both test mails passed verification for me.
Ian Zimmerman
2018-10-28 03:37:48 UTC
Permalink
Post by Claus Assmann
Just FYI: both test mails passed verification for me.
AAAARGH!! I am losing my mind!!

Here are the intermediate Received headers of the 2nd test mail as it
came back to me. Can you share the ones in your copy?

Received: from localhost (localhost [127.0.0.1])
by fraxinus.osuosl.org (Postfix) with ESMTP id D1C788647F;
Sun, 28 Oct 2018 00:22:33 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from fraxinus.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id Ml_6-3RG8bAN; Sun, 28 Oct 2018 00:22:33 +0000 (UTC)
Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34])
by fraxinus.osuosl.org (Postfix) with ESMTP id 3765E85F89;
Sun, 28 Oct 2018 00:22:33 +0000 (UTC)
X-Original-To: mutt-***@mutt.org
Delivered-To: mutt-***@osuosl.org
Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136])
by ash.osuosl.org (Postfix) with ESMTP id 10D901BF2F0
for <mutt-***@mutt.org>; Sun, 28 Oct 2018 00:22:32 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by silver.osuosl.org (Postfix) with ESMTP id 0E833220C2
for <mutt-***@mutt.org>; Sun, 28 Oct 2018 00:22:32 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from silver.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id IzrZogh12v0Z for <mutt-***@mutt.org>;
Sun, 28 Oct 2018 00:22:31 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Claus Assmann
2018-10-28 11:19:46 UTC
Permalink
Post by Ian Zimmerman
Post by Claus Assmann
Just FYI: both test mails passed verification for me.
AAAARGH!! I am losing my mind!!
Hopefully it's backed up somewhere...
Post by Ian Zimmerman
Here are the intermediate Received headers of the 2nd test mail as it
came back to me. Can you share the ones in your copy?
I deleted the mails already :-(
However, below are the Received headers of your most recent mail.

If you kept a copies of the mails which you originally sent
and which you got back: what's the "diff"?

Moreover, I downloaded the "raw" message from the MARC archive,
added some headers, and it verifies too. So I'm attaching that
as .gz file so it doesn't get messed up by some mail software.


Received: from localhost (localhost [127.0.0.1])
by silver.osuosl.org (Postfix) with ESMTP id A9B91227F9;
Sun, 28 Oct 2018 03:42:34 +0000 (UTC)
Received: from silver.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id R3tMY4OIzjA6; Sun, 28 Oct 2018 03:42:33 +0000 (UTC)
Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34])
by silver.osuosl.org (Postfix) with ESMTP id 4C728227C1;
Sun, 28 Oct 2018 03:42:33 +0000 (UTC)
Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138])
by ash.osuosl.org (Postfix) with ESMTP id C57471BF61D
for <mutt-***@mutt.org>; Sun, 28 Oct 2018 03:42:32 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by whitealder.osuosl.org (Postfix) with ESMTP id C2B3B86767
for <mutt-***@mutt.org>; Sun, 28 Oct 2018 03:42:32 +0000 (UTC)
Received: from whitealder.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id 9o-8499bbnC7 for <mutt-***@mutt.org>;
Sun, 28 Oct 2018 03:42:32 +0000 (UTC)
Received: from very.loosely.org (very.loosely.org [173.255.215.69])
by whitealder.osuosl.org (Postfix) with ESMTPS id 2D87886761
for <mutt-***@mutt.org>; Sun, 28 Oct 2018 03:42:32 +0000 (UTC)
Received: from itz by ahiker.mooo.com with local (Exim 4.91_26-9591063a)
(envelope-from <***@ahiker.mooo.com>) id 1gGbtd-0000PD-8F
for mutt-***@mutt.org; Sat, 27 Oct 2018 20:37:49 -0700
Ian Zimmerman
2018-10-28 15:55:00 UTC
Permalink
Post by Claus Assmann
Post by Ian Zimmerman
AAAARGH!! I am losing my mind!!
Hopefully it's backed up somewhere...
;-)
Post by Claus Assmann
If you kept a copies of the mails which you originally sent
and which you got back: what's the "diff"?
Indeed I have kept a file copy. The diff confirms my suspicion that the
dangling spaces were the trigger. I got this suspicion by looking at
Derek's messages, of which one verifies for me and the other does not.

@@ -33,7 +89,7 @@
Post by Claus Assmann
Let us check my own sweet key.
Ok, that worked. Now let's try this:=20
-This line ends with a couple of dangling spaces. =20
+This line ends with a couple of dangling spaces.=20=20=20=20

Now _that_ would be really stupid!

This does not tell us at which hop it happens, which software is
responsible, or (especially) why it depends on the _recipient_ aside
from all other things.
Post by Claus Assmann
Received: from localhost (localhost [127.0.0.1])
by silver.osuosl.org (Postfix) with ESMTP id A9B91227F9;
Sun, 28 Oct 2018 03:42:34 +0000 (UTC)
Received: from silver.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id R3tMY4OIzjA6; Sun, 28 Oct 2018 03:42:33 +0000 (UTC)
Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34])
by silver.osuosl.org (Postfix) with ESMTP id 4C728227C1;
Sun, 28 Oct 2018 03:42:33 +0000 (UTC)
Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138])
by ash.osuosl.org (Postfix) with ESMTP id C57471BF61D
Received: from localhost (localhost [127.0.0.1])
by whitealder.osuosl.org (Postfix) with ESMTP id C2B3B86767
Received: from whitealder.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
Sun, 28 Oct 2018 03:42:32 +0000 (UTC)
Perhaps there is a little clue here. Some kind of load balancing seems
to be in effect at osuosl, different hosts (silver, whitealder,
fraxinus, ash) are involved each time. Possibly different amavis
configuration on some of the hosts handling the outbound traffic. Yes,
I suspect amavis more than the other pieces; it should be the only piece
that takes apart the MIME structure.
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Kevin J. McCarthy
2018-10-28 16:35:47 UTC
Permalink
Post by Ian Zimmerman
Perhaps there is a little clue here. Some kind of load balancing seems
to be in effect at osuosl, different hosts (silver, whitealder,
fraxinus, ash) are involved each time. Possibly different amavis
configuration on some of the hosts handling the outbound traffic. Yes,
I suspect amavis more than the other pieces; it should be the only piece
that takes apart the MIME structure.
My Received headers match what you posted for the "Test number two"
email, and all the signatures verify for me. In fact, I don't recall
seeing any failing signatures on this list, before or after the switch
to OSUOSL.

I would suspect something between OSUOSL and your spoolfile is causing
the problem.
--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA
Ian Zimmerman
2018-10-28 01:29:00 UTC
Permalink
Post by Ian Zimmerman
This line ends with a couple of dangling spaces.
Now _that_ would be really stupid!
And so it is - that one failed :-(

The best available conclusion is that at one the the hops on osuosl.org,
mails get rewritten in a way similar to what I described at the link
below.

https://very.loosely.org/itz-blog/the-problem-with-gpg-signatures.html

Which is depressing because the software involved seems to be all FOSS -
mailman, postfix and amavis. Should I guess which one is responsible?
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Ian Zimmerman
2018-10-28 19:25:23 UTC
Permalink
Re-sending this to see if it is a reliable repro case.
Post by Ian Zimmerman
Post by Ian Zimmerman
Oh, I feel the itch again. Ow-ow, it's unbearable! I must scratch,
Has anyone tried to verify Derek's GPG signature on his message?
Checking, checking, checking gently,
Why it fails, that beats me.
Bad sigs rub so unpleasantly.
Let us check my own sweet key.
Ok, that worked. Now let's try this:
This line ends with a couple of dangling spaces.

Now _that_ would be really stupid!
--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
Derek Martin
2018-10-28 23:13:46 UTC
Permalink
Post by Ian Zimmerman
Post by Derek Martin
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
Oh, I feel the itch again. Ow-ow, it's unbearable! I must scratch,
Has anyone tried to verify Derek's GPG signature on his message?
It verifies for me. =8^)

It's been brought to my attention multiple times that some of my
messages do not properly verify, FOR SOME PEOPLE, while others do.
FWIW the first occurence of this happened long before any switch of
provider (either mine or Mutt's). Fun stuff.
--
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
Mark H. Wood
2018-10-29 13:25:34 UTC
Permalink
Derek's message verified here.
Both of Ian's test messages verified here.
--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu
Loading...