What are the current fetchmail/getmail and/or procmail/maildrop utilities?

[Chris Green]

I currently have my mail delivered to my desktop system using SMTP as
the system is on all the time and has a static IP.

However I always get paranoid when I reconfigure it and/or do other
maintenance so I'm considering moving back to a fetchmail/getmail based
system.

I also have a fairly complex mail filtering script I wrote myself in
Python which is fed mail via .forward.


What's the current "state of the art" way to collect mail and deliver it
through a filtering system to mutt? If I can do this all in one program
than so much the better but I'm happy with two programs if that would
work better. I can stay with my existing filter system but, again, if I
can consolidate things into one, easier to maintain, chunk then I'd be
happy.


I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.


What does everyone else here do for collecting mail and filtering mail
with mutt?

[Derek Martin]

Post by Chris Green
What does everyone else here do for collecting mail and filtering mail
with mutt?

Fetchmail and procmail. Ugly, but ubiquitous and reliable. A friend
pointed me at something "better" for mail filtering, but I can't
recall what it was... mainly because I haven't gotten around to
looking into it, on account of the fact that my current solution works
well and requires no learning curve. Learning a new mail filter
system is very low on the priority list.

[Peter Davis]

Post by Derek Martin

Post by Chris Green
What does everyone else here do for collecting mail and filtering mail
with mutt?

Fetchmail and procmail. Ugly, but ubiquitous and reliable.

Same here. I keep meaning to hook in an adaptive spam filter, but I
haven't bothered so far. Maybe mutt just makes it so easy to quickly
triage my mail that it hasn't seemed worth it.

-pd

[Mark H. Wood]

Post by Derek Martin

Post by Chris Green
What does everyone else here do for collecting mail and filtering mail
with mutt?

Fetchmail and procmail. Ugly, but ubiquitous and reliable. A friend
pointed me at something "better" for mail filtering, but I can't
recall what it was... mainly because I haven't gotten around to
looking into it, on account of the fact that my current solution works
well and requires no learning curve. Learning a new mail filter
system is very low on the priority list.

I'm guessing 'sieve'. Haven't tried it.

[Nikola Petrov]

Post by Chris Green
I currently have my mail delivered to my desktop system using SMTP as
the system is on all the time and has a static IP.
However I always get paranoid when I reconfigure it and/or do other
maintenance so I'm considering moving back to a fetchmail/getmail based
system.
I also have a fairly complex mail filtering script I wrote myself in
Python which is fed mail via .forward.
What's the current "state of the art" way to collect mail and deliver it
through a filtering system to mutt? If I can do this all in one program
than so much the better but I'm happy with two programs if that would
work better. I can stay with my existing filter system but, again, if I
can consolidate things into one, easier to maintain, chunk then I'd be
happy.
I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.
What does everyone else here do for collecting mail and filtering mail
with mutt?

I am using imapfilter with lua configuration file for my imap account.
That does the job for me and I like the fact that I declare my filters
with actual code(be it lua, python or what not)


Best, Nikola

[Chris Green]

Post by Nikola Petrov

Post by Chris Green
I currently have my mail delivered to my desktop system using SMTP as
the system is on all the time and has a static IP.
However I always get paranoid when I reconfigure it and/or do other
maintenance so I'm considering moving back to a fetchmail/getmail based
system.
I also have a fairly complex mail filtering script I wrote myself in
Python which is fed mail via .forward.
What's the current "state of the art" way to collect mail and deliver it
through a filtering system to mutt? If I can do this all in one program
than so much the better but I'm happy with two programs if that would
work better. I can stay with my existing filter system but, again, if I
can consolidate things into one, easier to maintain, chunk then I'd be
happy.
I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.
What does everyone else here do for collecting mail and filtering mail
with mutt?

I am using imapfilter with lua configuration file for my imap account.
That does the job for me and I like the fact that I declare my filters
with actual code(be it lua, python or what not)

How does that actually work? I've found its home at https://github.com/lefcha/imapfilter
but the documentation doesn't really tell me what it does (maybe my fault!).

Does it move E-Mails around on the IMAP server, or does it collect them
from the IMAP server and deliver them to you locally? Or does it do
something else?

[Nikola Petrov]

Post by Chris Green

Post by Nikola Petrov

Post by Chris Green
I currently have my mail delivered to my desktop system using SMTP as
the system is on all the time and has a static IP.
However I always get paranoid when I reconfigure it and/or do other
maintenance so I'm considering moving back to a fetchmail/getmail based
system.
I also have a fairly complex mail filtering script I wrote myself in
Python which is fed mail via .forward.
What's the current "state of the art" way to collect mail and deliver it
through a filtering system to mutt? If I can do this all in one program
than so much the better but I'm happy with two programs if that would
work better. I can stay with my existing filter system but, again, if I
can consolidate things into one, easier to maintain, chunk then I'd be
happy.
I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.
What does everyone else here do for collecting mail and filtering mail
with mutt?

I am using imapfilter with lua configuration file for my imap account.
That does the job for me and I like the fact that I declare my filters
with actual code(be it lua, python or what not)

How does that actually work? I've found its home at https://github.com/lefcha/imapfilter
but the documentation doesn't really tell me what it does (maybe my fault!).
Does it move E-Mails around on the IMAP server, or does it collect them
from the IMAP server and deliver them to you locally? Or does it do
something else?

No it doesn't deliver them to you. It sort of filters them online on the
server. You can then use something like offlineimap to deliver them
locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
far from happy with my setup at the moment.

Best, Nikola

[Chris Green]

Post by Nikola Petrov

Post by Chris Green

Post by Nikola Petrov
I am using imapfilter with lua configuration file for my imap account.
That does the job for me and I like the fact that I declare my filters
with actual code(be it lua, python or what not)

How does that actually work? I've found its home at https://github.com/lefcha/imapfilter
but the documentation doesn't really tell me what it does (maybe my fault!).
Does it move E-Mails around on the IMAP server, or does it collect them
from the IMAP server and deliver them to you locally? Or does it do
something else?

No it doesn't deliver them to you. It sort of filters them online on the
server. You can then use something like offlineimap to deliver them
locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
far from happy with my setup at the moment.

OK, thanks for the description, I don't think it's quite where I want to
be.

[Chris Bannister]

Post by Nikola Petrov
No it doesn't deliver them to you. It sort of filters them online on the
server. You can then use something like offlineimap to deliver them
locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
far from happy with my setup at the moment.

So what would you prefer?

[Chris Bannister]

Post by Chris Bannister

Post by Nikola Petrov
No it doesn't deliver them to you. It sort of filters them online on the
server. You can then use something like offlineimap to deliver them
locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
far from happy with my setup at the moment.

So what would you prefer?

Hi,
I am not sure I understand your question?

Ummm, Is "What would make you more happy?" a better question?

[Jamie Paul Griffin]

/ Nikola Petrov wrote on Wed 7.Nov'12 at 19:17:46 +0200 /

Post by Chris Green
I currently have my mail delivered to my desktop system using SMTP as
the system is on all the time and has a static IP.
However I always get paranoid when I reconfigure it and/or do other
maintenance so I'm considering moving back to a fetchmail/getmail based
system.

May I ask what it is that you are worried about using smtp delivery - I take it you have various protective measures in place with your configuration?

[Chris Green]

Post by Jamie Paul Griffin
/ Nikola Petrov wrote on Wed 7.Nov'12 at 19:17:46 +0200 /

Post by Chris Green
I currently have my mail delivered to my desktop system using SMTP as
the system is on all the time and has a static IP.
However I always get paranoid when I reconfigure it and/or do other
maintenance so I'm considering moving back to a fetchmail/getmail based
system.

May I ask what it is that you are worried about using smtp delivery - I take it you have various protective measures in place with your configuration?

No specific "protective measures" at all, it just relies on the sending
server retrying if my SMTP server isn't running (or connected). That's
one of the reasons I'd quite like to move away from SMTP. It *should*
be OK but I'm relying on the other end to behave properly.

[Derek Martin]

Post by Chris Green
No specific "protective measures" at all, it just relies on the sending
server retrying if my SMTP server isn't running (or connected). That's
one of the reasons I'd quite like to move away from SMTP. It *should*
be OK but I'm relying on the other end to behave properly.

It will. It has to. If it didn't, e-mail on the internet would be
horribly unreliable.

[David Champion]

Post by Derek Martin

Post by Chris Green
server retrying if my SMTP server isn't running (or connected). That's
one of the reasons I'd quite like to move away from SMTP. It *should*
be OK but I'm relying on the other end to behave properly.

It will. It has to. If it didn't, e-mail on the internet would be
horribly unreliable.

I hate to break it to you, but.... :)

I've used IMAP pickup in the past and it's OK for some IMAP servers. A
year or two ago my employer moved my mailbox to MS Exchange. Exchange
doesn't (necessarily?) hand you the exact e-mail it received. It
parses incoming mail, stores the parsed components, and reconstructs
the message the best it can figure when you pick it up via IMAP or POP.
Along the way it might modify or remove some components for no good
reason; for example, multipart/alternative with text/plan and text/html
invisibly becomes just a text/html message. I've also heard of its
breaking crypto, although I haven't seen that myself for a while.

So I forward my mail via SMTP away from my employer now.

[Derek Martin]

Post by David Champion

Post by Derek Martin

Post by Chris Green
server retrying if my SMTP server isn't running (or connected). That's
one of the reasons I'd quite like to move away from SMTP. It *should*
be OK but I'm relying on the other end to behave properly.

It will. It has to. If it didn't, e-mail on the internet would be
horribly unreliable.

I hate to break it to you, but.... :)
I've used IMAP pickup in the past and it's OK for some IMAP servers.

Right, we're talking about SMTP delivery to a machine that's sometimes
down or unreachable... Very reliable. :) Presumably that's exactly
why you're using SMTP to deal with your exchange problem!

[Jamie Paul Griffin]

/ David Champion wrote on Wed 7.Nov'12 at 16:33:58 -0600 /

Post by David Champion

Post by Derek Martin

Post by Chris Green
server retrying if my SMTP server isn't running (or connected). That's
one of the reasons I'd quite like to move away from SMTP. It *should*
be OK but I'm relying on the other end to behave properly.

It will. It has to. If it didn't, e-mail on the internet would be
horribly unreliable.

I hate to break it to you, but.... :)
I've used IMAP pickup in the past and it's OK for some IMAP servers. A
year or two ago my employer moved my mailbox to MS Exchange. Exchange
doesn't (necessarily?) hand you the exact e-mail it received. It
parses incoming mail, stores the parsed components, and reconstructs
the message the best it can figure when you pick it up via IMAP or POP.
Along the way it might modify or remove some components for no good
reason; for example, multipart/alternative with text/plan and text/html
invisibly becomes just a text/html message. I've also heard of its
breaking crypto, although I haven't seen that myself for a while.
So I forward my mail via SMTP away from my employer now.
--

Yes i think the benefits of using your own smtp delivery are worth it.

[Andre Klärner]

Post by Jamie Paul Griffin
Yes i think the benefits of using your own smtp delivery are worth it.

I can only agree. And to avoid issues when my landline is down I have a VM
on a big hoster that on one side delivers all my locally generated mails to
avoid the dialin IP address problem. And on the other side it acts as the
backup MX that stores my mails until my landline is back online and it can
be delivered at my home.

regards,
Andre

[Jeremy Kitchen]

Post by Andre Klärner

Post by Jamie Paul Griffin
Yes i think the benefits of using your own smtp delivery are worth it.

I can only agree. And to avoid issues when my landline is down I have a VM
on a big hoster that on one side delivers all my locally generated mails to
avoid the dialin IP address problem. And on the other side it acts as the
backup MX that stores my mails until my landline is back online and it can
be delivered at my home.

IMO, a better way to do this would be to have your current backup MX be
your primary (and only?) and set it to have a high retry time, possibly
even setting up something like ETRN[1] to trigger the remote MTA to flush
its queue to you.

You may also be able to configure your mail server to have a separate
retry time for incoming vs outgoing mail. I *think* postfix can do this?
Seems like something it would be able to do.

[1]: http://www.postfix.org/ETRN_README.html

I'm sure other MTAs support ETRN, but that was the first hit on google,
and I'm a postfix user (retired qmail user, and qmail does *not* support
ETRN) so it seemed prudent :)

-Jeremy

[Jeremy Kitchen]

Post by David Champion
I've used IMAP pickup in the past and it's OK for some IMAP servers. A
year or two ago my employer moved my mailbox to MS Exchange. Exchange
doesn't (necessarily?) hand you the exact e-mail it received. It
parses incoming mail, stores the parsed components, and reconstructs
the message the best it can figure when you pick it up via IMAP or POP.
Along the way it might modify or remove some components for no good
reason; for example, multipart/alternative with text/plan and text/html
invisibly becomes just a text/html message. I've also heard of its
breaking crypto, although I haven't seen that myself for a while.

I haven't had it break crypto, but I'm one of 2 people at the company
doing pgp signatures and both of us send *only* text/plain.

I have had it give me text/plain only when there was an html part, which
normally I wouldn't complain about, but if someone used an html link in
their email, I *never* see the link or the url.

Post by David Champion
So I forward my mail via SMTP away from my employer now.

I would love to do this, if for no other reason than I can have better
server-side filtering, but I very highly doubt the company would go for
it.

Otherwise, mutt seems to work just fine with exchange. I do need to set
up lbdb to pull from our exchange server at some point, but fortunately
I interact with only a very small subset of the company, so my aliases
file suffices for this, and if I need to look up someone's address I can
always open up OWA.

-Jeremy

[David Champion]

Post by Jeremy Kitchen
I haven't had it break crypto, but I'm one of 2 people at the company
doing pgp signatures and both of us send *only* text/plain.

My memory is fuzzy but I think it was more complex multipart signed
messages that it broke.

Post by Jeremy Kitchen
I have had it give me text/plain only when there was an html part, which
normally I wouldn't complain about, but if someone used an html link in
their email, I *never* see the link or the url.

I may have it backwards. OTOH it may have both problems.

Post by Jeremy Kitchen
Otherwise, mutt seems to work just fine with exchange. I do need to set
up lbdb to pull from our exchange server at some point, but fortunately
I interact with only a very small subset of the company, so my aliases
file suffices for this, and if I need to look up someone's address I can
always open up OWA.

True, mutt does pretty well. If you're compelled to use Exchange, mutt
remains a good option.

You can set a query_command that looks people up in AD, theoretically.
I've never done this but I suspect that I could. Maybe that's
functionally what you mean to do with lbdb, though?

(Our campus uses both AD and LDAP, and LDAP is used for first-pass mail
routing, so I'm querying it alone for now.

[Chris Green]

Post by David Champion

Post by Derek Martin

Post by Chris Green
server retrying if my SMTP server isn't running (or connected). That's
one of the reasons I'd quite like to move away from SMTP. It *should*
be OK but I'm relying on the other end to behave properly.

It will. It has to. If it didn't, e-mail on the internet would be
horribly unreliable.

I hate to break it to you, but.... :)
I've used IMAP pickup in the past and it's OK for some IMAP servers. A
year or two ago my employer moved my mailbox to MS Exchange. Exchange
doesn't (necessarily?) hand you the exact e-mail it received. It
parses incoming mail, stores the parsed components, and reconstructs
the message the best it can figure when you pick it up via IMAP or POP.
Along the way it might modify or remove some components for no good
reason; for example, multipart/alternative with text/plan and text/html
invisibly becomes just a text/html message. I've also heard of its
breaking crypto, although I haven't seen that myself for a while.
So I forward my mail via SMTP away from my employer now.

Yes, I realise there are good and bad places that deliver mail to you! :-)

If I move from SMTP delivery to collecting it myself (from POP3 or IMAP)
it will be from exactly the same source, my Tsohost web hosting service
which (in many people's opinions) is excellent.

All I have to do is change the ultimate destination of all my E-Mail
from zbmc.eu (whose MX record points at my home LAN) to a POP3 mailbox
on the TsoHost servers and then collect from that POP3 mailbox.

I'm thinking in fact that I'm going to stay with much the same system as
I already have but my Python filter script will collect E-Mail direct
from the Tsohost POP3 server instead of having it fed into its standard
input by the .forward. I already have *another* Python script that
collects mail from a Tsohost POP3 box so I know how to do that already,
all I need to do is merge a few bits of existing code.

[Jamie Paul Griffin]

/ Chris Green wrote on Thu 8.Nov'12 at 10:51:59 +0000 /

Post by Chris Green

Post by David Champion

Post by Derek Martin

Post by Chris Green
server retrying if my SMTP server isn't running (or connected). That's
one of the reasons I'd quite like to move away from SMTP. It *should*
be OK but I'm relying on the other end to behave properly.

It will. It has to. If it didn't, e-mail on the internet would be
horribly unreliable.

I hate to break it to you, but.... :)
I've used IMAP pickup in the past and it's OK for some IMAP servers. A
year or two ago my employer moved my mailbox to MS Exchange. Exchange
doesn't (necessarily?) hand you the exact e-mail it received. It
parses incoming mail, stores the parsed components, and reconstructs
the message the best it can figure when you pick it up via IMAP or POP.
Along the way it might modify or remove some components for no good
reason; for example, multipart/alternative with text/plan and text/html
invisibly becomes just a text/html message. I've also heard of its
breaking crypto, although I haven't seen that myself for a while.
So I forward my mail via SMTP away from my employer now.

Yes, I realise there are good and bad places that deliver mail to you! :-)
If I move from SMTP delivery to collecting it myself (from POP3 or IMAP)
it will be from exactly the same source, my Tsohost web hosting service
which (in many people's opinions) is excellent.
All I have to do is change the ultimate destination of all my E-Mail
from zbmc.eu (whose MX record points at my home LAN) to a POP3 mailbox
on the TsoHost servers and then collect from that POP3 mailbox.
I'm thinking in fact that I'm going to stay with much the same system as
I already have but my Python filter script will collect E-Mail direct
from the Tsohost POP3 server instead of having it fed into its standard
input by the .forward. I already have *another* Python script that
collects mail from a Tsohost POP3 box so I know how to do that already,
all I need to do is merge a few bits of existing code.
--
Chris Green

Hi Chris, personally, i'd stick with what your current set-up. Since I set up my own mta and security software i've never been happier with it. I much prefer the flexibility of being able to control almost all aspects of my mail delivery, reading and sending; including the DNS configuration etc...

What I did do, though, is look into OS's and software that provide the security we all need, whether collecting mail from remote IMAP storage or POP3, or having it routed directly using smtp. I stick with BSD systems and use a number of different security software on my end to deal with zombies, malware filtering, spam filtering, etc. They are all generally well documented as you no doubt know, and certainly great efforts have been made to make installation and configuration of these software easy(-ish).

The only downfall I think is the time involved with setting up and upgrading and monitoring these things. So, do you have the time and will to invest in doing that? As you've written some nice tools already, it sounds like you know what you're doing so stick with it i'd say. Plus, for me, I find it great fun and interesting learning about these things. I've learned loads by getting stuck in and engrossed in it all. Some people, however, have been there and done that and simply just can't be bothered with it anymore and/or would rather spend their time on something else.

Jamie

[Derek Martin]

Post by Jamie Paul Griffin
Hi Chris, personally, i'd stick with what your current set-up.

Ditto. I don't currently do this but that's only because port 25 is
blocked by my ISP. I've run my mail this way before and would do it
again if it were a practical option.

[Patrick Shanahan]

Post by Derek Martin

Post by Jamie Paul Griffin
Hi Chris, personally, i'd stick with what your current set-up.

Ditto. I don't currently do this but that's only because port 25 is
blocked by my ISP. I've run my mail this way before and would do it
again if it were a practical option.

Why not change postfix master.cf to use another port > 1024? Not a big
thing, one line to edit and restart postfix.

[Chris Green]

Post by Derek Martin

Post by Jamie Paul Griffin
Hi Chris, personally, i'd stick with what your current set-up.

Ditto. I don't currently do this but that's only because port 25 is
blocked by my ISP. I've run my mail this way before and would do it
again if it were a practical option.

Er, we're talking (or at least I'm talking) about mail delivery *from*
your ISP to your computer at home (or wherever). Your ISP blocking port
25 won't affect this at all, it's your receiving computer that needs to
have port 25 open.

[Jamie Paul Griffin]

/ Chris Green wrote on Thu 8.Nov'12 at 18:13:10 +0000 /

Post by Chris Green

Post by Derek Martin

Post by Jamie Paul Griffin
Hi Chris, personally, i'd stick with what your current set-up.

Ditto. I don't currently do this but that's only because port 25 is
blocked by my ISP. I've run my mail this way before and would do it
again if it were a practical option.

Er, we're talking (or at least I'm talking) about mail delivery *from*
your ISP to your computer at home (or wherever). Your ISP blocking port
25 won't affect this at all, it's your receiving computer that needs to
have port 25 open.
--
Chris Green

Yes as far as I know, the ISP blocking port 25 thing is for sending mail out, in which case you can use your mta to send through a smarthost. Receiving I set up DNS to forward mail to my IP, and port forwarding on my router at home to the machine that has an smtp daemon listening on port 25 or as Patrick said, listening on a different port like the submission port for example.

[Tim Gray]

Post by Chris Green
I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.
What does everyone else here do for collecting mail and filtering mail
with mutt?

I use getmail and dovecot deliver. Getmail is great, fast, and
flexible (and supports the OS X keychain, which I like). Dovecot is a
bit overkill for just a filtering solution since it's a full IMAP
server. However, I find dovecot deliver (which uses the sieve language
for filtering) to be much more readable/writable than procmail. An
added bonus is that my main IMAP account has sieve on the server, so I
can filter mail remotely there using the same syntax as I do with my
other accounts using getmail.

I've looked into other solutions that are more compatible with
offlineimap, like imapfilter (which I believe moves message server side,
but is run from as a remote client) and a few other lesser known local
solutions, like maildirproc, but in the end, sieve was the most straight
forward for my setup.

[Chris Green]

Post by Tim Gray

Post by Chris Green
I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.
What does everyone else here do for collecting mail and filtering mail
with mutt?

I use getmail and dovecot deliver. Getmail is great, fast, and

I had a look at dovecot deliver, I make take another look.

Post by Tim Gray
flexible (and supports the OS X keychain, which I like). Dovecot is
a bit overkill for just a filtering solution since it's a full IMAP
server. However, I find dovecot deliver (which uses the sieve
language for filtering) to be much more readable/writable than
procmail. An added bonus is that my main IMAP account has sieve on

I think anything is more readable than procmail! :-)

Post by Tim Gray
the server, so I can filter mail remotely there using the same
syntax as I do with my other accounts using getmail.
I've looked into other solutions that are more compatible with
offlineimap, like imapfilter (which I believe moves message server
side, but is run from as a remote client) and a few other lesser
known local solutions, like maildirproc, but in the end, sieve was
the most straight forward for my setup.

There seem to be very few currently maintained MDA/filters.

Thanks.

[Tony's unattended mail]

Post by Tim Gray
However, I find dovecot deliver (which uses the sieve language
for filtering) to be much more readable/writable than procmail.

Sieve does not include regular expressions -- I shit you not.

Dovecote needs regular expression capability to be shoe-horned in by
some hokey plugin. Regular expressions are quite fundamental to a
mail filtering language that has an appropriate amount of expressive
power. It's bizarre that sieve is presented as a thought out
successor to procmail complete with an RFC, and yet it excludes
something as essential as regular expressions.

I'm resisting sieve because the C-style makes the code look cluttered,
and it lacks expressive power. The one aspect that may compel a
switch to sieve is the fact that it is MIME-aware. MIME predates
procmail, and it's a shame that procmail has become unmaintained.

OTOH, I might rather have third party tools for MIME than third party
tools for regular expressions.

[Jamie Paul Griffin]

/ Tony's unattended mail wrote on Sat 10.Nov'12 at 22:37:04 +0000 /

Post by Tony's unattended mail

Post by Tim Gray
However, I find dovecot deliver (which uses the sieve language
for filtering) to be much more readable/writable than procmail.

Sieve does not include regular expressions -- I shit you not.
Dovecote needs regular expression capability to be shoe-horned in by
some hokey plugin. Regular expressions are quite fundamental to a
mail filtering language that has an appropriate amount of expressive
power. It's bizarre that sieve is presented as a thought out
successor to procmail complete with an RFC, and yet it excludes
something as essential as regular expressions.
I'm resisting sieve because the C-style makes the code look cluttered,
and it lacks expressive power. The one aspect that may compel a
switch to sieve is the fact that it is MIME-aware. MIME predates
procmail, and it's a shame that procmail has become unmaintained.
OTOH, I might rather have third party tools for MIME than third party
tools for regular expressions.

For what it's worth, I have used both of the above and now I only use procmail. Some experienced folk like Cameron have made very good points about its problems, but for me, it does the job nicely. I've been using it for about 3 years, maybe more and i still am learning about its complexities but it's a good tool in my opinion. I once had a set up that used procmail recipies in conjunction with dovecot deliver and that worked great for me. But I work from home all the time now so have no need for dovecot or any other IMAP server, not yet anyway.

[Chris Green]

Post by Tony's unattended mail

Post by Tim Gray
However, I find dovecot deliver (which uses the sieve language
for filtering) to be much more readable/writable than procmail.

Sieve does not include regular expressions -- I shit you not.
Dovecote needs regular expression capability to be shoe-horned in by
some hokey plugin. Regular expressions are quite fundamental to a
mail filtering language that has an appropriate amount of expressive
power. It's bizarre that sieve is presented as a thought out
successor to procmail complete with an RFC, and yet it excludes
something as essential as regular expressions.
I'm resisting sieve because the C-style makes the code look cluttered,
and it lacks expressive power. The one aspect that may compel a
switch to sieve is the fact that it is MIME-aware. MIME predates
procmail, and it's a shame that procmail has become unmaintained.
OTOH, I might rather have third party tools for MIME than third party
tools for regular expressions.

I simply have a custom script written in Python, hence I have all the RE
and/or other technology I need without much effort.

My filter rules *aren't* written in Python or any language as such, they
are in a format that is as user-friendly (well, me friendly) as possible
and are thus a straight text file with very little special syntax. The
*program* takes care of that, it's what computers are good at, why
should I faff about with funny characters, layout, etc. when the
computer can do it all for me.

My filter file has the following format:-

cheddar Li cheddar-***@lists.halon.org.uk
dabo Li dabo-***@leafe.com dabo-users
dbacruising Li ***@lists.shire.net
dbamain Li ***@lists.shire.net
dbamatters Li ***@lists.shire.net
dbasocial Li ***@lists.shire.net
dia Li dia-***@gnome.org
digitemp Li ***@googlegroups.com
dnsmasq Li dnsmasq-***@lists.thekelleys.org.uk Dnsmasq-discuss

Where the first column is both the mutt alias for the list *and* the
directory (under Li) where the incoming list mail is stored. The second
column is a destination directory (some things get directed to a Ju -
junk - hierarchy). The third column is a string to match in either To:
or Cc:. The fourth (optional) column is a string to remove from the
subject line if found between [].

My Python script to implement all this is only a 100 lines or so of code
in total.

When I subscribe to a new mailing list I just add the appropriate line
to the above filter file and that's it, nothing else to do at all. A
couple of very simple scripts get what's needed from the file to provide
(as I said) a mutt alias for the list and to add the list to provide
what is needed for mutt 'lists', 'subscribe' and 'mailboxes'.

... and as I was saying the filter file itself is in an incredibly
simple format, no XML, no indenting, no block structure, no funny
characters required. About the only 'special' thing is that you can add
comments by having lines starting with a #.

[Cameron Simpson]

On 07Nov2012 14:15, Chris Green <***@isbd.net> wrote:
| I also have a fairly complex mail filtering script I wrote myself in
| Python which is fed mail via .forward.
|
| What's the current "state of the art" way to collect mail and deliver it
| through a filtering system to mutt? If I can do this all in one program
| than so much the better but I'm happy with two programs if that would
| work better. I can stay with my existing filter system but, again, if I
| can consolidate things into one, easier to maintain, chunk then I'd be
| happy.
|
| I *don't* like procmail configuration files, they're one of the reasons
| I wrote my own.
|
| What does everyone else here do for collecting mail and filtering mail
| with mutt?

I collect email with getmail and deliver it to my "spool" folder.

I file mail with "mailfiler", a python program of my own, to monitor the
"spool", "spool-in", "spool-out" and "spool-spam-subj" maildirs. It
understands extremely easy to write and read filter rules, eg:

=spam SPAM-SUBJ subject:/^You have 24 hours left to TRIPLE your deposits

!=me,$EMAIL_IPHONE . to,cc:(ME)
from:(FAMILY|FRIENDS|[...snipped: other mail group names])

mutt Mutt-Users mutt-***@mutt.org

The first rule uses a regexp on the subject header and diverts matching
messages to my "spam" mail folder/septic-tank.

The second rule does proper address parsing of the named headers and (in
the example above) checks addresses against sets of addresses in my
maildb. Almost instant, and very reliable. The "!" means issue an
"alert", for "important" messages.

The third rule also does a proper address parse of to/cc/bcc and if
"mitt-***@mutt.org" is there, saves the message in the "mutt" folder
with the "Mutt-Users" X-Label header added.

I've actually described my setup at some length on the list just
recently, here and here:

http://www.mail-archive.com/mutt-***@mutt.org/msg45217.html
http://www.mail-archive.com/mutt-***@mutt.org/msg45215.html

outlining my setup, why I don't use procmail, what I used to do to beat
procmail into order in the past, etc.

In short, the rules for "spool" divert spam and copy other messages to
"spool-in". The rules for "spool-in" file messages into my main inbox
and multitudinous other folders for mailing lists. "spool-out" is for
cross filing copies of my outbound email.

Aside: "spool-spam-subj" is for recording the subject lines of all messages
filed there as to be considered spam. So I've got a mutt macro to save a
repetitious spam to that folder, and the rules there add a new rule to my
spam filter:-) Mailfiler notices rule updates on the fly.

Because mailfiler runs in the background, polling very regularly (1Hz,
and the machine load for that is insignificant), if I modify my rules
all I need to do to refile a message via the rules is to save it into
"spool-in" again. And off it goes. Of course there is a log file to tail
to watch this stuff happen.

There's a man page for the mailfiler rule syntax here:

https://bitbucket.org/cameron_simpson/css/src/tip/man/mailfiler.5.pod

and for mailfiler itself here:

https://bitbucket.org/cameron_simpson/css/src/tip/man/mailfiler.1.pod

Source code:

https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/app/mailfiler.py

Cheers,
--

2. Rather than auditing a lot of code, correcting a lot of coding mistakes,
like the OpenBSD security team has done, and still do, why not shift from C
to something, just as fast and powerfull as C, but more secure? Again like
Ada. (to completely avoid the possibilities of those errors).

why did you write your email in english? esperanto is simpler and
less error-prone.
- Ted Unangst <***@gmail.com> in misc.openbsd.org

[Mark H. Wood]

fetchmail + maildrop works for me.