On a number of occasions we have been informed by a few different clients that emails from particular (but varied) senders are being received multiple times - like 100's!

Looking at the logs our end, we don't see a problem - to us we just see the sending server make a connection, send an email which we forward onto the destination server. I've now been able to get hold of some sort of log file from one of the senders which are experiencing this 'duplication' of emails. From what I can gather, there system connects and sends SF the email, SF does whatever checks it does, but before SF accepts/confirms delivery, the senders end hits a time out, drops the connection and re-queues the email for another delivery attempt later. Despite the senders end timing out before our server has confirmed receipt or acceptance, our server still forwards the email - presumably because our server knows its received the email in its entirety having receiving the <CR>.<CR> instruction? So assuming the email passed spam checks, the email is sent on. 

The above course of events cause a loop, whereby our server receives and delivers the email, the senders continually times out before our server has confirmed acceptance, causing the sending server to re-queue, and re-deliver the email - and then the loop begins again.

If this is what's happening, I think either one of the following is wrong:

1) The senders end's time out is too short?

2) SF should not deliver an email if the connection is dropped/times out - just like it wouldn't if the connection was dropped part way through the DATA stage.

If number 2 is changed, so that SF does not deliver emails from which the connection is dropped even after a <CR>.<CR> is received, this will not prevent the senders end from timing out, so instead of receiving duplicates, our server would not forward any, meaning the email would never be received, and eventually a bounce back would be generated. What are your thoughts on this? And what do people thing the fix might be? 

The version of SpamFilter you're running is very old :-) That was a known issue and was solved a long time ago starting with SpamFilter v3.5.3.657. The partial release notes for that build are as follows:

My current storedprocedure utilizes MS SQL's temporary hold of records being added and based on the event, such as INSERT, it then triggers the storedprocedure and grabs the e-mail address, subtracts the domain and for that day it makes a count.

It is really easy and fast, and also fast to generate a bar-chart from that.

So for example a user can see the daily fluctuation of quarantined e-mails for their domain around 50,000 emails daily, and compare it to a graph next to that to their personal mailbox's quarantine with is around 50 for that particular e-mail account daily.

So if possible I'd like to avoid using raw logs. Server is busy enough already blocking junk e-mails based on the many filters.

I've written an enhanced web GUI(still working on it), using SQL DB from where I pickup the quarantined e-mail data.

As each new e-mail is added by the filter app, I trigger a counter that adds a new e-mail address to a separate table and a counter per day to count how many e-mails come in per day for that e-mail address.

Also do same for a domain name of that e-mail address to get totals.

SQL does all the work using a stored procedure.

But, I can only capture what the filter quarantines.

I'd love to see a more comprehensive counter per e-mail account and domain name.

- incoming e-mail count

- blocked e-mail count

- forwarded on good e-mail count

- quarantined e-mail count (this is all I have)

I build a chart based on the daily count so the user can see how traffic fluctuated over time. It is amazing how an account with average 30-50 quarantined spam per day can all of a sudden for a week drop to under 10 spams, and in another month all of a sudden jump for just one day into the hundreds.

But I'm only seeing quarantined.

So would it be possible, and would others also benefit from this? 

Basically the filter would do a one-way communication to the quarantine DB or a local file or cache and later written to file.

Per email address, per day one record in a counter table for each of the counters.

Perhaps call a storedprocedure and just let that do the updating and counting freeing up the Filter's process of it.

It would be great to know the total of good vs. bad e-mails per e-mail account.

Please, others add your comment/support for this feature if you'd like to see it!)

If the emails are in the quarantine, SpamFilter did indeed reject them, either thanks to your keyword or due to another filter.

Please note that, if you wish, you can configure most filter so that emails that are blocked by them are permanently rejected and not stored in the quarantine.

subject: === sale 70%
subject: =*= today 70%
subject: mr.
70%
80%
Pfizer
pfizer
viagra

Now, example of email in Quarantine:

Subject: =*= Today 70% OFF For 3brpena =*=

 

Question: Must this mail be rejected?

Because it is stored in Quarantina?

 

Thanks you

Please note that sometimes spammers will encode the subject lines in the email's source so that the rendered text will be different than the encoded one. In these cases you will need to view the actual email's source to see what the real subject header was.

If that is not the case, please contact us via email at support at logsat.com so we can ask you to provide us with the SpamFilter's activity logfile for a day this happened, the contents of your SpamFilter\Domains directory, and some of the original emails in question (please note that we'll need the original, unmodified email source - some email clients, notably Microsoft Outlook, completely change the email format making it impossible without registry tweaks to have access to the actual source).