MagicSpam: Auto Whitelisting and Log Filter

[fusion_builder_container hundred_percent = & #8220; no & #8220; equal_height_columns = #8220 NO #8220; menu_anchor = #8220; & #8220; hide_on_mobile = #8220; small-visibility, medium-visibility, large-visibility & #8220; class = & #8220; & #8220; id = & #8220; & #8220; background_color = #8220; & #8220; Background_Image = #8220; & #8220; background_position = & #8220; center center & #8220; background_repeat = #8220; no-repeat & #8220; fade = #8220 NO #8220; background_parallax = #8220; none & #8220; parallax_speed = #8220; 0.3 & #8243; video_mp4 = #8220; & #8220; video_webm = #8220; & #8220; video_ogv = #8220; & #8220; video_url = #8220; & #8220; video_aspect_ratio = #8220; 16: 9 & #8243; video_loop = #8220; yes & #8220; video_mute = #8220; yes & #8220; overlay_color = #8220; & #8220; video_preview_image = #8220; & #8220; border_size = #8220; & #8220; border_color = #8220; & #8220; border_style = #8220; solid & #8220; padding_top = #8220; & #8220; padding_bottom = #8220; & #8220; padding_left = #8220; & #8220; padding_right = & #8220; & #8220;] [fusion_builder_row] [fusion_builder_column type = & #8220; 1_1 & #8243; layout = & #8220; 1_1 & #8243; background_position = & #8220; left top & #8220; background_color = #8220; & #8220; border_size = #8220; & #8220; border_color = #8220; & #8220; border_style = #8220; solid & #8220; border_position = #8220; all & #8220; spacing = #8220; yes & #8220; Background_Image = #8220; & #8220; background_repeat = #8220; no-repeat & #8220; padding_top = #8220; & #8220; padding_right = #8220; & #8220; padding_bottom = #8220; & #8220; padding_left = #8220; & #8220; margin_top = #8220; 0px & #8220; margin_bottom = #8220; 0px & #8220; class = & #8220; & #8220; id = & #8220; & #8220; animation_type = #8220; & #8220; animation_speed = #8220; 0.3 & #8243; animation_direction = #8220; left & #8220; hide_on_mobile = #8220; small-visibility, medium-visibility, large-visibility & #8220; center_content = #8220 NO #8220; last = & #8220 NO #8220; min_height = #8220; & #8220; hover_type = #8220; none & #8220; link = & #8220; & #8220;] [fusion_text] Enclosed we found a very interesting and useful source with a pretty good description of how to make MagicSpam a bit more flexible. We changed the source code in two places to the original source, because there were two errors in the scripts and on the other hand the e-mail notification did not work. We use these scripts ourselves, all scripts are up-to-date and tested for the current posting date.

Test system: Ubuntu 16.04 + Plesk 17.5.3 Update #39
As script path we have with us: / Var / www / vhost / bin / created.

Thanks to the publisher of the origin source: https://marketmix.com/magicspam-auto-whitelisting-and-log-filter/

description

Get reliable, efficient and complete 2-step protection against spam and junk, as well as viruses, malware and spyware. Block potential threats, including images, videos, or attachments on the server side, before they arrive in your inbox, and cover all other attachments with comprehensive antivirus protection.

Suitable for:

  • Web professionals, agencies, and SMBs who need to protect themselves against security holes to prevent disruption or loss of business.
  • MSPs, CSPs and hosting providers who need to protect the inboxes and service integrity of their customers.

magic spam

What is a car whitelist?

In the whitelist from Magic spam are collected mail addresses that without Spam check will be let through. If the mail servers of important contacts violate various rules, therefore classified and blocked as spam senders, is the whitelist the tool of choice.
Manually putting many addresses on the whitelist is very tedious and time consuming. Auto-Whitelisting takes away this work: At regular intervals, the log files of their own mail server are scanned and all recipients of outgoing mails on the MagicSpam Whitelist set. This ensures that every recipient of your mails can also answer without landing in the spam filter.

Postfix: Differentiate autoresponder emails from other emails

In order for our solution to work, it is necessary to distinguish the autoresponder mails from all other mails in the Postfix log files. This is done with the following trick about the subject line of the autoresponder e-mails:

In Plesk, the subject of each autoresponder with the addition "(Auto-Reply)" Mistake:

Since Postfix by default does not write any subject lines in the log files, Postfix's configuration must be adjusted as follows:

Under /etc/postfix/main.cf the following line is added:

header_checks = regexp: / etc / postfix / header_checks

After that, create a new file called / Etc / postfix / header_checks with the following content:

/ ^ subject: / INFO

Now we restart Postfix via the following command:

sudo /etc/init.d/postfix restart sudo /etc/init.d/postfix restart

The new car whitelisting script for MagicSpam

The Auto whitelisting script looks like this:

#! / Bin / bash # # build-magic-whitelist.sh 1.02 # # Implements MagicSpam auto whitelisting by # Recipient recipient addresses from the Postfix log. # # Ignoring AUTORESPONDER mails requires the following Postfix modifications: # # EDIT: /etc/main.cf: # ADD: header_checks = regexp: / etc / postfix / header_checks # # CREATE: / etc / postfix / header_checks # ADD: / ^ subject: / INFO # # Restart Postfix to apply above mods. # # (c) 2016 Harald Schneider # # Config.start # # Clear this variable, if you use a unmodified Postfix AUTORESPONDER = "(Auto-Reply)" # # Config.end # Filter Postfix log # if [$AUTORESPONDER = "" ]; then cat / var / log / maillog | grep -v "relay = local" | grep "relay =" | grep "status = sent" | perl -ne 'print lc "$1 \ n" if / to=<(.* ?)> / '> /var/log/maillog-filtered.txt else cat / var / log / maillog | perl -MMIME :: QuotedPrint = decode_qp -e' print decode_qp join "", <> '| grep "info: header Subject: "| grep -v" Subject:. * $AUTORESPONDER "| perl -ne 'print lc" $1 \ n "if / to=<(.*?)>/'> / var / log / maillog-filtered. txt fi # Append existing Whitelist # cat / etc / magicspam / from_whitelist | awk '{print tolower ($0)}' >> /var/log/maillog-filtered.txt # Filter duplicates and write back to Whitelist # perl -ne ' print unless $seen {$ _} ++ '/var/log/maillog-filtered.txt> / etc / magicspam / from_whitelist rm /var/log/maillog-filtered.txt exit 0

Finally, a cron should be set up in the Plesk, which runs via root, the script once an hour.
With us this is & #8222;/var/www/vhost/bin/build-magic-whitelist.sh

Filter MagicSpam Log

Auto-whitelisting requires that a contact first receives an email before being sent "Unlocked" becomes. Therefore, it is important to keep an eye on the log files, especially in the phase immediately after installing MagicSpam. Here you can see exactly if a contact is wrongly classified as spam.

The following script reduces the daily logfile of MagicSpam to 3 columns, which can be checked very fast manually: Sender (FROM), Receiver (TO) and "RULE"ie the rule used to classify it as spam. The latter is helpful in deciding whether various settings in MagicSpam are switched off better.

The resulting and duplicate-corrected list will be automatically sent by email at the end together with the original log:

The magic spam log filter script in detail

#! / Bin / bash # # filter-magic-spamlist.sh 1.02 # # Filters spam sender's addresses from the # MagicSpam logs. # # (c) 2016 Harald Schneider # # Setup.start # LOG = / var / log / magicspam / mslog # Path of MagicSpam log file FROM=magicspam@yourdomain.com # Report mail sender TO=notify@yourdomain.com # Report mail recipient # # Setup.end # filter MagicSpam log # echo "Processing MagicSpam Log ..." cat $LOG | grep ": SPAM" | perl -ne 'print "FROM \ tTO \ tRULE \ n"; print "$2 \ t$3 \ t$1 \ n" if /: SPAM (. *?):. *? from = <(. *?)>. *? rcpt = <(.? *)> /; ' > /var/log/magicspam-tmp.txt # Filter duplicates and send as attachment # perl -ne 'print unless $see {$ _} ++' /var/log/magicspam-tmp.txt> / var / log / magicspam -senders.csv # Prepare original report # cp $LOG $LOG.csv echo "Sending reports ..." #send mail -f $FROM -t $TO -u "MagicSpam Report" -m "Reports attached. \ n \ n" -a $LOG. csv -a /var/log/magicspam-senders.csv a- Causes error mailx -r $FROM -s "MagicSpam Report" -a $LOG.csv $TO </var/log/magicspam-senders.csv # Cleanup # rm $LOG .csv rm /var/log/magicspam-senders.csv rm /var/log/magicspam-tmp.txt exit 0


Magic spam
creates an automatic task in Plesk (or other systems) that rotates the log files at 0:00. This means that / var / log / mslog becomes mslog.1, mslog.1 becomes mslog.2 etc. The last mslog.8 file is deleted. Then a new, empty mslog file is created in which again the current entries of the new day land.
There we are / Var / log / mslog process, it makes sense the script just before the Log rotation to start by Task Scheduler, eg daily at 23:55. So you have the evaluation in the mailbox the next morning.
With us this is & #8222;/var/www/vhost/bin/filter-magic-spamlist.sh

[/ Fusion_text] [/ fusion_builder_column] [/ fusion_builder_row] [/ fusion_builder_container]

en_USEnglish
de_DEDeutsch en_USEnglish
Scroll to Top