Protection of the mail system from spam and viruses. Anti-Spam Methods Using the Anti-Spam Email Code

The following technologies are used to protect mail servers:

There are two main methods for protecting spam: protecting against incoming spam when the server receives mail, and separating spam from the rest of the mail after it is received.

Blacklists.The blacklists contain IP addresses from which spam is sent.

Greylisting or greylisting.The way greylisting works is based on spamming tactics. As a rule, spam is sent in a very short time in large quantities from some server. The job of the greylisting is to deliberately delay receiving emails for a while. In this case, the address and time of transfer is entered into the gray list database. If a remote computer is real mail server, then he must keep the letter in the queue and repeat the forwarding within five days. Spam bots, as a rule, do not save letters in the queue, so after a short time they stop trying to forward the letter. When re-sending a letter from the same address, if the required amount of time has passed since the first attempt, the letter is accepted and the address is entered into the local white list for a sufficiently long period.

DNSBL (DNS blacklist) - lists of hosts stored using the DNS system. The mail server contacts the DNSBL and checks for the presence of the IP address from which it receives the message. If the address is in this list, then it is not accepted by the server, and the corresponding message is sent to the sender

Message limit... Setting a limit on the number of messages.

Program SpamAssasin (SA) allows you to analyze the content of an already delivered message. SpamAssassin comes with a large set of rules that determine which emails are spam and which are not. Most of the rules are based on regular expressions that match the body or header of the message, but SpamAssassin uses other techniques as well. The SpamAssassin documentation calls these rules "tests".

Each test has some "cost". If the message passes the test, this "cost" is added to the total score. Cost can be positive or negative, positive values \u200b\u200bare called "spam", negative "ham". The message goes through all the tests, the total score is calculated. The higher the score, the more likely the message is spam.

SpamAssassin has a configurable threshold, above which a message will be classified as spam. Typically, the threshold is such that an email must match several criteria; triggering only one test is not enough to exceed the threshold.

The following technologies are used to protect sites from spam:

1. Captcha picture... Those. the user is shown an arbitrary text that the user must enter to perform any action.

2. Text captchas - the subscriber must enter the answer to the proposed question to confirm his actions.

3. Interactive captcha - a little common, but very useful form of protection. For example, to confirm the actions, the user will be asked to solve an easy jigsaw puzzle - for example, to assemble a picture from three or four parts.

Introduction to the problem

We all know what spam is because we have either come across it or read about it. We all know how spammers collect email addresses. It is also no secret that spam cannot be completely defeated. The problem is how to maximally protect users who leave their contact details on your site with minimal effort.

Previously tested methods of protection

The biggest threat to mailboxes is represented by programs that download sites and take mail addresses from the text of pages. They download either only your site, or roam like search engines all over the web. If your site is small, the protection of this autocorrect text is quite enough:

] + href \u003d) ([""]?) mailto: (+) () @ ".
"() (+. (2,4)) 2 ([\u003e]) ~ i", "1" mailto: [email protected]"
onMouseover \u003d "this.href \u003d" mai "+" lto: 3 "+" 4 "+"% 40 "+" 5 "+" 6 ";" 7 ", $ text);?\u003e

Unfortunately, it won't work if you have a large site. Let's say spectator.ru, the author of which was one of the first to use this method. If I were a spammer, I would have climbed into personal settings, ticked "do not show ears", 1000 reviews per page, and caught cookies by Proxomitron. Then, with a rocking chair or a php script, I would download the pages with comments (substituting cookies with settings) and using a regular expression, I would catch the addresses. I would get a small base for advertising mailing.

There were also a couple of protection methods in which the mailto: link is automatically replaced with a cookie, but the effect remained the same - when you clicked on it, the system client would create a letter to the desired address. Both of them did not stand up to criticism.

Meet the iron fist

Obviously, it is difficult to think of another method of protection besides the already tried one - providing a form on the site to send a message. Let's start designing it. The advantages of this method are obvious: no one will be able to get the addresses for their spam database from your site. It will not work to send messages by hiding their address, as spammers do - the web server will fix its IP address. Lists of public anonymous proxy servers are regularly updated, and it is easy to block access from them.

Form sender

Let's start with him, because this is the hardest part.

When installing a form sender on a website, it is important to protect it from hooligan attacks, which can be no easier than spam. Therefore, we will have to make great efforts in this direction.

First, we will protect ourselves from silly double clicks and sending many identical requests. The idea is this: the message will not be sent if the user has not opened the page with the form before, and by opening the page with the form, you can send the message only once. This can be done using PHP's built-in sessions. When opening a page with a form, we will start a session in which we will save a variable, say $ flag. We'll display the session identifier as a hidden element at the very end of the form. The user enters a message and submits the form. Upon receiving the form, the script starts a session and checks the existence and value of the $ flag variable. If the variable does not exist, then this is a second click, the letter is not sent and an error message is displayed. If there is a variable, and the form data suits us (the required fields are filled in), the script sends a letter and deletes the session.

Secondly, we will protect ourselves from smart bullies by writing down message logs. If the user submits a correctly filled form, the script will look in the logs and check what is there. So, you need to ban

* send messages to the same address more often than a certain period
* send the same text to different addresses
* and just use the form sender too often - say, no more than 10 messages per day per user

We print the session ID at the very end of the form so that the hacker needs to download the entire form and parse it, which is more difficult than just sending HTTP requests. Naturally, the sender will issue messages about errors in writing a message, a request to indicate a return address, etc.

The resulting sender code was too large to be included in the text. It is archived on the site. It seems that the script is running and sending messages.

Replacing addresses in text

Now the form sender is ready, and you need to replace all emails with links to it. Of course, you shouldn't do this manually. For myself, I wrote a script that automatically replaces addresses with links to the sender.

... Disadvantages: more time for placing links (compensated by the catalog of links), the user, hovering the cursor over the link, does not see which address it will go to. (Dmitry Smirnov, "Ideal author's project, hypertextuality")

All the mentioned disadvantages can be easily eliminated if you use a code similar to the one I will now describe and show.

There is nothing complicated here, if these are links, then "more time for placement" is not required. On my site, I use an engine script that is called by all pages, so it's not a problem to add code to it or call from it a code that replaces addresses. Postal addresses are both written and written directly in the text of the pages, but before being displayed to the user, they are replaced with the desired text. Compiling a database of links or postal addresses is not a problem.

So what does an address substitute do. He looks for links "mailto:" in the text, selects addresses from them, sends a request to the database to count (count (*)) how many addresses from those on the page are in a special table. If there are new addresses on the page, then their number will be greater than the query result. In this case, a query is made, in which the values \u200b\u200bof addresses are selected, and those already existing in the table are excluded from the list. The remaining list is sent to the table with an INSERT query.

As for the ID addresses, in my opinion, it is better to use something that the site visitor could not pick up. Can you imagine the link /email.php?id\u003d10 leads to the sender? What a temptation to put in there 11, 12, etc. and try sending them all a message. Therefore, I decided to use the md5 hash from the addresses as identifiers. Hardly anyone will undertake to select a hash. In the case of a directory of links, you can get by with ID, but then you have to select all values \u200b\u200bfrom the database, and to replace addresses with their hashes, everything is much easier.

A command of the form

] + href \u003d) "." ([""]?) mailto :( [email protected]+ ".". (2,4)) 2 (. *?\u003e) ~ Ie "," "12" /email.php?email\u003d ". Urlencode (md5 (" 3 "))." "4" " , $ text);?\u003e

... which replaces addresses with their hashes. The rest of the addresses in the text, I did not dare to replace with links, but made a simple replacement for addresses like vasya_at_pupkin_dot_ru. The autocorrect code is also in the archive.

Outcome

Hiding email addresses from visitors is pretty easy. The autocorrect mechanism does not require additional effort, and you can write further pages of the site as if nothing happened. Difficulties arise when protecting the form sender from web bullies. This protection requires a lot of effort and complex code, so I have not yet used the written code on the site. You can download the archive with a substitute for addresses and a form sender, but I beg you: do not put it on your site in the form in which you downloaded it, I myself do not know how reliably it works.

What methods are there to combat spam?

There are two main methods of protecting a mail server from spam: protecting against incoming spam when it is received by the mail server and "separating spam" from the rest of the mail after it is received by the mail server.

Among the first method, the most popular are such methods as using the DNS Black List (DNSBL), Greylisting and various delays in sending mail; use of various technical means, such as checking the existence of a user on the sending side (callback), checking the "correctness" of the sending server by such methods as the presence of a record in the reverse DNS zone, the legality of the name when establishing an SMTP session (helo), checking the SPF record (for this to work in the DNS records for the host, the corresponding entry for the legitimate sender servers is used).

Among the methods for analyzing the content of the letter, the most popular methods are checking against various algorithms, such as searching for special keywords of an advertising nature or based on Bayes' theorem. The algorithm based on the Bayesian theorem contains elements of the theory of probability, is initially trained by the user for letters that, in his opinion, are spam, and then separates messages containing spam by their characteristic features.

So, let's take a closer look at these filtering methods. email.

Blacklists or DNSBL (DNS Black Lists)

The black lists contain addresses from which spam is sent. Lists such as “open relays” and “open proxies” are widely used, and various lists of dynamic addresses that are allocated by providers to end users. Because of their ease of implementation, these blacklists are used through DNS.

Greylisting or greylisting

The principle of greylisting is based on the tactics of sending spam. As a rule, spam is sent in a very short time in large quantities from some server. The job of the greylisting is to deliberately delay receiving emails for a while. In this case, the address and time of transfer is entered into the gray list database. If the remote computer is a real mail server, then it must keep the message in the queue and repeat the transfer within five days. Spambots, as a rule, do not save letters in the queue, therefore, after a short time, they stop trying to forward the letter. It has been experimentally established that the average time for sending spam is just over an hour. When re-sending a letter from the same address, if the required amount of time has passed since the first attempt, the letter is accepted and the address is entered into the local white list for a sufficiently long period.

Efficiency analysis

The first two methods allow you to filter out about 90% of spam even at the stage of delivery to mailbox... Mail that has already been delivered can be marked up by means of analyzing the message content, for example, using the SpamAssassin program. This product allows, on the basis of special algorithms, to add corresponding lines to the message headers, and the user, based on the mail filters in the mail client, can filter mail into the necessary folders of the mail program.

Conclusion

Of course, there are other ways to protect against spam, the most effective, unfortunately, at the moment are preventive measures, such as not leaving your real email inbox on sites, forums and message boards, using temporary addresses for such needs, which later you can delete, if it is necessary to publish a mailbox on the site, instead of text, use a graphic image and similar measures.

You can connect and configure GreyListing through the ISPmanager panel in the "Features" section

You can learn more about configuring anti-spam methods through the control panel here DNSBL and here Greylisting.

- € 55-250 million annually. 60% world mail traffic.
50-75% from all Russian mail traffic. Modern antispam tools filter 85-98% of spam. The global market for antispam filters and services in 2004 was approximately $ 500 million (IDC estimates).
Most antivirus vendors have included anti-spam components in their products. During the year, there were several purchases of anti-spam software vendors by anti-virus companies (notably the $ 340 million purchase of BrightMail by Symantec). In Russia, antispam filters have been installed by most of the holders of public mail services and most of the providers, which made it possible to remove the urgency of the problem of spam for their clients. The undoubted leader in Russia in terms of sales and the number of protected mailboxes is the Spamtest technology.
1. PREVENTION The # 1 anti-spam tool is to protect your email address. No spammers will know your address - no spam. Highlight your address on the network, you will have to throw it away and start a new one, it will only be a matter of time. And, as a result, tell all your friends and partners a new address again, and you may lose a number of contacts. To prevent this from happening Get two email addresses. One address for long-term contacts (do not shine it on the network).
Another address for making contacts, using the network (chats, message boards, etc.).
Then there should be no spam on the first address, because it is not known on the network.
When spam goes to the second address, just throw it away and start a new one.
2. CHOOSING A NAME People tend to get the most concise address. Let's say sergey@ mail.ru is cool and what a pity that all simple addresses are already taken. Rest assured that on [email protected] spam is pouring in without stopping. It's cool to have a laconic name for the site, but you still have to tell everyone the email address personally, even if it is from numbers or an original, not a hackneyed word. By the way, for this purpose, the leading mail gmail.com registers names of at least 6 characters. All short names have long been included in spam lists.
3. HTML SPECIAL CHARACTERS The simplest and most commonly used method of protecting against spiders is to encode the email address using special HTML characters. Instead of a dog - @ ... But today this method is hopelessly outdated.
Robots can easily find such addresses.
4. JAVASCRIPT On the Anti-Spam Code Generator page, you can generate your own script. Since these scripts for hiding the address are crafted, they are very motley and there are no programs that would be able to fetch email from JavaScript. It is the most secure address on the network today.
5. ANTISPAMMERS But, what if you are lit up, or you are so famous that it is impossible for you not to get noticed, then you cannot do without an anti-spammer. There are many antispam programs that you can download online.
What I do not advise.
I came to the conclusion that all these antispams are small and weak, and a sensible antispam cannot be handled by a person, only a solid company, such as Gmail.com, can do that. Their spam remains on the server, you can always go in and correct it. So my strong advice: get yourself a mail on Google.
I have not seen a better spam filter, all spam remains on the server, which, if desired, can always be viewed and corrected. Antispammers do not completely solve the problem, but make life easier in the problem.
6. POCKET PC AND WAP Spam has reached this level, but today there are fairly reliable means of protection. Therefore, the development of this issue is not relevant.

Hello dear readers of the blog site. Surely you have already encountered spam, even if you did not know exactly what this mess is called. And this is not even necessary. Spam with enviable regularity is poured into an ordinary mailbox - these are the very countless advertising brochures that they slip into you in the hope that you will order or buy something.

So, spam is just is the annoying one, climbing from all sides advertising (but not only advertising - sometimes worse). You did not order it, you do not need it, but it rushes and rushes from all the cracks. In this heap, the information you need can easily get lost and you have to spend a lot of time filtering it out.

In our computer age, the main source of spam. And besides advertising spam mailings can be dangerous both for your wallet (phishing, social engineering, wiring) and for your computer (viruses, worms, trojans).

What does the word Spam mean, what it is and how you can deal with it, you will learn by looking at this small note. Hope it will be interesting (well, definitely useful - I promise you).

Spam - what is it

WHAT DOES SPAM MEAN IN GERMAN?

Itself word Spam comes from the name of canned meat, which was fiercely advertised after the end of the Second World War (obviously, it was necessary to urgently sell the soldiers' meat rations).

The advertising was so aggressive, all-embracing and ubiquitous that this word (and the associated "sediment") was remembered, but already about the intrusive advertising that appeared along with it in conferences (then still in the feed, if anyone remembers).

The word stuck, especially since the obtrusive advertising did not become less, but rather the opposite. When e-mail gained popularity, unauthorized mass advertising and malicious mailing of letters became commonplace. For spammers, such mailings were beneficial, because the necessary information was communicated to a large number of people.

But e-mail is not limited to... Spam in a personal social networks, forums, messengers instant messaging, on message boards, in comments to blogs, is open for editing and adding text by everyone. They also send spam to your phone, for example, by advertising calls or by sending advertising SMS messages.

Where can you find him on the Internet

Email Is just a klondike for spammers. Bulk mailings can sell anything, you can cheat and rob, you can infect computers and send out worms. Databases for mass mailings are collected independently (with the help of programs), or purchased from those who do it professionally.
Forums, comments blogs, wikis and message boards - here, in fact, everyone is allowed to leave messages and it is difficult for spammers to resist not to sleep. This is not always advertising - often in this way, webmasters try to get a free link to their site so that it stands higher in the search results of Yandex or Google for various queries. This brings them traffic and money.
Social networks and dating sites - spam is very common among incoming personal messages. It is also available in the comments to the messages.
Messengers (type) are also subject to this scourge.
SMS- messages from people unknown to you of an advertising nature. Probably everyone is familiar.
Search spam - a rather specific thing, but familiar to everyone. You had such a thing that you enter a request into Yandex (Google), and in the responses there are entirely sites with some kind of bilibery that have nothing to do with the case. These are the so-called doorways (sites with automatically generated useless texts). They spam the results of search engines, but they make money on visits to them by visitors (in different ways).

Those. all letters in your mailbox (or messages on the forum, blog comments, messages in a personal message) of an advertising or other nature from a sender unknown to you - this is spam... True, the mailings to which you subscribed can be annoying, but you can still unsubscribe from them (refuse to receive them further).

By itself, spam is annoying and distracting, because it litters where it appears. Most often this is your inbox, and it can be very difficult to separate the wheat from the chaff when there are too many unwanted messages. But spam can also pose a real threat to both you and your computer. Let's figure it out with this.

Understanding the types of spam (harmless and dangerous)

Now and further, I will mainly talk about spam messages received by e-mail, for it is this channel that is considered particularly susceptible to this phenomenon. Everything else is not so launched and is not so actively used by spammers, and this does not apply to all network users. But the problem of their protection from unwanted messages is acute.

Types of harmless spam

Advertising of legal goods and services - in this case, the business owner simply chose spam mailings as one of the channels to attract customers and buyers, because it is not expensive, quickly and brings results. Naturally, he does not really think about the moral (or rather immoral) side of this case.
Prohibited advertising - there are goods and services that are prohibited by law to advertise, and for them spam mailings can be the main channel for attracting customers. By the way, spammers also advertise their services using mass mailings, because this activity is prohibited by law. This also includes advertising defaming competitors, because it is also prohibited by law (praise yourself, but do not scold others).
Impact on public opinion - very often spam becomes a good choice for those who are trying to manipulate public opinion in the right direction. It can be politics as well. Letters in general can be sent out ostensibly from some person in order to compromise him, change his opinion about him or use his authority for profit. But personally, this does not pose any danger to you.
Letters that ask for forwarding to others - these can be varieties of the so-called "letters of happiness" (send it to 10 friends and you will be happy), or asking to send information to friends for some other reason. Often such letters are used by spammers to collect or replenish the existing email database for subsequent mass mailings.

High-risk spam - what it can be

If regular (harmless) spam can only slightly affect your peace of mind and mental health, then the examples below can be very dangerous for your wallet or computer. And this is not a joke.

I myself (the one who teaches and teaches everyone) a couple of years ago "bought" this type of letter (phishing) and parted with several thousand rubles (read ""). I just wound myself up and did everything “on the machine” just to fall behind, but when I realized it, it was too late.

- highly effective method deceiving not only gullible, but simply busy or inattentive people (there is a hook for anyone). They send you a letter supposedly from your bank, e-money service or somewhere else. In this letter, you are sure to be dumbfounded (unsettled) and asked to log in to the site to solve the problem. You log in, but the site will be fake (although it looks like a real one) and the data you provided is immediately used to steal all your money.
Nigerian letters - you will find out that you can get a large amount of money (different pretexts - from unexpected inheritance to helping a prince in exile). You don't believe for a long time, but they convince you. When you believe it, you will be asked to transfer some money for "related expenses." You translate and nobody else will bother you.
Viruses, worms, trojans - the letter may contain an attached file with malware (or a link leading to a site with a virus). She can immediately cause problems with the operation of the computer, or she will sit quietly in a corner and will carefully write down all the passwords, logins and other useful things in the household you enter. Among other things, worms are also able to send themselves to your friends at the addresses found in the email contacts (as they will then be ...).

Spam protection

Where do spammers get databases with email addresses from?

Spammers (those who produce mass mailings) collect email addresses from all available sources. These can be forums, guest books, chats, social networks and other sites where postal addresses may be publicly available.
Hackers manage to get to some databases of addresses stored on sites.
The collection of emails is carried out in most cases by a programmatic method (with the help of search bots - harvesters) and this does not require much effort (only time is not very much). Moreover, similar email spam databases you can not collect, but buy them from those who specialize in this business ().
Billions of postal addresses are registered all over the world, and therefore you can simply try to generate emails using special programs using the appropriate dictionaries. Most likely, many of them will actually exist. Read about how spammers check the validity of addresses below.
There are special worms (viruses) that can send themselves to the database of addresses found on the victim's computer. The database collected in this way will already be cleared of non-working mailboxes.

How do spammers clean their databases from inactive email addresses?

Those who collect the base of addresses, in fact, do not care who owns this or that address - they send letters to everyone in a row, because there will still be someone who will react to them (as they say, they beat the squares).

But nevertheless, in order to optimize costs and increase the return on mailings, it is beneficial for them to clear the databases of non-existent addresses. How do they do it? Let's see.

The simplest thing is to place a picture in the letter (it may not even be visible - one pixel in size), which will be downloaded from the site belonging to the spammer when the user opens the letter. If the picture is loaded, it means that the letter was opened and the Email is valid.
Many e-mail clients (programs for working with e-mail) automatically send a message about reading the letter, which again plays into the hands of spammers.
The letter may contain a link calling to go somewhere, promising mountains of gold. We switched - consider that your email will now be marked as good. The most unpleasant thing is that such a link can be disguised as an unsubscribe button, which in fact will lead to the opposite effect.

How to reduce the likelihood of your email reaching the spam base?

In general, as soon as your mailbox is "confirmed", spammers won't just get off of you. It is important to understand that no one is immune from spam. But you can significantly reduce the likelihood of getting into such a spam databaseif you take specific precautions:

You can, of course, not publish your Email anywhere at all and not tell anyone about it. But in most cases this is difficult to do, so I advise except for the main box have one or two secondary ones that you will use to register on forums, etc. Often they can be useful and that can be obtained without registration at all.
Do not follow links in spam emails (even if there is an "Unsubscribe" button there is a trap) and, if possible, disable automatic download images in your email client program. There is a chance that your email will be counted as inactive and bulk mailings will not be received by a hundred per day.
If you have not registered a mailbox yet or are planning to start a new one (for example, due to the complete clogging of the old with spam), then proceed not from the convenience and ease of remembering it, but, on the contrary, make it more authentic and more complicated. Friends, you will still send it to in electronic format, but brute-force spam programs are unlikely to guess it.

What to do if spam doesn't let you breathe?

These were all preventive measures to combat spam (or rather to reduce its amount). But there is a possibility of effective struggle even in an already completely neglected situation. In this case, it becomes extremely important,.

The fact is that in such large services as or, there are powerful anti-spam filters.

They put all suspicious emails in a separate Spam folder, thereby freeing your Inbox from junk. Yes, there is no perfect spam cutter, and as far as possible, the contents of the "Spam" folder before cleaning it will be better to look diagonally for legitimate correspondence. But it's still much easier than digging through all this garbage all the time.

If you have a mailbox on another service, where the spam cutter is useless (for example, as in), then you should not despair. Get yourself an Email on Gmail or Yandex, and then set up mail forwarding to it from your old mailbox. Moreover, these settings can be made as in the old box (i.e., set up forwarding - shown in the screenshot):

In the same way, in the new mailbox, you can configure the collection of mail from the Emails you already have (the screenshot shows the settings for collecting mail in Gmail):

The same can be said for programs mail clients ... Most of them also have a built-in spam cutter.

But in this case, do not forget that on postal service will have its own spam folder, which will also need to periodically look through (or look for messages there that should come, but did not reach - for example, confirmation of registration anywhere is often cut by a spam filter), because mail from it will not be sent by default to your computer (although this can be corrected in the settings of the service or client program).

Good luck to you! See you soon on the pages of the blog site