How an email works

and above all because it ends up in spam

Since it is a complex subject that lends itself to extensive technical discussions, to make it understood by the "normal" public I try to simplify. Let's pretend we have two friends who write to each other, the sender A is called ARENA and recipient B is called BUGO.

Coso writes an email to Bugo. What happens behind the scenes?
Coso's Server A sends an email to Bugo's Server B.
In simple terms, cosi@vattelapesca.it writes to bugo@nonmissassare.com.

All ok for you. Where is the problem? It's actually not quite like that. It happens a lot. I omit the details of how the mail is "packaged" and sent in packets for several different routes around the internet universe to be reassembled by the Bugo mailserver because it becomes a very long thing and we would go off the rails.

Let's go back to cosi@vattelapesca.it who writes to bugo@nonmissassare.com.

Coso receives no response. Days go by and still get no response. Yet it is an important communication! So he calls Bugo on the phone and Bugo replies that he never received the email!

Generally, Bugo then calls his IT Manager or the hosting manager and then the one who supplies the mailboxes and the mailserver starts ranting: “Here! Give me my money back, thief! The emails don't work!!! I don't get emails from cosi@vattelapesca.it.

You've all had a bit of this problem. The point is that once everything passed, today with the security problems that exist, things have changed, thanks also to you who are great bunglers.

When Coso sends an email to Bugo, the Bugo mailserer checks back, does like the salmon and retraces the reverse path of the email received. And he checks a couple of things:

The first thing it checks is whether the sender is really who they say they are. It might seem trivial to you, but it's not at all! The fact that you are cosi@vattelapesca.it it means nothing at all.

In fact, the Bugo mailserver checks that the IP of the sender's host, i.e. "vattelapesca.com” in this case, matches the IP of the mailserver. Banal? No! It is often the first problem encountered. Internal IT Managers of companies often make the simple mistake of misconfiguring the domain's DNS zone and not correctly assigning the mailserver IP. This is especially true if there are several different domains on one server. Often, in the case of those who sell hosting services by partitioning the server they manage, they are wrong on this point. The big providers deal with this problem in a systemic way and therefore the problem is almost never encountered. But in large/medium companies, with IT managers not quite up to par, who manage their own mailservers internally, this problem becomes frequent.

Basically, the Bugo mailserver files the email as spam because it doesn't understand who the sender actually is. In these cases, even the email is burned completely and is not even recoverable from the spam folder.

Instead, let's make sure that the configuration of the sender's mailserver is correct and that the Bugo mailserver, by doing the reverse, certifies that the IP is correct and therefore the sender is who he says he is.

But the mail still does not arrive at its destination. Again Bugo's call to his IT manager etc… The IT manager checks and verifies that the SPF marking is missing. Ouch! What is the SPF marking?

Sender Policy Framework (SPF) is an email validation system designed to detect email spoofing attempts. The system offers the administrators of an e-mail domain a mechanism for defining the hosts authorized to send messages from that domain, allowing the recipient to check their validity.[ The list of hosts authorized to send e-mails for a given domain is published in the Domain Name System (DNS) for that domain, in the form of specially formatted TXT records. Phishing, and sometimes even spam, uses false sender addresses, so posting and verifying SPF records can be considered in part an anti-spam technique.

Translated, the Bugo mailserver checks that the Coso mailserver has the authorization to send mails from the Coso host, i.e. cosi@vattelapesca.it, thus verifying the validity of the sender and the list of authorized hosts. If the SPF marking is missing, many mailservers, at least the serious ones today, burn the mail, often you don't even find it in the spam box anymore.

Finished? No!

Let's pretend that cosi@vattelapesca.it also has the SPF marking correctly configured.

The email still doesn't arrive. Again Bugo's call to his IT manager and the IT manager who controls. And what does he find?

The DKIM marking is missing Appero! And what is this? What is this devilry?

DomainKeys Identified Mail (DKIM): allows domain managers to add a digital signature via private key to e-mail messages. DKIM therefore adds a further tool to verify the correspondence between the sender and its belonging domain.

Again, to simplify it, with the email, cosi@vattelapesca.it it also sends a random private key linked to a public key that resides in the domain's DNS zone vattelapesca.it and the recipient's mailserver checks that the keys are linked. If they are not, the email ends up in spam.

The perfect control is just Reverse+SPF+DKIM. If an email does not pass this check, the email is burned.

Often some customers ask me to whitelist for example the domain vattelapesca.it but if doing the reverse, the IP is still not correct, the Whitelist is useless. Moreover, it is an incorrect request for a simple reason: you cannot know if the sender in question is "in good faith" because sometimes, not even the sender himself knows that he is. It is as if you had a friend who lives near a nomad camp and you asked him to keep the front door open, "in good faith".

But let's proceed.

At the check, let's pretend the DKIM encryption is okay too. Or maybe it's not even necessary. But the email does not arrive.

Again Bugo's call to his by now exhausted IT Manager. The request is definitive: put cosi@vattelapesca.it whitelisted.

But you can't. You violate security protocols putting the entire company in serious danger. The IT Manager checks and …

The vattelapesca domain is in several RBLs/DNSBLs. Ah! Capers! But what are they?

A DNS-based Blackhole List (also DNSBL, Real-time Blackhole List or RBL) is a means by which it is possible to publish a list of IP addresses, in a special format that can be easily "queried" via the Internet. As the name suggests, the mechanism of operation is based on DNS (Domain Name System). DNSBLs are mainly used for publishing IP addresses related to spammers in some way. Most mail servers can be configured to reject or flag messages sent from hosts on one or more lists.

But at that point Bugo's IT Manager replies to his employer and says: "Guys, if they're registered in a RBL, it's not us who have to understand why and solve the problem, the sender's IT Manager has to think about it ”.

And the email ends up in the trash.

There would be a lot more to add but I want to stop here. However, let's say that all this is the indispensable minimum, that it is not enough to block the bad guys, that other controls are added and also problems inherent to the DKIM protocols which are open and interpretable protocols and therefore there is no uniformity, so much so that there are often problems sending/receiving from/to mailboxes on Libero, Virgilio, Gmail etc …. and they are very difficult problems to solve. Added to this are problems regarding the behavior of individuals who are often contrary to netiquette such as the incorrect use of email headers, the simultaneous sending to several different users unencrypted and so on.

A world opens up where technicians, computer scientists, engineers, mathematicians, physicists confront each other without however having succeeded in principle in finding a solution (and in my opinion they will never find it).

What I can say is that the choice of a good hosting service must go through various factors which are never the price and above all it must respond to security needs which should be the ABC of every company that today is exposed on the internet in the various forms and in various ways.