Urgently upgrade Exim to 4.92 - there is an active infection
Colleagues who use Exim versions 4.87 ... 4.91 on their mail servers - urgently upgrade to version 4.92, having previously stopped Exim itself to avoid hacking through CVE-2019-10149.
Several million servers around the world are potentially vulnerable, the vulnerability is rated as critical (CVSS 3.0 base score = 9.8 / 10). Attackers can run arbitrary commands on your server, in many cases from the root.
Please make sure that you are using a fixed version (4.92) or already patched.
Or patch an existing one, see the immaculate comment thread .
Update for centos 6 : see Theodor comment - for centos 7 it also works if it hasn’t arrived directly from epel yet.
UPD: Ubuntu are affected on April 18 and 18.10 , an update has been released for them. Versions 16.04 and 19.04 are not affected, unless custom options were installed on them. More details on their official website .
Information about the problem on Opennet
Information on the Exim website
Now the problem described there is being actively exploited (by a bot, presumably), I noticed an infection on some servers (running on 4.91).
Further reading is relevant only for those who have already "hit" - you must either transport everything to a clean VPS with fresh software, or look for a solution. Will we try? Write if someone can overcome this malvar.
If you, as an Exim user and reading this, still have not updated (you are not convinced of the availability of 4.92 or the patched version), please stop and run to update.
For those who are already caught, let's continue ...
UPD: supersmile2009 found a different kind of malware and gives the right advice:
The infection is noticeable like this: [kthrotlds] loads the processor; on weak VDS by 100%, on servers is weaker but noticeable.
After infection, the malware deletes entries in the crowns, registering only there in startup every 4 minutes, while the crontab file makes immutable. Crontab -e cannot save the changes, throws an error.
Immutable can be removed for example like this, and then delete the command line (1.5kb):
Then, in the crontab (vim) editor, delete the line and save: However, one of the active processes overwrites it again, I understand. At the same time, a bunch of active wget'ov (or curl'ov) hangs on the addresses from the installer script (see below), for now I am knocking them down like this, but they start again:
The trojan installer’s script found here (centos): / usr / local / bin / nptd ... I don’t post it to avoid, but if anyone is infected and understands shell scripts, please study carefully.
I will supplement as information is updated.
UPD 1: File demolition (with preliminary chattr -i) /etc/cron.d/root, / etc / crontab, rm -Rf / var / spool / cron / root did not help, as well as stopping the service - I had to completely crontab yet tear out (rename the bin file).
UPD 2: The installer of the trojan sometimes also wallowed in other places; search by size helped:
find / -size 19825c
UPD 3: Attention! In addition to disconnecting selinux, the trojan also adds its SSH keyat $ {sshdir} / authorized_keys! And it activates the following fields in / etc / ssh / sshd_config, if they have not already been registered as YES:
PermitRootLogin yes
RSAAuthentication yes
PubkeyAuthentication yes
echo UsePAM yes
PasswordAuthentication yes
UPD 4: Summarizing for now: disable exim, cron (with roots), urgently remove the trojans key from ssh and edit the sshd config, restart sshd! And then it’s not exactly what it will help, but without it, it’s generally a disaster.
Important information from the comments about patches / updates was taken to the beginning of the article so that readers can start with it.
UPD 5: Another Denny writes that Malware changed passwords in WordPress.
UPD 6: Paulmann has prepared a temporary medicinetesting! After rebooting or disabling the medicine, it seems to fly off, but so far at least.
Who will make (or find) a stable solution, please write, help a lot.
UPD 7: The user clsv writes:
You can clear the entire Exim queue like this:
exipick -i | xargs exim -Mrm
Checking the number of entries in the queue:
exim -bpc
UPD 8: Thanks again for the information. AnotherDenni : FirstVDS offered their version of the script for treatment, let's test!
UPD 9: It seems to work , thanks to Cyril for the script!
The main thing is not to forget that the server has already been compromised and the attackers could have time to add some other atypical nastiness (not specified in the dropper) to plant.
Therefore, it is better to move to a completely installed server (vds), or at least continue to monitor the topic - if something is new, write in comments here, because obviously, not everyone will move to a fresh installation ...
UPD 10: Thanks again clsv : it reminds that not only servers are infected, but for example Raspberry Pi and all kinds of virtual machines ... So after saving the servers do not forget to save your video consoles, robots, etc.
UPD 11: From the author of the healing script, an important note for “manually healing”:
(after applying this or that method to combat this malware)
UPD 12: supersmile2009 found another (?) Malware in its Exim queue and advises to first examine its specific problem before treatment begins.
UPD 13: lorc advises moving to a clean system as soon as possible and transferring files very carefully, as Malware is already in the public domain and can be used in other, less obvious and more dangerous ways.
UPD 14: reassuring themselves that as smart people do not start from the root - another urgent message from clsv :
UPD 15: when moving to a clean server from a compromised one, do not forget about hygiene, a useful reminder from w0den :
UPD 16: daykkin and savage_me ran into another problem : there was one version of Exim in the ports system, but in reality another one was running.
So everyone after the upgrade should make sure that they are using the new version!
The server used DirectAdmin and stood its old package da_exim (old version, no vulnerability).
At the same time, with the help of DirectAdmin's manager of custombuild packages, in fact then a newer version of Exim was installed, which is already vulnerable.
It was in this situation that the update through custombuild also helped.
Do not forget to backup before such experiments, and also make sure that before / after the update all Exim processes of the old version were stopped and not “stuck” in memory.
Several million servers around the world are potentially vulnerable, the vulnerability is rated as critical (CVSS 3.0 base score = 9.8 / 10). Attackers can run arbitrary commands on your server, in many cases from the root.
Please make sure that you are using a fixed version (4.92) or already patched.
Or patch an existing one, see the immaculate comment thread .
Update for centos 6 : see Theodor comment - for centos 7 it also works if it hasn’t arrived directly from epel yet.
UPD: Ubuntu are affected on April 18 and 18.10 , an update has been released for them. Versions 16.04 and 19.04 are not affected, unless custom options were installed on them. More details on their official website .
Information about the problem on Opennet
Information on the Exim website
Now the problem described there is being actively exploited (by a bot, presumably), I noticed an infection on some servers (running on 4.91).
Further reading is relevant only for those who have already "hit" - you must either transport everything to a clean VPS with fresh software, or look for a solution. Will we try? Write if someone can overcome this malvar.
If you, as an Exim user and reading this, still have not updated (you are not convinced of the availability of 4.92 or the patched version), please stop and run to update.
For those who are already caught, let's continue ...
UPD: supersmile2009 found a different kind of malware and gives the right advice:
There can be a great many malicious programs. By launching the medicine, the user will not be cured and, perhaps, will not know what he needs to be treated for.
The infection is noticeable like this: [kthrotlds] loads the processor; on weak VDS by 100%, on servers is weaker but noticeable.
After infection, the malware deletes entries in the crowns, registering only there in startup every 4 minutes, while the crontab file makes immutable. Crontab -e cannot save the changes, throws an error.
Immutable can be removed for example like this, and then delete the command line (1.5kb):
chattr -i /var/spool/cron/root
crontab -eThen, in the crontab (vim) editor, delete the line and save: However, one of the active processes overwrites it again, I understand. At the same time, a bunch of active wget'ov (or curl'ov) hangs on the addresses from the installer script (see below), for now I am knocking them down like this, but they start again:
dd
:wqps aux | grep wge[t]
ps aux | grep cur[l]
echo "Stopping..."
kill -9 `ps aux | grep wge[t] | awk '{print $2}'`
kill -9 `ps aux | grep cur[l] | awk '{print $2}'`
The trojan installer’s script found here (centos): / usr / local / bin / nptd ... I don’t post it to avoid, but if anyone is infected and understands shell scripts, please study carefully.
I will supplement as information is updated.
UPD 1: File demolition (with preliminary chattr -i) /etc/cron.d/root, / etc / crontab, rm -Rf / var / spool / cron / root did not help, as well as stopping the service - I had to completely crontab yet tear out (rename the bin file).
UPD 2: The installer of the trojan sometimes also wallowed in other places; search by size helped:
find / -size 19825c
UPD 3: Attention! In addition to disconnecting selinux, the trojan also adds its SSH keyat $ {sshdir} / authorized_keys! And it activates the following fields in / etc / ssh / sshd_config, if they have not already been registered as YES:
PermitRootLogin yes
RSAAuthentication yes
PubkeyAuthentication yes
echo UsePAM yes
PasswordAuthentication yes
UPD 4: Summarizing for now: disable exim, cron (with roots), urgently remove the trojans key from ssh and edit the sshd config, restart sshd! And then it’s not exactly what it will help, but without it, it’s generally a disaster.
Important information from the comments about patches / updates was taken to the beginning of the article so that readers can start with it.
UPD 5: Another Denny writes that Malware changed passwords in WordPress.
UPD 6: Paulmann has prepared a temporary medicinetesting! After rebooting or disabling the medicine, it seems to fly off, but so far at least.
Who will make (or find) a stable solution, please write, help a lot.
UPD 7: The user clsv writes:
If you haven’t said yet, the virus is resurrected thanks to an unsent email in Exim, if you try to send an email again, it is restored, look at / var / spool / Exim4
You can clear the entire Exim queue like this:
exipick -i | xargs exim -Mrm
Checking the number of entries in the queue:
exim -bpc
UPD 8: Thanks again for the information. AnotherDenni : FirstVDS offered their version of the script for treatment, let's test!
UPD 9: It seems to work , thanks to Cyril for the script!
The main thing is not to forget that the server has already been compromised and the attackers could have time to add some other atypical nastiness (not specified in the dropper) to plant.
Therefore, it is better to move to a completely installed server (vds), or at least continue to monitor the topic - if something is new, write in comments here, because obviously, not everyone will move to a fresh installation ...
UPD 10: Thanks again clsv : it reminds that not only servers are infected, but for example Raspberry Pi and all kinds of virtual machines ... So after saving the servers do not forget to save your video consoles, robots, etc.
UPD 11: From the author of the healing script, an important note for “manually healing”:
(after applying this or that method to combat this malware)
it is necessary to reboot - the malware is sitting somewhere in open processes and, accordingly, in memory, and writes itself in a new crown every 30 seconds
UPD 12: supersmile2009 found another (?) Malware in its Exim queue and advises to first examine its specific problem before treatment begins.
UPD 13: lorc advises moving to a clean system as soon as possible and transferring files very carefully, as Malware is already in the public domain and can be used in other, less obvious and more dangerous ways.
UPD 14: reassuring themselves that as smart people do not start from the root - another urgent message from clsv :
Even if it works outside of the root, hacking occurs ... I have a debianjessieUPD on OrangePi : stretch, exim is launched from Debian-exim, and there was a hack anyway, the crowns worn, and so on.
UPD 15: when moving to a clean server from a compromised one, do not forget about hygiene, a useful reminder from w0den :
When transferring data, pay attention not only to executable or configuration files, but also everything that may contain malicious commands (for example, in MySQL it can be CREATE TRIGGER or CREATE EVENT). Also, do not forget about .html, .js, .php, .py and other public files (ideally, these files, like other data, should be restored from local or other trusted storage).
UPD 16: daykkin and savage_me ran into another problem : there was one version of Exim in the ports system, but in reality another one was running.
So everyone after the upgrade should make sure that they are using the new version!
exim --versionThe server used DirectAdmin and stood its old package da_exim (old version, no vulnerability).
At the same time, with the help of DirectAdmin's manager of custombuild packages, in fact then a newer version of Exim was installed, which is already vulnerable.
It was in this situation that the update through custombuild also helped.
Do not forget to backup before such experiments, and also make sure that before / after the update all Exim processes of the old version were stopped and not “stuck” in memory.