Linux LKM rootkit
Skidmap Linux Rootkit: Traces and Offline Checks
Skidmap is a Linux cryptominer that loads kernel modules to hide files and fake CPU and network figures, and backdoors PAM. Its traces, checks and recovery.
- Kind
- Linux LKM rootkit · Kernel layer
- Platforms
- Linux
- First seen
- 2019
- Status
- In the wild
- Also known as
- SkidMap
- Checked by
- Linux kernel
Overview
Skidmap is a Linux cryptomining malware family that goes further than most miners. It loads its own kernel modules to hide the miner. Those modules make the CPU look idle and falsify network figures. Skidmap also leaves several ways back in, among them a backdoored PAM module that accepts one password for every account.
Trend Micro described it in September 2019. Trend Micro notes that the kernel modules make cleanup harder than for user-mode miners, and that the extra access routes let it reinfect a host.
In July 2023, Trustwave SpiderLabs reported a new variant on its honeypots. It broke in through Redis servers without authentication. It carried kernel-module packages for many distributions and kernels, Alibaba, openEuler, EulerOS and Anolis among them.
Neither vendor attributes Skidmap to a named actor. MITRE ATT&CK tracks it as S0468.
How it loads and hides
2019 version (Trend Micro).
- Access and install. A cron job fetches and runs an install script every minute. The script downloads the main binary to
/var/lib/pcand runs it. The binary weakens the host:- it runs
setenforce 0and edits/etc/selinux/configto disable SELinux; - it adds the attackers' public key to
authorized_keys; - it replaces
pam_unix.sowith a version that accepts a fixed password for any user.
- it runs
- Miner by distribution. On Debian-family systems the miner goes to
/tmp/miner2. On CentOS and RHEL, an archivecos7.tar.gzis downloaded, decrypted with OpenSSL (Triple DES) and installed. - Components:
kaudited(/usr/bin/kaudited) drops and loads the kernel modules, picking the build that matches the running kernel, and installs a watchdog for the miner;- a replacement
rmsometimes adds a cron job that downloads and runs a file; - the
iproutemodule hooksgetdentsto hide files; - the
netlinkmodule falsifies network statistics for chosen addresses and ports, hides thepamdicksprocess, and reports a low CPU load.
2023 variant (Trustwave).
- Access. The attackers write cron jobs through Redis keys named
HA. Every ten minutes a job fetches a script, namedb,corfon different hosts. - Dropper. The script installs an ELF dropper at
/var/lib/gif(earlierjpeg), marked immutable withchattr +i. The dropper reads/etc/*-releaseand picks one of several encrypted packages, staged under a new folder in/usr/include/:gold16,gold18,gold20,stream9,euler21,euler22. - Kernel modules:
mcpuinfo.kofakes system load through a replaced/prochandler (fake_loading_proc_show);kmeminfo.koregisters netfilter hooks (nf_register_net_hook) to inspect, change or drop packets;mzoneinfo.kois referenced but was never recovered.
- Other changes:
- the miner runs from
/tmp/.miner(or.mimer) on Debian and Ubuntu, and from memory on RHEL and CentOS; - attacker SSH keys go into files that Trustwave's text spells
/root/.ssh/authoried_keysandauthoried_keys2. Check both spellings; - a reverse shell is appended to
/etc/cron.hourly/prelink; - SELinux and auditd settings are changed;
- a
clean.shscript and a binary namedwtmpwipe logs.
- the miner runs from
Trustwave adds that the attackers appear to block the unhide tool. ClamAV and rkhunter reported nothing on the infected honeypot, while chkrootkit flagged malicious module indicators.
Traces on disk
| Location | What | Source |
|---|---|---|
| Crontabs fetching scripts every minute (2019) or ten minutes (2023) | Infection and reinfection | Trend Micro, Trustwave |
/var/lib/pc; /usr/bin/kaudited | 2019 main binary; module loader and watchdog | Trend Micro |
File names crypto514, iproute514, netlink514, kswaped, systemd_network | 2019 components listed as IoCs | Trend Micro |
/tmp/miner2; cos7.tar.gz | 2019 miner, by distribution | Trend Micro |
A pam_unix.so that differs from its package | Master-password backdoor | Trend Micro |
/var/lib/gif (immutable), earlier /var/lib/jpeg | 2023 dropper | Trustwave |
/usr/include/<new folder>/ holding gold*, stream*, euler* packages and a version.txt | 2023 staging | Trustwave |
mcpuinfo.ko, kmeminfo.ko | 2023 kernel modules | Trustwave |
/tmp/.miner, /tmp/.mimer | 2023 miner on Debian and Ubuntu | Trustwave |
/root/.ssh/authoried_keys, authoried_keys2, or authorized_keys with unknown keys | SSH backdoor | Trend Micro, Trustwave |
/etc/cron.hourly/prelink with a reverse shell appended | 2023 persistence | Trustwave |
/etc/selinux/config set to disabled; altered audit configuration | Weakened host | Trend Micro, Trustwave |
Gaps in wtmp and system logs | Log wiping | Trustwave |
Both reports list hashes and network indicators. Take them from the reports.
How to check a host
Live, with caution. The modules falsify CPU and network figures, so the usual "high CPU, find the process" approach fails. Look from outside and below:
cat /proc/sys/kernel/tainted
lsattr /var/lib/gif /var/lib/pc /usr/bin/kaudited 2>/dev/null # immutable files are unusual there
for m in /sys/module/*; do [ -e "$m/initstate" ] && ! grep -q "^${m##*/} " /proc/modules && echo "only in sysfs: ${m##*/}"; done
crontab -l; ls -la /etc/cron.* /var/spool/cron* 2>/dev/null
- Measure power draw, temperature or the hypervisor's view of the guest's CPU. Trustwave suggests unexplained fan noise and heat as a physical sign.
- Read network flows on the switch or the cloud provider's flow logs. The host's own counters may be faked.
- Capture memory before cleaning.
Offline.
R=/mnt/evidence
ls -la "$R"/var/lib/pc "$R"/var/lib/gif "$R"/var/lib/jpeg "$R"/usr/bin/kaudited "$R"/tmp/.miner "$R"/tmp/.mimer "$R"/tmp/miner2 2>/dev/null
find "$R" -xdev \( -name 'mcpuinfo.ko' -o -name 'kmeminfo.ko' -o -name 'mzoneinfo.ko' -o -name '*514' \) 2>/dev/null
ls -la "$R"/usr/include | grep -vE '\.h$' # unexpected folders among headers
cat "$R"/etc/cron.hourly/prelink; ls -la "$R"/root/.ssh/
debsums --root="$R" -c 2>/dev/null | grep -i pam # Debian/Ubuntu
rpm --root "$R" -V pam 2>/dev/null # RHEL family
zgrep -hE 'taint|module verification' "$R"/var/log/kern.log* "$R"/var/log/messages* 2>/dev/null
The modules may sit outside /lib/modules: neither report gives their path. Search the whole image by name and by type (file reports ELF 64-bit LSB relocatable for .ko files). For each module you find, see checking kernel modules offline.
What the checker looks for
The linuxkernel module can see:
- Module files under
lib/modules/: "Unsigned module among signed modules" and "Module unknown to the kernel build and to every package". Package ownership is read from the dpkg database only, so on a RHEL-family host the second finding rests onmodules.orderalone. Hook-target names such asgetdents, when the module carries them as strings, raise "Names kernel functions that rootkits hook"; netfilter registration imports appear only as context in that finding. - Kernel taint warnings in
kern.logormessages. The journal is not read. - A module hidden from
/proc/modules, if a module unlinks itself from/proc/modulesbut not from sysfs: "Loaded module missing from /proc/modules" from the collect script's_live/outputs. Neither report says what Skidmap's modules do there.
It does not read:
- cron, the Redis-written jobs or
authorized_keys; - the PAM module, so it cannot spot the replaced
pam_unix.so; - modules stored outside the module tree, which the reports suggest is likely.
Linux Persistence covers the PAM, cron and SSH key side.
Removal and recovery
- Preserve memory and the disk, including the immutable files and the staging folders.
- Expect reinfection.
- Cut the entry point first: Redis exposed without authentication (2023) or whatever let the 2019 install script run.
- Remove every cron entry, the SSH keys and the
prelinkreverse shell. - The 2019 replacement
rmand the watchdog exist to put things back.
- Rebuild. A replaced
pam_unix.someans any account could be entered with the master password. Kernel modules ran with full control. Reinstalling packages in place is not enough to trust the host. Rebuild it and restore data from backups. - Rotate every password and key that authenticated on the host, since the PAM backdoor saw them. Revoke the attacker keys wherever the same
authorized_keysmight have been copied. - Harden.
- Require authentication on Redis and keep it off the internet; Trustwave notes that protected mode alone was not enough.
- Keep SELinux enforcing, enforce module signatures, and alert on changes to
pam_unix.soand/etc/cron.*.
See Diamorphine and libprocesshider for the lighter hiding tools other mining crews use.
Sources
Primary public reports, newest first.
- 2023-07-30Honeypot Recon: New Variant of SkidMap Targeting RedisTrustwave SpiderLabs (LevelBlue)
- 2019-09-16Skidmap Linux Malware Uses Rootkit Capabilities to Hide Cryptocurrency-Mining PayloadTrend Micro
See also
Related families
- DiamorphineLinux LKM rootkit
- ReptileLinux LKM rootkit
- libprocesshiderLinux user-land rootkit