Managing a proxyware infection

TL;DR you, or someone you know is infected with proxyware, or RESIP. Evil people are secretly (probably) using their infected device to commit crimes from whatever network connection their device happens to be on, hiding behind a mass diversity of exit points, possibly including your internet service. Those crimes range from scraping in violation of ToS to evading defenses against nation-state level crime, cognitive warfare, massive fraud, DDOS attacks, credential stuffing attacks, and ransomware attacks. If you don’t want to be complicit in such acts, or don’t want your network to be found complicit in such acts, you need to learn how to detect RESIP on your network, how to stop it, and how to prevent it from being installed on your devices or the devices of people whose digital spray might spatter you.

Most of the papers and reports on the malware as a service industry focus on mitigating being a target of the malware, which is specifically designed to evade standard defensive measures, many of which rely on defensive tools like maintained, tractable lists of known infected/malicious hosts or repeated threat signatures, which this sort of malware generally evades. Nobody maintains lists of the hundreds of millions of RESIP infected devices already in the wild and tools like Snort/Suricata or Fail2Ban rely on repeat offenses creating temporary, often geometrically temporally extended bans for bad behavior as a balance between dynamic defense and unintentionally self-denial of service attacks. RESIP doesn’t (usually) attack your network, but uses your network to attack other people, making you and your network complicit in their crimes.

Some of the methods of RESIP endpoint detection and mitigation are sophisticated and may evolve, such as RTT timing anomalies from behind-the-proxy endpoint traffic. However, these defenses, if widely implemented, could also deny service to legitimate users using self-hosted VPNs to protect themselves against data collection and exploitation by criminals or rogue states, which would tend to have the same signatures.

I found little guidance on the steps to take to protect a small network from the reputational harm and bandwidth theft (and potential legal liability) a RESIP-infected endpoint causes. In the end, a fairly standard firewall-based defensive strategy should be effective against new infections and infected “guest” devices and with low penalty or overhead for legitimate use. The key steps are:

• Using a tool like pfBlockerNG DNS filter lists to block access to command and control endpoints, which tend to be hosted on VPS providers with minimal KYC checks or that engage in red flag behavior like accepting crypto payments. Standard blocklists need to be supplemented with in-situ detected RESIP endpoints to be effective.
• Use firewall rules to block direct access to DNS services, which RESIPs fail over to automatically, by port, both 53 and DoT on 853, forcing potentially infected devices through local DNS and so through DNS filtering.
• Use firewall rules to block access to known DNS over HTTP endpoints (DoH!). The malware I observed had a finite list of the major DoH providers, globally, that it cycled through. I don’t have a block list of DoH endpoints and so constructed one (provided below) by monitoring the RESIP traffic for a couple of days.
• Use pfBlockerNG to Block known RESIP C﹠C endpoints by IP. In this it was necessary to block the ASNs of many sketchy VPS providers of the sort that would host such malware. There is some risk of backscatter taking out legitimate VPS hosted services with this broad ASN or C-class approach, but the RESIP endpoints bounce IPs through VPS host ranges.
• Enable proxy detection rules in intrusion detection engines (Snort or Suricata typically) and monitor alerts to tune and expand the block lists, this is particularly helpful in detecting DoH connections, though they also have an obvious signal of being a 443 connection to a known DNS IP which doesn’t host web pages of interest to typical Facebook browsing users.
• Track traffic flows and measure results with traffic flow and analysis tools like ntopng.

All of this can be done with the free and open source pfSense firewall and with standard packages available with it.

An interesting artifact of working through the necessary defensive measures to cripple RESIP malware, short of simply blocking any infected device (a very reasonable brute force strategy and easier to implement than this fuss, but not user-friendly) is the degree to which the process would mirror the steps needed to monitor and disrupt censorship circumvention tools such as those that were, until DOGE cuts mostly eliminated funding for them, a key tool in providing support and access to democracy activists operating in democracy-challenged environments that previously were considered US adversaries. While it is clear that if it is possible to track down and disconnect RESIP with pfSense, a tool like SORM is certainly more capable. There was an arms-race between the groups the US helped support that were working to advance democracy in the world and those seeking to expand authoritarian autocracy before the US switched sides, that arms race continues with RESIPs very uncomfortably, in broad terms ∩=∪.

While the purposes of censorship circumvention tools the USAGM in general and specifically the Open Technology Fund supported in the pre-DOGE days, such as Tor, Signal, Lantern, Psiphon, OONI, Briar, and lots of smaller tools is very different from the malware networks run over RESIP, there is significant overlap in the functional requirements, such as hiding the content of and true endpoints of internet communications from observers that would want to “censor” that communication and/or prosecute the endpoints.

The economics diverge: RESIP malware is a volume-driven model with trivial value per endpoint. If a compromised endpoint is shut down, rolled up, or otherwise punished for the attacks they, typically unwittingly, facilitated, the loss to a 70m endpoint operating network is irrelevant. Activist protection tool developers, contrary, tend to be very protective of each user and would consider the loss to imprisonment or execution of any one a problem; an asymmetry that would likely reconverge at the perception of the endpoint themself in both cases. The capital economics also diverge, especially these days, with Bright Data alone reporting $300mm ARR vs. the consequences of the DOGE OTFpocalypse.

Discovering and shutting down a RESIP infected client without crippling my network

When you think of run-of-the-mill malware attacks, you don’t usually think of iPhones as being particularly vulnerable. One of the benefits of a closed ecosystem that denies users access to or control over their own hardware is a baseline level of relatively capable corporate security: sure iOS isn’t Graphene-grade, but it is, out of the box, generally perceived as more secure than a typical Android OEM offering, let alone a Windows box.

The result is that most users, who after all don’t care what the device is doing as long as it lets them insta and chat and post and scroll, find it less easy to hack themselves with unsavory software; but by the same token, it is a lot harder to clean that mess up when it does happen without a factory reset and reinstall.

Completely independently to the RESIP discovery, a person using my networks was a little confused and attributed Wi-Fi data reliability issues to ISP issues. Spoiler: turns out there is a bit more packet loss than there should be, 3-6 actual multi-second connectivity drops per day, which confounded out of hand dismissal, but real data collection invalidated the scope of the complaint while confirming the underlying premise the process uncovered an otherwise unnoticed RESIP infection.

The tool I developed to validate connectivity and attribute any detected failures to the network layer is published on codeberg.

A few glitches a day do show in the monitoring and as I can’t remotely instrument between the firewall and the gateway, I started walking through the logs and noticed some unexpected traffic, rather a lot from one dynamic DHCP (guest) device and noticed Snort was triggering on something I hadn’t seen before.

"ET TROJAN Observed DNS Query to Bandwidth Sharing Tool Domain (earn.fm)"
src=(my.exit.ip)  dst=8.8.8.8  port=53

Doing a full port scan with nmap, it was clearly an iPhone, as also indicated by some of the low-volume traffic flows captured in ntopng, but the high volume traffic was something nefarious. Turns out the device is infected with proxyware (or RESIP for RESidential IP Proxy), a category of malware, sometimes covertly bundled, browser hijacked, or otherwise “cracked,” and sometimes overtly foisted off on users as “passive income” or other scams. RESIP is basically a business that develops malware that users advertently or inadvertently install on their devices and which turns them into bot-net nodes. The RESIP gang then resells access to the resulting massive residential/business IP network to whoever wants to bypass cybersecurity protections by routing nefariously through real residential networks. It is Crime As A Service.

The uses for such a network vary from kinda legitimate network surveys that have no harmful nefarious intent to AI training engines circumventing IP-based restrictions on their spiders, to far more nefarious activities where avoiding attribution is critical to avoiding Interpol Red Letters.

You might use a commercial VPN, but your VPN endpoints are in data centers and… c’mon… everyone knows end-user browsing coming out of a data center IP is a VPN endpoint (e.g. https://github.com/X4BNet/lists_vpn/blob/main/output/vpn/ipv4.txt – 10,696 IPs or by commercial provider https://github.com/coocoobau/vpn-ip-lists). You can also find lists of Tor nodes: https://www.dan.me.uk/tornodes (14,756).

But I know of no RESIP IP list. Such a list would be so large as to be, itself, a network burden. A randomly selected single provider claims to have p0wned more than 70,000,000 devices worldwide into their botnet, and they’re just one of many; and they take crypto as payment for access to their nearly impossible to block network, just like Silk Road, surprise, surprise.

An iPhone? Srsly? Not the Windows Box?

iOS is heavily sandboxed compared to Windows. But it has one large hole: the users themselves. When an app asks for “VPN” or “Network Extension” permission, granting it gives the app license to route the device’s traffic however it wants. There’s no antivirus on iOS (in the meaningful sense) because there’s almost nothing for AV to scan. And the App Store review process, while real, doesn’t reliably catch apps whose primary monetization is a hidden bandwidth-sharing SDK silently included in something that appears to be a free VPN, a coupon finder, a screen-time tracker, or a “passive income” earner.

The iOS model relies on trusting the App Store (or the Apple Store genius bar) and believing that they wouldn’t allow anything “bad,” which is contradicted by some obvious troubling evidence and can fail if the user doesn’t fully comprehend the consequences of decisions.

Windows is far less secure (by CVSS), but at least you can fix it yourself without having to go to the Apple Store to have them wipe your data and tell you to start over.

So, yes, an iPhone was the malware host, not that it made any difference in the backend resolution. A complication with iOS is that it isn’t as easy as “run the anti-malware” as on other platforms. Android has a pretty robust anti-malware industry, which fits the intuition that it is less secure than iOS, and which is technically correct (yes, the best kind of correct), but only by a tiny margin and certainly not by enough to justify the absence of corrective tools on the iOS platform where security is more branding than reality.

Malware flavor identification

ntopng has nDPI integration that gives reasonable protocol identification with reverse DNS and TLS SNI inspection on flows which revealed the following dominating the iPhone’s traffic, by far:

socket-prod.earn.fm           ← EarnApp (Bright Data)
socket-backup.earn.fm         ← EarnApp backup
proxy.mystnodes.com           ← Mysterium Network
resi-api.pawns.app            ← Pawns / IPRoyal

Three different RESIP products were running simultaneously on this one device. That’s unusual but not unprecedented — these things tend to ship bundled inside the same “free VPN” wrapper apps, often marketed on dodgy TikTok or Youtube “influencer” spamvertisements, so it may be a single app with multiple malware tools integrated, or the user might have gone on a malware signup spree, embracing the “passive income” promise.

The iPhone was pushing ~21 MB of inbound UDP traffic per hour to a handful of DataPacket-hosted IPs on port 5001, plus another 10 MB outbound. Multiplied across a day, this was over 700 MB/day of malicious traffic serving an unknown breadth of criminal activities. The consequences of such traffic are real: an obvious likely outcome is ending up on IP block lists, which can result in email delivery failing, your own sites being flagged as malware purveyors by IP, IP-reputation list inclusion making browsing tedious as sites often trigger either outright blocks on access or require additional CAPTCHA tests before proceeding.

Malware Domain Blocking with pfBlockerNG

This isn’t sufficient, but it is necessary: create a feed of all the RESIP domains (and add public threat intel). The mechanism pfBlockerNG uses is to answer DNS requests for blocked domain IPs with a blackhole address like 10.10.10.1 instead of looking up the real IP address, which results in DNS failures for that specific domain.

This sort of malware is aggressive about circumventing DNS in a manner similar to censorship circumvention tools; direct DNS lookups and DoH and DoT where that fails, so this sort of block is of limited effect against the malware until the circumvention tricks are also locked down. Even so, these domains should be considered malware and blocked, it moves the needle (thousands of packets blocked from a single infected host):

# cat /var/db/pfblockerng/RESIP.txt

earn.fm
honeygain.com
earnapp.com
brightdata.com
luminati.io
bright-sdk.com
packetstream.io
peer2profit.com
repocket.com
traffmonetizer.com
proxyrack.com
soax.com
nodepay.org
grass.io
gradient.network
spare.app
honest.network
storj.io
filecoin.io
sia.tech
ipfs.io
hola.org
touchvpn.net
betternet.co
*.earn.fm
mystnodes.com
*.mystnodes.com
mysterium.network
*.mysterium.network
datapacket.com
*.datapacket.com
some-tools.com
*.some-tools.com
pawns.app
*.pawns.app
iproyal.com
*.iproyal.com
resi-api.pawns.app
linesso.com
*.linesso.com
linesdks.com
*.linesdks.com
dataprocess.one
*.dataprocess.one
kytira.cc
*.kytira.cc
mediadataflow.cloud
*.mediadataflow.cloud
datamassive.us.com
*.datamassive.us.com
floporin.com
*.floporin.com
zimpire.com
*.zimpire.com
glapoton.com
*.glapoton.com

The RESIP by domain name block list group I’m using also includes the following externally maintained lists:

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/pro.txt
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt

Note that these lists (and many others like them) are maintained by a variety of mechanisms that instrument the reputation of a domain name and when a domain name’s reputation is sufficiently bad, add the domain name to the naughty list so that admins can block it. If you handle your traffic reasonably fairly and don’t host malware or run scams or spam your domain should not be on one of these lists. If a user on your network brings a RESIP infection with them, you probably will end up on one of these lists and then the internet will become a desolate place of closed doors and unresponsive servers.

IP level blocking, specific and by ASN

pfBlockerNG also supports per-IP blocking. This is useful too, but malware uses DNS, not so much to make their domains mnemonic so humans can type them from memory but because the redirection allows the command and control (C﹠C) server to be moved around a VPS provider’s DNS space or from one VPS provider to another with a simple DNS update. Domain blocking tends to be stickier, where it works, than per-IP blocking, but per IP blocking is harder to circumvent.

The list of IPs I found:

# cat /var/db/pfblockerng/known_RESIP_ips.txt
49.51.252.53
45.78.213.100
43.102.107.48
43.102.214.14
43.166.230.158
170.106.177.230
107.150.106.113
67.213.124.165
89.222.99.110
89.222.99.115
103.195.100.222
185.110.216.0/23
5.161.0.0/16
178.156.0.0/16
178.63.0.0/16
185.228.168.9
185.228.168.10
185.228.168.168
185.228.169.9
185.228.169.11
185.228.169.168
135.148.0.0/16

This includes individual machines and subnets that appeared to be hosting a lot of malware. There is, always, some backscatter, but this is why you should check the reputation of the IPs of any hosting service you chose to use to host your own services. If they’re willing to host malware, you shouldn’t use them for honest services or you’ll likely be blocked. The above list resolves to 478,194 IPs (mostly the /16) and has blocked 2,071 packets.

And to continue the broad blocking needed to capture more of the problem servers, I needed to block entire ASNs. Some are obvious risks that many would block at the country-level, but I try to avoid country-level bans as excessive, plus I tend to work with people in some of those countries. The ASN block is more selective and is set up using pfBlockerNG’s IPv4 Source Definitions “ASN” list facility, which resolves the ASNs to IP ranges. This is a heavy list and definitely has some backscatter, but RESIP is pretty awful and any ASN that hosts it should seriously reconsider.

AS14576  [ Hosting Solution Ltd. ]
AS16276  [ OVH SAS ]
AS21859  [ Zenlayer Inc ]
AS23470  [ ReliableSite.Net LLC ]
AS26042  [ FiberState ]
AS44066  [ firstcolo GmbH ]
AS45102  [ Alibaba (US) Technology Co. ]
AS56630  [ Melbikomas UAB ]
AS60068  [ Datacamp Limited ]
AS62240  [ Clouvider ]
AS132203 [ Tencent Cloud International ]
AS135376 [ Rahanet Internet Service Provider ]
AS135377 [ UCLOUD INFORMATION TECHNOLOGY (HK) LIMITED ]
AS138549 [ Net Cafe ]
AS150436 [ Byteplus Pte. Ltd. ]
AS212238 [ Datacamp Limited ]
AS213230 [ Hetzner Cloud GmbH ]
AS396356 [ Latitude.sh ]

This short list of ASNs resolves to 5,825 individual IPs and has blocked 289,990 packets, all most probably malicious.

An important detail came out with Claude’s help: Hetzner runs two ASNs: AS24940 Hetzner Online, their traditional dedicated-server/colocation business with a customer base that is heavily small/medium European business (Mastodon instances, indie SaaS, personal sites), and AS213230 Hetzner Cloud GmbH, their on-demand VPS/cloud product. The customer base for the Cloud ASN skews much more toward “anyone who wants a cheap VM on short notice,” which includes a lot of proxyware operators.

By blocking only AS24940, initially I’d caught the less proxyware-relevant side of Hetzner’s network however, switching to blocking AS213230 (and leaving AS24940 allowed, since the latter has much higher legitimate-traffic exposure) blocks the actual proxyware infrastructure with minimal collateral.

Many large cloud providers run their cloud-VPS products on a separate ASNs from their main businesses: AWS, Google Cloud, Microsoft, OVH, and Hetzner all have similar splits. When blocking large providers, check for ASN level reputations.

Force Bad Hosts to use local DNS with port and IP aliases

RESIP, like circumvention tools, is aggressive about finding C﹠C servers and tries to work around using encrypted DNS over TLS (DoT) and the even sneakier DNS over HTTPS (DoH). However, they fall short of setting up VPS DoH servers with hardcoded IPs, this would be challenging for them to maintain and, as it would be extremely hard to block without subtle flow analysis. This is fortunate, as it is hard for a relatively under-resourced operators to block, especially if they were integrated into otherwise reputable VPS providers. Fortunately, for a RESIP operator, the return on investment of maintaining such an infrastructure is low, at least for now.

There are legitimate uses for remote DNS lookups, it is entirely reasonable to minimize DNS logging on whoever’s network you’re on to preserve privacy and is a very useful as a fallback to improve overall internet connectivity: RESIP infected hosts do not earn that privilege.

Allowed (or banned) Ports

First block the standard ports for DNS: 53 and 853, or better yet (the approach I recommend), only allow outbound connectivity on reasonable use ports to block an awful lot of dubious traffic:

OK TCP: 80, 443, 465, 587, 993, 995, 5222, 5223, 5228:5230
OK UDP: 443, 3478:3481, 19302:19309, 16384:32767

Block everything else, including direct DNS on 53 or DoT on 853.

Block DoH on 443

Note that 443 is allowed, which on which both HTTPS and DoH run, so the known DoH servers have to be blocked as well. As responsible users might have reason to use DoH servers, an alias should be configured and used with firewall rules applied specifically to irresponsible users (and/or all guests):

1.1.1.1, 1.0.0.1                               # Cloudflare
8.8.8.8, 8.8.4.4                               # Google
9.9.9.9, 149.112.112.112                       # Quad9
104.16.248.249, 104.16.249.249                 # cloudflare-dns.com
156.154.70.0/23                                # UltraDNS / Vercara
185.222.222.222, 45.11.45.11                   # DNS.SB
120.53.53.53, 1.12.12.12                       # DNSPod / AliDNS
185.228.168.0/22                               # CleanBrowsing
77.88.0.0/18                                   # Yandex extended
4.2.2.2                                        # Level 3
76.76.2.0/24, 76.76.10.0/24                    # NextDNS
223.5.5.5, 223.6.6.6                           # AliDNS
77.88.8.0/24                                   # Yandex DNS
185.228.168.0/22                               # cleanbrowsing.org

Most of these are generally good eggs and helpful for the interwebs, but irresponsible hosts lose the privilege when they abuse the privilege.

Block RESIP specific port-ranges

This should be applied as an alias as well. These port numbers aren’t assigned to malware, and so backscatter might impact legitimate use, however the following ports tend to dominate the RESIP flows:

5000:5010

I applied this block to all users as a preventive measure in the event a RESIP infection lands on a more trusted host.

How to kill all in-flight flows

In-flight flows (probably, depends on FW config) are not killed by adding/updating pfBlockerNG ban lists without a restarting the flows, so once the blocks are in place, kill all flows to the device, for example:

# Kill states where 89.222.99.0/24 (the external proxyware IP range)
# is destination, regardless of source
pfctl -k 0.0.0.0/0 -k 89.222.99.0/24

and/or

for entry in $(pfctl -t pfB_Proxy_ASNs_v4 -T show); do
    pfctl -k 0.0.0.0/0 -k "$entry"
    pfctl -k "$entry" -k 0.0.0.0/0
done

Snort and ntopng

Snort and ntopng are great tools. Snort detects some proxyware traffic and can help alert and block it on detection. ntopng makes the bad traffic flows far more visible and allows detailed investigation of who’s talking to what. It also has good tools for setting up alerts, for example triggering on high volume DoH traffic or proxyware application detection and sending an email, so the infection can be managed before it destroys IP reputation or generates legal complications.

iPhone help guide

I don’t use vanity devices because I don’t need people to see me holding a company logo to validate my device purchases but for people who do, a few steps might mitigate RESIP risks:

1. Settings → General → VPN ﹠ Device Management — check for unfamiliar VPN configurations. Delete anything you don’t actively use (NordVPN, ProtonVPN, etc. are legitimate; “Free VPN,” “Speed VPN,” “Turbo VPN,” and the like are not).
2. Settings → Cellular (or Cellular Data) — scroll through per-app data usage, look for apps using surprising amounts of data, especially ones you don’t actively use.
3. Settings → Battery — look for apps with high background activity that you haven’t opened.
4. Settings → General → iPhone Storage — review installed apps, delete anything unrecognized or that was installed as a “free” anything.
5. Settings → Wi-Fi → Configure DNS — if it’s set to Manual with non-firewall IPs you didn’t configure, switch to Automatic.
6. Reset Network Settings (Settings → General → Transfer or Reset iPhone → Reset → Reset Network Settings) — clears VPN profiles, custom DNS, and Wi-Fi passwords. Annoying (you re-enter Wi-Fi passwords) but nukes any lingering network plumbing.
7. If the above doesn’t help, factory reset and set up as new (not from backup, which would restore the malware).

Conclusion

The primary lesson for me was to watch the traffic logs on my network, especially if new guests are on the network.  It was also instructive to see how effective the open source ecosystem around network monitoring and protection actually is.  Tools like pfSense, pfBlockerNG, Snort, and ntopng are powerful and effective: 100% of the revenue traffic of the RESIP gang is blocked, all that remains is foiled attempts at DNS lookups.

While I must confess to a touch of schadenfreude whenever the iOS illusion of superiority is exposed as a sham, the scourge of proxyware is equal opportunity and while iOS’s illusion of security makes it (ironically) harder to clean, the user is the primary vulnerability of every platform.