For those of us - myself included - who run a hosts file list (either using dnsmasq like Pi-hole or directly), here are the sources that Pi-hole use so you can add to your own solution:
There's a few on there I don't use and will look to implement. There's also a few they seem to have missed (perhaps intentionally?) so below I have included the lists I use in case it's useful for anyone else:
uBlock actually has a very good list of blockfiles, most of which (from memory) correspond to those you list. Last I checked it was on the order of 60k hosts.
I've used that to populate my own hosts + dnsmasq blockfiles. Using just the winhelp2002 list does a passably good job on a DD-WRT imaged router (~13k entries).
It's just a shell script to manage dnsmasq running on my FreeBSD home server. It's not sophicated but equally it's written specifically for my server so no very portable either.
I did think about writing something to share but projects like Pi-hole have done a better job serving the community than i could have. So i just share the sources i use instead incase any like-minded sysadmins find it useful.
Thanks for the recommendation but the shell script i have works good enough and has been for a few years now. Plus the container (FreeBSD jail to be precise) is pretty low footprint so while i don't have an issue with Python itself, it's an additional package I don't really need.
ublock only acts on the browser(s) to which it's been added. PiHole, other DNS blockers, or firewall-type (CIDR-block) blockers defend all applications from either DNS-supported services (most of them), or direct network access (most of the rest). To bypass the second, you'd need apps which had, say, VNC or proxy access to somewhere, and I strongly suspect those proxies would be fairly trivial to add.
I've been running various blocking and reporting systems for coming on 20 years myself, and find that applying my own hygiene controls to network traffic is ... surprisingly effective. Not bulletproof by any means (though reasonably effective against "bulletproof" hosting providers), but it massively reduces attack service.
I've been thinking a lot about networks, size and scale, and the corresponding levels of abuse. To a rough approximation, the Internet grew by an order of magnitude from 1969 to 1970 (1 to 10 nodes), 1977, 1985, 1987, 1989, 1993, 1995, 2000, and 2012 (1 billion nodes).
The "Linux Sucks" guy (multiple presos at Linux conferences on the state of breakage within Linux) did a preso recently on the IoT, noting that we're going to be looking at roughly 80 billion nodes by 2020. I'm wondering what that will do to various forms of abuse.
In the 1970s there was Phreaking and John "Captain Crunch" Draper. In the 1980s, the first boot-sector viruses (Brain: 1986) and the Morris Worm (1988). War dialing was a thing.
Through the 1990s, there was Usenet spam, first jokes like Make Money Fast, then Green Card. With the spread of the WWW, the first banner ads, pop-ups, and click fraud.
2000s: adware and spyware (bug-for-bug compatible reimplimentation on Android now), ID theft, high-profile viruses (Nimda, Code Red, Welchia, Slammer...), DDoS, and Phishing.
In the 2010s, click fraud and various forms of doxxing and pranking lead to wholsesale attacks on liberal democracy itself, with A/B tested bots and informational attacks.
I'm looking for a concept of network scale and problems encountered out of this.
It would be nice for mobile browsers to support a blacklist/reporting of malicious ads. In particular I'm referring to the type that hijack the browser through redirects, preventing you from returning to the source page. I see these alot originating from major news sites. They're usually in the form of "Complete this 3 question Facebook survey to win an iPhone" or "Your device is infected!" The people originating these ought to be arrested for wire fraud.
and a community driven list of good advertisers that are transparent in how they deal with our information and take the quality of the adverts seriously.
https://github.com/pi-hole/pi-hole/blob/master/adlists.defau...
There's a few on there I don't use and will look to implement. There's also a few they seem to have missed (perhaps intentionally?) so below I have included the lists I use in case it's useful for anyone else: