t145 / black-mirror Goto Github PK

Blacklists and whitelists built by open code, so you know what goes into them.

License: GNU Affero General Public License v3.0

Shell 61.36% Dockerfile 27.20% Perl 11.44%

adblock-list blackhole-lists blacklist blacklist-data blacklist-domains blacklist-feed blacklist-ips blacklist-management blacklist-script blacklist-service blocklist daily-build dns privacy whitelist whitelists whitelists-hosts whitelists-ip

black-mirror's Introduction

-----BEGIN PGP PUBLIC KEY BLOCK----- Version: Keybase OpenPGP v2.0.76 Comment: https://keybase.io/crypto xsFNBGW+cAMBEACrJZFdqsZPOpMqxI8q5OPu4Bpp8ImUz1Ez7LqaduZzn5TmpDjy mwYy9EsVUwBcjGzgmZLYR4huV3uy+MPZNjcEsk/U4b8ve8p8cWz3xGZFvmYSd0rl mxrdR+5s9PG0K1P98pMo5jMz3llyrD3cocsoMAyc8sczwZbPJaPyuIsHBZjJZX/Y zHa7dkLRV9VrJCfxGmvm6qTeG+h00TW0r58hbGsn7kVIg2p2eFgBpGZ64C4YweLI I0YkBbdBG9hZIg/lJkh5BpT56GI3bQQI9IGIttM1lHv8T9EXZLgsGxBdt/q6j5L4 pSMy1BHB7zsheChuyYWi77tQdWKQUpb5eEzMLWYfznFKqX1m2vKsJaSmACzBzqKl 5G0Tx2clZ2obI4tE7I7oP2e1QZ+U85H/lKhztMznHjMyl16PhBUGdFBB/g1IWqHx KSR5+lTxQ5DLbrovcfV9b8GTYvxMz/CDWrbiBaL6fcM/MCsGPD01dxxqGCHdQzQP 7IDWEN8pSxOkw94obHSiODKYELiArjrRQtqf4tpTPULKxTfZR2bk3Lm6ye0IHSIE pkhoq4RIOv0yYwR6dtdYLY+Eb7G8eAlMKTCD+0Hte7n+MwD7u2cFmXxI4ru035ZN fizpf6W+bVcFSHZs/+sSZtKZc1tH0yTv79z6nB7fV91PbfFvoJipqg/TLQARAQAB zSNUYXlsb3IgU2h1bGVyIDxUMTQ1QHByb3Rvbm1haWwuY29tPsLBegQTAQoAJAUC Zb5wAwIbLwMLCQcDFQoIAh4BAheAAxYCAQIZAQUJAAAAAAAKCRBimS470qtAhr51 D/9rqiVW2pdBnD98L4yI5nIjC+JE1apa8FWkW9OCwn9WrOO+yg+ApFN3MJ+nDXAI 6hbHrmE2SolO5BDVdd2QcanNQGFc1xo8k50KsL4e7+GbeMBD2GVZWJL5fUz8EvHC 6j1Ph8Q2qnS1Y44iZKIDcUgx9WCmhjwJPuLnbw4AOBdSMUdhuSigKrdXl/zgo+ot AdvbGNDOjzvTeHYNEsVV3rndI2094g74/PQ9tJPFWeUXYKBIWpAbyGxe0E2KmAbe x5k4g2MHNpwbKfqxsLsBawMUQ5kh7nEEhJ+l0Ok59Sz0rmx/O39OwC3288sG3vt6 EW27LvEH7f2Z/Ce8SvQPwVgndAt49VzGesojTyYBfeOTfQGKqmqghO86JAHhZoFI ox1siX5ooHQePHxPwdbXpib9HnQ4Yj+aI7trG+7T082oPQP+xbgkKESb5Ij2RzB3 1mHIlAGIq5AP13KZiVHZOcxYh8chLjjRCAY8oO05b90ZIHpb1dKM1obYMTpBXwA1 NAmIsH42IjQ7pWh0a1AFApy3h9zigmgy9glXr+bJM1aW6pRoJmRqTnb8xfCdaseW omRnWkScad1YnZJfFCI7Kazwk1EvzpEugyS+Trkzrpb/VNnEk2vZNmu5T71Zez4i gRjwJs9sZSceDhwl7f/iGn2yRB21Z1U3n3yHapnaxTwoRs7BTQRlvnADARAAn5kI 2x8n0KEbMtE8JsRq8F19cNljH0ecIBr0U52e6xsIXBUo2X1VAHHMBPhePXGrlxI+ +5niC6im13nFz3AD2R/e+HoEDSs6GljV3kDk+Vko9YJfIaNCiu8aq3dTknqeBoq5 /My1JKWRq+EJJyW9NLzHbNYKRAK9/93ZdeIkIMzvc8GqJCV2G6bv7MFEP8NujVtS vlAoICSdv1wi9Y+o5PyARVedv+dkQGniIN1973+WcwSGngu6mmW5VeNzaDvUhaHY OYGHdmaomyYtk9UKEj7x5RkI8f2rcdbY/EMPeYcC4dHldGuXV87vUOPG8FBPd7kd A1f2zwgbE1j49NvmiiBy7cDZeLIy53tpXFTru30898hqT1BRGGea7NiwmHpU24Sa kdDJORRYSm5m4bMYw7nKfN9w9rj9aI1/k158arfzjCUnCaxdXxEJEv6P164GikUW nE2P4aX1dBPFu7hu7webiLnzqFqgANg7/yiVIwsJ9yg/o/LMS7njQLygDM0KNyo2 klsEA2YrFLf9gWgfhsQRphcRtM66Nt711X2v0Ia3CoC7EK69OztGIwVL9bMAg9MH PzEWvq8j4osQTLOtBcK4+0LGVlpQ1FwnUR7zYb9OovNfsgh+5IezIvQV3ft0HOVe lhyioS7V2x1837V+TU6GyXTVsbL25mnrSW6m0N8AEQEAAcLDhAQYAQoADwUCZb5w AwUJAAAAAAIbLgIpCRBimS470qtAhsFdIAQZAQoABgUCZb5wAwAKCRCNXaWUamWW ny42D/9OHrpbX1x46McAcBmzvX1x/rB0+Ldhgo4o10GlWJFm/ljzjwGzk69G4yxR FT1TxHVBfaCL+q/fZsJ7t7Rn+IgtXnOAJ/3KzygwMoTtx43YtuNGbqJsZe25O4rM sNPoE5YKb6vkQ/NxPQjvyMLbflXeF6sgq8T7RqZa0EdUK0lrISje3gPWeHISef0N nN6rk8IZ6ebYwEwXbi4ldhp9r8KsHc8KXLtOTd81MoOtmwVyLJwYDoJSA0nPrFaU XNMUijb88XIwueZP6FHo5LqE2qSge19hptR/LU7kgEtRV1+OrBshOHS7zHlM1AQc fukFnsS5PTrdSi8MWj5g8I7lL9XdW0cWkY9vxcV6/KZFr3vx+4oy0m2nhTvEfXvr 9HilZ9Xv8Kn4OQYuED/+k8yxeueBRGe+X7vCoQbHi14aly+OFojeyS49u5+2BeYw o0J/bCLagSTyoZ0fz1u5AfWjo6lyuJ9bHcsDzANA1LRczvjPjq/RVt/6fCJkPhYN VioLttuHMq0Pk84jffxQyM8aICIorc+bpyKs60jywUh8KTgs8GPkeptUzJlPXQZF EQeHxspFGSqVlATqd+5Vb+JxA9uwBVhxg+WPuu2ql+8WN4i+MunA6YKKbVqkqCup mMYnhdAtrqsgZHorc6afjg2t/aW1afrtsWTt3hokTd06p3AsU6HCD/9fY5Ch4GMj VlsMNimvfEnLMab1wIc/bOI6hBL9pmCqJgTMdwhBKeN0hsr2Cv9hfjccxfTiK8QP 1VV4DK+6Ir1EgTFU2sTNtBR6H6WdV5KgqJHfSVFvAv1xWRL1IKAcJJDzPrAMGQB1 deodtCltobHbWn+bvSyvL3PvXZ7xrepYH1SbBHXwtJsRTn8E5aap6uri0sBEiuGL K6y3YD+WkgAxiopJ/p6PrO7tPjIFjrA0oyJwVErjfQYbBeNCHVXWqD0fQ3mREkn8 oG+TNWalM7tnJvrkJCAlqalYp1U6lJrdsVn2PqUKLRJOa01YhsN0HvwgpbojM/FI Fp1kPxeR0WiagCLqUp2NpYoqNhQwVGu4Izdx35tzeRxsdbN7AbMC7ZfCBysSGKVT p+CVKlQ+Zms+FrOhL8uv3/hHBwYkgneT6Nnt1kvY/1VVNAfHwud/Obhsg65BeUvz NFy8bupxeaf9zPXsMRi339BLwGXL9nEborRBTCQrIsRU4aRp4oVRxMadsSF9gCFR v2I9xqmB6mo7I9ZZ0QNCdC0Xdf9n8A9hqUP6jM/O18SnC6lRHwdbCRp8qpecTIqS V7/KTCFxiF5IHn/xTDjBsQ/EXjrvbPAVKkmBIZ/esPYUreaTvEcSG1uplmLTPWIE DbWc8jeLY7nS/cmeMDsSNlcVJd+EsZXiqs7BTQRlvnADARAAz1BXvCUSpq2mgQ6Q 1A+SJg4j5IHTQcak0eX3HZU9hmBDlOw1k/ao/4IZoTi213OC0xrQEQWf8Z3k//SE 2ysuUdhZbQuY/8cpGUmI0HuvQdDAZxCueYRcs+r509rOUPeS7QwZ/AJBBvEKfZ/C troD7/rmjD8QMpNhInT8Unzb8GgRAm7f34FbwIpKmPcfyB5weDxGIazVmYrw3E1g 6OxBz3p/T2DCnFu0NH6vN7+5nZz6SF6Lzr+OMG6ODbGDyMSDbbDiQmSeqB5BICle aDf9EOlBA0C71/BzTKX4+oNcqYvmeI+q4MkoQpf5SR9Y1pWynx3H4BMPZsIYVgE+ hsdyR4YD7KkvCvmRhaaivEveBqdT4at08+vi0jUU92eh98aFznHMIg/vu42Rf52W nUmS3vOUO+Qc46YwOpv8JrSATfZUro1lwOun4jIv7RukEy26jBPRoIZtrb50KI2Z IBNnphTy2QAO/hQICMeENp2nqbyJqHw3Ct1DNyjjV/ZwG0JKDgOzUiHVmpurgF1S pexg4oBV6TQWHVh+w51ITCyPMDOYF6aVX1r7UjDef1d1VGOg/JqQaB4RyoAy3NHW UZjmYfOhtw7tCwFVmbIvhtYgnftz76Vd8e96ew78+ktIQivu3arK1Xb69URux5gN +Y5OXzmYDxpAYw+wSjtv1rbs+KcAEQEAAcLDhAQYAQoADwUCZb5wAwUJAAAAAAIb LgIpCRBimS470qtAhsFdIAQZAQoABgUCZb5wAwAKCRAEOTbePVQsctLxD/oCkoF2 O007LwcoWCWoFYsv3PJlPUm5iuyt/P5+LW7YC6n5N563ZaFxTPZvYOzuS8z8c1YN Y81c4iciaKvud8PBuREwD7lm0N6v4GcVI4Taz3ITdMFnoNY2K+dns0op3g4rsspd r4APsCiq9Qog5GFSSIE4j4TxNS7otWmwATs4R7PvjJomo2E7s9sslGXS0d3npIo3 +AMsgrVDiSgopKerY8uE1sUnIx+N3EsaQnXoL5+83c0mNxYZIFPEKWZfCJuE3P6H uAifbwwMAclYjL56TfoCGOVaKRHcVkeoNkhaDVeK4KXAF1J6dBn5Xh+WMqVfWXCj iwD2IepXAQ369UYf3gj9DT+7cxbfBvjl+OnU6wd9Trzk+Ax+4xR0yX9zRN9o1bWl NclQbh+nsJk0C+coMwDGhkpQJS7gvIDumdIleZf/ms24+lH/N8WC6VouLfD2oIZN QOGGoVqaGQDnoFe3yeiZWNIA7PE/KEe5JoDMwY6tF+1Fvrp3MpCvTNr0E5zKNBNL w0oXMzLNIuZXo0a8nORHnHCgo8oTczl88n9vUBI6yevRyCSd2uKNRPl/xO0rjj/h jF9lFDqbL95Rbi4n9y6jv7H04WG9GcU4uotx/0Z+HchBh5ZvF6dmC9vmDBt/Atfu OJAngHCuHE7ZA/T/d1gbfrhmeofhOgIgN/lCmBY2D/98yMyVE+sLDrsL1Se3m6aF 3E0eEEk/Z42Pj/TYt5YWdFZDdw6rrXog7+kt/TYNuaWPyaLXcwVHa98CGQ6e6zzT EXAhtfgWqBnt4r0B6psxbF6MnyGLpmmzywuEdxQBSk/DboE7zHPhLoCoF2KPbmPt 6q6VyxdXpWm8lGgvQR+gYjbV3y66MtArnwhEZEVmbVwhq3kYMO2xbJnjNUy4bJsJ KsXQBOBnAs32pJQ4kHLUeTbK9rEQhRNqNfBfdpupFOZJsAXbgCI9hng+Gl08jaaE uURUozgPK11mt5eP/KUbAl/O/EZJEexUC7JbqH3+xgsqD1vMWMAXkQ/OrmkQugO7 +Cc50maAsRUu3y0sWFZCiBVCIvhEYnAZinOAKPFheUmhgWMC5AL7tisPYx94GCDy xBUlii9zWXHwjiTcBPdqpDbqq3E0c5sTJKZOa3s2hKGSWBqv+WBUvX5hgY3cS62D rFtzlSAFczLo9x4l7QF0974gm1OpscMwSH2ZKjJuOQPOUvTy5JlhhZEuqc1BTaAf ate/6e18cg52Z7S/7SL61VN2/fiJ2evz37MbmqKOZ2FV1PXIHvpzX0UkBfEEIY2W J83snOy2GqJgP4OEW8I+PoalV6To8mjAZJRA9sRmsCVeAYeeUq64fsXVUAcOBQ32 TDFkpv3jd0Zn02MzlUu+aA== =x3ML -----END PGP PUBLIC KEY BLOCK-----

black-mirror's People

Contributors

Stargazers

Watchers

Forkers

mitchellkrogza opus-x trinib te-k webdienste johnbatts418 saas786 nicksondha shawnhank shazuwan96

black-mirror's Issues

[HOSTS] Add badmojr's 1Hosts takeover

https://github.com/badmojr/1Hosts#personaldnsfilter

[HOSTS] Add IPv4 and IPv6 addresses from IPverse

http://ipverse.net/

Link Checker Report

Errors were reported while checking the availability of links.

Full Github Actions output

Parse the betterfyi blockerList

https://better.fyi/blockerList.json

[feature]: Create a JSON schema file for sources.json

https://json-schema.org/

[FEATURE] Have separate IPv4 & IPv6 list versions

The null-route on IPv4 is 0.0.0.0, and the null-route on IPv6 is ::.

Here I'm using the term "null-route," but it's also referred to as a "black hole" address.

https://stackoverflow.com/questions/40189084/what-is-ipv6-for-localhost-and-0-0-0-0

Can be used to:

Blacklist porn sites
Whitelist common websites

[HOSTS] Add unused sources from the Ultimate Hosts Blacklist

https://github.com/Ultimate-Hosts-Blacklist/Ultimate.Hosts.Blacklist#information-data-sources-copyrights-and-credits

Create a separate IPv4 CIDR blocks release

Split individual addresses and CIDR blocks into separate builds. There will be a post-check that will determine if any IPs fall in CIDR blocks and remove them.

Track down Windscribe's Social & Cryptominers lists

Through some simple guesswork I was able to get all but Windscribe's social and cryptominers lists from their assets URL.
They'll be somewhere at: https://assets.windscribe.com/custom_blocklists/
Any help is appreciated!

Remove erroring sources in dnsmasq

köln-düsseldorfer-rhein-main.de
sendesık.com
sushicorner-würzburg.de

Probably just b/c their unicode I'm guessing.

Make a separate blacklist users can update and maintain with their own hosts

@EnergizedProtection/block Issue #728

Shrink the IPv4 build by grouping similar IPs into CIDR netblocks

IPverse uses this technique, and it can reduce the IPv4 build's size w/out having any noticeable impact.

[HOSTS] Blackbird's Windows hosts

https://getblackbird.net/blacklist/hosts.txt

I'd also encourage using the main application! 😉

Expand IPv4 CIDR blocks and IPv6 addresses so they can be handled properly

Specific IPs need be listed in order for blacklisting and whitelisting to work across all platforms. This should help reduce some more ineffective host entries. May tie into #29.

Add a comprehensive build in the *.deny format for Unix systems

This would just be the IPv4 list w/ ALL: prepended instead of 0.0.0.0.
https://linux.die.net/man/5/hosts.deny

Add Abuse.ch sources

https://abuse.ch/#about

Find a way to handle multiformat lists

Some lists have IPv4 addresses mixed with domains, others have have domains that are prepended with 127.0.0.1 alongside regular domains, etc.

Solution: Replace "rule" with a "filters" array. For each filter, apply it to the file and use the output.

New Solution: Create a map with a format as the key and a filter rule as the value. Then there can be varying operations done on a source while storing the results on a proper list.

[FEATURE] Use GitHub Actions to publish an archive release

This will help remove the massive text files from the repo, allowing cloning and small changes to not be wearisome.

https://github.com/marketplace/actions/publish-release

Use Public DNS Info as an IPv4 and IPv6 address whitelist

https://public-dns.info/

Update the README to show the number of blocked domains on each build

markdown README.md | tidy -f errs.txt -i -asxhtml | xmlstarlet ed --update '//td[@id="domain_size"]' --value '123'

        <!-- small box -->
        <div class="small-box bg-red no-user-select" title="Blocking list updated 01:54 (hh:mm) ago">
            <div class="inner">
                <p>Domains on Blocklist</p>
                <h3 class="statistic"><span id="domains_being_blocked" class="">82,723</span></h3>
            </div>
            <div class="icon">
                <svg class="svg-inline--fa fa-list-alt fa-w-16" aria-hidden="true" focusable="false" data-prefix="fas" data-icon="list-alt" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" data-fa-i2svg=""><path fill="currentColor" d="M464 480H48c-26.51 0-48-21.49-48-48V80c0-26.51 21.49-48 48-48h416c26.51 0 48 21.49 48 48v352c0 26.51-21.49 48-48 48zM128 120c-22.091 0-40 17.909-40 40s17.909 40 40 40 40-17.909 40-40-17.909-40-40-40zm0 96c-22.091 0-40 17.909-40 40s17.909 40 40 40 40-17.909 40-40-17.909-40-40-40zm0 96c-22.091 0-40 17.909-40 40s17.909 40 40 40 40-17.909 40-40-17.909-40-40-40zm288-136v-32c0-6.627-5.373-12-12-12H204c-6.627 0-12 5.373-12 12v32c0 6.627 5.373 12 12 12h200c6.627 0 12-5.373 12-12zm0 96v-32c0-6.627-5.373-12-12-12H204c-6.627 0-12 5.373-12 12v32c0 6.627 5.373 12 12 12h200c6.627 0 12-5.373 12-12zm0 96v-32c0-6.627-5.373-12-12-12H204c-6.627 0-12 5.373-12 12v32c0 6.627 5.373 12 12 12h200c6.627 0 12-5.373 12-12z"></path></svg><!-- <i class="fas fa-list-alt"></i> Font Awesome fontawesome.com -->
            </div>
        </div>

.small-box span {
  transition-duration: 0.5s;
}

.small-box span.glow {
  text-shadow: 0 0 5px currentColor;
}

.box,
.small-box {
  border-radius: 10px;
}

.bg-red {
  background-color: #913225 !important;
}

.no-user-select {
  -webkit-user-select: none;
  -moz-user-select: none;
  -ms-user-select: none;
  user-select: none;
}

https://github.com/pi-hole/AdminLTE/blob/master/img/donate.gif

Use Tranco 1M over Alexa 1M?

https://tranco-list.eu/

[BUG] Fix broken sources

  "openphish_feed": {
    "rule": "BEGIN{FS=\"/\"}/^http[s]?:\\/\\/([[:alnum:]_-]{1,63}\\.)+[[:alpha:]]+(\\/|$)/{print tolower($3)}",
    "url": "https://openphish.com/feed.txt"
  },
  "energized_ips": {
    "rule": "/^([[:alnum:]_-]{1,63}\\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($1)}",
    "url": "https://block.energized.pro/extensions/ips/formats/list.txt"
  },
  "reg_fi": {
    "rule": "BEGIN{FS=\"[|^]\"}/^\\|\\|([[:alnum:]_-]{1,63}\\.)+[[:alpha:]]+\\^(\\$third-party)?$/{print tolower($3)}",
    "url": "https://raw.githubusercontent.com/finnish-easylist-addition/finnish-easylist-addition/master/Finland_adb.txt"
  },
  "reg_id": {
    "rule": "BEGIN{FS=\"[|^]\"}/^\\|\\|([[:alnum:]_-]{1,63}\\.)+[[:alpha:]]+\\^(\\$third-party)?$/{print tolower($3)}",
    "url": "https://easylist-downloads.adblockplus.org/abpindo+easylist.txt"
  },
  "reg_kr": {
    "rule": "BEGIN{FS=\"[|^]\"}/^\\|\\|([[:alnum:]_-]{1,63}\\.)+[[:alpha:]]+\\^(\\$third-party)?$/{print tolower($3)}",
    "url": "https://raw.githubusercontent.com/List-KR/List-KR/master/filter.txt"
  },

Removed for now!

Add unique Sheriff53 sources

https://notabug.org/phronimon/Sheriff53/src/master/docs/THIRD_PARTY_LISTS.md
https://notabug.org/phronimon/sheriff53blocklists

Investigate threatcrowd's weird files

Found these while searching through their website:
https://threatcrowd.org/feeds/hardcore.txt - 520
https://threatcrowd.org/feeds/mar3launch.txt - 524
https://threatcrowd.org/inc/blacklist.txt - 200

https://assets.windscribe.com/extension/ws/malwaredomains.txt

Originally reported:
badmojr/1Hosts#171 (comment)

Hope you find it useful.
Cheers!

Regularly scan every source to determine if any are fully redundant, and create an issue reporting the analytics

If any sources are found to be fully redundant, they can be manually removed.

[FEATURE] Support file mirrors

aria2c has the option to download from mirrors as either a fallback or to increase download speeds. The technical feasability is quite probable.

https://aria2.github.io/manual/en/html/aria2c.html?highlight=mirror

Investigate I-BlockList

https://www.iblocklist.com/lists

ut-capitole contains improperly formatted 0.0.0.0 domains

This source clearly scrapes from other sources improperly.
Hosts listed as 0.0.0.0* are likely 0.0.0.0 * in their original source.

Other rules may be needed to extract the ill-formatted domains successfully.

Add unique pDNSf sources

From pDNSf:

Investigate scafroglia93's sources

https://github.com/scafroglia93/blocklists

Includes: https://github.com/pan-unit42/iocs

List Pihole-compatible sources in a single-line text file

Based off my idea in #42; tested by @pallebone.

jq -rj 'to_entries[] | select(.value.color == "black") | .value.mirrors | join (" "), " "'

Make a release step to print mirrors[0] for each source that contains a single domain filter using the mawk engine.

Be able to include whitelist sources alongside blacklist sources

https://github.com/EnergizedProtection/unblock#packs-1
https://github.com/anudeepND/whitelist#overview

Add unused sources from hl2guide's filterlist

https://raw.githubusercontent.com/hl2guide/Filterlist-for-AdGuard/master/filter_list_URLs.txt

Fix dnsmasq bad domain name errors by converting offenders to Punycode format

https://github.com/maravento/blackweb#debugging-punycode-idn

Offenders:

дольщикиспб.рф
шляхтен.рф
สารสกัดจากสมุนไพร.com

Add unique, active sources reported from 2016-2018

Need direct communication and granted access:

Found:

https://www.projecthoneypot.org/list_of_ips.php
https://www.dnsbl.info/dnsbl-list.php -- need to verify listed sources
- http://rsync-mirrors.uceprotect.net/ (Need to track down the whitelist http://www.whitelisted.org/?go=usage)

[FIX] Build around the Energized list to reduce StevenBlack redundancy

Extract each unique StevenBlack source so there's not much overlap. This should help reduce build time.

Add unique sources and maybe some mirrors from MISP's feeds

https://www.misp-project.org/feeds/

Requires importing

https://www.circl.lu/doc/misp/feed-osint/

Generic

IPv4/IPv6

https://home.nuug.no/~peter/pop3gropers.txt

IPv4

[HOSTS] Windows Spy Blocker extra and update lists

https://github.com/crazy-max/WindowsSpyBlocker/tree/master/data/hosts

[HOSTS] EnergizedProtection Extensions

https://github.com/EnergizedProtection/block#extensions-1

There are some sources listed in the extension descriptions that aren't included in the main pack descriptions. The Xtreme, Social, Regional, and IPs extensions all have unique hosts. The first two may require some whitelisting.

Add unique FireHOL lists to the IPv4 list

This will mean that raw IP addresses will not be included in the domains list.

https://github.com/firehol/blocklist-ipsets

Use dnsx to determine if associated hostnames have addresses

Due to some infrequent source updates and malicious domains getting taken down, it may be worth while to double-check every now and then.

https://github.com/projectdiscovery/dnsx

Add unique and found blocklist-tools sources

https://blocklist-tools.developerdan.com/blocklists

Interesting:

Found:

http://rlwpx.free.fr/WPFF/hosts.htm

"rule": "/^0\\.0\\.0\\.0[[:space:]]+([[:alnum:]_-]{1,63}\\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($2)}"

While the no IP version can use this rule:

"rule": "/^([[:alnum:]_-]{1,63}\\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($1)}"

The no IP version is likely best b/c its rule is a subset of the prior rule.

Will likely need to implement a whitelist to handle the Crypto, Facebook, and Torrent lists.

Use lychee to check for any dead source or README links

https://github.com/lycheeverse/lychee