Git Product home page Git Product logo

my-pihole-blocklists's Introduction

my-pihole-blocklists

Create custom pi-hole blocklists to block unwanted traffic to a network.

Disclaimer

This project is in no way affiliated with the core Pi-Hole project or organization. This project was created as a contribution to the community by providing high quality blocklists. Use at your own risk.

Screenshots

Pi-hole dashboard with the default, light, and heavy porn blocklists installed;

Running with default ruleset Top 1m (Light List) All (Heavy List)
default blocklist light blocklist heavy blocklist

And the affect of having the porn list installed;

Site Blocked!

Supported lists

Currently supported lists;

  1. porn (via create_blocklist_porn.pl which creates two lists)
    ... light list: pi_blocklist_porn_top1m.list (~21k domains). This list is a correlated list to only block porn sites that appear on Alex'a top 1m site list.
    ... heavy list: pi_blocklist_porn_all.list (~2m domains). This list is a slightly edited list from Université Toulouse 1 Capitole. NOTE: Use caution when using this list with embedded computers.
  2. more coming soon...

Requirements

  • Perl
  • Array::Utils (install via sudo perl -MCPAN -e 'install Array::Utils' aslo you may needed to use export PERL5LIB= after installing Array::Utils to add path to the library)

Installation

  1. SSH to deployed pi-hole
  2. Copy adlists.default to adlists.list sudo cp /etc/pihole/adlists.default /etc/pihole/adlists.list
  3. Edit sudo vi /etc/pihole/adlists.list to add one of the following lists;
    ... Light: https://raw.githubusercontent.com/chadmayfield/pihole-blocklists/master/lists/pi_blocklist_porn_top1m.list
    ... Heavy https://raw.githubusercontent.com/chadmayfield/pihole-blocklists/master/lists/pi_blocklist_porn_all.list
  4. Run pihole -g to update all lists
pi¾pi-hole:´ $ pihole -g
:::
::: Neutrino emissions detected...
:::
::: Pulling source lists into range... done!
:::
::: Getting raw.githubusercontent.com list... done
:::   Status: Success (OK)
:::   List updated, transport successful!
::: Getting mirror1.malwaredomains.com list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting sysctl.org list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting zeustracker.abuse.ch list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting s3.amazonaws.com list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting s3.amazonaws.com list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting hosts-file.net list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: 
::: Aggregating list of domains... done!
::: Formatting list of domains to remove comments.... done!
::: 128651 domains being pulled in by gravity...
::: Removing duplicate domains.... done!
::: 105011 unique domains trapped in the event horizon.
:::
::: Adding adlist sources to the whitelist... done!
::: Whitelisting 9 domains... done!
::: Nothing to blacklist!
::: No wildcards used!
::: Formatting domains into a HOSTS file... done!
:::
::: Cleaning up un-needed files... done!
:::
::: Refresh lists in dnsmasq... done!
::: DNS service is running
::: Pi-hole blocking is Enabled

Sample blocklist creation ouput

[chad@myhost:~] $ ./create_blocklist_porn.pl 
beginning downloads
download complete: top-1m.csv.zip
download complete: adult.tar.gz
extracting files
extracting file: top-1m.csv.zip
  loading alexa top-1m domains
  loaded successfully
extracting file: adult.tar.gz
  loading domains from adult.tar.gz
  loaded successfully
comparing lists for commonality
counting lines...done
*******************************************************************
Light blocklist created: pi_blocklist_porn_top1m.list (21492 lines)
Heavy blocklist created: pi_blocklist_porn_all.list (1980211 lines)
*******************************************************************

NOTE: This is a work in progress and is being actively developed (for now).

TODO

  • Cleanup the code! (most of this was written as PoC)
  • Log output from script to file rather than STDOUT
  • Add comparison to Alexa top 1 million sites to only block the top sites
  • Add cron script to run from cron daily/weekly/monthly
  • Add installation script for automated install
  • Add additional lists to combine into one mega list
  • Possibly generizie script to accept any list
  • Add instructions to host the list on the pi-host itself
  • Add the Cisco Umbrella OpenDNS Top 1 Million list
  • Add the Majestic Million list
  • Add the Stavoo Top 1 Million list

my-pihole-blocklists's People

Contributors

chadmayfield avatar mcleut avatar souhaiebtar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

my-pihole-blocklists's Issues

porn subdomains are not blocked

Hi Chad,

Since the pi-hole team told me that wildcard blocking was not working with lists, I open an issue just in case you find time to fix your huge list... (https://discourse.pi-hole.net/t/subdomains-not-being-blocked-v3-1-4/4787/3)

On my pi-hole (V3.1.4) the URL like "porn-domain.com" is blocked but if I enter "www.porn-domain.com" the page loads.
The pi-hole log confirms this :
Sep 8 23:29:25 dnsmasq[4743]: query[A] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: /etc/pihole/gravity.list myfreecams.com is 192.168.50.55
Sep 8 23:29:25 dnsmasq[4743]: query[A] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: /etc/pihole/gravity.list myfreecams.com is 192.168.50.55
Sep 8 23:29:25 dnsmasq[4743]: query[AAAA] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: forwarded myfreecams.com to 192.168.50.1
Sep 8 23:29:25 dnsmasq[4743]: query[AAAA] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: forwarded myfreecams.com to 192.168.50.1
Sep 8 23:29:25 dnsmasq[4743]: reply myfreecams.com is NODATA-IPv6
Sep 8 23:29:30 dnsmasq[4743]: query[A] www.myfreecams.com from 192.168.50.131
Sep 8 23:29:30 dnsmasq[4743]: forwarded www.myfreecams.com to 192.168.50.1
Sep 8 23:29:30 dnsmasq[4743]: reply www.myfreecams.com is 207.229.73.118
Sep 8 23:29:30 dnsmasq[4743]: reply www.myfreecams.com is 207.229.73.117
Sep 8 23:29:30 dnsmasq[4743]: query[A] www.myfreecams.com from 192.168.50.131
Sep 8 23:29:30 dnsmasq[4743]: cached www.myfreecams.com is 207.229.73.117
Sep 8 23:29:30 dnsmasq[4743]: cached www.myfreecams.com is 207.229.73.118

So perhaps doubling the number of entries by adding www. on top of every domain would do the trick. But this would lead to a tremendously huge list...

Hi

I have added the file in the adlist, it shows 1800000 domains blocked, but it is not working.

pronhub is going through when I check the logs.. its like it is not using the list?

Any idea?

fasthosts.co.uk - false positive

Hi, thanks for your great work on these lists.

Can you please consider adding fasthosts.co.uk and www.fasthosts.co.uk to your whitelist please as it's a regular web host and domain provider.

Regards

Lists need cleanup parse-wise

I was trying to troubleshoot an issue with my pihole-FTL daemon crashing about 1x/day. I ran across this article and took a look inside of my blocklists. Lo and behold, there it was:

https://i.imgur.com/5KwyxWH.png

I'm going to keep, but disable, your "all porn" blocklist for now. I'd edit the list myself, but a subsequent update would put them back.

Yes, I realize you made the list(s) as a PoC. I just thought I'd bring this to your attention.

yourbrainonporn.com - Probably false positive

I don't know exactly which site you want to block, only those with explicit content or event those that talk about porn in general.

But let me point out that the site yourbrainonporn.com site is in your blocklist. This scientific site talks about the effects of porn.

Maybe it should be removed, consider yourself.

website remove

Hi,
the website solotodo . com is not a porn site. It´s included in the following lists:

pi_blocklist_porn_all.list
pi_blocklist_porn_top1m.list

Thanks.

Remove domain from blocklist

Hello,

I noticed you have www.pbteen.com and pbteen.com added to the pi_blocklist_porn_top1m list. pbteen.com is actually PotteryBarn. Could you please remove it from your blocklist?

Thanks!

Non-domains in list

Your list has 4 lines of extra content (not devastating):

  • adult
  • porn
  • sex
  • xxx

I assume these are just categories

I'm not familiar with perl enough to help, but my parsing script uses sed to remove lines that don't meet FQDN requirements.

A valid domain name has :

  • at least one letter
  • at least one period
  • no symbol other than a period, underscore, or dash

I'm assuming that this is a small addon to your script to parse them out.

BTW,, people really like your lists, and your lists are featured on
https://wally3k.github.io/

Thanks again for such a great list!

Non-porn domains blocked

I use both your lists with my pihole and noticed it was blocking a number of non-porn sites.

I was looking for something cute for my wife and noticed that Victoria Secret is blocked, it sells lingerie/underwear, it's not porn. Though, I can understand why some people would it blocked, as it can have provocative images. This got me curious, so with the help of google, I have a list of lingerie/underwear type sites that your list is blocking. My list is likely incomplete.

If you don't want to unblock them that is fine with me just close the issue, but I figured you should know.

victoriasecret.com
www.victoriasecret.com
fredericks.com
www.fredericks.com
barenecessities.com
www.barenecessities.com
yandy.com
www.yandy.com
lingeriediva.com
www.lingeriediva.com
lasenza.com
www.lasenza.com
amiclubwear.com
www.amiclubwear.com
forplaycatalog.com
www.forplaycatalog.com
spicylingerie.com
www.spicylingerie.com
dear-lover.com
www.dear-lover.com
hipsandcurves.com
www.hipsandcurves.com
smartandsexy.com
www.smartandsexy.com
herroom.com
www.herroom.com
petitecherry.com
www.petitecherry.com
torrid.com
www.torrid.com

This one is a magazine, I'm pretty sure it isn't even adult focused
( https://en.wikipedia.org/wiki/Refinery29 )
refinery29.com
www.refinery29.com

Doordash.com

Hi,

I've been using your blocklists for a while and they're great! I just noticed that recently doordash.com was added to the blocklist. Doordash isn't porn, it's a food delivery service. I've added a whitelist on my Pi-hole for the time being but I'd figure I ask why/how it was blocked in the first place.

Thanks!

Just cant get script to run past line 12

Error I get is:
Can't locate Archive/Zip.pm in @inc (you may need to install the Archive::Zip module) (@inc contains: /home/pi/git/my-pihole-blocklists/ /etc/perl /usr/local/lib/arm-linux-gnueabihf/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/arm-linux-gnueabihf/perl5/5.28 /usr/share/perl5 /usr/lib/arm-linux-gnueabihf/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/arm-linux-gnueabihf/perl-base) at ./create_blocklist_porn.pl line 12.
BEGIN failed--compilation aborted at ./create_blocklist_porn.pl line 12.

I followed the below steps as best as I understood them. No issues installing Array::Utils. Not sure I understand the second step around 'export' in any case the script stops at trying to import Archive::Zip

Array::Utils (install via sudo perl -MCPAN -e 'install Array::Utils' aslo you may needed to use export PERL5LIB= after installing Array::Utils to add path to the library)

False Positive found

I have found a false positive. It's a news website. ( https://en.wikipedia.org/wiki/Heavy.com )

heavy.com 

I assume this is due to your upstream sources, if you don't want me to report false positives I find please let me know and I'll just whitelist things on my side.

Thanks so much for maintaining this list.

How to add support for Pi-hole 5 and some feedback for updating adlists

Hi,

I came around your project just wanted to leave a comment on these To-Dos from your README.md:

Add cron script to run from cron daily/weekly/monthly
Add additional lists to combine into one mega list
Possibly generizie script to accept any list

  1. I've written an auto updater to run daily with cronjob. Check it out here: https://github.com/JavanXD/ya-pihole-list#updating-your-adlists-automatically

  2. You don't need to combine the lists beause the gravity.sh from Pi-hole will do that for you. You should use it, because the block.list.txt will not longer be used. Pi-holes stores the adlists and the blocked domains in the sqlite3 Gravity-DB.

  3. The gravity.sh script can detect regex and other wildcards. It removes entrys which are not correct. Use the gravity.sh script by Pi-Hole for that.

Auto-Updating multiple Blocklists with Pi-hole 5

Pi-hole did a lot of changes. There is no adlists.list file anymore. Now they store it as explained in the sqlite database. I've written an updater which already supports the Pi-hole version 5: https://github.com/JavanXD/ya-pihole-list I am happy about feedback.

What about mobile domains?

First of all: Thank you for your list. Makes my network much cleaner than without. Very good work!

I have found that the domain "ladies.de" is always locked, but "m.ladies.de" is always unlocked. Is there a way to block all subdomains of a domain?

I am not sure if this is a mistake in Pi-Hole interpreting your list or an error in the list itself. If you look for possible subdomains, the runtime of your script will be extended exponentially. I also don't know if this is an easy task to solve.

Add Domains

Please add these domains to your Porn Block List:
anymad.com
bestindiansex.com
bestsouthindiansex.com
cdn.originalindianporn.com
cdn.xxindianporn.com
cdngangsta.com
crazydesisex.com
desifuckonline.com
desipapa.xxx
desipapa2.com
desipornfilm.com
desisex.tv
dirtypornphotos.com
extraindiansex.com
finaldesitube.com
freeindiansexscandals.com
freesexindians.com
fyredet.xyz
hardindianporn.com
homeindiansex.com
hotdesitube.com
hotindianporntube.com
hotnakedgirls.net
hyperspeed.pro
indian-sex-videos.net
indianfucklovers.com
indiangirlsclub.com
indianhamster.com
indiankinkygirls.com
indiansexchudai.com
indiansexphotos.com
indiansexvideos.porn
indiapornonline.com
mixindiansex.com
mostdesisex.com
netdna-ssl.com
newindianfuck.com
nicedesiporn.com
ohindiansex.com
onindiansex.com
ooodesi.com
originalindianporn.com
pepperindiaporn.com
prettyindiaporn.com
provocationstudio.ru
sexpussynude.com
sexyindiantube.com
sexzindian.com
soindianporn.com
soindianporn.com
th.cdngangsta.com
the-urbn.com
totaldesiporn.com
unseenmms.com
wellindianporn.com
wetindianporn.com
xbombo.com
xindiansex.com
xvideos-cdn.com
xxindianporn.com
xxxdesisexvideos.com
xxxindian.pro
ypncdn.com

site addition

hi chad,

i'm new to this github thing... :)

is this the right way to request an addition to the lists?

www.pornve.com

these lists are great... love your work... keep it up.

thanks,

shannon

New urls to add

Thanks for the work putting together the list. I have been testing the list and have come across some more to add. I have them in my local Pi-hole testing environment. Willing to pass them back to you for the community to use. Can you use the Pi-hole json file from the backup to add to the list?

Not actually an issue!

Hi, I occasionally skim through github for interesting pihole tweaks/projects.

Yours is definitely one that peaked my interest.

I have been working on a script to install other projects.

https://github.com/deathbybandaid/piadvanced

I don't want to steal anybodies work, and I try my hardest to make sure I give credit to the correct people.

I plan on keeping an eye on your project.

I didn't know how else to contact you, so feel free to close this when you see it.

Non porn glamuse.com

Hi,
You have glamuse.com in your blocklist which is a lingerie seller, probably blocked upstream.
Thank you for your job editing this list !

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.