Comments (17)
can confirm it chugs through my list of 750k domains quite happily on a Kali VM with 2gb ram
from subjack.
Hi @HellboundTKR,
I've ran into this problem in the past. To help you mitigate this issue I have a few questions,
-
- How big is your list?
-
- Assuming you're using mac or linux; can you run
ulimit -n
and provide me the output?
- Assuming you're using mac or linux; can you run
Thanks
from subjack.
Also, I would like to suggest running: cat all_subdomains.lst | sort | uniq > new_all_subdomains.lst
to remove duplicate subdomains (From the screenshot it looks like you have duplicates).
from subjack.
Okay, another update.
In an old, old version of subjack, when I was using net/http instead of fasthttp, I had a Connection: close
header set which mitigated this issue in the past.
I just pushed an update to readd this header to each request.
Please retest and confirm this is a working solution after you've:
- Sorted your list to remove duplicate subdomains
- Increased your file descriptors limit (if needed)
Thanks
from subjack.
Hi,
The llst is around 100k subdomains.
My OS is ParrotOS 64
ulimit - n was 1024
changed ulimit to unlimited but message still occuring even after patch
thanks
from subjack.
@HellboundTKR that's very strange if it's still occurring after you set it to unlimited in the same session.
I'm attempting to reproduce this issue with a list containing ~246k subdomains.
➜ subjack git:(master) ✗ ulimit -n
4864
So far gone through, over 50k of those subdomains without any errors using the following command, similar to the one you posted above:
./subjack -w cname_list.txt -t 50 -o subjackresults.txt -ssl
Do you have an estimate of how many subdomains you're able to enumerate before you experience the 'too many files' error?
from subjack.
Just an update, I surpassed 100k subdomains w/o any errors
from subjack.
Perhaps this is a low memory / low cpu issue? I ran into the "too many files" a long time ago on my 1gb ram ubuntu box... I'm doing this current scan with the 246k subdomain list on my macbook pro. Are you running this on Parrot OS in a VM? With low memory/cpu?
A possible workaround would be to split the large file into chunks.
from subjack.
Yeh its strange, I get to about 80k domains when it stops.
Im using it in a Virtualbox VM with 4cores set 4gb memory using i7 4770k @4,5ghz
Thing is, I can chug through my list no problems when using SubOver tool.
from subjack.
Subjack does make a lot more connections and requests than SubOver (which is based on an older version of Subjack) to accurately check for a possible subdomain takeover.
I’ll set up a Parrot OS VM this weekend, then run a series of tests including testing with the default Parrot OS instalization and testing with my suggested performance optimizations.
If anyone reading this issue is experiencing the same problem, please comment the OS, wordlist size, memory and CPU details and I will try to replicate that as well. Thanks.
from subjack.
Same here, but I don't think is a VM problem. You can easily fix it adjusting -t flag. As per 90mb's fiber I'm using -t 150 but if I tune it up to 200 -> too many open files. IS there any other way to fix it? How may I speed up it?
from subjack.
@HeisenbugHQ well, 200 threads is a lot.... I don't recommend going past 100, it's important to keep in mind that the more you increase, the harder your machine is going to have to work.
--
@everyone
The underlying issue here is that the 'too many open files' error occurs when there's too many connections open.
I've done as much research as I could, the only solution for the 'too many open files' error is to raise the ulimit (ulimit -n unlimited).
Taking all of this into consideration, the only solutions to remediate this issue are:
- Make Subjack slower (Cap number of connections being made) // Which I don't plan on doing.
- Properly, raise your box's ulimit
- Use on a better box
from subjack.
Hi guys! I having the same issue with a list of 14000 host
- ulimit is set to unlimited
- my box have 4 gb ram 2 cores 50 ssd
- This happen with 30 threads
from subjack.
I don't know why this happens. i had no errors with a list of +1 millions subdomains at 30-50 threads. but today i tried again at 80 threads with a list of 100k subdomains and got this error.
I will try again with no threads at all and update this comment.
EDIT:
Apparently if you just lower the threads you will run it just fine. play with it until you have no errors.
from subjack.
I ran into a similar but different issue while leveraging this tool. The error I ran into was specifically around fingerprints.json: too many open files
. I've created a PR which seems to address this issue and may address other folks' issues as well.
from subjack.
same error too many files to open. Any updates now.
from subjack.
same error too many files to open. Any updates now.
Sen türkmüsün?
from subjack.
Related Issues (20)
- Killed error while running script
- issues in building main.go file with go build
- fingerprints.json not found HOT 6
- add list of resolvers HOT 5
- Error
- -bash: ./subjack: is a directory
- No output from subjack HOT 1
- go get github.com/haccer/subjack No reply HOT 3
- subjack hangs and generates corupted json
- /src/github.com/haccer/subjack/fingerprints.json: no such file or directory HOT 2
- open /src/github.com/haccer/subjack/fingerprints.json: no such file or directory HOT 9
- 2021/11/07 20:10:05 invalid character '<' looking for beginning of value
- subjack installed into go/bin but no reply to subjack commands HOT 2
- Subjack not installing HOT 2
- 2022/06/02 22:31:47 invalid character '<' looking for beginning of value HOT 1
- Idont compile go build
- how to avoid the common problems related to this tool
- no idea where to start, problems from the word GO
- 2022/11/06 14:02:09 invalid character 'p' looking for beginning of value HOT 1
- Cant Exlcude For Output HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from subjack.