Comments (8)
On some systems I'm able to reproduce the issue, and on other I'm not. I'm looking into whether this is an overall performance issue with the tool or a system-level issue,
For now, using a local redis queue resolves the issue.
from changeme.
I have the same error on different Kali rolling machines.
Please install a redis server for better performance.
I have redis-server installed, how to use it?
from changeme.
@network23 apt-get install redis-server && /etc/init.d/redis-server start
should install the redis-server
and start it.
I'm going to try to work on a bug fix for this. My hunch is that the underlying problem is that each fingerprint and scanner instances has a reference to the credential dict and when that writes to the db, it's having to write an excessive amount of data. I'm going to move the creds dict to a global variable so it doesn't get stored along side the scanner or I'll do some more __getstate__
/__setstate__
magic.
from changeme.
@ztgrace Ah, it uses it automagically.
Still the same error, also when I'm using the docker file.
You could be right, I tried using "--protocols ssh" and it's working correctly. Even when scanning a local /24.
Traceback (most recent call last):
File "./changeme.py", line 6, in
core.main()
File "/home/X/Desktop/progz/SCAN/changeme/changeme/core.py", line 67, in main
check_for_interrupted_scan(config)
File "/home/X/Desktop/progz/SCAN/changeme/changeme/core.py", line 394, in check_for_interrupted_scan
scanners = FIFOSQLiteQueue(path=".", multithreading=True, name="scanners")
File "/home/X/.local/lib/python2.7/site-packages/persistqueue/sqlbase.py", line 70, in init
self._init()
File "/home/X/.local/lib/python2.7/site-packages/persistqueue/sqlqueue.py", line 40, in _init
super(SQLiteQueue, self)._init()
File "/home/X/.local/lib/python2.7/site-packages/persistqueue/sqlbase.py", line 85, in _init
self._conn.execute(self._sql_create)
sqlite3.OperationalError: database is locked
from changeme.
@ztgrace I see the same error:
Traceback (most recent call last):
File "./changeme-1.0.3/changeme.py", line 6, in <module>
core.main()
File "/opt/attack-host/changeme-1.0.3/changeme/core.py", line 70, in main
s.scan()
File "/opt/attack-host/changeme-1.0.3/changeme/scan_engine.py", line 65, in scan
while self.scanners.qsize() > 0:
File "/usr/local/lib/python2.7/dist-packages/persistqueue/sqlqueue.py", line 84, in qsize
return self.size
File "/usr/local/lib/python2.7/dist-packages/persistqueue/sqlqueue.py", line 81, in size
return self._count()
File "/usr/local/lib/python2.7/dist-packages/persistqueue/sqlbase.py", line 121, in _count
row = self._putter.execute(sql).fetchone()
sqlite3.OperationalError: database is locked
I noticed you are using persist-queue
package. So, I went to https://github.com/peter-wangxu/persist-queue
and noticed that they have a FAQ on their page which states:
persistquest open 2 connections for the db if multithreading=True, the SQLite database is locked until that transaction is committed. The timeout parameter specifies how long the connection should wait for the lock to go away until raising an exception. Default time is 10, increase timeout when creating the queue if above error occurs.
I then tried adding the timeout
on line 394 and 395 of core.py
to be 100.0
so that it increases from the default of 10.0
but no dice.
I also tried changing multithreading=True
to multithreading=False
but still kept getting the same error.
from changeme.
Yeah, I've experimented with timeouts, threads, etc and have the same issues. What I've noticed is that for a relatively small scan like a /24 the data.db is about 850 MB which is ridiculous. And it's only going to get worse as the number of creds grows.
I'm working on redesigning the fingerprint/scanner pattern so that each class instance doesn't contain a reference to the large creds dict. That way, when the fingerprint/scanner modules get written out to the sqlite db, the records should be considerably smaller.
from changeme.
Update:
The initial refactor has reduced the size of the data.db from ~850MB to 50MB! However, I'm still getting the dreaded database is locked
error.
Rather than beating my head against the wall trying to solve the sqlite issues (seems like it might be related to HAVE_USLEEP
compile options which varies by distro) I'll be switching the storage mechanism to :memory:
since that was fixed in persist-queue 0.3.1: peter-wangxu/persist-queue#22 and the only persistent option will be redis.
from changeme.
This should be resolved by v1.0.4.
from changeme.
Related Issues (20)
- paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
- Problem with lxml
- Not working
- Selenium PhantomJS driver is deprecated HOT 2
- User agent header evaluation order corner case HOT 1
- Did n't find tomcat credentials HOT 9
- Error Cannot Allocate Memory HOT 5
- Issue while adding default credential detection for Aruba's ClearPass HOT 6
- delete this HOT 1
- --shodan_query flag use cases HOT 1
- Option to disable screenshots
- Option to stop bruteforcing after at least one valid creds pair for each target was found HOT 3
- stucks on big input file HOT 4
- add to ssh.yml new crediantial , but, ssh credential not found HOT 2
- Adding RabbitMQ scanning capability HOT 1
- no module named cerberus HOT 1
- ImportError: No module named cerberus HOT 3
- Docker image is out of date
- Add custom ports to specific services
- pyodbc ... error HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from changeme.