shmilylty / oneforall Goto Github PK
View Code? Open in Web Editor NEWOneForAll是一款功能强大的子域收集工具
License: GNU General Public License v3.0
OneForAll是一款功能强大的子域收集工具
License: GNU General Public License v3.0
是否使用了最新代码
是
Bug描述
包导入路径不对
如何复现
复现步骤(可不写)
复现命令
python3 oneforall.py --target xx.com
预期结果
清晰而简洁的预期结果描述(可不写)
实际结果
importlib.import_module('.'+name, package)
包的导入路径问题
屏幕截图
运行环境
报错文本
Traceback (most recent call last):
File "oneforall.py", line 161, in <module>
fire.Fire(OneForAll)
File "/usr/local/Cellar/python/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fire/core.py", line 138, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/usr/local/Cellar/python/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fire/core.py", line 471, in _Fire
target=component.__name__)
File "/usr/local/Cellar/python/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fire/core.py", line 675, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "oneforall.py", line 154, in run
self.main()
File "oneforall.py", line 96, in main
collect.run()
File "/Users/xx/study/github/OneForAll/oneforall/collect.py", line 58, in run
self.import_func()
File "/Users/xx/study/github/OneForAll/oneforall/collect.py", line 48, in import_func
import_object = importlib.import_module('.'+name, package)
File "/usr/local/Cellar/python/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 973, in _gcd_import
File "<frozen importlib._bootstrap>", line 923, in _sanity_check
SystemError: Parent module 'modules.certificates' not loaded, cannot perform relative import
Alexa http://alexa.chinaz.com/
Googlect https://transparencyreport.google.com/https/certificates
Ilink
PassiveDNS http://ptrarchive.com/
Pgpsearch http://pgp.mit.edu/pks/lookup?search=baidu.com&op=index
ThreatCrowd https://www.threatcrowd.org/searchApi/v2/domain/report/?domain=baidu.com
用了最新代码
全部更新代码,并安装了re文件内的依赖包,报错:
Exception in thread gitee:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.8/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "/root/tools/OneForAll/oneforall/modules/search/gitee.py", line 70, in do
query.run()
File "/root/tools/OneForAll/oneforall/modules/search/gitee.py", line 56, in run
self.search()
File "/root/tools/OneForAll/oneforall/modules/search/gitee.py", line 36, in search
soup = BeautifulSoup(resp.text, 'lxml')
File "/usr/local/lib/python3.8/site-packages/bs4/init.py", line 225, in init
raise FeatureNotFound(
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?
运行环境
如何复现
python3 oneforall --target xxx.com run
中间有两段会报错
报错文本
复制完整的报错文本(必写)
Exception in thread baidu:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.8/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "/root/tools/OneForAll/oneforall/modules/search/baidu.py", line 105, in do
search.run()
File "/root/tools/OneForAll/oneforall/modules/search/baidu.py", line 78, in run
self.search(self.domain, full_search=True)
File "/root/tools/OneForAll/oneforall/modules/search/baidu.py", line 55, in search
subdomains = self.redirect_match(domain, resp.text)
File "/root/tools/OneForAll/oneforall/modules/search/baidu.py", line 24, in redirect_match
bs = BeautifulSoup(html, features='lxml')
File "/usr/local/lib/python3.8/site-packages/bs4/init.py", line 225, in init
raise FeatureNotFound(
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?
预期结果*
别报错就行,如何解决)
实际结果
清晰而简洁的实际结果描述(选写,如出现什么错误)
屏幕截图
完整OneForAll执行流程截图(建议上传)
日志上传
上传oneforall.log日志文件(复杂问题建议上传)
'query': 'parsed.names: example.com',此处没有替换成domain
应该是这样得:async_timeout==3.0.1,et_xmlfile==1.0.1
你写成async-timeout==3.0.1,et-xmlfile==1.0.1,pip识别不出来
是否使用了最新代码
是
Bug描述
一到域名爆破的时候就出现无效IP的错误提示.
如何复现
使用了自己的subname文件,不知道是不是因为文件里有特殊字符引起的.
预期结果
清晰而简洁的预期结果描述(可不写)
实际结果
清晰而简洁的实际结果描述(如出现什么错误)
运行环境
报错文本
Progress: 0%| | 0/366 [00:08<?, ?it/s] Traceback (most recent call last): File "oneforall.py", line 166, in <module> fire.Fire(OneForAll) File "C:\Python37\lib\site-packages\fire\core.py", line 138, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "C:\Python37\lib\site-packages\fire\core.py", line 471, in _Fire target=component.__name__) File "C:\Python37\lib\site-packages\fire\core.py", line 675, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "oneforall.py", line 159, in run self.main() File "oneforall.py", line 99, in main brute.run() File "E:\tools\OneForAll\oneforall\aiobrute.py", line 284, in run loop.run_until_complete(self.main(self.domain, rx_queue)) File "C:\Python37\lib\asyncio\base_events.py", line 584, in run_until_complete return future.result() File "E:\tools\OneForAll\oneforall\aiobrute.py", line 255, in main results = await pool.map(resolve.aiodns_query_a, task) File "C:\Python37\lib\site-packages\aiomultiprocess\core.py", line 434, in map return await self.results(tids) File "C:\Python37\lib\site-packages\aiomultiprocess\core.py", line 398, in results raise ProxyException(tb) aiomultiprocess.core.ProxyException: Traceback (most recent call last): File "C:\Python37\lib\site-packages\aiomultiprocess\core.py", line 298, in run result = future.result() File "E:\tools\OneForAll\oneforall\common\resolve.py", line 53, in aiodns_query_a resolver = aiodns_resolver() File "E:\tools\OneForAll\oneforall\common\resolve.py", line 41, in aiodns_resolver timeout=config.resolver_timeout) File "C:\Python37\lib\site-packages\aiodns\__init__.py", line 46, in __init__ self.nameservers = nameservers File "C:\Python37\lib\site-packages\aiodns\__init__.py", line 59, in nameservers self._channel.servers = value File "C:\Python37\lib\site-packages\pycares\__init__.py", line 433, in servers raise ValueError("invalid IP address") ValueError: invalid IP address
是否使用了最新代码
是
Bug描述
运行oneforall报错:
RuntimeError: Sorry but the OpenSSL being used by this python install (OpenSSL 1.0.2k-fips 26 Jan 2017) does not meet the minimum version (>= OpenSSL 1.1.1) in order to support TLS 1.3 required by Cloudflare.
但是已经升级到openssl 1.1.1d了:
[root@vultr oneforall]# openssl version
OpenSSL 1.1.1d 10 Sep 2019
如何复现
复现步骤(可不写)
复现命令
预期结果
清晰而简洁的预期结果描述(可不写)
实际结果
清晰而简洁的实际结果描述(如出现什么错误)
屏幕截图
运行环境
报错文本
复制完整的报错文本
是否使用了最新代码
是或否(如果不是的话尝试克隆最新的代码再跑一下)
Bug描述
测试多级域名时会没跑出数据。
如何复现
oneforall.py --target nju.edu.cn run
预期结果
应收集xxxx.nju.edu.cn
实际结果
但是收集的大多数都是edu.cn
屏幕截图
运行环境
报错文本
复制完整的报错文本
是否使用了最新代码
是
Bug描述
存在使用大端口范围时获取有效的子域结果数更少的问题
如何复现
复现命令
python3 oneforall.py --target caih.com --format xlsx --port large run
实际结果
使用大端口范围时获取有效的子域结果数更少
屏幕截图
运行环境
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 17616 and this is thread id 17904.
上手指南-->安装步骤 中,依赖安装指向
https://github.com/shmilylty/OneForAll/tree/master/docs/installation_dependency.md
因改为
https://github.com/shmilylty/OneForAll/tree/master/docs/Installation_dependency.md
大小写问题😂
db = records.Database(db_path) # 不存在数据库时会新建一个数据库
File "/usr/local/lib/python3.7/site-packages/records.py", line 262, in init
self._engine = create_engine(self.db_url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/init.py", line 450, in create_engine
return strategy.create(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 343, in dbapi
raise e
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 341, in dbapi
from sqlite3 import dbapi2 as sqlite # try 2.5+ stdlib name.
File "/usr/local/lib/python3.7/sqlite3/init.py", line 23, in
from sqlite3.dbapi2 import *
File "/usr/local/lib/python3.7/sqlite3/dbapi2.py", line 27, in
from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3'
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
是否使用了最新代码
是或否(如果不是的话尝试克隆最新的代码再跑一下)
是
Bug描述
清晰而简洁的Bug描述
python aiobrute.py
File "aiobrute.py", line 41
logger.log('INFOR', f'姝e湪鎺㈡祴{domain}鏄惁浣跨敤娉涜В鏋?)
^
SyntaxError: invalid syntax
如何复现
复现步骤(可不写)
复现命令
python aiobrute.py
File "aiobrute.py", line 41
logger.log('INFOR', f'姝e湪鎺㈡祴{domain}鏄惁浣跨敤娉涜В鏋?)
^
SyntaxError: invalid syntax
预期结果
清晰而简洁的预期结果描述(可不写)
实际结果
清晰而简洁的实际结果描述(如出现什么错误)
屏幕截图
运行环境
Windows 10 x64
git pull 最新的代码
报错文本
复制完整的报错文本
python aiobrute.py
File "aiobrute.py", line 41
logger.log('INFOR', f'姝e湪鎺㈡祴{domain}鏄惁浣跨敤娉涜В鏋?)
^
SyntaxError: invalid syntax
是否使用了最新代码
是
Bug描述
默认跑 duckduckgo 报错 19:29:20,608 [ERROR] duckduckgo:45 - list index out of range
如何复现
应该是列表容错错误
预期结果
连接错误/0域名/多数域名
实际结果
列表错误
屏幕截图
运行环境
解决办法,强制升级openpyxl库:
pip install -U openpyxl
是否使用了最新代码
是
Bug描述
valid 为none 其余默认,导出的excel中 部分实际为302跳转的 未看到记录
get_redirects = True
可以提取出 Resoponse中的Location字段
使用最新版
在运行的时候总是爆如下错误,看样子是异步模块里的SSL问题,不知道如何解决
ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x10c6d97b8>
transport: <_SelectorSocketTransport closing fd=53 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 763, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:1056)
09:55:02,488 [INFOR] request:113 - 正在异步进行子域的GET请求
Traceback (most recent call last):
File "oneforall.py", line 157, in
fire.Fire(OneForAll)
File "E:\ENV\Python3\lib\site-packages\fire\core.py", line 138, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "E:\ENV\Python3\lib\site-packages\fire\core.py", line 471, in _Fire
target=component.name)
File "E:\ENV\Python3\lib\site-packages\fire\core.py", line 675, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "oneforall.py", line 150, in run
self.main()
File "oneforall.py", line 128, in main
self.datas = loop.run_until_complete(task)
File "E:\ENV\Python3\lib\asyncio\base_events.py", line 571, in run_until_complete
self.run_forever()
File "E:\ENV\Python3\lib\asyncio\base_events.py", line 539, in run_forever
self._run_once()
File "E:\ENV\Python3\lib\asyncio\base_events.py", line 1739, in _run_once
event_list = self._selector.select(timeout)
File "E:\ENV\Python3\lib\selectors.py", line 323, in select
r, w, _ = self._select(self._readers, self._writers, [], timeout)
File "E:\ENV\Python3\lib\selectors.py", line 314, in _select
r, w, x = select.select(r, w, w, timeout)
ValueError: too many file descriptors in select()
Fatal Python error: PyImport_GetModuleDict: no module dictionary!
Current thread 0x00000fb4 (most recent call first):
File "E:\ENV\Python3\lib\site-packages\pycares_init_.py", line 387 in
是否使用了最新代码
是
Bug描述
config.py 中协程数和字典划分对aiobrute.py 无效
如何复现
复现步骤(可不写)
复现命令
py -3 aiobrute.py --target alimama.com --wordlist subdomains.txt --format txt --path 2.txt run
预期结果
清晰而简洁的预期结果描述(可不写)
预期对aiobrute.py 有效
实际结果
config.py 中协程数和字典划分对aiobrute.py 无效
屏幕截图
运行环境
报错文本
brute_process_num = os.cpu_count()
brute_coroutine_num = 256 # 爆破时每个进程下的协程数(不宜大于500)
brute_wordlist_path = data_storage_path.joinpath('subnames.txt')
brute_task_segment = 50000
py -3 aiobrute.py --target alimama.com --wordlist subdomains.txt --format txt --path 2.txt run
11:14:52,847 [INFOR] utils:128 - 获取到1个域名
11:14:52,878 [INFOR] aiobrute:286 - 开始执行AIOBrute模块爆破域名alimama.com
11:14:52,879 [INFOR] aiobrute:287 - 使用4进程乘64协程
11:14:52,881 [INFOR] aiobrute:41 - 正在探测alimama.com是否使用泛解析
11:14:52,937 [ALERT] aiobrute:55 - alimama.com使用了泛解析
11:14:52,940 [ALERT] aiobrute:56 - 01c19ea0.alimama.com 解析到域名: 01c19ea0.alimama.com. IP: {'223.82.248.117'} TTL: 60
11:14:52,943 [INFOR] aiobrute:210 - 使用subdomains.txt字典
Progress: 0%| | 0/1553 [00:00<?, ?it/s]aiobrute:129 - 爆破字典大小:776251
11:14:54,121 [INFOR] aiobrute:255 - 正在爆破alimama.com的域名
Progress: 0%| | 5/1553 [05:42<4:58:29, 11.57s/it]
如果你出现类似RuntimeError: !!Loop Protection!! We have tried to solve 3 time(s) in a row.的错误,请使用pip3更新你的依赖库cloudscraper
pip3 install --upgrade cloudscraper
Is your feature request related to a problem? Please describe.
Can you please do a tutorial on the subdomain scanner and takeover. I'm trying to use this tool
but i'm not sure what is going on...
i have used tool like Sublist3r to enum the sub-domains.
but how to get your tool to enum domains and check for the vulnerability to take over the sub-domain?
thanks.
我用aiobrute.py进行爆破 百万字典 一个晚上只能爆破进度到5%左右
配置如下
brute_process_num = os.cpu_count()
brute_coroutine_num = 1000 # 爆破时每个进程下的协程数(不宜大于1000)
brute_wordlist_path = data_storage_path.joinpath('subnames.txt')
brute_task_segment = 50000
请问怎么才能加快爆破速度呢?
Can't find README.en.md that mentioned in README.md
是否使用了最新代码
是或否(如果不是的话尝试克隆最新的代码再跑一下)
否
Bug描述
清晰而简洁的Bug描述
使用github模块时出现验证邮件:github Please verify your device
如何复现
复现步骤(可不写)
复现命令
在config.py中补充github模块
预期结果
清晰而简洁的预期结果描述(可不写)
脚本运行时出现验证邮件
实际结果
清晰而简洁的实际结果描述(如出现什么错误)
导致github模块无法使用
屏幕截图
运行环境
报错文本
复制完整的报错文本
无
##centos下使用
提示RuntimeError:python版本使用的openssl不兼容问题
解决方案:
重新编译openssl及python3,重新编译前删除干净openssl及python3
ref:https://blog.csdn.net/lkgCSDN/article/details/84403329
报错信息
SSL handshake failed
protocol: <asyncio.sslproto.SSLProtocol object at 0x12cb1add8>
transport: <_SelectorSocketTransport fd=30 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py", line 625, in _on_handshake_complete
raise handshake_exc
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/local/Cellar/python/3.7
修复方法
common文件夹下request.py文件
图中 的BaesException as e: 修改为except BaseException as e:
是否使用了最新代码
是
Bug描述
oneforall还是无法正常退出
运行环境
如何复现
if __name__ == '__main__': # fire.Fire(OneForAll) OneForAll('example.cn').run() # OneForAll('./domains.txt').run()
报错文本
无法退出
预期结果
正常退出
实际结果
清晰而简洁的实际结果描述(选写,如出现什么错误)
屏幕截图
日志上传
上传oneforall.log日志文件(复杂问题建议上传)
自己发现了新的子域名,请问是否可以添加手动批量导入子域名列表的功能
Describe the bug
如alipay.com
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
Add any other context about the problem here.
我用的mac系统,装pycares出现问题,网上找了一圈没有解决好,只有windows系统上的
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command 'clang' failed with exit status 1
Failed building wheel for pycares
Running setup.py clean for pycares
Failed to build pycares
Installing collected packages: pycares
Running setup.py install for pycares ... error
对于一些api接口网站比如chinaz,可以适当把keep_alive关掉,连续扫描子域名时候容易接口触发拒绝连接,关掉之后稳定一些
是否使用了最新代码
是
Bug描述
把config第51行的subnames.txt改成big_subnames.txt会报错
运行环境
如何复现
复现步骤(选写)
复现命令(必写)
python oneforall.py example.com
报错文本
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "oneforall.py", line 251, in run
self.main()
File "oneforall.py", line 176, in main
brute.run()
File "/Users//PycharmProjects/oneforall/aiobrute.py", line 283, in run
loop.run_until_complete(self.main(self.domain, rx_queue))
File "/Users//.pyenv/versions/3.8.0/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete
return future.result()
File "/Users//PycharmProjects/oneforall/aiobrute.py", line 261, in main
results = await resolve.aio_resolve(tasks, self.process, self.coroutine)
File "/Users/PycharmProjects/oneforall/common/resolve.py", line 168, in aio_resolve
return results
File "/Users//.local/share/virtualenvs/oneforall-EpEYzA6i/lib/python3.8/site-packages/aiomultiprocess/pool.py", line 130, in aexit
self.terminate()
File "/Users//.local/share/virtualenvs/oneforall-EpEYzA6i/lib/python3.8/site-packages/aiomultiprocess/pool.py", line 277, in terminate
self.close()
File "/Users/.local/share/virtualenvs/oneforall-EpEYzA6i/lib/python3.8/site-packages/aiomultiprocess/pool.py", line 272, in close
tx.put_nowait(None)
File "/Users//.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/queues.py", line 132, in put_nowait
return self.put(obj, False)
File "/Users/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/queues.py", line 84, in put
raise Full
queue.Full
预期结果
清晰而简洁的预期结果描述(选写,如正常情况应该是怎么样的)
实际结果
清晰而简洁的实际结果描述(选写,如出现什么错误)
屏幕截图
完整OneForAll执行流程截图(建议上传)
日志上传
上传oneforall.log日志文件(复杂问题建议上传)
是否使用了最新代码
是
Bug描述
Fofa api调用时出现500错误 Internal Server Error 8049
运行环境
如何复现
复现步骤(选写)
复现命令(必写)
报错文本
Internal Server Error 8049
预期结果
清晰而简洁的预期结果描述(选写,如正常情况应该是怎么样的)
实际结果
清晰而简洁的实际结果描述(选写,如出现什么错误)
屏幕截图
完整OneForAll执行流程截图(建议上传)
日志上传
上传oneforall.log日志文件(复杂问题建议上传)
import time
from config import logger
from common.query import Query
class Gov(Query):
def __init__(self, domain):
Query.__init__(self)
self.domain = self.register(domain)
self.module = 'Gov'
self.source = 'GovQuery'
self.addr = "http://114.55.181.28/check_web/databaseInfo_mainSearch.action?sEcho=2&iColumns=6&sColumns=,,,,,&iDisplayStart=0&iDisplayLength=10&mDataProp_0=dataNumber&bSortable_0=false&mDataProp_1=sitecode&bSortable_1=false&mDataProp_2=wzzgdw&bSortable_2=false&mDataProp_3=wzmc&bSortable_3=false&mDataProp_4=url&bSortable_4=false&mDataProp_5=caozuo&bSortable_5=false&iSortCol_0=0&sSortDir_0=asc&iSortingCols=1&term={}&searchType=url&isSearch=true&tt=Mon Mar 16 2020 23:14:59 GMT+0800 (China Standard Time)&is_exp=0,1,3&size=10&pos=1&pageNo={}&_t=1584371699717"
def query(self):
"""
向接口查询子域并做子域匹配
"""
page_num = 1
while True:
time.sleep(self.delay)
self.header = self.get_header()
self.proxy = self.get_proxy(self.source)
try:
resp = self.get(self.addr.format(self.domain, page_num))
except Exception as e:
logger.log('ERROR', e.args)
break
if not resp:
break
if resp.status_code != 200:
logger.log('ERROR', f'{self.source}模块搜索出错')
break
subdomains = self.match(self.domain, resp.text)
self.subdomains = self.subdomains.union(subdomains)
if not subdomains:
break
if page_num > 10:
break
page_num += 1
def run(self):
"""
类执行入口
"""
if self.domain.find("gov.cn") < 0:
return
self.begin()
self.query()
self.finish()
self.save_json()
self.gen_result()
self.save_db()
def do(domain):
"""
类统一调用入口
:param str domain: 域名
"""
query = Gov(domain)
query.run()
if __name__ == '__main__':
do('nanjing.gov.cn')
是否使用了最新代码
是
Bug描述
运行到异步进行子域的get请求时,开始出现大量的ssl报错
如何复现
复现步骤(可不写)
复现命令
预期结果
清晰而简洁的预期结果描述(可不写)
实际结果
11:13:40,548 [INFOR] request:146 - 正在异步进行子域的GET请求
Progress: 38%|▍| 3978/10345 [02:00<00:06, 1025.56it/s]ERROR:asyncio:SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f326d824470>
transport: <_SelectorSocketTransport fd=808 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/usr/lib/python3.7/asyncio/sslproto.py", line 207, in feed_ssldata
self._sslobj.unwrap()
File "/usr/lib/python3.7/ssl.py", line 767, in unwrap
return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)
Progress: 49%|▍| 5080/10345 [02:36<00:47, 111.25it/s]ERROR:asyncio:SSL handshake failed
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f3235e019b0>
transport: <_SelectorSocketTransport fd=787 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/sslproto.py", line 625, in _on_handshake_complete
raise handshake_exc
File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1056)
屏幕截图
运行环境
报错文本
ERROR:asyncio:SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f326d824470>
transport: <_SelectorSocketTransport fd=808 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/usr/lib/python3.7/asyncio/sslproto.py", line 207, in feed_ssldata
self._sslobj.unwrap()
File "/usr/lib/python3.7/ssl.py", line 767, in unwrap
return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)
Progress: 49%|▍| 5080/10345 [02:36<00:47, 111.25it/s]ERROR:asyncio:SSL handshake failed
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f3235e019b0>
transport: <_SelectorSocketTransport fd=787 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/sslproto.py", line 625, in _on_handshake_complete
raise handshake_exc
File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1056)
是否使用了最新代码
是或否(如果不是的话尝试克隆最新的代码再跑一下!)
是最新代码
Bug描述
生成结果错误
运行环境
如何复现
复现步骤(选写)
直接运行命令
复现命令(必写)
python3 oneforall.py --target domian run
报错文本
Request Progress: 80%|██████████████████▍ | 531/664 [00:09<00:02, 64.68it/s]ValueError: to
o many file descriptors in select()
预期结果
正常结果不太清楚,应该是正常生成文件把
实际结果
清晰而简洁的实际结果描述(选写,如出现什么错误)
日志上传
上传oneforall.log日志文件(复杂问题建议上传)
是否使用了最新代码
是
Bug描述
如何复现
do("360.cn")
预期结果
测试可以获取到400个子域名
实际结果
实际只获取了39个子域名,后面的请求都没发成功。
运行环境
报错文本
{'status': 'error', 'error_type': 'malformed_request', 'error': 'request is missing the required field query'}
运行到开启爆破的时候报错如下:
Python环境为Python3.7.0
Progress: 0%| | 0/47 [00:00<?, ?it/s]
Traceback (most recent call last):
File "oneforall.py", line 166, in
fire.Fire(OneForAll)
File "/usr/local/python3/lib/python3.7/site-packages/fire/core.py", line 138, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/usr/local/python3/lib/python3.7/site-packages/fire/core.py", line 471, in _Fire
target=component.name)
File "/usr/local/python3/lib/python3.7/site-packages/fire/core.py", line 675, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "oneforall.py", line 159, in run
self.main()
File "oneforall.py", line 99, in main
brute.run()
File "/OneForAll/oneforall/aiobrute.py", line 282, in run
loop.run_until_complete(self.main(self.domain, rx_queue))
File "/usr/local/python3/lib/python3.7/asyncio/base_events.py", line 568, in run_until_complete
return future.result()
File "/OneForAll/oneforall/aiobrute.py", line 253, in main
results = await pool.map(resolve.aiodns_query_a, task)
File "/usr/local/python3/lib/python3.7/site-packages/aiomultiprocess/core.py", line 434, in map
return await self.results(tids)
File "/usr/local/python3/lib/python3.7/site-packages/aiomultiprocess/core.py", line 398, in results
raise ProxyException(tb)
aiomultiprocess.core.ProxyException: Traceback (most recent call last):
File "/usr/local/python3/lib/python3.7/site-packages/aiomultiprocess/core.py", line 298, in run
result = future.result()
File "/OneForAll/oneforall/common/resolve.py", line 54, in aiodns_query_a
answers = await resolver.query(hostname, 'A')
aiodns.error.DNSError: (4, b'Domain name not found')
There is need to add some prefix keywords too while searching on crt.sh for specific domain.
Like if we see simple domain without using any prefix on crt.sh
https://crt.sh/?q=%25.yahoo.com
But what if we use prefixes like
api,Internal,Staging,Stag,Production,Product,dev,devs,prod,new
We will get different results from above.
https://crt.sh/?q=%25.api.yahoo.com
Please add prefixes in the tool to grab more subdomains related to a domain.
(hack) D:\haker\1-脚本\OneForAll\oneforall>python oneforall.py -target xj-tianye.com run
Traceback (most recent call last):
File "oneforall.py", line 13, in <module>
import config
File "D:\haker\1-脚本\OneForAll\oneforall\config.py", line 195, in <module>
sys.stderr.reconfigure(encoding='utf-8')
AttributeError: '_io.TextIOWrapper' object has no attribute 'reconfigure'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.