ring04h / wydomain Goto Github PK

`
Traceback (most recent call last):
File "wydomain.py", line 149, in
run(args)
File "wydomain.py", line 89, in run
logging.info("chaxunla fetcher subdomains({0}) successfully...".format(len(result)))
TypeError: object of type 'NoneType' has no len()

`

root@VM-131-71-ubuntu:~/wydomain# python wydomain.py wooyun.org

* Starting fofa plugin search

* No parent domain

* Starting get ns record

• ns2.dnsv2.com in whitelist

* ns1.dnsv2.com in whitelist

* Starting bruteforce subdomain task

* running: dnsdict6 -4 -t 32 wooyun.org domain_default.csv

* Starting subdomain search task

Traceback (most recent call last):
File "wydomain.py", line 337, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 298, in start_wydomain
subdomains_result = wy_subdomain_run(domain)
File "/root/wydomain/wysubdomain.py", line 56, in wy_subdomain_run
my_subdomains = get_subdomain_run(domain)
File "/root/wydomain/mysubdomain.py", line 200, in get_subdomain_run
mydomains.extend(links_get(domain))
TypeError: 'NoneType' object is not iterabl

安装phantomjs-2
在cp ./bin/phantomjs /usr/bin/前, 应cd phantomjs-1.9.8-linux-x86_64

TypeError: 'NoneType' object is not iterable

$ python wydomain.py baidu.com

• Starting fofa plugin search

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/X/X/X/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/X/X/X/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

import dns.query
import dns.resolver

_res = dns.resolver.Resolver(filename='/etc/resolv.conf', configure=True)
ns = _res.query('zju.edu.cn', 'NS')
for i in ns.response.answer:
  print i

通过多次获取可以获取到不同的NS记录。

File "wydomain/wydomain.py", line 41
os.makedirs(_cache_path, 0777)
^
SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers
(venv) (base)

* Starting fofa plugin search

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/root/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/root/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/lib/python2.7/json/init.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 383, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
几天前运行时非常正常，今天突然报错，求解

[INFO] Starting new HTTP connection (1): api.ysdm.net
('get verify_code():', None)

1 使用chaxun的api，有时候好有时候不好，没有超过额度，也是一天之内使用的cookie应该不会过期，('Connection aborted.', error(104, 'Connection reset by peer'))

2 我有一个列表批量查询的时候怎么做，或者我想要即使用字典又使用API能一个命令搞定么

3 看到 netcraft貌似最多只有500条记录，如果我想抓所有的.edu.cn的域名这500个明显是不够的，或者这种情况下怎么使用这个工具。或者是这个工具只是针对二级域名，对根域名不是很好。

4 使用过程中还提示
[INFO] Starting new HTTPS connection (1): www.threatminer.org
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning

➜ wydomain git:(master) python wydomain.py xxx.com

* Starting fofa plugin search

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/Users/user/git/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/Users/user/git/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

mysubdomain links_get代码中缺个while(True)
会导致在获取失败后，返回值为NoneType

[root@VM_15_187_centos wydomain]# python wydomain.py 163.com 
--------------------------------------------------
* Starting fofa plugin search
--------------------------------------------------
Traceback (most recent call last):
  File "wydomain.py", line 346, in <module>
    print start_wydomain(sys.argv[1])
  File "wydomain.py", line 53, in start_wydomain
    fofa_result = start_fofa_plugin(domain)
  File "/usr/local/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
    get_partner_domain(domain)
  File "/usr/local/wydomain/fofaplugin.py", line 181, in get_partner_domain
    jobId = json.loads(taskinfo)['jobId']
  File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.6/json/decoder.py", line 338, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

几个月前运行正常，今天再次安装后运行报如上错误。

#12 和这个报错一样吗？
https://fofa.so/lab/addtask/?taskaction=alldomains&domain=
这个接口不可用了？脚本就不可用了？

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 243, in start_fofa_plugin
get_partner_domain(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 186, in get_partner_domain
partner_result = json.loads(http_request_get(domian_jobInfo_url)['html'])
TypeError: 'Response' object has no attribute 'getitem'

应该为wydomain.csv

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 242, in start_fofa_plugin
get_partner_domain(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 180, in get_partner_domain
domian_jobInfo_url = 'http://fofa.so/lab/gettask?jobId=%s&t=%s' % (jobId, int(time.time()))
NameError: global name 'time' is not defined

方便调试，加了两句打印代码：
前后发起多次请求，发现有时泛域名没有解析到IP（泛域名确实存在）

# 能解析泛域名
2016-12-29 01:31:11,904 [INFO] starting bruteforce threading(16) : tjut.edu.cn
-- ehost ['wyspider0.tjut.edu.cn', 'wyspider1.tjut.edu.cn', 'wyspider2.tjut.edu.cn']
-- esets: [u'202.113.64.2', u'202.113.64.2']
-- extensive [u'202.113.64.2', u'202.113.64.2']

# 未解析
2016-12-29 01:32:16,314 [INFO] starting bruteforce threading(16) : tjut.edu.cn
-- ehost ['wyspider0.tjut.edu.cn', 'wyspider1.tjut.edu.cn', 'wyspider2.tjut.edu.cn']
-- esets: []
-- extensive []

我用2个线程 domain_larger.csv字典去尝试穷举 youku.com的域名，发现穷举结果为空。麻烦帮忙看一下什么原因。

最近使用猪哥的工具 发现返回结果没有域名的标题啥的
自己写了个小脚本 希望猪哥参考添加进去
`import requests
import json
from bs4 import BeautifulSoup

domain = set()
jsonname = ['alexa.json', 'chaxunla.json', 'dnsburte.json', 'ilinks.json', 'netcraft.json', 'sitedossier.json'
, 'threatminer.json']
def do_json(name):
with open(name) as f:
data = json.load(f)
return data

def get_title(url):
try:
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 UBrowser/6.0.1471.914 Safari/537.36'}
url = 'http://'+url
print(url)
response = requests.get(url=url, headers=headers,timeout=5,verify=False)
response.encoding = response.apparent_encoding
response = BeautifulSoup(response.text,'lxml')
title = response.html.title.get_text()
except:
title = '空'
return title

for i in jsonname:
for url in do_json(name=i):
domain.add(url)

with open('result.txt','w',errors='ignore') as f:
for i in domain:
title = get_title(url=i)
print(title)
f.write(i+' '+title+'\n')

`

root@kali:~/wydomain# python wydomain.py baidu.com\

---

* Starting fofa plugin search

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/root/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/root/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/lib/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

环境：

Python 2.7.3
Linux pentest 3.14-kali1-686-pae #1 SMP Debian 3.14.5-1kali1 (2014-06-07) i686 GNU/Linux

报错信息：

--------------------------------------------------
* Starting fofa plugin search
--------------------------------------------------
--------------------------------------------------
* Starting process [****.com] partner domain
--------------------------------------------------
* Starting get ns record
--------------------------------------------------
* ns1.dnsv2.com in whitelist
* ns2.dnsv2.com in whitelist
--------------------------------------------------
--------------------------------------------------
* Starting bruteforce subdomain task
--------------------------------------------------
* running: dnsdict6 -4 -t 32 ****.com domain_default.csv
--------------------------------------------------
* Starting subdomain search task
--------------------------------------------------
--------------------------------------------------
* Starting process [602.com] partner domain
--------------------------------------------------
* Starting get ns record
--------------------------------------------------
* ns1.dnsv2.com in whitelist
* ns2.dnsv2.com in whitelist
--------------------------------------------------
--------------------------------------------------
* Starting bruteforce subdomain task
--------------------------------------------------
* running: dnsdict6 -4 -t 32 602.com domain_default.csv
--------------------------------------------------
* Starting subdomain search task
--------------------------------------------------
Traceback (most recent call last):
  File "wydomain.py", line 346, in <module>
    print start_wydomain(sys.argv[1])
  File "wydomain.py", line 178, in start_wydomain
    subdomains_result = wy_subdomain_run(pdomain)
  File "/root/wydomain/wysubdomain.py", line 56, in wy_subdomain_run
    my_subdomains = get_subdomain_run(domain)
  File "/root/wydomain/mysubdomain.py", line 200, in get_subdomain_run
    mydomains.extend(links_get(domain))
TypeError: 'NoneType' object is not iterable

bug在dnsfunc.py的get_zone_record函数和get_ns_record函数。
get_zone_record 的54行，在for内循环中声明了变量a_record和cname_record，导致只能记录最后一次的循环解析，在循环超过两次的情况下会出现遗漏。
get_ns_record函数的204行，同样在for循环中声明了ns_record变量，导致只能记录一次。
例如get_ns_record("ce.cn")。