Git Product home page Git Product logo

shodanz's Introduction

Shodanz

Gem Version Yard Docs CI

A modern, async Ruby gem for Shodan, the world's first search engine for Internet-connected devices.

Installation

$ gem install shodanz

Usage

require "shodanz"

client = Shodanz.client.new(key: "YOUR_API_KEY")

Note You can also set the SHODAN_API_KEY environment variable instead of passing the API key as an argument when creating a client.

Optional Async Support

Shodanz utilizes async to provide asyncronous IO. This doesn't break any existing scripts using Shodanz, but now offers even more flexibility to write more awesome things, like this asyncronous honeypot detector:

Note To run any Shodanz method asyncronously, simply wrap it in a Async { ... } block. To wait for any other async operation to finnish in the block, call .wait on it.

REST API

The REST API provides methods to search Shodan, look up hosts, get summary information on queries and a variety of utility methods to make developing easier. Refer to the REST API documentation for more ideas on how to use it.

Shodan Search Methods

Search'n for stuff, are 'ya?

Host Information

Returns all services that have been found on the given host IP.

client.host("8.8.8.8")                # Default
client.host("8.8.8.8", history: true) # All historical banners should be returned.
client.host("8.8.8.8", minify: true)  # Only return the list of ports and the general host information, no banners.

Host Search

Search Shodan using the same query syntax as the website and use facets to get summary information for different properties.

client.host_search("mongodb")
client.host_search("nginx")
client.host_search("apache", after: "1/12/16")
client.host_search("ssh", port: 22, page: 1)
client.host_search("ssh", port: 22, page: 2)
client.host_search("ftp", port: 21, facets: { link: "Ethernet or modem" })

Search Shodan without Results

This method behaves identical to host_search with the only difference that this method does not return any host results, it only returns the total number of results that matched the query and any facet information that was requested. As a result this method does not consume query credits.

client.host_count("apache")
client.host_count("apache", country: "US")
client.host_count("apache", country: "US", state: "MI")
client.host_count("apache", country: "US", state: "MI", city: "Detroit")
client.host_count("nginx",  facets: { country: 5 })
client.host_count("apache", facets: { country: 5 })

Scan Targets

Use this method to request Shodan to crawl an IP or netblock.

client.scan("8.8.8.8")

Crawl Internet for Port

Use this method to request Shodan to crawl the Internet for a specific port.

This method is restricted to security researchers and companies with a Shodan Data license. To apply for access to this method as a researcher, please email [email protected] with information about your project. Access is restricted to prevent abuse.

client.crawl_for(port: 80, protocol: "http")

List Community Queries

Use this method to obtain a list of search queries that users have saved in Shodan.

client.community_queries
client.community_queries(page: 2)
client.community_queries(sort: "votes")
client.community_queries(sort: "votes", page: 2)
client.community_queries(order: "asc")
client.community_queries(order: "desc")

Search Community Queries

Use this method to search the directory of search queries that users have saved in Shodan.

client.search_for_community_query("the best")
client.search_for_community_query("the best", page: 2)

Popular Community Query Tags

Use this method to obtain a list of popular tags for the saved search queries in Shodan.

client.popular_query_tags
client.popular_query_tags(20)

Protocols

This method returns an object containing all the protocols that can be used when launching an Internet scan.

client.protocols

Ports

This method returns a list of port numbers that the Shodan crawlers are looking for.

client.ports

Account Profile

Returns information about the Shodan account linked to this API key.

client.profile

DNS Lookup

Look up the IP address for the provided list of hostnames.

client.resolve("google.com")
client.resolve("google.com", "bing.com")

Reverse DNS Lookup

Look up the hostnames that have been defined for the given list of IP addresses.

client.reverse_lookup("74.125.227.230")
client.reverse_lookup("74.125.227.230", "204.79.197.200")

HTTP Headers

Shows the HTTP headers that your client sends when connecting to a webserver.

client.http_headers

Your IP Address

Get your current IP address as seen from the Internet.

client.my_ip

Honeypot Score

Calculates a honeypot probability score ranging from 0 (not a honeypot) to 1.0 (is a honeypot).

client.honeypot_score('8.8.8.8')

API Plan Information

client.info

Streaming API

The Streaming API is an HTTP-based service that returns a real-time stream of data collected by Shodan. Refer to the Streaming API documentation for more ideas on how to use it.

Banners

This stream provides ALL of the data that Shodan collects. Use this stream if you need access to everything and/ or want to store your own Shodan database locally. If you only care about specific ports, please use the Ports stream.

client.banners do |data|
  # do something with banner data
  puts data
end

Banners Filtered by ASN

This stream provides a filtered, bandwidth-saving view of the Banners stream in case you are only interested in devices located in certain ASNs.

client.banners_within_asns(3303, 32475) do |data|
  # do something with banner data
  puts data
end

Banners Filtered by Country

This stream provides a filtered, bandwidth-saving view of the Banners stream in case you are only interested in devices located in certain countries.

client.banners_within_countries("DE", "US", "JP") do |data|
  # do something with banner data
  puts data
end

Banners Filtered by Ports

Only returns banner data for the list of specified ports. This stream provides a filtered, bandwidth-saving view of the Banners stream in case you are only interested in a specific list of ports.

client.banners_on_ports(21, 22, 80) do |data|
  # do something with banner data
  puts data
end

Banners by Network Alerts

Subscribe to banners discovered on all IP ranges described in the network alerts.

client.alerts do |data|
  # do something with banner data
  puts data
end

Banner Filtered by Alert ID

Subscribe to banners discovered on the IP range defined in a specific network alert.

client.alert("HKVGAIRWD79Z7W2T") do |data|
  # do something with banner data
  puts data
end

Exploits API

The Exploits API provides access to several exploit/ vulnerability data sources. Refer to the Exploits API documentation for more ideas on how to use it.

Search

Search across a variety of data sources for exploits and use facets to get summary information.

client.exploits_api.search("python")             # Search for python vulns.
client.exploits_api.search(port: 22)             # Port number for the affected service if the exploit is remote.
client.exploits_api.search(type: "shellcode")    # A category of exploit to search for.
client.exploits_api.search(osvdb: "100007")      # Open Source Vulnerability Database ID for the exploit.

Count

This method behaves identical to the Exploits API search method with the difference that it doesn't return any results.

client.exploits_api.count("python")             # Count python vulns.
client.exploits_api.count(port: 22)             # Port number for the affected service if the exploit is remote.
client.exploits_api.count(type: "shellcode")    # A category of exploit to search for.
client.exploits_api.count(osvdb: "100007")      # Open Source Vulnerability Database ID for the exploit.

License

The gem is available as open source under the terms of the MIT License.

shodanz's People

Contributors

dependabot-preview[bot] avatar dependabot[bot] avatar malwaremily avatar noraj avatar picatz avatar red0xff avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

shodanz's Issues

undefined method

/Users/qppn/.rvm/gems/ruby-3.2.2/gems/shodanz-2.0.6/lib/shodanz.rb:12:in <top (required)>': undefined method logger' for Async:Module (NoMethodError)

Async.logger.level = 4
^^^^^^^
from internal:/Users/qppn/.rvm/rubies/ruby-3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb:160:in require' from <internal:/Users/qppn/.rvm/rubies/ruby-3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:160:in rescue in require'
from internal:/Users/qppn/.rvm/rubies/ruby-3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb:40:in require' from /Users/qppn/Desktop/Projects/shodan.rb:1:in

'
internal:/Users/qppn/.rvm/rubies/ruby-3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb:86:in require': cannot load such file -- shodanz (LoadError) from <internal:/Users/qppn/.rvm/rubies/ruby-3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in require'
from /Users/qppn/Desktop/Projects/shodan.rb:1:in `'

Scan API request format is broken

I recently upgraded from 2.0.0 to 2.0.3 and found that client.scan no longer works. I dug in a bit, but I wasn't able to figure out what exactly was wrong with the request, but something about the format or the params is broken. I've verified with the Python Shodan library, that scanning still works as well as curl and I used the same API key in all cases. It looks like this could be an issue that was introduced when changing from Async::HTTP::Internet to Async::HTTP::Client, but the documentation is scant for those libraries and the source code didn't give any hints either. Below is the error response I get when trying to request a scan ($shodan is an instance of Client):

Portal (DEVELOPMENT) (main) 001:0> $shodan.scan "1.1.1.1"
JSON::ParserError: 784: unexpected token at '<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx</center>
</body>
</html>
'
from /home/chris/.rbenv/versions/2.5.5/lib/ruby/gems/2.5.0/gems/json-2.3.1/lib/json/common.rb:263:in `parse'

make a new release / bump version

Some commits happened today and @emiliska found an issue in the README, and a bug she is planning on making a PR for.

Should probably release a new version and bump version deps for this gem since it has been a few months.

[Chore] Ruby 2.7.0 changes

/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/shodanz-2.0.0/lib/shodanz/apis/streaming.rb:57: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/shodanz-2.0.0/lib/shodanz/apis/utils.rb:30: warning: The called method `slurp_stream' is defined here
/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/shodanz-2.0.0/lib/shodanz/apis/utils.rb:36: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/shodanz-2.0.0/lib/shodanz/apis/utils.rb:168: warning: The called method `sync_slurp_stream' is defined here
/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/shodanz-2.0.0/lib/shodanz/apis/utils.rb:170: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/shodanz-2.0.0/lib/shodanz/apis/utils.rb:113: warning: The called method `slurper' is defined here
/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/async-http-0.38.3/lib/async/http/url_endpoint.rb:163: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
/home/noraj/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/async-io-1.27.1/lib/async/io/host_endpoint.rb:98: warning: The called method `tcp' is defined here

Ruby 2.7

The shodanz warning must be due to Separation of positional and keyword arguments

/kernel_require.rb: 59:in require: cannot load such file - client

So I'm working with ruby api for future project.
And I've made like this.
I know there's a Client.rb file inside of the default folder so I've made a require for it.
and give me that issue.
Without hte client lib the script don't give me any result and the script back to the console
I' guess I'll have to add json and another lib right?

I'll stay here working on script at all night if I fix it I'll update but so far no luck
PS: I'll make the code of ruby looks better at future but I'm new on ruby so is not perfect code.
Ps 2: I don't want you to make that job I need just a direction.
Have niice night.

require "shodanz"
require "client"
def shodanz

if shodanz.client.new("MyAPI")
  puts shodanz.client.new
if endendendclient.rest_api.host
  puts endendendclient.rest_api.host_search
if client.rest_api.host("8.8.8.8", history: true) 
   puts client.rest_api.host_search
if client.rest_api.host_search("mongodb")
   puts client.rest_api.host_search
if client.rest_api.host_search("nginx")
   puts client.rest_api.host_search
if client.rest_api.host_search("apache", after: "1/12/16")
  puts client.rest_api.host_search
if client.rest_api.host_search("ssh", port: 22, page: 1)
  pits client.rest_api.host_search
if client.rest_api.host_search("ssh", port: 22, page: 2)
   puts client.rest_api.host_search
if client.rest_api.host_search("ftp", port: 21, facets: { link: "Ethernet or modem" })
   puts clients.rest_ap

Issue:

traceback (most recent call last):
	2: from shodanz.rb:2:in `<main>'
	1: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
/usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- client (LoadError)

Potentially unnecessary tripple error.

Could probably cleanup the error when a client is initialized without an API key.

[1] pry(main)> require "shodanz"
=> true
[2] pry(main)> client = Shodanz.client.new
No key has been found or provided!
No key has been found or provided!
No key has been found or provided!

This is happening because the client class is simply a proxy to the three other API classes; and so the key is stored once for each API. This allows for multiple API keys to be used by the same client / or for API keys to be changed later.

Maybe this feature could be removed to make things simpler. Or, at least the error could be caught once when the client is initialized.

This needs to be cleaned up further.

`client.api_rest.my_ip` not working

The client.rest_api.my_ip function is not working due to a typo in the URL endpoint

irb(main):007:0> client.rest_api.my_ip
TypeError: exception class/object expected
        from /var/lib/gems/2.3.0/gems/shodanz-1.0.2/lib/shodanz/apis/rest.rb:180:in `raise'
        from /var/lib/gems/2.3.0/gems/shodanz-1.0.2/lib/shodanz/apis/rest.rb:180:in `get'
        from /var/lib/gems/2.3.0/gems/shodanz-1.0.2/lib/shodanz/apis/rest.rb:164:in `my_ip'
        from (irb):7
        from /usr/bin/irb:11:in `<main>'

Suggested Fix: Update endpoint URL in shodanz/lib/shodanz/apis/rest.rb

Update README.md to correct typos in 'Search Shodan without Results'

Currently the README.md contains some small typos on lines 57 - 58:

client.rest_api.host_count("nginx".  facets: { country: 5 })
client.rest_api.host_count("apache". facets: { country: 5 })

I suggest the '.' after the first paramenter be replaced with ',' :

client.rest_api.host_count("nginx",  facets: { country: 5 })
client.rest_api.host_count("apache", facets: { country: 5 })

Help

Hi, i'm writing coding using you api but I have a doubt... Where is i insert API ?

exemple:
`require "shodanz"

client = Shodanz.client.new
client.rest_api.host("8.8.8.8") `

tks!

[Question] Async example working?

I'm able to make all the REST API examples work but not he async one, it always result nothign and exit without error. I quickly checked all the methods used seems not to have changed.

require 'async'
require 'shodanz'

client = Shodanz.client.new(key: "YOUR_API_KEY")

# Asynchronously stream banner info from shodan  and check any
# IP addresses against the experimental honeypot scoring service.
client.streaming_api.banners do |banner|
  if ip = banner['ip_str']
    Async do
      score = client.rest_api.honeypot_score(ip).wait
      puts "#{ip} has a #{score * 100}% chance of being a honeypot"
    rescue Shodanz::Errors::RateLimited
      sleep rand
      retry
    rescue # any other errors
      next
    end
  end
end

Exploit API

require 'shodanz'

client = Shodanz.client.new(key: 'xxx')

client.exploits_api.search('port: 10050')
# or
client.exploits_api.count('port: 22', page: 1)
# or
client.exploit_count('port: 22')

I'm never having an answer, like a forever timeout.
Are you able to make it work?

Directly it works.

curl -s 'https://exploits.shodan.io/api/count?query=python&key=xxx' | jq
{
  "matches": [],
  "total": 3263
}

I tried with async-http 0.50.0 and 0.38.3, async-io 1.27.1.

scan API failing

The client #scan method is broken. I suspect Shodan has changed their interface for this API endpoint. It now expects the IP(s) of the scan to be in a flat, non-json string submitted as a POST body. Their example:

$ curl -X POST "https://api.shodan.io/shodan/scan?key={YOUR_API_KEY}" -d 'ips=8.8.8.8,1.1.1.1'

Dependency issues causes Gem::ConflictError

@ninoseki let me know that shodanz isn't working on ruby 2.6 due to dependency issues:

Ruby version I tested:

$ ruby -v
ruby 2.6.0preview2 (2018-05-31 trunk 63539) [x86_64-darwin18]

Error when requiring shodanz:

$ pry
[1] pry(main)> require "shodanz"                                                                                                                                           
Gem::ConflictError: Unable to activate unirest-1.1.2, because json-2.1.0 conflicts with json (~> 1.8.1)
...

Add real tests

There are no real rspec tests at the moment.

This needs to be resolved eventually.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.