This project is inspired by Ruby's mechanize. The purpose is to cover all the features of original one.
-
Add the dependency to your
shard.yml:dependencies: mechanize: github: Kanezoh/mechanize.cr
-
Run
shards install
require "mechanize"
agent = Mechanize.new
page = agent.get("http://example.com/")
puts page.code # => 200
puts page.body # => html
puts page.title # => Example DomainYou can also send post request with data.
require "mechanize"
agent = Mechanize.new
query = {"foo" => "foo_value", "bar" => "bar_value"}
page = agent.post("http://example.com/", query: query)
# => request body is foo=foo_value&bar=bar_valueYou can add any query parameters and headers to requests.
require "mechanize"
agent = Mechanize.new
agent.request_headers = HTTP::Headers{"Foo" => "Bar"}
params = {"hoge" => "hoge"}
page = agent.get("http://example.com/", params: params)
# The actual URL is http://example.com/?hoge=hogeYou can fill and submit form by using field_with and submit. It enables you to scrape web pages requiring login.
require "mechanize"
agent = Mechanize.new
page = agent.get("#{web page contains login form}")
form = page.forms[0]
form.field_with("email").value = "[email protected]"
form.field_with("password").value = "xxxxxx"
agent.submit(form)
agent.get("#{web page only logged-in users can see}"You can use css selector to search html nodes by using #css method.
This method is from lexbor, so if you want to explore more, please refer the repository.
puts page.css("h1").first.inner_textFor activation, simply setup the log to :debug level
Log.setup("mechanize", :debug)You can access a page which is protected by basic auth, setting username and password for the url.
agent = Mechanize.new
agent.add_auth("http://example.com", "username", "password")
agent.get("http://example.com")- Fork it (https://github.com/Kanezoh/mechanize.cr/fork)
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create a new Pull Request
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent