Git Product home page Git Product logo

jiusanzhou / surferua Goto Github PK

View Code? Open in Web Editor NEW
15.0 2.0 1.0 93 KB

Surfer User-Agent (surferua) is a lightweight and high performance Golang package that generate HTTP User-Agent strings with particular attention to device type. 高性能 Golang 爬虫浏览器随机库。

Home Page: https://labs.zoe.im/surferua

License: Apache License 2.0

Go 95.57% Makefile 3.68% Shell 0.75%
golang user-agent fake-header fake-data useragent go-crawler zoe-lab

surferua's Introduction

surferua

Logo from uasurfer

Surfer UA

High performance User-Agent generator in Golang.

@Zoe on Twitter SurferUA on Sourcegraph


Surfer User-Agent (surferua) is a lightweight Golang package that generate HTTP User-Agent strings with particular attention to device type.

Start

Basic usage

  • Install with Go: go get go.zoe.im/surferua

  • Enjoy it!

Some example code below:

package main

import (
	"fmt"

	"go.zoe.im/surferua"
)

func main() {
	fmt.Println(surferua.New().String())

	// functions depends on your generated inputting data.

	fmt.Println(surferua.NewBot())
	fmt.Println(surferua.NewBotGoogle())
	fmt.Println(surferua.New().Phone().String())
	fmt.Println(surferua.New().Android().String())
	fmt.Println(surferua.New().Desktop().Chrome().String())
}

Customize the User-Agent database

  • Open the config.yml with any editor you like.
  • Edit the cyaml file as you can image what it do
# You can get all User-Agent from: http://www.webapps-online.com/online-tools/user-agent-strings
# If you want to add UA of some version or change them,
# you just need to edit this file and generate code with `go generate` again.
browsers:
  Firefox:
    engine:
      name: Gecko
    version:
      major: [35, 56]
      minor: [35, 56]
      patch: [0, 3]
  Chrome:
    engine:
      name: AppleWebKit
      version:
        major: [534, 603]
        minor: [35, 56]
    version:
      major: [39, 64]
      minor: 0
      patch: [0, 3000]
  Safari:
    engine:
      name: WebKit
      version:
        major: [534, 603]
        minor: [0, 21]
        patch: [0, 10]
    version:
      major: [5, 11]
      minor: [0, 2]
      patch: [0, 10]
platforms:
  Desktop:
    Linux:
    MacOS:
    Windows:
  Phone:
    iOS:
      version:
        major: [6, 11]
        minor: [0, 3]
        patch: [0, 3]
    Android:
      version:
        major: [4, 8]
        minor: [0, 4]
        patch: [0, 4]
bots: # Auto upper the first letter
  google: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
  bing: Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
  yahoo: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
  mj12: Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php)
  simplePie: SimplePie/1.3.1 (Feed Parser; http://simplepie.org; Allow like Gecko)
  blex: Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)
  yandex: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
  scoutJet: Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/)
  duck: DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)
  baidu: Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
  sogou: Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
  sogouOrin: Sogou Orion spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)
  exa: Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails)
  facebook: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
  alexa: ia_archiver (+http://www.alexa.com/site/help/webmasters; [email protected])
  msn: msnbot/2.0b (+http://search.msn.com/msnbot.htm)
  • go generate or make gen in *nix

TODO

  • Add functional way to create UA factory
  • Add testing

surferua's People

Contributors

jiusanzhou avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

76428778fada

surferua's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.