Git Product home page Git Product logo

cdp's Introduction

cdp

Build Status Coverage Status Go Report Card GoDoc

Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language. The bindings are generated (by cdpgen) from the latest tip-of-tree (tot) protocol definitions and are mainly intended for use with Google Chrome or Chromium, however, they can be used with any debug target (Node.js, Edge DevTools Protocol, Safari, etc.) that implement the protocol.

This package can be used for any kind of browser automation, scripting or debugging via the Chrome DevTools Protocol.

A big motivation for cdp is to expose the full functionality of the Chrome DevTools Protocol and provide it in a discoverable and self-documenting manner.

Providing high-level browser automation is a non-goal for this project. That being said, cdp hopes to improve the ergonomics of working with the protocol by providing primitives better suited for Go and automating repetitive tasks.

Features

  • Discoverable API for the Chrome DevTools Protocol (GoDoc, autocomplete friendly)
  • Contexts as a first-class citizen (for timeouts and cancellation)
  • Simple and synchronous event handling (no callbacks)
  • Concurrently safe
  • No silent or hidden errors
  • Do what the user expects
  • Match CDP types to Go types wherever possible
  • Separation of concerns (avoid mixing CDP and RPC)

Installation

$ go get -u github.com/mafredri/cdp

Documentation

See API documentation for package, API descriptions and examples. Examples can also be found in this repository, see the simple, advanced, logging and incognito examples.

Usage

The main packages are cdp and rpcc, the former provides the CDP bindings and the latter handles the RPC communication with the debugging target.

To connect to a debug target, a WebSocket debugger URL is needed. For example, if Chrome is running with --remote-debugging-port=9222 the debugger URL can be found at localhost:9222/json. The devtool package can also be used to query the DevTools JSON API (see example below).

Here is an example of using cdp:

package main

import (
	"bufio"
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"time"

	"github.com/mafredri/cdp"
	"github.com/mafredri/cdp/devtool"
	"github.com/mafredri/cdp/protocol/dom"
	"github.com/mafredri/cdp/protocol/page"
	"github.com/mafredri/cdp/rpcc"
)

func main() {
	err := run(5 * time.Second)
	if err != nil {
		log.Fatal(err)
	}
}

func run(timeout time.Duration) error {
	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()

	// Use the DevTools HTTP/JSON API to manage targets (e.g. pages, webworkers).
	devt := devtool.New("http://127.0.0.1:9222")
	pt, err := devt.Get(ctx, devtool.Page)
	if err != nil {
		pt, err = devt.Create(ctx)
		if err != nil {
			return err
		}
	}

	// Initiate a new RPC connection to the Chrome DevTools Protocol target.
	conn, err := rpcc.DialContext(ctx, pt.WebSocketDebuggerURL)
	if err != nil {
		return err
	}
	defer conn.Close() // Leaving connections open will leak memory.

	c := cdp.NewClient(conn)

	// Open a DOMContentEventFired client to buffer this event.
	domContent, err := c.Page.DOMContentEventFired(ctx)
	if err != nil {
		return err
	}
	defer domContent.Close()

	// Enable events on the Page domain, it's often preferrable to create
	// event clients before enabling events so that we don't miss any.
	if err = c.Page.Enable(ctx); err != nil {
		return err
	}

	// Create the Navigate arguments with the optional Referrer field set.
	navArgs := page.NewNavigateArgs("https://www.google.com").
		SetReferrer("https://duckduckgo.com")
	nav, err := c.Page.Navigate(ctx, navArgs)
	if err != nil {
		return err
	}

	// Wait until we have a DOMContentEventFired event.
	if _, err = domContent.Recv(); err != nil {
		return err
	}

	fmt.Printf("Page loaded with frame ID: %s\n", nav.FrameID)

	// Fetch the document root node. We can pass nil here
	// since this method only takes optional arguments.
	doc, err := c.DOM.GetDocument(ctx, nil)
	if err != nil {
		return err
	}

	// Get the outer HTML for the page.
	result, err := c.DOM.GetOuterHTML(ctx, &dom.GetOuterHTMLArgs{
		NodeID: &doc.Root.NodeID,
	})
	if err != nil {
		return err
	}

	fmt.Printf("HTML: %s\n", result.OuterHTML)

	// Capture a screenshot of the current page.
	screenshotName := "screenshot.jpg"
	screenshotArgs := page.NewCaptureScreenshotArgs().
		SetFormat("jpeg").
		SetQuality(80)
	screenshot, err := c.Page.CaptureScreenshot(ctx, screenshotArgs)
	if err != nil {
		return err
	}
	if err = ioutil.WriteFile(screenshotName, screenshot.Data, 0644); err != nil {
		return err
	}

	fmt.Printf("Saved screenshot: %s\n", screenshotName)

	pdfName := "page.pdf"
	f, err := os.Create(pdfName)
	if err != nil {
		return err
	}

	pdfArgs := page.NewPrintToPDFArgs().
		SetTransferMode("ReturnAsStream") // Request stream.
	pdfData, err := c.Page.PrintToPDF(ctx, pdfArgs)
	if err != nil {
		return err
	}

	sr := c.NewIOStreamReader(ctx, *pdfData.Stream)
	r := bufio.NewReader(sr)

	// Write to file in ~r.Size() chunks.
	_, err = r.WriteTo(f)
	if err != nil {
		return err
	}

	err = f.Close()
	if err != nil {
		return err
	}

	fmt.Printf("Saved PDF: %s\n", pdfName)

	return nil
}

For more information, consult the documentation.

Acknowledgements

The Go implementation of gRPC (grpc-go) has been a source of inspiration for some of the design decisions made in the cdp and rpcc packages. Some ideas have also been borrowed from the net/rpc package from the standard library.

Resources

cdp's People

Contributors

mafredri avatar nya3jp avatar jmuk avatar fd0 avatar powdercloud avatar marefr avatar rtomayko avatar jasonparekh avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.