mongodb-labs / mongoreplay Goto Github PK

JavaScript 1.00% Shell 0.89% Go 97.27% Makefile 0.01% C 0.81% C++ 0.03%

mongoreplay's Introduction

Mongoreplay

Mongoreplay is no longer supported or maintained.

Purpose

mongoreplay is a traffic capture and replay tool for MongoDB. It can be used to inspect commands being sent to a MongoDB instance, record them, and replay them back onto another host at a later time.

Use cases

Preview how well your database cluster would perform a production workload under a different environment (storage engine, index, hardware, OS, etc.)
Reproduce and investigate bugs by recording and replaying the operations that trigger them
Inspect the details of what an application is doing to a mongo cluster (i.e. a more flexible version of mongosniff)

Quickstart

Building Tools

To build the tools, you need to have Go version 1.9 and up. go get will not work; you need to clone the repository to build it.

git clone https://github.com/mongodb-labs/mongoreplay.git
cd mongodb-labs/mongoreplay.git

To use build/test scripts in the repo, you MUST set GOROOT to your Go root directory.

export GOROOT=/usr/local/go

Quick build

The build.sh script builds all the tools, placing them in the bin directory. Pass any build tags (like ssl or sasl) as additional command line arguments.

./build.sh
./build.sh ssl
./build.sh ssl sasl

Manual Build

Source set_goenv.sh and run the set_goenv function to setup your GOPATH and architecture-specific configuration flags:

. ./set_goenv.sh
set_goenv

Set the environment variable to use local vendor folder Pass tags to the go build command as needed in order to build the tools with support for SSL and/or SASL. For example:

mkdir bin
export GO111MODULE=on
export GOSUMDB=off
export GOFLAGS=-mod=vendor

go build -o bin/mongoreplay main/mongoreplay.go
go build -o bin/mongoreplay -tags ssl main/mongoreplay.go
go build -o bin/mongoreplay -tags "ssl sasl" main/mongoreplay.go

Use Mongoreplay

Please follow the instructions in https://docs.mongodb.com/manual/reference/program/mongoreplay/

Testing

To run unit and integration tests:

./runTests.sh

If TOOLS_TESTING_UNIT is set to "true" in the environment, unit tests will run. Delete the environment variable will disable unittest test. If TOOLS_TESTING_INTEGRATION is set to "true" in the environment, integration tests will run. Delete the environment variable will disable integration test.

Integration tests require a mongod (running on port 33333) while unit tests do not.

To run the tests inside pcap_test.go, you need to download the testing pcap files from Amazon S3 to mongoreplay/testPcap bucket: boxes.10gen.com path: build/mongotape/ If the pcap files are not available, the tests inside pcap_test will be skpped.

mongoreplay's People

Contributors

Stargazers

Watchers

Forkers

huan-mongo mtsmfm mlaspina justdba sunny4249 aaronkalair hudl kyontan myzhan fsnow

mongoreplay's Issues

mongoreplay play outputs `context.Execute error: error executing op: Closed explicitly`

Does this tool replay the same amount of operations even if I got this message?
If so, I think I can ignore this message though.

How to reproduce

Download this file https://mtsmfm.s3-ap-northeast-1.amazonaws.com/mongoreplay/test.playback.gz
Run the following command:

$ bin/mongoreplay play --speed 10 -p test.playback.gz --host mongo --gzip

https://gist.github.com/mtsmfm/38ec676b4119104a309745c7c321e829

Environment

golang:1.14.3-buster

Here's the VSCode remote container setting with docker-compose.yml

mtsmfm@b6c6e58

Additional info

I recorded this playback with the following queries:

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'mongo'
end

connection_count = 2000
loop_count = 1000

Mongo::Logger.logger.level = Logger::WARN
client = Mongo::Client.new("mongodb://mongo/testing", max_pool_size: connection_count, min_pool_size: connection_count)

collection = client['numbers']

collection.drop

connection_count.times do |index|
  collection.insert_one(index: index, value: 0)
end

threads = connection_count.times.map do |i|
  Thread.new(i) do |index|
    loop_count.times do |n|
      collection.update_one({index: index}, {value: n})
    end
  end
end

threads.each(&:join)

mongoreplay play feature doesnt seem to support secondarypreferred option

When trying to play the playback file on a secondary it didnt seem to work. Does it not support or am I missing something in the connection string?

--host mongodb://Admin:[email protected]:27017/?authSource=admin&readPreference=secondaryPreferred

--host mongodb://Admin:[email protected]:27017/?replicaSet=replica1&authSource=admin&readPreference=secondaryPreferred

linux build failed

go: downloading golang.org/x/sys v0.0.0-20190405154228-4b34438f7a67
go: downloading golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3
go: downloading github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d
go: downloading github.com/jtolds/gls v4.20.0+incompatible
go: downloading github.com/spacemonkeygo/spacelog v0.0.0-20160606222522-f936fb050dc6
go: downloading github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1
Building mongoreplay...
# github.com/google/gopacket/pcap
../go/pkg/mod/github.com/google/[email protected]/pcap/pcap_unix.go:34:18: fatal error: pcap.h: No such file or directory
 #include <pcap.h>
                  ^
compilation terminated.
Error building mongoreplay
build.sh: line 39: ./bin/mongoreplay: cannot execute binary file