google / mako Goto Github PK

View Code? Open in Web Editor NEW

57.0 10.0 14.0 3.35 MB

Mako - A performance testing library and service

License: Apache License 2.0

C++ 84.34% Go 10.91% C 0.07% Shell 0.39% Starlark 4.29%

mako's Introduction

Mako - A performance testing library and service

Mako is a set of tools for pre-production performance testing. It offers:

data storage
charting
statistical aggregation
automated regression analysis
workflows for triaging potential regressions

Mako is narrowly focused on meeting the performance testing needs of Google open source projects. At this time, Mako is not intended to be used by projects outside this limited scope.

Using Mako

Performance tests that use Mako

Mako client libraries are designed to be called from a client project’s performance test code. See the client project’s documentation for how to run their performance tests.

Since Mako performance tests upload data to https://mako.dev, they require access rights to run. Please see ACCESS.md.

Mako Dashboard

Results from performance tests that use Mako are visible to the world. Visit https://mako.dev to browse the results. Find dashboard documentation at https://mako.dev/help.

Mako Command-line Tool

The Mako command-line tool can be used to gain programmatic access to the same Mako data that’s available on the dashboard. This tool is used by benchmark owners to manage their benchmarks and runs.

To learn about how to use the Mako command-line tool, please read CLI.md.

Writing a new performance test using Mako

If you’re writing a new performance test that will use a Mako client to store results in https://mako.dev and to guard against performance regressions, please read GUIDE.md.

There are example performance tests in the examples/ folder.

Accessing Mako data with the storage libaries

The most common programmatic use of Mako is to create a new run from performance test data, and for that use case we recommend Quickstore. But if you need different kinds of access to the data (e.g. updating or deleting existing data), you’ll want to use the C++ or Go storage library.

mako's People

Contributors

Stargazers

Watchers

Forkers

grantr montaro pubfork chaodaig neotim hanhanw gyohuangxin melodylail isabella232 sfc-gh-psteijn phoenixdigitalfx ghas-results

mako's Issues

Please whitelist agones.dev project to use mako.dev

We are willing to use mako.dev for performance benchmarks.

The project repo:
https://github.com/googleforgames/agones

Note that while doing:
mako create_benchmark

I receive such an error:

 permission denied to write: caller [email protected] must be whitelisted. Contact [email protected] to request to be added.

Please allowlist exposure-notifications-verification-server project to use mako.dev

Hi Mako folks,

Cloud you please add exposure-notifications-verification-server project to the allowlist? We are planing to use mako for performance testing.

Thanks!

Errors when using Microservice Docker image

Hi, I followed the instruction Microservice Docker image

But, got errors:

/app/go/internal/quickstore_microservice/quickstore_microservice_mako: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /app/go/internal/quickstore_microservice/quickstore_microservice_mako)
/app/go/internal/quickstore_microservice/quickstore_microservice_mako: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /app/go/internal/quickstore_microservice/quickstore_microservice_mako)

Interested in using Mako for an open source project

Hi,

My name's Mou Sun, from Google's Firebase engprod team. Our team are doing SDK related performance testing, and we'd like to try Mako.

Best,
Mou

CLI build (and probably others) is broken on mac

macbookpro2 ~> which gcc
/usr/bin/gcc

macbookpro2 ~> gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin19.3.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

./bazel-0.24.1-darwin-x86_64 build --cxxopt='-std=c++14'

Gives the following errors:

INFO: Analysed target //cli:mako (83 packages loaded, 9603 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel/935c1634a455af65486597a2cbbf99c8/external/com_google_protobuf/BUILD:106:1: C++ compilation of rule '@com_google_protobuf//:protobuf_lite' failed (Exit 1) cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 36 argument(s) skipped)
 
Use --sandbox_debug to see verbose messages from the sandbox
external/com_google_protobuf/src/google/protobuf/arena.cc:147:1: warning: attribute declaration must precede definition [-Wignored-attributes]
PROTOBUF_FUNC_ALIGN(32)
^
/usr/local/include/google/protobuf/port_def.inc:207:51: note: expanded from macro 'PROTOBUF_FUNC_ALIGN'
#define PROTOBUF_FUNC_ALIGN(bytes) __attribute__((aligned(bytes)))
                                                  ^
/usr/local/include/google/protobuf/arena_impl.h:110:9: note: previous definition is here
  void* AllocateAligned(size_t n) {
        ^
external/com_google_protobuf/src/google/protobuf/arena.cc:148:18: error: redefinition of 'AllocateAligned'
void* ArenaImpl::AllocateAligned(size_t n) {
                 ^
/usr/local/include/google/protobuf/arena_impl.h:110:9: note: previous definition is here
  void* AllocateAligned(size_t n) {
        ^
external/com_google_protobuf/src/google/protobuf/arena.cc:192:47: error: redefinition of 'GetSerialArenaFast'
inline PROTOBUF_ALWAYS_INLINE bool ArenaImpl::GetSerialArenaFast(
                                              ^
/usr/local/include/google/protobuf/arena_impl.h:331:31: note: previous definition is here
  PROTOBUF_ALWAYS_INLINE bool GetSerialArenaFast(SerialArena** arena) {
                              ^
1 warning and 2 errors generated.
Target //cli:mako failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 3.843s, Critical Path: 1.91s
INFO: 19 processes: 19 darwin-sandbox.
FAILED: Build did NOT complete successfully

Please whitelist Asylo project to use Mako

The project is github.com/google/asylo

Is it OK to post owners here ?

Thanks!

Use klog instead of glog

https://github.com/kubernetes/klog

Reasons why are on the page and copied below.

Besides replacing the three import uses and change glog->klog, a call to klog.InitFlags(nil) must also be made; at package initialization time is just fine.

The bad design of glog makes it hard to interact with other systems.

The decision to create klog was one that wasn't made lightly, but it was necessary due to some drawbacks that are present in glog. Ultimately, the fork was created due to glog not being under active development.

glog presents a lot "gotchas" and introduces challenges in containerized environments, all of which aren't well documented.
glog doesn't provide an easy way to test logs, which detracts from the stability of software using it

Use google.golang.org/protobuf instead of github.com/golang/protobuf

As stated https://github.com/golang/protobuf#go-support-for-protocol-buffers, that:

This module (github.com/golang/protobuf) contains Go bindings for protocol buffers.

It has been superseded by the google.golang.org/protobuf module, which contains an updated and simplified API, support for protobuf reflection, and many other improvements. We recommend that new code use the google.golang.org/protobuf module.

Make bazel build compatible with Bazel 1.x

We're currently building and testing with Bazel 0.28.1. The build is known to be broken on Bazel 1.x. Ideally we'd be compatible with both Bazel 0.X and 1.X, but if we had to choose I think we should choose 1.X.

Whitelist Request

Dear google team,

Is it possible for my open source project to be whitelisted for Mako?

I plan to try it out and quite interested in the dashboard results.

Compiled proto and gRPC files in the repo

It appears that some files in this repo are compiled .proto files.

For example:
https://github.com/google/mako/blob/master/internal/quickstore_microservice/proto/quickstore_go_proto/quickstore.pb.go
https://github.com/google/mako/blob/master/internal/quickstore_microservice/proto/quickstore_go_grpc_pb/quickstore.pb.go

Can you provide context on why you have compiled protos here? Bazel should be able to generate these from .proto definitions, right?..

Any plans to make the server components available?

first

Determine how to accept data from non-Google users

Mako cannot yet accept data from non-Google users because we have not yet decided how to best license this data in a way that protects both Mako users and Google. It's our high priority to rectify this soon, in order to maximize the usefulness of Mako for the open source projects that will use Mako and for their communities of contributors. It’s our intention that all contributors feel comfortable and free using Mako to assess the performance of these software projects.

We’re considering many possibilities, and we’re getting advice from experts and interested parties. We’ll update this issue when we know more.

google / mako Goto Github PK

mako's Introduction

Mako - A performance testing library and service

Using Mako

Performance tests that use Mako

Mako Dashboard

Mako Command-line Tool

Writing a new performance test using Mako

Accessing Mako data with the storage libaries

mako's People

Contributors

Stargazers

Watchers

Forkers

mako's Issues

Recommend Projects

Recommend Topics

Recommend Org