google / perf_data_converter Goto Github PK
View Code? Open in Web Editor NEWTool to convert Linux perf files to the profile.proto format used by pprof
License: BSD 3-Clause "New" or "Revised" License
Tool to convert Linux perf files to the profile.proto format used by pprof
License: BSD 3-Clause "New" or "Revised" License
Not experienced enough how bazel build system works. I am running into this issue when building.
$ bazel build src:perf_to_profile
ERROR: /home/mschulte/.local/opt/perf_data_converter/src/quipper/BUILD:25:1: every rule of type proto_library implicitly depends upon the target '@com_google_protobuf//:protoc', but this target could not be found because of: error loading package '@com_google_protobuf//': Unable to find package for @rules_java//java:defs.bzl: The repository '@rules_java' could not be resolved.
ERROR: Analysis of target '//src:perf_to_profile' failed; build aborted: error loading package '@com_google_protobuf//': Unable to find package for @rules_java//java:defs.bzl: The repository '@rules_java' could not be resolved.
INFO: Elapsed time: 0.105s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 21 targets configured)
currently loading: @com_google_protobuf//
Here is some potentially useful information about my system that the issue could depend on
g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Build label: 0.26.1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Jun 6 11:05:05 2019 (1559819105)
Build timestamp: 1559819105
Build timestamp as int: 1559819105
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (build 1.8.0_292-8u292-b10-0ubuntu1~18.04-b10)
OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)
javac 11.0.11
libprotoc 3.6.1
This is to make sure quipper builds with libc++ as chromium os uses the github copy of quipper.
i find that you get addr from perf record.
and add addr with -1 to get right addr.
i find this could change the addr's kernel symbol.
give /proc/kallsyms:
[root@plat-sh-infra-testing-common-parca001 perfcheck]# cat /proc/kallsyms | head -n 34175 | tail
ffffffff92d83fe0 T tty_unlock_slave
ffffffff92d83fff T __lock_text_end
ffffffff92d84000 t perf_event_nmi_handler
ffffffff92d84000 T __kprobes_text_start
ffffffff92d84050 t perf_ibs_nmi_handler
ffffffff92d840c0 T save_paranoid
ffffffff92d84200 t common_interrupt
ffffffff92d8436a t ret_from_intr
ffffffff92d8437f t exit_intr
ffffffff92d8439d t retint_with_reschedule
the perf script would get ret_from_intr for addr ffffffff92d8436a:
[root@plat-sh-infra-testing-common-parca001 .parca-agent]# perf script | head -n 4240 | tail
ffffffffc04961f5 virtnet_poll+0x265 ([kernel.kallsyms])
ffffffff92c5057f net_rx_action+0x26f ([kernel.kallsyms])
ffffffff926a5435 __do_softirq+0xf5 ([kernel.kallsyms])
ffffffff92d9142c call_softirq+0x1c ([kernel.kallsyms])
ffffffff9262f715 do_softirq+0x65 ([kernel.kallsyms])
ffffffff926a57b5 irq_exit+0x105 ([kernel.kallsyms])
ffffffff92d92886 __irqentry_text_start+0x56 ([kernel.kallsyms])
ffffffff92d8436a ret_from_intr+0x0 ([kernel.kallsyms])
50c5d3 ZSTD_compressBlock_greedy_extDict_row+0x2d3 (/root/zstd/zstd)
9f5573aeed0b19a2 [unknown] ([unknown])
but with addr--, we would get common_interrupt for addr ffffffff92d8436a.
so with addr--, our result would diff from perf script output.
which is correct?
Hello up there.
I'm on latest perf_to_profile
(b86dd52) and on latest pprof
(4b386c29ec) and I've tried to use this tools to have better visualsiation for my call-graph profiles. While for user-space it seems to be working, I've found that
all kernel symbols are collapesd to `[[kernel.kallsyms]]` by perf_to_profile.
For example:
root@deco:/home/kirr# perf record dd if=/dev/urandom of=/dev/null bs=1K count=1M
1048576+0 записей получено
1048576+0 записей отправлено
1073741824 bytes (1,1 GB, 1,0 GiB) copied, 6,90417 s, 156 MB/s
[ perf record: Woken up 4 times to write data ]
[ perf record: Captured and wrote 1.073 MB perf.data (27575 samples) ]
root@deco:/home/kirr# perf report --stdio |head -50
no symbols found in /bin/dd, maybe install a debug package?
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 27K of event 'cycles:ppp'
# Event count (approx.): 23116095014
#
# Overhead Command Shared Object Symbol
# ........ ....... ................ ....................................
#
52.98% dd [kernel.vmlinux] [k] _extract_crng
31.96% dd [kernel.vmlinux] [k] chacha20_block
4.82% dd [kernel.vmlinux] [k] copy_user_enhanced_fast_string
2.41% dd [kernel.vmlinux] [k] _raw_spin_lock_irqsave
1.78% dd [kernel.vmlinux] [k] _raw_spin_unlock_irqrestore
1.10% dd [kernel.vmlinux] [k] urandom_read
0.84% dd [kernel.vmlinux] [k] __check_object_size
0.39% dd [kernel.vmlinux] [k] check_stack_object
0.38% dd [kernel.vmlinux] [.] system_call_fast_compare_end
0.30% dd [kernel.vmlinux] [k] __virt_addr_valid
0.27% dd [kernel.vmlinux] [k] entry_SYSCALL_64
0.26% dd [kernel.vmlinux] [k] extract_crng
0.23% dd [kernel.vmlinux] [k] _copy_to_user
0.21% dd [kernel.vmlinux] [k] fsnotify
0.20% dd [unknown] [k] 0x00007f67816e374e
0.14% dd [unknown] [k] 0x00007f67816e37ae
0.13% dd [kernel.vmlinux] [k] memset_erms
0.12% dd [kernel.vmlinux] [k] _crng_backtrack_protect
0.11% dd [kernel.vmlinux] [k] vfs_write
0.10% dd [kernel.vmlinux] [k] common_file_perm
0.09% dd [kernel.vmlinux] [k] vfs_read
0.09% dd [kernel.vmlinux] [k] security_file_permission
0.09% dd [kernel.vmlinux] [k] __fdget_pos
0.08% dd [kernel.vmlinux] [k] __fsnotify_parent
0.08% dd [kernel.vmlinux] [k] __fget_light
0.08% dd [kernel.vmlinux] [k] rw_verify_area
0.07% dd [kernel.vmlinux] [k] aa_file_perm
0.07% dd [kernel.vmlinux] [k] sys_write
0.05% dd [kernel.vmlinux] [k] crng_backtrack_protect
0.04% dd [kernel.vmlinux] [k] sys_read
0.04% dd [kernel.vmlinux] [k] entry_SYSCALL_64_fastpath
0.03% dd dd [.] 0x0000000000002ef8
0.03% dd dd [.] 0x0000000000004b38
0.03% dd [kernel.vmlinux] [k] apparmor_file_permission
0.02% dd dd [.] 0x0000000000004a80
0.02% dd dd [.] 0x0000000000002ef0
0.02% dd dd [.] 0x0000000000004bd6
0.01% dd dd [.] 0x0000000000004bb2
0.01% dd dd [.] 0x0000000000004c73
root@deco:/home/kirr# ./bin/perf_to_profile perf.data perf.pprof
[INFO:quipper/perf_reader.cc:847] Number of events stored: 27757
[ERROR:quipper/perf_parser.cc:217] Unknown event type: 79
[INFO:perf_data_handler.cc:414] guessing main for pid: 6901 /bin/dd
root@deco:/home/kirr# ./src/tools/go/g.env/bin/pprof perf.pprof
File: perf_4.13
perf-version:4.13.13
perf-command:/usr/bin/perf_4.13 record dd if=/dev/urandom of=/dev/null bs=1K count=1M
Type: cycles:ppp_event
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 22952170132, 99.29% of 23116095014 total
Dropped 5 nodes (cum <= 115580475)
flat flat% sum% cum cum%
22952170132 99.29% 99.29% 22952170132 99.29% [[kernel.kallsyms]]
From outside from a quick glance that seems to be happenning due to perf_to_profilie not handling kernel "dso" properly.
For workloads that depend performance-wise on proper in-kernel behaviour, instead of representing whole kernel integrally as one [[kernel.kallsyms]]
, it would be good to have pprof visualization (graph + flames) to be working for kernel symbols too.
Thanks beforehand,
Kirill
/cc @aalexand
I generated perf data with a clang linux build
$ perf record -e cycles:pp --output=arm64_clang_built_def_config.data --call-graph lbr -- make LLVM=1 -j72 ARCH=arm64
Which had a (quite large) size of:
$ du -h arm64_clang_built_def_config.data
18G arm64_clang_built_def_config.data
Invoking the perf_to_profile
tool yields this output:
$ perf_to_profile -i arm64_clang_built_def_config.data -o profile_arm64_clang_built_defconfig
[ERROR:src/quipper/perf_reader.cc:197] perf_file_section.size 19150652112 cannot exceed remaining perf data input size 1970824676 at offset 824
[ERROR:src/quipper/perf_reader.cc:813] Error reading header file section info.
[ERROR:src/perf_data_converter.cc:749] Could not read input perf.data
[FATAL:src/perf_to_profile.cc:33] Expected profile vector to have one element.
For reference, that first number 19150652112
is ~19gb and the second 1970824676
is ~1.9gb.
run the perf_to_profile
tool on a large perf.data. (I am not exactly sure how large of a file is required to reproduce, but I know for sure >=18gb does the trick)
edit: as per my comment (and fixes) below, the critical size is ~2.14gb.
Triggered when attempting to convert a perf capture of an OCaml program. Unfortunately, I can't share the code, but I might be able to produce a freestanding reproduction case if someone's available to chase this.
When trying to compile this project on both Ubuntu 18.04 and Debian 9 operating systems, I get the following error:
external/boringssl/src/crypto/fipsmodule/bcm.c:32:10: fatal error: fips_break_test.h: No such file or directory
#include "fips_break_test.h"
^~~~~~~~~~~~~~~~~~~
compilation terminated.
Target //src:perf_to_profile failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 68.100s, Critical Path: 9.56s
INFO: 534 processes: 8 internal, 526 processwrapper-sandbox.
Any ideas?
Thanks 🙏🏻
The branch stack entry reader will warn about reserved bits being non-zero, so the reader needs to know about all the fields that the kernel is producing:
diff --git a/src/quipper/kernel/perf_event.h b/src/quipper/kernel/perf_event.h
index 9f44a26..2a9ac28 100644
--- a/src/quipper/kernel/perf_event.h
+++ b/src/quipper/kernel/perf_event.h
@@ -1065,7 +1065,11 @@ struct perf_branch_entry {
in_tx : 1, /* in transaction */
abort : 1, /* transaction abort */
cycles : 16, /* cycle count to last branch */
- reserved : 44;
+ type : 4, /* branch type, or 15 to indicate new type */
+ spec : 2, /* speculation outcome */
+ new_type : 4, /* branch type (new types) */
+ priv : 3, /* privilege level */
+ reserved : 31;
};
} // namespace quipper
diff --git a/src/quipper/kernel/perf_internals.h b/src/quipper/kernel/perf_internals.h
index 3906e1f..f72122a 100644
--- a/src/quipper/kernel/perf_internals.h
+++ b/src/quipper/kernel/perf_internals.h
@@ -259,7 +259,11 @@ struct branch_flags {
u64 in_tx : 1;
u64 abort : 1;
u64 cycles : 16;
- u64 reserved : 44;
+ u64 type : 4;
+ u64 spec : 2;
+ u64 new_type : 4;
+ u64 priv : 3;
+ u64 reserved : 31;
};
struct branch_entry {
Probably, it should save at least some of this new information in the entry. The 'type' field may be needed by whoever is consuming this infomation, in order to ignore or distinguish some kinds of branches (e.g. interrupts). If type==15, new_type is used as an extended type field.
(Support for 'type' seems to have been added by the recent "update to latest internal version" merge, but "new_type" is still missing and we are already seeing kernels make use of that.)
$ sudo apt-get -y install g++ git libelf-dev libcap-dev linux-tools-`uname -r`
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package linux-tools-4.9.0-6-amd64
E: Couldn't find any package by glob 'linux-tools-4.9.0-6-amd64'
E: Couldn't find any package by regex 'linux-tools-4.9.0-6-amd64'
bazel version: 3.5.0
/root/.cache/bazel/_bazel_root/a08c2e4811c846650b733c6fc815a920/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in
ERROR: /root/.cache/bazel/_bazel_root/a08c2e4811c846650b733c6fc815a920/external/boringssl/BUILD:203:11: @platforms//os:linux is not a valid configuration key for @boringssl//:crypto
cc_library(
name = "crypto",
srcs = crypto_sources + crypto_internal_headers + crypto_sources_asm,
hdrs = crypto_headers + fips_fragments,
copts = boringssl_copts_c11,
includes = ["src/include"],
linkopts = select({
# Android supports pthreads, but does not provide a libpthread
# to link against.
"@platforms//os:android": [],
"@platforms//os:macos": [],
"@platforms//os:ios": [],
"@platforms//os:tvos": [],
"@platforms//os:watchos": [],
"@platforms//os:windows": ["-defaultlib:advapi32.lib"],
"//conditions:default": ["-lpthread"],
}),
visibility = ["//visibility:public"],
)
It is currently logged as an error, but it is not very much actionable for users, so it should be a warning.
[ERROR:perf_data_handler.cc:310] stat: missing_callchain_mmap 14317/113208
On Ubuntu Focal:
$ uname -a
Linux mark-t510 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
$ perf -v
perf version 5.11.22
To reproduce, I ran this command. It'd probably work with any other command too (where ninja.dot
is https://graphviz.org/Gallery/directed/ninja.gv.txt):
$ sudo perf record -F 4000 -g --call-graph dwarf -o ninja.data dot -Tsvg ninja.dot
Running this command, I get these fatal errors:
$ projects/perf_data_converter/bazel-bin/src/perf_to_profile -i ninja.data -o ninja_pprof.profile
[WARNING:src/quipper/perf_reader.cc:1247] Skipping 88 bytes of metadata: HEADER_CPU_TOPOLOGY
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event PERF_RECORD_CPU_MAP
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[ERROR:src/quipper/sample_info_reader.cc:385] PERF_SAMPLE_REGS_USER is not yet supported.
[ERROR:src/quipper/perf_reader.cc:1023] Couldn't read event PERF_RECORD_SAMPLE
[ERROR:src/perf_data_converter.cc:718] Could not read input perf.data
[FATAL:src/perf_to_profile.cc:33] Expected profile vector to have one element. Got:0
Looks like the root cause is the PERF_SAMPLE_REGS_USER is not yet supported.
message.
Everything works fine if I drop the --call-graph dwarf
and use the default frame-pointer-based callstack unwinding:
$ sudo perf record -F 4000 -g -o ninja-g.data dot -Tsvg ninja.dot
$ projects/perf_data_converter/bazel-bin/src/perf_to_profile -i ninja-g.data -o ninja_pprof.profile
[WARNING:src/quipper/perf_reader.cc:1247] Skipping 88 bytes of metadata: HEADER_CPU_TOPOLOGY
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event PERF_RECORD_CPU_MAP
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:src/quipper/perf_reader.cc:1039] Skipping unsupported event UNKNOWN_EVENT_18
[INFO:src/quipper/perf_reader.cc:1030] Number of events stored: 441
[INFO:src/quipper/perf_parser.cc:267] Parser processed: 183 MMAP/MMAP2 events, 2 COMM events, 0 FORK events, 1 EXIT events, 252 SAMPLE events, 98 of these were mapped, 0 SAMPLE events with a data address, 0 of these were mapped
[ERROR:src/quipper/perf_parser.cc:289] Only 38% of samples had all locations mapped to a module, expected at least 95%
[INFO:src/perf_data_handler.cc:87] Using the build id found for the file name: [kernel.kallsyms], build id: 9ae838e603db988d68009a8c3deb0cb1184c6824.
[WARNING:src/perf_data_handler.cc:463] stat: missing_callchain_mmap 593/1659
perf_to_profile drops symbol names when converting from perf output to pprof format.
$ perf record tree /usr
$ perf report
Samples: 4K of event 'cpu-clock:uhH', Event count (approx.): 1236250000
Overhead Command Shared Object Symbol
12.52% tree tree [.] 0x000000000000c964
11.06% tree libc-2.27.so [.] vfprintf
4.63% tree libc-2.27.so [.] write
4.33% tree libc-2.27.so [.] __lxstat64
3.84% tree libc-2.27.so [.] _IO_file_xsputn
3.01% tree libc-2.27.so [.] __fprintf_chk
2.75% tree libc-2.27.so [.] __strcoll_l
2.43% tree libc-2.27.so [.] c32rtomb
1.92% tree libc-2.27.so [.] malloc
1.80% tree libc-2.27.so [.] 0x000000000016af98
1.72% tree libc-2.27.so [.] 0x00000000000e00a8
1.62% tree libc-2.27.so [.] 0x000000000018e5a5
1.54% tree libc-2.27.so [.] cfree
1.38% tree libc-2.27.so [.] iswprint
1.01% tree libc-2.27.so [.] 0x000000000018e590
0.87% tree libc-2.27.so [.] __open_nocancel
0.87% tree libc-2.27.so [.] 0x0000000000169d90
0.75% tree libc-2.27.so [.] __close_nocancel
0.75% tree libc-2.27.so [.] __fxstat64
0.75% tree libc-2.27.so [.] readdir64
0.69% tree libc-2.27.so [.] 0x0000000000169d7a
0.69% tree libc-2.27.so [.] 0x000000000018e59c
0.67% tree libc-2.27.so [.] 0x0000000000169d98
0.67% tree libc-2.27.so [.] 0x000000000018e230
0.65% tree libc-2.27.so [.] 0x000000000018e69d
0.61% tree tree [.] 0x000000000000d672
0.59% tree tree [.] 0x0000000000007bbe
0.57% tree libc-2.27.so [.] _IO_default_xsputn
0.53% tree libc-2.27.so [.] 0x000000000018e690
0.51% tree tree [.] 0x000000000000c949
0.51% tree tree [.] 0x000000000000c95c
0.44% tree libc-2.27.so [.] 0x0000000000169d6c
0.42% tree libc-2.27.so [.] 0x000000000018e22b
0.42% tree tree [.] 0x000000000000d675
0.40% tree libc-2.27.so [.] 0x00000000000e00b0
0.40% tree libc-2.27.so [.] 0x000000000016af9e
0.38% tree libc-2.27.so [.] _IO_file_overflow
0.38% tree libc-2.27.so [.] 0x000000000018e227
0.36% tree libc-2.27.so [.] _IO_file_write
0.36% tree libc-2.27.so [.] mbstowcs
0.34% tree libc-2.27.so [.] 0x0000000000169d60
0.34% tree libc-2.27.so [.] 0x0000000000169de5
0.32% tree libc-2.27.so [.] _dl_mcount_wrapper_check
...
After conversion pprof data only contains the "Shared Object" name pprof accumulates all profiles under shared object names instead of symbols.
(pprof) top
Showing nodes accounting for 1236000000, 100% of 1236250000 total
Dropped 1 node (cum <= 6181250)
flat flat% sum% cum cum%
956250000 77.35% 77.35% 956250000 77.35% [libc-2.27.so]
279750000 22.63% 100% 279750000 22.63% [tree]
Would love to be able to get per-thread tags in the output, using this constant:
perf_to_profile
could take a command line argument for this?
The github.com links now seem to redirect to codeload.github.com but Bazel 4.1.0 just reports 404 even though the redirect works in a browser.
The error looks like this:
WARNING: Download from https://github.com/protocolbuffers/protobuf/archive/master.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
ERROR: An error occurred during the fetch of repository 'com_google_protobuf':
Traceback (most recent call last):
File "/home/jamcla02/.cache/bazel/_bazel_jamcla02/b2015f4763ecfa490ff19feafa4074f9/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl
download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/protocolbuffers/protobuf/archive/master.zip] to /home/jamcla02/.cache/bazel/_bazel_jamcla02/b2015f4763ecfa490ff19feafa4074f9/external/com_google_protobuf/temp14020832879063832399/master.zip: GET returned 404 Not Found
I see that PRs are not supported, so I will leave the patch that fixes it here in case anybody has the same issue. It can be applied with git am <patchfile>
:
From 30f5dcd29d92972434f0a9fa1e17ec834d278b9d Mon Sep 17 00:00:00 2001
From: James Clark <[email protected]>
Date: Mon, 2 Aug 2021 11:48:15 +0100
Subject: [PATCH] Fix build
The existing github links redirect to codeload.github.com
and redirection isn't supported by bazel so update them.
They also lost their extensions so add 'type' to force it.
python_rules also changed 'master' to 'main'
---
WORKSPACE | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/WORKSPACE b/WORKSPACE
index c416ba5..16c9be2 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -18,16 +18,18 @@ http_archive(
# rules_python is a dependency for protobuf.
http_archive(
name = "rules_python",
- urls = ["https://github.com/bazelbuild/rules_python/archive/master.tar.gz"],
- strip_prefix = "rules_python-master",
+ urls = ["https://codeload.github.com/bazelbuild/rules_python/tar.gz/main"],
+ strip_prefix = "rules_python-main",
+ type = "tar.gz"
)
# proto_library, cc_proto_library, and java_proto_library rules implicitly
# depend on @com_google_protobuf for protoc and proto runtimes.
http_archive(
name = "com_google_protobuf",
- urls = ["https://github.com/protocolbuffers/protobuf/archive/master.zip"],
+ urls = ["https://codeload.github.com/protocolbuffers/protobuf/zip/master"],
strip_prefix = "protobuf-master",
+ type = "zip"
)
http_archive(
@@ -35,8 +37,9 @@ http_archive(
# Gitiles creates gzip files with an embedded timestamp, so we cannot use
# sha256 to validate the archives. We must rely on the commit hash and
# https. Commits must come from the master-with-bazel branch.
- urls = ["https://github.com/google/boringssl/archive/master-with-bazel.zip"],
+ urls = ["https://codeload.github.com/google/boringssl/zip/master-with-bazel"],
strip_prefix = "boringssl-master-with-bazel",
+ type = "zip"
)
# zlib is a dependency of protobuf.
--
2.28.0
$ bazel build src:perf_to_profile
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/google/boringssl/archive/master-with-bazel.zip) = cec1a5180dc44d083573015694a5cc40a194920a1e7665dc005592a08d959597
DEBUG: Rule 'boringssl' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "cec1a5180dc44d083573015694a5cc40a194920a1e7665dc005592a08d959597"
DEBUG: Call stack for the definition of repository 'boringssl' which is a http_archive (rule definition at /usr/local/google/home/aalexand/.cache/bazel/_bazel_aalexand/f6d7d2335bdf63a7813e232f542cf0f2/external/bazel_tools/tools/build_defs/repo/http.bzl:262:16):
- /usr/local/google/home/aalexand/src/perf_data_converter/WORKSPACE:27:1
INFO: Analyzed target //src:perf_to_profile (19 packages loaded, 1079 targets configured).
INFO: Found 1 target...
INFO: From Compiling src/quipper/perf_reader.cc:
src/quipper/perf_reader.cc: In member function 'bool quipper::PerfReader::ReadCPUTopologyMetadata(quipper::DataReader*, size_t)':
src/quipper/perf_reader.cc:1525:23: warning: comparison of integer expressions of different signedness: 'int' and 'uint32_t' {aka 'unsigned int'} [-Wsign-compare]
for (int i = 0; i < nrcpus; ++i) {
~~^~~~~~~~
INFO: From Compiling src/perf_data_converter.cc:
src/perf_data_converter.cc: In member function 'perftools::ProcessProfiles perftools::{anonymous}::PerfDataConverter::Profiles()':
src/perf_data_converter.cc:663:21: warning: comparison of integer expressions of different signedness: 'int' and 'std::deque<perftools::profiles::Builder>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
for (int i = 0; i < builders_.size(); i++) {
~~^~~~~~~~~~~~~~~~~~
INFO: From Compiling src/quipper/perf_serializer.cc:
src/quipper/perf_serializer.cc: In member function 'bool quipper::PerfSerializer::DeserializeAuxtraceInfoEvent(const quipper::PerfDataProto_AuxtraceInfoEvent&, quipper::event_t*) const':
src/quipper/perf_serializer.cc:1006:21: warning: comparison of integer expressions of different signedness: 'quipper::u64' {aka 'long unsigned int'} and 'int' [-Wsign-compare]
for (u64 i = 0; i < sample.unparsed_binary_blob_priv_data_size(); ++i) {
~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/quipper/perf_serializer.cc: In member function 'const quipper::SampleInfoReader* quipper::PerfSerializer::GetSampleInfoReaderForEvent(const event_t&) const':
src/quipper/perf_serializer.cc:1467:24: warning: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'ssize_t' {aka 'long int'} [-Wsign-compare]
if (array_size <= event_id_pos) {
~~~~~~~~~~~^~~~~~~~~~~~~~~
src/quipper/perf_serializer.cc:1482:24: warning: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'ssize_t' {aka 'long int'} [-Wsign-compare]
if (array_size < event_id_pos) {
~~~~~~~~~~~^~~~~~~~~~~~~~
Target //src:perf_to_profile up-to-date:
bazel-bin/src/perf_to_profile
INFO: Elapsed time: 60.432s, Critical Path: 17.10s
INFO: 560 processes: 560 linux-sandbox.
INFO: Build completed successfully, 567 total actions
When I run converter i get:
[ERROR:src/quipper/sample_info_reader.cc:277] PERF_SAMPLE_REGS_USER is not yet supported.
[ERROR:src/quipper/sample_info_reader.cc:591] Read/skipped 56 bytes, expected 8440 bytes.
[ERROR:src/perf_data_converter.cc:659] Could not read input perf.data
[FATAL:src/perf_to_profile.cc:27] Expected profile vector to have one element.
Hello! We obtain the "perf.data" file by using "perf record -g -a ./my_execution", but it seems that the "perf.data" database does not include mapping information such as function names. After using "perf_to_profile" to convert the "perf.data" file to protobuf file, we still cannot extract function name information from it. Is it possible that our command is incorrect or lacks certain flags? Thanks for your time in advance.
perf record ./src/bitcoind -datadir=mainnet -server=0 -connect=127.0.0.1 -dbcache=16000
I ran the above command to capture a perf of a long running process (3-4 hours) and got the following error:
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/stubs/stringpiece.cc:50] size too big: 18446744072542064660 details: string length exceeds max size
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): size too big: 18446744072542064660 details: string length exceeds max size
fish: Job 1, '../perf_data_converter/result/b…' terminated by signal SIGABRT (Abort)
The perf.data
file generated by the perf
was 3GB. It's too big to attach in the github issues but I can provide it on request.
EDIT: I just saw #123 and should mention that the binary I was using was built on b1c2b82a8ec75b016c36da31c845491c75b9adae
Project build fails for me since yesterday (15.03.2021), it worked fine earlier. The problem seems to lie in boringssl dependency, this change in particular.
$ bazel build src:perf_to_profile
(...)
ERROR: /root/.cache/bazel/_bazel_root/9a84914462919c338e94f245491500b7/external/boringssl/BUILD:151:11: Compiling src/crypto/fipsmodule/bcm.c failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -MD -MF ... (remaining 34 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -MD -MF ... (remaining 34 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
external/boringssl/src/crypto/fipsmodule/bcm.c:82:10: fatal error: fips.c: No such file or directory
#include "fips.c"
^~~~~~~~
compilation terminated.
Target //src:perf_to_profile failed to build
(...)
Build this Dockerfile to reproduce:
FROM ubuntu:18.04
RUN apt update && apt install -y npm g++ git libelf-dev libcap-dev
RUN npm install -g @bazel/bazelisk
RUN git clone https://github.com/google/perf_data_converter.git /usr/src/perf_data_converter
WORKDIR /usr/src/perf_data_converter
# uncomment to make the build succeed
# RUN sed -i 's/\/google\/boringssl/\/pbetkier\/boringssl/' WORKSPACE
RUN bazel build src:perf_to_profile
Note the commented out sed
. After uncommenting the build does pass. It replaces the official boringssl dependency with my fork that moves history before the boringssl change linked in the beginning.
I'm not sure if this is the right place to file this problem, or should I rather file it in the boringssl project. My knowledge of the cpp ecosystem is also rather limited. I hope you can direct me to the right place :)
Hi team,
Quipper, a perf library used in this repro for Google's perf_data_converter, crashes with signs of memory corruption when parsing malformed perf .data files. Quick automated triage info below as you can check if the issue appears to be an exploitable vulnerability or just a crash as no further analysis was performed.
Quipper library code can be found here:
https://github.com/google/perf_data_converter/tree/master/src/quipper
Repro steps:
$ git clone https://github.com/google/perf_data_converter
(install deps and build)
$ ./perf_data_converter/bazel-bin/src/perf_to_profile -i repro.data -o test -f
repro.data (rename from repro.data.txt, had to add .txt to the extension for github issue file upload)
repro.data.txt
observe crash on console and within a debugger
Crash was found using litefuzz to mutate sample perf files. See below repro demonstration and debug output.
$ ./perf_data_converter/bazel-bin/src/perf_to_profile -i repro.data -o test -f
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted (core dumped)
Debug output from gdb from initial crash triage below.
Note: this uses electric-fence malloc debugger, install with sudo apt-get install electric-fence to trigger the instrumented crash as you can see.
$ gdb -q perf_data_converter/bazel-bin/src/perf_to_profile
Reading symbols from perf_data_converter/bazel-bin/src/perf_to_profile...
(No debugging symbols found in perf_data_converter/bazel-bin/src/perf_to_profile)
(gdb) set environment LD_PRELOAD=/usr/lib/libefence.so
(gdb) r -i ~/repro.data -o /tmp/test.data -f
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff79489ab in mprotect () at ../sysdeps/unix/syscall-template.S:78
(gdb) bt
#0 0x00007ffff79489ab in mprotect () at ../sysdeps/unix/syscall-template.S:78
#1 0x00007ffff7dc623e in Page_AllowAccess () from /usr/lib/libefence.so
#2 0x00007ffff7dc5745 in ?? () from /usr/lib/libefence.so
#3 0x00007ffff7dc5e7e in memalign () from /usr/lib/libefence.so
#4 0x00007ffff7ae9b39 in operator new(unsigned long) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00005555555e16ca in __gnu_cxx::new_allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >::allocate(unsigned long, void const*) ()
#6 0x00005555555e15b8 in std::allocator_traits<std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >::allocate(std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >&, unsigned long) ()
#7 0x00005555555e13a6 in std::_Vector_base<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >::_M_allocate(unsigned long) ()
#8 0x0000555555600284 in std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >::_M_default_append(unsigned long) ()
#9 0x00005555555fc17b in std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >::resize(unsigned long) ()
#10 0x00005555555f0328 in quipper::PerfReader::ReadCPUTopologyMetadata(quipper::DataReader*, unsigned long) ()
#11 0x00005555555ee39c in quipper::PerfReader::ReadMetadataWithoutHeader(quipper::DataReader*, unsigned int, unsigned long)::{lambda()#1}::operator()() const ()
#12 0x00005555555ee710 in quipper::PerfReader::ReadMetadataWithoutHeader(quipper::DataReader*, unsigned int, unsigned long) ()
#13 0x00005555555ede94 in quipper::PerfReader::ReadMetadata(quipper::DataReader*) ()
#14 0x00005555555f1717 in quipper::PerfReader::ReadFileData(quipper::DataReader*) ()
#15 0x00005555555e8ce6 in quipper::PerfReader::ReadFromData(quipper::DataReader*) ()
#16 0x00005555555e8b69 in quipper::PerfReader::ReadFromPointer(char const*, unsigned long) ()
#17 0x00005555555799ac in perftools::RawPerfDataToProfiles(void const*, unsigned long, std::map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::less<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, unsigned int, unsigned int, std::map<unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::less, std::allocator<std::pair<unsigned int const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&) ()
#18 0x00005555555746c4 in StringToProfiles(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, unsigned int, unsigned int) ()
#19 0x00005555555731e5 in main ()
(gdb) i r
rax 0xfffffffffffffff4 -12
rbx 0x7ffff703e730 140737337616176
rcx 0x7ffff79489ab 140737347094955
rdx 0x3 3
rsi 0xd6e6c3000 57687158784
rdi 0x7fff8894c000 140735484837888
rbp 0xd6e6c4000 0xd6e6c4000
rsp 0x7fffffffc9d8 0x7fffffffc9d8
r8 0xffffffff 4294967295
r9 0x0 0
r10 0x22 34
r11 0x202 514
r12 0x7ffff703e758 140737337616216
r13 0x7ffff703e758 140737337616216
r14 0xd6e6c2e80 57687158400
r15 0x7ffff7fc8128 140737353908520
rip 0x7ffff79489ab 0x7ffff79489ab <mprotect+11>
eflags 0x10202 [ IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
(gdb) exploitable
Description: Segmentation fault on program counter
Short description: SegFaultOnPc (3/22)
Hash: 434b871bba6df1ba821918d14d98847d.3c60d5a82288d85766b6290f18c589bb
Exploitability Classification: EXPLOITABLE
Explanation: The target tried to access data at an address that matches the program counter. This is likely due to the execution of a branch instruction (ex: 'call') with a bad argument, but it could also be due to execution continuing past the end of a memory region or another cause. Regardless this likely indicates that the program counter contents are tainted and can be controlled by an attacker.
Other tags: AccessViolation (21/22)
Recommendation/fix is to provide more input validation and bounds checking to ensure malformed files that could trigger error conditions are instead handled gracefully.
Sometimes, I failed to clone boringssl from googlesource, so I try to change the url of boringssl to its mirror on github. I can clone it correctly, but bazel always outputs "no such package '@boringssl//': BUILD file not found on package path and referenced by '//src/quipper:binary_data_utils'".
when I build perf-data-converter, I have this error, how I can solve it?
[root@opengauss-master perf_data_converter]# bazel build src:perf_to_profile
Starting local Bazel server and connecting to it...
DEBUG: Rule 'com_google_protobuf' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "7372397031f356341e932c4b0489489e4111152fc3a9e03660ede951e9b2d84d"
DEBUG: Repository com_google_protobuf instantiated at:
/root/perf/perf_data_converter/WORKSPACE:30:13: in
Repository rule http_archive defined at:
/root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in
DEBUG: Rule 'rules_python' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "12b37c991026c4ce136509fd3aeb4a194e98a29c0f328ecac53269a8c0af2906"
DEBUG: Repository rules_python instantiated at:
/root/perf/perf_data_converter/WORKSPACE:20:13: in
Repository rule http_archive defined at:
/root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in
INFO: Repository boringssl instantiated at:
/root/perf/perf_data_converter/WORKSPACE:37:13: in
Repository rule http_archive defined at:
/root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in
INFO: Repository bazel_skylib instantiated at:
/root/perf/perf_data_converter/WORKSPACE:5:13: in
Repository rule http_archive defined at:
/root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in
ERROR: /root/perf/perf_data_converter/src/BUILD:170:11: error loading package '@com_google_protobuf//': at /root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/rules_python/python/defs.bzl:20:6: at /root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/rules_python/python/py_runtime.bzl:19:6: at /root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/rules_python/python/private/common/py_runtime_macro.bzl:16:6: at /root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/rules_python/python/private/common/py_runtime_rule.bzl:21:6: at /root/.cache/bazel/_bazel_root/4a25792f71ccee4f60d59a36fdf7bb13/external/rules_python/python/private/common/py_internal.bzl:23:6: Unable to find package for @rules_python_internal//:py_internal.bzl: The repository '@rules_python_internal' could not be resolved: Repository '@rules_python_internal' is not defined. and referenced by '//src:builder'
ERROR: Analysis of target '//src:perf_to_profile' failed; build aborted:
INFO: Elapsed time: 9.419s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (39 packages loaded, 206 targets configured)
currently loading: @com_google_protobuf//
Fetching https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz
I tried to build with bazel 4.0.0, and got an error:
in proto_lang_toolchain rule @com_google_protobuf//:cc_toolchain: '@com_google_protobuf//:cc_toolchain' does not have mandatory provider 'ProtoInfo'.
system, archlinux:
Linux 5.10.13-arch1-2 #1 SMP PREEMPT Sat, 06 Feb 2021 11:07:29 +0000 x86_64 GNU/Linux
bazel version:
Build label: 4.0.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Jan 21 14:04:24 2021 (1611237864)
Build timestamp: 1611237864
Build timestamp as int: 1611237864
I found bazelbuild/bazel#12887, and try to upgrade protobuf to v13.0, but there are some other dependency issues.
I hava download all the source code, and use the command flowed:
root@x86-centos7:build$ cmake -DCMAKE_INSTALL_PREFIX=/home/name/test/autofdo-master/install -DLLVM_PATH=/home/name/llvm-dev_12x/install/bin ../
when run the make, I got this error:
CMake Error at /usr/local/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:164 (message):
Could NOT find Protobuf (missing: Protobuf_LIBRARIES Protobuf_INCLUDE_DIR)
Call Stack (most recent call first):
/usr/local/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/cmake-3.17/Modules/FindProtobuf.cmake:626 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:181 (find_package)
CMakeLists.txt:513 (config_with_llvm)
how can i resolve this?
perf_data_converter (master) ✔ bazel build src:perf_to_profile k8s [kubernetes-admin@kubernetes]
INFO: Invocation ID: 3c80c53b-bb6b-4ba4-82c2-1e2a29e678f2
DEBUG: Rule 'io_bazel' modified arguments {"sha256": "c290fbcc6de1e1a4d94408d32dd962aec90db6c670020784cf7a495385deb954"}
DEBUG: Rule 'boringssl' modified arguments {"sha256": "65991a30c481a2f753f2bca8f51ca84183d2d8be452514bd1fac1b774e478fb2"}
INFO: Analysed target //src:perf_to_profile (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Linking external/boringssl/libcrypto.a:
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/cpu-aarch64-fuchsia.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/cpu-aarch64-linux.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/cpu-arm-linux.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/cpu-arm.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/cpu-ppc64le.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/poly1305.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/poly1305_arm.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/deterministic.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/fuchsia.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/windows.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/refcount_c11.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/thread_none.pic.o has no symbols
/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/darwin-fastbuild/bin/external/boringssl/_objs/crypto/thread_win.pic.o has no symbols
ERROR: /Users/lpp/Desktop/perf_data_converter/src/quipper/BUILD:96:1: C++ compilation of rule '//src/quipper:dso' failed (Exit 1) cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 60 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
In file included from src/quipper/dso.cc:5:
In file included from src/quipper/dso.h:14:
In file included from src/quipper/data_reader.h:11:
src/quipper/binary_data_utils.h:8:10: fatal error: 'byteswap.h' file not found
#include <byteswap.h>
^~~~~~~~~~~~
1 error generated.
Target //src:perf_to_profile failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 19.511s, Critical Path: 3.27s
INFO: 239 processes: 239 darwin-sandbox.
FAILED: Build did NOT complete successfully
perf_data_converter (master) ✔ bazel build src:perf_to_profile k8s [kubernetes-admin@kubernetes]
perf_data_converter (master) ✔ k8s [kubernetes-admin@kubernetes]
perf_data_converter (master) ✔ git rev-parse HEAD k8s [kubernetes-admin@kubernetes]
Alias tip: g rev-parse HEAD
0a61af58974212f0ec3c40833b563fa587f9f9e3
perf_data_converter (master) ✔ k8s [kubernetes-admin@kubernetes]
perf_data_converter (master) ✔ uname -a k8s [kubernetes-admin@kubernetes]
Darwin nmiculinics-MacBook-Pro.local 18.2.0 Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST 2018; root:xnu-4903.231.4~2/RELEASE_X86_64 x86_64
❯ bazel build src:all
INFO: Repository com_google_googletest instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
/home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
WARNING: Download from https://github.com/google/googletest/archive/master.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
ERROR: An error occurred during the fetch of repository 'com_google_googletest':
Traceback (most recent call last):
File "/home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl
download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/google/googletest/archive/master.zip] to /home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/com_google_googletest/master.zip: GET returned 404 Not Found
INFO: Repository com_google_protobuf instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
/home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: Repository remote_coverage_tools instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
/home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: Repository boringssl instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
/home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
ERROR: /home/ahh/repo/perf_data_converter/src/compat/BUILD:27:11: //src/compat:test_compat depends on @com_google_googletest//:gtest_main in repository @com_google_googletest which failed to fetch. no such package '@com_google_googletest//': java.io.IOException: Error downloading [https://github.com/google/googletest/archive/master.zip] to /home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/com_google_googletest/master.zip: GET returned 404 Not Found
ERROR: Analysis of target '//src:intervalmap_test' failed; build aborted: Analysis failed
INFO: Elapsed time: 1.383s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
But it's not just tests:
❯ bazel build src:perf_to_profile
INFO: SHA256 (https://codeload.github.com/protocolbuffers/protobuf/zip/master) = 7138c6ec951e60f45efcdf4b2b5ab795a082cd50ae12f6ef79d052b864995218
DEBUG: Rule 'com_google_protobuf' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "7138c6ec951e60f45efcdf4b2b5ab795a082cd50ae12f6ef79d052b864995218"
DEBUG: Repository com_google_protobuf instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
/home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: Repository boringssl instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
/home/ahh/.cache/bazel/_bazel_ahh/b0269a7be5e98a894844c52eb8573a30/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
ERROR: /home/ahh/repo/perf_data_converter/src/quipper/BUILD:25:14: every rule of type proto_library implicitly depends upon the target '@com_google_protobuf//:protoc', but this target could not be found because of: error loading package '@com_google_protobuf//': Unable to find package for @rules_pkg//:mappings.bzl: The repository '@rules_pkg' could not be resolved.
ERROR: Analysis of target '//src:perf_to_profile' failed; build aborted: Analysis failed
INFO: Elapsed time: 1.671s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)
Fetching @bazel_skylib; fetching
Fetching @rules_proto; fetching
Fetching @rules_python; fetching
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.