tencent / yadcc Goto Github PK

build linux kernel, get some error
Failed on the cloud with [-126]. Failing back to local machine.

[45/1637] MAKE //thirdparty/nghttp2:nghttp2_build
......
libtool: install: /usr/bin/install -c .libs/libnghttp2.so.14.20.0 /home/name/workspace/yadcc/build64_release/thirdparty/nghttp2/lib/libnghttp2.so.14.20.0
/usr/bin/install: cannot stat '.libs/libnghttp2.so.14.20.0': No such file or directory

ll build64_release/thirdparty/nghttp2/nghttp2-1.41.0/lib/.libs

-rw-rw-r-- 1 wuminghui03 wuminghui03 819848 Oct 13 17:46 libnghttp2.a
lrwxrwxrwx 1 wuminghui03 wuminghui03     16 Oct 13 17:46 libnghttp2.la -> ../libnghttp2.la
-rw-rw-r-- 1 wuminghui03 wuminghui03   1023 Oct 13 17:46 libnghttp2.lai
lrwxrwxrwx 1 wuminghui03 wuminghui03     21 Oct 13 17:46 libnghttp2.so -> libnghttp2.so.14.20.0
lrwxrwxrwx 1 wuminghui03 wuminghui03     21 Oct 13 17:46 libnghttp2.so.14 -> libnghttp2.so.14.20.0
-rw-rw-r-- 1 wuminghui03 wuminghui03  27264 Oct 13 17:46 nghttp2_buf.o

安装报错 .libs/libnghttp2.so.14.20.0 不存在，查看目录 build64_release/thirdparty/nghttp2/nghttp2-1.41.0/lib/.libs , 有指向 libnghttp2.so.14.20.0 但是这个库是不存在的，这个库应该是新构建的才对，但是并未出现这个动态库链接报错。
已经按照 REAMD 更新了submodule, 并且在 build/external/flare 指向过 git lfs install/pull， 是有步骤不对少拉了文件吗
机器 CentOS Linux release 7.4.1708 (Core)。

E0803 21:09:16.471520 8840 task_dispatcher.cc:314] [10.243.2.241:48228] Unrecognized compilation environment [7446218a5a1cd826cd663c99919545490da3c56af4ffd1d53d1554cd0768b0ee] is requested by [10.243.2.241].

Most community contributors rarely know about blade, when we want to add address sanitizer option for this project, we get confused.

Great repo! And any develop roadmap on this ? I would a contributor on yadcc.

Is any guide or experience to compiling the AOSP code?

Android AOSP has several compiling systems per building target and android version.

Is there any plan to add support for heterogeneous build environments?
Like icecream did, pack the buildenv to tarball then send to remote peer....

这是编译日志：

[root@st1-gitci-1 yadcc]# ./blade build yadcc/...
Blade: Entering directory `/root/source/yadcc'
Blade(info): Loading config file "/root/source/yadcc/thirdparty/blade/blade.conf"
Blade(info): Loading config file "/root/source/yadcc/BLADE_ROOT"
Blade(info): Loading BUILD files...
Blade(info): Loading done.
Blade(info): Analyzing dependency graph...
Blade(info): Analyzing done.
Blade(info): Generating backend build code...
Blade(info): Generating done.
Blade(info): Building...
Blade(info): Adjust build jobs number(-j N) to be 16
[1/1781] LINK BINARY build64_release/yadcc/client/cxx/yadcc-cxx
FAILED: build64_release/yadcc/client/cxx/yadcc-cxx
g++ -o build64_release/yadcc/client/cxx/yadcc-cxx -Wl,--no-as-needed -lpthread -static-libgcc -static-libstdc++ @build64_release/yadcc/client/cxx/yadcc-cxx.rsp -ldl -lpthread -static
/usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -ldl
/usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -lm
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status

[2/1781] MAKE //thirdparty/ctemplate:ctemplate_build
ninja: build stopped: subcommand failed.
Blade(error): Build failure.
Blade(info): Cost time 23.6s
Blade(error): Failure

As official doc describes, PCH can be used in preprocess, and insert "#pragma GCC pch_preprocess "filename"" into the output. Furthermore, the path can be absolute or relative to the working dir. That's to say, it is possible to use PCH to achieve distribute build, which could improve the compilation efficiency. And i have skipped your code and found during the preparation, yadcc skips the PCH and just use -E, -fno-working-directory and -fdirectives-only to produce preprocess output. Now, i have two questions and wish your reply.

1、As i knew, -E option just preprocess the included headers directly without PCH, and should takes more time not only during the preprocess but also during the compilation. So, have u attemp to use PCH to achieve distributely build in yadcc, and if so, why not use it?

2、As gcc doc describes, -fworking-directory is default setting. However, yadcc turns it off and user -fno-working-directory, and some debug information like file path will not be inserted into the preprocess output. As i think, the purpose is to avoid absoulte path in the output which may cause missing cache in spite of the same source file. I have tried to handle the preprocess output of msvc compiler， and found without absolute path, cache will be hit if the source file is same in different computer. It helps much to save compilation time, but i can use the results to start executable files but can not use the results obj files or exp files or pdb files to debug if turning off -fworking-directory . Thus, i wonder that have yadcc found similar problems after turning path off.

Wish your reply, thank you very much.

yadcc能不能用于安卓系统AOSP构建加速

按照安装步骤执行
git clone https://github.com/Tencent/yadcc
cd yadcc
git submodule init
git submodule update .
编译时：./blade build yadcc/...
报错：
checking whether to build with libidn2... no
checking for pkg-config... no
configure: error: --with-nghttp2 was specified but could not find libnghttp2 pkg-config file.
[284/2107] CONFIGURE //thirdparty/glog:glog_configure
ninja: build stopped: subcommand failed.
Blade(error): Build failure.
Blade(info): Cost time 1m57.3s
Blade(error): Failure

环境：rhel 7.4 3.10.0-693.el7.x86_64 gcc11.2.0

在执行./blade build yadcc/...时报以下错误：

能否给daemon加个功能，当scheduler重启后自动重链

我运行的命令行是

./yadcc-daemon --scheduler_uri=flare://192.168.16.11:8336 token=lcy --poor_machine_threshold_processors=1

最终报错
F0316 15:04:56.197872 21353 execution_engine.cc:82] Check failed: parts.size() >= 3 (2 vs. 3) Unexpected cgroup setting: 0::/user.slice/user-0.slice/session-202.scope *** Check failure stack trace: *** @ 0x5632407132d3 (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x80f2d2) @ 0x563240715826 (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x811825) @ 0x563240712e8e (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x80ee8d) @ 0x563240715dcf (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x811dce) @ 0x56323ffc0639 (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0xbc638) @ 0x56323ffc0ea1 (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0xbcea0) @ 0x563240203572 (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x2ff571) @ 0x5632401ea47f (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x2e647e) @ 0x56324064deb3 (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x749eb2) @ 0x5632406586aa (/root/yadcc/build64_release/yadcc/daemon/yadcc-daemon+0x7546a9) Aborted (core dumped)

FAILED: lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o
/opt/buildtools/yadcc/client/cxx/yadcc-cxx -DGTEST_HAS_RTTI=0 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/workspace/llvm-project/tmp/lib/Support/BLAKE3 -I/home/workspace/llvm-project/llvm/lib/Support/BLAKE3 -I/home/workspace/llvm-project/tmp/include -I/home/workspace/llvm-project/llvm/include -fPIC -o lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -c /home/workspace/llvm-project/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_unix.S

buld llvm-project-11.0.0.tar.xz will pass
but build llvm main branch will fail
[2023-03-22 21:19:47.961755] [FATAL] [yadcc/client/common/utility.cc:157] Failed to find executable [-DGTEST_HAS_RTTI=0] in path.

编译环境：Intel CPU，Ubuntu22.04，gcc-11.3

yadcc v1.x 版本会拉取这个版本的flare-7a69d23，编译出错
/root/yadcc/build64_release/thirdparty/opentracing-cpp/opentracing-cpp-1.5.1/3rd_party/include/opentracing/catch2/catch.hpp:6631:45: error: size of array 'altStackMem' is not an integral constant-expression 6631 | char FatalConditionHandler::altStackMem[SIGSTKSZ] = {};
后面还会报一个error: macro "__has_attribute" requires an identifier的错误。
手动将flare更新到master最新版本可以修复：https://github.com/Tencent/flare/tree/365d1ac3670d21a62212251364b0b9f7e15ed07f

Code in master has long been superseded internally, using code in next brings us the following benefits:

• Natural support for more languages, from design perspective (code itself, protocol, source file layout);
• Support for --coverage (i.e., gcov);
• More backends for cache.
• ...

Yet, IIRC, it does regress in performance when compiling C++. This is mostly due to switch of protocol between client and daemon, from a dirty-and-quick one to a JSON based one. JsonCpp costs us quite some cpu cycles here.

That regression shouldn't hurt much, it's ~2%, and should be unnoticeable most of time.

compile linux kernel use cross_compile like this

export ARCH=arm64
export CROSS_COMPILE=aarch64-none-linux-gnu-
make  menuconfig  ARCH=arm64  CROSS_COMPILE=$CROSS_COMPILE
make ARCH=arm64 CROSS_COMPILE=$CROSS_COMPILE -j$(nproc) zImage

yadcc can not support it

client/common/command.cc:] Check failed: [WIFEXITED(status)]. Child process exited unexpectedly with status [13].

./blade build yadcc/...
./blade: line 7: /root/yadcc/thirdparty/blade/blade: No such file or directory

查了一下，这个路径里面写的blade/blade并不存在，去掉这个blade也不行

机器1：client（user，且不不接受任务）：ubuntu1804，gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
机器2：servant（专用编译机）：ubuntu2004，gcc-7 (Ubuntu 7.5.0-6ubuntu2) 7.5.0。
servant上支持的gcc-7的哈希值与user需要的不一致，对于这种情况有什么建议吗？ubuntu2004能编译出来和ubuntu1804哈希值一样的gcc吗？你们对于这样的场景，你们是怎么做的？

在12c的机器上测试 make -j60 ，限制本地轻任务并发为12，并在 RewriteFile 前面加日志作为预处理时间。发现正常情况下预处理在1秒左右的文件，预处理变成了12秒。

尝试从 RewriteFile 往下找，最后在似乎是由客户端调用的syscall。本地守护进程，设置了YADCC_LOG_LEVEL=0，也没打印预处理的日志。所以预处理是由客户端自己完成的？如果不是的话，能否告知在哪里加日志，可以获取预处理任务等待的时间？

When I do secondary development on yadcc daemon, the newly developed module causes segment errors. I feel confused as to what causes the problem.

the following cases:

• 1
  @ 0x7fe968d7dc70 (unknown)
  @ 0x55f7bc7b1ea3 tcmalloc::ThreadCache::ReleaseToCentralCache()
  @ 0x55f7bc7b21f5 tcmalloc::ThreadCache::ListTooLong()
• 2
  @ 0x7fb467824c70 (unknown)
  @ 0x560a2ffc98d3 tcmalloc::CentralFreeList::FetchFromOneSpans()
  @ 0x560a2ffc9b9c tcmalloc::CentralFreeList::FetchFromOneSpansSafe()
  @ 0x560a2ffc9c63 tcmalloc::CentralFreeList::RemoveRange()
  @ 0x560a2ffd1d02 tcmalloc::ThreadCache::FetchFromCentralCache()

The newly developed module:
Flare::Fiber used

构建了一个 arm 版本 yadcc， 其中发现资源库打包比较怪，打包的不是期望的构建出来的新库，
由于对于 blade 不熟悉， 没看出 src 中怎么引用其他库构建结果
只能是构建第一遍后，覆盖掉，然后再构建

请问这里是否可以优化呢？

yadcc/build64_release/thirdparty/opentracing-cpp/opentracing-cpp-1.5.1/3rd_party/include/opentracing/catch2/catch.hpp:6631:45: error: size of array ‘altStackMem’ is not an integral constant-expression
6631 | char FatalConditionHandler::altStackMem[SIGSTKSZ] = {};

glibc changed SIGSTKSZ to be a syscall instead
of being constant. #catchorg/Catch2#2178

当import module后，yadcc能否工作。

经过测试发现，将gcc编译之后的文件缓存至yadcc-cache没有问题，但是无法缓存g++编译之后的文件，并且使用YADCC_WARN_ON_NONCACHEABLE=1，这个参数也没有任何效果。

after install gcc by apt install in ubuntu 20.04
/usr/bin/gcc is a symlink file, the realpath of gcc is /usr/bin/x86_64-linux-gnu-gcc-9
the daemon can not find any compiler, because the bellow code skip symlink

  if (S_ISLNK(buf.st_mode)) {          // Symbolic link, ignore it.
    continue;
  }

错误日志如下 （Ubuntu22.10 + gcc 12.2 + cgroup-v2-workaround）
execution_engine.cc:82] Check failed: parts.size() >= 3 (2 vs. 3) Unexpected cgroup setting: 0::/user.slice/user-1000.slice/[email protected]/app.slice/app-org.gnome.Terminal.slice/vte-spawn-b6c5fa9c-c6b5-4316-b3d5-97fa888e8ef4.scope *** Check failure stack trace: *** @ 0x55fbd5c17233 google::LogMessage::Fail() @ 0x55fbd5c1997e google::LogMessage::SendToLog() @ 0x55fbd5c16dde google::LogMessage::Flush() @ 0x55fbd5c19f0f google::LogMessageFatal::~LogMessageFatal() @ 0x55fbd54b1d71 _ZZZN5yadcc6daemon5cloud12_GLOBAL__N_115IsCGroupPresentEvENKUlOT_OT0_E0_clImiEEDaS4_S6_ENKUlvE_clEv @ 0x55fbd54b273c _ZN5yadcc6daemon5cloud15ExecutionEngineC2Ev.cold @ 0x55fbd56fd8fa yadcc::daemon::cloud::ExecutionEngine::Instance() @ 0x55fbd56e358b yadcc::daemon::cloud::DaemonServiceImpl::Heartbeat() @ 0x55fbd5b4b791 _ZZN5flare8FunctionIFvvEE15ErasedCopySmallIZZNS_5fiber8SetTimerENSt6chrono10time_pointINS5_3_V212steady_clockENS5_8durationIlSt5ratioILl1ELl1000000000EEEEEESC_ONS0_IFvmEEEENUlT_E_clImEEDaSH_EUlvE_EEPKNS2_7TypeOpsEPvOSH_ENUlSO_E_4_FUNESO_ @ 0x55fbd5b55cda flare::fiber::detail::FiberProc() Aborted (core dumped)
能否提供一个docker镜像

after installing git-lfs，
./blade build yadcc/...
the "Blade(error): This repository need GIT LFS, see README.md" still occerd

{
    flare::NoncontiguousBuffer standard_output;
    ......
    auto result = flare::FlattenSlow(output->standard_output);
    auto deps_list = flare::Split(result, '\n');
}

a 400 MB file is cached in standard_output.
After I use it, I find that the memory is still occupied and cannot be released.

My machine: 16u32g

Is there a simple relationship?
flare_rpc_client_max_connections_per_server,flare_concurrency_hint,flare_rpc_server_stream_concurrency,flare_rpc_client_stream_concurrency, etc

Is it possible to provide a configuration guide document based on the configuration of personal computers?

文档中提到，“yadcc会按照命令行对源代码进行预处理，得到一个自包含的的预处理结果”
，如何理解“自包含”和yadcc是如何做到自包含的呢？ @chen3feng @0x804d8000

我架了4个编译节点， A:50核，B:50核， C:20核， D:20核。

当我make -j80的时候， A:50核占满，B:30核，C,和D，偶尔会有占用一两核。 C和D，在这种情况下，大部分时候没有编译任务。

想问下，有没有方案，让调度更平均一点，当make -j80的时候，按比例分配到节点
比如: A:28, B:28, C:12 D:12

W1022 14:18:14.378872 14776 distributed_cache_reader.cc:152] Failed to load compilation cache bloom filter from cache server: [1003] , last_bf_update: 355156, last_bf_full_update: 355158

last_bf_update < last_bf_full_update, but cache server requires last_bf_update >= last_bf_full_update

No logic error is found when working through distributed_cache_reader.cpp code.

yadcc尝试使用一个daemon未识别的编译器，守护进程等待五分钟放弃任务,并打印日志：
Task 3 cannot be started in time. Aborted.
，但是调度器仍不断打印 Unrecognized compilation environment。

$./blade build yadcc/...
thirdparty/protobuf/bin/x86_64/protoc-3.4.1: line 1: version: command not found
thirdparty/protobuf/bin/x86_64/protoc-3.4.1: line 2: oid: command not found

I have build yadcc suceessfully on a server but failed on another one. The error is at tho bottom.

on the suceess server, curl's configure is without idn2:
checking for libidn2 options with pkg-config... no
configure: IDN_LIBS: "-lidn2"
configure: IDN_LDFLAGS: ""
configure: IDN_CPPFLAGS: ""
configure: IDN_DIR: ""
checking if idn2_lookup_ul can be linked... no
checking idn2.h usability... no
checking idn2.h presence... no
checking for idn2.h... no
configure: WARNING: Cannot find libraries for IDN support: IDN disabled

on the failed server, there is a libidn2.so:
checking for libidn2 options with pkg-config... found
configure: pkg-config: IDN_LIBS: "-lidn2"
configure: pkg-config: IDN_LDFLAGS: ""
configure: pkg-config: IDN_CPPFLAGS: ""
configure: pkg-config: IDN_DIR: ""
checking if idn2_lookup_ul can be linked... yes
checking idn2.h usability... yes
checking idn2.h presence... yes
checking for idn2.h... yes

But if the env has libidn2.so, why yadcc do not link it?
I don't know blade very well, it seems to be an internal tool in tencent.
I have add -lidn2 in linkflags in BLADE_ROOT, but it do not work.

=========================================================

Blade: Entering directory /home/zhut/davidwang/yadcc' Blade(info): Loading config file "/home/zhut/davidwang/yadcc/thirdparty/blade/blade.conf" Blade(info): Loading config file "/home/zhut/davidwang/yadcc/BLADE_ROOT" Blade(info): Loading BUILD files... Blade(info): Loading done. Blade(info): Analyzing dependency graph... Blade(info): Analyzing done. Blade(info): Generating backend build code... Blade(info): Generating done. Blade(info): Building... Blade(info): Adjust build jobs number(-j N) to be 8 [1/3] LINK BINARY build64_release/yadcc/scheduler/yadcc-scheduler FAILED: build64_release/yadcc/scheduler/yadcc-scheduler g++ -o build64_release/yadcc/scheduler/yadcc-scheduler -Wl,--no-as-needed -lpthread -static-libgcc -static-libstdc++ -Wl,--whole-archive build64_release/yadcc/api/libscheduler_proto.a build64_release/yadcc/api/libenv_desc_proto.a build64_release/flare/init/libopenssl_initializer.a build64_release/flare/rpc/protocol/http/builtin/libexposed_vars_handler.a build64_release/flare/rpc/protocol/http/builtin/libgflags_handler.a build64_release/flare/rpc/protocol/http/builtin/libmisc_handler.a build64_release/flare/rpc/protocol/http/builtin/liboptions_handler.a build64_release/flare/base/liboption.a build64_release/flare/rpc/protocol/http/builtin/libprof_cpu_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_form_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_reflect_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_statistics_handler.a build64_release/flare/rpc/protocol/http/builtin/libstatic_resource_http_handler.a build64_release/flare/rpc/protocol/protobuf/libbaidu_std_protocol.a build64_release/flare/rpc/protocol/protobuf/libbaidu_std_rpc_meta_proto.a build64_release/flare/rpc/protocol/protobuf/libpoppy_protocol.a build64_release/flare/rpc/protocol/protobuf/libpoppy_rpc_meta_proto.a build64_release/flare/rpc/protocol/protobuf/libproto_over_http_protocol.a build64_release/flare/rpc/protocol/protobuf/libqzone_protocol.a build64_release/flare/rpc/protocol/protobuf/libnslb_registration.a build64_release/flare/rpc/load_balancer/libround_robin.a build64_release/flare/rpc/name_resolver/liblist.a build64_release/flare/rpc/message_dispatcher/libcomposited.a build64_release/flare/rpc/protocol/protobuf/libbinlog_proto.a build64_release/flare/rpc/librpc_options_proto.a build64_release/flare/rpc/protocol/protobuf/libstd_protocol.a build64_release/flare/rpc/binlog/text_only/libdumper.a build64_release/flare/rpc/binlog/text_only/libbinlog_proto.a build64_release/flare/rpc/binlog/util/libproto_binlog_proto.a build64_release/flare/rpc/protocol/protobuf/librpc_meta_proto.a build64_release/flare/rpc/protocol/http/libhttp.a build64_release/flare/rpc/protocol/http/libbinlog_proto.a build64_release/flare/base/compression/libgzip.a build64_release/flare/base/compression/liblz4.a build64_release/flare/base/compression/libsnappy.a build64_release/flare/base/compression/libzstd.a -Wl,--no-whole-archive @build64_release/yadcc/scheduler/yadcc-scheduler.rsp -lresolv -lm -ldl -lpthread -lrt /usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function conn_free':
url.c:(.text+0x2a6): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x2c2): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x2de): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x2fa): undefined reference to idn2_free'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_idnconvert_hostname': url.c:(.text+0x135b): undefined reference to idn2_check_version'
/usr/bin/ld: url.c:(.text+0x137b): undefined reference to idn2_lookup_ul' /usr/bin/ld: url.c:(.text+0x139b): undefined reference to idn2_strerror'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_free_idnconverted_hostname': url.c:(.text+0x13d1): undefined reference to idn2_free'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_connect': url.c:(.text+0x4769): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x4785): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x498e): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x49ac): undefined reference to idn2_free' collect2: error: ld returned 1 exit status [2/3] LINK BINARY build64_release/yadcc/cache/yadcc-cache FAILED: build64_release/yadcc/cache/yadcc-cache g++ -o build64_release/yadcc/cache/yadcc-cache -Wl,--no-as-needed -lpthread -static-libgcc -static-libstdc++ -Wl,--whole-archive build64_release/yadcc/api/libcache_proto.a build64_release/yadcc/cache/libdisk_cache_engine.a build64_release/yadcc/cache/libnull_cache_engine.a build64_release/flare/init/libopenssl_initializer.a build64_release/flare/rpc/protocol/http/builtin/libexposed_vars_handler.a build64_release/flare/rpc/protocol/http/builtin/libgflags_handler.a build64_release/flare/rpc/protocol/http/builtin/libmisc_handler.a build64_release/flare/rpc/protocol/http/builtin/liboptions_handler.a build64_release/flare/base/liboption.a build64_release/flare/rpc/protocol/http/builtin/libprof_cpu_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_form_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_reflect_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_statistics_handler.a build64_release/flare/rpc/protocol/http/builtin/libstatic_resource_http_handler.a build64_release/flare/rpc/protocol/protobuf/libbaidu_std_protocol.a build64_release/flare/rpc/protocol/protobuf/libbaidu_std_rpc_meta_proto.a build64_release/flare/rpc/protocol/protobuf/libpoppy_protocol.a build64_release/flare/rpc/protocol/protobuf/libpoppy_rpc_meta_proto.a build64_release/flare/rpc/protocol/protobuf/libproto_over_http_protocol.a build64_release/flare/rpc/protocol/protobuf/libqzone_protocol.a build64_release/flare/rpc/protocol/protobuf/libnslb_registration.a build64_release/flare/rpc/load_balancer/libround_robin.a build64_release/flare/rpc/name_resolver/liblist.a build64_release/flare/rpc/message_dispatcher/libcomposited.a build64_release/flare/rpc/protocol/protobuf/libbinlog_proto.a build64_release/flare/rpc/librpc_options_proto.a build64_release/flare/rpc/protocol/protobuf/libstd_protocol.a build64_release/flare/rpc/binlog/text_only/libdumper.a build64_release/flare/rpc/binlog/text_only/libbinlog_proto.a build64_release/flare/rpc/binlog/util/libproto_binlog_proto.a build64_release/flare/rpc/protocol/protobuf/librpc_meta_proto.a build64_release/flare/rpc/protocol/http/libhttp.a build64_release/flare/rpc/protocol/http/libbinlog_proto.a build64_release/flare/base/compression/libgzip.a build64_release/flare/base/compression/liblz4.a build64_release/flare/base/compression/libsnappy.a build64_release/flare/base/compression/libzstd.a -Wl,--no-whole-archive @build64_release/yadcc/cache/yadcc-cache.rsp -lresolv -lm -ldl -lpthread -lrt /usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function conn_free':
url.c:(.text+0x2a6): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x2c2): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x2de): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x2fa): undefined reference to idn2_free'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_idnconvert_hostname': url.c:(.text+0x135b): undefined reference to idn2_check_version'
/usr/bin/ld: url.c:(.text+0x137b): undefined reference to idn2_lookup_ul' /usr/bin/ld: url.c:(.text+0x139b): undefined reference to idn2_strerror'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_free_idnconverted_hostname': url.c:(.text+0x13d1): undefined reference to idn2_free'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_connect': url.c:(.text+0x4769): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x4785): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x498e): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x49ac): undefined reference to idn2_free' collect2: error: ld returned 1 exit status [3/3] LINK BINARY build64_release/yadcc/daemon/yadcc-daemon FAILED: build64_release/yadcc/daemon/yadcc-daemon g++ -o build64_release/yadcc/daemon/yadcc-daemon -Wl,--no-as-needed -lpthread -static-libgcc -static-libstdc++ -Wl,--whole-archive build64_release/flare/init/libopenssl_initializer.a build64_release/flare/rpc/builtin/libtcmalloc_profiler_http_handler.a build64_release/yadcc/api/libcache_proto.a build64_release/yadcc/api/libdaemon_proto.a build64_release/yadcc/api/libscheduler_proto.a build64_release/yadcc/daemon/local/libmessages_proto.a build64_release/flare/rpc/protocol/http/builtin/libexposed_vars_handler.a build64_release/flare/rpc/protocol/http/builtin/libgflags_handler.a build64_release/flare/rpc/protocol/http/builtin/libmisc_handler.a build64_release/flare/rpc/protocol/http/builtin/liboptions_handler.a build64_release/flare/base/liboption.a build64_release/flare/rpc/protocol/http/builtin/libprof_cpu_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_form_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_reflect_handler.a build64_release/flare/rpc/protocol/http/builtin/librpc_statistics_handler.a build64_release/flare/rpc/protocol/http/builtin/libstatic_resource_http_handler.a build64_release/flare/rpc/protocol/http/libhttp.a build64_release/flare/rpc/protocol/http/libbinlog_proto.a build64_release/flare/rpc/protocol/protobuf/libbaidu_std_protocol.a build64_release/flare/rpc/protocol/protobuf/libbaidu_std_rpc_meta_proto.a build64_release/flare/rpc/protocol/protobuf/libpoppy_protocol.a build64_release/flare/rpc/protocol/protobuf/libpoppy_rpc_meta_proto.a build64_release/flare/rpc/protocol/protobuf/libproto_over_http_protocol.a build64_release/flare/rpc/protocol/protobuf/libqzone_protocol.a build64_release/flare/rpc/protocol/protobuf/libnslb_registration.a build64_release/flare/rpc/load_balancer/libround_robin.a build64_release/flare/rpc/name_resolver/liblist.a build64_release/flare/rpc/message_dispatcher/libcomposited.a build64_release/flare/rpc/protocol/protobuf/libbinlog_proto.a build64_release/flare/rpc/librpc_options_proto.a build64_release/flare/rpc/protocol/protobuf/libstd_protocol.a build64_release/flare/rpc/binlog/text_only/libdumper.a build64_release/flare/rpc/binlog/text_only/libbinlog_proto.a build64_release/flare/rpc/binlog/util/libproto_binlog_proto.a build64_release/flare/rpc/protocol/protobuf/librpc_meta_proto.a build64_release/flare/base/compression/libgzip.a build64_release/flare/base/compression/liblz4.a build64_release/flare/base/compression/libsnappy.a build64_release/flare/base/compression/libzstd.a build64_release/yadcc/daemon/libcache_meta_proto.a build64_release/yadcc/api/libenv_desc_proto.a -Wl,--no-whole-archive @build64_release/yadcc/daemon/yadcc-daemon.rsp -lresolv -lm -ldl -lpthread -lrt /usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function conn_free':
url.c:(.text+0x2a6): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x2c2): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x2de): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x2fa): undefined reference to idn2_free'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_idnconvert_hostname': url.c:(.text+0x135b): undefined reference to idn2_check_version'
/usr/bin/ld: url.c:(.text+0x137b): undefined reference to idn2_lookup_ul' /usr/bin/ld: url.c:(.text+0x139b): undefined reference to idn2_strerror'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_free_idnconverted_hostname': url.c:(.text+0x13d1): undefined reference to idn2_free'
/usr/bin/ld: build64_release/thirdparty/curl/lib/libcurl.a(libcurl_la-url.o): in function Curl_connect': url.c:(.text+0x4769): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x4785): undefined reference to idn2_free' /usr/bin/ld: url.c:(.text+0x498e): undefined reference to idn2_free'
/usr/bin/ld: url.c:(.text+0x49ac): undefined reference to `idn2_free'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Blade(error): Build failure.
Blade(info): Cost time 4.43s
Blade(error): Failure

为了方便回溯，建议发布一个 v1.0 之类

我按技术文档配置，调度机和客户机都各自启动了scheduler和daemon，调度机检测到新的servant。

I1110 14:19:54.431716 1780984 init.cc:114] Flare started.
I1110 14:19:54.432881 1780984 runtime.cc:425] Using fiber scheduling profile [neutral].
I1110 14:19:54.432904 1780984 runtime.cc:220] Starting 8 worker threads per group, for a total of 1 groups. The system is treated as UMA.
I1110 14:19:54.436939 1780984 init.cc:122] Flare runtime initialized.
I1110 14:19:54.702870 1780997 task_dispatcher.cc:205] [192.168.33.5:46518] Discovered new servant at [192.168.33.5:8335]

但是运行YADCC_LOG_LEVEL=0 CXX='/home/ubuntu/.yadcc/symlinks/g++' LD='/home/ubuntu/.yadcc/symlinks/g++' CC='/home/ubuntu/.yadcc/symlinks/gcc' make -j8
提示

[2022-11-10 14:20:33.068854] [TRACE] [yadcc/client/yadcc.cc:261] Started
[2022-11-10 14:20:33.068886] [DEBUG] [yadcc/client/utility.cc:92] Looking up for [gcc] in [/home/ubuntu/.yadcc/symlinks].
[2022-11-10 14:20:33.068912] [DEBUG] [yadcc/client/utility.cc:92] Looking up for [gcc] in [/usr/local/sbin].
[2022-11-10 14:20:33.068919] [DEBUG] [yadcc/client/utility.cc:92] Looking up for [gcc] in [/usr/local/bin].
[2022-11-10 14:20:33.068926] [DEBUG] [yadcc/client/utility.cc:92] Looking up for [gcc] in [/usr/sbin].
[2022-11-10 14:20:33.068932] [DEBUG] [yadcc/client/utility.cc:92] Looking up for [gcc] in [/usr/bin].
[2022-11-10 14:20:33.068942] [TRACE] [yadcc/client/utility.cc:99] Found [gcc] at [/usr/bin].
[2022-11-10 14:20:33.068947] [TRACE] [yadcc/client/yadcc.cc:133] Using compiler: /usr/bin/gcc
[2022-11-10 14:20:33.068980] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:34.064286] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:34.065163] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:34.068817] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:34.069060] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:35.064416] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:35.065259] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:35.068913] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?
[2022-11-10 14:20:35.069152] [ERROR] [yadcc/client/task_quota.cc:61] Cannot contact delegate daemon. Daemon died?

请问可能是什么问题？

1、error occurred under gcc8 when building python2.7. compiled successfully under gcc7

2、python: can't decompress data; zlib not available when building python2.7.
solution

3、ubuntu 18.04.5(g++ 8.5)、ubuntu20.04(g++ 9.3)
The same compilation error occurred when i tryed to build yadcc on my PC which installed with ubuntu 18.04.5(g++ 8.5) or ubuntu20.04(g++ 9.3), but pthread.a(/usr/lib/x86_64-linux-gnu/libpthread.a) and pthread.so(/usr/lib/x86_64-linux-gnu/libpthread.so) exist. Strangely enough, it can be compiled successfully in the company environment which installed with ubuntu 18.04.

[3/75] LINK SHARED build64_release/flare/rpc/protocol/http/builtin/librpc_form_handler.so
FAILED: build64_release/flare/rpc/protocol/http/builtin/librpc_form_handler.so
g++ -shared -o build64_release/flare/rpc/protocol/http/builtin/librpc_form_handler.so  -Lthirdparty/jdk/lib -Xlinker --no-undefined build64_release/flare/rpc/protocol/http/builtin/rpc_form_handler.objs/rpc_form_handler.cc.o build64_release/flare/rpc/protocol/http/builtin/libstatic_resources.so build64_release/flare/rpc/protocol/http/libhttp.so build64_release/flare/rpc/protocol/http/libbinlog_proto.so build64_release/flare/rpc/protocol/http/libbuffer_io.so build64_release/flare/rpc/protocol/http/libhttp_handler.so build64_release/flare/rpc/protocol/http/libmessage.so build64_release/flare/base/libcompression.so build64_release/flare/base/compression/libgzip.so build64_release/flare/base/compression/liblz4.so build64_release/thirdparty/lz4/lib64/liblz4.so build64_release/flare/base/compression/libsnappy.so build64_release/thirdparty/snappy/lib64/libsnappy.so build64_release/flare/base/compression/libzstd.so build64_release/thirdparty/zstd/lib64/libzstd.so build64_release/flare/base/buffer/libcompression_output_stream.so build64_release/flare/base/compression/libutil.so build64_release/flare/base/compression/libcompression.so build64_release/flare/net/http/libpacket_desc.so build64_release/flare/net/http/libhttp_request.so build64_release/flare/net/http/libhttp_response.so build64_release/flare/net/http/libhttp_message.so build64_release/flare/net/http/libhttp_headers.so build64_release/flare/net/http/libtypes.so build64_release/flare/rpc/internal/libsession_context.so build64_release/flare/rpc/binlog/liblog_reader.so build64_release/flare/rpc/binlog/libdry_runner.so build64_release/flare/base/libstatus.so build64_release/flare/fiber/liblogging.so build64_release/flare/fiber/libtimer.so build64_release/flare/fiber/libwork_queue.so build64_release/flare/rpc/binlog/liblog_writer.so build64_release/flare/rpc/binlog/libdumper.so build64_release/flare/rpc/binlog/libpacket_desc.so build64_release/flare/base/buffer/libzero_copy_stream.so build64_release/flare/rpc/tracing/libtracing_ops.so build64_release/flare/rpc/tracing/libtracing_ops_provider.so build64_release/thirdparty/opentracing-cpp/lib/libopentracing.so build64_release/flare/rpc/protocol/libstream_protocol.so build64_release/flare/base/libbuffer.so build64_release/flare/base/buffer/libbuiltin_buffer_block.so build64_release/flare/init/libon_init.so build64_release/flare/rpc/protocol/libcontroller.so build64_release/flare/rpc/protocol/libmessage.so build64_release/flare/base/experimental/libuuid.so build64_release/flare/base/net/libendpoint.so build64_release/flare/fiber/libfiber_basic.so build64_release/flare/base/internal/libindex_alloc.so build64_release/flare/fiber/detail/libscheduling_parameters.so build64_release/flare/fiber/detail/libfiber_impl.so build64_release/flare/fiber/detail/libcontext.so build64_release/flare/fiber/detail/librun_queue.so build64_release/flare/fiber/detail/libstack_allocator.so build64_release/flare/base/libmonitoring.so build64_release/flare/base/internal/libdpc.so build64_release/flare/base/monitoring/libinit.so build64_release/flare/base/monitoring/libdispatcher.so build64_release/flare/base/monitoring/libmonitoring_system.so build64_release/thirdparty/yaml-cpp/lib/libyaml-cpp.a build64_release/flare/base/libobject_pool.so build64_release/flare/base/object_pool/libdisabled.so build64_release/flare/base/object_pool/libglobal.so build64_release/flare/base/object_pool/libmemory_node_shared.so build64_release/flare/base/libexposed_var.so build64_release/flare/base/libtsc.so build64_release/flare/base/write_mostly/libmetrics.so build64_release/thirdparty/jsoncpp/lib/libjsoncpp.so build64_release/flare/base/internal/libtime_keeper.so build64_release/flare/base/internal/libbackground_task_host.so build64_release/flare/base/internal/libthread_pool.so build64_release/flare/base/internal/libcpu.so build64_release/flare/base/libstring.so build64_release/flare/base/object_pool/libthread_local.so build64_release/flare/base/libdemangle.so build64_release/flare/base/liberased_ptr.so build64_release/flare/base/thread/libattribute.so build64_release/flare/base/thread/liblatch.so build64_release/flare/base/thread/libout_of_duty_callback.so build64_release/flare/base/liblogging.so build64_release/flare/base/libchrono.so build64_release/flare/base/internal/libmemory_barrier.so build64_release/flare/base/internal/liblogging.so build64_release/thirdparty/fmt/lib64/libfmt.so build64_release/flare/base/thread/libspinlock.so build64_release/thirdparty/protobuf-3.4.1/libprotobuf.so build64_release/thirdparty/protobuf-3.4.1/libprotobuf_lite.so build64_release/thirdparty/glog/lib/libglog.so build64_release/thirdparty/gflags/lib/libgflags.so build64_release/thirdparty/zlib/lib/libz.so build64_release/thirdparty/ctemplate/lib/libctemplate.so -ldl -lrt -lz -lpthread -lm
/usr/bin/ld: build64_release/flare/rpc/protocol/http/builtin/rpc_form_handler.objs/rpc_form_handler.cc.o: in function `flare::object_pool::detail::memory_node_shared::GlobalPoolDescriptor* flare::object_pool::detail::memory_node_shared::GetGlobalPoolDescriptor<flare::PolymorphicBuffer>()':
/usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:700: undefined reference to `pthread_once'
collect2: error: ld returned 1 exit status

即 ./yadcc-scheduler --acceptable_tokens=some_fancy_token
这个命令的some_fancy_token应该如何生成？急，求指导 @0x804d8000 @tencent-adm @hzlzh @chen3feng @darkinsky