romange / helio Goto Github PK

View Code? Open in Web Editor NEW

398.0 10.0 46.0 2.01 MB

A modern framework for backend development based on io_uring Linux interface

License: Apache License 2.0

CMake 1.96% C 25.61% C++ 71.84% Shell 0.31% JavaScript 0.20% CSS 0.08%

io-uring async fibers cpp17 hacktoberfest

helio's Introduction

Helio - backend development framework in C++ using io_uring and epoll event-loop.

=====

Async is a set of C++ primitives that allows you efficient and rapid development in c++17 on Linux systems. The focus is mostly on backend development, data processing, etc.

Dependency on abseil-cpp
Dependency on Boost 1.71
Uses ninja-build on top of cmake
Built artifacts are docker-friendly.
HTTP server implementation.
Fibers library and fiber-friendly synchronization primitives.

I will gradually add explanations for the most crucial blocks in this library.

Acknowledgments

Helio has utilized Boost.Fiber until May 2023. After this period, the fibers functionality was integrated directly into Helio code. This transition involved adapting and reworking portions of the original Boost.Fiber code to align with Helio's coding conventions and architecture.

The design and implementation of Helio fibers draw significant inspiration from Boost.Fiber. Certain segments of the Boost.Fiber code were selectively adopted and modified for seamless incorporation into Helio. We extend our deepest gratitude and recognition to @olk and all the contributors of Boost.Fiber for their foundational work.

The decision to replicate and modify the original code was driven by the necessity for greater flexibility. Specifically, it was easier to evolve Helio's polling mechanism when we had full control over the fibers internal interfaces.

Differrences between Boost.Fiber and helio fb2 fibers.

Helio fb2 library adopts similar non-intrusive, shared nothing design similarly to Boost.Fiber. However, helio's fb2 library is more opinionated and also provides full networking support for asynchronous multi-threaded execution. helio fibers allows injecting a custom dispatch policy "util::fb2::DispatchPolicy" directly into a dispatching fiber, while Boost.Fiber requires using a separate fiber to handle the I/O polling. In addition, Boost.Fiber inter-thread notification mechanism (aka remote ready queue) required locking while helio fibers use a lockless queue.

Setting Up & Building

> sudo ./install-dependencies.sh
> ./blaze.sh -release
> cd build-opt && ninja -j4 echo_server

third_party folder is checked out under build directories.

Then, from 2 tabs run:

  server> ./echo_server --logtostderr
  client> ./echo_server --connect=localhost --n 100000 --c=4

HTTP

HTTP handler is implemented using Boost.Beast library. It's integrated with the io_uring-based ProactorPool. Please see http_main.cc, for example. HTTP also provides support for backend monitoring (Varz status page) and for an extensible debugging interface. With monitoring C++ backend returns JSON object that is formatted inside status page in the browser. To check how it looks, please go to localhost:8080 while echo_server is running.

Self-profiling

Every http-powered backend has integrated CPU profiling capabilities using gperf-tools and pprof Profiling can be triggered in prod using magic-url commands. Enabled profiling usually has a very minimal impact on CPU performance of the running backend.

Logging

Logging is based on Google's glog library. The library is very reliable, performant, and solid. It has many features that allow resilient backend development. Unfortunately, Google's version has some bugs, which I fixed (waiting for review...), so I use my own fork. Glog library gives me the ability to control the logging levels of a backend at run-time without restarting it.

Tests

ASYNC uses a googletest+gmock unit-test environment.

Conventions

Third_party packages have TRDP:: prefix in CMakeLists.txt. absl libraries have prefix absl::....

helio's People

Contributors

Stargazers

Watchers

helio's Issues

statx is not implemented by musl-dev

we should check for statx support and avoid using it if it does not exist

replace libxml with pugixml

it's been used by minio-cpp, see https://github.com/minio/minio-cpp/blob/main/CMakeLists.txt#L70

Missing cgroups support

The current CPU count detector in Linux only uses affinity:

helio/util/fibers/proactor_pool.cc

Lines 39 to 44 in ceaa6f8

 static cpu_set_t OnlineCpus() { 

 cpu_set_t online_cpus; 

 CPU_ZERO(&online_cpus); 

 CHECK_EQ(0, sched_getaffinity(0, sizeof(online_cpus), &online_cpus)); 

 return online_cpus; 

 }

Modern containerized systems require use of cgroups that is used to limit resource usage and therefore it should be taken account also in Helio's CPU detection. Please, take a look at e.g. OpenJDK's implementation: https://github.com/openjdk/jdk/blob/37c2279148fa91627137e3af362bfcffde61acd0/src/hotspot/os/linux/cgroupSubsystem_linux.cpp#L449-L513

About ctx scheduler design

EpollProactor::Run
while (true) {
    task_queue_
    epoll_wait
    algo->SuspendIoLoop // suspend
}

We know that FiberSchedAlgo::main_cntx_ is inserted into the FiberSchedAlgo::queue_ when FiberSchedALgo::rqueue_ is
empty or IoLoop busy(

helio/util/fiber_sched_algo.cc

Line 50 in 124e97a

if (delta_us > 1000) { // 1ms

)

Why not replace suspend with yeild. It seems to have no impact on performance and better readability.

improve locking in helio

ParkingLot https://webkit.org/blog/6161/locking-in-webkit/

is a high-performant data structure that manages all the waiting queues globally.

prominent implementations are here: https://github.com/Amanieu/parking_lot/tree/master/core/src
and here: https://github.com/facebook/folly/blob/master/folly/synchronization/ParkingLot.h#L140

why are they good?
we can park fibers using helper data-structure that exists out of stack, meaning we do not need to manage lifetime of the synchronization primitive that lives on stack. For example, in the example below, if parking_lot would be on-stack object it would cause stack corruption because, task coult set b=true and then the waiting thread would exit and destroy the sync primitive.
With parking lot it does not happen.

{                  
atomic_bool = false;
auto fiber_key = this_fiber::parking_key(); 
launc_task([&b, fiber_key] { 
    b = true;  // serialization point, after each we do not access the original caller data.
    parking_lot::unpark(fiber_key);
  });

parking_lot::wait([] { return b;});        
}

fix ping_iouring_server.cc and add it to CI

we need to replace the old flags with new ones in ping_iouring_server

crash in fibers

F20230906 00:50:14.290956    10 synchronization.cc:51] Check failed: detail::kRemoteFree == (uintptr_t)MPSC_intrusive_load_next(*active) (1 vs. 281473299385088) 
*** Check failure stack trace: ***
    @     0xaaaac6f0973c  google::LogMessage::SendToLog()
    @     0xaaaac6f02820  google::LogMessage::Flush()
    @     0xaaaac6f0415c  google::LogMessageFatal::~LogMessageFatal()
    @     0xaaaac6af51d8  util::fb2::EventCount::wait_until()
    @     0xaaaac6b68d34  dfly::Transaction::WaitOnWatch()

AWS::Init should use multiple fallbacks to find credentials

Right now AWS::Init uses only AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to find credentials.

However, there are other ways to do it.
The order is:

To check environment variables https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
If it's not enough - check for credential file https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
(need to use some of the env vars to know what to look for)
if it's not enough - check the instance metadata (prod instance)

(one can check how aws cli checks for credentials via aws --debug s3api list-buckets for example)

To do (2) - we can use io::ini::Parse method from line_reader.h.
To do 3 - one can use https://github.com/aws/amazon-ec2-metadata-mock and then of course to do the final testing on ec2 instance.

spurious wakeup during EventCount::wait

When the active fiber resumes in this function, it's state suggests it's still subscribed to wait_list, suggesting it was not woken up by EventCount::notify.

Possible explanation:

The fiber flow looks like:

ec1.wait_until(...);
.....
ec2.wait(...);   // <- crash

the wait_until call expires on timeout, but in parallel, another thread notifies it via ScheduleFromRemote.
The fiber is activated via ProcessSleep and blocks on ec2.wait.
The dispatching fiber calls ProcessRemoteReady, which wakes up the fiber based on the previous notification.

derivative work of boost.fiber

I think helio is derivative work of boost.fiber.
Your implementation resembles a lot of boost.fiber - for instance the use of linked lists or the idea of a dispatching-fiber inside the scheduler or the kind of queues to handle sleeping/blocked/remote fibers.
You have removed all references to the original work of boost.fiber - including the license - and claims that all of this code is only a result of you thoughts - that is really is annoying and a slap int the face of the authors of boost.fiber.

s3 code testing coverage

we should provide a test coverage for our helio s3 code in a semi-mocked environment.
minio with helio unit tests assuming an s3 accessible local end point could be one way to design it.

extend DnsResolve to support ipv6

Some environments do not rely on DNS servers for dns resolution.
It seems that we do not support this case.

Expected: /etc/hosts would be taken into account.

a few leads to investigate:

https://c-ares.org/ares_init_options.html has char *hosts_path;
there is also a pure ares_gethostbyname_file call.

Seems that the first option would be a cleaner solution.

fix epoll controller code

We should do a major refresh of this code and align epoll functionality with iouring proactor.

The goal - to support dragonfly with iouring and epoll to allow an easier migration to dragonfly on legacy kernels.

the expired session token is not handled gracefully

here is what I get (redacted):

E20220924 22:13:38.256793 1244660 s3.cc:143] http error: HTTP/1.1 400 Bad Request
x-amz-request-id: BLA
x-amz-id-2: FOOO
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Sat, 24 Sep 2022 19:13:37 GMT
Server: AmazonS3
Connection: close

4ac
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Token-0>TOKENFOOBAR</Token-0><RequestId>FOO</RequestId><HostId>BARR</HostId></Error>
0

F20220924 22:13:38.258005 1244659 s3_demo.cc:51] Check failed: list_res

S3 object list api does not support paging

currently function ListObjects in s3.h check fails, if it encounters a truncated list of objects in the XML s3 response.

also the S3Bucket API does not support it. We must fix it before completing #10

crash in epoll_socket

F20230404 07:29:01.133536 9 epoll_socket.cc:67] Check failed: arm_index_ < 0 (11 vs. 0)

add support for io_uring latest features

io_uring_register_ring_fd
direct descriptors for sockets
IORING_RECVSEND_POLL_FIRST / IORING_CQE_F_SOCK_NONEMPTY
io_uring_register_buf_ring
io_uring_prep_msg_ring to replace eventfd in WakeRing if possible.

For reference: https://kernel.dk/axboe-kr2022.pdf

Fail to launch echo_server

Hi,

I follow the guide from "Setting Up & Building" and try to launch echo_server, but I got error message: Illegal instruction. I didn't get error message while building. Here is the full message after typing ./blaze.sh -ninja -release

Using launcher /usr/bin/ccache
++ cmake --version
++ grep -Eo '[0-9]+\.[0-9]+\.[0-9]+'
+ result=3.22.1
+ major=3
+ remainder=22.1
+ minor=22
+ '[' 3 -lt 3 ']'
+ '[' 3 -eq 3 ']'
+ '[' 22 -lt 4 ']'
+ cmake -L -B build-opt -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++ -GNinja -DCMAKE_CXX_COMPILER_LAUNCHER=/usr/bin/ccache
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- PROJECT_BINARY_DIR /home/eecheng/helio/build-opt GENERATOR Ninja
-- PROJECT_SOURCE_DIR /home/eecheng/helio CMAKE_SOURCE_DIR /home/eecheng/helio
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found Python: /usr/bin/python3.10 (found version "3.10.6") found components: Interpreter
-- Found Git: /usr/bin/git (found version "2.34.1")
-- git version: v0.2.0-7-g89870358 normalized to 0.2.0.7
-- Version: 0.2.0.7
-- Looking for shm_open in rt
-- Looking for shm_open in rt - found
-- Performing Test HAVE_CXX_FLAG_STD_CXX11
-- Performing Test HAVE_CXX_FLAG_STD_CXX11 - Success
-- Performing Test HAVE_CXX_FLAG_WALL
-- Performing Test HAVE_CXX_FLAG_WALL - Success
-- Performing Test HAVE_CXX_FLAG_WEXTRA
-- Performing Test HAVE_CXX_FLAG_WEXTRA - Success
-- Performing Test HAVE_CXX_FLAG_WSHADOW
-- Performing Test HAVE_CXX_FLAG_WSHADOW - Success
-- Performing Test HAVE_CXX_FLAG_WERROR
-- Performing Test HAVE_CXX_FLAG_WERROR - Success
-- Performing Test HAVE_CXX_FLAG_WSUGGEST_OVERRIDE
-- Performing Test HAVE_CXX_FLAG_WSUGGEST_OVERRIDE - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC
-- Performing Test HAVE_CXX_FLAG_PEDANTIC - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS - Success
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 - Failed
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS - Success
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED - Success
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WD654
-- Performing Test HAVE_CXX_FLAG_WD654 - Failed
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY - Failed
-- Performing Test HAVE_CXX_FLAG_COVERAGE
-- Performing Test HAVE_CXX_FLAG_COVERAGE - Success
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Looking for pfm_initialize in libpfm.a
-- Looking for pfm_initialize in libpfm.a - not found
Perf Counters support requested, but was unable to find libpfm.
-- Found Unwind: /usr/include/aarch64-linux-gnu (found version "1.3.2")
-- Looking for C++ include dlfcn.h
-- Looking for C++ include dlfcn.h - found
-- Looking for C++ include execinfo.h
-- Looking for C++ include execinfo.h - found
-- Looking for C++ include glob.h
-- Looking for C++ include glob.h - found
-- Looking for C++ include inttypes.h
-- Looking for C++ include inttypes.h - found
-- Looking for C++ include memory.h
-- Looking for C++ include memory.h - found
-- Looking for C++ include pwd.h
-- Looking for C++ include pwd.h - found
-- Looking for C++ include stdint.h
-- Looking for C++ include stdint.h - found
-- Looking for C++ include strings.h
-- Looking for C++ include strings.h - found
-- Looking for C++ include sys/stat.h
-- Looking for C++ include sys/stat.h - found
-- Looking for C++ include sys/syscall.h
-- Looking for C++ include sys/syscall.h - found
-- Looking for C++ include sys/time.h
-- Looking for C++ include sys/time.h - found
-- Looking for C++ include sys/types.h
-- Looking for C++ include sys/types.h - found
-- Looking for C++ include sys/utsname.h
-- Looking for C++ include sys/utsname.h - found
-- Looking for C++ include sys/wait.h
-- Looking for C++ include sys/wait.h - found
-- Looking for C++ include syscall.h
-- Looking for C++ include syscall.h - found
-- Looking for C++ include syslog.h
-- Looking for C++ include syslog.h - found
-- Looking for C++ include ucontext.h
-- Looking for C++ include ucontext.h - found
-- Looking for C++ include unistd.h
-- Looking for C++ include unistd.h - found
-- Looking for C++ include ext/hash_map
-- Looking for C++ include ext/hash_map - found
-- Looking for C++ include ext/hash_set
-- Looking for C++ include ext/hash_set - found
-- Looking for C++ include ext/slist
-- Looking for C++ include ext/slist - found
-- Looking for C++ include tr1/unordered_map
-- Looking for C++ include tr1/unordered_map - found
-- Looking for C++ include tr1/unordered_set
-- Looking for C++ include tr1/unordered_set - found
-- Looking for C++ include unordered_map
-- Looking for C++ include unordered_map - found
-- Looking for C++ include unordered_set
-- Looking for C++ include unordered_set - found
-- Looking for C++ include stddef.h
-- Looking for C++ include stddef.h - found
-- Check size of unsigned __int16
-- Check size of unsigned __int16 - failed
-- Check size of u_int16_t
-- Check size of u_int16_t - done
-- Check size of uint16_t
-- Check size of uint16_t - done
-- Looking for dladdr
-- Looking for dladdr - found
-- Looking for fcntl
-- Looking for fcntl - found
-- Looking for pread
-- Looking for pread - found
-- Looking for pwrite
-- Looking for pwrite - found
-- Looking for sigaction
-- Looking for sigaction - found
-- Looking for sigaltstack
-- Looking for sigaltstack - found
-- Performing Test HAVE_NO_DEPRECATED
-- Performing Test HAVE_NO_DEPRECATED - Success
-- Performing Test HAVE_NO_UNNAMED_TYPE_TEMPLATE_ARGS
-- Performing Test HAVE_NO_UNNAMED_TYPE_TEMPLATE_ARGS - Failed
-- Looking for pthread_threadid_np
-- Looking for pthread_threadid_np - not found
-- Looking for snprintf
-- Looking for snprintf - found
-- Looking for UnDecorateSymbolName in dbghelp
-- Looking for UnDecorateSymbolName in dbghelp - not found
-- Performing Test HAVE___ATTRIBUTE__
-- Performing Test HAVE___ATTRIBUTE__ - Success
-- Performing Test HAVE___ATTRIBUTE__VISIBILITY_DEFAULT
-- Performing Test HAVE___ATTRIBUTE__VISIBILITY_DEFAULT - Success
-- Performing Test HAVE___ATTRIBUTE__VISIBILITY_HIDDEN
-- Performing Test HAVE___ATTRIBUTE__VISIBILITY_HIDDEN - Success
-- Performing Test HAVE___BUILTIN_EXPECT
-- Performing Test HAVE___BUILTIN_EXPECT - Success
-- Performing Test HAVE___SYNC_VAL_COMPARE_AND_SWAP
-- Performing Test HAVE___SYNC_VAL_COMPARE_AND_SWAP - Success
-- Performing Test HAVE___DECLSPEC
-- Performing Test HAVE___DECLSPEC - Failed
-- Performing Test STL_NO_NAMESPACE
-- Performing Test STL_NO_NAMESPACE - Failed
-- Performing Test STL_STD_NAMESPACE
-- Performing Test STL_STD_NAMESPACE - Success
-- Performing Test HAVE_USING_OPERATOR
-- Performing Test HAVE_USING_OPERATOR - Success
-- Performing Test HAVE_NAMESPACES
-- Performing Test HAVE_NAMESPACES - Success
-- Performing Test HAVE_GCC_TLS
-- Performing Test HAVE_GCC_TLS - Success
-- Performing Test HAVE_MSVC_TLS
-- Performing Test HAVE_MSVC_TLS - Failed
-- Performing Test HAVE_CXX11_TLS
-- Performing Test HAVE_CXX11_TLS - Success
-- Performing Test HAVE_ALIGNED_STORAGE
-- Performing Test HAVE_ALIGNED_STORAGE - Success
-- Performing Test HAVE_CXX11_ATOMIC
-- Performing Test HAVE_CXX11_ATOMIC - Success
-- Performing Test HAVE_CXX11_CONSTEXPR
-- Performing Test HAVE_CXX11_CONSTEXPR - Success
-- Performing Test HAVE_CXX11_CHRONO
-- Performing Test HAVE_CXX11_CHRONO - Success
-- Performing Test HAVE_CXX11_NULLPTR_T
-- Performing Test HAVE_CXX11_NULLPTR_T - Success
-- Performing Test HAVE_LOCALTIME_R
-- Performing Test HAVE_LOCALTIME_R - Success
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Found Boost: /opt/boost/lib/cmake/Boost-1.76.0/BoostConfig.cmake (found suitable version "1.76.0", minimum required is "1.71.0") found components: context system fiber
-- Found Boost /opt/boost_1_76_0/lib 1_76 1.76.0
-- Performing Test COMPILER_SUPPORTS_CXX17
-- Performing Test COMPILER_SUPPORTS_CXX17 - Success
-- Performing Test COMPILER_SUPPORTS_CXX20
-- Performing Test COMPILER_SUPPORTS_CXX20 - Success
-- Compiler /usr/bin/g++, version: 11.3.0
-- Performing Test SUPPORT_ASAN
-- Performing Test SUPPORT_ASAN - Success
-- Performing Test SUPPORT_USAN
-- Performing Test SUPPORT_USAN - Success
-- Performing Test HAS_RAWMEMCHR
-- Performing Test HAS_RAWMEMCHR - Success
CXX_FLAGS -Wall -Wextra -g -fPIC -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -fno-omit-frame-pointer -Wno-unused-parameter -march=armv8.2-a+fp16+rcpc+dotprod+crypto  -std=c++20 -DHAS_RAWMEMCHR -fdiagnostics-color=always  -O3 -DNDEBUG
-- Found OpenSSL: /usr/local/lib/libcrypto.so (found version "3.0.0")
-- OpenSSL libs /usr/local/lib/libssl.so 3.0.0
-- Found LibXml2: /usr/lib/aarch64-linux-gnu/libxml2.so (found version "2.9.13")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/eecheng/helio/build-opt
-- Cache values
ABSL_BUILD_TESTING:BOOL=OFF
ABSL_ENABLE_INSTALL:BOOL=OFF
ABSL_GOOGLETEST_DOWNLOAD_URL:STRING=
ABSL_LOCAL_GOOGLETEST_DIR:PATH=/usr/src/googletest
ABSL_USE_EXTERNAL_GOOGLETEST:BOOL=OFF
ABSL_USE_GOOGLETEST_HEAD:BOOL=OFF
ANL_LIB:FILEPATH=/usr/lib/aarch64-linux-gnu/libanl.a
BENCHMARK_BUILD_32_BITS:BOOL=OFF
BENCHMARK_DOWNLOAD_DEPENDENCIES:BOOL=OFF
BENCHMARK_ENABLE_ASSEMBLY_TESTS:BOOL=OFF
BENCHMARK_ENABLE_DOXYGEN:BOOL=OFF
BENCHMARK_ENABLE_EXCEPTIONS:BOOL=ON
BENCHMARK_ENABLE_GTEST_TESTS:BOOL=ON
BENCHMARK_ENABLE_INSTALL:BOOL=OFF
BENCHMARK_ENABLE_LIBPFM:BOOL=ON
BENCHMARK_ENABLE_LTO:BOOL=OFF
BENCHMARK_ENABLE_TESTING:BOOL=OFF
BENCHMARK_ENABLE_WERROR:BOOL=ON
BENCHMARK_FORCE_WERROR:BOOL=OFF
BENCHMARK_INSTALL_DOCS:BOOL=ON
BENCHMARK_USE_BUNDLED_GTEST:BOOL=ON
BENCHMARK_USE_LIBCXX:BOOL=OFF
BUILD_DOCS:BOOL=ON
BUILD_GMOCK:BOOL=ON
BUILD_SHARED_LIBS:BOOL=OFF
Boost_CONTEXT_LIBRARY_RELEASE:STRING=/opt/boost_1_76_0/lib/libboost_context.a
Boost_FIBER_LIBRARY_RELEASE:STRING=/opt/boost_1_76_0/lib/libboost_fiber.a
Boost_INCLUDE_DIR:PATH=/opt/boost_1_76_0/include
Boost_SYSTEM_LIBRARY_RELEASE:STRING=/opt/boost_1_76_0/lib/libboost_system.a
CMAKE_BUILD_TYPE:STRING=Release
CMAKE_INSTALL_PREFIX:PATH=/usr/local
EXECINFO_LIBRARY:FILEPATH=EXECINFO_LIBRARY-NOTFOUND
FETCHCONTENT_BASE_DIR:PATH=/home/eecheng/helio/build-opt/_deps
FETCHCONTENT_FULLY_DISCONNECTED:BOOL=OFF
FETCHCONTENT_QUIET:BOOL=ON
FETCHCONTENT_SOURCE_DIR_ABSEIL_CPP:PATH=
FETCHCONTENT_SOURCE_DIR_BENCHMARK:PATH=
FETCHCONTENT_SOURCE_DIR_GLOG:PATH=
FETCHCONTENT_SOURCE_DIR_GTEST:PATH=
FETCHCONTENT_UPDATES_DISCONNECTED:BOOL=OFF
FETCHCONTENT_UPDATES_DISCONNECTED_ABSEIL_CPP:BOOL=OFF
FETCHCONTENT_UPDATES_DISCONNECTED_BENCHMARK:BOOL=OFF
FETCHCONTENT_UPDATES_DISCONNECTED_GLOG:BOOL=OFF
FETCHCONTENT_UPDATES_DISCONNECTED_GTEST:BOOL=OFF
HAVE_LIB_GFLAGS:BOOL=OFF
HAVE_RWLOCK:BOOL=OFF
INSTALL_GTEST:BOOL=ON
LIBRT:FILEPATH=/usr/lib/aarch64-linux-gnu/librt.a
PRINT_UNSYMBOLIZED_STACK_TRACES:BOOL=OFF
USE_MOLD:BOOL=OFF
WITH_CUSTOM_PREFIX:BOOL=ON
WITH_GFLAGS:BOOL=OFF
WITH_GTEST:BOOL=OFF
WITH_PKGCONFIG:BOOL=ON
WITH_SYMBOLIZE:BOOL=ON
WITH_THREADS:BOOL=ON
WITH_TLS:BOOL=ON
WITH_UNWIND:BOOL=ON

Here is the message after typing cd build-opt && ninja -j4 echo_server

[180/226] Building CXX object base/CMakeFiles/base.dir/histogram.cc.o
/home/eecheng/helio/base/histogram.cc: In static member function ‘static std::pair<double, double> base::Histogram::BucketLimits(unsigned int)’:
/home/eecheng/helio/base/histogram.cc:41:63: note: parameter passing for argument of type ‘std::pair<double, double>’ when C++17 is enabled changed to match C++14 in GCC 10.1
   41 | inline pair<double, double> Histogram::BucketLimits(unsigned b) {
      |                                                               ^
[226/226] Linking CXX executable echo_server

Improve TLS performance

We currently perform many calls to SSL_write in util::tls::TlsSocket::WriteSome. This has a negative impact on performance and should be improved.

replace yield instruction for aarch64 to isb

See https://github.com/Snapchat/KeyDB/pull/441/files submitted by Ali Saidi for context.

make msgring flag disabled by default

due to the following test being deadlocked:

./multi_test --gtest_filter=*MultiCauseUnblocking* --gtest_repeat=100000 --gtest_break_on_failure --singlehop_blocking=false --alsologtostderr --multi_exec_squash=false

macos compatibility

pmr::monotonic_buffer_resource does not exist on macos
sys/timerfd.h does not exist on macos
cpu_set_t does not exist on macos but there is this blog post that explains how to port them to macos
SOCK_NONBLOCK | SOCK_CLOEXEC do not exist on macos, instead one should use fcntl with FD_CLOEXEC etc.

support native reads and writes into S3

we currently can read only from a local file system. It could be nice if we could read natively from s3.
Also we should consider to support https://github.com/minio/minio (could probably help with e2e tests locally as well).
(I am not sure what does it mean to support min.io since they should be compatible with S3 API)

we already have a POC code here: https://github.com/romange/gaia/tree/master/util/aws
but it will require changes to be adopted into helio - better abstractions, better reliability, no check-fails, use helio sockets etc.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

	static cpu_set_t OnlineCpus() {
	cpu_set_t online_cpus;
	CPU_ZERO(&online_cpus);
	CHECK_EQ(0, sched_getaffinity(0, sizeof(online_cpus), &online_cpus));
	return online_cpus;
	}