shikokuchuo / nanonext Goto Github PK

nanonext - R binding for NNG (Nanomsg Next Gen)

Home Page: http://shikokuchuo.net/nanonext/

License: GNU General Public License v3.0

R 52.41% Shell 0.01% C 47.32% CSS 0.22% C++ 0.04%

rstats r-package nng nanomsg cran r socket-communication cryptographic-hash-functions messaging-library synchronization-primitives

nanonext's People

Contributors

Stargazers

Watchers

Forkers

hibiki-ai nviets spectral-cockpit wlandau jcheng5 gaborcsardi

nanonext's Issues

Patch for missing AI_NUMERICSERV: possible to add?

Could we add this somehow? nanomsg/nng#1651
Since nanonext downloads the source, I cannot patch it prior to installation.

Compilation issue on Ubuntu

When I try to install the current GitHub version nanonext on the rocker/verse:latest Docker image, I get the following issue. I think it relates to the use of a system copy of mbedtls. Is there a straight-forward fix for this?

#0 158.9 -- Install configuration: "Release"
#0 158.9 ** libs
#0 158.9 using C compiler: ‘gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0’
#0 158.9 gcc -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include   -I../install/include -I/usr/include  -fvisibility=hidden -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -Wall -pedantic -c aio.c -o aio.o
#0 158.9 gcc -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include   -I../install/include -I/usr/include  -fvisibility=hidden -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -Wall -pedantic -c cert.c -o cert.o
#0 158.9 cert.c:41:10: fatal error: mbedtls/build_info.h: No such file or directory
#0 158.9    41 | #include <mbedtls/build_info.h>
#0 158.9       |          ^~~~~~~~~~~~~~~~~~~~~~
#0 158.9 compilation terminated.
#0 158.9 make: *** [/usr/local/lib/R/etc/Makeconf:191: cert.o] Error 1
#0 158.9 ERROR: compilation failed for package ‘nanonext’
#0 158.9 * removing ‘/tmp/RtmpE82I5p/pkg-libda7245ba586/nanonext’
#0 158.9 ---
#0 158.9 Backtrace:
#0 158.9 1. base::source("030-targets.R")
#0 158.9 2. | base::withVisible(eval(ei, envir))
#0 158.9 3. | base::eval(ei, envir)
#0 158.9 4. | base::eval(ei, envir)
#0 158.9 5. pak::pak(c("targets", "tarchetypes", "wlandau/crew", "shikokuchuo/mirai", …
#0 158.9 6. pak::pkg_install(pkg, ...)
#0 158.9 7. pak:::remote(function(...) get("pkg_install_do_plan", asNamespace("pak"))(...), …
#0 158.9 8. err$throw(res$error)
#0 158.9 ---
#0 158.9 Subprocess backtrace:
#0 158.9  1. base::withCallingHandlers(cli_message = function(msg) { …
#0 158.9  2. get("pkg_install_do_plan", asNamespace("pak"))(...)
#0 158.9  3. proposal$install()
#0 158.9  4. pkgdepends::install_package_plan(plan, lib = private$library, num_workers = nw, …
#0 158.9  5. base::withCallingHandlers({ …
#0 158.9  6. pkgdepends:::handle_events(state, events)
#0 158.9  7. pkgdepends:::handle_event(state, i)
#0 158.9  8. pkgdepends:::stop_task(state, worker)
#0 158.9  9. pkgdepends:::stop_task_build(state, worker)
#0 158.9 10. base::throw(new_pkg_build_error("Failed to build source package {pkg}", …
#0 158.9 11. | base::signalCondition(cond)
#0 158.9 12. global (function (e) …
#0 158.9 Execution halted
#0 159.0 run-parts: /srv/r-installation/030-targets.sh exited with return code 1

Condition variables to reduce polling in crew

nextget("cv") and mirai(.signal = TRUE) are incredibly useful. We have already discussed cv_value(nextget("cv")) as a constant-time way to count unresolved tasks (shikokuchuo/mirai#78 (reply in thread)). Development crew already does this.

Now, I am thinking of ways that nextget("cv") could reduce polling in both crew and targets. There is a ton of potential. The crux is where crew either:

Waits until at least one task resolves, or
Waits until all tasks resolve.

I would like to use something like until(nextget("cv")). It's simple in theory, but I find myself struggling:

until() decrements the cv. This is a challenge because crew thinks cv_value(nextget("cv")) is total cumulative number of resolved tasks. To use the cv correctly for shikokuchuo/mirai#78 (reply in thread), I have to add back 1 if the cv was originally positive. And it's hard to tell if the cv was originally positive without hitting a race condition.
until() only sleeps if cv_value() is positive. I am not sure how to wait until all tasks resolve. It would be easier if the sleep could start at any value and simply wait for a change. If there were a way to wait until the cv climbs up to a user-defined fixed positive value, then this would be easier still.

Compiler choice building bundled nng 1.6.0

Something does not work:

[ 90%] Building C object CMakeFiles/nng.dir/src/supplemental/http/http_server.c.o
[ 92%] Building C object CMakeFiles/nng.dir/src/supplemental/sha1/sha1.c.o
[ 92%] Building C object CMakeFiles/nng.dir/src/supplemental/tls/mbedtls/tls.c.o
/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/nanonext-0.8.2/nng-8e1836f/src/supplemental/tls/mbedtls/tls.c:605:10: error: 'nng_tls_engine_conn_ops' {aka 'struct nng_tls_engine_conn_ops_s'} has no member named 'peer_cn'
  605 |         .peer_cn        = conn_peer_cn,
      |          ^~~~~~~
/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/nanonext-0.8.2/nng-8e1836f/src/supplemental/tls/mbedtls/tls.c:606:10: error: 'nng_tls_engine_conn_ops' {aka 'struct nng_tls_engine_conn_ops_s'} has no member named 'peer_alt_names'
  606 |         .peer_alt_names = conn_peer_alt_names,
      |          ^~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/nng.dir/src/supplemental/tls/mbedtls/tls.c.o] Error 1
make[1]: *** [CMakeFiles/nng.dir/all] Error 2
gmake: *** [Makefile:156: all] Error 2
-- Install configuration: "Release"
CMake Error at src/cmake_install.cmake:41 (file):
  file INSTALL cannot find
  "/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/nanonext-0.8.2/nng-8e1836f/libnng.a":
  No such file or directory.
Call Stack (most recent call first):
  cmake_install.cmake:42 (include)


** libs
/opt/local/bin/gcc-mp-12 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -isystem/opt/local/include/LegacySupport -I/opt/local/include  -I../install/include -I/opt/local/include   -fPIC  -pipe -Os -arch ppc  -c aio.c -o aio.o
aio.c: In function 'rnng_aio_result':
aio.c:378:7: warning: implicit declaration of function 'nng_aio_busy'; did you mean 'nng_aio_begin'? [-Wimplicit-function-declaration]
  378 |   if (nng_aio_busy(aiop->aio))
      |       ^~~~~~~~~~~~
      |       nng_aio_begin
/opt/local/bin/gcc-mp-12 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -isystem/opt/local/include/LegacySupport -I/opt/local/include  -I../install/include -I/opt/local/include   -fPIC  -pipe -Os -arch ppc  -c core.c -o core.o
/opt/local/bin/gcc-mp-12 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -isystem/opt/local/include/LegacySupport -I/opt/local/include  -I../install/include -I/opt/local/include   -fPIC  -pipe -Os -arch ppc  -c init.c -o init.o
/opt/local/bin/gcc-mp-12 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -isystem/opt/local/include/LegacySupport -I/opt/local/include  -I../install/include -I/opt/local/include   -fPIC  -pipe -Os -arch ppc  -c protocol.c -o protocol.o
/opt/local/bin/gcc-mp-12 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -isystem/opt/local/include/LegacySupport -I/opt/local/include  -I../install/include -I/opt/local/include   -fPIC  -pipe -Os -arch ppc  -c thread.c -o thread.o
/opt/local/bin/gcc-mp-12 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -isystem/opt/local/include/LegacySupport -I/opt/local/include  -I../install/include -I/opt/local/include   -fPIC  -pipe -Os -arch ppc  -c tls.c -o tls.o
/opt/local/bin/gcc-mp-12 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -isystem/opt/local/include/LegacySupport -I/opt/local/include  -I../install/include -I/opt/local/include   -fPIC  -pipe -Os -arch ppc  -c utils.c -o utils.o
/opt/local/bin/gcc-mp-12 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/opt/local/Library/Frameworks/R.framework/Resources/lib -Wl,-headerpad_max_install_names -Wl,-rpath,/opt/local/lib/libgcc -L/opt/local/lib -lMacportsLegacySupport -arch ppc -o nanonext.so aio.o core.o init.o protocol.o thread.o tls.o utils.o -L../install/lib -L/opt/local/lib -lnng -lmbedtls -lmbedx509 -lmbedcrypto -latomic -F/opt/local/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/destroot/opt/local/Library/Frameworks/R.framework/Versions/4.2/Resources/library/00LOCK-nanonext-0.8.2/00new/nanonext/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘nanonext’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/destroot/opt/local/Library/Frameworks/R.framework/Versions/4.2/Resources/library/00LOCK-nanonext-0.8.2/00new/nanonext/libs/nanonext.so':
  dlopen(/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/destroot/opt/local/Library/Frameworks/R.framework/Versions/4.2/Resources/library/00LOCK-nanonext-0.8.2/00new/nanonext/libs/nanonext.so, 6): Symbol not found: _nng_aio_busy
  Referenced from: /opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/destroot/opt/local/Library/Frameworks/R.framework/Versions/4.2/Resources/library/00LOCK-nanonext-0.8.2/00new/nanonext/libs/nanonext.so
  Expected in: dynamic lookup

Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/destroot/opt/local/Library/Frameworks/R.framework/Versions/4.2/Resources/library/nanonext’
Command failed:  cd "/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/nanonext-0.8.2" && /opt/local/bin/R CMD INSTALL . --library=/opt/local/var/macports/build/_opt_PPCRosettaPorts_R_R-nanonext/R-nanonext/work/destroot/opt/local/Library/Frameworks/R.framework/Versions/4.2/Resources/library 
Exit code: 1

nng 1.5.2 builds fine.

nanonext : which kind of protocol we must use for working with 'future' package ?

Hi,

I'm doing some experiments with your package because it provides interesting capacities, especially the one to transport objects. In the tests i'm doing, i try to coordinate message status between my workers and the main r session. The first functional prototype is based on liteq but limited to text messaging. So I'm trying to exteding it with your package.

the code is structured as follows:

library(future)
library(lgr)
library(promises)
library(data.table)
library(stringr)
library(nanonext)
library(magrittr)

dt_tasks <- data.table(
  ID = 1:3,
  name = c( "UC1", "UC2", "UC3"),
  memory = c(100, 150, 200),
  time = c(2, 4, 6),
  treated = rep(F,3)
)

# define a wrapper for initializing workers' environment
nb_workers <- 3
with_two_workers <- function( expr ) {
  old_plan <- future::plan(future::multisession(workers = nb_workers))
  on.exit({future::plan(old_plan)}, add = TRUE)
  force(expr)
  while(!later::loop_empty()) {Sys.sleep(0.1); later::run_now()}
  invisible()
}

nb_of_completed_ops <- 0
with_two_workers(
  {
    # define the workers' tasks
    for(i in 1:nrow(dt_tasks)){
      future_promise(
        {
          s1 <- socket("pair")
          send_aio(s1, str_c( dt_tasks[i,name], "|", str_pad( "Start Treatment",width = 15, side = "right" ), "|PID=", Sys.getpid() ) )
          Sys.sleep( dt_tasks[i,time] )
          send_aio(s1, str_c( dt_tasks[i,name], "|", str_pad( "End Treatment" ,width = 15, side = "right" ), "|PID=", Sys.getpid() ) )
          send_aio(s1, str_c( dt_tasks[i,name], "|", "DONE" ) )
          close(s1)
        },
        packages = c("data.table", "nanonext"), seed = NULL
      ) 
    }
    
    # define Main tasks to monitor the workers alongside
    promise_resolve(
      {
        s2 <- socket("pair")
        lgr$warn( str_c( "MAIN|",str_pad("Start Treatment",width = 15,side = "right"), "|PID=", Sys.getpid()) )
        nb_of_completed_ops <- 0
        while( nb_of_completed_ops < nrow(dt_tasks)) {
          lc_msg <- recv_aio(s2)
          if( unresolved(lc_msg) ){
            lgr$debug( str_c( "MAIN|",str_pad("Waiting...",width = 15,side = "right"),"|PID=", Sys.getpid(), ">nb of completed ops: ", nb_of_completed_ops) )
          } else {
            lgr$info( lc_msg$data )
            if (lc_msg$data == "DONE") {
              nb_of_completed_ops <- nb_of_completed_ops + 1
            }
          }
          Sys.sleep(1)
        }
        lgr$warn( str_c( "MAIN|",str_pad("End Treatment",width = 15,side = "right"), "|PID=", Sys.getpid()) )
        close(s2)
      }
    )
  }
)

the issue is i don't know how to combined nanonext with future/promises:

if i create s1 and s2 outside the workers' creation, future will "detect a non exportable reference"
if i create the two paired socket inside the promises, i got an issue labelled as follows"later: exception occurred while executing callback: Evaluation error: object of type 'closure' is not subsettable."

I'm sure i'm missing something but in your examples, you presents asynchronous interaction in a sequential manner.
So how should I proceed while coding futures / promises ?

Many thanks in advance

Regards,
Bruno

Forwarding to ephemeral condition variables

For controller groups in crew, I would like to be able to wait on an ephemeral condition variable which gets signals from %~>% and whose lifespan is a call to a function. When I simulate many such calls, my R session becomes unresponsive.

library(nanonext)
cv1 <- cv()
f <- function(cv1) {
  cv2 <- cv()
  cv1 %~>% cv2
  until(cv2, 50)
}
for (i in 1:100) {
  print(i)
  f(cv1)
}
1 + 1

Release a version that uses dynamically linked NNG (which is also released)

As I mentioned in wlandau/crew#80, I am working on packaging nanonext for conda-forge. The conda-forge policy is to prefer dynamically linked dependencies rather than bundled, and to very strongly prefer officially released versions. It seems that the current release of nanonext depends on the unreleased v1.6.0 of NNG; while the unreleased version of nanonext has reduced this to requiring only v1.5.1, which is released. Would it be too much trouble for you to push a new release of nanonext which uses the released NNG? Alternatively, do you have insight into when NNG 1.6.0 will be published?

not compiling on university cluster (no sudo access)

Hi,

The install seems to compile MBED fine but then doesn't find it when compiling libnng. Should this just work or am I missing something?

> install.packages("nanonext")
Installing package into ‘/scale_wlg_persistent/filesets/home/kenny.bell/R/library/4.2’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/nanonext_0.9.2.tar.gz'
Content type 'application/x-gzip' length 1101151 bytes (1.1 MB)
==================================================
downloaded 1.1 MB

* installing *source* package ‘nanonext’ ...
** package ‘nanonext’ successfully unpacked and MD5 sums checked
** using staged installation
No existing 'libmbedtls' >= 2 found
Detecting 'cmake'...
/usr/bin/cmake
Detecting 'xz'...
/opt/nesi/CS400_centos7_bdw/XZ/5.2.5-GCCcore-11.3.0/bin/xz
Compiling 'libmbedtls' from source ...
-- The C compiler identification is GNU 11.3.0
-- Check for working C compiler: /opt/nesi/CS400_centos7_bdw/GCCcore/11.3.0/bin/gcc
-- Check for working C compiler: /opt/nesi/CS400_centos7_bdw/GCCcore/11.3.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/mbedtls-1873d3b
Scanning dependencies of target mbedcrypto
[  1%] Building C object library/CMakeFiles/mbedcrypto.dir/aes.c.o
[  2%] Building C object library/CMakeFiles/mbedcrypto.dir/aesni.c.o
[  3%] Building C object library/CMakeFiles/mbedcrypto.dir/aesce.c.o
[  4%] Building C object library/CMakeFiles/mbedcrypto.dir/aria.c.o
[  5%] Building C object library/CMakeFiles/mbedcrypto.dir/asn1parse.c.o
[  6%] Building C object library/CMakeFiles/mbedcrypto.dir/asn1write.c.o
[  7%] Building C object library/CMakeFiles/mbedcrypto.dir/base64.c.o
[  8%] Building C object library/CMakeFiles/mbedcrypto.dir/bignum.c.o
[  9%] Building C object library/CMakeFiles/mbedcrypto.dir/bignum_core.c.o
[ 10%] Building C object library/CMakeFiles/mbedcrypto.dir/bignum_mod.c.o
[ 12%] Building C object library/CMakeFiles/mbedcrypto.dir/bignum_mod_raw.c.o
[ 13%] Building C object library/CMakeFiles/mbedcrypto.dir/camellia.c.o
[ 14%] Building C object library/CMakeFiles/mbedcrypto.dir/ccm.c.o
[ 15%] Building C object library/CMakeFiles/mbedcrypto.dir/chacha20.c.o
[ 16%] Building C object library/CMakeFiles/mbedcrypto.dir/chachapoly.c.o
[ 17%] Building C object library/CMakeFiles/mbedcrypto.dir/cipher.c.o
[ 18%] Building C object library/CMakeFiles/mbedcrypto.dir/cipher_wrap.c.o
[ 19%] Building C object library/CMakeFiles/mbedcrypto.dir/constant_time.c.o
[ 20%] Building C object library/CMakeFiles/mbedcrypto.dir/cmac.c.o
[ 21%] Building C object library/CMakeFiles/mbedcrypto.dir/ctr_drbg.c.o
[ 23%] Building C object library/CMakeFiles/mbedcrypto.dir/des.c.o
[ 24%] Building C object library/CMakeFiles/mbedcrypto.dir/dhm.c.o
[ 25%] Building C object library/CMakeFiles/mbedcrypto.dir/ecdh.c.o
[ 26%] Building C object library/CMakeFiles/mbedcrypto.dir/ecdsa.c.o
[ 27%] Building C object library/CMakeFiles/mbedcrypto.dir/ecjpake.c.o
[ 28%] Building C object library/CMakeFiles/mbedcrypto.dir/ecp.c.o
[ 29%] Building C object library/CMakeFiles/mbedcrypto.dir/ecp_curves.c.o
[ 30%] Building C object library/CMakeFiles/mbedcrypto.dir/entropy.c.o
[ 31%] Building C object library/CMakeFiles/mbedcrypto.dir/entropy_poll.c.o
[ 32%] Building C object library/CMakeFiles/mbedcrypto.dir/error.c.o
[ 34%] Building C object library/CMakeFiles/mbedcrypto.dir/gcm.c.o
[ 35%] Building C object library/CMakeFiles/mbedcrypto.dir/hash_info.c.o
[ 36%] Building C object library/CMakeFiles/mbedcrypto.dir/hkdf.c.o
[ 37%] Building C object library/CMakeFiles/mbedcrypto.dir/hmac_drbg.c.o
[ 38%] Building C object library/CMakeFiles/mbedcrypto.dir/lmots.c.o
[ 39%] Building C object library/CMakeFiles/mbedcrypto.dir/lms.c.o
[ 40%] Building C object library/CMakeFiles/mbedcrypto.dir/md.c.o
[ 41%] Building C object library/CMakeFiles/mbedcrypto.dir/md5.c.o
[ 42%] Building C object library/CMakeFiles/mbedcrypto.dir/memory_buffer_alloc.c.o
[ 43%] Building C object library/CMakeFiles/mbedcrypto.dir/nist_kw.c.o
[ 45%] Building C object library/CMakeFiles/mbedcrypto.dir/oid.c.o
[ 46%] Building C object library/CMakeFiles/mbedcrypto.dir/padlock.c.o
[ 47%] Building C object library/CMakeFiles/mbedcrypto.dir/pem.c.o
[ 48%] Building C object library/CMakeFiles/mbedcrypto.dir/pk.c.o
[ 49%] Building C object library/CMakeFiles/mbedcrypto.dir/pk_wrap.c.o
[ 50%] Building C object library/CMakeFiles/mbedcrypto.dir/pkcs12.c.o
[ 51%] Building C object library/CMakeFiles/mbedcrypto.dir/pkcs5.c.o
[ 52%] Building C object library/CMakeFiles/mbedcrypto.dir/pkparse.c.o
[ 53%] Building C object library/CMakeFiles/mbedcrypto.dir/pkwrite.c.o
[ 54%] Building C object library/CMakeFiles/mbedcrypto.dir/platform.c.o
[ 56%] Building C object library/CMakeFiles/mbedcrypto.dir/platform_util.c.o
[ 57%] Building C object library/CMakeFiles/mbedcrypto.dir/poly1305.c.o
[ 58%] Building C object library/CMakeFiles/mbedcrypto.dir/ripemd160.c.o
[ 59%] Building C object library/CMakeFiles/mbedcrypto.dir/rsa.c.o
[ 60%] Building C object library/CMakeFiles/mbedcrypto.dir/rsa_alt_helpers.c.o
[ 61%] Building C object library/CMakeFiles/mbedcrypto.dir/sha1.c.o
[ 62%] Building C object library/CMakeFiles/mbedcrypto.dir/sha256.c.o
[ 63%] Building C object library/CMakeFiles/mbedcrypto.dir/sha512.c.o
[ 64%] Building C object library/CMakeFiles/mbedcrypto.dir/threading.c.o
[ 65%] Building C object library/CMakeFiles/mbedcrypto.dir/timing.c.o
[ 67%] Building C object library/CMakeFiles/mbedcrypto.dir/version.c.o
[ 68%] Building C object library/CMakeFiles/mbedcrypto.dir/version_features.c.o
[ 69%] Linking C static library libmbedcrypto.a
[ 69%] Built target mbedcrypto
Scanning dependencies of target mbedx509
[ 70%] Building C object library/CMakeFiles/mbedx509.dir/pkcs7.c.o
[ 71%] Building C object library/CMakeFiles/mbedx509.dir/x509.c.o
[ 72%] Building C object library/CMakeFiles/mbedx509.dir/x509_create.c.o
[ 73%] Building C object library/CMakeFiles/mbedx509.dir/x509_crl.c.o
[ 74%] Building C object library/CMakeFiles/mbedx509.dir/x509_crt.c.o
[ 75%] Building C object library/CMakeFiles/mbedx509.dir/x509_csr.c.o
[ 76%] Building C object library/CMakeFiles/mbedx509.dir/x509write_crt.c.o
[ 78%] Building C object library/CMakeFiles/mbedx509.dir/x509write_csr.c.o
[ 79%] Linking C static library libmbedx509.a
[ 79%] Built target mbedx509
Scanning dependencies of target mbedtls
[ 80%] Building C object library/CMakeFiles/mbedtls.dir/debug.c.o
[ 81%] Building C object library/CMakeFiles/mbedtls.dir/mps_reader.c.o
[ 82%] Building C object library/CMakeFiles/mbedtls.dir/mps_trace.c.o
[ 83%] Building C object library/CMakeFiles/mbedtls.dir/net_sockets.c.o
[ 84%] Building C object library/CMakeFiles/mbedtls.dir/ssl_cache.c.o
[ 85%] Building C object library/CMakeFiles/mbedtls.dir/ssl_ciphersuites.c.o
[ 86%] Building C object library/CMakeFiles/mbedtls.dir/ssl_client.c.o
[ 87%] Building C object library/CMakeFiles/mbedtls.dir/ssl_cookie.c.o
[ 89%] Building C object library/CMakeFiles/mbedtls.dir/ssl_debug_helpers_generated.c.o
[ 90%] Building C object library/CMakeFiles/mbedtls.dir/ssl_msg.c.o
[ 91%] Building C object library/CMakeFiles/mbedtls.dir/ssl_ticket.c.o
[ 92%] Building C object library/CMakeFiles/mbedtls.dir/ssl_tls.c.o
[ 93%] Building C object library/CMakeFiles/mbedtls.dir/ssl_tls12_client.c.o
[ 94%] Building C object library/CMakeFiles/mbedtls.dir/ssl_tls12_server.c.o
[ 95%] Building C object library/CMakeFiles/mbedtls.dir/ssl_tls13_keys.c.o
[ 96%] Building C object library/CMakeFiles/mbedtls.dir/ssl_tls13_server.c.o
[ 97%] Building C object library/CMakeFiles/mbedtls.dir/ssl_tls13_client.c.o
[ 98%] Building C object library/CMakeFiles/mbedtls.dir/ssl_tls13_generic.c.o
[100%] Linking C static library libmbedtls.a
[100%] Built target mbedtls
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/mbedtls-1873d3b
No existing 'libnng' >= 1.5 found
Detecting 'cmake'...
/usr/bin/cmake
Detecting 'xz'...
/opt/nesi/CS400_centos7_bdw/XZ/5.2.5-GCCcore-11.3.0/bin/xz
Compiling 'libnng' from source ...
-- The C compiler identification is GNU 11.3.0
-- Check for working C compiler: /opt/nesi/CS400_centos7_bdw/GCCcore/11.3.0/bin/gcc
-- Check for working C compiler: /opt/nesi/CS400_centos7_bdw/GCCcore/11.3.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring for NNG version 1.6.0-pre
-- Performing Test NNG_HIDDEN_VISIBILITY
-- Performing Test NNG_HIDDEN_VISIBILITY - Success
-- Looking for strlcpy
-- Looking for strlcpy - not found
-- Looking for strnlen
-- Looking for strnlen - found
-- Looking for strcasecmp
-- Looking for strcasecmp - found
-- Looking for strncasecmp
-- Looking for strncasecmp - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Looking for lockf
-- Looking for lockf - found
-- Looking for flock
-- Looking for flock - found
-- Looking for getrandom
-- Looking for getrandom - not found
-- Looking for arc4random_buf
-- Looking for arc4random_buf - not found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for sem_wait in pthread
-- Looking for sem_wait in pthread - found
-- Looking for pthread_atfork in pthread
-- Looking for pthread_atfork in pthread - found
-- Looking for pthread_set_name_np in pthread
-- Looking for pthread_set_name_np in pthread - not found
-- Looking for pthread_setname_np in pthread
-- Looking for pthread_setname_np in pthread - found
-- Looking for gethostbyname in nsl
-- Looking for gethostbyname in nsl - found
-- Looking for socket in socket
-- Looking for socket in socket - not found
-- Looking for __atomic_load_1 in atomic
-- Looking for __atomic_load_1 in atomic - found
-- Looking for AF_UNIX
-- Looking for AF_UNIX - found
-- Looking for backtrace_symbols_fd
-- Looking for backtrace_symbols_fd - found
-- Performing Test NNG_HAVE_MSG_CONTROL
-- Performing Test NNG_HAVE_MSG_CONTROL - Success
-- Looking for eventfd
-- Looking for eventfd - found
-- Looking for kqueue
-- Looking for kqueue - not found
-- Looking for port_create
-- Looking for port_create - not found
-- Looking for epoll_create
-- Looking for epoll_create - found
-- Looking for epoll_create1
-- Looking for epoll_create1 - found
-- Looking for getpeereid
-- Looking for getpeereid - not found
-- Looking for SO_PEERCRED
-- Looking for SO_PEERCRED - found
-- Performing Test NNG_HAVE_SOCKPEERCRED
-- Performing Test NNG_HAVE_SOCKPEERCRED - Failed
-- Looking for LOCAL_PEERCRED
-- Looking for LOCAL_PEERCRED - not found
-- Looking for LOCAL_PEERPID
-- Looking for LOCAL_PEERPID - not found
-- Looking for getpeerucred
-- Looking for getpeerucred - not found
-- Looking for atomic_flag_test_and_set
-- Looking for atomic_flag_test_and_set - found
CMake Error at cmake/FindmbedTLS.cmake:78 (file):
  file STRINGS file
  "/tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/nng-c5e9d8a/src/supplemental/tls/mbedtls/MBEDTLS_INCLUDE_DIR-NOTFOUND/mbedtls/version.h"
  cannot be read.
Call Stack (most recent call first):
  cmake/NNGHelpers.cmake:46 (find_package)
  src/supplemental/tls/mbedtls/CMakeLists.txt:21 (nng_find_package)


CMake Error at cmake/FindmbedTLS.cmake:82 (string):
  string sub-command REGEX, mode REPLACE needs at least 6 arguments total to
  command.
Call Stack (most recent call first):
  cmake/NNGHelpers.cmake:46 (find_package)
  src/supplemental/tls/mbedtls/CMakeLists.txt:21 (nng_find_package)


CMake Error at /usr/share/cmake3/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find mbedTLS (missing: MBEDTLS_TLS_LIBRARY MBEDTLS_CRYPTO_LIBRARY
  MBEDTLS_X509_LIBRARY MBEDTLS_INCLUDE_DIR MBEDTLS_LIBRARIES)
Call Stack (most recent call first):
  /usr/share/cmake3/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindmbedTLS.cmake:84 (find_package_handle_standard_args)
  cmake/NNGHelpers.cmake:46 (find_package)
  src/supplemental/tls/mbedtls/CMakeLists.txt:21 (nng_find_package)


-- Configuring incomplete, errors occurred!
See also "/tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/nng-c5e9d8a/CMakeFiles/CMakeOutput.log".
See also "/tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/nng-c5e9d8a/CMakeFiles/CMakeError.log".
gmake: *** No targets specified and no makefile found.  Stop.
-- Configuring for NNG version 1.6.0-pre
CMake Error at cmake/FindmbedTLS.cmake:78 (file):
  file STRINGS file
  "/tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/nng-c5e9d8a/src/supplemental/tls/mbedtls/MBEDTLS_INCLUDE_DIR-NOTFOUND/mbedtls/version.h"
  cannot be read.
Call Stack (most recent call first):
  cmake/NNGHelpers.cmake:46 (find_package)
  src/supplemental/tls/mbedtls/CMakeLists.txt:21 (nng_find_package)


CMake Error at cmake/FindmbedTLS.cmake:82 (string):
  string sub-command REGEX, mode REPLACE needs at least 6 arguments total to
  command.
Call Stack (most recent call first):
  cmake/NNGHelpers.cmake:46 (find_package)
  src/supplemental/tls/mbedtls/CMakeLists.txt:21 (nng_find_package)


CMake Error at /usr/share/cmake3/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find mbedTLS (missing: MBEDTLS_TLS_LIBRARY MBEDTLS_CRYPTO_LIBRARY
  MBEDTLS_X509_LIBRARY MBEDTLS_INCLUDE_DIR MBEDTLS_LIBRARIES)
Call Stack (most recent call first):
  /usr/share/cmake3/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindmbedTLS.cmake:84 (find_package_handle_standard_args)
  cmake/NNGHelpers.cmake:46 (find_package)
  src/supplemental/tls/mbedtls/CMakeLists.txt:21 (nng_find_package)


-- Configuring incomplete, errors occurred!
See also "/tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/nng-c5e9d8a/CMakeFiles/CMakeOutput.log".
See also "/tmp/RtmpcgXGvI/R.INSTALL24ff35ac08d59/nanonext/nng-c5e9d8a/CMakeFiles/CMakeError.log".
** libs
gcc -I"/opt/nesi/CS400_centos7_bdw/R/4.2.1-gimkl-2022a/lib64/R/include" -DNDEBUG   -I/opt/nesi/CS400_centos7_bdw/OpenSSL/1.1.1k-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/nodejs/16.15.1-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/Java/17/include -I/opt/nesi/CS400_centos7_bdw/GMP/6.2.1-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/NLopt/2.7.0-GCC-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/cURL/7.83.1-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/SQLite/3.36.0-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/libxml2/2.9.10-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/libpng/1.6.37-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/ncurses/6.2-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/libreadline/8.1-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/PCRE2/10.40-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/XZ/5.2.5-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/bzip2/1.0.8-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/zlib/1.2.11-GCCcore-11.3.0/include -I/opt/nesi/CS400_centos7_bdw/imkl/2022.0.2/mkl/2022.0.2/include  -fvisibility=hidden -fpic  -O3 -march=x86-64-v3 -mno-recip  -c aio.c -o aio.o
In file included from aio.c:20:
nanonext.h:22:10: fatal error: nng/nng.h: No such file or directory
   22 | #include <nng/nng.h>
      |          ^~~~~~~~~~~
compilation terminated.
make: *** [aio.o] Error 1
ERROR: compilation failed for package ‘nanonext’
* removing ‘/scale_wlg_persistent/filesets/home/kenny.bell/R/library/4.2/nanonext’

The downloaded source packages are in
        ‘/tmp/RtmpV1QCTU/downloaded_packages’
Warning message:
In install.packages("nanonext") :
  installation of package ‘nanonext’ had non-zero exit status

Performance of TLS encryption with a small number of small tasks

Discussed in #26

^{Originally posted by wlandau January 2, 2024}
Hi @shikokuchuo,

I hope you enjoyed your holidays!

Since mirai and nanonext are more robust than ever, I find myself thinking about performance, specifically when it comes to TLS encryption.

On my local Ubuntu machine, the following example with 100 tasks and TLS encryption takes almost 9 seconds to complete. I notice similar encryption-specific overhead when testing crew.aws.batch using real-life AWS Batch jobs.

library(nanonext) # 0.11.0.9005
library(mirai) # 0.11.3.9003
daemons(n = 1L, dispatcher = TRUE, url = host_url(ws = TRUE, tls = TRUE))
url <- rownames(status()$daemons)
cv <- nextget("cv")
cv2 <- cv()
cv %~>% cv2
system(launch_remote(url = url), wait = FALSE)
msleep(5000)
n <- 1e2
start <- proc.time()["elapsed"]
tasks <- replicate(n, mirai(TRUE))
while (cv_value(cv) < n) {
  until_(cv2, 500)
}
lapply(tasks, unresolved)
elapsed <- unname(proc.time()["elapsed"] - start)
daemons(n = 0L)
print(elapsed)

When I turn off encryption, the same workload completes in under a tenth of a second.

library(nanonext) # 0.11.0.9005
library(mirai) # 0.11.3.9003
daemons(n = 1L, dispatcher = TRUE)
url <- rownames(status()$daemons)
cv <- nextget("cv")
cv2 <- cv()
cv %~>% cv2
launch_local(url = url)
msleep(2000)
n <- 1e2
start <- proc.time()["elapsed"]
tasks <- replicate(n, mirai(TRUE))
while (cv_value(cv) < n) {
  until_(cv2, 500)
}
lapply(tasks, unresolved)
elapsed <- unname(proc.time()["elapsed"] - start)
daemons(n = 0L)
print(elapsed)

I know there is inevitable overhead from encryption/decryption and maybe extra steps in the handshake, but I would not have expected the slowdown to be quite this much, especially given how fast HTTP and HTTPS both load web pages.

You may already know where the bottleneck comes from, but I set up a quick profiling study anyway using gperftools and a few modifications in wlandau@dda175d:

library(nanonext) # my fork
library(mirai) # 0.11.3.9003
daemons(n = 1L, dispatcher = TRUE, url = host_url(ws = TRUE, tls = TRUE))
url <- rownames(status()$daemons)
cv <- nextget("cv")
cv2 <- cv()
cv %~>% cv2
system(launch_remote(url = url), wait = FALSE)
msleep(5000)
n <- 1e2
samples <- tempfile()
profiler_start(path = samples)
tasks <- replicate(n, mirai(TRUE))
while (cv_value(cv) < n) {
  until_(cv2, 500)
}
daemons(n = 0L)
profiler_stop()
proffer::serve_pprof(pprof = samples)

Unfortunately the flame graph is not useful. Looks like either a standard library function or system call is taking the most time, or the bottleneck comes from a non-interactive external process or thread.

Trouble using until()

I am trying to use mirai::nextget("cv") in controller$wait() and controller$map() in crew in order to avoid the worst aspects of polling. But I am finding that nanonext::until() somehow decrements the condition variable. This leaves me with the unfortunate choice of (1) do not wait on nextget("cv") as I would like to do in #17, or (2) go back to the inefficient way of counting unresolved tasks (i.e. revert wlandau/crew#131) because cv_value(nextget("cv")) can no longer be trusted to count resolved ones. Reprex:

library(mirai)
library(nanonext)
url <- "ws://127.0.0.1:5700"
daemons(n = 1L, url = url)
launch_local(url = url)
cv <- nextget("cv")
cv_value(cv)
#> [1] 0
task <- mirai(TRUE, .signal = TRUE)
Sys.sleep(0.25)
cv_value(cv)
#> [1] 1
task <- mirai(Sys.sleep(1), .signal = TRUE)
until(cv, 2000)
cv_value(cv) # I would expect 2.
#> [1] 0
daemons(n = 0L)

Building errors - [ Fedora using system libs ]

I'm having trouble bringing nanonext to this repository for Fedora, as you can see from the list of failed builds. Both required libraries are available in Fedora, but I get the following errors:

** using staged installation
NANONEXT_SYS flag set
Using 'libnng' and 'libmbedtls' installed in '/usr' ...
** libs
gcc -m64 -I"/usr/include/R" -DNDEBUG   -I/usr/local/include  -I/usr/include  -fvisibility=hidden -fpic  -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -c aio.c -o aio.o
aio.c: In function 'rnng_aio_result':
aio.c:148:7: warning: implicit declaration of function 'nng_aio_busy'; did you mean 'nng_aio_begin'? [-Wimplicit-function-declaration]
  148 |   if (nng_aio_busy(aiop->aio))
      |       ^~~~~~~~~~~~
      |       nng_aio_begin
gcc -m64 -I"/usr/include/R" -DNDEBUG   -I/usr/local/include  -I/usr/include  -fvisibility=hidden -fpic  -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -c core.c -o core.o
gcc -m64 -I"/usr/include/R" -DNDEBUG   -I/usr/local/include  -I/usr/include  -fvisibility=hidden -fpic  -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -c init.c -o init.o
gcc -m64 -I"/usr/include/R" -DNDEBUG   -I/usr/local/include  -I/usr/include  -fvisibility=hidden -fpic  -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -c opts.c -o opts.o
gcc -m64 -I"/usr/include/R" -DNDEBUG   -I/usr/local/include  -I/usr/include  -fvisibility=hidden -fpic  -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -c sha.c -o sha.o
sha.c: In function 'rnng_sha224':
sha.c:78:8: error: void value not ignored as it ought to be
   78 |     xc = mbedtls_sha256(buf, sz, output, 1);
      |        ^
sha.c: In function 'rnng_sha256':
sha.c:159:8: error: void value not ignored as it ought to be
  159 |     xc = mbedtls_sha256(buf, sz, output, 0);
      |        ^
sha.c: In function 'rnng_sha384':
sha.c:240:8: error: void value not ignored as it ought to be
  240 |     xc = mbedtls_sha512(buf, sz, output, 1);
      |        ^
sha.c: In function 'rnng_sha512':
sha.c:321:8: error: void value not ignored as it ought to be
  321 |     xc = mbedtls_sha512(buf, sz, output, 0);
      |        ^
make: *** [/usr/lib64/R/etc/Makeconf:168: sha.o] Error 1
ERROR: compilation failed for package 'nanonext'
* removing '/builddir/build/BUILDROOT/R-CRAN-nanonext-0.5.5-1.fc36.copr4863362.x86_64/usr/local/lib/R/library/nanonext'
error: Bad exit status from /var/tmp/rpm-tmp.XWG99o (%install)

You can check the full log here.

On the other hand, it would be great if the package could automatically detect if the system libraries are available under /usr (not only /usr/local) and use them in such a case (here I patched the configure to use /usr instead of /usr/local).

Installation error "bad variable name"

I am trying to install nanotext for use with mirai and targets. I get the following error:

> install.packages("nanotext")
Installing package into ‘/home/eric/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
Warning message:
package ‘nanotext’ is not available for this version of R

A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages 
> install.packages("nanonext", repos = "https://shikokuchuo.r-universe.dev")
Installing package into ‘/home/eric/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
trying URL 'https://shikokuchuo.r-universe.dev/src/contrib/nanonext_0.10.0.9000.tar.gz'
Content type 'application/x-gzip' length 1164744 bytes (1.1 MB)
==================================================
downloaded 1.1 MB

* installing *source* package ‘nanonext’ ...
** using staged installation
./configure: 16: export: -std: bad variable name
ERROR: configuration failed for package ‘nanonext’
* removing ‘/home/eric/R/x86_64-pc-linux-gnu-library/4.0/nanonext’

The downloaded source packages are in
        ‘/home/eric/tmp/RtmpVhNrdR/downloaded_packages’
Warning message:
In install.packages("nanonext", repos = "https://shikokuchuo.r-universe.dev") :
  installation of package ‘nanonext’ had non-zero exit status

Any advice? My R version is 4.0.3 and I am using debian buster. Thank you.

Wait on multiple condition variables?

I really like nextget("cv") in mirai, and I think it could make polling in targets more gentle. However, in crew, there is a feature called controller groups which aggregates multiple controllers under a single interface. This feature meets the request from shikokuchuo/mirai#28 for heterogeneous workers. Each controller in the controller group has its own condition variable, and it would be great to be able to wait until at least one of these condition variables gets signaled.

nanonext is mutating the `NA` value shared across all R packages

Originally reported at r-lib/tidyselect#265.

Expected behaviour:

fn <- function() NA
fn <- compiler::cmpfun(fn)

attributes(fn())
#> NULL

Unexpected:

library(nanonext)
s <- socket(dial = "inproc://nano")
r <- recv_aio(s)
r$data

attributes(fn())
#> $class
#> [1] "unresolvedValue"

This is a critical bug that breaks all sorts of things in other packages (for instance devtools::load_all() fails after that).

The problem occurs here:

nanonext/src/aio.c

Lines 276 to 277 in 2dfe7e6

 PROTECT(res = Rf_ScalarLogical(NA_LOGICAL)); 

 Rf_classgets(res, Rf_mkString("unresolvedValue"));

. Before using mutating functions like classgets(), you should check that you own the object. ScalarLogical() returns shared objects. You can duplicate() before setting the class to fix this. Note that there is also a protection error, you should protect the result of mkString() before calling classgets() since the latter allocates, which triggers GC.

Unrelatedly, since S3 classes are not namespaced, you should in principle soft-namespace all S3 classes with your package name. For instance the nanoContext class belongs in principle to the nano package. A proper class name for your package would be nanonextContext.

Installation issue on Raspberry Pi 4

Hi, thanks for this package.

I want to install nanonext on a Raspberry Pi 4. I build R (v4.12) and libnng from source.
Installation from cran gives an error:

Thanks for any suggestions


* installing *source* package ‘nanonext’ ...
** package ‘nanonext’ successfully unpacked and MD5 sums checked
** using staged installation
Found libnng system installation...
** libs
gcc -I"/opt/R/4.1.2/lib/R/include" -DNDEBUG   -I/usr/local/include  -I/usr/local/include  -fvisibility=hidden -fpic  -g -O2  -c core.c -o core.o
gcc -I"/opt/R/4.1.2/lib/R/include" -DNDEBUG   -I/usr/local/include  -I/usr/local/include  -fvisibility=hidden -fpic  -g -O2  -c init.c -o init.o
gcc -I"/opt/R/4.1.2/lib/R/include" -DNDEBUG   -I/usr/local/include  -I/usr/local/include  -fvisibility=hidden -fpic  -g -O2  -c opts.c -o opts.o
gcc -I"/opt/R/4.1.2/lib/R/include" -DNDEBUG   -I/usr/local/include  -I/usr/local/include  -fvisibility=hidden -fpic  -g -O2  -c sockets.c -o sockets.o
gcc -I"/opt/R/4.1.2/lib/R/include" -DNDEBUG   -I/usr/local/include  -I/usr/local/include  -fvisibility=hidden -fpic  -g -O2  -c utils.c -o utils.o
gcc -shared -L/opt/R/4.1.2/lib/R/lib -L/usr/local/lib -o nanonext.so core.o init.o opts.o sockets.o utils.o -L/usr/local/lib -lnng -L/opt/R/4.1.2/lib/R/lib -lR
installing to /home/XXX/R/armv7l-unknown-linux-gnueabihf-library/4.1/00LOCK-nanonext/00new/nanonext/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘nanonext’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/home/XXX/R/armv7l-unknown-linux-gnueabihf-library/4.1/00LOCK-nanonext/00new/nanonext/libs/nanonext.so':
  /home/XXX/R/armv7l-unknown-linux-gnueabihf-library/4.1/00LOCK-nanonext/00new/nanonext/libs/nanonext.so: undefined symbol: __atomic_fetch_sub_8
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/home/XXX/R/armv7l-unknown-linux-gnueabihf-library/4.1/nanonext’

The downloaded source packages are in
        ‘/tmp/RtmpT6GSCD/downloaded_packages’
Warning message:
In install.packages("nanonext") :
  installation of package ‘nanonext’ had non-zero exit status

System endianness bug affecting nanonext 0.9.2

nanonext 0.9.2 brought performance enhancements to serialisation / unserialisation by moving this to a lower level.

Part of the gains was to allow use of system endianness for serialisation, which is fine as unserialisation can automatically account for this. So this is not an issue even for sending/receiving between systems with different endianness.

However the same underlying functions were adopted by the sha / base64 functions, which is now problematic as the output will differ depending on system endianness.

	PROTECT(res = Rf_ScalarLogical(NA_LOGICAL));
	Rf_classgets(res, Rf_mkString("unresolvedValue"));