Git Product home page Git Product logo

prqlc-r's Introduction

prqlr

prqlr status badge CRAN status

R bindings for the prqlc Rust library, powered by savvy.

This version supports PRQL 0.11.2.

Installation

Requires R 4.2.0 or later.

This package can be installed from CRAN or R-universe. If available, a binary package will be installed.

# Install from CRAN
install.packages("prqlr")
# Install from R-universe
install.packages("prqlr", repos = "https://prql.r-universe.dev")

For source installation, pre-built Rust libraries may be available if the environment variable NOT_CRAN is set to "true". (Or, set LIBPRQLR_BUILD to "false")

Sys.setenv(NOT_CRAN = "true")
install.packages("prqlr")

Or, the Rust toolchain (Rust 1.65 or later) must be configured to build the Rust library.

Please check the https://github.com/r-rust/hellorust repository for about Rust code in R packages.

Examples

library(prqlr)

"from mtcars | filter cyl > 6 | select {cyl, mpg}" |>
  prql_compile() |>
  cat()
#> SELECT
#>   cyl,
#>   mpg
#> FROM
#>   mtcars
#> WHERE
#>   cyl > 6
#> 
#> -- Generated by PRQL compiler version:0.11.2 (https://prql-lang.org)

PRQL’s pipelines can be joined by the newline character (\n), or actual newlines in addition to |.

"from mtcars \n filter cyl > 6 \n select {cyl, mpg}" |>
  prql_compile() |>
  cat()
#> SELECT
#>   cyl,
#>   mpg
#> FROM
#>   mtcars
#> WHERE
#>   cyl > 6
#> 
#> -- Generated by PRQL compiler version:0.11.2 (https://prql-lang.org)
"from mtcars
filter cyl > 6
select {cyl, mpg}" |>
  prql_compile() |>
  cat()
#> SELECT
#>   cyl,
#>   mpg
#> FROM
#>   mtcars
#> WHERE
#>   cyl > 6
#> 
#> -- Generated by PRQL compiler version:0.11.2 (https://prql-lang.org)

Thanks to the {tidyquery} package, we can even convert a PRQL query to a SQL query and then to a {dplyr} query!

"from mtcars
filter cyl > 6
select {cyl, mpg}" |>
  prql_compile() |>
  tidyquery::show_dplyr()
#> mtcars %>%
#>   filter(cyl > 6) %>%
#>   select(cyl, mpg)

{knitr} integration

Using {prqlr} with {knitr} makes it easy to create documents that lists PRQL queries and a translated SQL queries, or documents that lists PRQL queries and tables of data retrieved by PRQL queries.

Please check the vignette vignette("knitr", "prqlr") for details.

prqlc-r's People

Contributors

eitsupi avatar dependabot[bot] avatar yutannihilation avatar sorhawell avatar yihui avatar

Stargazers

Srikanth K S avatar David Ranzolin avatar Maximilian Muecke avatar  avatar Sebastien Vaucouleur avatar Charlie Gao avatar José de Jesus Filho avatar  avatar hatayou avatar Kirill Müller avatar Jone Keat Lim avatar Dan avatar Nicholas Erskine avatar Kenneth Blake Vernon avatar Michael Sumner avatar  avatar Daniel Fahey avatar Josiah Parry avatar Romain Lesur avatar Andrew Allen Bruce avatar Philipp Baumann avatar Turgut avatar b4D8 avatar Frans van Dunné avatar  avatar Xiangyun Huang avatar  avatar Aaron Simumba avatar E. David Aja avatar John MacKintosh avatar Senku avatar Indrajeet Patil avatar Maximilian Roos avatar Michael Milton avatar th3888 avatar Taras Novak avatar Justin Marciszewski avatar Carl Boettiger avatar  avatar  avatar Pedro Z avatar Mohammed Hamdy avatar Uchida Mizuki avatar Leo Lee avatar  avatar timelyportfolio avatar Shinya Uryu avatar ITO Takafumi avatar  avatar Justin Singh-M. - NOAA avatar Tobias Brandt avatar

Watchers

timelyportfolio avatar Maximilian Roos avatar  avatar  avatar

prqlc-r's Issues

CRAN acceptable inst/AUTHORS format

I received an email:

Thanks, we see:

 person(given = "The authors of the dependency Rust crates", role = c("aut"),
        comment = "see inst/AUTHORS file for details"))

is not acceptable, especially as the file has things like

addr2line (version 0.21.0):
  addr2line authors

anstyle (version 1.0.2):
  anstyle authors

anstyle-parse (version 0.2.1):
  anstyle-parse authors

anstyle-query (version 1.0.0):
  anstyle-query authors

anstyle-wincon (version 1.0.2):
  anstyle-wincon authors

is-terminal (version 0.4.9):
  softprops, Dan Gohman

itertools (version 0.10.5):
  bluss

itertools (version 0.11.0):
  bluss

Please fix and resubmit.

Originally posted by @eitsupi in #167 (comment)

Feature request: Allow for the use of ?variables on prql code chunks

I would like to be able to use R variables inside prql code chunks as you can do with SQL code chunks, it would be useful for parameterized documents. For example:

```{r}
#| echo: false

library(DBI)
library(prqlr)
con <- dbConnect(RSQLite::SQLite(), ":memory:")
dbWriteTable(con, "mtcars", mtcars)

cyl_number <- 6
```

```{prql}
#| connection: con

from mtcars
filter cyl > ?cyl_number
select [cyl, mpg]
derive [mpg_int = round 0 mpg]
take 3
```

Or maybe is already possible but the syntax is different and is not documented?

Update for "Using Rust in CRAN packages"

2023-07-12, CRAN added a new policy for R packages including Rust codes.
https://cran.r-project.org/web/packages/using_rust.html

Packages wishing to use Rust code should either

  • include the code in the package, or
  • download a specific version from a secure and reliable site and check that the download is the expected code by some sort of checksum. The expected checksum needs to be embedded in the source package.

In either case, the authorship and copyright information for the Rust code must be included in the DESCRIPTION file. That includes any Rust sources included as dependencies (see the CRAN policy).

The package should declare
SystemRequirements: Cargo (Rust's package manager), rustc

The configure/configure.win script should check for the presence of commands cargo and rustc, and check their versions if required. This includes checking for system versions on the path and personal versions in ~/.cargo/bin (which are often not on the path). The Linux servers on the CRAN check farm use system versions, and Linux distributions are often slow to update these so version requirements need to be conservative.

cargo build -j N defaults to the number of ‘logical CPUs’. This usually exceeds the maximum allowed in the CRAN policy, so needs to be set explicitly to N=1 or 2.

Downloading should be avoided if at all possible. The package would become uninstallable if the Internet resources are temporarily or permanently unavailable, and CRAN packages are kept available for many years. CRAN does not regard github.com (which hosts the index of crates.io and many of the crates) as sufficiently reliable.

  • Adds inst/AUTHORS (#150, #154, #160, #169, #172)
  • Use SystemRequirements: Cargo (Rust's package manager), rustc (#153)
  • Adds configure scripts (#149)
  • Set cargo build -j 2 if not NOT_CRAN=true (#151)
  • Vendoring Rust dependent crates (#152, #159)
  • LICENSE.note should includes build time dependencies. (#156)

Incorrect target name in query header may not result in an error

Ideally, both of the following should be errors.

"prql target:sql.duckd\nfrom a" |> prqlr::prql_compile()
#> Error in unwrap(compile(prql_query, target, format, signature_comment)) :
#>   dialect `"duckd"` not found
"prql target:duckd\nfrom a" |> prqlr::prql_compile()
#> "SELECT\n  *\nFROM\n  a\n\n-- Generated by PRQL compiler version:0.5.0 (https://prql-lang.org)\n"

Support arm64 cross-compile on R-universe

See mlondschien/changeforest#165 (comment)

Currently, there is also an issue with not correctly detecting pre-built binaries.

In polars:

Run r-universe-org/build-and-check/macos-cross@v1
  with:
    sourcepkg: polars_0.12.1.9000.tar.gz
  env:
    MY_UNIVERSE: https://etiennebacher.r-universe.dev/
    GH_APP_ID: 87942
    R_HOME_CROSS: /opt/R-4.3-arm64
    PKG_CONFIG_PATH: /opt/R/x86_64/lib/pkgconfig:/opt/R/x86_64/share/pkgconfig:/usr/lib/pkgconfig:/opt/X11/lib/pkgconfig:/opt/X11/share/pkgconfig
    PROJ_LIB: /opt/R/x86_64/share/proj
    R_LIBS_USER: /Users/runner/work/_temp/Library
    TZ: UTC
    _R_CHECK_SYSTEM_CLOCK_: FALSE
    JAVA_HOME: /Users/runner/hostedtoolcache/Java_Temurin-Hotspot_jdk/11.0.21-9/x64/Contents/Home/
    MACOSX_DEPLOYMENT_TARGET: 11.0
Run sed -i.bak 's|x86_64-apple-darwin20|aarch64-apple-darwin20|g' $(R RHOME)/etc/Makeconf
* installing *source* package ‘polars’ ...
** using staged installation

--------------------- [SETTING FOR R-UNIVERSE] ---------------------
It seems that this is on R-universe <https://etiennebacher.r-universe.dev/>.
Trying to download pre-built binary.
--------------------------------------------------------------------


---------------- [TRY TO DOWNLOAD PRE-BUILT BINARY] ----------------
Found pre-built binary at <https://github.com/pola-rs/r-polars/releases/download/lib-v0.36.0/libr_polars-0.36.0-x86_64-apple-darwin.tar.gz>.
Downloading...
Checking SHA256 for </var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//RtmpICrxFs/file1670368ba777.tar.gz>...
SHA256 matches for </var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//RtmpICrxFs/file1670368ba777.tar.gz>.
Extracted pre-built binary to </private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/RtmpVh7J7E/R.INSTALL126026d484aa/polars/tools> directory.
--------------------------------------------------------------------

Rust lib binary release

Upload the binaries to the GitHub releases as string2path used to do.
#187 has made Makefiles cross-compilation compatible, so I think we can release arm64 binaries as well.

I would like to record the base URL and lib version in the DESCRIPTION file and set it up to try to download binaries when NOT_CRAN=ture (like the arrow package does).

Installation issue on Windows

When I test with devtools::check_win_release() on 7f12537, installation failed with this log:

* installing *source* package 'prqlr' ...
** using staged installation
** libs
Warning: this package has a non-empty 'configure.win' file,
so building only the main architecture

using C compiler: 'gcc.exe (GCC) 12.2.0'
rm -Rf "prqlr.dll" "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target/x86_64-pc-windows-gnu/release/libprqlr.a" "entrypoint.o"
gcc  -I"D:/RCompile/recent/R-4.3.1/include" -DNDEBUG     -I"d:/rtools43/x86_64-w64-mingw32.static.posix/include"     -pedantic -Wstrict-prototypes -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c entrypoint.c -o entrypoint.o
if [ -f "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/vendor.tar.xz" ]; then \
	mkdir -p "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/vendor" && \
	d:/rtools43/usr/bin/tar --extract --xz --file "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/vendor.tar.xz" -C "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/vendor" && \
	mkdir -p "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/.cargo" && \
	cp "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/vendor-config.toml" "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/.cargo/config.toml"; \
fi
mkdir -p "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target/libgcc_mock"
# `rustc` adds `-lgcc_eh` flags to the compiler, but Rtools' GCC doesn't have
# `libgcc_eh` due to the compilation settings. So, in order to please the
# compiler, we need to add empty `libgcc_eh` to the library search paths.
#
# For more details, please refer to
# https://github.com/r-windows/rtools-packages/blob/2407b23f1e0925bbb20a4162c963600105236318/mingw-w64-gcc/PKGBUILD#L313-L316
touch "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target/libgcc_mock/libgcc_eh.a"
# CARGO_LINKER is provided in Makevars.ucrt for R >= 4.2
if [ "" != "true" ]; then \
	export CARGO_HOME="/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/.cargo"; \
	export CARGO_BUILD_JOBS=2; \
fi && \
	export CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER="x86_64-w64-mingw32.static.posix-gcc.exe" && \
	export LIBRARY_PATH="${LIBRARY_PATH};/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src//d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target/libgcc_mock" && \
	cargo build --target="x86_64-pc-windows-gnu" --lib --release --manifest-path="/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/Cargo.toml" --target-dir "/d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target"
   Compiling proc-macro2 v1.0.66
   Compiling unicode-ident v1.0.11
   Compiling cc v1.0.83
error: linking with `x86_64-w64-mingw32.static.posix-gcc.exe` failed: exit code: 1
  |
  = note: "x86_64-w64-mingw32.static.posix-gcc.exe" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsbegin.o" "D:\\temp\\rustcNUbtb4\\symbols.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.0.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.1.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.10.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.11.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.12.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.13.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.14.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.15.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.2.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.3.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.4.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.5.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.6.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.7.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.8.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.build_script_build.bd9bb062-cgu.9.rcgu.o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.4xuhmu3oxs63p4ch.rcgu.o" "-L" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\deps" "-L" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-Wl,-Bstatic" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd-4208b5b050761eab.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libpanic_unwind-c24e86db72ac50d0.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libobject-8130621e6418bc58.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libmemchr-08ae56106143c1db.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libaddr2line-ccfb784edb974a4a.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libgimli-4ab6c884d279676f.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_demangle-9ddceef41b9e9295.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd_detect-d8366e410ad5f9d7.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libhashbrown-ce69f37d51637f0b.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libminiz_oxide-19f6886d9ebc20bf.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libadler-25358c2b66d0aa3d.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_alloc-d18d5ff4d2dae958.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libunwind-0cef8242de7339bc.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcfg_if-edcd03acb0b5d165.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liblibc-21b2a42330a7f2e3.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc-ee725b426657593b.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_core-d6f1a8662e8b71a9.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcore-84ddc19bfcf64c94.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcompiler_builtins-6b61207aa476e831.rlib" "-Wl,-Bdynamic" "-lkernel32" "-ladvapi32" "-luserenv" "-lkernel32" "-lws2_32" "-lbcrypt" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-L" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-o" "D:/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target\\release\\build\\proc-macro2-a6c5247ccfea3f6e\\build_script_build-a6c5247ccfea3f6e.exe" "-Wl,--gc-sections" "-no-pie" "-nodefaultlibs" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsend.o"
  = note: D:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot find -lgcc_eh: No such file or directory
          collect2.exe: error: ld returned 1 exit status
          

error: could not compile `proc-macro2` due to previous error
warning: build failed, waiting for other jobs to finish...
make: *** [Makevars.win:26: /d/temp/RtmpoxnIQf/R.INSTALL2ec8461d0491e/prqlr/src/rust/target/x86_64-pc-windows-gnu/release/libprqlr.a] Error 101
ERROR: compilation failed for package 'prqlr'
* removing 'd:/RCompile/CRANguest/R-release/lib/prqlr'

Maybe due to 4d0bef5?

Cache Rust on CI

Like:

      - name: Cache Rust
        uses: Swatinem/rust-cache@v2
        with:
          shared-key: build
          workspaces: ./src/rust/

Rust version compatibility

Both R-hub builder and R-universe builder (ubuntu:jammy) failed installation with the following error after using prql-compiler 0.4.0.
https://github.com/r-universe/eitsupi/actions/runs/3941624440/jobs/6752975644#step:3:802

  error[E0658]: `let...else` statements are unstable
     --> /tmp/RtmpnXSnJF/R.INSTALLfa93bec8912/prqlr/src/.cargo/registry/src/github.com-1ecc6299db9ec823/prql-compiler-0.4.0/src/semantic/lowering.rs:453:13
      |
  453 | /             let pl::ExprKind::All { except, .. } = expr.kind else {
  454 | |                 // base case
  455 | |                 r.push(self.declare_as_column(expr, is_aggregation)?);
  456 | |                 continue;
  457 | |             };
      | |______________^
      |
      = note: see issue #87335 <https://github.com/rust-lang/rust/issues/87335> for more information

The `compile()` and `prql_to_pl()` will freeze if it contains a large number of errors? (from prql-compiler 0.6.0)

Execute the following to stop the session.

"from a | select {{b}} | select {{c}}" |> prqlr:::compile(target = NULL, format = FALSE, signature_comment = FALSE)

Note that prql-python and prql-js output errors normally.

>>> import prql_python as prql
>>> prql.compile("from a | select {{b}} | select {{c}}")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SyntaxError: Error:
   ╭─[:1:17]
   │
 1from a | select {{b}} | select {{c}}
   ·                 ┬
   ·                 ╰── unexpected {
───╯
Error:
   ╭─[:1:20]
   │
 1from a | select {{b}} | select {{c}}
   ·                    ┬
   ·                    ╰── unexpected }
───╯
Error:
   ╭─[:1:32]
   │
 1from a | select {{b}} | select {{c}}
   ·                                ┬
   ·                                ╰── unexpected {
───╯
Error:
   ╭─[:1:35]
   │
 1from a | select {{b}} | select {{c}}
   ·                                   ┬
   ·                                   ╰── unexpected }
───╯

...and it works on prqlr

> "from a | select {b" |> prqlr:::compile(target = NULL, format = FALSE, signature_comment = FALSE)
$ok
NULL

$err
[1] "Error:\n   ╭─[:1:17]\n\n 1 │ from a | select {b\n   ·                 ┬\n   ·                 ╰── unexpected {\n───╯\n"

attr(,"class")
[1] "rust_result"

build fails on nightly (or wasm target only?)

Just for sharing. tkaitchuck/aHash#200 is the issue and this is probably just a temporary one.

error[E0635]: unknown feature `stdsimd`
  --> /tmp/RtmpAfozKq/file16ea87038/prqlr/src/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ahash-0.8.6/src/lib.rs:99:42
   |
99 | #![cfg_attr(feature = "stdsimd", feature(stdsimd))]
   |                                          ^^^^^^^

For more information about this error, try `rustc --explain E0635`.
error: could not compile `ahash` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
make: *** [Makevars:19: /tmp/RtmpAfozKq/file16ea87038/prqlr/src/rust/target/wasm32-unknown-emscripten/release/libprqlr.a] Error 101
ERROR: compilation failed for package 'prqlr'

https://github.com/yutannihilation/savvy-webr-test/actions/runs/7860163904/job/21447049235#step:4:613

pre-built binaries shoud be built with R 4.2?

If binaries built with the new R version are used for the old R version, warnings will appear.

Like:

 Warning: Warning: .drectve `-exclude-symbols:"_ZN4core3ptr110drop_in_place$LT$std..collections..hash..map..HashMap$LT$alloc..string..String$C$alloc..string..String$GT$$GT$17he0d6d8373ff25b65E.llvm.11343610586586519693" ' unrecognized

From #211

Set Taskfile.yml

https://github.com/PRQL/prql uses Taskfile.yml for development.
It would be useful to set that up here as well.

(When update prql-compiler, we should run rextendr::document() -> devtools::load_all() -> devtools::test() -> devtools::build_readme() -> Rscript dev/generate-license-note.R...)

prql code chunks use R code autocompletion and syntax highlighting

I guess prql autocompletion would need to be implemented on the RStudio IDE side but could it be possible to disable autocompletion as if it was R code and maybe use SQL syntax highlighting?

I just noticed this only happens when you have the "Source" markdown editor enabled on the RStudio IDE

sqldf

Note that this works:

library(prqlr)
library(sqldf)
"from mtcars | filter cyl == 4" |> prql_compile() |> sqldf()

giving

    mpg cyl  disp  hp drat    wt  qsec vs am gear carb
1  22.8   4 108.0  93 3.85 2.320 18.61  1  1    4    1
2  24.4   4 146.7  62 3.69 3.190 20.00  1  0    4    2
3  22.8   4 140.8  95 3.92 3.150 22.90  1  0    4    2
4  32.4   4  78.7  66 4.08 2.200 19.47  1  1    4    1
5  30.4   4  75.7  52 4.93 1.615 18.52  1  1    4    2
6  33.9   4  71.1  65 4.22 1.835 19.90  1  1    4    1
7  21.5   4 120.1  97 3.70 2.465 20.01  1  0    3    1
8  27.3   4  79.0  66 4.08 1.935 18.90  1  1    4    1
9  26.0   4 120.3  91 4.43 2.140 16.70  0  1    5    2
10 30.4   4  95.1 113 3.77 1.513 16.90  1  1    5    2
11 21.4   4 121.0 109 4.11 2.780 18.60  1  1    4    2

extendr_api::throw_r_error could leak a bit of memory

Hello wonderful person, what a nice project! I personally never liked the raw SQL syntax variants ^^, maybe this is good bridge for me.

I noticed the use of extendr_api::throw_r_error. I guess in the case prqlr all the inputs or intermediate variabels of the function compile() could potentially be leaked. I suppose it is not large amounts of bytes as query strings are small. The linked thread have several alternative approaches, but no unified best approach for any use case.

pub fn compile(
    prql_query: &str,
    dialect: Option<String>,
    format: bool,
    signature_comment: bool,
) -> String {
    let dialect = prql_compiler::sql::Dialect::from_str(dialect.as_deref().unwrap_or_default())
        .map(From::from)
        .ok();

    let options: Option<prql_compiler::sql::Options> = Some(prql_compiler::sql::Options {
        format,
        dialect,
        signature_comment,
    });

    let result = Ok(prql_query)
        .and_then(prql_compiler::prql_to_pl)
        .and_then(prql_compiler::pl_to_rq)
        .and_then(|rq| prql_compiler::rq_to_sql(rq, options.map(prql_compiler::sql::Options::from)))
        .map_err(|e| e.composed("", prql_query, false));

    unwrap_or_throw(result)
}

fn unwrap_or_throw(result: anyhow::Result<String, prql_compiler::ErrorMessages>) -> String {
    match result {
        Ok(v) => v,
        Err(e) => {
            throw_r_error(e.to_string());
            unreachable!()
        }
    }
}

Build failuar on CRAN Windows builder

I sent 6be3f1f to CRAN but it does not pass the incoming checks automatically.

* installing *source* package 'prqlr' ...
** using staged installation

--------------------------- [RUST FOUND] ---------------------------
cargo 1.76.0 (c84b36747 2024-01-18)

rustc 1.76.0 (07dca489a 2024-02-04)
binary: rustc
commit-hash: 07dca489ac2d933c78d3c5158e3f43beefeb02ce
commit-date: 2024-02-04
host: x86_64-pc-windows-gnu
release: 1.76.0
LLVM version: 17.0.6
--------------------------------------------------------------------

** libs
using C compiler: 'gcc.exe (GCC) 12.3.0'
rm -Rf "prqlr.dll" "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target/x86_64-pc-windows-gnu/release/libprqlr.a" "init.o"
gcc  -I"D:/RCompile/recent/R/include" -DNDEBUG     -I"d:/rtools43/x86_64-w64-mingw32.static.posix/include"     -pedantic -Wstrict-prototypes -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c init.c -o init.o
if [ -f "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/../tools/libprqlr.a" ]; then \
	mkdir -p "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target/x86_64-pc-windows-gnu/release" ; \
	mv "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/../tools/libprqlr.a" "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target/x86_64-pc-windows-gnu/release/libprqlr.a" ; \
	exit 0; \
fi && \
if [ -f "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/vendor.tar.xz" ]; then \
	mkdir -p "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/vendor" && \
	d:/rtools43/usr/bin/tar --extract --xz --file "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/vendor.tar.xz" -C "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/vendor" && \
	mkdir -p "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/.cargo" && \
	cp "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/vendor-config.toml" "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/.cargo/config.toml"; \
fi && \
if [ "" != "true" ]; then \
	export CARGO_HOME="/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/.cargo"; \
	export CARGO_BUILD_JOBS=2; \
fi && \
	export CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER="x86_64-w64-mingw32.static.posix-gcc.exe" && \
	cargo build --lib --manifest-path="/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/Cargo.toml" --target-dir "/d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target" --target="x86_64-pc-windows-gnu" \
		--profile="release" --features=""
   Compiling proc-macro2 v1.0.71
   Compiling unicode-ident v1.0.12
   Compiling cc v1.0.83
error: linking with `x86_64-w64-mingw32.static.posix-gcc.exe` failed: exit code: 1
  |
  = note: "x86_64-w64-mingw32.static.posix-gcc.exe" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsbegin.o" "D:\\temp\\rustcS2JgtJ\\symbols.o" "D:/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target\\release\\build\\proc-macro2-999638963ec1b075\\build_script_build-999638963ec1b075.build_script_build.822b784807005e7d-cgu.0.rcgu.o" "D:/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target\\release\\build\\proc-macro2-999638963ec1b075\\build_script_build-999638963ec1b075.build_script_build.822b784807005e7d-cgu.1.rcgu.o" "D:/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target\\release\\build\\proc-macro2-999638963ec1b075\\build_script_build-999638963ec1b075.build_script_build.822b784807005e7d-cgu.2.rcgu.o" "D:/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target\\release\\build\\proc-macro2-999638963ec1b075\\build_script_build-999638963ec1b075.24epqwbzz8zcwbci.rcgu.o" "-L" "D:/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target\\release\\deps" "-L" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-Wl,-Bstatic" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd-efaf1f98469bce6b.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libpanic_unwind-d359dc6e62e55626.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libobject-3c268a13c68ea8e2.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libmemchr-9f58f527e2b91dae.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libaddr2line-c77bb5c52ef40ec2.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libgimli-351e9eebebf3246b.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_demangle-98eb1ba61e5b7801.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd_detect-7e82d8ed3fcb6f19.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libhashbrown-b42813017cfa0a7e.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_alloc-ebd56e19753fc0ec.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libminiz_oxide-633cedffad10403c.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libadler-65d6646cd32deaaa.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libunwind-24b4e764e2c1853b.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcfg_if-9fcf4cdda0a238d5.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liblibc-56337887c3a2ba98.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc-257f356590128be9.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_core-4a3f13ff983c2d26.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcore-76b1656df760b6c0.rlib" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcompiler_builtins-6c9e2f948258138b.rlib" "-Wl,-Bdynamic" "-lkernel32" "-ladvapi32" "-lbcrypt" "-lkernel32" "-lntdll" "-luserenv" "-lws2_32" "-lkernel32" "-lws2_32" "-lkernel32" "-lntdll" "-lkernel32" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-L" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-o" "D:/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target\\release\\build\\proc-macro2-999638963ec1b075\\build_script_build-999638963ec1b075.exe" "-Wl,--gc-sections" "-no-pie" "-nodefaultlibs" "C:\\Users\\CRAN\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsend.o"
  = note: D:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot find -lgcc_eh: No such file or directory
          collect2.exe: error: ld returned 1 exit status
          

error: could not compile `proc-macro2` (build script) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
make: *** [Makevars.win:23: /d/temp/RtmpEDufiR/R.INSTALLa42c41d937a6/prqlr/src/rust/target/x86_64-pc-windows-gnu/release/libprqlr.a] Error 101
ERROR: compilation failed for package 'prqlr'
* removing 'd:/RCompile/CRANincoming/R-devel/lib/prqlr'

It seems that the same error occurred on #184.

@yutannihilation Sorry for bothering you, but is it possible that the lines below that you removed on #252 are still needed?

prqlc-r/src/Makevars.win

Lines 23 to 30 in 97fa7ce

# `rustc` adds `-lgcc_eh` flags to the compiler, but Rtools' GCC doesn't have
# `libgcc_eh` due to the compilation settings. So, in order to please the
# compiler, we need to add empty `libgcc_eh` to the library search paths.
#
# For more details, please refer to
# https://github.com/r-windows/rtools-packages/blob/2407b23f1e0925bbb20a4162c963600105236318/mingw-w64-gcc/PKGBUILD#L313-L316
mkdir -p "$(TARGET_DIR)/libgcc_mock"
touch "$(TARGET_DIR)/libgcc_mock/libgcc_eh.a"

prqlr fails to compile on Linux arm64

prqlr complains about the Rust version that comes from the OS repositories on Debian-based distros (tried with Ubuntu 22.04 LTS and RPi OS) but even after installing the latest Rust version with rustup (which solves the problem for Linux amd64) compilation still fails on arm64 with this error message.

 --- stderr
  thread 'main' panicked at 'Cannot find libR-sys bindings file for R 4.2.2false on linux in bindings. Consider compiling with --features use-bindgen.', /tmp/RtmpjWO0lb/R.INSTALL463401756134e/prqlr/src/.cargo/registry/src/github.com-1ecc6299db9ec823/libR-sys-0.3.0/build.rs:448:13

manual entry for prqlr

First thing after installing pkg for me is checking its manual

library(prqlr)
?prqlr
#No documentation for 'prqlr' in specified packages and libraries:
#you could try '??prqlr'

IMO could work for prqlr as well

Docs about vendoring

Vendoring would not pass the CRAN check due to the 12 MB size, so I removed the vendor.tar.xz file and submitted 0.5.0 to CRAN.
This needs to be documented.

Originally posted by @eitsupi in #148 (comment)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.