Comments (28)
Jonah-Foley
commented on July 19, 2024
1
That has worked! Thank you. Clearly my environment was the problem. I shall attempt to diagnose what cause the issue and then make a follow up for future reference.
It is slightly surprising that the problem persists despite specifying -i with nix develop.
from artiq.
sbourdeauducq
commented on July 19, 2024
Unmodified artiq-7 builds fine on the CI server and should not have the issue. Can you let us know what exactly you did to cause the problem to appear?
from artiq.
Jonah-Foley
commented on July 19, 2024
My steps for recreating this issue are
mkdir test && cd test
git clone --branch release-7 https://git.m-labs.hk/M-Labs/artiq-zynq.git
cd artiq-zynq
nix develop
cd src
mkdir -p ../build
python gateware/kasli_soc.py -r ../build/pl.rs -c ../build/rustc-cfg -m ../build/mem.rs ../kasli-soc-master.json
cd runtime && XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot cargo xbuild --release --target-dir ../../build/firmware --no-default-features --features=target_kasli_soc
At which point I see
error: edition 2021 is unstable and only available with -Z unstable-options.
error: could not compile `alloc`
I then got around this by modifying artiq-zynq/src/.cargo/config to
[target.armv7-none-eabihf]
rustflags = [
"-C", "link-arg=-Tlink.x",
"-C", "target-feature=a9,armv7-a,neon",
"-C", "target-cpu=cortex-a9",
]
[build]
target = "armv7-none-eabihf.json"
[unstable]
build-std = ["alloc"]
(I couldnt get the fix in 8ca75a3 to do anything after applying it to the flake.nix of artiq-zynq)
This worked, however then at a later stage in the build you then see
rror[E0464]: multiple matching crates for `core`
|
= note: candidates:
error[E0464]: multiple matching crates for `core`
|
= note: candidates:
error: aborting due to previous error
error: aborting due to previous error
error[E0464]: multiple matching crates for `core`
|
= note: candidates:
error: aborting due to previous error
error: could not compile `vcell`
And this is currently where i'm at. I'd appreciate any advice on how to proceed.
from artiq.
sbourdeauducq
commented on July 19, 2024
git clone --branch 7.0 [email protected]:m-labs/artiq.git
I don't know where you got this command from, it is wrong and likely a source of the problem.
from artiq.
sbourdeauducq
commented on July 19, 2024
Please also check your local python environment for conflicting packages. Use the nix environment isolation features as required.
from artiq.
Jonah-Foley
commented on July 19, 2024
git clone --branch release-7 https://git.m-labs.hk/M-Labs/artiq-zynq.git
This is my mistake, you can do without this entirely and still see the problem.
Please also check your local python environment for conflicting packages. Use the nix environment isolation features as required.
Does entering into the nix environment by running nix develop not achieve this for me?
from artiq.
sbourdeauducq
commented on July 19, 2024
Try with --ignore-environment or using nix build.
from artiq.
Jonah-Foley
commented on July 19, 2024
I tried --ignore-environment and the problem seems to persist. I cannot use nix build as my end-goal is to develop and deploy a custom Phy on kasli-soc (however the problem described persists even on unmodified artiq).
Checking the hydra build logs for artiq-zynq im performing all the same build steps, however I noticed the CI tests haven't seem to be ran in ~2 weeks (2023-12-03), so perhaps you could try running them again?
from artiq.
Jonah-Foley
commented on July 19, 2024
running which cargo inside the nix develop devshell I see
/nix/store/lp3wfd7jqiyljq5q43zkglcval0dyhv5-rust-1.51.0-nightly-2021-01-28-c0b64d97b/bin/cargo
should this not be nightly-2018 if the cargo-xbuild postpatch is working as intended?
from artiq.
sbourdeauducq
commented on July 19, 2024
@fsagbuya please look into this
from artiq.
Jonah-Foley
commented on July 19, 2024
Some more info: Trying to recreate the build on hydra using:
nix build git+https://git.m-labs.hk/m-labs/artiq-zynq.git?ref=release-7&rev=4ec1ef125b14c42049c7350257b1201e7d072015#hydraJobs.kasli_soc-master-firmware
I see:
error: flake 'git+https://git.m-labs.hk/m-labs/artiq-zynq.git?ref=release-7' does not provide attribute 'packages.x86_64-linux.default' or 'defaultPackage.x86_64-linux'
from artiq.
fsagbuya
commented on July 19, 2024
I have followed the steps you gave to reproduce the issue in the latest unmodified artiq-zynq:
mkdir test && cd test
git clone --branch release-7 https://git.m-labs.hk/M-Labs/artiq-zynq.git
cd artiq-zynq
nix develop
cd src
mkdir -p ../build
python gateware/kasli_soc.py -r ../build/pl.rs -c ../build/rustc-cfg -m ../build/mem.rs ../kasli-soc-master.json
cd runtime && XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot cargo xbuild --release --target-dir ../../build/firmware --no-default-features --features=target_kasli_soc
And I haven't encounter the error mentioned.
cd runtime && XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot cargo xbuild --release --target-dir ../../build/firmware --no-default-features --features=target_kasli_soc
WARNING: There is no root package to read the cargo-xbuild config from.
Compiling libc v0.1.0 (/home/user/dev/artiq-zynq/src/libc)
Compiling libsupport_zynq v0.0.0 (https://git.m-labs.hk/M-Labs/zynq-rs.git#91bae572)
Compiling unwind v0.0.0 (/home/user/dev/artiq-zynq/src/libunwind)
Compiling libboard_artiq v0.0.0 (/home/user/dev/artiq-zynq/src/libboard_artiq)
Compiling runtime v0.1.0 (/home/user/dev/artiq-zynq/src/runtime)
Compiling io v0.0.0 (/home/user/dev/artiq-zynq/src/libio)
Compiling dwarf v0.0.0 (/home/user/dev/artiq-zynq/src/libdwarf)
Finished release [optimized + debuginfo] target(s) in 15.32s
Have you run the steps before or after the fix has been commited? Can you check the latest commit from your cloned repo by git log --oneline. Please also note that the cargo-xbuild package in artiq-zynq is tracking it from zynq-rs (as denoted here) and not from ARTIQ-7, so no need to manually apply the patch from there. It was already committed.
You can also share some additional info that you think might be helpful on debugging the issue, otherwise I suggest to rerun the commands again without modifying anything.
from artiq.
Jonah-Foley
commented on July 19, 2024
Thank you for trying, its unfortunate that you could not reproduce the problem. I have check the git log and I'm definitely on the most recent commit to release-7. However, when looking at your output, the list of what is compiled is significantly different to mine, which is:
(nix:artiq-zynq-dev-shell-env) [jfoley@localhost src]$ cd runtime && XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot cargo xbuild --release --target-dir ../../build/firmware --no-default-features --features=target_kasli_soc
WARNING: There is no root package to read the cargo-xbuild config from.
Compiling core v0.0.0 (/nix/store/lp3wfd7jqiyljq5q43zkglcval0dyhv5-rust-1.51.0-nightly-2021-01-28-c0b64d97b/lib/rustlib/src/rust/library/core)
Compiling compiler_builtins v0.1.39
Compiling rustc-std-workspace-core v1.99.0 (/nix/store/lp3wfd7jqiyljq5q43zkglcval0dyhv5-rust-1.51.0-nightly-2021-01-28-c0b64d97b/lib/rustlib/src/rust/library/rustc-std-workspace-core)
Compiling alloc v0.0.0 (/tmp/nix-shell.xsP83p/cargo-xbuildlspE6H)
error: edition 2021 is unstable and only available with -Z unstable-options.
error: could not compile `alloc`
For example mine starts with Compiling core v0.0.0, whereas yours is Compiling libc v0.1.0, along with your paths pointing to local directories, while mine point to the nix store, suggesting we are doing something different.
However looking at the CI log here it seems closer to mine than yours. Do you have any idea what would be causing these differences?
from artiq.
Jonah-Foley
commented on July 19, 2024
In particular, the failing command is
rustc --crate-name alloc --edition=2021 /nix/store/lp3wfd7jqiyljq5q43zkglcval0dyhv5-rust-1.51.0-nightly-2021-01-28-c0b64d97b/lib/rustlib/src/rust/library/alloc/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C panic=abort -C linker-plugin-lto -C codegen-units=1 -C debuginfo=2 -Z force-unstable-if-unmarked -C metadata=8e0a9e602491346a -C extra-filename=-8e0a9e602491346a --out-dir /tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps --target /home/jfoley/test/artiq-zynq/src/armv7-none-eabihf.json -L dependency=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps -L dependency=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/release/deps --extern compiler_builtins=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps/libcompiler_builtins-ac5bf2f28729e6ef.rmeta --extern core=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps/libcore-0820182884c96dd9.rmeta -Cembed-bitcode=yes
from artiq.
Jonah-Foley
commented on July 19, 2024
@sbourdeauducq @fsagbuya Please excuse my persistence, but will you be able to provide any further support on this matter?
from artiq.
fsagbuya
commented on July 19, 2024
For example mine starts with
Compiling core v0.0.0, whereas yours isCompiling libc v0.1.0, along with your paths pointing to local directories, while mine point to the nix store, suggesting we are doing something different.However looking at the CI log here it seems closer to mine than yours. Do you have any idea what would be causing these differences?
I have built it before and just rebuild it, so it only shows the tail part or the build log. But the full log in mine is similar with yours and the CI.
In particular, the failing command is
rustc --crate-name alloc --edition=2021 /nix/store/lp3wfd7jqiyljq5q43zkglcval0dyhv5-rust-1.51.0-nightly-2021-01-28-c0b64d97b/lib/rustlib/src/rust/library/alloc/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C panic=abort -C linker-plugin-lto -C codegen-units=1 -C debuginfo=2 -Z force-unstable-if-unmarked -C metadata=8e0a9e602491346a -C extra-filename=-8e0a9e602491346a --out-dir /tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps --target /home/jfoley/test/artiq-zynq/src/armv7-none-eabihf.json -L dependency=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps -L dependency=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/release/deps --extern compiler_builtins=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps/libcompiler_builtins-ac5bf2f28729e6ef.rmeta --extern core=/tmp/nix-shell.xsP83p/cargo-xbuildKmbhpN/target/armv7-none-eabihf/release/deps/libcore-0820182884c96dd9.rmeta -Cembed-bitcode=yes
From this failing command it shows that you are building from the unpatched version by the flag --edition=2021 which is causing the problem.
I'm suspecting this might be a nix issue. Can you switch to another directory and clone again artiq-zynq release-7 and run your build commands? Or if you have a VM, might as well run a test there.
from artiq.
sbourdeauducq
commented on July 19, 2024
What's the output of "which rustc" and "which cargo"? Did you use rustup?
from artiq.
fsagbuya
commented on July 19, 2024
Additionally, you can also check if the you are using the patched cargo-xbuild by nix show-derivation.
- Run nix develop and run
which cargo-xbuild. - Copy the nix store path.
- Run
nix show-derivation <cargo-xbuild-nix-store-path> - Look for the attribute
postPatch.
"pname": "cargo-xbuild",
"postPatch": "substituteInPlace src/sysroot.rs --replace 2021 2018",
"postUnpack": "eval \"$cargoDepsHook\"\n\nexport RUST_LOG=\n",
If you don't see postPatch, the cargo-xbuild is still in the original unpatched version.
from artiq.
Jonah-Foley
commented on July 19, 2024
What's the output of "which rustc" and "which cargo"? Did you use rustup?
/nix/store/lp3wfd7jqiyljq5q43zkglcval0dyhv5-rust-1.51.0-nightly-2021-01-28-c0b64d97b/bin/rustc
/nix/store/lp3wfd7jqiyljq5q43zkglcval0dyhv5-rust-1.51.0-nightly-2021-01-28-c0b64d97b/bin/cargo
Given the postPatch only applies to the alloc rust lib (looking at https://github.com/rust-osdev/cargo-xbuild/blob/master/src/sysroot.rs) then I dont believe the above being version 2021 to be wrong? I may be wrong however and it may be the presence of these 2021 rust instanlls overriding the postPatch when cargo-xbuild is invoked.
If you don't see
postPatch, the cargo-xbuild is still in the original unpatched version.
Upon running the commands as stated I do see the postPatch field,
"env": {
"PKG_CONFIG_ALLOW_CROSS": "0",
"buildInputs": "",
"builder": "/nix/store/5n8r5fgscx6f5ifmqqpd8gxjf44a0y2l-bash-5.1-p16/bin/bash",
"cargoBuildFeatures": "",
"cargoBuildNoDefaultFeatures": "",
"cargoBuildType": "release",
"cargoCheckFeatures": "",
"cargoCheckNoDefaultFeatures": "",
"cargoCheckType": "release",
"cargoDeps": "/nix/store/2f5marzkwr46hcfn6rknmy84g6kbpmpm-cargo-xbuild-0.6.5-vendor.tar.gz",
"cargoSha256": "sha256-qMPJC61ZVW9olMgNnGrvcQ/je4se4J5gOVoaOpNMUo8=",
"configureFlags": "",
"configurePhase": "runHook preConfigure\nrunHook postConfigure\n",
"depsBuildBuild": "",
"depsBuildBuildPropagated": "",
"depsBuildTarget": "",
"depsBuildTargetPropagated": "",
"depsHostHost": "",
"depsHostHostPropagated": "",
"depsTargetTarget": "",
"depsTargetTargetPropagated": "",
"doCheck": "1",
"doInstallCheck": "",
"name": "cargo-xbuild-0.6.5",
"nativeBuildInputs": "/nix/store/3gpchxhmm0r9f704hh394508ypxw6rv7-nss-cacert-3.86 /nix/store/5qlm7fwbaa46b2nm9bcrh8kbgn93k20g-git-minimal-2.36.2 /nix/store/4z6bvjiba1zp1z9agyndzhhwag6xiihg-cargo-build-hook.sh /nix/store/f8ydp07n6sc4is6cylfjjyvhffa00fha-cargo-check-hook.sh /nix/store/71x327ykrnmzz5qcd6gmvyziwwr19y6k-cargo-install-hook.sh /nix/store/9rg8n1xlrgi6ikvhdz5pz653q8kvp6m7-cargo-setup-hook.sh /nix/store/hqzbm4d3dcmjhidv506vi73h958b7ldi-rustc-1.60.0",
"out": "/nix/store/93c28y91k0ng47s09dk7galwypnaxhij-cargo-xbuild-0.6.5",
"outputs": "out",
"patchRegistryDeps": "/nix/store/nk6b2ckznjic5wj8ddw0wgdrn4mbz3lg-patch-registry-deps",
"patches": "",
"pname": "cargo-xbuild",
"postPatch": "substituteInPlace src/sysroot.rs --replace 2021 2018",
"postUnpack": "eval \"$cargoDepsHook\"\n\nexport RUST_LOG=\n",
"propagatedBuildInputs": "",
"propagatedNativeBuildInputs": "",
"src": "/nix/store/1xjj7cbxs5yvil8ac5pajdg3cxfqcnia-source",
"stdenv": "/nix/store/fq0pag250md12fzzfvd1mf6m5g6gslk9-stdenv-linux",
"strictDeps": "1",
"system": "x86_64-linux",
"version": "0.6.5"
},
which suggests the patch is being recognised but not applied.
I tried cloning into a fresh directory and performing the build, to which I saw the same results.
from artiq.
sbourdeauducq
commented on July 19, 2024
Maybe you have some problematic rust user settings? Try a new user account with an empty HOME.
Or "nix build"...
from artiq.
sbourdeauducq
commented on July 19, 2024
That doesn't block access to your home folder. But nix build does.
from artiq.
Jonah-Foley
commented on July 19, 2024
The problem was having a .cargo folder in my home directory which evidently overwrote some of the settings set by nix. It seems a common use case that someone would have rust installed (with a .cargo folder in ~) and also want to do development on Artiq via a nix develop environment. Is there a good way to ensure these don't clash as I have just experienced?
from artiq.
sbourdeauducq
commented on July 19, 2024
Take it up with the Rust developers. They are the one constantly breaking compatibility which means that you need to have very specific versions installed, and the ones who came up with this silly .cargo folder.
from artiq.
sbourdeauducq
commented on July 19, 2024
Otherwise nix build with the full sandbox is not affected, as I pointed out several times.
from artiq.
sbourdeauducq
commented on July 19, 2024
Also try the CARGO_HOME environment variable.
from artiq.
Jonah-Foley
commented on July 19, 2024
Otherwise nix build with the full sandbox is not affected, as I pointed out several times.
I appreciate nix build is fully sandboxed. If the recommended advice is to use this, what is the reason to use nix develop? Can I do development of artiq within a nix build environment?
from artiq.
sbourdeauducq
commented on July 19, 2024
Mostly, nix develop allows incremental builds as well as partial builds (eg firmware only) and gives you more manual control.
from artiq.
Jonah-Foley
commented on July 19, 2024
I see, it's an unfortunate trade off to have to make. Thank you for the insight.
from artiq.
Related Issues (20)
- mirny/ADF5356 driver doesn't have 52-bits precision HOT 1
- mirny: Unstable initiliazation with MUXOUT not low/too high errors
- Impossibility of using spare Urukul alongside with SUServo HOT 1
- LLVM Linker (ld.lld) Invocation Fails on Compilation with MSYS2 HOT 31
- Python 3.12 compatibility HOT 1
- dashboard dataset editor conversion to float64 HOT 2
- artiq_master command line reference not built
- artiq_corelog: command not found HOT 1
- Grabber input_mu function ROI limits HOT 1
- Override DAC values and DDS attenuation in dashboard applet
- Allow additional fields and cards in JSON descriptions HOT 4
- handle cancelled interactive argument requests properly
- expand interactive argument example
- interactive args: unclear error message when supplying args simultaneously
- issues with moving existing item in `DictSyncModel`
- What happens in the gatware when there is a delay/jitter on the reference clock provided to the KC705? HOT 3
- dashboard/master: Calling setattr_argument() for the same key but different type raises a TypeError on the master HOT 4
- Conda installation on windows couldn't find artiq HOT 3
- Fastino garbage output with long sequences HOT 4
- DRTIO aux packets do not use the whole space HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from artiq.