It is useful find a link to PR just by looking at the commit log. The PR may have review and other details that helps establish the context of the patch.

For example arcanist would add Differential Revision to each commit message:
apple/llvm-project@332b8cf This helps related a PR to patch and vice versa.

The move-native crate contains mostly tricky unsafe code. It has a test suite that has decent but probably not complete coverage. Run these tests under miri, the rust interpreter, and fix any issues it finds.

Add analysis passes for basic optimizations like reaching definitions:

we have a pre-commit hook for cargo fmt. Add a setup script that devs can run to setup that hook.

To support various llvm backends. Support eBPF and X86 triples at least.

Since it is not built currently, cargo test fails for the tests requiring move-ir-compiler.

I thought there was an issue for this but I'm not finding it.

I spent some time on this today, trying to adapt the upstream GitHub actions. I didn't quite succeed though, mostly because without a custom build of LLVM there's not much we can test, and it's not obvious to me how to do that build efficiently on CI.

Without the LLVM build we can still:

• run cargo x lint
• run cargo x fmt
• run the tests for move-native

We can't run cargo x clippy or even run cargo check on move-mv-llvm-compiler

• Parse function decl
• Parse abilities
• Parse generics

vector is a built-in type whose layout is known to both the compiler and runtime and whose operations are implemented by the runtime, and which requires emitting type descriptors. Implementing it will unlock a bunch of stuff.

First declare all the structs, per #25

Ops likely to be involved here include: Load, BorrowField, GetField, Destroy.

Don't know yet how to destroy structs that need destructors - that will probably need to wait until we have vectors.

8fa1969

🐛 Bug

With bc7c05b cargo build failed with the following steps.

To reproduce

$ mkdir -p $HOME/sbf-tools/releases/v1.32 && cd  $HOME/sbf-tools/releases/v1.32 
$ wget https://github.com/solana-labs/sbf-tools/releases/download/v1.32/solana-bpf-tools-linux.tar.bz2
$ tar -xf  solana-bpf-tools-linux.tar.bz2
$ cd $HOME/move/language/tools/move-mv-llvm-compiler
$ export SBF_TOOLS_ROOT=$HOME/sbf-tools/releases/v1.32
$ clang -v
clang version 15.0.6 (Fedora 15.0.6-2.fc37)
$ rustc --version
rustc 1.65.0 (897e37553 2022-11-02)

$ cargo build

 = note: ld.lld: error: undefined symbol: LLVMInitializeSBFTargetInfo
          >>> referenced by llvm.rs:28 (language/tools/move-mv-llvm-compiler/src/stackless/llvm.rs:28)
          >>>               move_mv_llvm_compiler-9f44bf544af0c8f6.3mxs1fhp8ozmebsf.rcgu.o:(move_mv_llvm_compiler::stackless::llvm::initialize_sbf::h9ac631b3ed0e3818) in archive /run/media/tb_half/move/target/debug/deps/libmove_mv_llvm_compiler-9f44bf544af0c8f6.rlib
...

to repro:

$ cargo test -p move-mv-llvm-compiler --test rbpf-tests

error[E0412]: cannot find type `Option` in this scope
  --> /home/adityak/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.152/src/std_error.rs:45:25
   |
45 |     fn source(&self) -> Option<&(Error + 'static)> {
   |                         ^^^^^^ not found in this scope
   |
help: there is an enum variant `de::Unexpected::Option`; try using the variant's enum
   |
45 |     fn source(&self) -> de::Unexpected {
   |                         ~~~~~~~~~~~~~~


error[E0425]: cannot find value `None` in this scope
  --> /home/adityak/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.152/src/std_error.rs:46:9
   |
46 |         None
   |         ^^^^ not found in this scope

Some errors have detailed explanations: E0405, E0412, E0425, E0432, E0433, E0463, E0531.
error: could not compile `serde` due to 4284 previous errors
thread 'run_test::assert.move' panicked at 'running SBF cargo failed', language/tools/move-mv-llvm-compiler/tests/rbpf-tests.rs:135:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'thread 'run_test::abort.moverun_test::multi-module.move' panicked at '' panicked at 'Once instance has previously been poisonedOnce instance has previously been poisoned', ', language/tools/move-mv-llvm-compiler/tests/rbpf-tests.rslanguage/tools/move-mv-llvm-compiler/tests/rbpf-tests.rs::110110::1111

thread 'run_test::script.move' panicked at 'Once instance has previously been poisoned', language/tools/move-mv-llvm-compiler/tests/rbpf-tests.rs:test run_test::assert.move ... FAILED110
:11
test run_test::abort.move ... FAILED
test run_test::multi-module.move ... FAILED
test run_test::script.move ... FAILED

failures:
    run_test::assert.move
    run_test::abort.move
    run_test::multi-module.move
    run_test::script.move

test result: FAILED. 0 passed; 4 failed; 0 filtered out

Move constants probably end up as Load instructions, which are already partially implemented, so implementing this may just be a matter of writing tests and filling in missing Load implementations.

move-ir-tests are probably sufficient, not rbpf-tests since we probably can't make useful assertions about it.

Add initial plumbing to support mvir verification when mvir-llvm-compiler is supplied an .mvir file

Move allows functions to return multiple values (but does not have first-class tuples). The code today has todo!s in all the multiple-return branches. It should be a relatively straightforward thing to fill in these branches.

I'm not sure about the ABI but it can probably be anything that works. If LLVM functions can return structs then just define these functions to return an anonymous struct, then the caller can copy the fields into the correct locals; let LLVM decide how structs are returned.

The move model contains a list of struct definitions for every module. When translating the module, before translating functions, we should translate struct definitions and cache them in the module context.

Switch to the sbf-solana-solana target.

Per comments here #24 (comment) the solana tooling has already switched the the sbf target, so our code should use that instead.

I think this will just involve changing target triples to sbf-solana-solana and using the sbf tools instead of the bpf tools, changing the BPF_TOOLS_ROOT variable to SBF_TOOLS_ROOT.

There are some llvm features, I think +sbfv1 and +sbfv2, that I don't understand yet. Hopefully though the sbf target will turn on sbfv1 and sbfv2 is opt-in.

In file: language/tools/move-mv-llvm-compiler/tests/move-ir-tests/eq.move

In the following test summary, one test was ignored but in the summary report of test result, it shows '0' filtered out tests.

running 6 tests
ignoring tests/ir-tests/small.mvir
test run_test::small.mvir ... ok
test run_test::struct-pair.mvir ... ok
test run_test::friend.mvir ... ok
test run_test::empty-module.mvir ... ok
test run_test::struct.mvir ... ok
test run_test::struct_arguments.mvir ... ok

test result: ok. 6 passed; 0 failed; 0 filtered out

Add runtime calls to builtins Resources: A Safe Language Abstraction for Money, page# 7 if there is no direct mapping to Solana Bytecode (eBPF). To start with, we can have each of the following as part of runtime library and make optimizations as needed.

- [x] local variable instructions: MvLoc ⟨x⟩ | CpLoc ⟨c⟩ | StLoc ⟨x⟩ | BorrowLoc ⟨x⟩
- [x] reference instructions: ReadRef | WriteRef | FreezeRef
- [x] record instructions: Pack | Unpack | BorrowField ⟨f ⟩
- [ ] global state instructions: MoveTo ⟨s⟩ | MoveFrom ⟨s⟩ | BorrowGlobal ⟨s⟩ | Exists ⟨s⟩
- [x] stack instructions: Pop | LoadConst ⟨a⟩ | Op
- [x] procedure instructions: Call ⟨h⟩ | Ret

Driver program for move-bytecode frontend. move-mv-llvm-compiler takes move-bytecode as input and targets llvm supported back ends.

/target/debug/move-mv-llvm-compiler -b function.mv -o a

Create a standalone driver program to take .mv. Add the following options to start with.

• -v for verbose
• --version for version
• -S to emit assembly
• -a to target specific architecture
• -b path to bytecode module/script
• -triple See: https://github.com/llvm/llvm-project/blob/main/clang/test/AST/ast-dump-aarch64-sve-types.c#L7

• Scaffolding to use stackless bytecode as starting point for lowering to llvm-ir
  • Interface to llvm context
  • Abstraction for llvm module
  • Lower function declaration and return type
• Write llvm-ir to file
• Documentation of command line interface
• Test harness

To test in rbpf, move programs need to call a few runtime functions, particularly abort and logging functions. The rbpf test harness should compile the move-native crate (in my move-native branch) to a C static library, and include it in the link step.

Ensure that calls to move_rt_abort work, and extend the test harness to understand expected aborts.

Many runtime calls require type descriptors. The std::debug::print runtime call is an easy way to test them. Probably easiest to do this prior to trying to implement vector in #53

🐛 Bug

We have .move, .mvir, and .mv files. The test harness should be able to invoke the right tools for each of these and invoke the compiler with the .mv files.

To reproduce

Code snippet to reproduce

# Your code goes here
# Please make sure it does not require any external dependencies

Stack trace/error message

// Paste the output here

Expected Behavior

System information

Please complete the following information:

Additional context

Add any other context about the problem here.

• CastU8
• CastU16
• CastU32
• CastU64
• CastU128
• Not

A single .move file may contain multiple modules, and to run rbpf tests that actually call functions will require multiple modules - the rbpf test harness runs scripts, and scripts are modules with only a single function, so calling another function will require a move file with both a script and a module.

The move model allows multiple bytecode files to be loaded at once, but they have to be topologically sorted. The test harness calls move-build to generate move modules, and it'll have get the dependency list from it somehow and pass it to move-mv-llvm-compiler.

Figuring out how to do this is a big indicator of whether we can manage to do the compilation based only on bytecode without immediate access to the source.

The build directory can accumulate stale files that confuse the test runner. Before rebuilding delete all .mv files from the build directory.

Pretty much everything in this chapter https://move-language.github.io/move/references.html

Write some test cases and see what bytecode needs to be implemented. It should be relatively straightforward. Also do reference arguments and return types.

Address is a primitive type that can be copied around and passed as arguments. Implement that much.

Move supports addresses of multiple sizes depending on configuration, so make sure to take that into account. The address size is also dependent on the chain, so make sure to make it conditional on the target. Solana addresses are 32 bytes. Address has a layout known to the runtime - u8 array of the target-specific length, defined in move_native::rt_types::Address.

Don't try to figure out global storage yet, and the exists, borrow_global, borrow_global_mut, move_from, move_to ops. Unclear what we're going to do with that yet.

• Load move programs compiled to bpf
• Run the bpf program in rbpf

Known failing tests will be added to help contributors work on specific tasks. Once that task is done the failing test will be enabled. In the mean time developers should be able to run those with a flag that is passed to cargo test

Only the most basic scalar ops are implemented. It should be straightforward to fill in all of them.

Run this before committing

cargo fmt --check 

Developers should run cargo fmt before committing.

Test harness that takes move source code as input and verifies the generated llvm-ir output.

Probably we want to monomorphise generics.

It would be nice to do this in two passes: before translating functions discover all the instantiations of generic functions, do type substitutions, emit their declarations and cache them in a way the callers can retrieve; then translate the type-substituted functions along with all the non-generic functions.

If it's not possible to discover all the instances ahead of time, then we can emit and cache them at the same time as emitting their callers.

Just get the most basic thing working as a proof of concept. I don't know the best patterns for monomorphization so it may help to study the rust compiler for a while to see how they are doing it now.

Ref: https://github.com/move-language/move/blob/main/language/move-prover/bytecode/src/stackless_bytecode.rs#L112

• arithmetic
  • Add
  • Sub
  • Mul
  • Div
  • Mod
  • CastU256
• boolean
  • Lt
  • Gt
  • Le
  • Ge
  • Or
  • And
  • Eq
  • Neq
• bitwise
  • BitOr
  • BitAnd
  • Xor
  • Shl
  • Shr

Currently, build and test are independent of each other. So one has to run the following for testing.

cargo build -p move-ir-compiler
cargo test --test ir-tests

This isn't ideal and may result issues later. We should make cargo test dependent on cargo build such that before testing, the required targets are build.

Top level tasks to parse global decls (structs, functions etc)

From move_model::ty::PrimitiveType
https://github.com/move-language/move/blob/main/language/move-model/src/ty.rs#L49

• Bool
• U8
• U32
• U64
• U128
• U256
• Address
• Signer

Per #34 the +solana feature is unnecessary.

Need to know before lowering these.

https://llvm.org/docs/LangRef.html#lshr-instruction

Running cargo test in move/language/tools/move-mv-llvm-compiler should do the following
For each .mvir file in move/language/tools/move-mv-llvm-compiler/tests

• Generate move bytecode from move-ir (move-ir-compiler -m a.mvir)
• Generate bitcode file from move ($MOVE_HOME/target/debug/move-mv-llvm-compiler -b a.mv)
• Add assertions as applicable (e.g., assert(FunctionName == "my_fun")
• Print cargo test stats

Documentation of BTF debug info in llvm
https://reviews.llvm.org/D53736

For BPF targets, BTF maybe getting generated automatically for types known to LLVM but we may want additional type info for move types.

BTF Types: https://docs.kernel.org/bpf/btf.html

https://github.com/TheDan64/inkwell

At the moment function declarations are translated immediately prior to translating the function bodies. Instead, so that functions can call other functions, all function declarations should be translated first, their definitions cached in the module context; then the function bodies translated.