Git Product home page Git Product logo

ipfs-execute's Introduction

IPFS execute!

ipfs-execute is a simple script that combines sandboxing provided by Bubblewrap with data storage provided by IPFS. The tool is proof of concept for "pure computations in IPFS".

ipfs-execute fetches description of what to do from IPFS, executes computation in sandbox and then pushes result back to IPFS.

More in depth description

ipfs-execute takes following arguments in order:

  • IPFS path of filesystem snapshot (.tar.gz file)
  • IPFS path of argument - it will be mounted at /input inside container.
  • arbitrary number of arguments which will be passed to /sbin/init

ipfs-execute unpacks root fs, mounts input, mounts empty directory at /output and another one at /tmp. After that container is executed. Entry point is /sbin/init.

When container finishes whole /output is added to IPFS. Final hash is written to stdout.

/ and /input is mounted read-only. All persistent efects must be located in /output tree. This potentially allows to reuse unpacked rootfs or input.

Containers don't have access to internet. Output should be deterministicaly computed from rootfs and input.

Dependencies

  • IPFS (daemonized)
  • Bubblewrap 2.0
  • ipfs-api-mount
  • sh + tar + gzip + some other obvious things
  • GNU Make (only if you want to build example images or use install script)
  • ipfs-shell-utlis (only if you want to build example images)

Try it

[jan@aaa ipfs-execute]$ ipfs-execute $(cat examples/busybox) $(cat examples/alpine_repository) sh
/ # ls /input
aarch64  armhf    x86_64
/ # echo blabla > /output/test
/ # exit
QmeugpFaBp7aafV2uN7GC3E2GbAycnEU3c2Q3FS7mSLbo4

What happened? Image located at path specified in examples/busybox got executed with IPFS path specified in examples/alpine_repository mounted at /input. When container finished /output was pushed to IPFS. Let's inspect:

[jan@kukla:~/ipfs-execute]$ ipfs cat QmeugpFaBp7aafV2uN7GC3E2GbAycnEU3c2Q3FS7mSLbo4/test
blabla

Next lets install some packages!

For this we need prepared directory with repository and base rootfs. I'll use ipfs-mkdir from ipfs-shell-utils, but you can do it different way. ipfs-mkdir usage is as follows:

ipfs-mkdir repository $(cat examples/alpine_repository) rootfs.tar.gz $(cat examples/busybox) > /tmp/busybox-with-repo
cat /tmp/busybox-with-repo
# QmVLwnKjWMrxXdRdqwvR5h94uUdT2vqyWCypSWv2TDno98

We are ready to call ipfs-execute.

ipfs-execute $(cat examples/apk) $(cat /tmp/busybox-with-repo) file > /tmp/alpine-file
ipfs ls $(cat /tmp/alpine-file)
# QmcNqqRByj6kbw11sbjxXrBASECh2JRg63W6iw3FGezJkU 1280118 rootfs.tar.gz
ipfs-execute $(cat /tmp/alpine-file)/rootfs.tar.gz $(cat examples/empty) sh

and inside spawned container:

file /bin/busybox.static
# /bin/busybox.static: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped

Example images

Hashes and build scripts of example containers are listed in examples/.

  • busybox - just an unpacked busybox-static alpine package. Init is set to call busybox with supplied args.
  • apk - alpine linux apk package with init pointing to script that installs stuff. This container expects directory with rootfs.tar.gz and repository (dir with alpine repo structure) as argument.

In order to build images you need to have scripts in your path. make install PREFIX=~/.local/bin will copy them to ~/.local/bin.

Makefile in examples/ dir is just for building images. Do make -C examples clean to delete prebuilt images and then call make -C examples.

TODOs

  • security considerations if mounting /dev and /proc is safe.
  • RAM, disk, CPU time limits
  • build consistency - don't depend on current time etc

ipfs-execute's People

Contributors

suprasummus avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

dulumao

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.