Git Product home page Git Product logo

docker-to-linux's Introduction

docker-to-linux - make bootable Linux disk image abusing Docker

! Support development of this project > patreon.com/iximiuz

There is no real goal behind this project. Just out of my curiosity what if:

  • launch a base Linux container (debian, alpine, etc)
  • pull in Linux kernel & init system (systemd, OpenRC, etc)
  • dump container's filesystem to a disk image
  • install bootloader (syslinux) to this image...

Then it should be probably possible to launch a real virtual machine with such an image!

Try it out:

# 1. Build the image.
#    Depending on your setup, you may need to preceed `make` with `sudo`.
make debian  # or ubuntu, or alpine

# 2. Run it! Use username `root` and password `root` to log in.
qemu-system-x86_64 -drive file=debian.img,index=0,media=disk,format=raw -m 4096
# 2. Alternate
qemu-system-x86_64 -hda debian.qcow2 -m 512

# 3. Clean up when you are done.
make clean

It works!

You can increase or decrease the size of the VM hard disk via the environment variable VM_DISK_SIZE_MB (1024 by default).
For example for a 3GB disk:

make debian VM_DISK_SIZE_MB=3072  # or ubuntu, or alpine

Check out Makefile for more details or read my article on iximiuz.com.

Features

  • Real quick build a bootable Linux image with a single command!
  • 3 target distributives: Ubuntu 20.04, Debian Bullseye, Alpine 3.13.5
  • Build from macOS (including M1 chips) or Linux hosts

FAQ

  • Q: I'm getting an error about "read-only filesystem". How can I make it writable?

    A: It's Linux default behaviour to mount the / filesystem as read-only. You can always remount it with mount -o remount,rw /.

  • Q: How can I access network from the VM / How can I SSH into the VM?

    A: Networking is not configured at the moment. If you want to configure it yourself, search for TUN/TAP/bridge devices. Don't forget to open a PR if you come up with a working solution.

Release notes

2021-05-24

  • Start using ext4 instead of ext3.

2021-05-07

  • Fix - Ubuntu 20.04 stopped working because of the changed path to vmlinuz and initrd files.

2021-05-02

  • Fix macOS support #10 (thanks to @xavigonzalvo for reporting and suggesting the fix)
    • move losetup call from Makefile to the builder container
    • explicitly select amd64 architecture in target distr Dockerfiles to support builds on ARM hosts (aka M1)
  • Upgrade target distr versions
    • Ubuntu 18.04 -> 20.04
    • Debian Stretch -> Bullseye
    • Alpine 3.9.4 -> 3.13.5

2020-02-29

  • Improve Alpine support #7 (creds @monperrus)

2019-08-02

  • Fix loopback device lookup #3 (creds @christau)

2019-06-03

  • Initial release

TODO

  • add basic networking support
  • support different image formats (e.g. VirtualBox VDI)
  • support different target architectures (e.g. ARM)

Similar projects

...or tangential projects relying on the similar technique:

docker-to-linux's People

Contributors

henritel avatar iximiuz avatar monperrus avatar pagousmile avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-to-linux's Issues

user/pass

any idea what is the default user/password in ubuntu?

This is better (more stable) than Vagrant

Thanks for your code and great docs!

I'm using this to create some forensics practice disk images.

I found this repo early in my endeavor, however, I did not think it was the best choice to start with, surely Vagrant would be a better choice? (Perhaps for mainstream distros, but not for small distros) Surely installing something like tinycore from an ISO directly with vboxmanage would be the ticket? (It worked, but wasn't easily reproducible, and I didn't want to mess with my home network to connect the vm to the Internet). Well, maybe docker-to-linux will work?

And it did! and the image is even smaller than tinycore disk images *shuffles nervously*, but an image for forensic practice doesn't have to be perfect fidelity, and one can boot qemu off this image anyway, so it's pretty darn good!

Anyway, I thought you might be curious and pleased to know how I've been using this "for fun" project.

Support for arm64 Debian docker containers to bootable linux image

I know you mentioned this is in your TODO but I can't seem to get it to work on my end. I tried replacing the Debian AMD image with the equivalent arm64, but when It comes time to make the boot loader with extlinux, it fails because it's apparently not available. Do you have any advice on how to implement this? I went down a u boot hole but didn't get that working either. x86_64 works great tho

Trying on real HD

Hi again Ivan,

Have you try putting your raw image onto real HD?

extlinux --install /os/mnt/boot/

This line is the reason that I'm asking as the extlinux was the boot loader that I've been using for years, until it gave me the trouble when me duplicating system from one machine to another --

A I have my specific ways of doing things, I usually do a brand new installation in one machine, and fully customize it to my comfort, then rsync it to another machine. The problem is, the extlinux boots fine from the first machine but just refuse to boot on the second machine no matter what I tried. I even tried to exclude all its relevant files when doing rsync it to another machine, then do a brand new extlinux installation there. But my new extlinux still refuse to boot.

I really hope that you've tried it and find it not a problem. thx.

Practical use: versionable docker based linux distribution

I want to make an OS that doesn't require any maintenance, because I believe system maintenance sucks.

I'm currently on XUbuntu 16.04 and I'm about to switch distribution. I have to wipe out all of my system data (yes, after a backup) because there's really old building-from-source clones, executables and whatnots on my drive, and I don't know where to begin the cleanup. Wouldn't it be great if I never had to cleanup or upgrade my distribution in the first place?

Maybe I'm daydreaming, but in order to achieve that, I've been thinking of using docker as a "package manager" for applications. Custom Dockerfiles and application data would be stored somewhere in the file system tree, where application binaries would only reside within docker images and containers. This idea makes me think about exo kernels, although exo kernels are lower-level and can technically support Windows and Linux applications concurrently without a virtual layer.

Now, about the "versionable" part in the title: backups can come in quite handy sometimes, and I like git for it's ability to roll back any change and keep an history of what happened in a repository. Wouldn't it be nice if I could version my system configuration and simply create a new bootable image in one line when my computer dies? To achieve this, I would store Dockerfile / docker-compose.yml files in a repository, along with their configuration files. Once I succeed to install an application in a docker image, I commit the image configuration to a new folder in my repository. If I ever have to run the app somewhere else, it's simple: just clone, build and run; perhaps even just pull from Docker Hub. If I ever want to completely remove an application from my disk, I can also simply do that with docker rmi.

Another advantage of this idea is that distribution specific packages could be installed without having to bring potentially incompatible configuration on my system. For example, it would be possible to use any pacman package along with any apt-get package, effortlessly.

However, how can I be sure that my system configuration never gets out of hand? For me, this is where docker-to-linux comes into play. It allows me to store the fundamental OS configuration as a dockerfile. It gives me full control over what's in my system, regardless of the depth of my knowledge about Unix-like operating systems, basic tools, bootloaders, partitions, initramfs, bios, assembly, etc.

Yes, I might encounter issues along this journey. For instance, docker images usually take a lot of space. There might not be a lot of images on Docker Hub that fit exactly my use case, forcing me to write a big bunch of custom docker files (and this looks like maintenance to me). Maybe something exists that already solves the issues mentioned above, and I just don't know about it. So I can't tell whether the advantages of this idea over traditional system maintenance would be worth it.

Make filesystem writable before booting

I've been trying to install docker in the alpine dockerfile, on my local copy of the repository:

FROM alpine:latest
RUN apk add --no-cache linux-virt openrc docker
RUN rc-update add docker boot
RUN echo "root:root" | chpasswd

The problem I have currently is that, when the VM boots, the docker service cannot start. It seems to be unable to create mandatory files:

1623574205

I know that I can make the filesystem writeable and successfully start the service with this script:

mount -o remount,rw /
service docker start

However, I simply cannot afford to be forced to run this every time I want to use the image! Simply rebooting makes the filesystem read-only again...

Is there a way to tell qemu to mount the file system as rw, or update the image configuration from the builder container to automate this process?

aarch64 support for M1 Mac Silicon

It works fine on a Mac M1 but since it doesn't run in native mode, without acceleration, the result is a bit slow.

I try to get a working solution starting from your work with

Has anyone tried with an arm version or has the information to build a bootable linux system for Mac M1?

make alpine: Fails with mount: /dev/loop0 is write-protected, mounting read-only

$ make alpine

Fails with message like:


[Format partition with ext3]
mke2fs 1.43.4 (31-Jan-2017)
Discarding device blocks: done                            
Creating filesystem with 261888 4k blocks and 65536 inodes
Filesystem UUID: 603c1544-a156-4188-bcb4-aca0eccf23cb
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done


[Copy linux directory structure to partition]
mount: /dev/loop0 is write-protected, mounting read-only
mount: cannot mount /dev/loop0 read-only
Makefile:51: recipe for target 'linux.img' failed
make[1]: *** [linux.img] Error 32
make[1]: Leaving directory '/root/docker-to-linux'
Makefile:38: recipe for target 'alpine.img' failed
make: *** [alpine.img] Error 2

Any thoughts on this?

disk size issue

Hi!,
I've been having trouble with these steps using a bigger disk size. Just changing count to 2 in sudo dd if=/dev/zero of=linux.img bs=${IMG_SIZE} count=2 break the image at boot time.

My custom debian image is a little bigger, like 2.5gb , but i've checked using the same debian:stretch used in this examples and when i try to create a bigger disk size, it breaks. (it works flawlessly with sudo dd if=/dev/zero of=linux.img bs=${IMG_SIZE} count=1

I've would like to know more details about the sizes in the bootable part of the disk so i can change virtual disk size at will.

Practical use

Hi Ivan,

I don't use tweeter, so let me put my practical use-case here.

I've been looking for such docker-to-linux solution for years, and I'll search for it every now and there, but only yesterday did I find your project in google. I think your project, after being existing for two years, is taking mainstream and is becoming top hits from google.

Congratulations!

If you’re aware of the real use of the docker-to-linux project, please drop me a message and share your experience.

For me, I'll be using it to automate my Debian system installation. It's only for my personal use, which varies dramatically from the massive system installation that enterprises use (like Vagrant etc), as I have my specific ways of doing things, like alias that I'be been using for decades. Not being able to get into my accustomed environment in a brand new machine is the "dark age" to me, and I need to get over that period as short as possible.

However, there aren't many ready-made solutions that has both the power and versatility that allows me to do that, as far as real HD is concerned. I.e,

  • For VM, I use Debian Preseed installation
  • For Docker, I use packer

But for real HD installation, when it is only a bare-metal that I have to install everything from scratch, that's truly a dark age to me, that's why I've been looking for such docker-to-linux solution for years.

Thanks!

Error with extlinux

Observing the following issue when building the container:

#6 0.611 E: Package 'extlinux' has no installation candidate

rm dist.dir isn't working

This is so weird, at the very end of the build (make ubuntu), something is calling rm ubuntu.dir (and failing), but I can't figure out what! Any ideas?

rm ubuntu.dirmake: unlink: ubuntu.dir: Operation not permitted
 ubuntu.tar

Error using qemu with the Ubuntu build/img

First off, just want to say what you're doing with this is absolutely amazing!

Steps to reproduce:

make ubuntu

qemu-system-x86_64 -drive file=linux.img,index=0,media=disk,format=raw -m 4096

See image for the error:

Screen Shot 2021-05-05 at 22 12 05

[Question] Scratch images

Hi, this project looks great, and I'm curious if it's possible to run scratch images same way?
Let's say pure c printf to terminal and infinity loop, or just a standalone web server with all required dependencies included, excluding systemd and other unnecessary parts?

Script fails with "failed to setup loop device"

Hi and thanks for sharing your work,
I wanted to test this, but I always get

[Format partition with ext3]
losetup: /os/linux.img: failed to set up loop device: Device or resource busy
Makefile:51: recipe for target 'linux.img' failed
make[1]: *** [linux.img] Error 1

I ran the following cmd line

sudo make ubuntu

It also failed with the same message when doing

sudo make debian

I'm running on an ubuntu 18.04.
Did I make any mistake in calling your script?

Update: I already checked issue #1 and added '--privileged=true' but the error still came up.

ubuntu 14.04

Can you do the same with ubuntu 14.04 and grub2?

Create Multiple Partitions

Is there a way to make multiple partitions? I'd like to break up the drive into three different partitions

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.