larsks / undocker Goto Github PK

License: GNU General Public License v3.0

Python 100.00%

undocker's Introduction

Unpacks a Docker image.

Usage

usage: undocker.py [-h] [--ignore-errors] [--output OUTPUT] [--verbose]
                   [--debug] [--list] [--layer LAYER]
                   image

positional arguments:
  image

optional arguments:
  -h, --help            show this help message and exit
  --ignore-errors, -i   Ignore OS errors when extracting files
  --output OUTPUT, -o OUTPUT
                        Output directory (defaults to ".")
  --verbose, -v
  --debug, -d
  --list, --ls          List layers in an image
  --layer LAYER, -l LAYER
                        Extract only the specified layer

Examples

Extract an entire image:

$ docker save busybox | undocker -i -o busybox busybox

The -i option is necessary here because I am not running as root, and the extract operation will fail when it attempts to create device nodes.

List the layers in an image:

$ docker save busybox | undocker --list
511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
df7546f9f060a2268024c8a230d8639878585defcc1bc6f79d2728a13957871b
ea13149945cb6b1e746bf28032f02e9b5a793523481a0a18645fc77ad53c4ea2
4986bf8c15363d1c5d15512d5266f8777bfba4974ac56e3270e7760f6f0a8125

Extract only specific layers:

$ docker save busybox |
  undocker -o busybox -v \
  -l 4986bf8c15363d1c5d15512d5266f8777bfba4974ac56e3270e7760f6f0a8125 \
  busybox
INFO:undocker:extracting image busybox (4986bf8c15363d1c5d15512d5266f8777bfba4974ac56e3270e7760f6f0a8125)
INFO:undocker:extracting layer 4986bf8c15363d1c5d15512d5266f8777bfba4974ac56e3270e7760f6f0a8125

License

undocker -- a tool for decomposing Docker images
Copyright (C) 2015 Lars Kellogg-Stedman [email protected]

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

undocker's People

Contributors

Stargazers

Watchers

undocker's Issues

any chance to push this to PyPI?

thank you very much for this tool. Any chance to push this to PyPI? I'd love to get it install/updated with ease.

KeyError: "filename 'repositories' not found"

After pulling busybox and running the following command :

I get this error, what is wrong?

Permissions issue

Apparently unable to read a docker layer or part of it. Looks like a permissions issue.

$docker save jenkins/jenkins:lts-alpine | undocker -i -o unpacked jenkins/jenkins:lts-alpine
Traceback (most recent call last):
  File "/usr/local/bin/undocker", line 9, in <module>
    load_entry_point('undocker==5', 'console_scripts', 'undocker')()
  File "build/bdist.linux-x86_64/egg/undocker.py", line 131, in main
  File "build/bdist.linux-x86_64/egg/undocker.py", line 61, in find_layers
AttributeError: 'ExFileObject' object has no attribute 'readable'

Is this a problem with the image or the undocker project??

Push new python3 compatible version to PyPI

Could you push a new version to PyPI?
The latest release is from Feb 17th 2016 and does not include the fix for python3 compatibility yet (b786b9c).

OSError: [Errno 2] No such file or directory: 'etc/rcS.d/.wh.S02mountdevsubfs.sh'

I'm trying to extract the contents of my build: https://hub.docker.com/r/ericfont/armv7hf-debian-qemu-appimagekit/builds/bnywuskzyttfjagcp6o6at5/

docker save ericfont/armv7hf-debian-qemu-appimagekit:helloworld | /usr/bin/python2 ~/undocker.py -o armv7hf-debian-qemu-appimagekit-helloworld ericfont/armv7hf-debian-qemu-appimagekit:helloworld

produces this error:

Traceback (most recent call last):
  File "/home/e/undocker.py", line 145, in 
    main()
  File "/home/e/undocker.py", line 140, in main
    os.unlink(path)
OSError: [Errno 2] No such file or directory: 'etc/rcS.d/.wh.S02mountdevsubfs.sh'

I don't know what that means. I'm on Parabola linux x86-64 and armv7a. I'm also noticing that the output folder doesn't seem to contain everything, becuase I ran apt-get install inside the build, but I don't see those resulting files in the output folder.

Does this work for windows containers, in windows OS

using powershell I cant tell if
docker pull <image> | undocker -o <image>

is working.

undocker --help works fine. but I cant tell if the | (pipe) in powershell is working properly

OSError: [Errno 21] Is a directory: 'var/lib/apt/lists/partial'

Hey there

Thanks for coming up with this tool; I'm giving it a try and am running into the following issue:

root@fit06:~/image# docker save fitr2lab/ubuntu16 | python3 ../undocker/undocker.py
Traceback (most recent call last):
  File "../undocker/undocker.py", line 174, in <module>
    main()
  File "../undocker/undocker.py", line 167, in main
    os.unlink(newpath)
IsADirectoryError: [Errno 21] Is a directory: 'var/lib/apt/lists/partial'

I expect the issue to be easily reproducible from that same image which is published at dockerhub
The directory that I am trying to populate is empty
I am using the latest version of undocker.py as available at github (which btw does not seem to be the exact same one as what I first got with pip, but another matter..)

The host OS is this:

root@fit06:~/image# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"

root@fit06:~/image# python3 --version
Python 3.5.2

Getting IsADirectoryError when trying to run examples

undocker -i -o seems to be broken.

When I try to run the example from the README:

% docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
Digest: sha256:a9286defaba7b3a519d585ba0e37d0b2cbee74ebfe590960b0b1d6a5e97d1e1d
Status: Image is up to date for busybox:latest
docker.io/library/busybox:latest
% docker save busybox | undocker -i -o busybox busybox
Traceback (most recent call last):
  File "/home/nc/.local/bin/undocker", line 8, in <module>
    sys.exit(main())
  File "/home/nc/.local/lib/python3.8/site-packages/undocker.py", line 99, in main
    open(args.image, 'rb') if args.image
IsADirectoryError: [Errno 21] Is a directory: 'busybox'

Running undocker 7:

% pip3 show undocker
Name: undocker
Version: 7
Summary: Unpack docker images
Home-page: http://github.com/larsks/undocker
Author: Lars Kellogg-Stedman
Author-email: [email protected]
License: UNKNOWN
Location: /home/nc/.local/lib/python3.8/site-packages
Requires:
Required-by:

Make undocker work on a saved image

Since the first thing undocker does is to take the stdin and write to a temp file, would it be possible to allow the user to save to disk first, and then just run undocker on the already saved image?

For instance, something like this:

docker save busybox -o /tmp/busybox.tar
undocker -i -o /tmp/filesystem busybox /tmp/busybox.tar

failed to find image ubuntu with tag latest

Hi,
I am trying to get layers of an image. Image name is ubuntu:14.04.
But I face this error.

docker save ubuntu:14.04 | undocker --layers ERROR:undocker:failed to find image ubuntu with tag latest

I appreciate if you could guide to solve it.

Too many levels of symbolic links

I'm looking for guidance on how to debug where is the problem and work around it.

The image is debian:8 with many packages installed, including tzdata. Also, it's indeterministic, It always fails on different symlinks.

root=/media/data/infra-state
rm -rf $root
mkdir $root
cd $root
docker save ${name} |sudo ~/bin/python ~/bin/undocker.py --output .

Traceback (most recent call last):
  File "app_main.py", line 75, in run_toplevel
  File "/home/core/bin/undocker.py", line 129, in <module>
    main()
  File "/home/core/bin/undocker.py", line 125, in main
    layer.extractall(path=args.output)
  File "/home/core/pypy/lib-python/2.7/tarfile.py", line 2070, in extractall
    self.extract(tarinfo, path)
  File "/home/core/pypy/lib-python/2.7/tarfile.py", line 2107, in extract
    self._extract_member(tarinfo, os.path.join(path, tarinfo.name))
  File "/home/core/pypy/lib-python/2.7/tarfile.py", line 2183, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/home/core/pypy/lib-python/2.7/tarfile.py", line 2223, in makefile
    with bltn_open(targetpath, "wb") as target:
IOError: [Errno 40] Too many levels of symbolic links: './usr/share/zoneinfo/Australia/Broken_Hill'

# It extracted
$ sudo ls -lh /media/data/infra-state/usr/share/zoneinfo/Australia/ |fgrep -i 'broken_hill'
lrwxrwxrwx  1 root root   10 Apr 22 01:34 Broken_Hill -> Yancowinna
lrwxrwxrwx  1 root root   11 Apr 22 01:34 Yancowinna -> Broken_Hill

# Here's what is supposed to be in image (maybe it's only correctly constructed by another layer):
$ docker run --rm -it infra-state ls -lh /usr/share/zoneinfo/Australia/ |egrep 'Broken_Hill|Yancowinna'
-rw-r--r--  8 root root 2.2K Apr 14 14:50 Broken_Hill
lrwxrwxrwx  1 root root   11 Apr 14 14:50 Yancowinna -> Broken_Hill

Allow extracting only files that match pattern

I need to get files matching a pattern from an image, without creating a container.

Would it be possible to extend undocker to take a pattern and only extract files matching that pattern from each layer?

Images may be bigger than available memory

Traceback (most recent call last):
  File "app_main.py", line 75, in run_toplevel
  File "/home/core/bin/undocker.py", line 125, in <module>
    main()
  File "/home/core/bin/undocker.py", line 70, in main
    fd.write(sys.stdin.read())
MemoryError

AUFS .wh. whiteouts are not not honored

As a result, the extracted layers are not correct when files are moved or removed

No layers listed when we execute the command: ./undocker.py -v --list

Issue

No layers listed when we execute the command ./undocker.py -v --list

docker save busybox | ./undocker.py -v --list
busybox: latest

but using the command docker save busybox | tar -tf- list the files correctly

cabb9f684f8ba3edb303d578bfd7d709d853539ea1b420a3f6c81a08e85bb3d7.json
f98c3de345c52d9949bd57f470c385d035a6d36315356db5719851b6ec513763/
f98c3de345c52d9949bd57f470c385d035a6d36315356db5719851b6ec513763/VERSION
f98c3de345c52d9949bd57f470c385d035a6d36315356db5719851b6ec513763/json
f98c3de345c52d9949bd57f470c385d035a6d36315356db5719851b6ec513763/layer.tar
manifest.json
repositories

OS: Centos7

Docker info

docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 21
  Running: 2
  Paused: 0
  Stopped: 19
 Images: 155
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.42.2.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.14GiB
 Name: h01-121
 ID: JNFA:3SZC:3VZ7:XJ6H:RSGL:WCHZ:JQWV:W37I:3VSR:QWGW:22T5:EJ65
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Turn image layers into standalone images

This is a question. Docker prevents to use the source layers as an image, the ID is missing (<missing> is shown).

My question is, can I use this too to extract all image layers into standalone images with image ID available for each layer?

license?

Hi @larsks : what would be the license for your script?

Handle image names that contain host + port number

Images pulled from a non-standard registry have names of the form e.g. myregistry.local:5000/test-image. The presence of the port number seems to confuse the script, causing it to raise an error:

ERROR:__main__:failed to find image myregistry.local with tag 5000/test-image

This is true whether the image is specified on the command line or extracted from the repositories metadata. Fixing it in the latter case should be simple, as the image names in the repositories metadata never contain the tag. The command line case requires some more complex parsing of the argument.

Pull request to follow.

python3 compatibility

This program is possible to install using pip3!

But it doesn't work when installed as a python3 program. The problem I encountered was the print '' notation instead of print('') notation. 2to3 agrees with me about the print notation, but it also wants to wrap repos.keys() and repos.items() in list(), but that I dunno what's good for.

--- undocker.py (original)
+++ undocker.py (2to3)
@@ -82,15 +82,15 @@
             repos = json.load(repos)

             if args.list:
-                for name, tags in repos.items():
-                    print '%s: %s' % (
+                for name, tags in list(repos.items()):
+                    print('%s: %s' % (
                         name,
-                        ' '.join(tags))
+                        ' '.join(tags)))
                 sys.exit(0)

             if not args.image:
                 if len(repos) == 1:
-                    args.image = repos.keys()[0]
+                    args.image = list(repos.keys())[0]
                 else:
                     LOG.error('No image name specified and multiple '
                               'images contained in archive')
@@ -112,7 +112,7 @@
             layers = list(find_layers(img, top))

             if args.layers:
-                print '\n'.join(reversed(layers))
+                print('\n'.join(reversed(layers)))
                 sys.exit(0)

             if not os.path.isdir(args.output):

Alternatively, maybe this shouldn't be installable from pip3.