Comments (18)
@mjambon I agree with you. In the meantime, have you looked at using docker system prune
with a label filter? In our infra we've used two prune jobs in the past, one for everything, and then a specific one that allowed keeping specially tagged items longer.
/usr/bin/docker system prune -a -f --filter "label!=com.docker.ucp.version" --filter "until=120m"
would delete anything that does not have the com.docker.ucp.version
label AND is older than 120 minutes.
/usr/bin/docker container prune -f --filter "until=120m"
would prune all stopped containers older than 120 minutes. Thus, we would be left with running containers, and all images/networks/dangling images/dangling build cache items that have been used within 120 minutes and have the com.docker.ucp.version
label.
So you could tag your images/containers/etc with dev=true
and run /usr/bin/docker system prune -a -f --filter "label=dev" --filter "until=30m" and
/usr/bin/docker system prune -a -f --filter "label!=dev" --filter "until=120m". This would get rid of dev items taking up storage space much quicker, while keeping around other stuff.
from roadmap.
... Are there other problems you want solved related to clean up not yet mentioned?
Thanks for asking. I'll ping with a reminder about the point I'd made above about helping folks (wanting to clean up DD-related disk usage) with an easy way to move all the images to another drive, especially when running DD for Win with wsl 2. Or might anyone say that's been resolved since my last comment here, a year ago: #13 (comment)?
from roadmap.
Love the new feature π
The round / circular diagram is quite "hungry" on screen real-estate; I wonder if it has been considered to use a horizontal diagram instead, as I posted in my older comment above (matching macOS's approach); #13 (comment)
Doing so would likely not allow to have the "legend", which could be solved using a hover box;
from roadmap.
I also have frequently full disks on my development machines as well as our CI hosts running Jenkins, where the solution is to clear the cache nightly with a cron job, requiring base images to be downloaded again and again, defeating the point of having a cache.
I asked on ServerFault if there was a way to delete unused objects but keep the ones that have been recently used (I haven't received an answer at this time). I think it would make the cache more effective and greatly enhance the user experience when disk space is limited (it usually is) and network is slow.
I don't know anything about Docker internals but I imagine that tracking the usage of image layers would be a first step toward detecting unused objects worth deleting. Usage could be based on container lifetime, i.e. the termination timestamp of the last container that used a certain image layer. An approximate date would already be useful for caching purposes.
Food for thought.
Edit: I now realize that Docker Desktop is a specific part of the product, which is not particularly tied to the comments I made. Please let me know if should create a separate issue or more generally how to be helpful.
from roadmap.
I don't think this is a desktop issue. All installations have this problem.
from roadmap.
@tiborvass You're right, all installations potentially have this problem, but AIUI this issue is specifically about exposing the existing docker prune via Desktop UI. Enhancements to docker prune maybe belong in a separate ticket.
from roadmap.
For Docker Desktop, it would be great to also have an overview how disk space is consumed. There's parts that cannot be removed inside the container (e.g., docker
and containerd
itself), and parts that can be cleaned up (e.g. through docker system prune
).
Having an indicator to see what takes up space would be useful; something similar to macOS's presentation;
For cleaning up; I think both "automatic" and "manual" could make sense. For automatic cleaning, some care should be taken, because it's harder to decide if something is "ok" to be removed. When using BuildKit as a builder, build-cache can be automatically "garbage collected", and provides some configuration to customise (see moby/moby#37846)
For other resources both "manual" and "automatic" should default to NOT remove volumes, as they may contain important data (even if unused by a container, we cannot assume they can be cleaned up).
[ Cleanup resources ]
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
Possibly have checkboxes to customize. For example, docker image prune
(by default) only cleans up "untagged images", but allows using --all
to remove all images that are not currently used by a container, and docker builder prune
allows "keeping" X gigabytes of cache and/or only removing cache that hasn't been used longer in the last X hours
Cleanup resources
[ ] customize
[x] dangling images
[ ] all unused images
[x] stopped containers
[x] build-cache
[x] unused networks
[ ] unused volumes
from roadmap.
Yes but the feature first needs to be in engine anyway via daemon settings which would out-of-the-box work in Desktop via the daemon settings pane. Happy to have nicer UI on top of that, but I would argue that's not the highest priority. Priority is to have the feature in the first place.
from roadmap.
Does Docker Desktop do any automatic clean up today? If so, part of the UI that would be nice is to show objects queued for cleanup in the near future. Maybe also a log of what has been cleaned up, if there is any automation.
from roadmap.
@JimCodified Desktop does not do any automated clean-up, and I'm not sure we'd want to do that either because we'd be deleting the user's files. Certainly in the first version it would just be about reporting space better and allowing the user to select to clean it up.
from roadmap.
The problem still exists.
Scripts for pruning didn't help.
Win10, LSTC 2018, All updates installed.
from roadmap.
I realize this could be regarded as a separate issue, but it's definitely related to "resource cleanup": it sure would be nice if DD had a ui feature to move itself (or at least the vm location) from one drive to another.
People often install it and only find over time after many pulls and builds that DD is filling their drive--often their system drive, to great peril. And it's not trivial to move.
Yes, there's the experimental ui and its keywords (if you know them, and accurately track their changes over the years). But a single button click--which knew how to handle things for us (on per-os basis, or for Wsl vs WSL2, etc) would sure make life easier for many.
from roadmap.
@carehart is the "Disk image location" option in the preferences what you're looking for? https://docs.docker.com/docker-for-windows/#resources
Disk image location: Specify the location of the Linux volume where containers and images are stored.
You can also move the disk image to a different location. If you attempt to move a disk image to a location that already has one, you get a prompt asking if you want to use the existing image or replace it.
from roadmap.
Linking another ticket that's somewhat related (switching default logging driver to reduce disk space taken up by container logs); #163
from roadmap.
@carehart is the "Disk image location" option in the preferences what you're looking for? https://docs.docker.com/docker-for-windows/#resources
Disk image location: Specify the location of the Linux volume where containers and images are stored.
You can also move the disk image to a different location. If you attempt to move a disk image to a location that already has one, you get a prompt asking if you want to use the existing image or replace it.
That works for those on Hyper-V, yes (and I see I missed it was added/moved from being an experimental arg instead). But for those using WSL/WSL2, it doesn't appear. And sure, someone may say "take up the problem with MS", I'll just say that if DD creates the image, then it's reasonable to ask it (as part of this "cleanup" feature suggestion) that it (DD) could help with that use case as well.
But I will understand if the decision is made that it feels not DD's responsibility. Just wanted to put it on the table for consideration.
from roadmap.
Hi all! Somewhat relatedly to this item we have been prototyping two things we might build:
- Improving the clean up interface that we have on images and also making it available for the volume management interface
- Multi select check boxes on images and volumes for easy removal without having to enter the clean up mode (note that we plan on adding the disk space saved on this screen too like in clean up screen above).
With the goal of allowing you more control for freeing up space that is being used by volumes. Some mock ups attached and we would love any feedback you have (What do you like/not like? Whatβs missing from making this useful? Is anything confusing?)
Curious if there is any interest in having a clean up mode across resources rather than doing it separately?
from roadmap.
Also, based on the other feedback on this ticket, it sounds like there is a need for:
- Overall more visibility into what resources take up space
- The ability to set policies for automatic clean up on some frequency
- Exposing more of the prune capabilities within the Docker Dashboard (the UI) for example delete images older than X minutes
Are any the most important to you all on this? Are there other problems you want solved related to clean up not yet mentioned?
from roadmap.
Hi all! In 4.8 we shipped a disk usage extension to help with visibility and reclaiming space. We'd love to hear your feedback!
from roadmap.
Related Issues (20)
- Personal Access Token only for specific repository HOT 1
- Improvement in the integration of docker and/or docker-compose real-time volumes. HOT 4
- New HOT 2
- Dockerd as a python flask server:
- Better handling of Proxy-Authenticate via 407
- Allow changing of docker config location for WSL integration
- Docker hub rate limit for pro members HOT 1
- [Docker Desktop Linux] .desktop file to launch the Docker Desktop Dashboard
- [Docker Hub] Default team option for new repos HOT 2
- 323232
- 11
- 00
- 5555
- Option to disable word wrapping for container Logs in Docker Desktop for Windows
- Improve UI usability in Docker Desktop UI Container List View Action Buttons HOT 1
- Ability to mount docker volumes to WSL seamlessly via configuration
- Multiselect and bulk activity for Members view for Org Admins
- Default repo HOT 1
- OCI Volume Source HOT 1
- Enhanced access and visibility
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from roadmap.