Comments (6)
@hamirmahal I'm not a developer but want to respond to your ticket.
Diagnosis
The pip install -r requirements.txt
step in your workflow benefits from caching. However, the setup-python action doesn't control pip's behavior, and cannot reduce the number of "Collecting" and "Using" lines that you're seeing.
However, "Collecting" and "Using" aren't actually consuming much time -- it's the installation itself that consumes the vast majority of time. You can verify this by reviewing the raw logs, which contain a timestamp for every output line:
Looking at those raw logs, pip spends ~6 seconds (from 2024-05-06T02:41:05.2881468Z
to 2024-05-06T02:41:11.4118495Z
) printing "Collecting" and "Using" lines. It then spends ~26 seconds (from 2024-05-06T02:41:11.4118495Z
to 2024-05-06T02:41:37.9340734Z
) actually installing your dependencies.
Best practice
My recommendation is to disable setup-python's pip caching entirely and focus exclusively on caching an entire virtual environment.
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
id: setup-python
with:
python-version: "3.11"
# Write the exact Python version to a file for cache-busting.
- run: |
echo "${{ steps.setup-python.outputs.python-version }}" > ".installed-python"
# THIS! This is where you'll save all your time!
# Never cache pip dependencies! Cache virtual environments!
- uses: "actions/cache@v4"
id: "restore-cache"
with:
key: "venv-${{ hashFiles('.installed-python', 'requirements.txt') }}"
path: |
.venv/
# If Python 3.11.x upgrades to 3.11.y, or if requirements.txt gets updated,
# the cache lookup above will miss, and the venv needs to be recreated.
- name: "Create a virtual environment"
if: "steps.restore-cache.outputs.cache-hit == false"
run: |
python -m venv .venv
.venv/bin/python -m pip install --upgrade pip setuptools wheel
.venv/bin/python -m pip install -r requirements.txt
- run: .venv/bin/python src/main.py
Caching an entire virtual environment is going to save you a ton of time. The only thing you need to watch out for is cache-busting. The example above busts the cache based on the exact Python version (like "3.11.7") and based on your requirements.txt
file. If you're running this workflow on multiple platforms you'll need to include that in the cache key, too.
@HarithaVattikuti I think that this ticket can be closed; the report is referring to pip behavior, not setup-python behavior.
from setup-python.
https://github.com/hamirmahal/cache-pip-install/commits/continue-caching-with-setup-python/
from setup-python.
Initial Run
Run / python-program (push) Successful in 1m Details
Cached Run
Run / python-program (push) Successful in 54s Details
from setup-python.
Hello @hamirmahal
Thank you for creating this issue. We will investigate it and get back to you as soon as we have some feedback.
from setup-python.
You're welcome @HarithaVattikuti.
from setup-python.
Hello @hamirmahal 👋,
Regarding pip caching, it works by storing downloaded packages in a directory so that they don't need to be re-downloaded when they're needed in the future. This indeed saves time by skipping the downloading step.
Thanks to @kurtmckee for the insightful explanation. As rightly pointed out, the "Collecting" and "Using" stages in pip's process are essential and cannot be bypassed entirely, even when caching is employed. The "Collecting" stage is where pip identifies the necessary packages for installation (including dependencies), and the "Using" stage is where pip indicates that it's utilising a cached package, thus avoiding the need for a new download.
Hope this clarifies and I am proceeding to close this issue. Thank you for your patience and cooperation:)
Please feel free to reach us out incase of any further concerns!
from setup-python.
Related Issues (20)
- Missing support for Python 3.8 on Ubuntu 24.04 runner HOT 2
- Version not found when using new arm-based linux runners (ubuntu 22.04) HOT 4
- Add Python 3.12.4 (released 2 days ago) HOT 6
- [Feature Request] Add Python debug builds with `Py_DEBUG` flag HOT 1
- Python 3.12.4: Extract downloaded archive; gzip: stdin: not in gzip format HOT 22
- is python 3.12.4 download assets broken HOT 7
- Python 3.13 fails to install on windows: Offset to Central Directory cannot be held in an Int64. HOT 6
- Verify checksum of downloaded version archive HOT 1
- Support os-release standard HOT 5
- setup-python CI workflows have been broken for more than 2 months HOT 1
- Sporadically missing patch versions on python-versions for arm64 HOT 5
- `python-version-file` does not support multiple versions HOT 2
- Cannot parse version number from TOML 1.0.0 pyproject.toml file HOT 5
- Associate .py with python on Windows HOT 1
- Ability to set desired version of pip, or to not update it at all HOT 3
- Python version '3.12.x' with architecture 'arm64' was not found for Ubuntu 24.04 HOT 3
- Occasionally getting `candidates is not iterable` error HOT 5
- Self-hosted Windows Runner Fails to Install Python HOT 6
- Python 3.9/3.10 on Darwin runners fails: Error: dyld[78135]: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from setup-python.