saltstack-formulas / apt-formula Goto Github PK
View Code? Open in Web Editor NEWHome Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
License: Other
Home Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
License: Other
Why have update and upgrade only run when sources.list or sources.list.d are changed?
This means it can't be run as a maintenance job. I'm new to Saltstack formulas, so am I missing something obvious?
I modified and tested the formula to use additional pillar data, "always_update" and "always_upgrade" to enable the maintenance behavior, e.g.
{% from "apt/map.jinja" import apt as apt_map with context %}
{% set apt = pillar.get('apt', {}) %}
{% set always_update = apt.get('always_update', apt_map.always_update) %}
apt-get -y update:
{% if always_update %}
cmd.run
{% else %}
cmd.wait:
- watch:
- file: /etc/apt/sources.list
- file: /etc/apt/sources.list.d
{% endif %}
Any interesting in a pull request?
httpredir is now deprecated (eventough it works by redirecting to deb.debian.org) see this message on the mailing list for more info https://lists.debian.org/debian-mirrors/2017/02/msg00000.html
On debian (at least buster), the systemd service "apt-daily-upgrade.service", which install package upgrades is controlled by apt-daily-upgrade.timer.
Currently the service is triggered at 6:00 plus a random delay of 0-60 minutes :
# cat /lib/systemd/system/apt-daily-upgrade.timer
[Unit]
Description=Daily apt upgrade and clean activities
After=apt-daily.timer
[Timer]
OnCalendar=*-*-* 6:00
RandomizedDelaySec=60m
Persistent=true
[Install]
WantedBy=timers.target
Would it be possible to have a way of controlling the OnCalendar and RandomizedDelaySec parameters ?
Thanks
an example configuration would be :
apt:
unattended:
service:
upgrade:
calendar: "*-*-* 6:00"
delay: "60m"
The file /etc/apt/apt.conf.d/10periodic
is managed via the apt:unattended
pillar, but not all APT::Periodic
options are supported.
Those missing include, but are not limited to, MaxAge
, MaxSize
, and CleanInterval
.
As reported in the Slack #testing channel (which will disappear after a while).
https://saltstackcommunity.slack.com/archives/C7KKN9P6W/p1588231385039600
Rebuilt our pre-salted images at https://github.com/netmanagers/salt-image-builder after the CVE release and now beginning to run the tests of the master branch across the SaltStack Formulas. Got our first failures in the
apt-formula
, with aneon
warning that's been reintroduced by adevelop
=>master
port 8 days ago (https://github.com/saltstack/salt/blame/master/salt/states/pkgrepo.py#L313-L318).https://travis-ci.org/github/myii/apt-formula/jobs/681357767#L1517-L1530
RuntimeError: The warning triggered on filename '/usr/local/lib/python3.7/dist-packages/salt/states/pkgrepo.py', line number 315, is supposed to be shown until version 3000 is released. Current version is now 3000. Please remove the warning.
- Out of interest, why is a
salt.utils.versions.warn_until()
significant enough to trigger aRuntimeError
?
This line will need to be removed/adjusted:
apt-formula/apt/repositories.sls
Line 81 in baa83f2
the preferences.d files shipped via the pillars are quite nice, it would be nice to be able to do the same with apt.conf.d
When I run update.sls and upgrade.sls through highstate run.
`----------
ID: apt-get update
Function: cmd.run
Result: True
Comment: Command "apt-get update" run
Started: 14:55:03.296087
Duration: 7082.109 ms
Changes:
----------
pid:
21886
retcode:
0
stderr:
stdout:
Hit http://security.debian.org jessie/updates InRelease
Hit http://security.debian.org jessie/updates/main amd64 Packages
Hit http://security.debian.org jessie/updates/main Translation-en
Ign http://http.debian.net jessie InRelease
Hit http://http.debian.net jessie-backports InRelease
Hit http://repo.saltstack.com jessie InRelease
Ign http://httpredir.debian.org jessie InRelease
Hit http://http.debian.net jessie Release.gpg
Hit http://repo.saltstack.com jessie/main amd64 Packages
Get:1 http://http.debian.net jessie-backports/main Sources/DiffIndex [27.8 kB]
Hit http://httpredir.debian.org jessie-updates InRelease
Get:2 http://http.debian.net jessie-backports/main amd64 Packages/DiffIndex [27.8 kB]
Hit http://httpredir.debian.org jessie-backports InRelease
Get:3 http://http.debian.net jessie-backports/main Translation-en/DiffIndex [27.4 kB]
Hit http://httpredir.debian.org jessie Release.gpg
Ign http://repo.saltstack.com jessie/main Translation-en
Hit http://http.debian.net jessie Release
Get:4 http://httpredir.debian.org jessie-updates/main amd64 Packages/DiffIndex [1012 B]
Hit http://http.debian.net jessie/main Sources
Get:5 http://httpredir.debian.org jessie-updates/main Translation-en/DiffIndex [736 B]
Hit http://http.debian.net jessie/main amd64 Packages
Hit http://http.debian.net jessie/main Translation-en
Get:6 http://httpredir.debian.org jessie-backports/main Sources/DiffIndex [27.8 kB]
Get:7 http://httpredir.debian.org jessie-backports/main amd64 Packages/DiffIndex [27.8 kB]
Get:8 http://httpredir.debian.org jessie-backports/main Translation-en/DiffIndex [27.4 kB]
Hit http://httpredir.debian.org jessie Release
Hit http://httpredir.debian.org jessie/main amd64 Packages
Hit http://httpredir.debian.org jessie/main Translation-en
Fetched 168 kB in 3s (45.7 kB/s)
ID: apt-get upgrade
Function: cmd.run
Result: False
Comment: Command "apt-get upgrade" run
Started: 14:55:10.378360
Duration: 268.491 ms
Changes:
----------
pid:
21941
retcode:
1
stderr:
stdout:
Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be upgraded:
libdns-export100 libirs-export91 libisc-export95 libisccfg-export90
linux-image-3.16.0-4-amd64 python-requests python-urllib3
7 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 34.6 MB of archives.
After this operation, 204 kB disk space will be freed.
Do you want to continue? [Y/n] Abort.
Succeeded: 1 (changed=2)
Total states run: 2
ERROR: Minions returned with non-zero exit code
`
Ubuntu has ubuntu-keyring, Mint has linuxmint-keyring. Forcing debian-archive-keyring just adds a useless package on those systems.
The formula handles the following syntax
- name: {{ r_type }} {{ r_arch }} {{ r_url }} {{ r_distro }} {{ r_comps }}
It would useful to have another freestyle option for things such as [trusted=yes]
I thought I'd be able to use this apt.update
state to update the apt db on servers, so I tried
salt 'minion' state.sls apt.update
It fails with
ID: apt-get update
Function: cmd.wait
Result: False
Comment: The following requisites were not found:
watch:
file: /etc/apt/sources.list
file: /etc/apt/sources.list.d
Started: 13:42:21.398096
Duration: 0.061 ms
Changes:
README lists this state so it looks like it's callable standalone (not just for internal use) but it seems not.
I'm using the most recent master bea590b
Salt Version:
Salt: 3002.6
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.11.3
libgit2: 1.1.0
M2Crypto: 0.37.1
Mako: Not Installed
msgpack: 1.0.0
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.9.7
pygit2: 1.4.0
Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
python-gnupg: 0.4.6
PyYAML: 5.3.1
PyZMQ: 20.0.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: debian 11 bullseye
locale: utf-8
machine: x86_64
release: 5.10.0-22-amd64
system: Linux
version: Debian GNU/Linux 11 bullseye
master and minion is the same machine.
apt:
remove_sources_list: true
repositories:
salt:
distro: bullseye
url: https://repo.saltproject.io/salt/py3/debian/11/amd64/3007
key_url: https://repo.saltproject.io/salt/py3/debian/11/amd64/SALT-PROJECT-GPG-PUBKEY-2023.gpg
opts:
signed-by: /etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg
This role does not seem to know the aptkey
false from pkgrepo.managed
and does not download the key into /etc/apt/keyrings
. apt-key itself is deprecated. Looking through the source cannot seem to find the option to set - aptkey: False
in this role.
Use above pillar. See that it errors on the salt repo:
local:
----------
ID: deb salt
Function: pkgrepo.managed
Name: deb [ signed-by=/etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg ] https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye main
Result: True
Comment: Configured package repo 'deb [ signed-by=/etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg ] https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye main'
Started: 19:23:01.724969
Duration: 813.624 ms
Changes:
----------
repo:
deb [ signed-by=/etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg ] https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye main
----------
ID: apt.refresh_db
Function: module.run
Name: pkg.refresh_db
Result: False
Comment: Module function pkg.refresh_db threw an exception. Exception: W: GPG error: https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 64CBBC8173D76B3F
E: The repository 'https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye InRelease' is not signed.
Started: 19:23:02.541283
Duration: 4257.558 ms
Changes:
Summary for local
--------------
Succeeded: 130 (changed=1)
Failed: 1
--------------
Total states run: 131
Total run time: 6.392 s
I'd like to use aptkey: False
.
Look through the documentation (pillar) and source code.
v0.10.3
Saltmaster:
Salt Version:
Salt: 3002.6
Dependency Versions:
cffi: 1.14.5
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.10
libgit2: 0.27.7
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 0.5.6
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.19
pycrypto: Not Installed
pycryptodome: 3.6.1
pygit2: 0.27.4
Python: 3.7.3 (default, Jul 25 2020, 13:03:44)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 17.1.2
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.1
System Versions:
dist: debian 10 buster
locale: utf-8
machine: x86_64
release: 4.19.0-14-amd64
system: Linux
version: Debian GNU/Linux 10 buster
Saltminion:
Salt Version:
Salt: 3003
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.10
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 0.5.6
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.6.1
pygit2: Not Installed
Python: 3.7.3 (default, Jan 22 2021, 20:04:44)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 17.1.2
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.1
System Versions:
dist: debian 10 buster
locale: utf-8
machine: x86_64
release: 4.19.0-16-amd64
system: Linux
version: Debian GNU/Linux 10 buster
{#- URLs fuer Repositories und GnuPG-Keys #}
{%- set repo_url = 'https://baltocdn.com/helm/stable/debian/' %}
{%- set key_url = 'https://baltocdn.com/helm/signing.asc' %}
# Name der Debian-Version ermitteln
{%- if salt['grains.get']('osmajorrelease') == 10 %}
{%- set distro = 'buster' %}
{%- elif salt['grains.get']('osmajorrelease') == 11 %}
{%- set distro = 'bullseye' %}
{%- else %}
{%- set distro = 'buster' %}
{%- endif %}
# Architektur des Systems erkennen und in Variable festhalten
{%- set osarch = salt['grains.get']('osarch') %}
### Major-Release-Nummer in Variable speichern
{%- set release = salt['grains.get']('osmajorrelease') %}
apt:
repositories:
# Repo von Saltstack fuer stretch und buster Hosts
helm:
distro: "all main"
#comps: [all, main]
url: {{ repo_url }}
arch: [amd64]
type: [binary]
keyurl: {{ key_url }}
The signing key from the helm docs isn't working for trusting the repo.
I also tried to forced to trust it with the option "trusted: yes" but this does not do the trick.
Err:5 https://baltocdn.com/helm/stable/debian all InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 294AC4827C1A168A
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://baltocdn.com/helm/stable/debian all InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 294AC4827C1A168A
W: Failed to fetch https://baltocdn.com/helm/stable/debian/dists/all/InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 294AC4827C1A168A
W: Some index files failed to download. They have been ignored, or old ones used instead.
Minion debug log:
1 local:
2 ----------
3 ID: debian-archive-keyring
4 Function: pkg.installed
5 Result: True
6 Comment: All specified packages are already installed
7 Started: 10:17:23.416126
8 Duration: 34.819 ms
9 Changes:
10 ----------
11 ID: /etc/apt/sources.list
12 Function: file.managed
13 Result: True
14 Comment: File /etc/apt/sources.list exists with proper permissions. No changes made.
15 Started: 10:17:23.455036
16 Duration: 2.422 ms
17 Changes:
18 ----------
19 ID: /etc/apt/sources.list.d
20 Function: file.directory
21 Result: True
22 Comment: The directory /etc/apt/sources.list.d is in the correct state
23 Started: 10:17:23.457792
24 Duration: 1.846 ms
25 Changes:
26 ----------
27 ID: deb debian10-saltstack
28 Function: pkgrepo.managed
29 Name: deb [arch=amd64 ] https://repo.saltproject.io/py3/debian/10/amd64/3003/ buster main
30 Result: True
31 Comment: Configured package repo 'deb [arch=amd64 ] https://repo.saltproject.io/py3/debian/10/amd64/3003/ buster main'
32 Started: 10:17:23.460649
33 Duration: 783.444 ms
34 Changes:
35 ----------
36 repo:
37 deb [arch=amd64 ] https://repo.saltproject.io/py3/debian/10/amd64/3003 buster main
38 ----------
39 ID: deb helm
40 Function: pkgrepo.managed
41 Name: deb [arch=amd64 ] https://baltocdn.com/helm/stable/debian/ all main main
42 Result: True
43 Comment: Configured package repo 'deb [arch=amd64 ] https://baltocdn.com/helm/stable/debian/ all main main'
44 Started: 10:17:24.244553
45 Duration: 211.369 ms
46 Changes:
47 ----------
48 repo:
49 deb [arch=amd64 ] https://baltocdn.com/helm/stable/debian all main main
50 ----------
51 ID: apt.refresh_db
52 Function: module.run
53 Name: pkg.refresh_db
54 Result: True
55 Comment: Module function pkg.refresh_db executed
56 Started: 10:17:24.457909
57 Duration: 2357.477 ms
58 Changes:
59 ----------
60 ret:
61 ----------
62 http://deb.debian.org/debian buster InRelease:
63 None
64 http://deb.debian.org/debian buster-updates InRelease:
65 None
66 http://security.debian.org/debian-security buster/updates InRelease:
67 None
68 https://baltocdn.com/helm/stable/debian all InRelease:
69 False
70 https://download.docker.com/linux/debian buster InRelease:
71 None
72 https://packages.gitlab.com/runner/gitlab-runner/debian buster InRelease:
73 None
74 https://repo.saltproject.io/py3/debian/10/amd64/3003 buster InRelease:
75 None
76
77 Summary for local
78 ------------
79 Succeeded: 6 (changed=3)
80 Failed: 0
81 ------------
82 Total states run: 6
83 Total run time: 3.391 s
The Helm repo should be trusted.
don't know
Hello,
I'm using this formula to manage my apt repositories, I only include apt.repositories
in my states, and my pillar looks like
{% set oscodename = grains['oscodename'] %}
apt:
remove_sources_list: true
clean_sources_list_d: true
remove_preferences: true
clean_preferences_d: true
repositories:
{{ oscodename }}:
distro: {{ oscodename }}
url: http://ftp.debian.org/debian
comps: [main, contrib, non-free]
type: [binary]
[...]
When I run a highstate or only the apt.repositories state, I got this error
[INFO ] Loading fresh modules for state activity
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://apt/repositories.sls'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://apt/map.jinja'
[INFO ] Running state [debian-archive-keyring] at time 15:26:43.653378
[INFO ] Executing state pkg.installed for debian-archive-keyring
[INFO ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}\n', '-W'] in directory '/root'
[INFO ] Package debian-archive-keyring is already installed.
[INFO ] Completed state [debian-archive-keyring] at time 15:26:43.866063
[INFO ] Running state [/etc/apt/sources.list] at time 15:26:43.868531
[INFO ] Executing state file.absent for /etc/apt/sources.list
[INFO ] File /etc/apt/sources.list is not present
[INFO ] Completed state [/etc/apt/sources.list] at time 15:26:43.870025
[INFO ] Running state [/etc/apt/sources.list.d] at time 15:26:43.870463
[INFO ] Executing state file.directory for /etc/apt/sources.list.d
[INFO ] {'removed': ['/etc/apt/sources.list.d/wheezy-binary.list']}
[INFO ] Completed state [/etc/apt/sources.list.d] at time 15:26:43.872370
[INFO ] Running state [deb http://ftp.debian.org/debian wheezy main contrib non-free] at time 15:26:43.873656
[INFO ] Executing state pkgrepo.managed for deb http://ftp.fr.debian.org/debian wheezy main contrib non-free
[WARNING ] could not open file '/etc/apt/sources.list'
[WARNING ] could not open file '/etc/apt/sources.list'
[INFO ] Executing command 'apt-get -q update' in directory '/root'
[WARNING ] could not open file '/etc/apt/sources.list'
[...]
The error/warning (could not open file '/etc/apt/sources.list') appears several times, at least twice for each apt repository.
Do you have any idea of what can be the reason for this warning?
Warnings:
The following package(s) are "virtual package" names: apt-transport-https. These will no longer be supported as of the Fluorine release. Please update your SLS file(s) to use the actual package name.
apt-transport-https:
pkg.installed
should be:
apt-transport-https:
pkg.installed:
- name: apt-transport-https
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.