voxpupuli / puppet-elastic_stack Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Yum can't find the kibana packages in the https://artifacts.elastic.co/packages/oss-6.x/yum repository.
Yum can find the kibana packages in https://artifacts.elastic.co/packages/6.x/yum repository.
So when oss is set to true, kibana-oss package installation fails.
First of all, thank you for provide this functionality (although it has become now a requirement of the ES Module since version 6.3.0
).
Previously in the ES Module we were able pin an specific release we wanted, such as for example:
elasticsearch::repo_version: '1.7.6'
now the suggestion from voxpupuli/puppet-elasticsearch#959, is to do something such as this:
elastic_stack::repo::version: 1.7.6
and you might expect this to work, but since this is not a valid Integer it will fail as you may expect:
Error: Evaluation Error: Error while evaluating a Function Call, Class[Elastic_stack::Repo]: parameter 'version' expects an Integer value, got Float
Can you please modify this parameter so that we can keep pinning specific releases (yes, it's an old release but hey, world is not perfect) without breaking the compatibility?
For testing purposes, this is not specified on the README file, It seems like ES 1.x version is not supported either. On my hiera
configuration I have set the following settings:
elasticsearch::version: '1.7.6'
elastic_stack::repo::version: 1
This is what you get:
Notice: /Stage[main]/Apt::Update/Exec[apt_update]/returns: W: The repository 'https://packages.elastic.co/elasticsearch/1.x/debian stable Release' does not have a Release file.
Notice: /Stage[main]/Apt::Update/Exec[apt_update]/returns: E: Failed to fetch https://packages.elastic.co/elasticsearch/1.x/debian/dists/stable/main/binary-amd64/Packages 404 Not Found
Notice: /Stage[main]/Apt::Update/Exec[apt_update]/returns: E: Some index files failed to download. They have been ignored, or old ones used instead.
Error: /Stage[main]/Apt::Update/Exec[apt_update]: Failed to call refresh: '/usr/bin/apt-get update' returned 100 instead of one of [0]
Error: /Stage[main]/Apt::Update/Exec[apt_update]: '/usr/bin/apt-get update' returned 100 instead of one of [0]
This is the resulting file with the repository /etc/apt/sources.list.d/elastic.list
:
deb https://packages.elastic.co/elasticsearch/1.x/debian stable main
PS: I'm running this under stack Ubuntu Xenial and Puppet module 6.3.0
using elastic_stack to manage repo at fresh debian bullseye installation:
class { 'elastic_stack::repo':
version => 7,
}
Error: certificate verify failed (unable to get local issuer certificate)
like to have "weak_ssl" option for apt::source module as workaround:
Boolean $apt_weak_ssl = false,
.. and then:
case $facts['os']['family'] {
'Debian': {
include apt
apt::source { 'elastic':
ensure => 'present',
comment => $description,
location => $base_url,
release => 'stable',
repos => 'main',
key => {
'id' => $key_id,
'source' => $key_source,
'weak_ssl' => $apt_weak_ssl,
},
include => {
'deb' => true,
'src' => false,
},
pin => $priority,
}
}
Error: Could not set 'present' on ensure: SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate) (file: /etc/puppetlabs/code/environments/production/modules/apt/manifests/key.pp, line: 55)
Error: Could not set 'present' on ensure: SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate) (file: /etc/puppetlabs/code/environments/production/modules/apt/manifests/key.pp, line: 55)
Wrapped exception:
SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)
Error: /Stage[first]/Elastic_stack::Repo/Apt::Source[elastic]/Apt::Key[Add key: 46095ACC8548582C1A2699A9D27D666CD88E42B4 from Apt::Source elastic]/Apt_key[Add key: 46095ACC8548582C1A2699A9D27D666CD88E42B4 from Apt::Source elastic]/ensure: change from 'absent' to 'present' failed: Could not set 'present' on ensure: SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate) (file: /etc/puppetlabs/code/environments/production/modules/apt/manifests/key.pp, line: 55)
The release tarball for 6.2.4 contains someone's local .git
directory:
$ tar tvf elastic-elastic_stack-6.2.4.tar.gz
drwxrwxr-x 1000/1000 0 2018-09-17 01:12 elastic-elastic_stack-6.2.4
-rw-rw-r-- 1000/1000 177 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.fixtures.yml
drwxrwxr-x 1000/1000 0 2018-09-17 01:12 elastic-elastic_stack-6.2.4/.git
-rw-rw-r-- 1000/1000 361 2018-09-17 01:09 elastic-elastic_stack-6.2.4/.git/COMMIT_EDITMSG
-rw-rw-r-- 1000/1000 215 2018-09-17 01:08 elastic-elastic_stack-6.2.4/.git/FETCH_HEAD
-rw-rw-r-- 1000/1000 23 2018-07-17 02:37 elastic-elastic_stack-6.2.4/.git/HEAD
-rw-rw-r-- 1000/1000 41 2018-09-17 01:11 elastic-elastic_stack-6.2.4/.git/ORIG_HEAD
drwxrwxr-x 1000/1000 0 2018-09-17 01:12 elastic-elastic_stack-6.2.4/.git/branches
-rw-rw-r-- 1000/1000 474 2018-08-28 01:51 elastic-elastic_stack-6.2.4/.git/config
-rw-rw-r-- 1000/1000 73 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/description
drwxrwxr-x 1000/1000 0 2018-09-17 01:12 elastic-elastic_stack-6.2.4/.git/hooks
-rwxrwxr-x 1000/1000 478 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/applypatch-msg.sample
-rwxrwxr-x 1000/1000 896 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/commit-msg.sample
-rwxrwxr-x 1000/1000 3327 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/fsmonitor-watchman.sample
-rwxrwxr-x 1000/1000 189 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/post-update.sample
-rwxrwxr-x 1000/1000 424 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/pre-applypatch.sample
-rwxrwxr-x 1000/1000 1642 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/pre-commit.sample
-rwxrwxr-x 1000/1000 1348 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/pre-push.sample
-rwxrwxr-x 1000/1000 4898 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/pre-rebase.sample
-rwxrwxr-x 1000/1000 544 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/pre-receive.sample
-rwxrwxr-x 1000/1000 1492 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/prepare-commit-msg.sample
-rwxrwxr-x 1000/1000 3610 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/hooks/update.sample
-rw-rw-r-- 1000/1000 2001 2018-09-17 01:11 elastic-elastic_stack-6.2.4/.git/index
drwxrwxr-x 1000/1000 0 2018-09-17 01:12 elastic-elastic_stack-6.2.4/.git/info
-rw-rw-r-- 1000/1000 240 2018-05-30 22:10 elastic-elastic_stack-6.2.4/.git/info/exclude
drwxrwxr-x 1000/1000 0 2018-09-17 01:12 elastic-elastic_stack-6.2.4/.git/logs
-rw-rw-r-- 1000/1000 5561 2018-09-17 01:11 elastic-elastic_stack-6.2.4/.git/logs/HEAD
This confuses r10k into endlessly and needlessly redeploying the module, especially when running r10k deploy environment -p
. Relates to #19. I believe a new release of elastic_stack should be cut that does not contain a .git
directory in order to fix this issue.
In case of a server not able to access the internet it is required to point the module to an internal mirror of the gpg key file, as it already is possble with the base_repo_url for packages.
Currently the priority parameter is set to 10 by default, which is one of the lowest options for Debian systems and therefore breaks the installation of elasticsearch installation if it is also available in the main repositories of e.g. Ubuntu 16.04.
The default should be undef.
It will submit a PR.
So,
I noticed on forge you have released 6.1.0 however there is no corresponding 6.1.0 tag/release in git. The elasticsearch module requires 6.1 - can you update one/the other/both for consistency?
Could you please update the Project URL on https://forge.puppet.com/elastic/elastic_stack which currently points to https://github.com/jarpy/puppet-elastic-stack and it doesn't exist?
Could you also fill the changelog as is very useful to see how this project proceeds, please?
Allow for a repo_url
parameter to be set and if defined then override the $_repo_url
variable.
This allows for alternative/private mirrors.
This module requires the puppet/yum
module however I can't see it used anywhere, unlike the apt module.
This statement Class['apt::update'] -> Package <| |>
[1] inserts a dependency in all packages defined not only by this module but for every other .pp file.
In many cases we want to install some packages before running the apt::update
but this module makes in impossible. One example would be the package 'apt-transport-https'.
My suggestion is to only add the dependency to the specific packages the modules depends on instead of using the "wildcard".
[1] https://github.com/elastic/puppet-elastic-stack/blob/master/manifests/repo.pp#L34
Puppet have released stdlib version 5. Ensure this module can use it and bless it in metadata.json
.
Hi folk! I have been tracking a bizarre issue with the elastic_stack module for a bit now. I run r10k every 5 minutes currently to update our environments, but I'd been noticing that elastic_stack::repo keeps not being located on catalog runs. Turns out it's because every 5 minutes the module is deleted and reinstalled via r10k. This does not happen with any other modules, but I also have no idea at all why it's happening. I wanted to bounce it off of y'all to see if you have any ideas. I looked around your code and I don't see a single thing that explains it. I have quite a few modules enabled (somewhere between 200 and 300), but here is a snippit with the elastic ones:
mod 'elastic/kibana', '6.3.0'
mod 'elastic/elastic_stack', '6.2.4'
mod 'elastic/elasticsearch', '6.3.3'
mod 'elastic/logstash', '6.1.4'
elastic_stack is the only one that continuously updates despite pointing at a specific version. For reference this is:
I switched to pointing directly at your git repository and that has resolved the issue for now.
There are way to many mentions of "elasticsearch". As this module should be a provider of the elastic repositories, there should any mentions (or even cross module variable access) to elasticsearch.
I will submit a PR.
After running this:
exec { 'puppet_module_install':
command => "puppet module install elastic/elastic_stack",
unless => "puppet module list | grep 'elastic/elastic_stack'",
path => ['/bin', '/usr/bin', '/opt/puppetlabs/bin']
}
class { 'elastic_stack::repo':
prerelease => false,
}
include elastic_stack::repo
I get:
Error: Evaluation Error: Error while evaluating a Resource Statement, Could not find declared class elastic_stack::repo (file: /......../Packs/App_ElasticStack.pp, line: 9, column: 1) on node elastistack
I then run another pack (not at the same time, I am running them one at a time) and get this error:
Error: Evaluation Error: Error while evaluating a Function Call, Could not find class ::mtulio::ssh for elastistack (file: /......../Packs/Config_SSH-Server_AllowRoot.pp, line: 7, column: 1) on node elastistack
Note that this file (SSH-Server) makes no reference to elastistack:
exec { 'puppet_module_install':
command => "puppet module install mtulio/ssh",
unless => "puppet module list | grep 'mtulio/ssh'",
path => ['/bin', '/usr/bin', '/opt/puppetlabs/bin']
}
include mtulio::ssh
class { '::ssh::sshd_config':
permitrootlogin => 'yes'
}
The puppet module list command does NOT list the elastistack command as being installed...
Manually running puppet module install elastic/elastic_stack command installs the module, but does not resolve the issue with other packs.
Uninstalling it also does not resolve the issue.
Puppet is effectively killed on this box now. Ideas?
This was a clean install (with a few other puppet packs installed) just to build out the elastic stack process.
Applying class 'elasticsearch' without specifying version couldn't add the RHEL repository giving this error
Warning: /etc/puppetlabs/code/environments/elasticprod/modules/elasticsearch/hiera.yaml: Use of 'hiera.yaml' version 4 is deprecated. It should be converted to version 5
(file: /etc/puppetlabs/code/environments/elasticprod/modules/elasticsearch/hiera.yaml)
Warning: Defining "data_provider": "hiera" in metadata.json is deprecated.
(file: /etc/puppetlabs/code/environments/elasticprod/modules/elasticsearch/metadata.json)
Error: Could not autoload puppet/type/yumrepo: undefined method `sensitive' for Puppet::Type::Yumrepo::Proxy_password:Class
Error: Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/yumrepo: undefined method `sensitive' for Puppet::Type::Yumrepo::Proxy_password:Class (file: /etc/puppetlabs/code/environments/elasticprod/modules/elastic_stack/manifests/repo.pp, line: 89, column: 7) on node gules.domain.com
Modules Installed:
── elastic-elastic_stack (v6.2.4)
├── elastic-elasticsearch (v6.3.3)
├── puppet-archive (v3.2.0)
├── puppet-yum (v3.1.0)
├── puppetlabs-apt (v6.0.0)
├── puppetlabs-concat (v4.2.1)
├── puppetlabs-java (v3.1.0)
├── puppetlabs-stdlib (v4.25.1)
├── puppetlabs-yumrepo_core (v1.0.1)
└── richardc-datacat (v0.6.2)
I have to disable the include apt
here since it conflicts with the other settings in my environment.
diff --git a/manifests/repo.pp b/manifests/repo.pp
index 774291c..c035860 100644
--- a/manifests/repo.pp
+++ b/manifests/repo.pp
@@ -66,7 +66,7 @@ class elastic_stack::repo (
case $::osfamily {
'Debian': {
- include apt
+ # include apt
apt::source { 'elastic':
ensure => 'present',
if ! defined (Class['apt']) {include apt}
didn't help either, since this one comes before my own settings.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.