example42 / hdm Goto Github PK
View Code? Open in Web Editor NEWHiera Data Manager
Hiera Data Manager
I am still wondering whether we can have a real footer (at the bottom of the page).
The text I added is placed directly underneath the latest data.
Offer users a selector, which data type they would like to store.
Example for pipe:
long_pipe: |
cluster.name: dev_cluster
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: %{facts.fqdn}
http.port: 9200
discovery.seed_hosts: ["localhost"]
cluster.initial_master_nodes: ["localhost"]
action.destructive_requires_name: true
xpack.security.enabled: true
Example for block:
long_block: >
ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh
jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y
l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd
/HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm
IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==]
When using puppetdb, all nodes from all environments are shown.
The list of nodes should only show the ones from the selected environment.
When choosing to remove a value, hdm shows OK, but value does not get removed.
rename master to old.
Which new default branch name should we use?
main?
Within a hiera hierarchy, one has the option to set the merge behavior using the key "lookup_options".
Example:
data/common.yaml
lookup_options:
<key>:
merge: <merge behavior>
convert_to: <Data Type> # Usually "Sensitive"
The following merge behavior options are possible:
The hiera key lookup_options should always be the very first in the list of keys (if set).
HDM should show the lookup_options providing a text message above the top hierarchy.
lookup option: merge: first, convert_to: <Data Type>
Note: lookup options can be set in every hiera hierarchy.
Updated test code is in mulitple_environmenrs.
Node sse8epsu.example42.training has lookup option for profile::auth::sshd_config_allowgroups set to 'first' and in common it is set to 'unique'.
Hiera uses deep merge on lookup_options.
e.g.
lookup_options:
<key1>:
merge: deep
<key2>:
merge: first
convert_to: 'Sensitive'
lookup_options:
<key1>:
merge: first
see DEVELOPMENT.md
After selecting a node, the list of all hiera keys is shown.
As this list might be very long: provide a filter field above the list of keys to liimit the keys shown.
It would be nice to have a description of what this is and what it does. Maybe a screenshot, if it has an user interface.
Example from common.yaml:
Prior editing:
hdm::boolean: true
Changing value to false creates a new entry:
hdm::boolean.json: false
The original key-value pair is still in common.yaml
At Puppetize PDX several attendees requested a read-only view on data.
Add a feature flag (set in hdm config file), which disables editing of values.
Check if test environments are used by travis CI tests!
Hiera has the option to set merge behavior via lookup_options. HDM should be able to parse these, too.
In large environment one can have several thousands of nodes.
Instead of showing a pulldown menu, also allow to write node names and filter within the pulldown list.
I did:
starting nginx, accessing website causes error.
Steps for reproduction:
Disabling http puppet redirect hiera data:
puppet_enterprise::profile::console::proxy::http_redirect::enable_http_redirect: false
selinux:
sealert -a /var/log/audit/audit.log
/sbin/restorecon -v /opt/example42/hdm/public/favicon.ico
ausearch -c 'nginx' --raw | audit2allow -M hdm-nginx
semodule -i hdm-nginx.pp
server {
listen 80 default_server;
server_name hdm.demo.example42.training;
# Tell Nginx and Passenger where your app's 'public' directory is
root /opt/example42/hdm/public;
# Turn on Passenger
passenger_enabled on;
passenger_ruby /opt/puppetlabs/puppet/bin/ruby;
passenger_friendly_error_pages on;
passenger_env_var HDM__CONFIG_DIR /etc/puppetlabs/code/environments;
passenger_env_var HDM__PUPPET_DB__ENABLED true;
passenger_env_var HDM__PUPPET_DB__SELF_SIGNED_CERT true;
passenger_env_var HDM__PUPPET_DB__TOKEN 0e-ZhnlpGdpEM9_Yh1zQzHef4BzQdvbnZSOQZrVWCW7A;
passenger_env_var HDM__PUPPET_DB__SERVER https://localhost:8081;
passenger_env_var PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/puppetlabs/puppet/bin;
}
Error message from passenger:
Error: The application encountered the following error: exit (SystemExit)
/opt/example42/hdm/vendor/ruby/2.5.0/gems/webpacker-4.0.2/lib/webpacker/railtie.rb:48:in `exit'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/webpacker-4.0.2/lib/webpacker/railtie.rb:48:in `block in <class:Engine>'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/initializable.rb:32:in `instance_exec'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/initializable.rb:32:in `run'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/initializable.rb:61:in `block in run_initializers'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:228:in `block in tsort_each'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:431:in `each_strongly_connected_component_from'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:349:in `block in each_strongly_connected_component'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:347:in `each'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:347:in `call'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:347:in `each_strongly_connected_component'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:226:in `tsort_each'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/tsort.rb:205:in `tsort_each'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/initializable.rb:60:in `run_initializers'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/application.rb:361:in `initialize!'
/opt/example42/hdm/config/environment.rb:5:in `<top (required)>'
config.ru:3:in `require_relative'
config.ru:3:in `block in <main>'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/builder.rb:55:in `instance_eval'
/opt/example42/hdm/vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/builder.rb:55:in `initialize'
config.ru:1:in `new'
config.ru:1:in `<main>'
/usr/share/passenger/helper-scripts/rack-preloader.rb:101:in `eval'
/usr/share/passenger/helper-scripts/rack-preloader.rb:101:in `preload_app'
/usr/share/passenger/helper-scripts/rack-preloader.rb:189:in `block in <module:App>'
/usr/share/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:380:in `run_block_and_record_step_progress'
/usr/share/passenger/helper-scripts/rack-preloader.rb:188:in `<module:App>'
/usr/share/passenger/helper-scripts/rack-preloader.rb:30:in `<module:PhusionPassenger>'
/usr/share/passenger/helper-scripts/rack-preloader.rb:29:in `<main>'
Prepared testing environment (eyaml).
Provide a config switch, if we want to decrypt values in hdm.conf.
EYaml is configured in hiera.yaml file, by setting the lookup_key to eyaml_lookup and specifying options for private and public key.
If the private key is readable by HDM: provide the option to "decrypt" the data.
Preparation for storing data:
If a user decrypts a value, sets a new one and wants to save it in plain text: provide a warning, and the user must confirm to store the value.
Please consider that we might have usermanagement RBAC where we want to limit users to decrypt values.
You are using nodejs 8. It is EOL since 2020-01-01.
See https://nodejs.org/en/about/releases/
Feature request: Create a page where to pass as parameter in the the name of an hiera key to analyse (for easy links wherever this key is shown) which shows what are the values for this key on all the nodes.
Ideally the list should be compressed to group nodes which have the same value and show information of the affected node (how many nodes have a given value) and possibly an option to expand them.
It would be nice, also, in the same page or another, to show where, for a given node, there's a duplicated and redundant entry which can be removed. For example where the same key with the same value is in common.yaml and in a node specific yaml, in this case the node specific entry is redundant and there should be a button to quickly remove it.
HDM running against a new PE 2019.2.2 installation, with issue #9 resolved by PR #10.
Steps to reproduce:
puppet_enterprise::master::puppetserver::jruby_max_active_instances: 1
in $codedir/environments/production/data/common.yaml
.puppet_enterprise::master::puppetserver::jruby_max_active_instances
as the key.Expected behavior:
HDM displays its value (1
in the case of this lab system).
Observed behavior:
Error:
undefined method `gsub!' for nil:NilClass
Full function:
def content_for_key(key)
return nil unless content
return nil unless content.has_key?(key)
value = content[key]
return "true" if value == true
return "false" if value == false
return value if value.is_a?(String)
value_string = value.to_yaml
value_string.gsub!("---\n", '').gsub!(/^$\n/, '')
value_string
end
Flagged line is:
value_string.gsub!("---\n", '').gsub!(/^$\n/, '')
Rails.root:
/root/hdm
App trace:
app/models/hiera/read_file.rb:37:in `content_for_key'
app/models/hiera.rb:55:in `block (2 levels) in search_key'
app/models/hiera.rb:48:in `map'
app/models/hiera.rb:48:in `block in search_key'
app/models/hiera.rb:46:in `each_pair'
app/models/hiera.rb:46:in `search_key'
app/concepts/keys/operation/show.rb:28:in `node_data!'
Request parameters:
{"on"=>:member, "environment_id"=>"production", "node_id"=>"master.test.vlan", "id"=>"puppet_enterprise::master::puppetserver::jruby_max_active_instances"}
--
Web console output:
>> value
=> 1
>> value_string
=> "--- 1\n"
value_string
is the result of value.to_yaml
. If the value were a String instead of an Integer, return value if value.is_a?(String)
would return before this.
The behavior of Integer.to_yaml
:
>> 1.to_yaml
=> "--- 1\n"
>> 10.to_yaml
=> "--- 10\n"
>> "1".to_yaml
=> "--- '1'\n"
Since "--- 1"
is apparently expected from 1.to_yaml
, then:
>> value_string.gsub!("---\n", '').gsub!(/^$\n/, '')
NoMethodError: undefined method `gsub!' for nil:NilClass
from /root/hdm/app/models/hiera/read_file.rb:37:in `content_for_key'
"---\n"
matches nothing in "--- 1"
. gsub!
returns nil
if there's no match, so nil
is also expected. We could convert the first result to a String/.to_s
, but it feels like the integer value would still be lost unless the match itself were changed.
For reference:
>> content
=> {"puppet_enterprise::profile::amq::broker::heap_mb"=>"96", "puppet_enterprise::master::puppetserver::jruby_max_active_instances"=>1, "puppet_enterprise::master::puppetserver::reserved_code_cache"=>"96m", "puppet_enterprise::profile::master::java_args"=>{"Xmx"=>"384m", "Xms"=>"128m", "XX:MaxPermSize"=>"=96m", "XX:PermSize"=>"=64m", "XX:+UseG1GC"=>""}, "puppet_enterprise::profile::puppetdb::java_args"=>{"Xmx"=>"128m", "Xms"=>"64m", "XX:MaxPermSize"=>"=96m", "XX:PermSize"=>"=64m", "XX:+UseG1GC"=>""}, "puppet_enterprise::puppetdb::read_maximum_pool_size"=>4, "puppet_enterprise::puppetdb::write_maximum_pool_size"=>2, "puppet_enterprise::profile::console::java_args"=>{"Xmx"=>"128m", "Xms"=>"64m", "XX:MaxPermSize"=>"=96m", "XX:PermSize"=>"=64m", "XX:+UseG1GC"=>""}, "puppet_enterprise::trapperkeeper::database_settings::activity::maximum_pool_size"=>2, "puppet_enterprise::trapperkeeper::database_settings::classifier::maximum_pool_size"=>2, "puppet_enterprise::trapperkeeper::database_settings::rbac::maximum_pool_size"=>2, "puppet_enterprise::profile::console::delayed_job_workers"=>1, "puppet_enterprise::profile::database::shared_buffers"=>"4MB", "puppet_enterprise::profile::orchestrator::java_args"=>{"Xmx"=>"128m", "Xms"=>"128m", "XX:+UseG1GC"=>""}}
There are several integer values here, and they would be common in many PE installs. I'd expect any of them to fail.
Attempting to return an Integer value prevents the error when attempting to display the key's value when selecting it in HDM:
--- a/app/models/hiera/read_file.rb
+++ b/app/models/hiera/read_file.rb
@@ -31,7 +31,7 @@ class Hiera
value = content[key]
return "true" if value == true
return "false" if value == false
- return value if value.is_a?(String)
+ return value if value.is_a?(String) || value.is_a?(Integer)
But after making this change, then editing the selected value in HDM's interface (for instance from 1
to 2
in this example), HDM appends the new value to a new key in data/common.yaml
with the same name but .json
suffixed, instead of overwriting the existing key's value.
# grep -R max_active_instances $(puppet config print codedir)
...
/etc/puppetlabs/code/environments/production/data/common.yaml:puppet_enterprise::master::puppetserver::jruby_max_active_instances: 1
/etc/puppetlabs/code/environments/production/data/common.yaml:puppet_enterprise::master::puppetserver::jruby_max_active_instances.json: 2
HDM is configurable by setting environmetn variables. We want HDM to be configurable via a config file.
/etc/hdm/hdm.conf (ini-File or YAML)
At the moment HDM uses a default webtheme. We want to change layout to be compiant with example42 color scheme.
Only a subset of the hierarchies is shown.
see fixtures multiple_hierarchies test data.
HDM in read/write mode must be able to store the new data. Data path must be in hiera yaml file and in hdm config file.
Any reason why we still need this directory?
I suppose that this is a leftover from test data when building prototype.
Webinterface shows "Oops, something went wrong"
Changed data are written back to file.
Application throws the following error message:
NoMethodError (undefined method `persisted?' for nil:NilClass):
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/polymorphic_routes.rb:266:in `handle_model'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/polymorphic_routes.rb:280:in `handle_model_call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/url_for.rb:190:in `full_url_for'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/url_for.rb:169:in `url_for'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/redirecting.rb:110:in `_compute_redirect_to_location'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/redirecting.rb:63:in `redirect_to'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/flash.rb:58:in `redirect_to'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/instrumentation.rb:66:in `block in redirect_to'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `block in instrument'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `instrument'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/instrumentation.rb:65:in `redirect_to'
vendor/ruby/2.5.0/gems/turbolinks-5.2.0/lib/turbolinks/redirection.rb:12:in `redirect_to'
vendor/ruby/2.5.0/gems/reativo-0.1.5/app/controllers/reativo/crud_controller.rb:101:in `block (2 levels) in update'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/mime_responds.rb:203:in `respond_to'
vendor/ruby/2.5.0/gems/reativo-0.1.5/app/controllers/reativo/crud_controller.rb:99:in `update'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/abstract_controller/base.rb:194:in `process_action'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/rendering.rb:30:in `process_action'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
vendor/ruby/2.5.0/gems/react-rails-2.5.0/lib/react/rails/controller_lifecycle.rb:31:in `use_react_component_helper'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:136:in `run_callbacks'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/abstract_controller/callbacks.rb:41:in `process_action'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/rescue.rb:22:in `process_action'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `block in instrument'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `instrument'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
vendor/ruby/2.5.0/gems/activerecord-5.2.3/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/abstract_controller/base.rb:134:in `process'
vendor/ruby/2.5.0/gems/actionview-5.2.3/lib/action_view/rendering.rb:32:in `process'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal.rb:191:in `dispatch'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_controller/metal.rb:252:in `dispatch'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/route_set.rb:34:in `serve'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/journey/router.rb:52:in `block in serve'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/journey/router.rb:35:in `each'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/journey/router.rb:35:in `serve'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/route_set.rb:840:in `call'
vendor/ruby/2.5.0/gems/warden-1.2.8/lib/warden/manager.rb:36:in `block in call'
vendor/ruby/2.5.0/gems/warden-1.2.8/lib/warden/manager.rb:34:in `catch'
vendor/ruby/2.5.0/gems/warden-1.2.8/lib/warden/manager.rb:34:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/tempfile_reaper.rb:15:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/etag.rb:25:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/conditional_get.rb:38:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/head.rb:12:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/session/abstract/id.rb:259:in `context'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/session/abstract/id.rb:253:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/cookies.rb:670:in `call'
vendor/ruby/2.5.0/gems/activerecord-5.2.3/lib/active_record/migration.rb:559:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:98:in `run_callbacks'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/executor.rb:14:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
vendor/ruby/2.5.0/gems/web-console-3.7.0/lib/web_console/middleware.rb:135:in `call_app'
vendor/ruby/2.5.0/gems/web-console-3.7.0/lib/web_console/middleware.rb:22:in `block in call'
vendor/ruby/2.5.0/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `catch'
vendor/ruby/2.5.0/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/rack/logger.rb:38:in `call_app'
vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/rack/logger.rb:26:in `block in call'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:71:in `block in tagged'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:28:in `tagged'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:71:in `tagged'
vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/rack/logger.rb:26:in `call'
vendor/ruby/2.5.0/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/request_id.rb:27:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/method_override.rb:22:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/runtime.rb:22:in `call'
vendor/ruby/2.5.0/gems/activesupport-5.2.3/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/executor.rb:14:in `call'
vendor/ruby/2.5.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/static.rb:127:in `call'
vendor/ruby/2.5.0/gems/rack-2.0.8/lib/rack/sendfile.rb:111:in `call'
vendor/ruby/2.5.0/gems/webpacker-4.0.2/lib/webpacker/dev_server_proxy.rb:29:in `perform_request'
vendor/ruby/2.5.0/gems/rack-proxy-0.6.5/lib/rack/proxy.rb:57:in `call'
vendor/ruby/2.5.0/gems/railties-5.2.3/lib/rails/engine.rb:524:in `call'
vendor/ruby/2.5.0/gems/puma-3.12.2/lib/puma/configuration.rb:227:in `call'
vendor/ruby/2.5.0/gems/puma-3.12.2/lib/puma/server.rb:674:in `handle_request'
vendor/ruby/2.5.0/gems/puma-3.12.2/lib/puma/server.rb:476:in `process_client'
vendor/ruby/2.5.0/gems/puma-3.12.2/lib/puma/server.rb:334:in `block in run'
vendor/ruby/2.5.0/gems/puma-3.12.2/lib/puma/thread_pool.rb:135:in `block in spawn_thread'
At the moment there is no login in front of hdm. We want to be able to manage users (locally)
Do you think it is possible to run this project in docker?
@alvagante David Sandilands asked if this repo can be archived and maybe redirect to betadots hdm.
betadots/hdm#105
Data: test/fixtures/files/puppet/environments/
Nodes and Facts: test/fixtures/files/puppet/nodes/
available file and data present: bold
file available, no data inside: normal
file unavailable: italic
The following should not be part of the git repo.
This information must be added to a document describing the way how to package HDM.
Rails uses different environments. At the moment we use development.
For packaging, we need production environment.
Default data sets are stored in a config yaml file.
e.g.
hdm_user_data.yaml
hash:
key: value
key2: value
array:
- el1
- el2
boolean: true
integer: 10
float: 1.1
string: 'a string'
``´
a hiera.yaml file is fully optional.
If no file is available, hiera uses internal defaults
HDM expects hiera data to be in environmentpath/data
But data can be set using hiera.yaml file:
---
version: 5
defaults:
datadir: modules/hieradata/data
Defaults are in hiera ruby gem:
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/hiera/config.rb
class Hiera::Config
class << self
##
# load takes a string or hash as input, strings are treated as filenames
# hashes are stored as data that would have been in the config file
#
# Unless specified it will only use YAML as backend with a
# hierarchy of 'nodes/%{::trusted.certname}' and 'common', and with a
# console logger.
#
# @return [Hash] representing the configuration.
def load(source)
@config = {:backends => ["yaml"],
:hierarchy => ["nodes/%{::trusted.certname}", "common"],
:merge_behavior => :native }
if source.is_a?(String)
if File.exist?(source)
config = begin
yaml_load_file(source)
rescue TypeError => detail
case detail.message
when /no implicit conversion from nil to integer/
false
To verify with a Rails expert:
Should file config/credentials.yml.enc be kept in the repo or removed and added to .gitignore?
It can be regenerated with rails:credentials:edit
Maybe describe in setup instructions or via the Puppet module.
HDM running against a new PE 2019.2.2 installation.
Steps to reproduce:
production
for "Choose the environment".Expected behavior:
The default PE keys for that node appear.
Observed behavior:
Error:
undefined method `has_key?' for nil:NilClass
Extracted source reported:
def default_yaml_data?
defaults.has_key?("data_hash") && defaults["data_hash"] == "yaml_data"
end
Rails.root:
/root/hdm
App trace:
app/models/hiera/config_file.rb:15:in `default_yaml_data?'
app/models/hiera/config_file.rb:25:in `block in initialize_environments'
app/models/hiera/config_file.rb:24:in `map'
app/models/hiera/config_file.rb:24:in `initialize_environments'
app/models/hiera/config_file.rb:7:in `initialize'
app/models/hiera.rb:85:in `new'
app/models/hiera.rb:85:in `config_file'
app/models/hiera.rb:14:in `paths'
app/models/hiera.rb:27:in `all_keys'
app/concepts/nodes/operation/show.rb:21:in `node_data!'
Request parameters:
{"on"=>:member, "environment_id"=>"production", "id"=>"master.test.vlan"}
--
defaults
is defined in Hiera::ConfigFile as:
def defaults
content["defaults"]
end
Ruby web console inspection of content
after the error:
>> content
=> {"version"=>5, "defaults"=>nil, "hierarchy"=>[{"name"=>"Per-node data (yaml version)", "path"=>"nodes/%{::trusted.certname}.yaml"}, {"name"=>"Other YAML hierarchy levels", "paths"=>["common.yaml"]}]}
content
is defined in app/models/hiera.rb
:
def content
@content ||= YAML.load(
File.read(Pathname.new(Settings.config_dir).join(environment, "hiera.yaml"))
)
end
It reads the environment's hiera.yaml
:
@content ||= YAML.load(
File.read(Pathname.new(Settings.config_dir).join(environment, "hiera.yaml"))
)
./code/environments/production/hiera.yaml
with comments removed:
---
version: 5
defaults:
hierarchy:
- name: "Per-node data (yaml version)"
path: "nodes/%{::trusted.certname}.yaml"
- name: "Other YAML hierarchy levels"
paths:
- "common.yaml"
defaults
has no value, so nil
is expected.
content
is consumed in initialize_environments
:
def initialize_environments
return @environments = [] unless content.has_key?("hierarchy")
@environments = content['hierarchy'].map do |hierarchy|
default_data_hash = default_yaml_data? ? 'yaml_data' : nil
Hierarchy.new(hierarchy: hierarchy, default_data_hash: default_data_hash)
end
end
So if default_yaml_data
is truthy, return 'yaml_data'
. Otherwise, return nil
.
The problem appears to be that def default_yaml_data?
attempts to use has_key
on a nil
-value defaults
and fails. def defaults
results in nil
if content["defaults"]
is nil
, as it is in the above hiera.yaml
.
Therefore, def default_yaml_data?
should at least check if defaults
itself is truthy/exists before attempting to use methods on it.
def default_yaml_data?
defaults && defaults.has_key?("data_hash") && defaults["data_hash"] == "yaml_data"
end
Configure the following options:
Tests check for files with absolute path from individual location:
/Users/fernandes/src/hdm/puppet
Please add all missing test files.
Needed to add travis-ci.
Provide a documentation on how to test against a real puppetdb using psick/vagrant setup
Restore layout from master branch in rails6 branch
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.