aspiers / ansible-viz Goto Github PK

Not yet sure why this happens.

CodeClimate or similar

Currently it relies on being run in place.

938a490 added tracking of originating file pathnames for each component. References (such as includes) from one component to another, such as ../../../roles/myrole/tasks/mytask.yml are currently handled via quite specific regexps, whereas we could just calculate the full path, clean it up via Pathname.cleanpath, and perform a reverse lookup (doesn't exist currently) to find the object. This would allow even crazy stuff like ../../../roles/myrole1/../myrole2 to work.

Currently this barfs, because the scoper doesn't pick up those roles early enough, so scoping other roles which depend on them will fail.

Workaround: just touch an empty task file in the role.

It would be so much easier to use that way.

Currently classes are used more or less as libraries rather than in a true OO fashion, e.g.

• State is passed everywhere via parameters (e.g. dict and g), rather than being stored in instances.
• The instances have little or no state, and yet the methods are mostly instance methods not class methods.
• There is no relationship modelled between classes, whereas there is ample opportunity for composition via dependency injection.
• Core objects such as playbooks, roles, tasks, templates, variables etc. are modelled as Hashes rather than as objects. Commonality between the objects (e.g. name, fqn, parent, originating file path) is not enforced in any rigorous way - this is crying out for inheritance.

I followed https://github.com/ArdanaCLM/ardana-wiki/wiki/Ansible-call-graph#usage (but with ruby2.5 from openSUSE Tumbleweed) and got:

sudo zypper in ruby2.5-rubygem-bundler
bundle --path suse/bundle
bundle exec ruby ansible-viz.rb ~/devel/suse-provo/ardana/ardana-dev-tools/ansible
[...]
deployer::pre-init-deployer
deployer::_upload_iso_to_deployer
deployer::build
vagrant::validate
dev-env::validate
- no scope for libvirt yet
- no scope for product-ardana yet
= set full scope for product-ardana from product-ardana::main
- no scope for pip-consumer yet
- no scope for ccache yet
- no scope for dev-env yet
Traceback (most recent call last):
	8: from ansible-viz.rb:135:in `<main>'
	7: from ansible-viz.rb:97:in `render'
	6: from /home/tom/devel/ansible-viz/scoper.rb:38:in `process'
	5: from /home/tom/devel/ansible-viz/scoper.rb:38:in `each'
	4: from /home/tom/devel/ansible-viz/scoper.rb:44:in `block in process'
	3: from /home/tom/devel/ansible-viz/scoper.rb:146:in `calc_scope'
	2: from /home/tom/devel/ansible-viz/scoper.rb:146:in `flat_map'
	1: from /home/tom/devel/ansible-viz/scoper.rb:146:in `each'
/home/tom/devel/ansible-viz/scoper.rb:148:in `block in calc_scope': dependency product of role dev-env is missing scope (RuntimeError)

Roles that get pulled in via the new import_role don't show up in the visualization

Currently you can only run the whole test suite via test-viz.rb, and all the separate test files are loaded via require. This is likely not the best way to do it, and probably prevents things like #4. Also, rspec would probably be a better choice then minitest.

This shouldn't be possible but is clearly visible via puts debug statement.

There are still 2 tests failing.

Run options:

# Running tests:

[33/35] TC_Scoper#test_order_tasks = 0.00 s
  1) Failure:
TC_Scoper#test_order_tasks [/home/adam/.GIT/3rd-party/ansible-viz/test-scoper.rb:26]:
<["taskB", "main", "taskA", "task2", "main", "task1"]> expected but was
<["taskB", "taskA", "task1", "task2", "main", "main"]>.

[34/35] TC_Scoper#test_scope = 0.01 s
  2) Failure:
TC_Scoper#test_scope [/home/adam/.GIT/3rd-party/ansible-viz/test-scoper.rb:46]:
roleA taskA: missing/extra items.
<[[], []]> expected but was
<[["factB", "meow", "varAmaininc", "factAmain", "factB"], []]>.

Finished tests in 0.093157s, 375.7111 tests/s, 944.6451 assertions/s.
35 tests, 88 assertions, 2 failures, 0 errors, 6 skips

ruby -v: ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-linux]
Coverage report generated for Unit Tests to /home/adam/.GIT/3rd-party/ansible-viz/coverage. 633 / 874 LOC (72.43%) covered.

When processing the first playbook at hands, ansible-viz runs into a No MethodError at some point.

Loading ...
Postprocessing ...
Traceback (most recent call last):
	11: from bin/ansible-viz.rb:17:in `<main>'
	10: from bin/ansible-viz.rb:10:in `main'
	9: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/cli.rb:96:in `graph_from_data'
	8: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:15:in `process'
	7: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:15:in `each'
	6: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:15:in `block in process'
	5: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:26:in `process_role'
	4: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:26:in `each'
	3: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:28:in `block in process_role'
	2: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:28:in `each'
	1: from /home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:28:in `block (2 levels) in process_role'
/home/yala/src/github.com/aspiers/ansible-viz/lib/ansible_viz/postprocessor.rb:36:in `process_vars': undefined method `each' for #<String:0x0000000001056530> (NoMethodError)

How would I debug this further?

Similar to #22, if the playbook directory has an ansible.cfg file within it that specifies a roles_path variable, it will override the system ansible.cfg definition. As such, this should read that directory for roles.

Note: this is all assuming you don't just ls the directories and assume all of them are required.

Need to use Guard for TDD red/green cycles. Probably blocked by #2.

Following the README.md "Run:" instructions I get the following:

[user@fedajh ansible-viz]$ gem install bundle
Fetching: bundle-0.0.1.gem (100%)
Successfully installed bundle-0.0.1
Parsing documentation for bundle-0.0.1
Installing ri documentation for bundle-0.0.1
Done installing documentation for bundle after 0 seconds
1 gem installed
[user@fedajh ansible-viz]$ bundle install
Traceback (most recent call last):
	2: from /home/ahuffman/bin/bundle:23:in `<main>'
	1: from /usr/share/rubygems/rubygems.rb:308:in `activate_bin_path'
/usr/share/rubygems/rubygems.rb:289:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executable bundle (Gem::GemNotFoundException)

Some more info:

[user@fedajh ansible-viz]$ cat /etc/redhat-release
Fedora release 28 (Twenty Eight)

Maybe blocked by #2 ?

There are other places for roles to live other than <playbook>/roles, and as such this should probably not produce a RuntimeError:

$ bundle exec ruby bin/ansible-viz.rb ~/ansible_dev/<playbook>
Loading ...
Traceback (most recent call last):
        3: from bin/ansible-viz.rb:17:in `<main>'
        2: from bin/ansible-viz.rb:9:in `main'
        1: from /Users/<user>/ansible_dev/ansible-viz/lib/ansible_viz/loader.rb:59:in `load_dir'
/Users/<user>/ansible_dev/ansible-viz/lib/ansible_viz/loader.rb:36:in `ls': No such directory: /Users/<user>/ansible_dev/<playbook>/roles (RuntimeError)

I'll put in corresponding other PRs related to this:

• #22
• #23

Need a common way to output debug info (and possibly other stuff), which doesn't reinvent wheels.

version:
Ubuntu 18.04.1 LTS

1. Gemfile.lock
   $ sudo bundle install
   Traceback (most recent call last):
   2: from /usr/local/bin/bundle:23:in <main>' 1: from /usr/lib/ruby/2.5.0/rubygems.rb:308:in activate_bin_path'
   /usr/lib/ruby/2.5.0/rubygems.rb:289:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executable bundle (Gem::GemNotFoundException)

Solution:
I had to remove it.

2. ffi dependency
   $ sudo bundle install
   Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.
   Fetching gem metadata from https://rubygems.org/.............
   Fetching gem metadata from https://rubygems.org/.
   Resolving dependencies...
   Fetching rake 12.3.2
   Installing rake 12.3.2
   Fetching ansi 1.5.0
   Installing ansi 1.5.0
   Fetching mustache 0.99.8
   Installing mustache 0.99.8
   Fetching word_wrap 1.0.0
   Installing word_wrap 1.0.0
   Using ansible-viz 0.1.0 from source at .
   Fetching builder 3.2.3
   Installing builder 3.2.3
   Using bundler 2.0.1
   Fetching coderay 1.1.2
   Installing coderay 1.1.2
   Fetching diff-lcs 1.3
   Installing diff-lcs 1.3
   Fetching docile 1.3.1
   Installing docile 1.3.1
   Fetching ffi 1.10.0
   Installing ffi 1.10.0 with native extensions
   Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

   current directory: /var/lib/gems/2.5.0/gems/ffi-1.10.0/ext/ffi_c
   /usr/bin/ruby2.5 -r ./siteconf20190318-3639-ryqexi.rb extconf.rb
   mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h

extconf failed, exit code 1

Gem files will remain installed in /var/lib/gems/2.5.0/gems/ffi-1.10.0 for inspection.
Results logged to /var/lib/gems/2.5.0/extensions/x86_64-linux/2.5.0/ffi-1.10.0/gem_make.out

An error occurred while installing ffi (1.10.0), and Bundler cannot continue.
Make sure that gem install ffi -v '1.10.0' --source 'https://rubygems.org/' succeeds before bundling.

In Gemfile:
guard-rspec was resolved to 4.7.3, which depends on
guard was resolved to 2.15.0, which depends on
listen was resolved to 3.1.5, which depends on
rb-inotify was resolved to 0.10.0, which depends on
ffi

Solution:
$ sudo apt install ruby-dev

Currently this only looks at one place: <playbook>/roles. This should be more intelligent and also look at the entire roles directory list: http://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#role-search-path (note, only one extra is required right now).

But as an extension, should probably look at the global ansible.cfg file to figure out which other places to look defined in roles_path.

we have a huge number of roles. therefore we organized our roles in subdirectories

we have for example

roles/third-party-services/grafana

executing ansible-viz causes the warning
WARNING: Couldn't find role 'third-party-services/grafana' (invoked by playbook 'tools')

and the dependency is not created properly in the output graph.