azavea / ansible-hdfs Goto Github PK

I did set up a small hdfs-cluster for playing. I got it working so far, but no matter what i try, *_properties entries will lead to playbook errors. Can you give me a hint whats wrong?

error message is

RUNNING HANDLER [azavea.hdfs : Restart Hadoop HDFS NameNode] *******************
fatal: [odysseus]: FAILED! => {"changed": false, "failed": true, "msg": "stopping namenode\n * Stopped Hadoop namenode: \nstarting namenode, logging to /var/log/hadoop-hdfs/
hadoop-hdfs-namenode-odysseus.out\n * Failed to start Hadoop namenode. Return value: 1\n"}

The (probably relevant) expert for the task in the site.yml for the master follows. I cropped away the spark stuff...

- name: set up cluster master                                        
  hosts: spark-masters                                               
  vars:                                                              
    java_version: "8u77**"                                           
    java_flavor: "oracle"                                            
    java_major_version: "8"                                          
    java_oracle_accept_license_agreement: "True"                     
    hdfs_namenode: "True"                                            
    hdfs_namenode_host: odysseus                                     
#    hdfs_core_properties:                                           
#      - { name: "hadoop.proxyuser.hue.hosts", value: "*" }          # doesnt work
#      - { name: "hadoop.proxyuser.hue.groups", value: "*" }                   # doesnt work either
    hdfs_namenode_properties:                                        
      - { name: "dfs.permissions.enabled", value: "false" }                    # the culprit
#      - { name: "dfs.webhdfs.enabled", value: "true" }                        # same here
  roles:                                                             
    - azavea.java                                                    
    - azavea.hdfs                                                    

  pre_tasks:                                                         
    - name: Update APT cache                                         
      apt: update_cache=yes                                          

It would be nice to see how to use this role to setup multiple nodes.

Exposing all options to the Ansible mount module through hdfs_disks may allow us to choose all of the same options cloud-init does when mounting volumes via user data, so that mount tasks can actually be idempotent.

Using code from the right branch:

~/dev/ansible-hdfs/examples> git remote -v
origin https://github.com/azavea/ansible-hdfs.git (fetch)
origin https://github.com/azavea/ansible-hdfs.git (push)

And it is up-to-date

~/dev/ansible-hdfs/examples> git status
On branch develop
Your branch is up-to-date with 'origin/develop'.
nothing to commit, working directory clean

vagrant up breaks

~/dev/ansible-hdfs/examples> vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ubuntu/trusty64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'ubuntu/trusty64' is up to date...
==> default: Setting the name of the VM: examples_default_1425502066892_3479
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
==> default: Forwarding ports...
default: 50070 => 50070 (adapter 1)
default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
default: Warning: Connection timeout. Retrying...
default: Warning: Remote connection disconnect. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
default: /vagrant => /Users/alexanderjardim/dev/ansible-hdfs/examples
==> default: Running provisioner: ansible...

PLAY [all] ********************************************************************

GATHERING FACTS ***************************************************************
ok: [default]

TASK: [Update APT cache] ******************************************************
ok: [default]

TASK: [azavea.java | Install OpenJDK JRE (headless)] **************************
changed: [default]

TASK: [azavea.java | Install OpenJDK JRE] *************************************
changed: [default]

TASK: [azavea.java | Install OpenJDK] *****************************************
changed: [default]

TASK: [azavea.java | Set OpenJDK as the default] ******************************
ok: [default]

TASK: [azavea.java | Configure the WebUpd8 repository] ************************
skipping: [default]

TASK: [azavea.java | Accept Oracle license] ***********************************
skipping: [default]

TASK: [azavea.java | Install Oracle Java] *************************************
skipping: [default]

TASK: [azavea.java | Set Oracle Java as the default] **************************
skipping: [default]

TASK: [azavea.hdfs | Configure Cloudera APT key] ******************************
changed: [default]

TASK: [azavea.hdfs | Configure the Cloudera APT repositories] *****************
changed: [default]

TASK: [azavea.hdfs | Pin Cloudera APT repositories] ***************************
changed: [default]

TASK: [azavea.hdfs | Install Hadoop HDFS NameNode] ****************************
failed: [default] => {"failed": true}
msg: pkgname wildcard and version can not be mixed

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
to retry, use: --limit @/Users/alexanderjardim/site.retry

default : ok=9 changed=6 unreachable=0 failed=1

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.