• Configure the CM repo to install the latest 5.9 release on on the second node listed before
  • List the command and output of ls /etc/yum.repos.d in challenges/labs/2_cm.md
  • Copy the cloudera-manager.repo file to challenges/labs/2_cloudera-manager.repo.md
• Use the scm_prepare_database.sh script to write your db.properties file, list the full command line in 2_cm.md
• Start the Cloudera Manager server. Then in challenges/labs/2_db.properties.md:
  • Add the first line from your server log
  • Add the log line that contains the phrase "Started Jetty server"
  • Copy your db.properties file to challenges/labs/2_db.properties.md

YARN/RM Lab

• Doing the Math
• Static Service Pools
• Tuning for YARN

CM Lab

• Monitoring Lab
• Create a Custom Dashboard
• Use the API
• Upgrade Cloudera Manager

• Install the latest CDH release; deploy Coreset services only
• Rename your cluster after your GitHub handle
• Create user directories in HDFS for neymar and ronaldo
• Add the following to 3_cm.md:
  • Command and output for hdfs dfs -ls /user
  • The output from the CM API call ../api/v14/hosts
• Login to Hue and install the Hive sample data
  • Capture a Hue screen that lists the Hive tables to challenges/labs/3_hue_hive.png

HDFS Lab tasks

• Replicate to another cluster
• Test HDFS performance
• Test HDFS Snapshot
• Enable HDFS HA

• Create the file challenges/labs/0_setup.md with:
  • List the cloud provider you are using (AWS, GCE, Azure, other)
  • List the Linux release you have chosen
  • Show that the disk space on each node is at least 30 GB
  • List the command and output for yum repolist enabled
• Add the following Linux accounts to all nodes
  • User neymar with a UID of 2010
  • User ronaldo with a UID of 2016
• Create the group barca and add ronaldo to it
• Create the group merengues and add neymar to it
• List the /etc/passwd entries for neymar and ronaldo in your setup file
• List the /etc/group entries for barca and merengues in your setup file

• Install a MySQL 5.6.x server on the node you listed first
• Copy /etc/yum.repos.d/mysql-community.repo to challenges/labs/1_my-database-server.repo.md
• Install the MySQL client package and JDBC connector jar on all clients
• Start the mysqld service
• Create the following databases
  • scm
  • rman
  • hive
  • oozie
  • hue
  • sentry
• Put the following in the file challenges/labs/1_db-server.md
  • The hostname of your MySQL node
  • The command and output for mysql --version
  • The command and output for listing MySQL databases

overview/intro.md contains syntax errors over the href tags in Course Objectives and Course Plan section.

CM Install Lab

• System Configuration Checks
• Configure MySQL with a replica server
• CM/CDH Install Lab
• Create a local parcel repo (manual) (BONUS)

• As user neymar, use teragen to generate a 65,536,000-record dataset into eight files
  • Set the block size to 16 MB
  • Name the target directory tgen640
  • Use the time command to capture job duration
• Put the following in the file challenges/labs/4_teragen.md
  • The full teragen command
  • The output of the time command
  • The command and output of hdfs dfs -ls /user/neymar/tgen640
  • Show how many blocks are associated with this directory

• Authenticate using Linux users/groups
• Security or Availability?

CDH Security

• Implement TLS Level 1 Security
• Security Labs Preparation
• Integrating Kerberos with Cloudera Manager
• Sentry Lab

• Update Cloudera Manager to the latest available release
  • Capture the command used to determine the API version available, along with the output.
  • Store them in the file 6_cm_latest.png
• Upgrade CDH to the latest available release
  • Capture the CDH version and the services that are running
  • Store it in the file 6_cdh_latest.png

Based on the labs and challenge work submitted, I am pleased to offer you a Full Pass. Congratulations!

You will receive an email notification when all the work submitted for the Madrid class has been processed.

Thank you for your hard work and diligence throughout the class.

• Install an MIT KDC on the same node as the CM server
  • Name your realm after your GitHub handle
  • Use ES as a suffix
  • For example: MFERNEST.ES
• Create Kerberos principals for neymar, ronaldo, and cloudera-scm
  • Give cloudera-scm the privileges needed to create principals and keytabs
• Enable Kerberos for the cluster
• Run the terasort program as neymar using /user/neymar/tsort640m
  • Copy the command and output to challenges/labs/5_terasort.md
• Run the Hadoop pi program as the user ronaldo
  • Copy the command and output to challenges/labs/5_pi.md
• Copy only text files in /var/kerberos/krb5kdc/ to your repo as follows:
  • Add the prefix 5_ and the suffix .md
  • Example: 5_kdc.conf.md

Your setup looks good so far. If you have not already invited rsiwicki as a Collaborator, please do so.

Once he has accepted, you may remove the incomplete label from this Issue and close it.

Good luck this week!