asdaraujo / edge2ai-workshop Goto Github PK

View Code? Open in Web Editor NEW

68.0 68.0 102.0 96.01 MB

Edge2AI Workshop

License: Apache License 2.0

Python 53.03% Shell 38.60% HCL 5.65% HTML 1.37% Dockerfile 0.45% Scala 0.83% JavaScript 0.06%

edge2ai-workshop's People

Contributors

Stargazers

Watchers

edge2ai-workshop's Issues

az "a" doesn't work sometimes

It seem "a" is used to describe which az subnet will be set up in.
https://github.com/asdaraujo/edge2ai-workshop/blob/master/setup/terraform/network.tf#L66

But I hit an Error told me some instance types are not supported in the "xxxx-a" az.
Can I add an optional Env var to describe postfix of az which user perfer to and use it for subnet setup?

If OK, I'll do the modification

ExecuteProcess screenshot

Hi there Andre,

As mentioned in TKO, Lab 1 has us create an ExecuteProcess processor, but then that is lost during the remaining screenshots (yet is critical to the lab methodology)

Kind Regards,

Mike

Text copy-pastable configuration for LAB 6

It would be great to have a copy-paste version of all the SRM replication for LAB 6:
replication.factor=1
heartbeats.topic.replication.factor=1
checkpoints.topic.replication.factor=1
offset-syncs.topic.replication.factor=1
offset.storage.replication.factor=1
config.storage.replication.factor=1
status.storage.replication.factor=1

I made the mistake of incorrectly typing from the screenshot and it took me a long time to figure out the error.

The link for"kudu-spark2_2.11-1.9.0.jar" was unresolved

I searched from web, and got it at "https://repo1.maven.org/maven2/org/apache/kudu/kudu-spark2_2.11/1.9.0/kudu-spark2_2.11-1.9.0.jar".
The doc has unresolved link "http://central.maven.org/maven2/org/apache/kudu/kudu-spark2_2.11/1.9.0/kudu-spark2_2.11-1.9.0.jar".
Maybe it's a temporary outage. Then, just ignore this issue.

Flink link in the web UI is not correct

It points to a generic one and won't show you the current UI for the a newly deployed flink app. that app will have it's own unique URL

Running with deploy_cdsw_model=true frequently causes build to fail

Tim Spann reports during setup that deploying with deploy_cdsw_model=true (the default) causes the install to fail and leaves the cluster in an unusable state.

Workaround is to set deploy_cdsw_model=false and have students run Lab 11 to set up CDSW model, but that can take 30-60 minutes.

Remove dependency to external source for the code used in streaming_flink.adoc

The workshop is based on https://github.com/zBrainiac/streaming-flink/ and it's currently broken it seems.
We should fix any issues and make it self-contained in the edge2ai repo.

Broken image links in streaming_flink.adoc

The workshop documentation has some broken image links that need fixing.

pytest fails if TF_VAR_deploy_cdsw_model is False

The current pytest tests suite assumes that CDSW has been configured and the Model is deployed.

Failure on -> ./setup/terraform/launch.sh default

Hi Team,

I'm receiving the error 'Error: error executing "/tmp/terraform_3605898.sh": Process exited with status 1' after a little over 7 minutes (even after running the launch command again). All the details on the error below. Let me know if you'd like any other information. I'll be on PTO, returning on March 9th (that I'll respond with).

My steps on set up:

I followed https://docs.google.com/document/d/1eMTCVrjGxD3BWlitojee1PRiAkrYLmIdGU-sCdkJ1Us/edit# (pulling down the latest and greatest -> first-time docker pull and clone of github)
On #9 -> I needed to modify for CEM and CM (along with CFM and CSP).
After I modified my .env file and ./setup/scripts/stack.sh (I included below so you could validate)

Now to the errors: (I ran four times, all with the same error -> after installing Maven)

First time:

aws_instance.cluster[0] (remote-exec): -- Install Maven
aws_instance.cluster[0] (remote-exec): + curl https://mirrors.sonic.net/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
aws_instance.cluster[0] (remote-exec): % Total % Received % Xferd Average Speed Time Time Time Current
aws_instance.cluster[0] (remote-exec): Dload Upload Total Spent Left Speed
aws_instance.cluster[0] (remote-exec): 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
aws_instance.cluster[0] (remote-exec): 33 9283k 33 3088k 0 0 3271k 0 0:00:02 --:--:-- 0:00:02 3271k
aws_instance.cluster[0] (remote-exec): 100 9283k 100 9283k 0 0 8232k 0 0:00:01 0:00:01 --:--:-- 8237k
aws_instance.cluster[0] (remote-exec): ++ get_homedir centos
aws_instance.cluster[0] (remote-exec): ++ local username=centos
aws_instance.cluster[0] (remote-exec): ++ getent passwd centos
aws_instance.cluster[0] (remote-exec): ++ cut -d: -f6
aws_instance.cluster[0] (remote-exec): + tar -C /home/centos -zxvf /tmp/apache-maven-bin.tar.gz
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/LICENSE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/NOTICE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/cdi-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-cli.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-io.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-lang3.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guava.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guice.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/javax.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jcl-over-slf4j.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsoup.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsr250-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.plexus.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-cipher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-component-annotations.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-interpolation.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-sec-dispatcher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-utils.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/slf4j-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/libjansi.jnilib
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/m2.conf
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnyjp
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/simplelogger.properties
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/settings.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/toolchains.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds-2.6.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-embedder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-plugin-api-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-builder-support-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-api-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-util-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-shared-utils-3.2.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-io-2.5.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guice-4.2.1-no_aop.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guava-25.1-android.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/javax.inject-1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsr250-api-1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-utils-3.2.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-sec-dispatcher-1.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-cipher-1.7.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/slf4j-api-1.7.29.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-lang3-3.8.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-core-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-repository-metadata-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-artifact-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-provider-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-impl-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-spi-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.inject-0.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-component-annotations-2.1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-compat-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-interpolation-1.25.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-provider-api-3.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.plexus-0.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/cdi-api-1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-cli-1.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-http-3.3.4-shaded.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsoup-1.12.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jcl-over-slf4j-1.7.29.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-file-3.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-connector-basic-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-transport-wagon-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-slf4j-provider-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-1.17.1.jar
aws_instance.cluster[0] (remote-exec): + rm -f /tmp/apache-maven-bin.tar.gz
aws_instance.cluster[0] (remote-exec): ++ tail -1
aws_instance.cluster[0] (remote-exec): +++ get_homedir centos
aws_instance.cluster[0] (remote-exec): +++ local username=centos
aws_instance.cluster[0] (remote-exec): +++ getent passwd centos

Error: error executing "/tmp/terraform_476292196.sh": Process exited with status 1

Second time:

aws_instance.cluster[0]: Still creating... [17m59s elapsed]
aws_instance.cluster[0] (remote-exec): Running setup.py install for thriftpy2: finished with status 'done'
aws_instance.cluster[0] (remote-exec): Running setup.py install for impyla: started
aws_instance.cluster[0] (remote-exec): Running setup.py install for impyla: finished with status 'done'
aws_instance.cluster[0] (remote-exec): Successfully installed MarkupSafe-1.1.1 attrs-19.3.0 bitarray-1.2.1 certifi-2019.11.28 cffi-1.14.0 chardet-3.0.4 cm-client-40.0.3 cryptography-2.8 deepdiff-3.3.0 docker-4.2.0 idna-2.9 importlib-metadata-1.5.0 impyla-0.16.2 jinja2-2.11.1 jsonpickle-1.3 lxml-4.3.5 more-itertools-8.2.0 nipyapi-0.14.0 packaging-20.1 paho-mqtt-1.5.0 pluggy-0.13.1 ply-3.11 psycopg2-binary-2.8.4 py-1.8.1 pyOpenSSL-19.1.0 pycparser-2.19 pyparsing-2.4.6 pytest-5.3.5 python-dateutil-2.8.1 pyyaml-5.3 requests-2.23.0 ruamel.yaml-0.16.5 ruamel.yaml.clib-0.2.0 setuptools-45.2.0 six-1.14.0 thrift-0.13.0 thriftpy2-0.4.10 urllib3-1.25.8 wcwidth-0.1.8 websocket-client-0.57.0 zipp-3.0.0
aws_instance.cluster[0] (remote-exec): + ln -s /opt/rh/rh-python36/root/bin/python3 /usr/bin/python3
aws_instance.cluster[0] (remote-exec): + ln -s /opt/rh/rh-python36/root/bin/pip3 /usr/bin/pip3
aws_instance.cluster[0] (remote-exec): + echo '-- Install JDBC connector'
aws_instance.cluster[0] (remote-exec): -- Install JDBC connector
aws_instance.cluster[0] (remote-exec): + cp /usr/share/java/postgresql-jdbc.jar /usr/share/java/postgresql-connector-java.jar
aws_instance.cluster[0] (remote-exec): + chmod 644 /usr/share/java/postgresql-connector-java.jar
aws_instance.cluster[0] (remote-exec): + echo '-- Install Maven'
aws_instance.cluster[0] (remote-exec): -- Install Maven
aws_instance.cluster[0] (remote-exec): + curl https://mirrors.sonic.net/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
aws_instance.cluster[0] (remote-exec): % Total % Received % Xferd Average Speed Time Time Time Current
aws_instance.cluster[0] (remote-exec): Dload Upload Total Spent Left Speed
aws_instance.cluster[0] (remote-exec): 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
aws_instance.cluster[0] (remote-exec): 9 9283k 9 864k 0 0 1241k 0 0:00:07 --:--:-- 0:00:07 1239k
aws_instance.cluster[0] (remote-exec): 100 9283k 100 9283k 0 0 9321k 0 --:--:-- --:--:-- --:--:-- 9320k
aws_instance.cluster[0] (remote-exec): ++ get_homedir centos
aws_instance.cluster[0] (remote-exec): ++ local username=centos
aws_instance.cluster[0] (remote-exec): ++ getent passwd centos
aws_instance.cluster[0] (remote-exec): ++ cut -d: -f6
aws_instance.cluster[0] (remote-exec): + tar -C /home/centos -zxvf /tmp/apache-maven-bin.tar.gz
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/LICENSE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/NOTICE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/cdi-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-cli.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-io.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-lang3.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guava.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guice.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/javax.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jcl-over-slf4j.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsoup.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsr250-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.plexus.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-cipher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-component-annotations.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-interpolation.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-sec-dispatcher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-utils.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/slf4j-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/libjansi.jnilib
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/m2.conf
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnyjp
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/simplelogger.properties
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/settings.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/toolchains.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds-2.6.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-embedder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-plugin-api-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-builder-support-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-api-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-util-1.4.1.jar

Error: error executing "/tmp/terraform_1718602607.sh": Process exited with status 1

Third:

aws_instance.cluster[0]: Still creating... [7m11s elapsed]
aws_instance.cluster[0] (remote-exec): Running setup.py install for thriftpy2: finished with status 'done'
aws_instance.cluster[0] (remote-exec): Running setup.py install for impyla: started
aws_instance.cluster[0] (remote-exec): Running setup.py install for impyla: finished with status 'done'
aws_instance.cluster[0] (remote-exec): Successfully installed MarkupSafe-1.1.1 attrs-19.3.0 bitarray-1.2.1 certifi-2019.11.28 cffi-1.14.0 chardet-3.0.4 cm-client-40.0.3 cryptography-2.8 deepdiff-3.3.0 docker-4.2.0 idna-2.9 importlib-metadata-1.5.0 impyla-0.16.2 jinja2-2.11.1 jsonpickle-1.3 lxml-4.3.5 more-itertools-8.2.0 nipyapi-0.14.0 packaging-20.1 paho-mqtt-1.5.0 pluggy-0.13.1 ply-3.11 psycopg2-binary-2.8.4 py-1.8.1 pyOpenSSL-19.1.0 pycparser-2.19 pyparsing-2.4.6 pytest-5.3.5 python-dateutil-2.8.1 pyyaml-5.3 requests-2.23.0 ruamel.yaml-0.16.5 ruamel.yaml.clib-0.2.0 setuptools-45.2.0 six-1.14.0 thrift-0.13.0 thriftpy2-0.4.10 urllib3-1.25.8 wcwidth-0.1.8 websocket-client-0.57.0 zipp-3.0.0
aws_instance.cluster[0] (remote-exec): + ln -s /opt/rh/rh-python36/root/bin/python3 /usr/bin/python3
aws_instance.cluster[0] (remote-exec): + ln -s /opt/rh/rh-python36/root/bin/pip3 /usr/bin/pip3
aws_instance.cluster[0] (remote-exec): + echo '-- Install JDBC connector'
aws_instance.cluster[0] (remote-exec): -- Install JDBC connector
aws_instance.cluster[0] (remote-exec): + cp /usr/share/java/postgresql-jdbc.jar /usr/share/java/postgresql-connector-java.jar
aws_instance.cluster[0] (remote-exec): + chmod 644 /usr/share/java/postgresql-connector-java.jar
aws_instance.cluster[0] (remote-exec): + echo '-- Install Maven'
aws_instance.cluster[0] (remote-exec): -- Install Maven
aws_instance.cluster[0] (remote-exec): + curl https://mirrors.sonic.net/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
aws_instance.cluster[0] (remote-exec): % Total % Received % Xferd Average Speed Time Time Time Current
aws_instance.cluster[0] (remote-exec): Dload Upload Total Spent Left Speed
aws_instance.cluster[0] (remote-exec): 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
aws_instance.cluster[0] (remote-exec): 67 9283k 67 6288k 0 0 6662k 0 0:00:01 --:--:-- 0:00:01 6661k
aws_instance.cluster[0] (remote-exec): 100 9283k 100 9283k 0 0 9215k 0 0:00:01 0:00:01 --:--:-- 9218k
aws_instance.cluster[0] (remote-exec): ++ get_homedir centos
aws_instance.cluster[0] (remote-exec): ++ local username=centos
aws_instance.cluster[0] (remote-exec): ++ getent passwd centos
aws_instance.cluster[0] (remote-exec): ++ cut -d: -f6
aws_instance.cluster[0] (remote-exec): + tar -C /home/centos -zxvf /tmp/apache-maven-bin.tar.gz
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/LICENSE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/NOTICE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/cdi-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-cli.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-io.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-lang3.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guava.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guice.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/javax.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jcl-over-slf4j.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsoup.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsr250-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.plexus.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-cipher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-component-annotations.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-interpolation.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-sec-dispatcher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-utils.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/slf4j-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/libjansi.jnilib
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/m2.conf
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnyjp
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/simplelogger.properties
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/settings.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/toolchains.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds-2.6.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-embedder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-plugin-api-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-builder-support-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-api-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-util-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-shared-utils-3.2.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-io-2.5.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guice-4.2.1-no_aop.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guava-25.1-android.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/javax.inject-1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsr250-api-1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-utils-3.2.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-sec-dispatcher-1.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-cipher-1.7.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/slf4j-api-1.7.29.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-lang3-3.8.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-core-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-repository-metadata-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-artifact-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-provider-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-impl-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-spi-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.inject-0.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-component-annotations-2.1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-compat-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-interpolation-1.25.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-provider-api-3.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.plexus-0.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/cdi-api-1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-cli-1.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-http-3.3.4-shaded.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsoup-1.12.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jcl-over-slf4j-1.7.29.jar

Error: error executing "/tmp/terraform_1571355228.sh": Process exited with status 1

Fourth:

aws_instance.cluster[0]: Still creating... [7m12s elapsed]
aws_instance.cluster[0] (remote-exec): Running setup.py install for thriftpy2: finished with status 'done'
aws_instance.cluster[0] (remote-exec): Running setup.py install for impyla: started
aws_instance.cluster[0] (remote-exec): Running setup.py install for impyla: finished with status 'done'
aws_instance.cluster[0] (remote-exec): Successfully installed MarkupSafe-1.1.1 attrs-19.3.0 bitarray-1.2.1 certifi-2019.11.28 cffi-1.14.0 chardet-3.0.4 cm-client-40.0.3 cryptography-2.8 deepdiff-3.3.0 docker-4.2.0 idna-2.9 importlib-metadata-1.5.0 impyla-0.16.2 jinja2-2.11.1 jsonpickle-1.3 lxml-4.3.5 more-itertools-8.2.0 nipyapi-0.14.0 packaging-20.1 paho-mqtt-1.5.0 pluggy-0.13.1 ply-3.11 psycopg2-binary-2.8.4 py-1.8.1 pyOpenSSL-19.1.0 pycparser-2.19 pyparsing-2.4.6 pytest-5.3.5 python-dateutil-2.8.1 pyyaml-5.3 requests-2.23.0 ruamel.yaml-0.16.5 ruamel.yaml.clib-0.2.0 setuptools-45.2.0 six-1.14.0 thrift-0.13.0 thriftpy2-0.4.10 urllib3-1.25.8 wcwidth-0.1.8 websocket-client-0.57.0 zipp-3.0.0
aws_instance.cluster[0] (remote-exec): + ln -s /opt/rh/rh-python36/root/bin/python3 /usr/bin/python3
aws_instance.cluster[0] (remote-exec): + ln -s /opt/rh/rh-python36/root/bin/pip3 /usr/bin/pip3
aws_instance.cluster[0] (remote-exec): + echo '-- Install JDBC connector'
aws_instance.cluster[0] (remote-exec): -- Install JDBC connector
aws_instance.cluster[0] (remote-exec): + cp /usr/share/java/postgresql-jdbc.jar /usr/share/java/postgresql-connector-java.jar
aws_instance.cluster[0] (remote-exec): + chmod 644 /usr/share/java/postgresql-connector-java.jar
aws_instance.cluster[0] (remote-exec): + echo '-- Install Maven'
aws_instance.cluster[0] (remote-exec): -- Install Maven
aws_instance.cluster[0] (remote-exec): + curl https://mirrors.sonic.net/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
aws_instance.cluster[0] (remote-exec): % Total % Received % Xferd Average Speed Time Time Time Current
aws_instance.cluster[0] (remote-exec): Dload Upload Total Spent Left Speed
aws_instance.cluster[0] (remote-exec): 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
aws_instance.cluster[0] (remote-exec): 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
aws_instance.cluster[0] (remote-exec): 100 9283k 100 9283k 0 0 9204k 0 0:00:01 0:00:01 --:--:-- 9209k
aws_instance.cluster[0] (remote-exec): ++ get_homedir centos
aws_instance.cluster[0] (remote-exec): ++ local username=centos
aws_instance.cluster[0] (remote-exec): ++ getent passwd centos
aws_instance.cluster[0] (remote-exec): ++ cut -d: -f6
aws_instance.cluster[0] (remote-exec): + tar -C /home/centos -zxvf /tmp/apache-maven-bin.tar.gz
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/LICENSE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/NOTICE
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/cdi-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-cli.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-io.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-lang3.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guava.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guice.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/javax.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jcl-over-slf4j.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsoup.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsr250-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.inject.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.plexus.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-cipher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-component-annotations.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-interpolation.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-sec-dispatcher.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-utils.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/slf4j-api.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds.license
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/freebsd64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux32/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/linux64/libjansi.so
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/osx/libjansi.jnilib
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows32/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/windows64/jansi.dll
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/m2.conf
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug.cmd
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvn
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnDebug
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/bin/mvnyjp
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/logging/simplelogger.properties
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/settings.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/conf/toolchains.xml
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/ext/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-native/README.txt
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/boot/plexus-classworlds-2.6.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-embedder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-settings-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-plugin-api-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-model-builder-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-builder-support-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-api-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-util-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-shared-utils-3.2.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-io-2.5.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guice-4.2.1-no_aop.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/guava-25.1-android.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/javax.inject-1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsr250-api-1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-utils-3.2.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-sec-dispatcher-1.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-cipher-1.7.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/slf4j-api-1.7.29.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-lang3-3.8.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-core-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-repository-metadata-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-artifact-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-provider-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-impl-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-spi-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.inject-0.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-component-annotations-2.1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-compat-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/plexus-interpolation-1.25.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-provider-api-3.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/org.eclipse.sisu.plexus-0.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/cdi-api-1.0.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/commons-cli-1.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-http-3.3.4-shaded.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jsoup-1.12.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jcl-over-slf4j-1.7.29.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/wagon-file-3.3.4.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-connector-basic-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-resolver-transport-wagon-1.4.1.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/maven-slf4j-provider-3.6.3.jar
aws_instance.cluster[0] (remote-exec): apache-maven-3.6.3/lib/jansi-1.17.1.jar
aws_instance.cluster[0] (remote-exec): + rm -f /tmp/apache-maven-bin.tar.gz
aws_instance.cluster[0] (remote-exec): ++ tail -1
aws_instance.cluster[0] (remote-exec): +++ get_homedir centos
aws_instance.cluster[0] (remote-exec): +++ local username=centos
aws_instance.cluster[0] (remote-exec): +++ getent passwd centos
aws_instance.cluster[0] (remote-exec): +++ cut -d: -f6
aws_instance.cluster[0] (remote-exec): ++ ls -d1tr /home/centos/apache-maven-3.6.3/bin
aws_instance.cluster[0] (remote-exec): + MAVEN_BIN=/home/centos/apache-maven-3.6.3/bin
aws_instance.cluster[0] (remote-exec): + echo 'export PATH=$PATH:/home/centos/apache-maven-3.6.3/bin'
aws_instance.cluster[0] (remote-exec): ++ get_homedir centos

Error: error executing "/tmp/terraform_3605898.sh": Process exited with status 1

My .env file: (provisioning only one cluster -> to verify this will work)

Set the number of clusters you want to deploy

export TF_VAR_cluster_count=1

Set limit on how many clusters should be deployed in parallel

export TF_VAR_parallelism=25

Set your admin parameters

export TF_VAR_owner=rcicak
export TF_VAR_web_server_admin_email=[email protected]
export TF_VAR_web_server_admin_password=******

Set your VM tags - MUST PROVIDE

export TF_VAR_enddate=02292020
export TF_VAR_project="self development"

Set your credentials

export TF_VAR_aws_region=us-east-2
export TF_VAR_aws_access_key_id=********
export TF_VAR_aws_secret_access_key=**********

Other options

export TF_VAR_ssh_username=centos
export TF_VAR_ssh_password=supersecret1
export TF_VAR_cluster_instance_type=m5.4xlarge
export TF_VAR_deploy_cdsw_model=true

OPTIONAL: AMI Selection - if not set, a default image will be selected based on the set region (see below)

#export TF_VAR_base_ami=

OPTIONAL: If using an image create by packer, set this AMI to your packer ami

#export TF_VAR_cluster_ami=$TF_VAR_base_ami

OPTIONAL: If the below variables are not specified a VPC will be created automatically

Uncomment the below if you want to bring your own VPC

#export TF_VAR_vpc_id=<CHANGE_ME>
#export TF_VAR_cidr_block_1="10.1.0.0/16"

DO NOT CHANGE BELOW THIS LINE

If not set above, chooses the default AMI based on the selected region

if [ "${TF_VAR_base_ami:-}" == "" ]; then
case "$TF_VAR_aws_region" in
"ap-northeast-1") TF_VAR_base_ami=ami-045f38c93733dd48d ;;
"ap-northeast-2") TF_VAR_base_ami=ami-06cf2a72dadf92410 ;;
"ap-south-1") TF_VAR_base_ami=ami-02e60be79e78fef21 ;;
"ap-southeast-1") TF_VAR_base_ami=ami-0b4dd9d65556cac22 ;;
"ap-southeast-2") TF_VAR_base_ami=ami-08bd00d7713a39e7d ;;
"ca-central-1") TF_VAR_base_ami=ami-033e6106180a626d0 ;;
"eu-central-1") TF_VAR_base_ami=ami-04cf43aca3e6f3de3 ;;
"eu-north-1") TF_VAR_base_ami=ami-5ee66f20 ;;
"eu-west-1") TF_VAR_base_ami=ami-0ff760d16d9497662 ;;
"eu-west-2") TF_VAR_base_ami=ami-0eab3a90fc693af19 ;;
"eu-west-3") TF_VAR_base_ami=ami-0e1ab783dc9489f34 ;;
"sa-east-1") TF_VAR_base_ami=ami-0b8d86d4bf91850af ;;
"us-east-1") TF_VAR_base_ami=ami-02eac2c0129f6376b ;;
"us-east-2") TF_VAR_base_ami=ami-0f2b4fc905b0bd1f1 ;;
"us-west-1") TF_VAR_base_ami=ami-074e2d6769f445be5 ;;
"us-west-2") TF_VAR_base_ami=ami-01ed306a12b7d1c96 ;;
esac
export TF_VAR_base_ami
fi
export TF_VAR_cluster_ami=${TF_VAR_cluster_ami:-$TF_VAR_base_ami}

My ./setup/scripts/stack.sh file:
#!/usr/bin/env bash

Mandatory component: BASE

Common components to CDH and CDP: CDSW, FLINK, HBASE HDFS, HIVE, HUE, IMPALA, KAFKA, KUDU,

NIFI, OOZIE, SCHEMAREGISTRY, SMM, SRM, SOLR, SPARK_ON_YARN, YARN,

ZOOKEEPER

CDP-only components: ATLAS, LIVY, RANGER, ZEPPELIN

CM_SERVICES=BASE,ZOOKEEPER,HDFS,YARN,HIVE,HUE,IMPALA,KAFKA,KUDU,NIFI,OOZIE,SCHEMAREGISTRY,SPARK_ON_YARN,SMM,CDSW,FLINK,SOLR,HBASE,ATLAS,LIVY,ZEPPELIN
ENABLE_KERBEROS=no

Cloudera public repository credentials - only needed if using subscription-only URLs below; leave it blank otherwise

Must always be set. If not needed, set to "dummy"

REMOTE_REPO_USR=dummy
REMOTE_REPO_PWD=dummy

Java Package

JAVA_PACKAGE_NAME=java-1.8.0-openjdk-devel

Maven binary

MAVEN_BINARY_URL=https://mirrors.sonic.net/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz

BASE_URI=https://archive.cloudera.com

CM

CM_VERSION=7.0.3
CM_MAJOR_VERSION=${CM_VERSION%%.*}
CM_REPO_AS_TARBALL_URL=https://archive.cloudera.com/cm7/7.0.3/repo-as-tarball/cm7.0.3-redhat7.tar.gz
#CM_REPO_FILE_URL=${BASE_URI}/cm${CM_MAJOR_VERSION}/${CM_VERSION}/redhat7/yum/cloudera-manager-trial.repo

CDH

CDH_VERSION=7.0.3
CDH_BUILD=${CDH_VERSION}-1.cdh${CDH_VERSION}.p0.1635019
CDH_MAJOR_VERSION=${CDH_VERSION%%.*}
CDH_BASE_URI=${BASE_URI}
CDH_PARCEL_REPO=${CDH_BASE_URI}/cdh${CDH_MAJOR_VERSION}/${CDH_VERSION}.0/parcels/

CFM

CFM_VERSION=2.0.0.0
CFM_BUILD=${CFM_VERSION}-106
NIFI_VERSION=1.10.0
NIFI_REGISTRY_VERSION=0.5.0
CFM_BASE_URL=CFM_BASE_URL=http://s3.amazonaws.com/dev.hortonworks.com/CFM/centos7/2.x/BUILDS/${CFM_BUILD}/tars
CFM_PARCEL_REPO=${CFM_BASE_URL}/parcel
CFM_NIFI_CSD_URL=${CFM_BASE_URL}/parcel/NIFI-${NIFI_VERSION}.${CFM_BUILD}.jar
CFM_NIFICA_CSD_URL=${CFM_BASE_URL}/parcel/NIFICA-${NIFI_VERSION}.${CFM_BUILD}.jar
CFM_NIFIREG_CSD_URL=${CFM_BASE_URL}/parcel/NIFIREGISTRY-${NIFI_REGISTRY_VERSION}.${CFM_BUILD}.jar

Anaconda

ANACONDA_VERSION=5.1.0.1
ANACONDA_PARCEL_REPO=https://repo.continuum.io/pkgs/misc/parcels/archive/

CDSW

If version is set, install will be attempted

CDSW_VERSION=1.6.1
CDSW_BUILD=${CDSW_VERSION}.p1.1504243
CDSW_PARCEL_REPO=${BASE_URI}/cdsw1/${CDSW_VERSION}/parcels/
CDSW_CSD_URL=${BASE_URI}/cdsw1/${CDSW_VERSION}/csd/CLOUDERA_DATA_SCIENCE_WORKBENCH-CDH6-${CDSW_VERSION}.jar

CSP

CSP_VERSION=3.0.0.0-104
CSP_MAJOR_VERSION=${CSP_VERSION%%.*}
SCHEMAREGISTRY_VERSION=0.8.1
SCHEMAREGISTRY_BUILD=${SCHEMAREGISTRY_VERSION}.${CSP_VERSION}
STREAMS_MESSAGING_MANAGER_VERSION=2.1.0
STREAMS_MESSAGING_MANAGER_BUILD=${STREAMS_MESSAGING_MANAGER_VERSION}.${CSP_VERSION}
STREAMS_REPLICATION_MANAGER_VERSION=1.0.0
STREAMS_REPLICATION_MANAGER_BUILD=${STREAMS_REPLICATION_MANAGER_VERSION}.${CSP_VERSION}
CSP_BASE_URL=http://s3.amazonaws.com/dev.hortonworks.com/CSP/centos7/${CSP_MAJOR_VERSION}.x/BUILDS/${CSP_VERSION}/tars
CSP_PARCEL_REPO=${CSP_BASE_URL}/parcel/
SCHEMAREGISTRY_CSD_URL=${CSP_BASE_URL}/parcel/SCHEMAREGISTRY-${SCHEMAREGISTRY_VERSION}.jar
STREAMS_MESSAGING_MANAGER_CSD_URL=${CSP_BASE_URL}/parcel/STREAMS_MESSAGING_MANAGER-${STREAMS_MESSAGING_MANAGER_VERSION}.jar
STREAMS_REPLICATION_MANAGER_CSD_URL=${CSP_BASE_URL}/parcel/STREAMS_REPLICATION_MANAGER-${STREAMS_REPLICATION_MANAGER_VERSION}.jar

CEM

CEM_VERSION=1.0.0.0
CEM_BUILD=${CEM_VERSION}-25
CEM_MAJOR_VERSION=${CEM_VERSION%%.*}
EFM_VERSION=1.0.0
MINIFI_VERSION=0.6.0

PUBLIC TARBALL

CEM_BASE_URL=${BASE_URI}/CEM/centos7/${CEM_MAJOR_VERSION}.x/updates/${CEM_VERSION}
CEM_TARBALL=CEM-${CEM_VERSION}-centos7-tars-tarball.tar.gz
#CEM_URL=${CEM_BASE_URL}/${CEM_TARBALL}

INDIVIDUAL

CEM_BASE_URL=${BASE_URI}
EFM_TARBALL_URL=${CEM_BASE_URL}/efm/efm-${EFM_VERSION}.${CEM_BUILD}-bin.tar.gz
MINIFI_TARBALL_URL=${CEM_BASE_URL}/minifi/minifi-${MINIFI_VERSION}.${CEM_BUILD}-bin.tar.gz
MINIFITK_TARBALL_URL=${CEM_BASE_URL}/minifi/minifi-toolkit-${MINIFI_VERSION}.${CEM_BUILD}-bin.tar.gz

CSA

CSA_VERSION=1.0.0.0
CSA_BUILD=NA
FLINK_VERSION=1.9.0
CSA_BASE_URL=${BASE_URI}
FLINK_BUILD=${FLINK_VERSION}-csa${CSA_VERSION}-cdh6.3.0
CSA_BASE_URL=${BASE_URI}/csa/${CSA_VERSION}
CSA_PARCEL_REPO=${CSA_BASE_URL}/parcels/
FLINK_CSD_URL=${CSA_BASE_URL}/csd/FLINK-${FLINK_BUILD}.jar

Parcels to be pre-downloaded during install.

Cloudera Manager will download any parcels that are not already downloaded previously.

PARCEL_URLS=(
hadoop "$CDH_BUILD" "$CDH_PARCEL_REPO"
nifi "$CFM_VERSION" "$CFM_PARCEL_REPO"
cdsw "$CDSW_BUILD" "$CDSW_PARCEL_REPO"
Anaconda "$ANACONDA_VERSION" "$ANACONDA_PARCEL_REPO"
schemaregistry "$SCHEMAREGISTRY_BUILD" "$CSP_PARCEL_REPO"
streamsmsgmgr "$STREAMS_MESSAGING_MANAGER_BUILD" "$CSP_PARCEL_REPO"
streamsrepmgr "$STREAMS_REPLICATION_MANAGER_BUILD" "$CSP_PARCEL_REPO"
flink "$FLINK_BUILD" "$CSA_PARCEL_REPO"
)

CSD_URLS=(
$CFM_NIFI_CSD_URL
$CFM_NIFICA_CSD_URL
$CFM_NIFIREG_CSD_URL
$CDSW_CSD_URL
$SCHEMAREGISTRY_CSD_URL
$STREAMS_MESSAGING_MANAGER_CSD_URL
$STREAMS_REPLICATION_MANAGER_CSD_URL
$FLINK_CSD_URL
)

new commit causing deployment failure

Haven't had time to troubleshoot this yet, but the same configs were working last week before i pulled in the latest commit

Mon Jun 29 02:41:14 2020
Cmd ID: 1546333147, Name: ClusterTemplateImport, Status: FAILED (java.lang.IllegalArgumentException: Unknown configuration attribute 'xml.authorizers.userGroupProvider.composite-configurable-user-group-provider.property.User Group Provider 1' for service (type: 'NIFI', name: 'nifi').)
 +- Cmd ID: 1546333155, Name: DeployParcels, Status: Succeeded (The Following parcels successfully activated : Anaconda-5.1.0.1,CDH-7.1.1-1.cdh7.1.1.p0.3266817,CDSW-1.7.1.p1.1850946,CFM-2.0.1.0,FLINK-1.10.0-csa1.2.0.0-cdh7.1.1.0-326-2901802.) (Cluster: OneNodeCluster)
Traceback (most recent call last):
  File "/tmp/resources/create_cluster.py", line 347, in <module>
    CLUSTER_CREATOR.create_cluster(options.template)
  File "/tmp/resources/create_cluster.py", line 286, in create_cluster
    raise RuntimeError('Failed to deploy cluster template')
RuntimeError: Failed to deploy cluster template

LAB 6: SMM option needed to to show replication UI in SMM

In order to see the SRM metrics in SMM, you have to set the following option in the SMM config:

Streams Messaging Manager Rest Admin Server Advanced Configuration Snippet (Safety Valve) for staging/streams-messaging-manager-srm.xml

New Config:
Name:SRM
Value: edge2ai-1.dim.local

Restart SMM, and then also restart SRM afterwards.

Lacking port no for SSB in web server

Web server has no port number for SSB:

CDSW: MLOPS_MODELMETRICS_ENABLE disabled by default

It would be nice if MLOPS_MODELMETRICS_ENABLE were enabled by default so ML Ops demos worked without a config change.

Superfluous parameters should not block execution

Especially in stack.sh there are some things that block execution if combined.

For example 'you cannot fill in link a and link b at the same time'

This is quite a puzzle for infrequent users. Please let these things generate a warning or confirmation dialog if needed, but simply pick a default one to use when too much is filled in.

Switch from system python2 to python3

Schema Registry: Error 401 Authentication required

Access to Schema Registry end with a Error 401 Authentication required
URI: http://<public_dns>:7788/

stack.sh

CM_VERSION=7.0.3
SCHEMAREGISTRY_VERSION=0.8.0

Tested with:

pulled the latest version from github
ENABLE_KERBEROS=yes and /or no

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.