Git Product home page Git Product logo

zoomdata-formula's Introduction

zoomdata-formula

Install, configure and run the Zoomdata services.

IMPORTANT!

This code is experimental and still in development. It is not officially supported by Zoomdata, Inc. and provided for evaluation purposes only.

NOTE

See the full Salt Formulas installation and usage instructions.

Available states

Bootstrap the Zoomdata services from scratch or upgrade existing installation.

Make backup of the Zoomdata installation state and metadata (PostgreSQL) databases.

Prepare a directory on local filesystem to store backups.

Write compressed dumps of PostgreSQL databases.

Remove old backups. Keep last 10 by default.

Write a Pillar SLS file that describes current Zoomdata installation state.

Disable the Zoomdata services and uninstall the Zoomdata packages.

Configure package repositories for installing the Zoomdata packages.

Restore the Zoomdata installation from previously made backup.

Restore the Zoomdata databases in a PostgreSQL cluster.

Install, configure, enable and start the Zoomdata services.

Install the Zoomdata packages and write the configuration files.

Start the Zoomdata services.

Stop the Zoomdata services.

Setup initial runtime parameters for the Zoomdata server.

Install additional explicitly defined packages from tools repository.

zoomdata-formula's People

Contributors

minifygb avatar niyakiy avatar srikanthinsight avatar vutny avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zoomdata-formula's Issues

Zoomdata Screenshot service is not working with provided Pillar example

Description

If using provided configuration in the pillar.example file, the zoomdata-screenshot-service is not able to create any screenshots for Zoomdata visualizations, dashboards and bookmarks.
The Pillar should contain the following:

zoomdata:
  post_install:
    zoomdata-screenshot-service:
      - /opt/zoomdata/docs/screenshot-service/install-dependencies.sh

Symptoms

The service log file at /opt/zoomdata/logs/screenshot-service.log contains the exception like this:

2019-01-08 12:56:58,013 INFO  [c.z.s.c.ScreenshotController] [334c9d3f0b654168, 09fd1740ae63edfc, true] Received screenshot request for: http://localhost:8080/zoomdata/visualization/5c349e195c30a43e92334125?__target=widget&screenshot_capture=true&hidecontrol=timetable,vistitle&key=10t6N47ijX 
2019-01-08 12:56:58,124 ERROR [o.s.c.s.i.w.ExceptionLoggingFilter] [334c9d3f0b654168, 09fd1740ae63edfc, true] Uncaught exception thrown 
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.concurrent.ExecutionException: java.lang.IllegalStateException: The path to the driver executable must be set by the webdriver.chrome.driver system property; for more information, see https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver. The latest version can be downloaded from http://chromedriver.storage.googleapis.com/index.html

The affected zoomdata-screenshot-service package version is 3.7.3-4.

Reason

This is not an issue with the formula, but the packaged post-installation script!

The install-dependencies.sh script contains a shameful bug causing chromedriver binary would not be unpacked from downloaded archive.

Workaround

Run the following commands on the machine where Zoomdata has been installed:

CHROMEDRIVER_LATEST_VERSION=$(curl -s https://chromedriver.storage.googleapis.com/LATEST_RELEASE)
curl -s -o /tmp/chromedriver.zip "https://chromedriver.storage.googleapis.com/${CHROMEDRIVER_LATEST_VERSION}/chromedriver_linux64.zip"
sudo unzip /tmp/chromedriver.zip -d /usr/bin/ && rm -f /tmp/chromedriver.zip

On Ubuntu GNU/Linux operating systems you may need to additionally install curl in advance.

sudo apt-get install curl

Resolution

The issue with the packaged script is expected to be fixed in Zoomdata LTS patch release 3.7.4 and upcoming 4.0.0 cutting-edge release.

Compatibility with Salt 2019.2 (Fluorine)

Got this error during state.apply with latest stable Salt 2019.2.1:

Data failed to compile:
----------
    Rendering SLS 'base:zoomdata.backup.metadata' failed:
    Jinja variable 'dict object' has no attribute 'zoomdata_properties'

Deprecate `tls` states and configuration

The states in tls.sls file are not being used and tested for a while.
The production Zoomdata setup requires external load-balancer or proxy to efficiently offload TLS, such as AWS ELB, Nginx or HAproxy.
Also, we do not ship keytool with bundled JVM anymore.

The function `zoomdata.list_repos` misdetects repo settings

The function zoomdata.list_repos, when called with compact=True argument, returns wrong results if configured Zoomdata release is not latest.
For example:

  base_url: https://repo.zoomdata.com
  components:
  - stable
  gpgkey: https://repo.zoomdata.com/ZOOMDATA-GPG-KEY.pub
  release: tools
  repositories:
  - '3.7'

Expected result is:

...
  release: '3.7'
  repositories:
  - tools

Other side effect reveals when legacy repositories are present in the system:

  base_url: http://repo.zoomdata.com
  components:
  - stable
  - unstable
  gpgkey: https://repo.zoomdata.com/ZOOMDATA-GPG-KEY.pub
  release: latest
  repositories:
  - tools
  - tools
  - '3.1'

PostgreSQL passwords with colon (:) aren't properly quoted in zoomdata.backup state

Heaving this in pillar:

spring.datasource.password:
    zQkHK3rHD22z%]Q:

note colon at the end of password!
zoomdata.backup state rendering fails with this:

Data failed to compile:
----------
    Rendering SLS 'base:zoomdata.backup' failed: mapping values are not allowed here; line 99

---
[...]
        gzip
        --stdout  >
        /opt/zoomdata/data/backups/18-04-23_14:45:29/zoomdata-scheduler_postgre.sql.gz
    - env:
      - PGUSER: zoomdata
      - PGPASSWORD: zQkHK3rHD22z%]Q:    <======================
    - onchanges:
      - file: zoomdata_backup_dir
    - onchanges_in:
      - file: zoomdata_backup_retension
    - require:
[...]

@vutny FYI

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.