johannestang / bigdata_stack Goto Github PK

Hi @johannestang thanks for the quick start repo you have made!

i am also thinking of setting up a big data stack for a docker cluster and possibly doing so using helm charts for k8s.

Before i start testing with your stack i would to ask a couple of questions to understand the direction to take when making possible improvements:

• first off, what do you think should be improved first in this project? do you have a desired roadmap ahead?
• secondly, if i understand correctly, the HIVE service is actually only needed as a requirement to use Presto to run SQL right? Is it possible to get rid of Hive if we only want to use Presto / Impala or is it not possible currently?
• thirdly, in your blog post you state that "There are of course many other interesting big data SQL engines, e.g. Impala, Spark SQL, and Drill. For background on these (and more) have a look at this great post." Does it mean that if one wants to use Spark (and thus Spark SQL) one can use Hive and remove Presto from the stack, or would you still recommend connecting SparkSQL to Presto to run queries?

thanks for the clarification.

P.S. have you published any newer Blog Post since 2019 ? Let me know

jessie-backports repo was removed from debian ftp, so the repo needs to replace by this one:

RUN echo "deb [check-valid-until=no] http://archive.debian.org/debian jessie-backports main" >> /etc/apt/sources.list

see https://unix.stackexchange.com/questions/508724/failed-to-fetch-jessie-backports-repository