User Story

As a contributor to Greenplum Database
I'd like to have ad-hoc interaction with my compiled database
So that I can have confidence in my changes
And I can stop making Jesse sad by forking pipelines

Scenario

Given I have my changes to zero or more of:

1. GPOS
2. ORCA
3. GPDB

When I run streamline-master/uber.bash,
Then it will build the database and set up the cluster
And it will prompt me how to get an interactive shell into the container, where I can run psql or make installcheck-good

Background

Currently uber.bash does all of these 4 things:
0. it builds the universe
0. it initializes a cluster
0. it starts the cluster
0. it runs installcheck with ORCA

See if we can split that last guy off

This comes up whenever I think about #37 . The nice thing about our current approach seems to be that containers have unique names. The problem with that approach is it becomes harder to refer to the container "from the outside", e.g. when you need to docker exec or when you want to start a privileged container sharing some namespace (pid, ipc, or network) with the Greenplum container.

This does not entirely supplant #37 but should enable a couple use cases that #37 was meant to address.

This should help quickly validate a pair of ORCA and GPDB pull requests

It's like, --interactive and ICG got married and this will be their child.

yum is yuck

This might enable two use cases:

1. Getting another shell in the container by docker exec -ti <name> /bin/bash
2. Running psql directly against the database with something like

   docker run --rm -ti --link <name>:gpdb postgres psql -h gpdb -p 15432 -U gpadmin -d postgres

   or

   docker run --rm -ti --net container:<name> postgres psql -h localhost -p 15432 -U gpadmin -d postgres

• Build 32bit/64bit
• Build with/without GPORCA
• Build with/without CodeGen
• Run ICG with/without GPORCA
• Run ICB with/without GPORCA

Given the declarative nature of Dockerfile, it may be tempted to just not test them. However, in times of big changes, I still sorely miss having some tests (with clear intents of course).

My buddy @xinzweb, who's an existential thinker, feels the enormous emptiness of life when the deep dark terminal screen is not animating. All of a sudden, every second feels like 10 years. He turned on env DEBUG=1, but there are still a few places that are not noisy enough:

1. docker building an image
2. compiling ORCA
3. any other place that's not producing pointless output

Let's make him happy

It didn't feel like the output of make commands provided much value, consider silencing it with make --silent, at least when DEBUG is unset?

After #1, the script will not delete the container unless something fails before entering interactive shell. Maybe show a message about the command to delete the container?

On a fast (like 2016) system, running unit tests with make -s -j8 -C /build/gpdb/src/backend unittest-check has a high chance of failing compilation because the test directories stampede on each other when trying to build the mocks.

User story

As a Greenplum database core maintainer,
I want to a consistent way to verify pull requests,
which gives me the same level of confidence as a forked CI pipeline would have given me

Hello the ORCA and Greenplum community,
This is my first try at doing an RFC.

Background

I don't know about you but besides Linux, I also use a Mac, and boy it is a slow one (2012 MacBook Air, 2 physical cores with 4 "hyperthreading" cores). The uber script currently works in the following way:

0. The uber script mounts the orca source code at /workspace/gporca as a read-only volume. Similarly, other repositories needed are available readonly under /workspace.

1. For orca, gp-xerces, and (optionally) gpos, it uses out-of-tree build, generating a build tree outside the source tree, but referring to source tree from /workspace/gporca. The container path /workspace/gporca is a read-only mount from our Macs.

2. For the Postgres-based server of Greenplum, it clones the Greenplum database source code from /workspace/gpdb to inside the container at /build/gpdb, and generates the build tree in-tree at /build/gpdb.

The original intent was to use out-of-tree build as much as possible because we envisioned a workflow where a developer could make a 3-line change, and build the entire product in a few seconds. And the clone-followed-by-in-tree build method for Greenplum code base was purely incidental, mostly motivated by the fact that until Postgres 9.0, the autoconf build system in Postgres wasn't robust enough to support out-of-tree build.

The only pain here, is the fact that when we are not working on a Linux workstation, the file system used for /workspace/orca and friends is one of the slowest (fuse-based osxfs, vboxsf for Virtual Box, or hgfs for VMware). An empirical evidence is that the uber script builds way way faster on a underpowered 2-core Linux host than a 8-core Mac host:

Proposal

I propose we switch to cloning ORCA into the container before building, just like how we build the Greenplum server. This is not strictly an improvement, but a trade-off:

Pros

Vastly faster builds on Macs, and in any situation that use a desktop-grade VM to host Docker really. Here's a new piece of anecdotal evidence:

Cons

The following workflow that we originally envisioned will stop working:

0. Make some small changes to Orca on your Mac, don't commit
1. Run Uber script to spin up a cluster running with your changes

Instead, we need to commit our changes before running the uber script. Is this acceptable?

Other concerns

• Can everybody take it for a spin and report back the timing?
• Documentation update

in 4.3_STABLE, drop_if_exists fails if USER env var is unset.

We often forget to do add -l to /bin/bash -l. Either show a message so user can copy-paste, or do your wizardry.

Here is a scenario:

• I run the full ICG and ICB
• One test failed
• I want to repro just that one test failure instead of running the entire tests.

To reproduce, run uber.bash, and press Ctrl-C during ORCA compilation. This only interrupts the docker client that is waiting for the container that's building ORCA, but it does not really stop or remove the build container.

Given that Omer, an ORCA contributor is fixing a planner issues
When Omer runs uber.bash --no-optimizer
Then uber script should run installcheck without turning on optimizer.

There are a few scenarios where we found it really valuable to run the linux perf tools to profile the databases:

1. When we recently found out that mirrors go nuts in the middle of installcheck-good, and we want to be good citizens and collect perf data for the storage team
2. When we are thinking about hot spots and bottlenecks in ORCA, and want a scientific way to profile it
3. In the context of some C++ refactoring, we want to look into using perf probe to accurately count the number of invocations of certain functions that we aim to hide away using resource wrappers, but were afraid of introducing performance regressions (the plan B being using a global volatile int64_t to just count ...)

This is not impossible but might be painful without either #41 or #37

Even if it's just running shellcheck

User Story

As a lazy person who contributes to Greenplum Database,
I'd like to run a single script to have all my code built,
and my database cluster initialized,
AND I'd like to be able to proceed to testing after that.

Scenario

Given my code changes to zero or more of

1. ORCA
2. GPOS
3. GPDB

When I run streamline-master/uber.bash --interactive
After it builds the universe and it initializes the database,
Then it presents me with an interactive shell, which is

1. primed with the right environment (PGPORT, and possibly PATH)
2. cd'd into src/test/regress

When I'm using a Mac I find myself doing a trick inside the container to incrementally build:

git fetch origin HEAD && git reset --hard FETCH_HEAD && make install -s -j8 -l12 -C /build/gpdb && gpstop -ari

I heard @hardikar complaining about this today, might as well document it

When I run ~/bug-free-fortnight/pg/uber.bash --interactive on the pg branch, I get the following error

Get:91 http://archive.ubuntu.com/ubuntu/ trusty-updates/main libc6-dbg amd64 2.19-0ubuntu6.13 [3462 kB]
Fetched 46.2 MB in 24s (1887 kB/s)
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/c/curl/libcurl3-gnutls_7.35.0-1ubuntu2.11_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/l/linux/linux-libc-dev_3.13.0-133.182_amd64.deb  404  Not Found [IP: 91.189.91.26 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get install -y 	gdb 	ccache 	make 	gcc 	bison 	flex 	vim 	zlib1g-dev 	libreadline-dev 	runit 	byobu 	git 	rsync' returned a non-zero code: 100

Just found out Asim has two checkouts: gpdb-postgres-merge and gpdb

17.04 zesty was an awkward short-living release (like all non-LTS rolling releases). It was already out of the "rolling" designation by the time we chose it. And we only chose it because of new compiler warnings from GCC 7. Now that these have all been fixed, let's keep
rolling forward!

This should enable us to automatically upgrade to future rolling releases should they become available.

I am starting to feel there's too much noise in the output, especially considering that I get diminishing value out of the output: what do I get out of the quickly scrolling screen? The feeling that time is elapsing?

I have a few options to make this more quiet, but before even shaving this yak, I want to gather some feedback from you guys, @oarap @xinzweb and @hsyuan:

1. Do you feel strongly about this? Choose one out of three:
   • I hate it, it's so noisy!!!
   • I love it, noise gives me a good existential feeling!
   • Noise. Meh. C'est la vie
2. Am I the only one annoyed by the ORCA compilation output at the beginning of the build process?
3. If you find it valuable, can you leave a comment below and let me know what value you get out of the output
4. If you feel "meh", can you help me decide which improvement item gives you more value:
   1. debug build
   2. suppress ORCA output unless it fails

It's long overdue

This happens when:

1. Your Docker host (read: the Linux distro on which containerd runs, not really the host where a VM runs) is Ubuntu with AppArmor configured (default)
2. Your Docker version is lower than 18.03.1 (Community Edition)

This is not my bad. The bug is fixed upstream and undergoing further verification and testing. See the following upstream bugs:

1. moby/moby#36466
2. docker/docker-ce#470

Not that I care, but a lot of people seem annoyed by zombies.
Sort out the pid 1 problem maybe

User Story

We should build GPOS and ORCA with exactly the same toolchain and image as in CI
Before we experiment with bumping the compiler version independently
Because this allows us to draw confidence in the installcheck test run that artifacts from different compilers actually interoperate correctly

Because it's LTS, or because it's new, whatever

Heard this question from @oarap today.

askingforafriend

We've all been there, breaking 'em unintentionally. Wouldn't it be nice if we'd known better?

Or something that can be better tested ...

I'm gonna sound very pretentious and claim that I'm a Bash guru. However, I'm also very fed up with every surprise the Bash (and POSIX sh overall) gives to everybody else. The cost of doing the right thing in Bash is too high that I'm willing to write this (maybe) in C++...

GPDB actually records a version of ORCA it should be built with, as a non-ORCA developer, this may come in handy?