Drake platform reviewers should be able to approve Shambhala PRs. At a minimum, we want to allow priority fixes to be merged by any of the well-staffed and trusted review crew, instead of blocking on a very busy @stonier.

Relates to #15, but we could fix the permissions in GH earlier without going through all of reviewable.

• drake-shambhala#5
• drake-shambhala#6

The failures are occuring for drake_cmake_external, here:

@tinyobjloader//:install: '@tinyobjloader//:install_cmake_config' does not have mandatory provider 'install_actions'

In the latter, it is with octomap instead of tinyobjloader. Oddly enough, it is working fine from a build on my pc.

Per this pull request build:
https://circleci.com/gh/RobotLocomotion/drake-shambhala/235

Perhaps the prereqs should be updated to include system gflags?

$ cmake ../ -Ddrake_DIR=~/tmp/drake/lib/cmake/drake

CMake Error at /usr/local/Cellar/cmake/3.9.4_1/share/cmake/Modules/CMakeFindDependencyMacro.cmake:37 (find_package):
  Could not find a package configuration file provided by "VTK" (requested
  version 8.0.1) with any of the following names:

    VTKConfig.cmake
    vtk-config.cmake

  Add the installation prefix of "VTK" to CMAKE_PREFIX_PATH or set "VTK_DIR"
  to a directory containing one of the above files.  If "VTK" provides a
  separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
  /Users/soonhok/tmp/drake/lib/cmake/drake/drake-config.cmake:78 (find_dependency)
  CMakeLists.txt:39 (find_package)


-- Configuring incomplete, errors occurred!
See also "/Users/soonhok/work/sham/drake_cmake_installed/build/CMakeFiles/CMakeOutput.log".

Env:

• macOS 10.13
• cmake-3.9.4 installed (via homebrew)
• vtk-8.0.1 installed (via homebrew)

See the following two failures:

• https://drake-jenkins.csail.mit.edu/view/Continuous%20Production/job/RobotLocomotion/job/drake-shambhala/job/master/56/execution/node/18/log/
• https://drake-jenkins.csail.mit.edu/view/Nightly%20Production/job/RobotLocomotion/job/drake-shambhala/job/master/56/execution/node/18/log/

Relates to #61.

Without diving into any plan yet, would like to fathom exactly what is going on. There are some aspects of the linking I haven't come across before and some of this is having surprising effects for us (in Delphyne).

Minimal Example: Particle demo at https://github.com/stonier/drake-shambhala/tree/stonier/link_paths.

The only differences here is the library is built shared and both library and particle executable are installed. Build with:

cmake -DCMAKE_INSTALL_PREFIX=../install -Ddrake_DIR=/opt/drake/lib/cmake/drake ..
make
make install

Resulting links in the build directory:

ldd build/src/particles/uniformly_accelerated_particle
        libparticles.so => /mnt/mervin/workspaces/shambhala/drake_cmake_installed/build/src/particles/libparticles.so (0x00007fea3504e000)
        /opt/drake/lib/libdrake.so (0x00007fea33a02000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fea33680000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fea33377000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fea33161000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fea32d97000)
        /opt/drake/lib/libdrake_bullet_collision.so (0x00007fea32a88000)
        /opt/drake/lib/libdrake_bullet_linear_math.so (0x00007fea32864000)
        /opt/drake/lib/libdrake_ignition_rndf0.so (0x00007fea3262f000)
        /opt/drake/lib/libdrake_ignition_math4.so (0x00007fea323f6000)
        /opt/drake/lib/libdrake_lcm.so (0x00007fea321dd000)
        libvtkFiltersSources-8.0.so.1 => /opt/drake/lib/libvtkFiltersSources-8.0.so.1 (0x00007fea31d08000)

In the install directory:

ldd install/bin/uniformly_accelerated_particle
        libparticles.so => not found
        libgflags.so.2 => /usr/lib/x86_64-linux-gnu/libgflags.so.2 (0x00007f9c78706000)
        /opt/drake/lib/libdrake.so (0x00007f9c772db000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9c76f59000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9c76c50000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9c76a3a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9c76670000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9c76453000)
        libdrake_ignition_rndf0.so => /opt/drake/lib/libdrake_ignition_rndf0.so (0x00007f9c7621e000)
        libdrake_ignition_math4.so => /opt/drake/lib/libdrake_ignition_math4.so (0x00007f9c75fe5000)
        libvtkFiltersSources-8.0.so.1 => /opt/drake/lib/libvtkFiltersSources-8.0.so.1 (0x00007f9c75d2d000)

Questions:

• Why is the libdrake.so path locked in?
• Why do the libdrake.so and dependency paths not stripped on install as is done for libparticles.so? (not rpaths?)

Use the same one (already in?) used by Drake.

Per discussion in #76, the PCL + Drake example (and linking test) should show non-trivial functionality.

At present, this is blocked by the failure shown in #90.

\cc @jamiesnape

The drake_bazel_external documentation doesn't give any hint about how to turn on the commercial solvers for mathematical programs (SNOPT, MOSEK, Gurobi) when using Drake as a Bazel external. We should document it.

@jamiesnape you added MIT licenses here, however I just noticed that Drake is BSD licensed, not MIT.

Should have them matching if we can.

Hi guys,
So here is the project I am working on. https://github.com/siyuanfeng-tri/rand_obj_picking
This repo has 4 main sub components and a executable that depends on them. The particular problem I am running into is during linking for that application.

As is, this should build just fine. There is a short readme explaining how to make. To illustrate the problem, uncomment https://github.com/siyuanfeng-tri/rand_obj_picking/blob/master/src/test_robot_bridge.cc#L4 and https://github.com/siyuanfeng-tri/rand_obj_picking/blob/master/src/test_robot_bridge.cc#L42
You should see something like

Scanning dependencies of target test_robot_bridge
[ 50%] Building CXX object CMakeFiles/test_robot_bridge.dir/test_robot_bridge.cc.o
[100%] Linking CXX executable test_robot_bridge
/usr/bin/c++   -g -O2 -Wall     CMakeFiles/test_robot_bridge.dir/test_robot_bridge.cc.o  -o test_robot_bridge  -L/home/sfeng/code/rand_obj_picking/build/lib -lperception -lgrasp_gen -lrobot_bridge /home/sfeng/drake_bazel_build/lib/libdrake.so /home/sfeng/drake_bazel_build/lib/libBulletCollision.so /home/sfeng/drake_bazel_build/lib/libLinearMath.so /home/sfeng/drake_bazel_build/lib/libfcl.so /home/sfeng/drake_bazel_build/lib/libccd.so /home/sfeng/drake_bazel_build/lib/liboctomap.so /home/sfeng/drake_bazel_build/lib/libignition_rndf.so /home/sfeng/drake_bazel_build/lib/liblcm.so -lglib-2.0 -lpthread /home/sfeng/drake_bazel_build/lib/libprotobuf.so /home/sfeng/drake_bazel_build/lib/libyaml_cpp.so /home/sfeng/drake_bazel_build/lib/libfmt.so /home/sfeng/drake_bazel_build/lib/libscsdir.so /home/sfeng/drake_bazel_build/lib/libsdformat.so /home/sfeng/drake_bazel_build/lib/libignition_math.so -ltinyxml /home/sfeng/drake_bazel_build/lib/libtinyobjloader.so -ltinyxml2 -Wl,-rpath,/home/sfeng/code/rand_obj_picking/build/lib:/home/sfeng/drake_bazel_build/lib: 
/usr/bin/ld: CMakeFiles/test_robot_bridge.dir/test_robot_bridge.cc.o: undefined reference to symbol '_ZN20vtkDebugLeaksManagerD1Ev'
/home/sfeng/drake_bazel_build/lib/libvtkCommonCore-8.0.so.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
CMakeFiles/test_robot_bridge.dir/build.make:112: recipe for target 'test_robot_bridge' failed
make[6]: *** [test_robot_bridge] Error 1
CMakeFiles/Makefile2:70: recipe for target 'CMakeFiles/test_robot_bridge.dir/all' failed
make[5]: *** [CMakeFiles/test_robot_bridge.dir/all] Error 2
Makefile:130: recipe for target 'all' failed
make[4]: *** [all] Error 2
CMakeFiles/clutter_proj.dir/build.make:114: recipe for target 'clutter_proj-prefix/src/clutter_proj-stamp/clutter_proj-build' failed
make[3]: *** [clutter_proj-prefix/src/clutter_proj-stamp/clutter_proj-build] Error 2
CMakeFiles/Makefile2:107: recipe for target 'CMakeFiles/clutter_proj.dir/all' failed
make[2]: *** [CMakeFiles/clutter_proj.dir/all] Error 2
CMakeFiles/Makefile2:119: recipe for target 'CMakeFiles/clutter_proj.dir/rule' failed
make[1]: *** [CMakeFiles/clutter_proj.dir/rule] Error 2
Makefile:131: recipe for target 'clutter_proj' failed
make: *** [clutter_proj] Error 2

For this particular executable, it links to perception and robot_bridge, which both links against drake::drake. I have tried to make individual executable that links to either one, and it would work, but not both. I have also tried to explicitly link against libvtkCommonCore-8.0.so.1, but that ends up complaining about more vtk stuff upstream. Any ideas about what I am doing wrong here???
cc @jamiesnape @EricCousineau-TRI @stonier
I am also blocked on this, so faster responses would be greatly appreciated.

https://github.com/RobotLocomotion/drake-external-examples/blob/master/scripts/setup/linux/ubuntu/xenial/install_prereqs duplicates Drake's list of dependencies. This is a maintenance burden. We need to find a way to allow for everyday dependency changes to Drake without PR'ing to two separate places.

Conceivably, more intricate changes such as bumping Bazel, or changing the compiler, could have duplicated logic. But "Add system libfoo-dev" should not require a separate PR here.

Get this working. It's a common use case for a drake developer.

In the interest of keeping it as simple as possible, I'd like to have these working on drake_cmake_installed only since it is the exemplary case. Move what scripts are relevant and document there.

We should probably also exemplify how to pull in an installed drake and use it inside a python project.

@sammy-tri are there pure python only users of drake?

now that we've seen issues with finding resources (with FindResource) from external repos, I think it's important that we capture that in our example here. @siyuanfeng-tri was the one who battled through it most recently.

Hello,
I am using MAC OS X, and want to use drake_cmake_installed.
In the README file, there is an instruction to install drake binaries for UBUNTU but not for MAC.
I would like to install binaries from the source, so want to use the method similar to sudo make install in cmake but I don't know the equivalent method in bazel.

Is there any way for me to install binaries from the drake source by using a method similar to make install in MAC?

Should I follow this instruction below?

# 3) Manual Installation
# git clone https://github.com/RobotLocomotion/drake.git
# (mkdir drake-build && cd drake-build && cmake -DCMAKE_INSTALL_PREFIX=/opt/drake ../drake && make)

# 4) Manual Installation w/ Licensed Gurobi
# Install & setup gurobi (http://drake.mit.edu/bazel.html?highlight=gurobi#install-on-ubuntu)
# git clone https://github.com/RobotLocomotion/drake.git
# (mkdir drake-build && cd drake-build && cmake -DCMAKE_INSTALL_PREFIX=/opt/drake -DWITH_GUROBI=ON ../drake && make)

@jamiesnape -- can't tell you how excited i am to see the progress in this repo. it's looking great and will be a huge contribution. thank you!

i have a question/proposal that i asked @stonier and @jwnimmer-tri yesterday: what if we were to zap this repo, and instead have separate repos titled drake-bazel-external, drake-cmake-external, and drake-cmake-installed? those directories would have the travis.yml file in their root (the right spot), etc, and would therefore make it even simpler for a user to get started. they could essential clone the repo, push to their new repo, and be on travis/circle immediately. I admit it's only moderately harder to clone this repo, then copy only one subdirectory, move the travis.yml file, etc, but I think that difference gets bigger for novice users.

The downside @stonier articulated is that some of the scripts are being shared between the subfolders. But I would prefer to increase the cost slightly of us maintaining 3 repos if it can decrease the cost to a novice user.

Side benefit: the repo names will make sense to everyone. ;-)

What do you think?

I was debugging linker errors on a different project which uses drake as an external and discovered that building with clang++ solves the linker issues (will submit this as a separate issue once I understand it better) but introduces a libprotobuf exception at runtime.

This is on a fresh install of Ubuntu 16.04. This problem does not occur on macOS.

to reproduce (following the README.md instructions in drake_cmake_installed):
mkdir build
cd build
cmake -Ddrake_DIR=$HOME/drake/build/lib/cmake/drake -DCMAKE_CXX_COMPILER=/usr/bin/clang++ ..
which produces the following output:

-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is Clang 3.8.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/clang++
-- Check for working CXX compiler: /usr/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find GTest (missing:  GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) 
-- Could NOT find GTest (missing:  GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/momap/exp/sham/drake_cmake_installed/build

then make -j and cd src/particles followed by ./uniformly_accelerated_particle_demo which results in:

[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:58] File already exists in database: google/protobuf/any.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1394] CHECK failed: generated_database_->Add(encoded_file_descriptor, size): 
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: generated_database_->Add(encoded_file_descriptor, size): 
Aborted (core dumped)

The whole pipeline works fine if you omit the -DCMAKE_CXX_COMPILER=/usr/bin/clang++ and I wouldn't care - except my other project won't link if I don't use clang.

An example failure in my custom project:

[ 81%] Linking CXX executable test_ik
libdracula.a(constraint_solver.cc.o): In function `ConstraintSolver::getJointNames[abi:cxx11]()':
constraint_solver.cc:(.text+0x2291): undefined reference to `RigidBodyTree<double>::get_position_name(int) const'
collect2: error: ld returned 1 exit status

Thanks for the help! In the meantime I'll try and get the other project building without clang.

Need to have this run nightly to make sure that it tests against the latest drake build.

https://drake-jenkins.csail.mit.edu/job/RobotLocomotion/job/drake-shambhala/job/master/

To put under CI and be instructive about how to pull Drake into a CMake project. Pulling is via find_package or ExternalProjectAdd.

@RussTedrake would also like to show (I agree) a lightweight Travis build using a hosted tarball of the installation.

This is a pre-requisite of having #11 working and under test.

These add much noise and aren't useful from an instructive point of view. They are useful to provide complete coverage on all subprojects and the platforms they build on though.

In #81, we added the drake_bazel_external implementation. We should add CI coverage for it.

Russ mentioned that it might be worth pulling the underactuated textbook into this repository from Drake itself.

If we start doing this, then Shambhala starts to become as it's name suggests - the portal to getting started with Drake, not just an exemplary instance of how to build a package using Drake.

@RussTedrake this might be a good question to pose at the first Drake Developers meeting?

A package with a couple of simple programs to get up and going:

• Simple system plant demo
• Simple traffic car demo
• Simple autodiff'd car optimisation demo

Testing so it can be verified that it is continuously working?

The LICENSE file says "Copyright (c) 2017 Daniel Stonier". I believe this was done as TRI work-for-hire, so should have corporate copyright not personal.

If a ROS environment is sourced, with ros-kinetic-octomap installed, then Drake's octomap module will fail to find it's own (v1.7.2) and abort after finding ROS' (v1.8.1). This leaves the octomap::octomap target empty.

From many of those generic rules in drake - e.g. github.bzl. Akin to bazel_tools. Drake is a heavy dependency just to bootstrap some rules.

The https://github.com/RobotLocomotion/drake-external-examples/blob/master/drake_cmake_installed/README.md instructions point users to https://drake-packages.csail.mit.edu/drake/nightly/drake-latest-xenial.tar.gz. However, it appears that even if the Nightly CI fails, the package is still uploaded and the "latest" link changed to use it? That means users will see broken nightlies, which seems awful.

Is it easy to change the build to not bump the "latest", or to have a different "latest-passing" or something? Or worst case for now just don't upload anything?

I just had a user try to use the broken nightly and have to ask for help.

Failure here: https://drake-jenkins.csail.mit.edu/job/RobotLocomotion/job/drake-shambhala/job/PR-35/10/console

Analyzing: target //apps:simple_continuous_time_system (10 packages loaded)
ERROR: /home/ubuntu/.cache/bazel/_bazel_ubuntu/4ec442eb70ae159fd7ea51f91a92aa37/external/drake/drake/common/BUILD.bazel:52:1: no such package '@stx//': The repository could not be resolved and referenced by '@drake//drake/common:essential'

This popped up due to the change in Drake #7241.

Stx got shifted to utilising a new_local_repository approach. This can't be replicated when drake is brought in as an external because the path to those headers can't be retrieved.

It's possibly also not the correct thing to do - the path for a new_local_repository is supposed to absolute according to the docs, though obviously the bazel executable doesn't actually back that up. I am guessing that the spirit of this mechanism is for experimenting with dependencies on your local system, not for managing sandboxed sets of files within the project.

(Forked off from #37.)

While I love the MIT license better than BSD, the fact that Shamb is MIT and Drake is BSD means that when we copy code between them, we are going to have to fiddle with license grant badging in annoying ways.

Given the tightly-coupled relationship between Drake and Shamb, please consider having Shamb change to use the BSD license ASAP, to reduce developer friction.

testing with the drake_cmake_installed workflow on mac. it almost works. is this something simple?

russt$ src/simple_continuous_time_system/simple_continuous_time_system
dyld: Symbol not found: __ZN8ignition4math5Angle4ZeroE
  Referenced from: /Users/russt/drake-build/install/lib/libdrake.so
  Expected in: /Users/russt/drake-build/install/lib/liblcm.so
 in /Users/russt/drake-build/install/lib/libdrake.so
Abort trap: 6

Do we have a guarantee that this test is run on CI?
If not, can we require it?

I ask because I had copied from find_resource.cc.in for test_find_resource.py for #85, and then seemed to have run into RobotLocomotion/drake#7809

When I tried building / running the test on my machine, it reports GTEST_NOT_FOUND, even though I have libgtest-dev installed.

\cc @jamiesnape @fbudin69500

For all of the packages just illustrating use with drake.

Pick something very core that isn't likely to change and build an app around that.

• readme for find_resource example
• update the root readme
• caveats about the find_package(drake) method
• caveats about linking with Drake & PCL
• not supporting Mac yet, list up what problems we have

Continuous, Nightly

Or some form of it - the external add project would probably be sufficient.

EDIT (eric): Helps remove delay when checking build interface + consumed APIs.

It looks innocuous, but running it on my desktop (xenial) forced an install of gles versions of opengl everywhere, including qt, which then resulted in blowing away most of KDE.

While not important on CI, some people are likely to try and use those prereqs as a guide and we're going to have to consider instructions to install pre-requisites in README instructions anyway.

Haven't root caused it yet.

Similar to #6 but with ExternalProject_Add instead of find_package.

Only caveat is that it should be ExternalProject_Add(drake … SOURCE_DIR "${CMAKE_BINARY_DIR}/drake-src" …) so multiple build types can work.

I am trying to make pendulum example work and here is the modified code: pendulum example, but got Segmentation fault (core dumped)

I am running drake visualizer &. I am running docker ubuntu from archlinux. pendulum example works with bazel build.

I used gdb and got that:

Program received signal SIGSEGV, Segmentation fault.
0x00000000009ced86 in drake::TypeSafeIndexdrake::systems::SystemConstraintTag:
:AssertValid (this=0x0, source=0xb29389 "Converting to an int.")
at /home/drake/drake-build/include/drake/common/type_safe_index.h:420

which was invoked from drake-build/include/drake/systems/framework/diagram_builder.h in function ThrowIfAlgebraicLoopsExist() line 303

 // Populate more edges based on direct feedthrough.
    for (const auto& system : registered_systems_) {
      for (const auto& pair : system->GetDirectFeedthroughs()) {
        PortIdentifier src_port{system.get(), pair.first};
        PortIdentifier dest_port{system.get(), output_to_key(pair.second)};
        if (nodes.count(src_port) > 0 && nodes.count(dest_port) > 0) {
          // Track direct feedthrough only on port pairs where *both* ports are
          // connected to other ports at the diagram level.
          edges[src_port].insert(dest_port);
        }
      }
    }

• How can I solve this error?

thanks

The README says:

https://github.com/RobotLocomotion/drake-shambhala/blob/5e8a897c0f53e4f36e777466aa7d9dd38b86b828/drake_cmake_installed/README.md#L47-L49

... but I think we need a cd drake-shambhala/drake_cmake_installed between the git clone and the mkdir build for that to work? At least, I needed it.

So that it is noticed when things break, at least nightly.

Currently failing on the master branch. See here - not finding make_unique (c++14 feature).

Since PCL is a very common library, it should be incorporated into some downstream project as an example.

The steps I can think of at the moment:

• Incorporate a custom-built Drake-compatible version of PCL that can be used in Bazel (as an archive) - see #37 for a prototype. Focus is first on Xenial, possibly on Mac thereafter.
• Incorporate a custom-built Drake-compatible version of PCL within the CMake external (preferrably cmake_install, using the existing archive that Bazel consumes)
• Once Drake is compatible with a more widely distributed ROS- / Ubuntu-package, incorporate this into CI.
• Incorporate into drake/examples
• Incorporate into drake/ if needed

\cc @RussTedrake @sammy-tri

Not sure if this is the right idea or not, but a drake-latest.tar.gz would automate the CI checks here so that we are not drifting away from Drake's latest releases.

Should illustrate:

• bazel library
• bazel app
• bazel test

Created a new master branch to replace the existing devel so that it aligns with the convention used by drake.

@jamiesnape I suspect jenkins/circle/travis need to be updated to follow suit before deleting the devel branch.