This broker was inspired by the cf-containers-broker. It supports Docker and Bosh-based service deployments. More details on the implemented Cloud Foundry contract can be found here. Read the Big Picture behind Service Fabrik Broker.

1. Local Development Setup
2. Installing Docker
3. Installing NVM
4. Installing Bosh Lite
5. Installing Cloud Foundry
6. Installing the Broker
7. Launch the Broker
8. Register the Broker
9. Upload Bosh Director Based Service Releases
10. Run a Service Lifecycle
11. How to obtain Support
12. Advanced Debugging

Certainly when you are a broker developer, but also if you are a service developer, you may want to work locally with the broker without packaging it into a Bosh release and deploying it. You can do so using Bosh Lite and starting and registering the broker locally:

The famous Docker will be required for a local start of the broker. You can avoid this by removing any Docker service defintion from the broker settings/configuration/catalog.

• Follow instructions at https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/

If you're using a Mac, we recommend to use Docker for MAC.

• https://download.docker.com/mac/beta/Docker.dmg

Post installation of docker make modifications to manage docker as a non-root user

NVM helps you manage node versions and isolated environments.

• Follow instructions at https://github.com/creationix/nvm/#install-script

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.3/install.sh | bash
source ~/.nvm/nvm.sh
nvm install node
nvm use node

Bosh Lite is a lightweight local development environment for BOSH using Warden/Garden containers in a Vagrant box.

• Follow instructions at https://github.com/cloudfoundry/bosh-lite/blob/master/README.md#install-bosh-lite
• Run vagrant ssh and then remove 8.8.8.8 from /etc/resolv.conf to avoid 5s timeouts
• If you want to work with BOSH2.0 and want to deploy bosh-deployment, then please follow the bosh-lite installation guide as described in https://bosh.io/docs/bosh-lite.

Well, you know what Cloud Foundry is, otherwise you wouldn't be interested into a service broker to begin with.

• Follow instructions at https://docs.cloudfoundry.org/deploying/cf-release/boshlite/create_a_manifest.html
• You may like to shorten the polling interval for asynchronous service instance operations in cf-release/bosh-lite/deployments/cf.yml:

properties:
  cc:
    broker_client_default_async_poll_interval_seconds: 1

• Next follow instructions at https://docs.cloudfoundry.org/deploying/cf-release/common/deploy.html to deploy Cloud Foundry
• If you run into strange errors using bosh and OpenSSL (when generating the manifest), update OpenSSL and patch the scripts:

wget http://www.openssl.org/source/openssl-1.0.1p.tar.gz
tar -xvzf openssl-1.0.1p.tar.gz
cd openssl-1.0.1p
./config --prefix=/usr/
make
sudo make install
# add 2>/dev/null to the bosh <command> calls in ./scripts/generate-bosh-lite-dev-manifest

• If you run into certificate errors from consul like "Unexpected response code: 500 (rpc error: failed to get conn: x509: certificate has expired or is not yet valid)". This means the certificates has expired and it is required to recreate the certificates. Follow the steps below:

  • Generate new certificates using the script cf-release/scripts/generate-consul-certs.
  • Copy the certificates in the relevant section of cf-release/templates/cf-infrastructure-bosh-lite.yml
  • Regenerate the cf-release deployment manifest
  • Now recreate the release, upload and deploy it as follows:

    bosh create release && bosh upload release && bosh deploy
    

• Follow instructions at https://docs.cloudfoundry.org/cf-cli/install-go-cli.html to install cf cli

• If you have installed bosh-lite using bosh-deployment, then you can deploy cf using the following commands, using cf-deployment.

git clone https://github.com/cloudfoundry/cf-deployment
cd cf-deployment
bosh -e vbox upload-stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent
bosh -e vbox update-cloud-config iaas-support/bosh-lite/cloud-config.yml
bosh -e vbox -d cf deploy cf-deployment.yml -o operations/bosh-lite.yml --vars-store deployment-vars.yml -v system_domain=bosh-lite.com

• Target, Login, Prepare Cloud Foundry Usage

cf api --skip-ssl-validation api.bosh-lite.com
cf login -u admin -p admin

Password will be different in case if you are using using cf-deployment.

export CF_ADMIN_PASSWORD=$(bosh int ./deployment-vars.yml --path /cf_admin_password)
cf auth admin $CF_ADMIN_PASSWORD

Now, sample orgs and spaces can be created to start using it.

cf create-org dev
cf create-space -o dev broker
cf target -o dev -s broker

• Clone this repo (assuming your working directory is ~/workspace)

cd ~/workspace

• Clone and setup fork and git-secrets ( requires hub and git secrets to be installed)

  sh -c "$(curl -fsSL https://raw.githubusercontent.com/cloudfoundry-incubator/service-fabrik-broker/master/bin/clone-for-development)"
  cd service-fabrik-broker
  git checkout -b my-new-feature
  # make code changes
  git push <github_username> my-new-feature
  

• Install dependencies

npm install

• Optional: To locally run all unit test

npm run -s test

Then, we need to upload the cloud-config required for service-fabrik on bosh.

For bosh-lite, you can upload cloud-config in the following manner:

cd templates
bosh –e bosh upload-cloud-config cloud-config-boshlite.yml

The file cloud-config-boshlite.yml can be found here: https://github.com/cloudfoundry-incubator/service-fabrik-boshrelease/blob/master/templates/cloud-config-boshlite.yml

For AWS, we need to update the vars-files for the cloud-config. The vars file to be edited is cloud-config-aws-vars.yml. It can be found in the templates directory. Once the vars file is filled with proper details, the cloud-config can be uploaded:

cd templates
bosh –e bosh upload-cloud-config --vars-store=cloud-config-aws-vars.yml cloud-config-aws.yml

The required files mentioned above can be found here: https://github.com/cloudfoundry-incubator/service-fabrik-boshrelease/tree/master/templates

Useful prerequisites: When working with the broker, install curl (sudo apt-get install curl), jq, and yaml2json.

If you need to change the settings.yml configuration you should copy the file and point the broker to your settings file via the environment variable SETTINGS_PATH.

# env vars you may like to set to different than these default values
# export NODE_ENV=development ## For bosh2.0, use the environment boshlite2, as the passwords and BOSH IP are different.
# cp $(pwd)/config/settings.yml $(pwd)/config/my-settings.yml
# export SETTINGS_PATH=$(pwd)/config/my-settings.yml
npm run -s start

Check endpoint with curl

curl -sk -u broker:secret -H "X-Broker-Api-Version: 2.9" https://127.0.0.1:9293/cf/v2/catalog | jq .

You have to do this only once or whenever you modify the catalog. Then of course, use update-service-broker instead of create-service-broker.

• Registration

cf create-service-broker service-fabrik-broker broker secret https://10.0.2.2:9293/cf # host IP reachable from within the Vagrant box
cf service-brokers # should show the above registered service broker
curl -sk -u broker:secret -H "X-Broker-Api-Version: 2.9" https://127.0.0.1:9293/cf/v2/catalog | jq -r ".services[].name" | xargs -L 1 -I {} cf enable-service-access {}
cf service-access # should show all services as enabled, cf marketplace should show the same

In order for the broker to provision bosh director based services, the releases used in the service manifest templates must be manually uploaded to the targetted bosh director. The catalog (config/settings.yml) only contains our own blueprint-service which we are using for internal development, testing and documentation purposes of the Service Fabrik. If you want to provision our blueprint-service as bosh director based service, follow the steps mentioned in blueprint-boshrelease repository.

You will need a Cloud Foundry application, let's call it my-app (see below). If you have no specific one, you can use our blueprint-app.

cf create-service blueprint v1.0-container my-service
cf bind-service my-app my-service
# take a look at the generated binding with cf env my-app
cf restart my-app # do this a.) to make binding information available in environment of the app and b.) to activate the security group created with the service
# verify the application sees the service; if you have deployed the above app, run curl -skH "Accept: application/json" "https://my-app.bosh-lite.com/test"
cf unbind-service my-app my-service
cf delete-service -f my-service

If you need any support, have any question or have found a bug, please report it in the GitHub bug tracking system. You can also reach us out on our Slack Channel.

Now you can start restarting your builds in Debug mode with an API request. To trigger a debug job:

1. Get the API token using the travis CLI and send a POST request to /job/:job_id/debug replacing the TOKEN and JOB_ID values below:

curl -s -X POST \
   -H "Content-Type: application/json" \
   -H "Accept: application/json" \
   -H "Travis-API-Version: 3" \
   -H "Authorization: token <TOKEN>" \
   -d '{ "quiet": true }' \
   https://api.travis-ci.org/job/<JOB_ID>/debug

The Job ID is displayed in the build log after expanding "Build system information".

2. Head back to the web UI and in the log of your job. you should see the following lines to connect to the VM:

Setting up debug tools.
Preparing debug sessions.
Use the following SSH command to access the interactive debugging environment:
ssh [email protected]

3. Connect from your computer using SSH into the interactive session, and once you're done, just type exit and your build will terminate.

Please note that when the debug build is thus initiated, the job will skip the remaining phases after debug. Also, please consider removing the build log after you've finished debugging.

Finally, once in the SSH session, these bash functions will come in handy to run the different phases in your build:

This project is licensed under the Apache Software License, v. 2 except as noted otherwise in the LICENSE file.