semaphoreci / docs Goto Github PK

Use case is when a user installs a Ruby version which is not preinstalled on Semaphore. This is a time-consuming operation since it involves download over network and compilation. They want to do it once and keep the stuff available in cache "forever". Let's add a recipe for that using cache to the Ruby guide. Over time we can add similar tip for other languages too.

How to install one, and any prerequisites, similar to the corresponding Ruby section. https://github.com/semaphoreci/docs/blob/master/language-ruby_5bb9e082042863158cc7236d.md#c-extensions--system-dependencies

We have some pieces in the Ruby guide, but this should be one full production-ready example that people can easily copy and modify. Based on a recent Uncut episode.

Based on https://github.com/renderedtext/issues/issues/1026.

The documentation would fit nicely in here https://github.com/semaphoreci/docs/blob/master/debugging-with-ssh-access_5bbcbae0042863158cc73802.md#exploring-the-build-environment.

A user on support recently outlined a complete scenario:

I have a Go 1.10 project, using MySQL for data storage.

• The first step should do some linting with gofmt, go-checkstyle.
• Then building a docker image.
• Next do unit tests.
• Finally the image should pushed to a Kubernetes cluster.

The value of the SEMAPHORE_WORKFLOW_ID environment variable is the ID of the workflow of the current job.

of the current job. part

It's what makes possible things like:

cache restore gems-$SEMAPHORE_GIT_BRANCH-$(checksum Gemfile.lock),gems-$SEMAPHORE_GIT_BRANCH-,gems-master-

And so since restore does partial matching, we should probably clarify that other commands do full matching.

Applies to caching reference + guided tour page.

A user reported:

In your Ruby docs you provide a demonstration on how to cache your gems in vendor/bundle and then use the BUNDLE_PATH environment in the next task to tell bundler where to find the dependencies. However I couldn’t get this to work at all like the docs described - I had to run bundle check --path vendor/bundle after restoring the cache to get bundler to recognise the dependency location. Is this an issue with the docs or something I’m doing?

You can use sem create -f to create a new job that will be running without being added to an existing Pipeline.

There is no example for that.

Then we talk about "on demand" jobs:

There is a quick way to create a job and executing it right way:

sem create job [name] --project [existing project name] --command "[Valid command]"

This sentence in S1 migration guide compresses too much info:

You can set up conditions for manual or automatic promotion that trigger other pipelines, and more.

Expand it a bit (can be a separate paragraph) and link to promotions guide.

There's a lot to digest and re-construct in your head as your read it. A diagram will help. Would include all building blocks, hints what's sequential and what's parallel, etc.

Example found on these pages:

markdown file don't
helpscout docs don\'t

If you just copy for example the production-deploy.yml from https://docs.semaphoreci.com/article/67-deploying-with-promotions, your pipeline with fail (missing version, maybe also name).

Scenario: my project is using some Docker images as a starting point. For example my service runs on golang:alpine parent image and also connects to a mongodb base image via docker compose. I don’t want to pull my parent or dependency images from scratch in every workflow. These images should stay in cache "indefinitely".

This one comes up on support occasionally and would be a useful demo and addition to the Use cases section.

Scenario

1. Scan changes in current revision and determine which subcomponents of our app need to be rebuilt.
2. Launch parallel jobs that execute a bin/ci script that will run the test suite for each affected subcomponent.

Solution outline

1. In the first block of a pipeline, run a script to determine which subcomponents need to be tested. Store that info in cache.
2. In the second block, define jobs for all subcomponents. At the start of each job fetch data created in the first step, check whether current subcomponent should be tested and run ci script only if it’s true.

It’s a demo so subcomponents can simply be two subdirectories.

• (Only) Node version v8.11.3 is installed and set as default. Alias 8.11.
• nvm is installed, version 0.33.2
• nvm ls gives a long lis, basically every version can be installed minus the latest one via nvm install <version>
• yarn dependency manager is installed, version 1.10.1
• bower package manager is also installed, version 1.8.4

Current Python docs show use of pipenv, which is fairly new and not preinstalled.

Pip is preinstalled and common so let's have show how to use pip in dependency caching section first, before pipenv.

This will help people get green Semaphore pipeline right away for their Rails apps.

"Use Cases" category will be the home of all hands-on examples that show how to apply S2's general features to do something with XYZ technology.

Kramdown does not format codeblocks in the way we want.

For example, we sometimes receive a message like

I've been trying to add a project to Semaphore. Unfortunately, using sem init appears not to work appropriately, since on both my machine, as well as the machine of a Git admin, running sem init fails with "{"message":"admin permisssions are required on the repository in order to add the project to Semaphore"}".

Raw response:

In order to add a project to Semaphore 2.0, the user must be a member of your Semaphore 2.0 organization and also have Admin permission level for the project on GitHub.
Currently, Semaphore is "mirroring" permission rights from GitHub which means that a user will have the same level of permission on Semaphore 2.0 as on GitHub.

Also, if the person has Admin permission level on GitHub and is a member of your Semaphore 2.0 organization and still cannot add a project, could you please check if the access for Semaphore 2.0 was granted within your GitHub organization? You can check that here: https://github.com/settings/connections/applications/328c742132e5407abd7d.
If it has been granted, there should be a green checkmark next to your organization name. If there isn't one, that means that the access hasn't been granted for Semaphore 2.0 and you cannot add a project to it.

We'd add a potentially more concise explanation and any other similar FAQ to the guided tour.

checksum source code is in https://github.com/semaphoreci/toolbox

Use the Rails guide as a blueprint.

Let's complete #172 before starting work on this one. This would be a new take which would supersede #74, so please create a new pull request.

Scenario: I build Docker images based on my code in one block and then want to carry them to the next blocks as fast as possible. After we're done with testing phases and the image is promoted — let's say I've published my service/app to a registry — the image(s) are no longer needed in cache. So we can delete them from cache at the end of the workflow. This approach keeps cache use at a normal level.

Reading sem reference with fresh eyes is a difficult task without examples. There are examples, but they're very far down, so we ask the reader to make a lot of effort to go back and forth across a very long piece of text. Unless there was a strong argument for the current approach, let's not have a separate Examples section but merge the examples with the core reference.

Some details I noticed :).

• I'd say we don't need the word reference in the title?
• I don't know if it makes sense to make S capital in Sem command line tool title?
• Not sure about capitalizing letters in titles

Again, not sure if it makes sense to stick to rule of the capital letter in the first bullet point and the small letter in the second.

See also part suggests also the articles which aren't yet published so we should check if this can be fixed anyhow.

These are just details which I was only able to make comments on due to lack of the technical knowledge :).

Mirror what came out in 0.8.12 and #140.

We have an article about it in Semaphore 1 docs: https://semaphoreci.com/docs/building-project-with-private-gems.html

The difference is that credentials in Semaphore 2.0 need to be stored in secrets.

The final page should be published in the Use Cases category.

When bundle install --path vendor/bundle is not included, execution of bundle exec rake tasks fail with message similar to the following:

Could not find rake-x.x.x in any of the sources
Run `bundle install` to install missing gems.

Command sequence should look like:

cache restore gems-$(checksum Gemfile.lock)
bundle install  --path vendor/bundle

This came up a few times on support. Let’s make it more clear that a database or any service isn't shared between blocks and users need to start them within each block and migrate / populate data.

Pages to update:

• https://docs.semaphoreci.com/article/32-ubuntu-1804-image#databases-and-services
• https://docs.semaphoreci.com/article/65-customizing-the-build-environment

We had several similar questions in the community and support.

So far I am directing them to https://docs.semaphoreci.com/article/66-environment-variables-and-secrets.

Opening the issue because this is a very common use case that is not addressed directly in the docs.

Semaphore Classic autogenerates this file for every Ruby on Rails project. Let's make transition easier by having an example they can copy and paste.

Currently, all URLs in docs do not have target="_blank". Please make redcarpet do that.

Simple application of cache. The point is to download once, and then in all future workflows avoid network roundtrips which can vary. In the example let's use a .deb package.

A simple recipe, including how to set up secrets and a promotion, eg:

Released today with v0.8.12

We currently send something like this via support:

docker run --net=host --rm -d -e MYSQL_ALLOW_EMPTY_PASSWORD=true --name mysql mysql:5.7

This sets up MySQL listening on 0.0.0.0:3306.

We should have a verified example for both MySQL and PG. Verification should include connecting and using the provisioned database (but this shouldn't be in the recipe).

Depending on final length we can add this either to the Ubuntu reference or create a new page.

Sharing links to subsections of a docs page is a common action while doing support.

Currently, we need to "right click > inspect > copy id > paste into urlbar".

Can use the app from #165 as an example. The Heroku page can then be connected as one of the possible next steps to any web framework use case.