pdmlab / docker-compose Goto Github PK
View Code? Open in Web Editor NEWManage Docker-Compose via Node.js
Home Page: https://pdmlab.github.io/docker-compose/
License: MIT License
Manage Docker-Compose via Node.js
Home Page: https://pdmlab.github.io/docker-compose/
License: MIT License
Make alpine
service blocking to not finish container after starting or use other base image with blocking call.
I'm having difficulty using compose.exec. Part of this is because I think the docs are wrong. Here's what docs say:
exec(container, command, options)
However, you definitely have to include information about the docker-compose.yaml. This at least runs:
compose.exec('test_db',
psql, {cwd: path.join(initDir, 'pushkin'), config: 'docker-compose.dev.yml'}, '-U postgres',
-c 'create database test_db')
However, I get the following error:
'psql: FATAL: role "root" does not exist\n'
So it looks like the -U tag is being ignored. I also tried doing it this way:
compose.exec('test_db',
psql -U postgres -c 'create database test_db', {cwd: path.join(initDir, 'pushkin'), config: 'docker-compose.dev.yml'})
with the same result. Here's the critical part of my docker-compose:
test_db:
image: 'postgres:11'
environment:
POSTGRES_PASSWORD: example
Since I didn't set a root user, docker goes with the default 'postgres'. However, trying to set the user to 'root' didn't help:
test_db:
image: 'postgres:11'
environment:
POSTGRES_USER: root
POSTGRES_PASSWORD: example
Files and folders to exclude:
.idea
(JetBrains).vscode
(VS Code).circleci
(CircleCI)Add .npmignore
to exclude tests, ESLint and EditorConfig
I would like to set other child_process.exec options on the docker compose calls. Specifically the env
object so I may set my docker-machine environment variables.
As per: https://nodejs.org/api/child_process.html#child_process_event_close
Child processes expose their exit codes when closing/exiting. It might be preferable to use exit codes returned from docker-compose
calls to determine whether command has run OK or failed than trying to determine the result from stdout/stderr contents.
EDIT: or consider exit code different than zero to be a condition for rejection of the promise.
Hi folks ๐
as you know sharing is caring and we want to deliver more value to our users by not just providing this little module but also share examples of usage of the docker-compose
module (not docker-compose
itself) for several platforms like GitLab CI, Jenkins, Azure DevOps and so on.
If possible it would be great if you can contribute describing your use case like integration or e2e testing and a .gitlab-ci.yml
, Jenkinsfile
or whatever.
We're looking forward to see what you're building ๐ using docker-compose
module.
Please send a PR including a .md
-file named <yourusername>.md
in the folder community-samples
where you're describing how you're using your CI/CD using the docker-compose
module.
Also please reference your <yourusername>.md
in the readme.md
inside the community-samples
folder.
Thanks all ๐
โโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Moderate โ Denial of Service โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Package โ js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Dependency of โ composefile [dev] โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Path โ composefile > parser-yaml > js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ More info โ https://nodesecurity.io/advisories/788 โ
โโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Moderate โ Denial of Service โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Package โ js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Dependency of โ composefile [dev] โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Path โ composefile > write-yaml > js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ More info โ https://nodesecurity.io/advisories/788 โ
โโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Moderate โ Denial of Service โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Package โ js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Dependency of โ eslint [dev] โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Path โ eslint > js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ More info โ https://nodesecurity.io/advisories/788 โ
โโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ High โ Code Injection โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Package โ js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Dependency of โ composefile [dev] โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Path โ composefile > parser-yaml > js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ More info โ https://nodesecurity.io/advisories/813 โ
โโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ High โ Code Injection โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Package โ js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Dependency of โ composefile [dev] โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Path โ composefile > write-yaml > js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ More info โ https://nodesecurity.io/advisories/813 โ
โโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ High โ Code Injection โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Package โ js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Dependency of โ eslint [dev] โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Path โ eslint > js-yaml โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ More info โ https://nodesecurity.io/advisories/813 โ
โโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
found 6 vulnerabilities (3 moderate, 3 high) in 379 scanned packages
Blocked by PDMLab/composefile#1
mocha
has a nice one:
https://github.com/mochajs/mocha/blob/master/CHANGELOG.md
Can I just pass in a yaml string instead of yaml file name(s)? Because my config yaml is dynamic generated and no need to be saved.
Add support to stopOne() command as it is supported upOne or restartOne
This PR Implaments Streaming API and class API
const cwd = process.cwd()
const dockerCompose = require('docker-compose/class')
const compose = new dockerCompose({ cwd })
compose.upAll()
compose.upAllStream()
compose.logsStream()
Will Write more soon and Push my Integration for this module
I'm using this project to create a CLI that wraps some docker-compose stuff to spin up a local environment. It would be nice to be able to have a command that would stream docker-compose logs.
Would this be possible, or is there an alternative you recommend?
Hi, thanks for providing this great library! I'm incorporating it into https://github.com/testcontainers/testcontainers-node and it works great, I just have a couple of questions:
As added to express-http-problem-details recently
@Steveb-p thoughts?
The current implementation exposes very low-level information on the result of a command with the interface
export interface IDockerComposeResult {
exitCode: number | null;
out: string;
err: string;
}
This has the disadvantage of leaving the interpretation of the output to the user, leading in a lot of boilerplate code that all users of docker-compose have to write. Instead it would be better to provide more introspective into the result by typing it:
export interface IDockerComposeResult<T> {
exitCode: number | null;
out: string;
err: string;
result: T | null;
}
This should be given for every mapped subcommand of docker-compose. An example for the port
subcommand:
interface IDockerComposePortResult {
address: string;
port: number;
}
If an error occurs, the result should be null, since the error output will already be present in the err
key.
We should use github's project to track, which subcommands have already been enhanced.
Originally posted by @stefanmeschke in #52 (comment)
I have a use case where I plan to use this package to startup a collection of services for development, based on what I am working on. It would be nice to not send '-d' to the command.
I could see this being an option on the options object:
option.interactive: true
where the '-d' being passed on each command would be skipped.
Would this be a valid change?
Instead of forcing the file name to be docker-compose.yml
, arbitrary file names should be allowed.
This seems to be neccesary: https://circleci.com/docs/2.0/docker-compose/
when using, "buildOne"/"run" (maybe more commands), the output response from the promise returns { exitCode: NUMBER, "err": "string", "out": "string }.
the "err" that gets printed is not actually an error when exiting on code.
example:
{ exitCode: 0,
err: 'Building docker-example\n',
out:
'Step 1/13 : bal bal bla ....' }
{
exitCode: 0,
err: 'Creating network "docker-example" with the default driver\n',
out: 'docker-example-test\n' }
So while i was implementing an npm build system for my docker-compose project i had this idea of changing existing API for this module (or adding it on top of an existing one) to make this module more OOPeish and to avoid repetion of cwd
, log
and other parameters.
Would you be interested in something like this?
const DockerCompose = require('docker-compose')
// DockerCompose.method will still be a thing (legacy support), but the main course is
const compose = new DockerCompose({
cwd: '.',
log: false,
config: 'docker-compose.yml'
})
await compose.build()
await compose.up()
const container = 'node'
const command = 'npm install'
await compose.exec(container, command, {
log: true, // will override initial constructor settings
workdir: './var/www' // cli args for exec and other commands will be also supported
})
Add recently added #90 pullOne
, pullMany
, pullAll
to readme.md command list.
docker-compose
allows us to call docker-compose config
and receive configuration yaml as a result (for it's "main" functionality). This is different than just merging yaml files since all environment variables are replaced accordingly as docker-compose does (also taking into account possible .env
files I believe).
This command also allows for viewing service lists when called with --services
options, volumes with --volumes
option and container hashes (as I understand it) with --hash="*"
option (filtered as well).
From my point of view, config
call would allow one to see what services are available, even if they have not yet been created (since those would not show up in docker-compose ps
).
I'll gladly provide a PR for it, since I'll be probably using this functionality in my own application.
A docker-compose.yml
allows to forward a service port to an undefined, ephemeral host port:
version: "3"
services:
database:
build: .
ports:
- "3000"
(reference: https://docs.docker.com/compose/compose-file/#ports)
The forwarded host port can be retrieved by the port
subcommand: https://docs.docker.com/compose/reference/port/
Currently if I run an up method with the following command option:
commandOptions: ['--abort-on-container-exit']
I'll receive the following error message:
{ exitCode: 1, err: '-d cannot be combined with --abort-on-container-exit or --attach-dependencies.\n', out: '' }
This is because upAll, upMany, and upAll all always pass the -d
flag.
To make --abort-on-container-exit
usable we have it replace -d
when the user passes --abort-on-container-exit
as a commandOption.
Happy to open up a PR for this.
Here is an example of a script where I want to run a yarn install from a specific workdir on my container
docker-compose run --workdir="/app/backend" node yarn install
The equivalent would be
await dockerCompose.run('node', 'yarn install', {
composeOptions: [['--workdir','/app/backend']]
});
But this call will generate a call with bad argument order, passing the workdir to the command inside the container and not to the container itself
docker-compose run node yarn install --workdir="/app/backend"
Is there a workaround I don't get or is it a bug ?
Create a dedicated docs folder/document describing usage of each function in detail with examples.
I'm sitting here with a very, very bad internet connection and without docker images preloaded on my laptop (I'm using a different one that usually) and I've noticed that there are two different versions of MongoDB service being pulled - one 3.4
and second 3.4.0
. The same goes for alpine, but considering it's a lot smaller...
Anyway, I think we should use the same images in all tests?
EDIT: Also preferably we could use smaller "service" image instead of Mongo? Like nginx:alpine? (150 MB~ish worth of docker layers would become around 7 MB?)
Examples:
const options = [["--build"], ["--timeout", "30"]]
await compose.upAll({cwd: __dirname, log: true, options});
const options = ["--build", ["--timeout", "30"]]
await compose.upAll({cwd: __dirname, log: true, options});
debug
~> 2.6.9
lodash
~> 4.17.5
Currently, TypeScript typings are ignore in the local test/index.js
and WebStorm and VS Code won't recognise the docker-compose
methods.
When supplying log: true
in a commands options the output only logs when the command is finished. In my case that can be a few minutes before it's done. It would be nice to see the progress in real time.
I made a pull request #13 that pipes the output to the process, but another alternative would be to just return the ChildProcess object on all the calls instead of the stdout and stderr in an object.
If child-process-promise was used it could returns a modified Promise with childProcess as an extra field.
I'm using this module in typescript and the typing is not match to actual function calls.
ensure run and exec are working
test is broken because the mongo:3.4
image has matured from debian
to ubuntu
.
Solution: pin mongo
version to 3.4.0
in test/docker-compose.yml
I tried to run docker-compose with buildkit by adding environment variables:
const options: dockerCompose.IDockerComposeBuildOptions = {
cwd,
env: {
COMPOSE_DOCKER_CLI_BUILD: '1',
DOCKER_BUILDKIT: '1',
},
log: true,
};
await dockerCompose.buildAll(options);
but it throws error:
Error: spawn docker-compose ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:268:19)
at onErrorNT (internal/child_process.js:468:16)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
Why?
As reported by Jins George on slack:
Hello, I am trying to use docker-compose in a hapijs project running on node 6.3.1 and running into following error. Any idea what might be the issue?
lab --timeout 60000 -C -v -l -c -r lcov -o ./test-reports/coverage/lcov.info test/*.js
/opt/dpp/node_modules/docker-compose/index.js:235
);
^
SyntaxError: Unexpected token )
at Object.exports.runInThisContext (vm.js:76:16)
at Module._compile (module.js:513:28)
at Object.require.extensions.(anonymous function) [as .js] (/opt/dpp/node_modules/lab/lib/coverage.js:37:28)
at Module.load (module.js:458:32)
at tryModuleLoad (module.js:417:12)
at Function.Module._load (module.js:409:3)
at Module.require (module.js:468:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/opt/dpp/test/00project.test.js:6:17)
docker compose version=0.17.0
Immediate resolution was removing of trailing commas in buildAll
, buildMany
and push
functions.
The typescript interfaces of options are not exported anymore, making it impossible to allow type-checking in compositions:
/// /dist/index.d.ts
interface IDockerComposeOptions {
cwd?: string;
config?: string | string[];
log?: boolean;
composeOptions?: string[] | (string | string[])[];
commandOptions?: string[] | (string | string[])[];
env?: NodeJS.ProcessEnv;
}
[...]
See the following function signatures:
exec(container, command, options)
run(container, command, options)
Both use container
for their first argument name. In the docker compose documentation/help command these are refered to as service
. It would be benefical if argument name choice as well as the README documentation was consistent with the docker-compose documentation.
This might be nit picky, but when I first came across this npm module I wasn't sure if the container
argument actually meant the container's name/id generated by docker-compose. I expected it to be the service name (which it is), but from the README and the function argument it self I had to check.
When the docker-compose command has a lot of output the maxBuffer is easily exceeded on the child process.
I would love to be able to run docker-compose build
with this library.
// A single service
build('my_service')
// Multiple services
build(['my_service', 'my_other_service'])
// All services
build()
Bonus could be adding the --build
flag to the up()
function.
What do you think about TypeScript?
If you would be willing to have the source written in TypeScript I would be willing to make a pull request with the source converted to TypeScript.
or
If you would be willing to accept a pull request with typings I would be willing to maintain them for this repository.
I'm attempting to do something fairly simple.
We have some integration tests that were running against local images of MySQL DBs.
I've dockerized this with a docker-compose file that looks like this
version: '3.1'
services:
db:
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
command: --explicit_defaults_for_timestamp=ON
ports:
- 3306:3306
restart: always
volumes:
- ./scripts/db_scripts:/docker-entrypoint-initdb.d/b.sql
environment:
MYSQL_ROOT_PASSWORD: example
TZ: UTC
adminer:
image: adminer
restart: always
ports:
- 8080:8080
and have tried to do something like this in my DbSetup file (which I would like to set up the docker container for DB tests)
export default class DbSetup {
private static rootDirectory: string = path.join(__dirname, '../../..');
private static dockerComposeOptions: compose.IDockerComposeOptions = {
cwd: DbSetup.rootDirectory,
log: true,
};
public static async setup(): Promise<void> {
// Protection against running tests in non-local environment
if (process.env.DB_HOST !== 'localhost') {
throw new Error('DB tests can only be run against localhost');
}
await compose.upAll(DbSetup.dockerComposeOptions);
}
This waits for the container to spin up, but it does not wait for the Db to be available to accept connections. Is there a way to await for some sort of health check to pass as well before resolving the promise?
Hi!
Usage rm:
rm [options] [SERVICE...]
Why not?
export const rm = function (options?: IDockerComposeOptions, container?: string): Promise<IDockerComposeResult>;
I need compose not only to remove containers but also volumes defined in the compose file.
So far i am not sure, how to properly pass -v, --volumes to compose.down()
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.