labzero / bootleg Goto Github PK
View Code? Open in Web Editor NEWSimple deployment and server automation for Elixir.
License: MIT License
Simple deployment and server automation for Elixir.
License: MIT License
I'm trying to start run custom task via mix bootleg.invoke
and getting nothing. So, this is the strategy:
# lib/bootleg/tasks/other.ex
defmodule Bootleg.Tasks.Other do
use Bootleg.Task do
task :other do
Bootleg.UI.info("World?")
end
end
end
@spec invoke(atom) :: :ok
def invoke(task) when is_atom(task) do
IO.inspect("Invoking task #{inspect task}")
invoke_task_callbacks(task, :before_hooks)
module_name = module_for_task(task)
IO.inspect("Module name is #{inspect module_name}")
if Code.ensure_compiled?(module_name) do
IO.inspect("Module is loaded")
apply(module_name, :execute, [])
else
IO.inspect("Module is NOT loaded")
end
invoke_task_callbacks(task, :after_hooks)
end
$ mix bootleg.invoke other
"Invoking task :other"
"Module name is Bootleg.DynamicTasks.Other"
"Module is NOT loaded"
$
phoenix_digest
from bootleg_phoenix package is working:$ mix bootleg.invoke phoenix_digest
"Invoking task :phoenix_digest"
"Module name is Bootleg.DynamicTasks.PhoenixDigest"
"Module is loaded"
....
$
other
is also working via iex shell:iex(1)> Bootleg.Config.invoke :other
"Invoking task :other"
"Module name is Bootleg.DynamicTasks.Other"
"Module is loaded"
World?
:ok
iex(2)>
I'm totally missed in this reality, hope my debuggings will help you with fixing this bug.
I believe there is sufficient externally useful bits in UI now after @Virviil's prompt additions that it might be useful to have some documentation around the helpers available in UI. In fact, this might also be a good opportunity to break out the existing README docs for hooks/tasks/invoke/remote/ui into a separate doc file which could cover customization as a whole, maybe something like CUSTOMIZATION.md
Currently there is no way to tell exactly what tasks/hooks will get executed when a particular task is invoked, especially when another package is providing some of those tasks for you. The bootleg mix tasks would support a --dry-run
flag, which would cause bootleg to load and parse its config files like normal, and then "run" the tasks with remote
turned into a no-op (or maybe patch into SSHKit's dry run functionality). During the run, it would print out each task/hook as they are entered/exited.
The flexibility of the DSL is a bit of a disadvantage here, as things like invoke
can be conditionalised on the results of a remote
call. Doing AST traversal, it should be possible to detect those cases and maybe the output would reflect both branches? Or warn that the results are unknown (we don't want to follow too many conditional branches if the user has a bunch of conditional invoke
s).
This feature is probably going to need to rely on AST parsing instead of actual code execution to avoid side-effects like notifying APMs or touching files via system calls.
Example output:
$ mix bootleg.build --dry-run
Dry run...
task :build started
task :compile started
remote :build, MIX_ENV=prod mix deps.compile
remote :build, MIX_ENV=prod mix compile
task :compile ended
task :generate_release started
remote :build, MIX_ENV=prod mix release
task :generate_release ended
task :build ended
task :update do
invoke :build
invoke :deploy
invoke :start
end
If the app is already running, Distillery start
will not stop it and start it again on its own.
Unlike restart
, Distillery's start
and stop
will not throw errors if the running status is unexpected, so it's possible we can just add an invoke :stop
in our :update
task to fix the problem, but this may require some more work.
After upgrading to bootleg 0.5.0 the format of the output looks like the following. Notice the quotes look weird now. Some output has lines split across multiple lines of output as well, which may have been a problem before.
[MY_IP_ADDRESS ] removed �deps/postgrex/lib/postgrex/extensions/int4.ex�
[MY_IP_ADDRESS ] removed direct
[MY_IP_ADDRESS ] ory: �deps/postgrex/lib/postgrex/extensions�
I don't get any errors though like I did previously.
Is there a reason why there's an extra space after the IP address in all bootleg output?
[a.b.c.d ]
use Bootleg.Config
role :build, "build.example.com"
role :app, ["app1.example.com", "app2.example.com"]
task :do_on_all do
remote :all do
"echo This will echo on build, app1, and app2"
end
end
Create a shortcut rollup command that is the functional equivalent to running
mix bootleg.build
mix bootleg.deploy
mix bootleg.start
Proposed additional task would be called update
and would be exposed via a mix task of mix bootleg.update
I'm seeing a rather puzzling and opaque internal error:
$ mix bootleg.build production
Creating remote context at '/tmp/bootleg/build'
[MY_BUILD_HOST] (export REPLACE_OS_VARS="true" && /usr/bin/env mkdir -p /tmp/bootleg/build)
** (SSHError) SSHKit returned an internal error on MY_BUILD_HOST: {:function_clause, [{:public_key, :pem_entry_decode, [nil], [file: 'public_key.erl', line: 141]}, {SSHClientKeyAPI, :user_key, 2, [file: 'lib/ssh_client_key_api.ex', line: 102]}, {:ssh_auth, :get_public_key, 2, [file: 'ssh_auth.erl', line: 145]}, {:ssh_connection_handler, :is_usable_user_pubkey, 2, [file: 'ssh_connection_handler.erl', line: 1769]}, {:ssh_connection_handler, :"-init_ssh_record/4-lc$^0/1-0-", 2, [file: 'ssh_connection_handler.erl', line: 477]}, {:ssh_connection_handler, :init_ssh_record, 4, [file: 'ssh_connection_handler.erl', line: 476]}, {:ssh_connection_handler, :init, 1, [file: 'ssh_connection_handler.erl', line: 412]}, {:ssh_connection_handler, :init_connection_handler, 3, [file: 'ssh_connection_handler.erl', line: 374]}]}
lib/bootleg/ssh.ex:70: anonymous fn/2 in Bootleg.SSH.run/2
(elixir) lib/enum.ex:1294: Enum."-map/2-lists^map/1-0-"/2
lib/bootleg/ssh.ex:163: Bootleg.SSH.run!/2
lib/bootleg/ssh.ex:87: Bootleg.SSH.validate_workspace/3
lib/bootleg/ssh.ex:51: Bootleg.SSH.init/2
lib/strategies/build/distillery.ex:9: Bootleg.Strategies.Build.Distillery.init/0
lib/strategies/build/distillery.ex:16: Bootleg.Strategies.Build.Distillery.build/0
lib/bootleg/config.ex:356: Bootleg.Config.invoke/1
I'm using and ed25519 key if that helps. This is a brand new server. Might you be able to offer some insight or means of debugging this error?
I am having some issues getting the assets for phoenix 1.3 working:
this is the default phoenix task in my deploy.exs:
after_task :compile do
mix_env = Keyword.get(config(), :mix_env, "prod")
remote :build do
"[ -f package.json ] && npm install || true"
"[ -f brunch-config.js ] && [ -d node_modules ] && ./node_modules/brunch/bin/brunch b -p || true"
"[ -d deps/phoenix ] && MIX_ENV=#{mix_env} mix phoenix.digest || true"
end
end
I can successfully build, deploy and start the server, however, the priv/static directory will remain empty and no assets are built. I can see the server running but without assets. I assume this is because in Phoenix 1.3 the package.json is in the /assets
directory. However adding "cd ./assets"
directly after remote :build does fail.
Anyone has a clue? Thanks
By default it would be ["*"]
, which would clean the entire workspace. A user or package could add additional locations or change the location to something more specific, say just priv/static
or something.
A location not being found would not be an error, so things like ["*", "priv/static"]
would just work. All relative paths would be relative to the workspace, and absolute paths would work as expected.
The proposed configuration name is clean_locations
.
So trying to symlink my prod.secret.exs and getting an error here before invoke :compile
, thus not allowing me to symlink/copy with the before_task :compile
task. So mix local.rebar --force
is the specific command looking for the prod.secret.exs file. I also tried after_task :clean
as well with no dice. Eventually I used replace_os_vars
in the prod secret file and removed from gitignore. If you think a PR could come out of this, I'd be happy to help. Otherwise, I can close.
** (SSHError) Command exited on 127.0.0.1 with non-zero status (1)
cmd: MIX_ENV=prod mix local.rebar --force
stderr: ** (Mix.Config.LoadError) could not load config config/prod.secret.exs
** (File.Error) could not read file "/tmp/bootleg/build/config/prod.secret.exs": no such file or directory
(elixir) lib/file.ex:272: File.read!/1
(mix) lib/mix/config.ex:180: Mix.Config.read!/2
(mix) lib/mix/config.ex:217: anonymous fn/3 in Mix.Config.read_wildcard!/2
(elixir) lib/enum.ex:1811: Enum."-reduce/3-lists^foldl/2-0-"/3
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
(stdlib) erl_eval.erl:878: :erl_eval.expr_list/6
(stdlib) erl_eval.erl:404: :erl_eval.expr/5
lib/bootleg/ssh.ex:169: Bootleg.SSH.run_result/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
lib/strategies/build/distillery.ex:24: Bootleg.Strategies.Build.Distillery.build/0
lib/bootleg/config.ex:356: Bootleg.Config.invoke/1
(mix) lib/mix/task.ex:301: Mix.Task.run_task/3
(mix) lib/mix/cli.ex:75: Mix.CLI.run_task/2
If you try to filter by attribute in remote/3
, the list of hosts will not be filtered. To reproduce:
use Bootleg.Config
role :app, "validhost.example.com"
[] = remote :app, [foo: :bar], "true"
This should not raise an error (or establish any SSH connections), instead it raises a match error and establishes a connection to validhost.example.com
.
Right now bootleg only supports a single config, found in config/deploy.exs
. This makes it hard to target different deployment environments (development, staging, UAT, production, etc) without lots of reliance on system ENVs.
To support multiple deployment environments, it makes sense to have another set of optional config files in config/deploy
, named for the particular environment (config/deploy/staging.exs
, config/deploy/production.exs
) that provide additional configuration for that environment. The config for that environment would be loaded after the main config/deploy.exs
file is loaded. In addition, the current environment would be accessible from the config agent via the key bootleg_env
.
Each bootleg mix
command would then take an optional target environment (defaults to production).
For example:
mix bootleg.deploy production
mix bootleg.build staging
This is separate from MIX_ENV
, as we need to target deploy environments separately from MIX_ENV
. For example, MIX_ENV
will likely be prod
for both production
and staging
bootleg environments.
If build server host is the same as app server host, copy release from :build
workspace to :app
workspace instead of uploading release from local.
For example:
remote :db, primary: true do
"rm -rf /"
end
mix bootleg.init
could create a basic template version of config/deploy.exs
use Bootleg.Config
# Configure the following roles to match your environment.
# `build` defines what remote server your distillery release should be built on.
# `app` defines what remote servers your distillery release should be deployed and managed on.
#
# Some available options are:
# - `user`: ssh username to use for SSH authentication to the role's hosts
# - `password`: password to be used for SSH authentication
# - `identity`: local path to an identity file that will be used for SSH authentication instead of a password
# - `workspace`: remote file system path to be used for building and deploying this Elixir project
role :build, "build.example.com", workspace: "/tmp/bootleg/build"
role :app, ["app1.example.com", "app2.example.com"], workspace: "/var/app/example"
# Phoenix has some extra build steps which can be defined as task after the compile step runs.
#
# Uncomment the following task definition if this is a Phoenix application. To learn more about
# hooks and adding additional behavior to your deploy workflow, please refer to the bootleg
# README which can be found at https://github.com/labzero/bootleg/blob/master/README.md
# after_task :compile do
# remote :build do
# "[ -f package.json ] && npm install || true"
# "[ -f brunch-config.js ] && [ -d node_modules ] && ./node_modules/brunch/bin/brunch b -p || true"
# "[ -d deps/phoenix ] && mix phoenix.digest || true"
# end
# end
In addition to the creation of the template config, perhaps we can print out some basic bootleg command usage information
In the case that a user does not need to build on a remote host, bootleg should provide a way to run a build locally without the need to ssh.
When I run mix bootleg.build
I pretty regularly (but intermittently) get an error similar to the following during the workspace cleanup step.
** (ArgumentError) argument error
(stdlib) :io.put_chars(:standard_io, :unicode, [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[...] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[MY_IP_ADDRESS ] "] | "\e[0m"], "removed ‘_build/prod/rel/myapp/lib/db_connection-1.1.2/ebin/db_connection.app’"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[MY_IP_ADDRESS ] "] | "\e[0m"], "removed ‘_build/prod/rel/myapp/lib/db_connection-1.1.2/ebin/Elixir.DBConnection.Sojourn.Broker.beam’"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[MY_IP_ADDRESS ] "] | "\e[0m"], "removed ‘_build/prod/rel/myapp/lib/db_connection-1.1.2/ebin/Elixir.DBConnection.App.beam’"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[MY_IP_ADDRESS ] "] | "\e[0m"], "removed ‘_build/prod/rel/myapp/lib/db_connection-1.1.2/ebin/Elixir.DBConnection.Sojourn.Supervisor.beam’"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[MY_IP_ADDRESS ] "] | "\e[0m"], "removed ‘_build/prod/rel/myapp/lib/db_connection-1.1.2/ebin/Elixir.DBConnection.Poolboy.Worker.beam’"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[MY_IP_ADDRESS ] "] | "\e[0m"], "removed ‘_build/prod/rel/myapp/lib/db_connection-1.1.2/ebin/Elixir.DBConnection.Pool.beam’"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[MY_IP_ADDRESS ] "] | "\e[0m"], "removed ‘_build/prod/rel/myapp/lib/db_connection-1.1.2/ebin/Elixir.DBConnection.Query.beam’"] | "\e[0m"], 10])
lib/bootleg/ssh.ex:90: Bootleg.SSH.capture/3
lib/sshkit/ssh/channel.ex:224: SSHKit.SSH.Channel.loop/4
lib/sshkit/ssh.ex:125: SSHKit.SSH.run/3
lib/bootleg/ssh.ex:70: anonymous fn/2 in Bootleg.SSH.run/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
lib/bootleg/ssh.ex:109: Bootleg.SSH.run!/2
deps/bootleg/lib/bootleg/tasks/build.exs:31: anonymous fn/3 in Bootleg.DynamicTasks.Clean.execute/0
I haven't taken the time to look into this yet, so forgive me if this is a ticket for sshkit or some other dependency.
In #164 we've discussed changing how filters are specified in the remote
macro. Given that it may take some time to get there, and it represents a breaking change, we should update the README to remove the filtering example, or at least call out how it may change in the (relatively) near future.
Similar to bootleg.ping
and other built-in commands.
Downloading a file from the remote host can be done by using Bootleg.SSH.download/3
, but its usage is undocumented and it requires a raw SSHKit.Context. Exposing this functionality via the DSL with proper role/host support is a much better alternative.
use Bootleg.Config
after_task :deploy do
# dumps to the remote workspace
download :app, "path/to/a/remote/file"
# or with an explicit destination
download :app, "path/to/a/remote/file", "local/path"
end
Uploading a file to the remote host can be done by using Bootleg.SSH.upload/3
, but its usage is undocumented and it requires a raw SSHKit.Context
. Exposing this functionality via the DSL with proper role/host support is a much better alternative.
use Bootleg.Config
after_task :deploy do
# dumps to the remote workspace
upload :app, "path/to/a/local/file"
# or with an explicit destination
upload :app, "path/to/a/local/file", "remote/path"
end
build_type: :docker
would run a build inside a docker container.
=== stream of conscience ===
Dockerfile name could be defined as config variable (or maybe via :build
role definition?)
could default to ./Dockerfile
or ./Dockerfile.build
docs should present any required configuration lines
The documentation should link to the appropriate source code.
As @pmeinhardt pointed out in [#213] it may be handy to have BOOTLEG_ENV
environment variable always set for remote commands
Currently this is supported by manually setting the environment variable on the :app
hosts, but it might be nice to be able to set this via a config setting. perhaps something as obvious as
use Bootleg.Config
config :replace_os_vars, true
alternatively, during role definition
use Bootleg.Config
role :app, ["host1", "host2"], replace_os_vars: true
Hi there 👋,
I was wondering how I could deploy a branch other than master
?
I saw a :refspec
option mentioned in the README, but adding that to the role :build
did not have any effect. Am I doing something wrong?
Thanks a lot ☀️
This is just a leftover from having first supported username and password.
Having our README examples define username and password is probably both a turn-off and encouraging of a bad practice, as deploy configs are likely checked into source control.
Identity file should be the default authentication mechanism.
Right now :generate_release
and :compile
are the only bundled tasks that use the config agent, but they should use it via the config/0
macro.
# messy
config()[:mix_env] || "prod"
# better
Keyword.get(config(), :mix_env, "prod")
# ideal
# ...[future]
I would like to symlink my prod.secret.exs files from outside the git repo after git clone but before a build. I was hoping there might be a git_clone task or something similar that I can create an after_task hook for. Does that sound like a reasonable feature to add? If there's a better way to handle secrets I'd love to hear.
Thanks, and I'm excited to see where the project leads. :)
When trying to build and deploy an application, I am getting the error below. Since this lib uses sshkit under the hood, I was wondering if anybody has encountered the same problems and got around it? Here is the related issue in sshkit.
Ref Issue: bitcrowd/sshkit.ex#88
** (SSHError) SSHKit returned an internal error on 165.227.58.185: 'User interaction is not allowed'
lib/bootleg/ssh.ex:70: anonymous fn/2 in Bootleg.SSH.run/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
lib/bootleg/ssh.ex:163: Bootleg.SSH.run!/2
lib/bootleg/ssh.ex:87: Bootleg.SSH.validate_workspace/3
lib/bootleg/ssh.ex:51: Bootleg.SSH.init/2
deps/bootleg/lib/bootleg/tasks/ping.exs:5: anonymous fn/3 in Bootleg.DynamicTasks.Ping.execute/0
(elixir) lib/enum.ex:1811: Enum."-reduce/3-lists^foldl/2-0-"/3
deps/bootleg/lib/bootleg/tasks/ping.exs:5: Bootleg.DynamicTasks.Ping.execute/0
When I implemented local_build
in #229 I think I had the short-term blinders on. After giving it a bit more thought I think this should be implemented as a more general build_type
config option that we can use into the future. Right now i see at least three use-cases:
build_type: :remote
- (DEFAULT) build using remote build host
build_type: :local
- run build locally
build_type: :docker
- run locally inside a docker container (to be implemented in the future)
I'm trying to write a custom task that has to cd into different directories and run commands. It does something like the following.
remote :build do
"cd apps/myapp_web/assets && yarn install"
"cd apps/myapp_web/assets && ./node_modules/brunch/bin/brunch b -p"
"cd apps/myapp_web && mkdir priv/static && MIX_ENV=#{mix_env} mix phx.digest"
end
The problem I'm running into is that these commands are prefixed with cd /path/to/workspace && /usr/bin/env
, so after the cd
completes I'm back to my original context.
$ pwd
/tmp
$ env cd dir1 && pwd
/tmp
I can't wrap my entire command in quotes since env will say it can't find the command (thinking the whole string is my command path). The only way I can think of to deal with this with the current code is to explicitly invoke my shell and pass in the commands, like bash -c "cd ... && ..."
. Alternatively perhaps the remote
block could take a custom directory to cd into within the build workspace? I'm not really sure what is ideal but I thought I'd provide the feedback anyway in case it is helpful.
There is currently nothing documented in the README about these macros. We should at minimum call out:
config :refspec, "branch"
config :app, "appname"
config :version, "1.0.0"
config :env, :production
config :ex_path, "path/to/elixir/proj/"
According to the doc if the :build
role is not set the release should be done locally. If I remove the role :build,...
line in config/deploy.exs
I get ** (ArgumentError) You must supply a %Host{}, a %Role{} or a defined role_name.
What is the correct way to do it?
Do you plan to add the ability to build a release by pulling code from repository (github..) instead by pushing local code ? (both would be useful)
Would directly invoke a specific task that has been defined by Bootleg or an bootleg task or extension packages
mix bootleg.invoke [task_name]
It seams that rollup updates are not supported. Any intention on supporting them?
Hi, I hope you don’t mind me bothering you 🙂 Thanks a lot for your quick replies!
I was wondering what your strategy for environment variables with bootleg is.
More specifically: I would like to pass a PORT
environment variable when starting an app. For now what I did is to override the :start
task in the environment – obviously a hack:
# config/deploy/production.exs
use Bootleg.Config
# …
task :start do
remote :app do
"PORT=20673 bin/#{Bootleg.Config.app} start"
end
Bootleg.UI.info "#{Bootleg.Config.app} started"
:ok
end
Unfortunately I don't have any control over what environment the system sets by default.
I bet there's a better way, maybe you can help me out?
I have a Phoenix umbrella project with two apps (myapp and myapp_web). At the top level of the repo my rel/config.exs looks like the following.
# Import all plugins from `rel/plugins`
# They can then be used by adding `plugin MyPlugin` to
# either an environment, or release definition, where
# `MyPlugin` is the name of the plugin module.
Path.join(["rel", "plugins", "*.exs"])
|> Path.wildcard()
|> Enum.map(&Code.eval_file(&1))
use Mix.Releases.Config,
# This sets the default release built by `mix release`
default_release: :default,
# This sets the default environment used by `mix release`
default_environment: Mix.env()
# For a full list of config options for both releases
# and environments, visit https://hexdocs.pm/distillery/configuration.html
# You may define one or more environments in this file,
# an environment's settings will override those of a release
# when building in that environment, this combination of release
# and environment configuration is called a profile
environment :dev do
set dev_mode: true
set include_erts: false
set cookie: :"SECRET_COOKIE"
end
environment :prod do
set include_erts: true
set include_src: false
set cookie: :"SECRET_COOKIE"
end
# You may define one or more releases in this file.
# If you have not set a default release, or selected one
# when running `mix release`, the first release in the file
# will be used by default
release :myapp do
set version: "0.1.0"
set applications: [
:runtime_tools,
myapp: :permanent,
myapp_web: :permanent
]
end
If I build the release with mix release
it builds fine and creates _build/prod/rel/tree/releases/0.1.0/myapp.tar.gz
. If I build using mix bootleg.build
the same file is created, but I get the following output after the build completes. Is there some special configuration needed to make this work with umbrella projects?
[10.2.0.49 ] ==> Assembling release..
[10.2.0.49 ] ==> Building release myapp:0.1.0 using environment prod
[10.2.0.49 ] ==> One or more direct or transitive dependencies are missing from
[10.2.0.49 ] :applications or :included_applications, they will not be included
[10.2.0.49 ] in the release:
[10.2.0.49 ]
[10.2.0.49 ] :elixir_make
[10.2.0.49 ]
[10.2.0.49 ] This can cause your application to fail at runtime. If you are sure
[10.2.0.49 ] that this is not an issue, you may ignore this warning.
[10.2.0.49 ] ==> Including ERTS 9.0 from /usr/lib/erlang/erts-9.0
[10.2.0.49 ] ==> Packaging release..
[10.2.0.49 ] ==> Release successfully built!
[10.2.0.49 ] You can run it in one of the following ways:
[10.2.0.49 ] Interactive: _build/prod/rel/myapp/bin/myapp console
[10.2.0.49 ] Foreground: _build/prod/rel/myapp/bin/myapp foreground
Downloading release archive
[10.2.0.49 ] DOWNLOAD /data/bootleg/build/_build/prod/rel//releases/0.0.1/.tar.gz -> releases/0.0.1.tar.gz
unsupported user key algorithm :"ssh-dss"
CentOS: Access or use of this system is restricted to authorized users only. The following are strictly prohibited: unauthorized disruption of the traceable, correct, and complete functions of this system inclusive of communication, processing and software, destruction, alteration or disclosure of data or cryptographic keys contained. Misuse of this system, software, data, or cryptography is subject to both civil and criminal prosecution.** (RuntimeError) SCP download error: Invalid SCP directive received: scp: /data/bootleg/build/_build/prod/rel/releases/0.0.1/.tar.gz: No such file or directory
lib/bootleg/ssh.ex:122: Bootleg.SSH.download/3
lib/strategies/build/distillery.ex:142: Bootleg.Strategies.Build.Distillery.download_release_archive/2
lib/bootleg/config.ex:311: Bootleg.Config.invoke/1
deps/bootleg/lib/bootleg/tasks/update.exs:4: Bootleg.DynamicTasks.Update.execute/0
lib/bootleg/config.ex:311: Bootleg.Config.invoke/1
(mix) lib/mix/task.ex:301: Mix.Task.run_task/3
(mix) lib/mix/cli.ex:75: Mix.CLI.run_task/2
I've been trying to get bootleg to work for a while now but seem to be completely stuck on even getting commands to run on the server. Anyone having a clue what is wrong?
I can ssh into the remote server the regular way without issues so that rules out the any problem with the key.
My config:
role :build, "my-domain.com",
user: "deploy",
identity: Path.expand("~/.ssh/id_rsa"),
workspace: "~/tmp/bootleg/build"
Stacktrace from mix bootleg.build production
(using elixir 1.5.1 and OTP 20)
Creating remote context at '~/tmp/bootleg/build'
[my-domain.com] /usr/bin/env mkdir -p ~/tmp/bootleg/build
** (SSHError) SSHKit returned an internal error on my-domain.com: {:function_clause, [{:public_key, :pem_entry_decode, [{:RSAPrivateKey, <<191, 8, 24, 153, 216, 76, 147, 2, 19, 72, 42, 182, 142, 174, 157, 112, 161, 94, 6, 160, 222, 224, 247, 247, 52, 200, 16, 19, 244, 44, 128, 219, 71, 131, 153, 55, 179, 128, 124, 232, 217, 140, 10, ...>>, {'AES-128-CBC', <<84, 47, 220, 211, 67, 173, 224, 77, 235, 222, 244, 234, 184, 159, 23, 18>>}}], [file: 'public_key.erl', line: 131]}, {SSHClientKeyAPI, :user_key, 2, [file: 'lib/ssh_client_key_api.ex', line: 102]}, {:ssh_auth, :get_public_key, 2, [file: 'ssh_auth.erl', line: 145]}, {:ssh_connection_handler, :is_usable_user_pubkey, 2, [file: 'ssh_connection_handler.erl', line: 1744]}, {:ssh_connection_handler, :"-init_ssh_record/4-lc$^0/1-0-", 2, [file: 'ssh_connection_handler.erl', line: 467]}, {:ssh_connection_handler, :init_ssh_record, 4, [file: 'ssh_connection_handler.erl', line: 466]}, {:ssh_connection_handler, :init, 1, [file: 'ssh_connection_handler.erl', line: 401]}, {:ssh_connection_handler, :init_connection_handler, 3, [file: 'ssh_connection_handler.erl', line: 364]}]}
lib/bootleg/ssh.ex:66: anonymous fn/2 in Bootleg.SSH.run/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
lib/bootleg/ssh.ex:109: Bootleg.SSH.run!/2
lib/bootleg/ssh.ex:83: Bootleg.SSH.validate_workspace/3
lib/strategies/build/distillery.ex:9: Bootleg.Strategies.Build.Distillery.init/0
lib/strategies/build/distillery.ex:16: Bootleg.Strategies.Build.Distillery.build/0
lib/bootleg/config.ex:311: Bootleg.Config.invoke/1
(mix) lib/mix/task.ex:301: Mix.Task.run_task/3
Hi, thanks for the great tool!
I had this task in my deploy.exs
:
after_task :compile do
remote :build do
"[ -f package.json ] && npm install || true"
"[ -f brunch-config.js ] && [ -d node_modules ] && ./node_modules/brunch/bin/brunch b -p || true"
"[ -d deps/phoenix ] && mix phoenix.digest || true"
end
end
Suddenly my builds start failing with this output:
...
[my-build-host] │ │ │ │ │ └── [email protected]
[my-build-host] │ │ │ │ └─┬ [email protected]
[my-build-host] │ │ │ │ └── [email protected]
[my-build-host] │ │ │ ├── [email protected]
[my-build-host] │ │ │ ├─┬ [email protected]
[my-build-host] │ │ │ │ └─┬ [email protected]
[my-build-host] │ │ │ │ ├── [email protected]
[my-build-host] │ │ │ │ ├── [email protected]
[my-build-host] │ │ │ │ └── [email protected]
[my-build-host] │ │ │ ├── [email protected]
[my-build-host] │ │ │ └── [email protected]
[my-build-host] │ │ ├─┬ [email protected]
[my-build-host] │ │ │ └─┬ [email protected]
[my-build-host] │ │ │ ├── [email protected]
[my-build-host] │ │ │ └── [email protected]
[my-build-host] │ │ └── [email protected]
[my-build-host] │ └─┬ [email protected]
[my-build-host] │ ├─┬ [email protected]
[my-build-host] │ │ ├── [email protected]
[my-build-host] │ │ └── [email protected]
[my-build-host] │ └─┬ [email protected]
[my-build-host] │ └─┬ [email protected]
** (ArgumentError) argument error
(stdlib) :io.put_chars(:standard_io, :unicode, [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[...] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[my-build-host] "] | "\e[0m"], " ├─┬ [email protected]"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[my-build-host] "] | "\e[0m"], " │ ├─┬ [email protected]"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[my-build-host] "] | "\e[0m"], " │ │ ├─┬ [email protected]"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[my-build-host] "] | "\e[0m"], " │ │ │ └── [email protected]"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[my-build-host] "] | "\e[0m"], " │ │ └── [email protected]"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[my-build-host] "] | "\e[0m"], " │ ├── [email protected]"], "\n"] | "\e[0m"] | "\e[1m"] | "\e[34m"], "[my-build-host] "] | "\e[0m"], " │ └── [email protected]"] | "\e[0m"], 10])
lib/bootleg/ssh.ex:90: Bootleg.SSH.capture/3
lib/sshkit/ssh/channel.ex:224: SSHKit.SSH.Channel.loop/4
lib/sshkit/ssh.ex:125: SSHKit.SSH.run/3
lib/bootleg/ssh.ex:70: anonymous fn/2 in Bootleg.SSH.run/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
lib/bootleg/ssh.ex:109: Bootleg.SSH.run!/2
(elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
I resolved this problem by replacing:
"[ -f package.json ] && npm install || true"
to
"[ -f package.json ] && npm install --silent > /dev/null || true"
Output results returned from remote/3
drop data in some cases where multiple roles are used with differing host counts.
For example:
use Bootleg.Config
role :app, ["app1.example.com", "app2.example.com"]
role :build, ["build.example.com"]
remote [:build, :app], do: "hostname" # should return 3 results, will only return 2
The issue only occurs if the host counts between roles does not match. In the above example, the following would behave correctly:
remote :build, do: "hostname" # returns 1 result
remote :app, do: "hostname" # returns 2 results
In all cases the remote command is executed on all requested hosts, and the the streamed output back to STDOUT
will include output for all requested hosts. Only the return value from remote/3
is impacted.
While poking through the build task functional tests I noticed there are two tests that don't actually assert anything. These tests shoulda have at least one assertion
test "builds the application"
test "builds the application with an absolute workspace path"
Argument error on parsing env from tasks
mix bootleg.update production
Raise an error
** (ArgumentError) argument error
:erlang.hd([])
lib/bootleg/tasks.ex:53: Bootleg.Tasks.parse_env_task/1
lib/mix/tasks/update.ex:2: Mix.Tasks.Bootleg.Update.run/1
(mix) lib/mix/task.ex:314: Mix.Task.run_task/3
(mix) lib/mix/cli.ex:80: Mix.CLI.run_task/2
The problem seems to come from https://github.com/labzero/bootleg/blob/master/lib/bootleg/tasks.ex#L50 trying to get head of an empty list.
This was recently observed when phoenix asset digest failed but its error didn't cause the build to be aborted. The assets appeared to still have been bundled into and deployed with the app, but were actually old assets from a previous build.
To repro:
echo "foo" > /tmp/bootleg/build/priv/static/bar
(using your build workspace)mix bootleg.update
file /opt/app/lib/app-0.0.1/priv/static/bar
(using your app name and version)Expected: "bar: No such file or directory"
Occurred: "bar: ASCII text"
Is it possible to suppress the banner text for ssh connections? It produces a lot of noise in the output. I was hoping perhaps quiet_mode
or something can be passed to Erlang's ssh client, but I haven't spent much time researching.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.