aztfmod / rovergo Goto Github PK
View Code? Open in Web Editor NEWThe next version of Rover, the command line tool for Azure CAF Landingzones. Developed in Go
License: MIT License
The next version of Rover, the command line tool for Azure CAF Landingzones. Developed in Go
License: MIT License
Rover v1 was ONLY usable inside a devcontainer
Rover v2 should also support running from a devcontainer
Unit tests are the wild new craze taking the software development world by storm.
We should get on this new hotness and some of these so called "unit tests" to rover v2
To be agreed:
Currently only launchpad run
works for deploying and destroying in launchpad mode. Also for CD the level must be launchpad: true
.
Landingzone mode is a simple extension (most if not all of existing code in pkg/landingzone applies) however the required vars need to be set and state handled differently. etc
This should bring us to close to an usable but 'alpha' version
Task #21 makes up a large part (possibly all, pending investigation) of this effort
Current CI command reads both symphony YAML and also YAML in ci_tasks directory, each of these YAMLs defines a task that can be run in CI mode
These files will need to be parsed, and will form the core of the ci
command implementation. Once the file is passed it takes the extra params from symphony YAML (e.g. working directory, path to vars, level etc) and executes the task by dropping down to the shell
For terraform commands I think tf-exec should be used rather shell
This task is just for parsing the YAML
Building the task runner is in task xxxx
When I tried this command with an incorrect -c parameter, the command tried to run anyway, gave incorrect results.
rover lp destroy -c $HOME/caf_config -s $HOME/git/caf-terraform-landingzones -l level0
I left off /demo/level0 from the -c parameter.
Take the existing 'CAF sample application' from here https://github.com/aztfmod/symphony/tree/master/caf
In case changes are required I propose making a copy into rovergo repo
Try to deploy it! First the launchpad and then the various levels etc
Investigate the code flow
I think the only difference for launchpad mode is it calls initialize_state
(confusing function name as it's ONLY used by launchpad) which calls init without a backend, then after calls upload_tfstate
Suggestion we use a $HOME/.rover for everything we can
$HOME/.rover/logs/
$HOME/.rover/hist.txt
This epic tracks the overall effort to rewrite and reimplement rover in Go.
The motivations for doing this are to replace the bash code with a more robust, easy to maintain code base and improve the user experience at the command line.
Makes sense this is not a top level command as it's rarely used, and it only applies to landingzones so more intuative it goes here
Secrets will need to be fetched, as part of #21
Placeholder to not forget this needs working on
az login
do all we need?The ci_tasks folder must be statically located, i.e. always in the same place, so having a flag for its location makes no sense.
Task #35 is a pre-req for this one.
This goes hand in hand with the work for #12
Considerations
integration testing may use terratest, but in general the pattern is:
terraform plan, validate
Both landingzone and launchpad commands will need to call TF plan and apply, hopefully they can share the same code and design
The current behaviour is:
./custom_actions
is looked forcustom_actions
directory is not found a warning is outputIssues:
custom_actions
in the .rover directory?Specifically the extraction of TF_VAR_tfstate_storage_account_name etc vars from the keyvault
This is a broad tasks and might require breaking up, further investigation is required
Rover v1 supports a way to list all deployed landingzones for some reason ๐คทโโ๏ธ
Rather than supply /foo/bar/caf_launchpad or /foo/bar/caf_solution to the lp and lz commands, accept a source flag but change it to be just the "root" of the landingzone source directory, no need to add caf_launchpad or caf_solution as we know what one we want from the "mode" (lp or lz)
Get a list of all Azure CLI commands the current rover tool executes.
This will then feed into a list of tasks here - to implement corresponding calls using the Azure APIs and Go SDK, and shape a package containing those calls in a way they can be used but other parts of rovergo, e.g. wrappers and helpers
I see there being a mix of
It would improve test-ability if there was some output to examine along with an error object.
For example:
{
"command": "lp",
"subCommand": "init",
"outcome": "success"
}
I tried it a couple of ways:
rover cd destroy
rover lp destroy -c $HOME/caf_config/demo/level0 -s $HOME/git/caf-terraform-landingzones -l level0
This may be due to the fact that I changed some things in caf_config. As follows:
resource_groups = {
level0 = {
name = "lp-l0"
tags = {
level = "level0"
}
} ... etc
Currently all output is done via pkg/console.go which supports the following message levels, the levels are mapped to colours
The only one of these which is controllable is debug, which will only be output if the global --debug
flag is passed to rover
We could replace --debug with a --log-level flag and map that level to the existing 5 levels or do we want to step back and examine logging & output in more depth
Thoughts / ideas:
This issue is to discuss & provide feedback on the proposal for Rover v2, which is hosted on the wiki for easier editing
Note. This is a "living document" and constantly evolving
This is a legacy of an earlier phase in the project, it was created automatically for us as part of running the cobra new
command when the project was initialised.
Removing this has one big advantage we can replace the lengthy --symphony-config
flag with simply --config
.
It's also one less config file, and artefact for the user to keep track of
runCITaskSubCommand in ci.go
Not really any reason we shouldn't check for these as rover starts, the user isn't going to be able to get very far without either of these
Current working draft is on the wiki
As there is no way to comment on wiki pages, discussion and feedback should be done here, on this issue
${TF_DATA_DIR}/tfstates/${TF_VAR_level}/${TF_VAR_workspace}/${TF_VAR_tf_name}
TF_VAR_workspace
is set to "tfstate" unless the user passes --workspace
to set itTF_DATA_DIR
to $HOME/.terraform.cache
terraform init
is called with -backend-config
so it uses remote state. The storage account to use is found using the environment tags and the level-tfstate
parameter to rover sets the state key that is used, i.e. the blob nameWe need to authenticate in order to call Terraform - however this has been solved see current implementation login.go
, azure.go
and terraform.go
HOWEVER we also need to call the Azure APIs via the Go SDK, we need to understand how this can be unified and done without
end goal - no hard coded endpoints / dns names other than in the internal list in #1 above
Function in functions.sh of rover v1
Checks role with
role=$(az role assignment list --role "${1}" --assignee ${TF_VAR_logged_user_objectId} --include-inherited --include-groups)
This should be done as part of runLaunchpadInit (probably)
Opening this issue to discuss the proposed changes here:
https://github.com/aztfmod/rovergo/wiki/Symphony-v2---Draft-Proposal
Add stubs with cobra add foo
and cobra add bar --parent foo
Output of task #4 drives this and is a pre-req
This design is probably not final but adding these stubs gives a good sense for what it will look and feel like, and what work is remaining
Commands required:
Because this is run inside an init() function it means rover won't run if ci_tasks is missing, even if the command you are running is NOT rover ci
Line 41 in 45bbb2e
CD should be the focus now, i.e. the entry point should be some form of rover cd --config=foo
command to carry out landing zones and launchpad operations, these can then be internally "ported" over to launchpad and landingzone commands and the settings which are in symphony YAML.
The internal implementation, namely the terraform actions, interaction with state etc will remain the same regardless.
This task is to get stub functionality for CD working, reading symphony YAML, looping over it or locating a level or stack in the YAML and outputing simple log messages for now
Then we can follow up with deeper implementation
#14 is a pre-req
rover cd
always requires a symphony.yaml config file as inputRover can switch the identity it runs terraform under by fetching details from a keyvault
I don't think this affects any other calls to Azure e.g. the az commands
and therefor any calls we make to the API with the SDK
Initial work for CI and CD, we need to work off the symphony.yaml
file
If az account show
reveals that the user is a managed identity:
"user": { "assignedIdentityInfo": "MSI", "name": "systemAssignedIdentity", "type": "servicePrincipal" }
use the Azure Instance Metadata Service to get the vm's name and resource group name. feed those into az vm identity show
(the REST equivalent) to get the principalId.
This principalId is (we think) the same as the object-id acquired by az ad signed-in-user show
. Which is required for various api calls in rovergo.
Add capability to run any ad hoc CI command, such as tflint, etc. i.e. one that requires an executable in the environment (whether devcontainer, pipeline agent, etc). spec of the command is in a ci_tasks folder yaml file.
Also, refactor CI commands similar to CD - as methods on the symphony struct object.
This task is to refactor the current action verbs to meet the current Rover v2 spec
This involves moving the action from a command flag, to sub commands, one per action. This will result in many new sub-commands but should result in a logical structure. These sub-commands will all follow the same flow, the difference being the landingzone.Action they pass into the Execute()
The current semantics of rover v2 maps actions to terraform verbs almost 1:1, the exception being the action deploy
which is a synonym for running terraform plan
+ terraform apply
I think switching back to the action named apply
(which actually does a terraform plan
+ terraform apply
) is best for now and will prevent confusion. However I'd welcome input and comments
See https://github.com/aztfmod/caflint
It's written in Go, so...
This would either be a new top level command e.g. rover caflint
or an value for the action flag of landingzone run
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.