Git Product home page Git Product logo

iohk-ops's Introduction

Build status

Collection of tooling and automation to deploy IOHK infrastructure.

Structure

  • deployments - includes all NixOps deployments controlled via .hs scripts
  • modules - NixOS modules
  • lib.nix - wraps upstream <nixpkgs/lib.nix> with our common functions
  • scripts - has bash scripts not converted to Haskell/Turtle into Cardano.hs yet
  • default.nix - is a collection of Haskell packages
  • static includes files using in deployments
  • jobsets is used by Hydra CI
  • terraform - other AWS infrastructure
  • nix-darwin - deployment script and configurations for MacOS X machines

Getting SSH access

  1. Fork https://github.com/input-output-hk/iohk-ops
  2. Check out the master branch
  3. Add your username and SSH public key to the appropriate developer section of lib/ssh-keys.nix. Keys should remain sorted alphabetically by username.
  4. Submit a PR against master and let DevOps know.
  5. Wait until the DevOps team deploys the infrastructure cluster.

The io command

Sources for the iohk-ops tool are in the iohk directory.

Usage

After cloning this repo, start a nix-shell.

% nix-shell
[nix-shell:~/iohk/iohk-ops]$ io --help

For more documentation, see docs/iohk-ops-reference.md.

Development

To hack on the iohk-ops tool, use

% nix-shell -A ioSelfBuild
[nix-shell:~/iohk/iohk-ops]$ type io
io is a function
io ()
{
    cabal exec iohk-ops -- "$@"
}
[nix-shell:~/iohk/iohk-ops]$ io --help

This will provide a Haskell environment where you can use io to run the script or ghci for development:

[nix-shell:~/iohk/iohk-ops]$ ghci -iiohk/common
GHCi, version 8.2.2: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/rodney/config/.ghc/ghci.conf
λ> :l iohk/iohk-ops.hs

Run from anywhere

$(nix-build --no-out-link https://github.com/input-output-hk/iohk-ops/archive/master.tar.gz -A iohk-ops)/bin/iohk-ops --help

iohk-ops's People

Contributors

adinapoli-iohk avatar angerman avatar cleverca22 avatar codiepp avatar craigem avatar deepfire avatar disassembler avatar dnadales avatar domenkozar avatar dorin100 avatar erikd avatar georgeee avatar iohk-bors[bot] avatar jbgi avatar jmitchell avatar johnalotoski avatar krisajenkins avatar ksaric avatar manveru avatar mhuesch avatar nc6 avatar ntfreem avatar picnoir avatar robcohen avatar ruhatch avatar rvl avatar sectore avatar shmish111 avatar vagoum avatar volhovm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

iohk-ops's Issues

Release notes automation for daedaluswallet.io

Tomas updates the release notes (and perhaps other content?) on daedaluswallet.io every release. When DevOps proposes the new mainnet installers on the blockchain, we also update the daedalus-latest.version.json so the installer links on the website are updated without manual intervention.

I suggest we work toward having all standard release-related updates encoded in the daedalus-latest-version.json file. Otherwise, we have complicated timing issues where DevOps has been asked to do the update in the middle of Tomas' night.

Previous issue: https://iohk.myjetbrains.com/youtrack/issue/DEVOPS-904

Build of iohk-ops fails

Build of iohk-ops fails with the the next message:
error: build of ‘/nix/store/jf1jwbkii4b3cyd19k701wr6rbivdsan-cardano-sl-36f9fcc.drv’ failed

If I checkout to slow-fallocate branch and move to slow-fallocate directory, then build fails too with similar message:
error: build of ‘/nix/store/jf1jwbkii4b3cyd19k701wr6rbivdsan-cardano-sl-36f9fcc.drv’ failed

New benchmarking guide

Currently we have a document benchmarks/Benchmarks.md which explains how to launch benchmarking on a cluster and collect results. Unfortunately, this document is incomplete and contains some out-of-date info. We have to improve it.

standard port instead of 3000

(same suggestion posted in deadalus repository)

It looks like a noticeable number of users has problems using Daedalus just because their computer (AV Suite) or local network (LAN/WAN) does not allow connections to the 7 bootstrap servers listening on port 3000
I would suggest introducing a fallback on port 443 (HTTPS) or a complete switch to such a standard port.

Better UX when adding new repos to hydra

Currently adding a new repo to hydra for CI involves editing at least three files. Which can be seen for example in #578. This is quite ergonomically and prone to errors (I think).

It would be pretty cool if we had a json file that looked something like this:

[
  ...
  , { "loony tools": "https://github.com/input-output-hk/tools" }
  ...
]

from which this scaffolding that's currently needed would be driven. Maybe optionally have a key bors: true if borsng should be enabled on that repo or not.

gac: `deployments/configurator.nix` has AWS-specific code in it

This file is used by all GAC-based deployments for the purpose of sharing
configuration parameters between NixOS VMs/instances and the Nixops configs that
deploy them. It also contains some definitions of AWS resources. While the
configuration-parameter management aspect of it is needed by all GAC
deployments, the AWS resources are only needed by deployments to AWS instances.
We therefore need to be able to include those parts separately.

Change hydra project name

  • configure hydra to report both serokell and Cardano jobs
  • fix PR building for cardano-shell
  • update branch protection settings on iohk-ops
  • report statuses for cardano-wallet
  • report Cardano status for daedalus
  • update ouroboros-network branch protection
  • update cardano-sl branch protection
  • update plutus branch protection
  • verify bors.toml is updated on cardano-ledger

# Prometheus Integration

Need to have a monitoring server with the following for each deployment:

  • prometheus
  • alertmanager
  • grafana
  • all services behind oauth_proxy
  • integration with pagerduty
  • statsd exporter for metrics for applications providing metrics
  • node exporter on all nodes in deployment
  • nginx /w vts monitoring on any nginx hosts (including monitoring host)
  • deadmansnitch to alert if prometheus server is down (PD integration as well)

Hydra Issues

  • Notification Queue
  • Eval times
  • Slow mac builds
  • Number of builds per slave to low
  • sending/receiving inputs/outputs between slave and master

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.