Git Product home page Git Product logo

avhclient's Introduction

GitHub main-branch Build Workflow Status Unit tests results Codecov coverage report PyPI - Python Version PyPI - Wheel PyPI PyPI - License

Arm Virtual Hardware Client (avhclient)

The Arm Virtual Hardware Client (avhclient) is a python module that provides an interface for deploying and using Arm Virtual Hardware (AVH).

It enables uniform implementation of CI operations in various environments with reference examples provided for the following use cases:

  • Jenkins CI pipelines
  • GitHub-Actions workflows
  • Local use with AVH targets

Other environments can be supported using demonstrated concepts as well.

Example projects using AVH Client


Installation

Installing (development snapshot) directly from GitHub main branch:

pip install git+https://github.com/ARM-software/avhclient.git@main

Backend Setup

avhclient can control different backends with Arm Virtual Hardware. Following options are currently available:

  • aws (default) - interacts with AVH AMI available through AWS Marketplace
  • local - operates with AVH Targets installed locally.

The backend can be specified with -b option preceding the actual avhclient command.

Depending on the backend certain environment setup is expected.

AWS Backend Setup

avhclient accesses AWS services via Boto3 AWS SDK and for that requires a set of parameters to be available in the environment.

AWS Credentials

It is necessary to expose avhclient with the AWS credentials for your account. You can either export your AWS credentials:

export AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID"
export AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_ACCESS_KEY"
export AWS_SESSION_TOKEN="YOUR_AWS_SESSION_TOKEN"

or create a AWS credential file on ~/.aws/credentials (Linux & Mac) or `%USERPROFILE%.awscredentials (Windows)`:

[default]
aws_access_key_id=YOUR_AWS_ACCESS_KEY_ID
aws_secret_access_key=YOUR_AWS_SECRET_ACCESS_KEY
aws_session_token=YOUR_AWS_SESSION_TOKEN

More info AWS CLI config and credentials

AWS Account info

In order for avhclient to create or access an AVH instance following parameters need to be defined in the execution environment of avhclient:

  • When creating and running a new AVH instance:

    Mandatory info:

    export AWS_IAM_PROFILE='YOUR_IAM_PROFILE'
    export AWS_SECURITY_GROUP_ID='YOUR_AWS_SECURITY_GROUP_ID'
    export AWS_SUBNET_ID='YOUR_SECURITY_GROUP_ID'
    export AWS_S3_BUCKET_NAME='YOUR_B3_BUCKET_NAME'

    Optional info (examples):

    export AWS_AMI_ID=DESIRED_AVH_AMI_ID
    export AWS_AMI_VERSION=1.1.2
    export AWS_EFS_DNS_NAME=fs-066cf410af2428e2f.efs.eu-west-1.amazonaws.com
    export AWS_EFS_PACKS_DIR=packs
    export AWS_KEEP_EC2_INSTANCES=true
    export AWS_KEY_NAME=YOUR_AWS_KEYPAIR_NAME
    export AWS_INSTANCE_TYPE=t2.micro
    export AWS_INSTANCE_NAME=MY_AVH_INSTANCE
    • If AWS_AMI_VERSION is not set, the avhclient will use the latest available version of AVH AMI.
    • If AWS_EFS_DNS_NAME is set, the AVH Client will try to mount it during the cloud-init phase. The only scenario supported for now is using Packs.
    • If AWS_EFS_PACKS_DIR is set, the mount path is relative to /home/ubuntu folder. Default folder is packs and if it exists locally will be then replaced by the EFS mount. Only used when AWS_EFS_DNS_NAME env is set.

    AWS Cloudformation can be used to create the AWS resources required for AVH operation, as shown in this template

  • When reusing an AVH Instance:

    export AWS_INSTANCE_ID=YOUR_INSTANCE

Local Backend Setup

Operation with a local backend requires no specific environment parameters, but assumes that necessary toolchain, AVH targets and utilities are installed locally on the machine and configured for execution in command line.


Usage

Getting Help

To get the brief descriptions of all commands and options available with avhclient execute:

avhclient -h

You can also use option -h with a specific command to get help for it. For example for execute command:

avhclient execute -h

Execute command

  • Create a new AWS AVH instance and run AVH project

    The execute command bundles all necessary steps to build your avh project:

    • prepare the backend.
    • upload your files
    • run your commands
    • download the results
    • cleanup the backend

    Inform the path for the avh.yml file for your AVH project (example):

    avhclient -b aws execute --specfile AVH-GetStarted/basic/avh.yml
  • You can also run in AVH commands in your local computer by selecting local backend:

    avhclient -b local execute --specfile AVH-GetStarted/basic/avh.yml
  • There are also backend specific info you can provide to the tool:

    avhclient -b aws -h (get full list)
    avhclient -b aws --instance-name MY_NEW_NAME execute --specfile AVH-GetStarted/basic/avh.yml (seeting a new AVH instance name)
    avhclient -b aws --ami-version 1.1.0 --specfile AVH-GetStarted/basic/avh.yml (Create a new AVH instance from a v1.1.0 AVH AMI)
    avhclient -b aws --ami-version >1.1.0 --specfile AVH-GetStarted/basic/avh.yml (Create a new AVH instance from a >v1.1.0 AVH AMI)

AVH YML file syntax

avhclient execute command requires a specfile in YML format that describes details of individual steps to be executed on AVH. The file syntax is explained below.

A JSON schema for automatic checks and auto-completion is in schema/avh.schema.json.

Fields

Format of the specfile:
        name: (optional) The name of the workload.
        workdir: (optional) The local directory to use as the workspace, defaults to specfile's parent.
        backend: (optional) Dictionary with backend specific parameters.
          aws: (optional) Dictionary with AWS backend specific parameters. (see backend help)
          local: (optional) Dictionary with local backend specific parameters. (see backend help)
        upload: (optional) List of glob patterns of files (relative to workdir) to be sent to the AVH backend. (see glob format)
        steps: (mandatory) List of steps to be executed on the AVH backend.
          - run: String written into a bash script and executed on the AVH backend inside the workspace directory.
        download: (optional) List of glob patterns of files (relative to workdir) to be retrieved back from the AVH backend. (see glob format)
    Glob format:
        The list of glob patterns is evaluated in order.
        Wildcard '*' matches all files but no directory except hidden files (starting with '.').
        Wildcard '**' matches all files and directories except hidden files/directories (starting with '.').
        Inclusive matches (no prefix) are added to the file list.
        Exclusive (prefixed with '-:') matches are removed from current file list.

Example

# yaml-language-server: $schema=https://raw.githubusercontent.com/ARM-software/avhclient/main/schema/avh.schema.json

name: "AVH GetStarted Example"
workdir: ./
backend:
  aws:
    ami-version: ~=1.1
    instance-type: t2.micro
upload:
  - RTE/**/*
  - -:RTE/**/RTE_Components.h
  - basic.debug.cprj
  - build.py
  - main.c
  - requirements.txt
  - retarget_stdio.c
  - vht_config.txt
  - README.md
steps:
  - run: |
      pip install -r requirements.txt
      python build.py --verbose build run
download:
  - RTE/**/RTE_Components.h
  - Objects/basic.axf
  - Objects/basic.axf.map
  - basic-*.xunit
  - basic-*.zip

avhclient's People

Contributors

spcaipers-arm avatar jonatanantoni avatar vovamarch avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.