Git Product home page Git Product logo

hdl-component-manager's Introduction

HDL Component Manager (HCM)

Manage HDL code as components so they can shared as IP.

Github Release

PyPI Version

Build Status

Test Coverage

Read The Docs

Codacy

Maintainability

image

Table of Contents

Overview

HCM was created after a frustrating attempt to merge changes from one program to another. Even after carefully performing the merge, there were issues.

It was turning into a nightmare just to pass updates between multiple concurrently running programs. I was inspired by PIP, and how easy it is to install python packages. I wanted to bring that same level of ease to HDL design.

Key Benefits

  • Provides a method to control versions of IP
  • Controls the distribution of HDL code
  • Follows the Major.Minor.Patch method of version control
  • Language independent (VHDL, Verilog, System Verilog)
  • Can be used to control vendor IP

Key Features

  • Works with SVN repositories
  • Automates publishing of code to a central location
  • Automates installing and upgrading of code
  • Supports multiple IP repositories
  • Supports dependencies between components

Installation

You can get the latest released version of HCM via pip.

pip install hcm

The latest development version can be cloned...

git clone https://github.com/jeremiah-c-leary/hdl-component-manager.git

...and then installed locally...

python setup.py install

Usage

HCM can be invoked by issuing hcm at the command line prompt:

$ hcm
usage: hcm [-h]
           {browse,create,download,install,uninstall,list,publish,show,validate,version}
           ...

Provides configuration management for HDL components.

positional arguments:
  {browse,create,download,install,uninstall,list,publish,show,validate,version}
    browse              List components available for installation.
    create              Creates a component repo
    download            Downloads components without installing them.
    install             Adds a component from the component repo
    uninstall           Removes installed components
    list                Lists components and their versions
    publish             Adds components to the component repo
    show                Displays information about installed components
    validate            Verifies manifest of installed component
    version             Displays HCM version information

optional arguments:
  -h, --help            show this help message and exit

HCM has eight subcommands: browse, create, install, uninstall, list, publish, show, and validate.

browse

Use the browse subcommand to list components available for installation.

image

The arguments for the subcommand can be listed using the -h option:

$ hcm browse -h
usage: hcm browse [-h] [component]

positional arguments:
  component   Component to browse

optional arguments:
  -h, --help  show this help message and exit

create

Use the create subcommand to create a component directory in the repository.

image

The arguments for the subcommand can be listed using the -h option:

$ hcm create -h
usage: hcm create [-h] url

positional arguments:
  url         location to create the base component repo

optional arguments:
  -h, --help  show this help message and exit

download

Use the download subcommand to pull a version from the repository without installing it.

image

install

Use the install subcommand to add or upgrade a component from a repository.

image

The arguments for the subcommand can be listed using the -h option:

$ bin/hcm install -h
usage: hcm install [-h] [--version VERSION] [--url URL] [--force] [--external]
                   [--dependencies] [--upgrade]
                   component

positional arguments:
  component          Component name to install

optional arguments:
  -h, --help         show this help message and exit
  --version VERSION  Major.Minor.Patch version of component to install.
  --url URL          location of component directory in repo
  --force            Install component ignoring any local changes
  --external         Install as an external
  --dependencies     Install dependencies
  --upgrade          Upgrade dependencies to latest version

uninstall

Use the uninstall subcommand to remove installed components.

image

The arguments for the subcommand can be listed using the -h option:

$ hcm uninstall -h
usage: hcm uninstall [-h] component

positional arguments:
  component   Installed Component name to install

optional arguments:
  -h, --help  show this help message and exit

list

Use the list subcommand to check the versions of components you have installed.

image

The arguments for the subcommand can be listed using the -h option:

$ hcm list -h
usage: hcm list [-h] [--all]

optional arguments:
  -h, --help  show this help message and exit
  --all       Includes directories that are not under HCM control

publish

Use the publish subcommand to push a version of a component to a repository.

image

The arguments for the subcommand can be listed using the -h option:

$ hcm publish -h
usage: hcm publish [-h] (-m M | -f F) [--url URL] component version

positional arguments:
  component   Component name to publish
  version     Major.Minor.Patch version to publish

optional arguments:
  -h, --help  show this help message and exit
  -m M        Commit message
  -f F        File to use as commit message
  --url URL   Base URL of the component repository

show

Use the show subcommand to display information about an installed component.

image

The arguments for the subcommand can be listed using the -h options:

$ hcm show -h
usage: hcm show [-h] [--manifest] [--upgrades] [--updates] [--modifications]
                component

positional arguments:
  component        Component to display information

optional arguments:
  -h, --help       show this help message and exit
  --manifest       Displays manifest for all files in component
  --upgrades       Lists upgrade versions and their log entries
  --updates        Lists versions with newer publishes and their log entries
  --modifications  Lists committed modifications for component

validate

Use the validate subcommand to compare the component manifest against what is currently installed.

image

The arguments for the subcommand can be listed using the -h options:

positional arguments:
  component   Component to display information

optional arguments:
  -h, --help  show this help message and exit
  --report    Reports differences

Environment Variables

HCM will use the HCM_URL_PATHS environment variable as a replacement for the --url command line option. HCM uses the paths in the variable to know which component repositories to interact with.

Documentation

All documentation for HCM is hosted at read-the-docs.

Contributing

I welcome any contributions to this project. No matter how small or large.

There are several ways to contribute:

  • Bug reports
  • Code base improvements
  • Feature requests
  • Pull requests

Please refer to the documentation hosted at read-the-docs for more details on contributing.

hdl-component-manager's People

Contributors

jeremiah-c-leary avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

00mjk

hdl-component-manager's Issues

Add show argument

The show argument could output the hcm.json file in a readable format.

Fix formatting of list output

The list output is a fixed size.
This causes the output to not align if some component names are really long.

Should also include the URL of the component.

attempting to install using an external when there are no externals already causes HCM to crash

comps$ hcm install --external rook
INFO:Installing component rook
INFO:Validating all files for component rook are committed.
INFO:Removing local component directory
INFO:Updating externals
Traceback (most recent call last):
File "/home/jeremiah/projects/hdl-component-manager/bin/hcm", line 14, in
main()
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/main.py", line 128, in main
subcommand.install(commandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 12, in install
install_component(oCommandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 75, in install_component
update_externals(sFinalUrlPath, sComponent)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 159, in update_externals
lExternals = svn.get_externals('.').split('\n')[:-1]
AttributeError: 'NoneType' object has no attribute 'split'

add --upgrades to show subcommand

The --upgrades option would show all available upgrades and the svn log for each release. This could help someone in determining if they want to upgrade.

Add dependency support

There should be a method to indicate if a block requires another block.
There should also be a way to indicate if a block is not compatible with another block.

I would suggest a yaml file:

requires : 
    rook : 
        url : http://svn/..../rook
        version : 1.0.0

compatibility: 
     castle : 
        url : http://svn/.../castle
        version : 3.2.1

Need a download option

It would be nice to have the ability to pull down a specific version of component without installing it. This would be useful when you have to merge two versions.

subcommand list will crash if there is an error in the hcm.json file

comps$ hcm list
Traceback (most recent call last):
File "/home/jeremiah/projects/hdl-component-manager/bin/hcm", line 14, in
main()
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/main.py", line 130, in main
subcommand.sub_list(commandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/sub_list.py", line 36, in sub_list
copy_url(dVersions, dConfig, sDirectory)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/sub_list.py", line 80, in copy_url
dVersions['components'][sDirectory]['url'] = utils.get_url(dConfig)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/utils.py", line 29, in get_url
return dHcmConfig['publish']['url']
KeyError: 'publish'

Need a way to check the log entries for versions

When using the list subcommand, it just shows the current version.
Using the show subcommand only shows upgraded log files.
I would like to view the log entries for every version of a component.

publish is not working

Getting the following error when trying to publish:

Traceback (most recent call last):
File "../../bin/hcm", line 14, in
main()
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/main.py", line 64, in main
subcommand.publish(commandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/publish.py", line 162, in publish
dHcmConfig = create_default_hcm_dictionary(oCommandLineArguments.name, oCommandLineArguments.version, sUrl)
AttributeError: 'Namespace' object has no attribute 'name'

HCM will attempt to publish a directory that does not exist.

This ends in the following error:

INFO:Publishing component pawwn as version 3.1.0
INFO:Validating all files for component pawwn are committed.
INFO:Searching for hcm.json file...
WARNING:Did not find hcm.json for component pawwn.
INFO:Creating default hcm.json file...
INFO:Validating component exists in component directory...
INFO:Creating component in component directory.
INFO:Updating version...
INFO:Updating source URL...
Traceback (most recent call last):
File "../../bin/hcm", line 14, in
main()
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/main.py", line 105, in main
subcommand.publish(commandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/publish.py", line 184, in publish
update_source_url(dHcmConfig)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/publish.py", line 73, in update_source_url
lOutput = svn.issue_command(['svn', 'info', '-R', dHcmConfig['hcm']['name']]).split('\n')
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/svn.py", line 10, in issue_command
raise e
subprocess.CalledProcessError: Command '['svn', 'info', '-R', 'pawwn']' returned non-zero exit status 1

Update README.md

The project has progressed enough to enhance the README.md file.

HCM does not cleanly handle the case where a path can not be found in the repo when using dependencies

../../bin/hcm install king --force
INFO:Installing component king
INFO:Removing local component directory
{'requires': {'castle': None, 'pawn': None}}
castle
pawn
INFO:Installing component castle
Traceback (most recent call last):
File "../../bin/hcm", line 14, in
main()
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/main.py", line 118, in main
subcommand.install(commandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 30, in install
sFinalUrlPath = validate_urls(lUrl, sComponent, sVersion)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 95, in validate_urls
sUrlPath = build_url_path(sUrl, sComponent, sVersion)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 71, in build_url_path
return sReturn + utils.get_latest_version(sUrl + '/' + sComponent)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/utils.py", line 51, in get_latest_version
lOutput = svn.issue_command(['svn', 'list', sUrl]).split('\n')[:-1]
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/svn.py", line 10, in issue_command
raise e
subprocess.CalledProcessError: Command '['svn', 'list', 'file:///home/jeremiah/svn/my_repo/comps/castle']' returned non-zero exit status 1

Add a version argument

Having a version option will allow users to check the version being used without having to use pip.

Add scenario for adding a component that already exists.

I ran into a situation where I had an updated version of a component that was not under HCM control. I wanted to ensure I merged what was already published with the unpublished version.

Detail how to deal with this situation.

Create presentation

A presentation would be handy to have in case I need to give a presentation about HCM.

HCM crashes when attempting to install an external over a non externalled component.

comps$ hcm install bishop
INFO:Installing component bishop
INFO:Validating all files for component bishop are committed.
INFO:Removing local component directory
INFO:Installation complete
comps$ hcm list

Component Version Upgrade Status URL


bishop 2.0.0 None U file:///home/jeremiah/svn/my_repo/comps
queen 3.0.0 None E file:///home/jeremiah/svn/my_repo/comps
rook 3.0.0 4.0.0 E file:///home/jeremiah/svn/my_repo/comps
comps$ svn status
M .
A + bishop
X queen
X rook

Performing status on external item at 'queen':

Performing status on external item at 'rook':
comps$ hcm install bishop --external
INFO:Installing component bishop
INFO:Validating all files for component bishop are committed.
ERROR:The following files must be committed or removed:
A + bishop
comps$ hcm install bishop --external --force
INFO:Installing component bishop
INFO:Removing local component directory
INFO:Updating externals
Traceback (most recent call last):
File "/home/jeremiah/projects/hdl-component-manager/bin/hcm", line 14, in
main()
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/main.py", line 128, in main
subcommand.install(commandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 12, in install
install_component(oCommandLineArguments)
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/subcommand/install.py", line 76, in install_component
svn.issue_command(['svn', 'update', '.'])
File "/home/jeremiah/projects/hdl-component-manager/bin/../hcm/svn.py", line 10, in issue_command
raise e
subprocess.CalledProcessError: Command '['svn', 'update', '.']' returned non-zero exit status 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.