metanorma / ci Goto Github PK
View Code? Open in Web Editor NEWBuild scripts for Metanorma, works with continuous integration
Build scripts for Metanorma, works with continuous integration
puppetteer has been supplanted by apache XSL:FO
for:
nist ogc csd iso iec rsd
@CAMOBAP795 we need to remove AppVeyor for all builds migrated to GHA.
cc @ronaldtse
Currently we have a new macro from https://github.com/metanorma/metanorma-plugin-lutaml gem - lutaml_diagram, this macro render uml diagrams with graphviz dot
command, so we need this tool in our build and docker images.
@ronaldtse should we?
The Makefile in mn-samples-itu and mn-samples-ogc are now identical, and I've tried it for mn-samples-ietf as well. It should work for all mn-samples-*.
This task is to add the Makefile to cimas config and apply them across all mn-samples-*.
Hi there, Recently while I was working on the CLI then I came across an issue related to the caching gems, and I suspect this might be related to an old version for cache action. Especially even after changing the *.gemspec
file, it's was using the old hash key.
I temporarily updated it to cache@v2
in the CLI and it seems to be fixing the issue for now, What do you guys think about updating the cache action version globally here?
Ref: metanorma/metanorma-cli#180
// cc: @CAMOBAP, @ronaldtse
Currently, when we doing a release and push tag, execution also happens for branch too, if there is a way to restrict it we should try to implement it
Error: POST https://api.github.com/repos/metanorma/metanorma-ietf/pulls: 403 - You have triggered an abuse detection mechanism and have been temporarily blocked from content creation. Please retry your request again later. // See: https://docs.github.com/v3/#abuse-rate-limits
I faced with this issue today, this is potential blocker for cimas tool
To be continued...
Two files must be present in every Ruby repository: .hound.yml and .rubocop.yml. Following content seems optimal for Metanorma:
Extracted from riboseinc/oss-guides#44.
For IETF related repositories:
We often see timeouts like:
Reading workgroups from https://tools.ietf.org/wg/...
bundler: failed to load command: metanorma (/Users/runner/work/mn-templates-ietf/mn-templates-ietf/vendor/bundle/ruby/3.0.0/bin/metanorma)
/Users/runner/work/mn-templates-ietf/mn-templates-ietf/vendor/bundle/ruby/3.0.0/gems/metanorma-ietf-2.3.1/lib/asciidoctor/ietf/validate.rb:48:in `initialize': No such file or directory @ rb_sysopen - https://tools.ietf.org/wg/ (Errno::ENOENT)
from /Users/runner/work/mn-templates-ietf/mn-templates-ietf/vendor/bundle/ruby/3.0.0/gems/metanorma-ietf-2.3.1/lib/asciidoctor/ietf/validate.rb:48:in `open'
from /Users/runner/work/mn-templates-ietf/mn-templates-ietf/vendor/bundle/ruby/3.0.0/gems/metanorma-ietf-2.3.1/lib/asciidoctor/ietf/validate.rb:48:in `cache_workgroup_ietf'
from /Users/runner/work/mn-templates-ietf/mn-templates-ietf/vendor/bundle/ruby/3.0.0/gems/metanorma-ietf-2.3.1/lib/asciidoctor/ietf/validate.rb:75:in `block in cache_workgroup'
from /Users/runner/work/mn-templates-ietf/mn-templates-ietf/vendor/bundle/ruby/3.0.0/gems/metanorma-ietf-2.3.1/lib/asciidoctor/ietf/validate.rb:74:in `open'
from /Users/runner/work/mn-templates-ietf/mn-templates-ietf/vendor/bundle/ruby/3.0.0/gems/metanorma-ietf-2.3.1/lib/asciidoctor/ietf/validate.rb:74:in `cache_workgroup'
...
This can be fixed by caching ~/.metanorma-ietf-workgroup-cache.json
like already done here:
One cons: this IETF specific logic and it will be in all other CI configuration
FYI @ronaldtse
List of repos which uses ruby-xslt
Remove ruby-xslt
installation once it will be removed from repos above
cc @ronaldtse
issue so far restricted to metanorma-standoc, OSX builds; other gems building fine:
Still running (12 of 20): brew install --HEAD https://raw.githubusercontent.com/metanorma/homebrew-metanorma/master/Formula/metanorma.rb
2402The command brew install --HEAD https://raw.githubusercontent.com/metanorma/homebrew-metanorma/master/Formula/metanorma.rb exited with 1.
2403
2404Log:
2405
2406Warning: Your Xcode (9.4.1) is outdated.
2407Please update to Xcode 10.1 (or delete it).
2408Xcode can be updated from the App Store.
2409
2410
....
/Users/travis/.travis/functions: line 553: 14616 Terminated: 15 travis_jigger "${!}" "${timeout}" "${cmd[@]}"
2564The command "if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then travis_wait brew install --HEAD https://raw.githubusercontent.com/metanorma/homebrew-metanorma/master/Formula/metanorma.rb ; fi" failed and exited with 1 during .
2565
2566
https://gist.github.com/jcypret/608731eed983dd46a9e7447791c86cdb
@ronaldtse how do you think about this?
For ubuntu only flows
Based on discussion with @ronaldtse
According to https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/
GHA recently introduced a new workflow_dispatch
event which allows to trigger workflows manually, with some input
@ronaldtse AFAIR that in the past we had some cases where this can be applied, could you please remind if you remember
I came this CI failure here - mn-templates-cc/pull/23/checks and looks like it's because of the recently deleted metanorma setup
interface. Most of the ci configuration still referencing it and that's causing those failures, should we consider removing this from the actions and update all the repos?
//cc: @CAMOBAP, @ronaldtse
Add packed-mn testing on public samples repos alongside with docker & ruby metanorma
@ronaldtse how do you think, make sense?
Take metanorma/mn-templates-ietf#3 approach as reference
Problem https://github.com/metanorma/mn-samples-iso/runs/632875985?check_suite_focus=true
Fix metanorma/mn-samples-iso@3f42418
Or explicitly setup path if using v2
cc @ronaldtse
The solution which was introduced by me brew install plantuml
take much more time in comparison with the previous one
- unset _JAVA_OPTIONS
- wget "http://downloads.sourceforge.net/project/plantuml/plantuml.jar?r=&ts=1424308684&use_mirror=jaist" -O plantuml.jar
- sudo mkdir -p /opt/plantuml
- sudo cp plantuml.jar /opt/plantuml
- echo "#! /bin/sh" > plantuml.sh
- echo 'exec java -jar /opt/plantuml/plantuml.jar "$@"' >> plantuml.sh
- sudo install -m 755 -D plantuml.sh /usr/bin/plantuml
- plantuml -version
I need to check it on OSX and try different URL (maybe https://vorboss.dl.sourceforge.net/project/plantuml/1.2019.4/plantuml.1.2019.4.jar) to download because the current one takes up to 1 min to get it
For now, when we change specific GHA workflow file all flows will be triggered
Use https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#excluding-paths to run only changed flow
I noticed that for some repositories we run CI on both
Example:
Is there any good reason for:
on: [push]
instead of:
on: [push, pull_request]
Running tests on push
event will run tests on the last commit of given branch, but will not take into account commits, which are present on master
branch, but not on tested branch. Also, I suppose, no tests will be run for pull requests coming from forked repositories.
"""
I'm thinking that we should always clean up the gh-pages branch entirely on deployment
There's no need to carry the gh-pages history
https://github.com/peaceiris/actions-gh-pages looks good to use
"""
Regression after merge all workflows to a single one
@ronaldtse subj
https://github.com/metanorma/mn-samples-itu/pull/68/checks?check_run_id=522010507 & all mn-samples-* repos
Kind of known issue knu/ruby-unf_ext#52
I will check what we can do about this
make failed, exit code 2
Gem files will remain installed in
C:/hostedtoolcache/windows/Ruby/2.5.7/x64/lib/ruby/gems/2.5.0/gems/unf_ext-0.0.7.6
for inspection.
Results logged to
C:/hostedtoolcache/windows/Ruby/2.5.7/x64/lib/ruby/gems/2.5.0/extensions/x64-mingw32/2.5.0/unf_ext-0.0.7.6/gem_make.out
An error occurred while installing unf_ext (0.0.7.6), and Bundler cannot
continue.
Make sure that `gem install unf_ext -v '0.0.7.6' --source
'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
metanorma-cli was resolved to 1.2.10.1, which depends on
metanorma-acme was resolved to 1.4.4, which depends on
metanorma-standoc was resolved to 1.3.21, which depends on
relaton-iev was resolved to 0.1.3, which depends on
relaton was resolved to 0.10.1, which depends on
relaton-un was resolved to 0.1.0, which depends on
http-cookie was resolved to 1.0.3, which depends on
domain_name was resolved to 0.5.20190701, which depends on
unf was resolved to 0.1.4, which depends on
unf_ext
make: *** [Makefile.win:109: bundle] Error 5
The relaton-iev build scripts contain
steps:
- uses: actions/checkout@master
with:
submodules: recursive
submodules is not supported in GitHub Actions, and there are not submodules in relaton-iev anyway. I could not find any instances of submodules in metanorma/metanorma-build-scripts, but just in case, please check that submodules are not being added now into any GitHub Action scripts. (This is the only instance I could find in my local copy of relaton and metanorma.) I have deleted the lines from the build scripts.
cimas-config/cimas.yml
need to be updated too, please let me know if you need help with this
@CAMOBAP could you help run Cimas to check if the rfc-* workflows work? Thanks!
Originally posted by @ronaldtse in #55 (comment)
@ronaldtse could you please confirm that we can remove puppeteer form all gems?
cc @opoudjis
https://help.github.com/en/articles/workflow-syntax-for-github-actions
jobs.<job_id>.strategy.fail-fast
When set to true, GitHub cancels all in-progress jobs if any matrix job fails. Default: true
jobs.<job_id>.strategy.max-parallel
The maximum number of jobs that can run simultaneously when using a matrix job strategy. By default, GitHub will maximize the number of jobs run in parallel depending on the available runners on GitHub-hosted virtual machines.
strategy:
max-parallel: 2
Possible solution https://github.com/marketplace/actions/purge-artifacts
Looks like Appbeyor doesn't support _trunk
out of box, but this can be fixed by https://github.com/MSP-Greg/appveyor_ruby
cc @ronaldtse
@ronaldtse could you please check if there is some restrictions where METANORMA_CI_PAT_TOKEN can be used
Based on metanorma/metanorma#95 (comment)
We need to implement it for samples and templates
We don't fail Travis builds if the latest version of Ruby fails, but the ruby head build, Travis Ruby 2.6.*, is now failing because we're using an obsolete version of Builder:
Bundler could not find compatible versions for gem "bundler":
In Gemfile:
bundler (~> 2.0.1)
Current Bundler version:
bundler (2.1.0.pre.1)
This Gemfile requires a different version of Bundler.
Perhaps you need to update Bundler by running `gem install bundler`?
Could not find gem 'bundler (~> 2.0.1)' in any of the relevant sources:
the local ruby installation
The command "bundle update" failed and exited with 6 during .
Should we globally switch to bundler ~> 2
?
cc @ronaldtse
@CAMOBAP795 in our templates the Travis script is calling this:
Typically the script is run like this:
sudo bash -c "curl -L https://raw.githubusercontent.com/metanorma/metanorma-linux-setup/master/ubuntu.sh | bash"
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.