Git Product home page Git Product logo

easybuild-docs's Introduction

EasyBuild documentation

This repository contains the sources of the EasyBuild documentation, which is hosted at https://docs.easybuild.io.

Format

The documentation sources are located in the docs subdirectory.

They are written in MarkDown format, and rendered using MkDocs, and the Material for MkDocs theme.

Useful links:

Writing

When writing content for the EasyBuild documentation, it is useful to get a live preview of how things will look.

Next to the letting mkdocs serve a local copy of the full EasyBuild documentation so you can view it in your browser (see below), you can also use online services for this. This will allow you to see a live version of standard Markdown, but it will not give you a live preview of the MkDocs specific Markdown extensions.

At https://markdownlivepreview.com you can see a live preview while editing content in MarkDown format.

HackMD (https://hackmd.io) is a MarkDown note service which provides live previewing of the rendered note while you edit in MarkDown format, and collaborative editing of content.

Create a new note via https://hackmd.io/new to preview your content while editing it in MarkDown, and share the note URL with others if you would like to work together on some content.

To save work-in-progress notes and get back to them later, create a dedicated HackMD account, or sign in using your GitHub/Google/... account, via https://hackmd.io/login.

Publishing

The documentation is automatically built and published to https://docs.easybuild.io on every push to the main branch of this repository.

This is taken care of by the deploy GitHub Actions workflow.

Using mkdocs

Installing mkdocs

To install all the required mkdocs Python packages, use the provided requirements.txt file:

pip install -r requirements.txt

Building

To build the documentation, use:

mkdocs build

or use the Makefile that is provided in this repository:

make

Testing

To test whether the documentation is building correctly, and whether all (internal) links are correct, use:

mkdocs build --strict

or

make test

These commands will exit with a non-zero exit code if mkdocs produces any errors or warnings.

Previewing

To see a local preview of the rendered documentation in your browser, use

mkdocs serve

or

make preview

and click the link that is provided, for example:

INFO     -  Documentation built in 0.24 seconds
INFO     -  [17:52:07] Watching paths for changes: 'docs', 'mkdocs.yml'
INFO     -  [17:52:07] Serving on http://127.0.0.1:8000/

This preview of the rendered documentation will automatically refresh when the documentation sources are updated!

EasyBuild Framework API

The API documentation for the EasyBuild Framework is generated automatically if the EasyBuild framework code is found in src:

mkdir src
cd src
git clone [email protected]:easybuilders/easybuild-framework.git

If the EasyBuild Framework is not found then the mkdocs build of the documentation will be faster, but this will cause a number of warnings messages to be printed.

Contributing

To contribute to the EasyBuild documentation, you should make a pull request to the main branch.

  • Fork the easybuild-docs repository to your GitHub account, clone the repository and set up a remote for your fork:

    git clone [email protected]:easybuilders/easybuild-docs.git
    cd easybuild-docs
    git add remote YOUR_GITHUB_ACCOUNT [email protected]:YOUR_GITHUB_ACCOUNT/easybuild-docs.git
  • Create a branch for your changes, push it to your fork on the easybuild-docs repository, and open the pull request:

    vim docs/example.md
    git checkout -b example_branch
    git add docs/example.md
    git commit -m "this is just an example"
    git push YOUR_GITHUB_ACCOUNT example_branch
    # create pull request in GitHub:
    # https://github.com/YOUR_GITHUB_ACCOUNT/easybuild-docs/pull/new/example_branch

easybuild-docs's People

Contributors

bartoldeman avatar bedroge avatar boegel avatar branfosj avatar deniskristak avatar hattom avatar jfgrimm avatar lexming avatar micket avatar ocaisa avatar orbsmiv avatar petrkralcz avatar sebastianachilles avatar smoors avatar verdurin avatar xaviercs-dev avatar xdelaruelle avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

easybuild-docs's Issues

Document how to create a patch file

While the documentation mentions what kind of format that patches need to be in there is no information on how one would create a patch to begin with. The question has come up a few times on Slack.

For example, my workflow for creating a patch is something like this, I'm sure there's a lot of variations on it among contributors and maintainers:

  • unpack the source tarball twice, once into foo-1.2.3.orig and once into foo-1.2.3
  • make the desired changes to foo-1.2.3
  • diff -ruN foo-1.2.3.orig foo-1.2.3 > foo-1.2.3_fix-some-problem.patch
  • edit the patch file and add a plain text section at the top of the file describing what it solves, where it's sourced from, credits/authors
  • add to desired easyconfig together with the computed checksum or suggest --inject-checksums

Missing documentation for `modulename`, especially for PythonPackage

The question just came up on slack: What to do if the name of a PythonPackage (on e.g. PyPi) doesn't match its import name (i.e. what is used in python -c 'import <module-name>'?

Trying to find "modulename" in the docs turned up nothing and it isn't listed in any help I found, e.g. eb -a -e PythonPackage

Especially worth mentioning would be that dashes in the names are replaced by underscores by default!

Toolchains bug (foss description) + improvement (compatible toolchains)

Looking at the documentation of the toolchains [https://docs.easybuild.io/version-specific/toolchains/].

foss | GCC | OpenMPI | OpenBLAS, ScaLAPACK | FFTW

Presumably this should now read FlexiBLAS instead of OpenBLAS (but probably OpenBLAS under the hood)?

I was thinking that the idea of compatible toolchains (e.g. foss> {gfbf, gompi} > GCC > GCCcore) doesn't appear to be particularly clearly documented. Would this fit well into the toolchain docs page?

Docs needed on when to use the Python EasyConfig, Bundle, or PythonPackage to install Python packages (similar for R?)

My question was "When/why would one use Bundle, or choose for PythonPackage? What’s the difference?"

@boegel 's answers:

boegel [1:53 PM]
@caspar van Leeuwen We use a Bundle when a Python package depends on a bunch of other Python packages, which are all installed as extensions (and are version-locked). exts_defaultclass indicates what the default easyblock is to install extensions with.

[1:54]
@caspar van Leeuwen If a Python package doesn't depend on Python packages (that are not yet covered through dependencies), then you can use the PythonPackage easyblock directly, otherwise you'd have a bundle with a single extension, which is weird.

[1:55]
@caspar van Leeuwen Some Python packages, like PyYAML are provided as dependency (rather than extesnion), because it requires other dependencies (e.g. libyaml).

[1:55]
It's probably worthwhile to have some clear documentation on this, yes, since there are several options (as extension in Python easyconfig vs separate module, as a bundle of extensions or straight up Python package, etc.)

[1:56]
@caspar van Leeuwen Please open an issue with a request to document this, feel free to include my comments above it in as a place to start from.

Hooks - update link

Update hooks.md to replace the link configuration-easyconfigs-repository with easyconfigs_repo. (Due to conflict in PR #12.)

support building docs without API part

With the changes in #79, having easybuild-framework cloned into src is a hard requirement for the docs build to work:

$ ls src
ls: src: No such file or directory
$ make preview
mkdocs serve
INFO     -  Building documentation...
INFO     -  Cleaning site directory
FileNotFoundError: [Errno 2] No such file or directory: 'py3.py' -> 'src/easybuild/tools/py2vs3/py2.py'
make: *** [preview] Error 1

That's because the gen_ref_pages.py script blindly assumes that src contains a copy of the easybuild-framework repository.

We should allow building the documentation without the API part, to speed up local development of the documentation.

While doing that, we should also use src/easybuild-framework instead of just src.

Internal links in auto-generated content

In order to preserve the current docs' internal links, the auto-generated content (e.g. that in version-specific directory) will need to include the following links as attributes (e.g. {: #link_name }):

(N.B. please update the following list as instances are discovered.)

  • version-specific/eb-list-toolchains.md:
    • toolchains_table
  • version-specific/toolchain-opts.md:
    • avail_toolchain_opts
  • version-specific/eb-a-wrf.md:
    • eb_a_e_EB_WRF
  • version-specific/help.md
    • basic_usage_help
  • version-specific/easyconfig_parameters.md
    • easyconfig_params

review locks page

  • https://docs.easybuild.io/locks/
  • older example: OpenFOAM-7-foss-2019b.eb
  • --wait-on-lock without an argument throws:
    TypeError: '>' not supported between instances of 'tuple' and 'int'
    • this is wrong, since an argument isn't said to be required in the docs
    • --wait-on-lock is deprecated, replaced by --wait-on-lock-interval, so docs should be updated
  • everything else is OK

Suggestion: Add easyconfig templates and try to leverage the easyconfigs library for helping newcomers

Apologies from the get go is this suggestion is vague.
Being a newcomer to the easybuild community I've struggled with creating easyconfigs for the packages I deal with.

I find that a set of standard template example easyconfigs to provide a starting point for the most common types of easyconfig would have been very helpful.

In addition, something I think would be very helpful is learning from examples. If there were a summary of some existing easyconfigs+easyblocks and what problems they are trying to solve and in what way, this would likely significantly lower the entry threshold.

Title Capitlisation

We should decide on a title case and use it consistently. See https://titlecaseconverter.com/ for a convertor and information on possiblities.

It would also be helpful to have a test check the capitalisation.

  • grep all the header lines in markdown files
  • check they conform (already a tester for this or would we need to regex them?)

This might be more complicated due to technical terms and commands in the titles

Suggestion: List of common easyblocks

A page listing the features of common easyblocks and how to decide when to use them would be helpful.
I'm thinking especially of common queries such as picking Tarball versus PackedBinary versus Binary.

If this is already there, perhaps better links are needed?

build fails with mkdocs-material > 9.0.6

We fixed the mkdocs-material version used to build the docs to 9.0.6 in d974bab (included via #109), to work around failing build with mkdocs-material==9.0.7 and mkdocs-material==9.0.8

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.9/x64/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/mkdocs/__main__.py", line 250, in build_command
    build.build(cfg, dirty=not clean)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/mkdocs/commands/build.py", line 320, in build
    _build_theme_template(template, env, files, config, nav)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/mkdocs/commands/build.py", line 118, in _build_theme_template
    output = _build_template(template_name, template, files, config, nav)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/mkdocs/commands/build.py", line 97, in _build_template
    output = template.render(context)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/404.html", line 4, in top-level template code
    {% extends "main.html" %}
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/main.html", line 4, in top-level template code
    {% extends "base.html" %}
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/base.html", line 148, in top-level template code
    {% block site_nav %}
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/base.html", line 156, in block 'site_nav'
    {% include "partials/nav.html" %}
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/partials/nav.html", line 27, in top-level template code
    {% include "partials/nav-item.html" %}
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/partials/nav-item.html", line 91, in top-level template code
    {{ render(nav_item, path, level) }}
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/jinja2/runtime.py", line 777, in _invoke
    rv = self._func(*arguments)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/partials/nav-item.html", line 56, in template
    {{ render(nav_item, path ~ "_" ~ loop.index, level + 1) }}
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/jinja2/runtime.py", line 777, in _invoke
    rv = self._func(*arguments)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/material/partials/nav-item.html", line 48, in template
    <nav class="md-nav" data-md-level="{{ level }}" aria-labelledby="{{ path }}_label" aria-expanded="{{ nav_item.active | tojson }}">
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/jinja2/filters.py", line 1688, in do_tojson
    return htmlsafe_json_dumps(value, dumps=dumps, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/jinja2/utils.py", line 658, in htmlsafe_json_dumps
    dumps(obj, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Undefined is not JSON serializable
Error: Process completed with exit code 1.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.