Description:

Every time, when we add some commits/PR, etc to the repo, there is no any run with sanity checks (e.g.unit-tests). We can add, for example Travis-CI feature to solve that problem.

Steps:

1. Authenticate with OAuth on https://travis-ci.org/ (should do the maintainer of the repository).
2. Flick the repository switch on here:
   https://travis-ci.org/profile/$USERNAME
3. Add to root file .travis.yml such as:

language: bash

sudo: false

install:
    - pip install -U pip
    - pip install --user -U rpm

before_script: cd src/test

script:
- bash test.sh 

4. Somehow skip tests, that are expected to fail. From beakerlib/src/test/README

Some of the tests are expected to fail (backup functions
need root permission and the coverage test suite fails because...
well because the coverage is not complete... yet :-)

I didn't add them because I faced with another problem (specific for beakerlib project): check out build log here:
https://travis-ci.org/mnalband/beakerlib/builds/346725708

beakerlib uses some unbound variables which makes it impossible to run with set -u:

$ cat test.sh 
set -u

. /usr/share/beakerlib/beakerlib.sh

$ sh test.sh 
./beakerlib.sh: line 34: __INTERNAL_SOURCED: unbound variable
posix          	on
./infrastructure.sh: line 987: BEAKERLIB_DIR: unbound variable

CC @atodorov

If main.fmf is available in the test's directory a Makefile is skipped completely.
However, main.fmf does not need to contain all the metadata as some of them may be inherited from higher level's which is normally resolved by tmt.
Beakerlib should concatenate both sources if available.

I am getting this error on Fedora 29+.

Failed to convert timestamp attribute to int.                        
Error: invalid literal for int() with base 10: '"1538625769"'
Exiting unsuccessfully.

Looks like quotes are not striped from the attribute_value variable.

Suspect:

I'm trying to run command in watchdog and I wanted to capture the output using -s option. However I don't have it set even after rlRun exited correctly (was not killed by watchdog).

    rlPhaseStartTest
        rlWatchdog "rlRun -s 'echo yes'" 10
        declare -p rlRun_LOG
    rlPhaseEnd

When execututed, I don't have rlRun_LOG set

:: [ 10:16:02 ] :: [  BEGIN   ] :: Running 'rlWatchdog "rlRun -s 'echo yes'" 10'
:: [ 10:16:02 ] :: [   LOG    ] :: Runnning rlRun -s 'echo yes', with 10 seconds timeout
:: [ 10:16:02 ] :: [  BEGIN   ] :: Running 'echo yes'
yes
:: [ 10:16:02 ] :: [   PASS   ] :: Command 'echo yes' (Expected 0, got 0)
:: [ 10:16:03 ] :: [   LOG    ] :: Command ended itself, I am not killing it.
[2]+  Terminated              eval "sleep $timeout; touch __INTERNAL_TIMEOUT"
:: [ 10:16:04 ] :: [   PASS   ] :: Command 'rlWatchdog "rlRun -s 'echo yes'" 10' (Expected 0, got 0)
./runtest.sh: line 39: declare: rlRun_LOG: not found
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

As discussed in teemtee/tmt/issues/391 it would be useful to be able to import beakerlib library from the same directory where the test is located. The proposed syntax is:

rlImport .

This will simplify writing library self tests for cases when directory traversal does not detect the library.

as a library maintainer I want to be able to have my library in a custom location

Currently the library name is required to be in format <COMPONENT>/<LIB_NAME>. While we enabled libraries to be placed in individual repos on github the names might be simplified as well. E.g. currently there's library squid/squid in https://github.com/beakerlib/squid. The name here is forced to be double squid.
I think we may ease the requirement for it to be basically any string. If there was / the part before first / would be treated as a component, the rest as a lib_name. This would allow one to have a library in the git root and not to be forced to artificially created another names.
For the squid example the library name would be simply squid.

@psss, @hegerj, @lukaszachy, any thoughts?

It might be useful to have a simple list of phases and their statuses at the end of the report or included in the result file.
Use-case:
I have a test with a lot of phases (hundreds), if I run it on old and new package I want to be able to easily compare the lists to quickly find the differences.

Context

We'd like to run some of our existing test code on OracleLinux. So far we aim for OracleLinux 7/8/9.

Problem

To my knowledge, BeakerLib does not support OracleLinux. Where this definitely matters is rlIs*() family
of functions but there might be more places.

Solution

Ideally we'd

1. In ourt test code, replace all of our rlIsRHEL() with a generic function as suggested in #47 and #95,
2. make sure that this generic function also supports OracleLinux.

so for the matter, I'd consider this RFE to be blocked on one of the other RFE's (they look like dupes).

That said, there could be more places where the RHEL-fork-ness matters so this Solution might not be complete.

Final report doesn't contain consolidated test result.
Below image shows how it used to generate the consolidated test results before.

and how it looks like now:

As per the email communication:

your test / library parses those data form the text journal form which was redesigned a bit. This text form was never meant to be machine processed. It would be better to go other way. There's a results file referenced by __INTERNAL_TEST_RESULTS, where those information might be available to you. Unfortunately it appears to be still undocumented.

Hence filing this RFE.

when doing some new library with copying lib.sh sources from another library I've was unable to debug the the issue

:: [ 04:43:48 ] :: [   FAIL   ] :: Command 'rlImport cockpit/atomicstart' (Expected 0, got 1)
:: [ 04:43:48 ] :: [  ERROR   ] :: rlImport: Import of library cockpit/atomicstart was not successful (callback failed)

after @sopos investigation He found that I did not chaged: # library-prefix = cockpit
so that it did not called proper callback function. and also output is that just show that callback failed, and it seems that there is some error inside this function.

So there should be better message in this case and show that there is not defined proper callback function, and what is expected like

:: [ 04:43:48 ] :: [  ERROR   ] :: rlImport: Import of library cockpit/atomicstart was not successful (missing callback function cockpitLibraryLoaded in lib.sh file (or defined  library prefix is bad see #   library-prefix = cockpit))

Apparently, there are some pieces of code which are really slow.
Normally one would not expect such huge slow down, so some kind of profiling would help to identify slow parts.

I came upon this strange behavior in beakerlib-1.18-7.el7bkr.

I'm writing a library, say, d/foo that needs to import another one, say, d/bar opportunistically. That is, under (and only under) certain circumstances, d/foo can import `d/bar.

if test d_bar__var == ok; then
    rlImport d/bar
    d_bar__call ..
fi

So in my d/foo Makefile I have

...
@echo "RhtsRequires:    library(d/bar)" >> $(METADATA)
...

And in my runtest.sh I have just

rlImport d/foo

The thing is, when I run my test, beakerlib seems to be importing d/bar right away! This of course fails on validation as the circumstances where d/bar is valid aren't met.

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   init
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 15:42:30 ] :: [   INFO   ] :: rlImport: Found 'd/foo' in /mnt/tests
:: [ 15:42:30 ] :: [   INFO   ] :: rlImport: Will try to import d/foo from /mnt/tests/CoreOS/d/Library/foo/lib.sh
:: [ 15:42:30 ] :: [   INFO   ] :: rlImport: Found 'd/bar', version '17.4-15' in /mnt/tests
:: [ 15:42:30 ] :: [   INFO   ] :: rlImport: Will try to import d/bar from /mnt/tests/CoreOS/d/Library/bar/lib.sh
:: [ 15:42:31 ] :: [  ERROR   ] :: d_bar error!
:: [ 15:42:31 ] :: [   FAIL   ] :: (INTERNAL TEST ERROR) 
:: [ 15:42:31 ] :: [  ERROR   ] :: rlImport: Import of library d/bar was not successful (callback failed)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This is caused by how the macro is implemented:

I guess many tests use just rhIsRHEL, but the tests should work fine on all clones, like CentOS or even other. It would be nice to have a macro that matches all such clones, so even users of Scientific Linux or Oracle Linux can contribute with tests (in epel for example).

Currently LibraryDir is set during LibraryLoaded, which makes it impossible to use it in context like defining of global variables:

For example, where ns-prefix is distribution_hack__:

distribution_hack__path=${distribution_hack__path:-$distribution_hack__LibraryDir/hacks}

This should set the path global to 'hacks' sub-directory, but it won't works because the LibraryDir is not set. It should be possible to find it out already; it's just dirname of file we're reading.

As discussed in teemtee/tmt#391 it would be good if library's runtest.sh is able to find its lib.sh during upward traversal even if the Library is not present in the path.

On large systems, this fragment from src/journal.sh seems to be responsible for most of the delay in starting new tasks:

    local line size
    # CPU info
    if [ -f "/proc/cpuinfo" ]; then
        local count=0
        local type="unknown"
        local cpu_regex="^model\sname.*: (.*)$"
        while read -r line; do
            if [[ "$line" =~ $cpu_regex ]]; then
                type="${BASH_REMATCH[1]}"
                let count++
            fi
        done < "/proc/cpuinfo"
        __INTERNAL_WriteToMetafile hw_cpu -- "$count x $type"
        __INTERNAL_LogText "    CPUs          : $count x $type" 2> /dev/null
    fi

bash reads /proc/cpuinfo in pieces of 128 bytes, and the kernel really does not like that from a because of the way the proc file system is implemented. (I tried bash-4.4.19-10.el8.x86_64 and kernel-4.18.0-193.el8.x86_64.) The reported counts are wrong as well for some reason.

To give a perspective on the performance, running the modified script

    if [ -f "/proc/cpuinfo" ]; then
        count=0
        type="unknown"
        cpu_regex="^model\sname.*: (.*)$"
        while read -r line; do
            if [[ "$line" =~ $cpu_regex ]]; then
                type="${BASH_REMATCH[1]}"
                let count++
            fi
        done < "/proc/cpuinfo"
        echo hw_cpu -- "$count x $type"
        echo "    CPUs          : $count x $type" 2> /dev/null
    fi

results on an unloaded system in:

$ time bash cpu-script
hw_cpu -- 256 x AMD EPYC 7742 64-Core Processor
    CPUs          : 256 x AMD EPYC 7742 64-Core Processor

real    0m16.720s
user    0m0.095s
sys     0m16.565s

It's much worse on larger systems.

Would it be acceptable to use nproc? Copying the data to a temporary file instead will help as well because it eliminates the source of super-linear performance.

Since tests are going to be moving away from Makefiles and using fmf, beakerlib functions which depend on Makefiles should be modified to be able to use fmf file if it is available.
(There will be a period of time with a portion of tests in fmf and a portion in the older Makefile format)
According to a discussion in Testing Tools Notes gdoc these are some examples of such functions:
rlImport --all
rlCheckMakefileRequires / rlGetMakefileRequirements
rlAssertRequired
There may be more. Current status of required package definition is also being discussed in Jira ticket TT-119.

Command rlCheckMakefileRequires ends with [ WARNING ] :: requirement '1.23' not satisfied.

File metadata.yaml contains

require:
- beakerlib >= 1.23

I'd say that rlGetYAMLdeps doesn't quote string properly.

This was found in test suite of tmt run in teemtee/tmt#667

Seems in some cases we would like if the tempfile created by rlRun to be reused, so we can cleanup easily with one command. Now it always creates a new file.

This can be a new option to rlRun for example, where we pass in the tempfile for example.

rlRun "tmpfile=$(mktemp)"
rlRun -s -f $tmpfile "...."
rlAssertGrep "..." $rlRun_LOG
rlRun -s -f $tmpfile "...."
rlAssertGrep "..." $rlRun_LOG
rlRun "rm -f $rlRun_LOG"

cc @psss @sopos

If there's a need to ship some test and execute it in unknown environment where presence of required libraries is not guaranteed, one could just copy all the necessary libraries to the test directory to bundle them tohetgher with the test. Currently rlImport starts to traverse upwards in directories starting one directory up from the test's one. To overcome this a BEAKERLIB_LIBRARY_PATH variable can be used but if it is already used in the test environment where the tests are executed, it will collide.

There are bunch of variables intended to be used by a user to alter beakerlib's behaviour, most of them are not documented.
New section in man page should be added to describe these variables.

Hello,

What would it take to have xUnit results integrated into beakerlib, so that say provided some option, running beakerlib tests would give xUnit output? Is this a possibility?

Thanks

kernel/base library is available as part of beakerlib-libraries, and it seems since beakerlib 1.17 it does not handle the library correctly.

Tested with https://upstreamfirst.fedorainfracloud.org/nfs-utils/blob/master/f/exportfs/runtest.sh

# rpm -q beakerlib
beakerlib-1.17-4.fc28.noarch

# dnf copr enable mvadkert/beakerlib-libraries -y
Repository successfully enabled.

# dnf install beakerlib-libraries
<snip>

# ./runtest.sh 

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   do-Setup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 13:55:26 ] :: [   INFO   ] :: rlImport: Found 'kernel/base' in /usr/share/beakerlib-libraries
:: [ 13:55:26 ] :: [   INFO   ] :: rlImport: Will try to import kernel/base from /usr/share/beakerlib-libraries/kernel/Library/base/lib.sh
/usr/share/beakerlib/logging.sh: line 104: /mnt/tests/unknown/output.log: No such file or directory
:: [ 13:55:27 ] :: [   INFO   ] :: output log will be stored in /mnt/tests/unknown/output.log
{INFO} Test env info:
------------------------------------------------
Time & CURDIR : [2017-11-21 13:55:27 @/root/nfs-utils/exportfs]
Case Name     : unknown
$HOSTNAME     : localhost.localdomain
/usr/share/beakerlib-libraries/kernel/Library/base/lib.sh: line 299: lsb_release: command not found
Distro Info   :  : 
NVR & host    : Linux localhost.localdomain 4.15.0-0.rc0.git1.1.fc28.x86_64 #1 SMP Mon Nov 13 19:54:17 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
cmdline       :
    BOOT_IMAGE=/boot/vmlinuz-4.15.0-0.rc0.git1.1.fc28.x86_64 root=UUID=46f142a6-b4a4-4795-adc3-726609b3e796 ro no_timer_check console=tty1 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 LANG=en_US.UTF-8
Package Info  :
    rpm: no arguments given for query
------------------------------------------------

:: [ 13:55:27 ] :: [   INFO   ] :: using '/var/tmp/beakerlib-flcBsc8/backup' as backup destination
:: [ 13:55:27 ] :: [   INFO   ] :: using '/var/tmp/beakerlib-flcBsc8/backup' as backup destination

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   unknown
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

[13:55:27 root@ ~~]# rm -f /etc/exports.d/*
:: [ 13:55:27 ] :: [   PASS   ] :: Running 'rm -f /etc/exports.d/*' (Expected 0, got 0)
--------------------------------------------------------------------------------

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   unknown
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

[13:55:27 root@ ~~]# mkdir -p $nfsmp $expdir
:: [ 13:55:27 ] :: [   PASS   ] :: Running 'mkdir -p $nfsmp $expdir' (Expected 0, got 0)
--------------------------------------------------------------------------------

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   unknown
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

[13:55:27 root@ ~~]# echo "$expdir *(rw,no_root_squash,fsid=0)" > /etc/exports
:: [ 13:55:27 ] :: [   PASS   ] :: Running 'echo "$expdir *(rw,no_root_squash,fsid=0)" > /etc/exports' (Expected 0, got 0)
--------------------------------------------------------------------------------
Redirecting to /bin/systemctl start nfs.service
________________________________________________________________________________
:: [ 13:55:28 ] :: [   LOG    ] :: Duration: 1s
:: [ 13:55:28 ] :: [   LOG    ] :: Assertions: 1 good, 0 bad
:: [ 13:55:28 ] :: [   PASS   ] :: RESULT: unknown


::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   do-Test-exportfs-with-wrong-option
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

<snip>

I'd like to propose an option to modify behavior of rlAssertRpm --all when it comes to $COLLECTIONS. Currently, each item of $COLLECTIONS list is considered to be an rpm, representing a metapackage of a given collection, e.g. for devtoolset-8, rlAssertRpm expects to find devtoolset-8 RPM installed.

While this may be true for many collections, it is often not that simple in the case of Developer Toolset collections (aka "DTS"). These behave more like a set of sub-collections or tools, sharing a top-level metapackage. Customer is free to install just some of the tools from the DTS collection, e.g. devtoolset-8-gcc, which would pul gcc's dependencies including devtoolset-8-runtime RPM, which is the RPM that carries all necessary data for SCL to "install" devtoolset-8 collection. Customer can happily live without the top-level metapackage, therefore this metapackage may be missing in the otherwise valid scenario.

rlAssertRpm --all would then fail because of missing devtoolset-8 which was not installed, on purpose. The crucial RPM that makes the collection available to the customer is devtoolset-8-runtime, not devtoolset-8, and we'd like to modify rlAssertRpm to let us assert the presence of this kind of collection RPM.

An opt-in way would be nice, to avoid spoiling the default behavior, for example in a form of an option, let's say --collections-runtime. The implementation would then check for ${collection}-runtime RPM. The question is whether it should first check for $collection first and try ${collection}-runtime only when the first check failed.

Any comments, ideas?

cc @marticak

Currently, the TestResults file is generate by rlJournalEnd function. It would make sense to generate it in rlPhaseEnd as well and introduce a new variable, e.g. TESTRESULT_STATE with value complete / incomplete signlalizing the actual state.

Context

As part of in-place upgrade testing initiative, we already have speial
tests that (simplified) will do setup on RHEL-N and test on RHEL-N+1.

A newer, initiative -- convert2rhel has similar needs: there we have
use cases for "upgrade" tests that start on CentOS and end on RHEL.

Both types of tests are identical in structure:

• get source distro (RHEL-7 / CentOS-7 / ...),
• set up -- here we use BeakerLib code,
• upgrade or convert (to RHEL-7 / ...)
• test -- here we use same BeakerLib code again,

and because some components are long-term stable in CLI and behavior,
even the content is often almost identical.

Hence, we want to re-use the code as much as possible.

But the CLI/behavior is not 100% identical - they need to code branch
per distro -- hence the use of rlIsRHEL() is common.

Problem

When we try to take a test code that uses rlIsRHEL and re-use it in
a convert2rhel testing, we end up having to go to the code and change from

if rlIsRHEL '>6'; then
...

to

if rlIsRHEL '>6' || rlsCentOS '>6'; then
...

which can become really messy and confusing, especially if the rlIsRHEL condition is already part of
a more complex boolean expression.

Furthermore, there's a perspective that we may want to run this on other
distros, eg. OracleLinux. BeakerLib does not have rlIsOracleLinux
but I think you can see where this would go.

Solution

In most (not all, though) cases, the behavior of RHEL and RHEL-like
distro can be assumed to be identical. In these cases, rlIsRHEL
could be effectively replaced by something like this:

rlIsRHELLike() {
    #
    # True if rlIsRHEL $@ OR rlIsCentOS $@ would be true
    #
    rlIsCentOS "$@" && return 0
    rlIsRHEL   "$@" && return 0
    return 1
}

This would make the code re-usable on CentOS without further complexity.

Furthermore, the rlIsRHELLike function could be easily extended to
support other RHEL forks (where the major/minor assumption can be made),
making tests re-usable with no changes.

rlFileSubmit will print things like "Uploading ... .done" to stdout, which breaks use cases where stdout is being generated and needs to have proper format. (E.g. meta-programming.)

Reproduced with local build: beakerlib-1.17-13.el7bkr

There is no description at the top of the homepage, nor in the README.

Something short like "Test infrastructure used to build RHEL"?

First, I have to admit that I'm from the Copr maintenance team, and we are
using beakerlib for testing Copr. So I might be a bit biased, but still.

We use a container environment to run our integration testing that
is using beakerlib - and we try to move from Fedora N to Fedora N+1
usually very soon, when the N+1 release is out. But our processes
always choke on the /etc/yum.repos.d/beaker-client-Fedora.repo
file. It usually takes too much time till the new Fedora repository
starts working (e.g. F34 is still not available at this point in time).

In Fedora Copr, there's the "follow fedora branching" option which
would automatically create Fedora N+1 repository for you (and us,
users) at (almost) appropriate time.

if rlService* is called the current status is check first. This output is actually related to the previous service run so it is confusing if e.g. rlService Start is called but the status output says 'failed' even if the service start actually passed because the previous run failed.

If would be good to hide the initial service state check and eventually run service state check after the actual action so the relevant state is visible.

The operation below takes tens of minutes or sometimes even an hour to finish with larger journals:
/usr/bin/beakerlib-journalling --metafile journal.meta --journal journal.xml
The older beakerlib did not suffer from this, or at least it seemed so. Thanks.

There's already a concept of libraries which can be sourced by rlImport function. But the infrastructure is left on user's infrastructure. This makes things harder while migrating tests from custom infrastructure to upstream.
There might be a dedicated repository here at github for libraries.
One idea is to have branches per library. It would also prevent prefix collisions. Beakerlib would than be able to search for specific libraries also in such repository.
Other way could be to create special organization where every library would have its own repository which would also allow them to have devel and other branches.

Bug 1916894 - rlServiceStart fails due to missing chkconfig

Description of problem:
rlServiceStart fails if chkconfig is not installed:

/usr/share/beakerlib/infrastructure.sh: line 1375: chkconfig: command not found
:: [ 05:51:20 ] :: [ WARNING ] :: rlServiceStart: service initial-setup status returned 127
:: [ 05:51:20 ] :: [ WARNING ] :: rlServiceStart: Guessing that original state of initial-setup is stopped
/usr/share/beakerlib/infrastructure.sh: line 1388: chkconfig: command not found
:: [ 05:51:20 ] :: [ ERROR ] :: rlServiceEnable: Enablement of service initial-setup failed
:: [ 05:51:20 ] :: [ ERROR ] :: Status of the failed service:
:: [ 05:51:20 ] :: [ LOG ] :: Redirecting to /bin/systemctl status initial-setup.service
:: [ 05:51:20 ] :: [ LOG ] :: ● initial-setup.service - Initial Setup configuration program
:: [ 05:51:20 ] :: [ LOG ] :: Loaded: loaded (/usr/lib/systemd/system/initial-setup.service; disabled; vendor preset: disabled)
:: [ 05:51:20 ] :: [ LOG ] :: Active: inactive (dead)

beakerlib should require chkconfig.

Version-Release number of selected component (if applicable):
beakerlib-1.21-1.fc34

Currently you either have TEST empty and Package is detected from PACKAGE or PACKAGES
Or you have nice TEST name but Packages is dected from its value and forcing you to follow its naming convention.

Please create environment variable to override package detection, e.g. TESTPACKAGE (similar to TESTVERSION)

I could not find any instructions in how to properly open pull request. For example: which branch to code against? What should be covered by tests? Is there a project-wide coding style?

It would be nice to have this in file like README.md or CONTRIBUTING.md.

Hello beaker-devs,
in lorax-composer we would like to have rlAssert statements (and anything else that can report FAIL really) to terminate the test script immediately. It is fine with us if this is not enabled by default but we need some way to make it happen.

Can you point me in the right direction? Where do I look internally in beakerlib to see if we can hack something up? Ideally an ENV variable or a config file will work great.

CC @larskarlitski

Beakerlib has rlCmpVersion and rlTestVersion, however when one wants to compare NVR of an installed package he needs to:

• check if the package is installed at first
• use rpm --query ... to get the package version and pass it as arguments

It would be handy to have functions like rlCmpPkgVersion and rltestPkgVersion that does the above on behalf of the user. It would be called e.g. as follows:
rlCmpPkgVersion bash 1.2-3
rlTestPkgVersion bash 1.2-3

Few more notes:

• an error should be printed if the package is not installed
• not sure if Version in the name is appropriate, maybe NVR would be better
• cannot use simple NVR comparison, imagine the user would provide only Version and still we would like it to work properly, e.g. "rlTestPkgVersion bash 1.2" should return '=' both for bash-1.2-1 and bash-1.2-9. Release should be considered for comparison only if it is provided.

Currently, rlImport --all imports also recommended libraries. This is not really ideal as the recommended library may not be present at the run time but from the nature of the thing is should not cause a failure unless specifically handled by the requester.

During my attempts to use multiple operands of rlIsRHEL, I have noticed problem behaviour in respect to ranges.

[root@qeos-46 ]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 Beta (Maipo)
[root@qeos-46 ]# rlIsRHEL '<7' '<7.6'; echo $?
1
[root@qeos-46 ]# rlIsRHEL '<7' '<7.7'; echo $?
1
[root@qeos-46 ]# rlIsRHEL '<8' '<7.7'; echo $?
0
[root@qeos-46 ]# rlIsRHEL '<7' '<7.7'; echo $?
1
[root@qeos-46 ]# rlIsRHEL '7' '<7.7'; echo $?
0
[root@qeos-46 ]# rlIsRHEL '<7' '<7.7'; echo $?
1
[root@qeos-46 ]# rlIsRHEL '7' '7.5'; echo $?
0
[root@qeos-46 ]# rlIsRHEL '6' '7.5'; echo $?
0
[root@qeos-46 ]# rlIsRHEL '<6' '7.5'; echo $?
1
[root@qeos-46 ]# rlIsRHEL '<8' '7.5'; echo $?
0

rlGetDistro*() functions seem to rely on naming of RPMs, but this does not work for rlGetDistroVariant() on CentOS systems (tested with CentOS 6 and CentOS 7):

# rlGetDistroVariant; echo $?

0

I haven't yet figured out where/how to reliably get the info. (#35 does solve it for CentOS 7 and all /etc/os-release distros, but that does not entail CentOS 6.) - edit: no it doesn't: CentOS does not have such thing as Variants.

Documentation contains output examples with the pre-journal update format.

On RHEL-6 (and probably other) ComputeNode, rlGetDistroVariant will print Computenode which is incorrectly cased.

This can lead to further errors reading variant-named artifacts or URL's containing variant.

Apparently the error is caused by this naive attempt to "fix" casing:

 rpm -q --qf="%{NAME}" --whatprovides redhat-release | cut -c 16- | sed 's/.*/\u&/'

This is a solution that gets casing right (for one-letter variants, anyway (which are all RHEL-6 variants)):

 grep -Eoe "Linux [^ ]+ release" /etc/system-release | cut -d' ' -f2

Note that for RHEL-7 and above (or any systemd distros), it makes most sense to read VARIANT key from /etc/os-release using function like this:

__eor_key() {
    #
    # Read key $1 from /etc/os-release
    #
    # /etc/os-release supports syntax similar to shell assignment,
    # so we'll run it in a sandbox.
    #
    local key=$1
    test -f "/etc/os-release" || return 2
    ( eval "$(grep -m1 "^$key=" /etc/os-release); echo \$$key"; )
}

called:

$ __eor_key VARIANT
ComputeNode

Most, if not all our taskout.log's are filled with these:

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   /distribution/crashes/check-abrt
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

** (rstrnt-report-log:21753): WARNING **: ...: Option -T deprecated! Update your code.

Uploading journal.xml 

Versions:

beakerlib-1.17-13.el7bkr.1.noarch

I'm looking to create an install script for beakerlib which is based only on Bash and doesn't use make or rpm. The idea is to install beakerlib in environments where make or rpm aren't available similar to what other popular open source tools do. An example command will look like
curl -o- https://raw.githubusercontent.com/beakerlib/beakerlib/master/install.sh | bash

Can you give me a general direction what would be acceptable ?

I would recommend using rpm.labelCompare instead.

$ rlCmpVersion 'ab' 'b'
>
$ python
Python 2.6.6 (r266:84292, Jun 11 2019, 11:01:44) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm
>>> rpm.labelCompare(('1','1','ab'), ('1', '1', 'b'))
-1
>>> rpm.labelCompare(('1','1','1'), ('1', '1', '2'))
-1

How to reproduce:
rlAssertLesser "0 is not lesser than 0 => Failed" 0 0
rlAssertLesser "1 is not lesser than 1 => Failed" 9 9
rlAssertLesser "-1 is not lesser than -1 => Failed" -1 -1

As we already talked with @sopos , we think it will be beneficial to have the way of specifying custom rlIsRHEL (and others) behaviour.
Rationale behind this is that if I am testing new package version (say for RHEL-X.Y), at least in initial phases it is run on previous Y-stream version (RHEL-X.(Y-1)) - whole compose might be available much later. If I test new feature as part of existing test, I decide on running the particular part of test based on rlIsRHEL, therefore it would be really beneficial to have an option to make beakerlib think it is already running on the newer version.
Suggested implementation is that rlIsRHEL considers some environment variable (say BEAKERLIB_OS_RELEASE) and if it is set, information is used directly and not found out from /etc/{os,redhat}-release.