Git Product home page Git Product logo

ibm / elm-python-client Goto Github PK

View Code? Open in Web Editor NEW
23.0 10.0 11.0 7.15 MB

Python client for IBM Enterprise Lifecycle Management applications with example commandline applications for OSLC Query export to CSV from DOORS Next (DN/DNG), Enterprise Workflow Management (EWM/RTC) and Enterprise Test Management (ETM/RQM), ReqIF import/export, and DOORS Next Reportable REST export to CSV/XML

License: Other

Python 19.14% HTML 53.34% CSS 6.41% JavaScript 20.91% Makefile 0.10% Yacc 0.07% Jinja 0.03%
oslc query oslc-query reqif automation exporter python enterprise lifecycle management

elm-python-client's Introduction

Python client for IBM Enterprise Lifecycle Management applications

Python client for IBM Enterprise Lifecycle Management applications

(c) Copyright 2021- IBM Inc. All rights reserved

SPDX-License-Identifier: MIT

version="0.26.2"

What's New?

11-Dec-2023

  • Added _validate.py to provide the Validate API, available on all apps (may need to restrict this to just ccm/gc/qm/rm?)
  • Added examples/validate.py as an example of using the validate API - for most options refer to oslcquery readme

16-Aug-2023

  • tested with Python 3.11.4 - worked OOTB. Now developing using 3.11.4 - won't be back-tested with older Pythons but should work back to 3.9 at least.

4-May-2023

  • Deprecated RM load_folders() - use find_folder() instead.
  • Added RM create_folder() - this doesn't require doing a forced reload of all folders, because it inserts the new folder into the internal info about folders
  • Added example dn_simple_createfolderpath.py - allows exercising the create_folders() function
  • Added rm-only implementation of create/deliver changeset NOTE these aren't fully finished, specifically hte delivery result may not be obvious(!)
  • Added example dn_simple_typesystemimport_cs.py which shows creating/delivering a changeset around typesystem import

Introduction

The aim of this code is to provide a Python client for the IBM Enterprise Lifecycle Management (ELM) applications.

IMPORTANT NOTES:

  • This code is not developed, delivered or supported in any way as part of the IBM ELM applications
  • This code is not supported other than by the efforts of the author and contributors in this github repository
  • This code is not intended to be complete/comprehensive - it provides functionality needed by the examples and little else.

Included in this package are a few examples using elmclient which are reasonably functional:

  • OSLC Query able to work with DOORS Next (DN), Engineering Test Management (ETM), Engineering Workflow Management (EWM) and Global Configuration management (GCM)
  • ReqIF import/export for DOORS Next
  • Basic Reportable REST for DOORS Next - very basic!
  • More general Reportable REST, currently only for DOORS Next but intended to expand to cover EWM and ETM.

There are links to these examples below.

Installation

Either method of install described below installs the elmclient package and puts example commands (such as oslcquery into your path so a) they can be run simply by typing the command, e.g. oslcquery and b) as you edit the source code these commands automatically use the latest code.

Requirements: Python 3.11/3.10/3.9 - NOTE I'm developing using Python 3.11.4 and compatibility with older versions is NOT checked.

Overview

Step 1: Install Python so it can be run from a command prompt Either Step 2a: Quickest and easiest to get started: install elmclient from pypi Or Step 2b: If you want to change elmclient code

Step 1

Install Python so you can run Python from the commandline - might be python3 if you're on *nix. On Windows the command is python - you can find install guides all over the internet.

Step 2a - Quickest and easiest to just use elmclient

This method is also easiest to update with new versions of elmclient.

at a command prompt:

  • for Windows type pip install elmclient
  • For *nix use pip3 install elmclient

To update:

  • for Windows type pip install -U elmclient
  • For *nix use pip3 install -U elmclient

Using this method you could copy the examples from where they're lurking in your Python installation's library to a different folder, rename if using one of the commands such as oslcquery to a different name, and edit them in this folder separate from the elmclient install.

Test that all was successful by running oslcquery -h you should get a version number then a swathe of text with all the options.

Step 2b - If you want to edit the code in elmclient

This method assumes you have developer knowledge how to modify and merge code.

By far the preferred method is to first fork the github repository. You'll then get a folder on your PC which has a sub-folder `elmclient'.

Open a command prompt in the folder which has a subfolder elmclient and run the command (Windows) pip install -e . or (*nix) pip3 install -e .

Test that all was successful by running oslcquery -h you should get a version number then a swathe of text with all the options.

Coding using the elmclient

You code will import elmclient, then use it.

The basis of using the elmclient is to first create a "server", then add the needed application(s) to it - typically just one application such as rm, or perhaps more applications such as rm and gc.

Then you can use the API functions to find projects, components, configurations, etc.

The DN reportable REST example provides a simple functional example with hard-coded values for the project, configuration and the artifact ID to be queried. This is the easy way to get into using elmclient - by modifying this example.

The other examples add fairly complex details around the use of elmclient to provide a commandline interface and should provide again a starting point for further development.

Authentication (in httpops.py)

The auth code works with:

  • form authentication using Liberty in local user registry
  • LDAP (using JTS setup for LDAP) and OIDC (Jazz Authorisation Server, which might be configured for LDAP)

Other authentication methods haven't been tested.

You'll have to provide a username and password; that username will determine the permissions to read/write data on your server, just as they would through a browser UI.

The examples oslcquery and reqif_io layer authentication enhancements on top of this to allow saving obfuscated credentials to a file so you don't have to provide these on the commandline every time. See the code for these examples.

Handling different context roots

It's possible to install the ELM applications to run on non-standard context roots like /rm1 for DOORS Next, or /scm for EWM. This is handled in elmclient using APPSTRINGs. These specify the domain of an application using rm, jts, gc, ccm, qm, and the context root as a string without the leading /. So for example /rm1 would be specified as rm:rm1, or /scm would be specified as ccm:scm.

For example, if your DN is on /rm then just specify rm. Or, if it's on /rm23 then specify rm:rm23.

If more than one application is needed then use a comma separate list (without spaces). The main application is specified first, but if jts is also on /jts1 then your APPSTRING could be rm:rm1,jts:jts1.

Example code provided

These examples drove the evolution of elmclient:

  • OSLC Query - read more here - this is the largest example by quite a margin. It enables commandline export to CSV from the supported applications using an abstract syntax for OSLC Query - commandline options
  • ReqIF import/export - read more here - this allows limited CRUD of reqif definitions in DOORS Next, performing multiple reqif export using a definition, and multiple reqif import - commandline options
  • General Reportable REST export to XML/CSV - read more here - currently only implemented for DOORS Next but with potential to expand to EWM and ETM - commandline options
  • Simple Simple DOORS Next Module Structure API - read more here - Access a module structure and print out the indentend artifiact titles with section number - hardcoded
  • Simple DOORS Next Reportable REST - read more here - this is a very simple example of using elmclient to access the DOORS Next Reportable REST API - hardcoded
  • .. and more, check the examples folder...

ELM APIs

This code provides examples of using various ELM APIs:

  • DN

** Process - elmclient

** OSLC including OSLC Query - oslcquery.py for user use, internally oslcqueryapi.py implements OSLC Query parsing and querying

** Module Structure - dn_simple_modulestructure.py - currently external to elmclient

** ReqIF - reqif_io.py - currently reqif API is external to elmclient

** Reportable REST (incomplete for qm and ccm) - represt.py for user use, internally for each application in _rm.py, _ccm.py, _qm.py

  • ETM

** Process - elmclient

** OSLC including OSLC Query - oslcquery.py for user use, internally oslcqueryapi.py implements OSLC Query parsing and querying

** Reportable REST (incomplete for qm and ccm) - represt.py for user use, internally for each application in _rm.py, _ccm.py, _qm.py

  • EWM

** Process - elmclient

** OSLC including OSLC Query - oslcquery.py for user use, internally oslcqueryapi.py implements OSLC Query parsing and querying

** Reportable REST (incomplete for qm and ccm) - represt.py for user use, internally for each application in _rm.py, _ccm.py, _qm.py

  • GCM

** Process - elmclient

** OSLC including OSLC Query - oslcquery.py for user use, internally oslcqueryapi.py implements OSLC Query parsing and querying

Reporting issues, and contributing

If you find a problem with elmclient you can report it on the github issues https://github.com/IBM/ELM-Python-Client/issues - note this is just for issues with elmclient code. All other issues will likely be closed immediately.

You can do a pull request to propose updates - there's no guarantee of if/when/how these will be merged but we certainly hope to benefit from contributions!

elm-python-client's People

Contributors

barny avatar jjrevuel avatar stevemar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

elm-python-client's Issues

oslcquery with DN 7.0.2 - Error trying to process rule "identifier" when using oslc select with custom identifier

When using "oslc.select" statement with a custom identifier/attribute such as "NEWID", I got a runtime error:

Exception has occurred: VisitError
Error trying to process rule "identifier":

Name resolution for NEWID failed!
File "C:\Users\REDACTED\elmclient_queryparser.py", line 619, in identifier
raise Exception("Name resolution for %s failed!" % (resultname))
Exception: Name resolution for NEWID failed!

What could be the problem? It was working before with DOORS v 6.x.x. We just upgraded to DN 7.0.2
When running the types report, we got the variable/property name with the URI.

Thank you

Add ability to convert a list of results into a count of the list length

When querying for example for modules rdm_types:ArtifactFormat=jazz_rm:Module and specifying for select oslc_rm:uses, the results contain all the artifacts in each module - want to add the ability to totalize the list so putting a count into that column - using this you can count artifacts/module

Might want to disable name resolution while doing this, because the results will be as large as the total number of arrtifacts across all modules, and name resolution is needed to do this.

Provide a query syntax for use in oslcquery to say 'find the configuration with this name'

This might be used for example to enable commandline querying such as with gc for finding uses of a contribution, which uses a query that embeds the local contribution URL, e.g. for gcm:

So there needs to be a way to specify the application (and project/component?) that will be searched for the configuration name, perhaps a prefix like rm: or gc:

And there needs to be syntax to delimit it.

A challenge with this for RM is that there's no way to search globally for configurations so have to open each project and scan all its configurations.

dn_simple_modulestructure.py fails with an exception when provided a module name that also has other similarly named requirement artifacts

When running the python program dn_simple_modulestructure.py, the following exception error is raised:
More than one module with that name in project <project name> component <component name> configuraition <configuration name>

It seems that the filter parameters passed using whereterms on line 175 are only partially used - just the OSLC "title" property is matched, and second filter term, the Artifact Format type of Module, seems to be ignored.

When I made the Module name (OSLC title property) unique from all the other requirement artifact names in the component, I was able to successfully execute the program.

I was also unable to fix the whereterms even though I used the fully qualified (using the full namespace URI for jazz_rm) uri for Module Artifact format name (as returned by the query). i.e. the following change didn't work either:

   whereterms = [
        ["dcterms:title", "=", f'"{mod}"'],
        [
            "rdm_types:ArtifactFormat",
            "=",
            f'"{rdfxml.RDF_DEFAULT_PREFIX["jazz_rm"]}Module"',
        ],
    ]

Add (for at least RM) a Python API for OSLC Configuration Management (within a project/component)

DNG (at least) supports OSLC Configuration Management - there's current implementation to find/select a configuration which works across dng and ETM, but it would be useful to also be able to create (and deliver) changeset, stream, baseline. This will have to handle opt-out for 7.x and hopefully 6.0.6.1 (which are different) - will likely need a refacter/redsign of the Project to be object-oriented, and of the apps that need it to be able to create the appropriate project area type. A demonstrator would be reqif import which: creates baseline before, create changeset for the import, import into the CS, deliver the CS, take a baseline after.

RM Update artifact with a custom attribute

Hi, I was able to update the primary text using update artifact for a Requirement Artifact,
and I'm looking to find out how to search the tags for a custom attributes, I saw the print out of the tags, but nothing comes familiar in the available tags when "print(f"{ET.tostring(theartifact_x)=}\n")"
after

now get the artifact content and its etag

theartifact_x, etag = c.execute_get_rdf_xml(theartifact_u, return_etag=True, intent="Retrieve the artifact")

is the there a specific resource I need to sub-navigate to find out the custom attributes?
Thank you

How to Use -s/-n/-v property with an attribute with spaces

Discussed in #32

Originally posted by pablogarijo March 2, 2023
How can I use the -s/-n/-v property with an attribute with spaces (F.E "Planned SW Release Version" or 'Satisfied By Architecture Element')?

I am getting the next error:

dcterms:title,Planned SW Release Version,dcterms:identifier,r
^
Expected one of:

  • COMMA
  • LBRACE
  • DOT
  • COLON

I have tried introducing the data with "", with '' and with both at the same time, but I keep getting errors.

oslcquery -J https://doorsng.grupoantolin.com:9443/ -U -P -p "" -C "SW_DALU" -s dcterms:title -F "SW_DALU_V3.0" -q "((oslc:instanceShape='SW Non-Functional Requirement') || (oslc:instanceShape='SW Functional Requirement'))" -n "Planned SW Release Version" -O result3.csv

this is my query

examples for oslcquery with IBM cloud (trial)?

is there a possibility to provide a working example to use oslcquery with IBM cloud? in our case, it is trial, as we are testing a new version.

we set:
set QUERY_JAZZURL=https://jazz.net
set QUERY_USER=xxx
set QUERY_PASSWORD=xxx
set QUERY_JTS=sandbox01-qm

and then we run:
oslcquery -A qm:sandbox01-qm,gc:sandbox01-gc -G "abc (Global Configuration) Initial Development" -p "abc (Quality Management)" -E "abc (Global Configuration)" -O testcases_jazz.csv -s *

similar composition and parameters work with our current server.

We are getting errors like:

Version 0.26.2
Exception on retrying request. URL: https://jazz.net/sandbox01-gc/process/project-areas, 401, 
Error 401: CRJZS0360I The remote authorization server is requesting user authorization to access the remote service provider&#39;s resources.
...

  File "C:\Program Files\Python312\Lib\site-packages\elmclient\httpops.py", line 622, in _execute_one_request_with_login
    response.raise_for_status()
  File "C:\Users\xxx\AppData\Roaming\Python\Python312\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://jazz.net/sandbox01-gc/process/project-areas

We were also checking for a solution here: #43 , without sucess

thanks for any help!

RM how to retrieve value/label from enumerated attribute

I've been working in a project that has made extensive use of custom artifacts and attributes, and was wondering if there was a way to translate the URI returned for enumerated attributes by oslc_query methods into the corresponding value/label.

In the artifact XML returned by theartifact_x, etag = self.config.component.execute_get_rdf_xml( theartifact_u, return_etag=True, intent="Retrieve the artifact" ) we've been able to retrieve and modify our custom attributes by parsing the output, but for enumerated attributes the only return value is an 'rdf:resource' URI (example provided below).

<j.0:enumAttribute rdf:resource="https://jazz.external.company.com/rm/types/AT_YizZiKjKEe2vXr_tc4cF9Q#_9s-KxivuIym9A1QTfgQjpA"/>

How can I retrieve the value and label associated with this enumerated URI?

Login issue

Hello,

I'm starting now with the library and I do succeed to login to the system. The Jazz is configured to have automatic (Windows) login via Active Directory. I specified the environment variables QUERY_JAZZURL, QUERY_USER and QUERY_PASSWORD with the correct values but the query:

reqif_io.exe -F "Initial Stream" -C "Overall requirements" "Rocket project" list

throws:
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://<<jazzurl>>/rm/process/project-areas

What do I wrong or what I need to specify to get connected.

The oslcquery command works with the same settings.

The dotted field names are not being parsed when using do_complex_query

Hi,
When I try to use do_complex_query -> select kwarg like select='rtc_cm:x.y.z' I get lark.exceptions.UnexpectedCharacters error with message

No terminal matches '.' in the current parser context, at ... 
rtc_cm:x.y.z
        ^
Expected one of: 
         * COMMA 
         * LBRACE

Same select statement seems fine when using execute_oslc_query.

oslcquery speed up large --percontribution querying by parallelising query with processing of results from previous query

One example doing counting of artifacts/module in a very large GC project with 1253 contributions containing 25886 modules and total of 6722401 bindings was spending about 50% of the time processing results, sequentially with the queries; if the processing can be done in parallel with the next query that's a significant speedup. Probably doesn't need anything more fancy/complicated than processing in parallel with the next query, but I suppose some more speedup might be possible with a controlled number of queries in parallel, at the cost of server load.

-q rdm_types:ArtifactFormat=jazz_rm:Module -s oslc_rm:uses,dcterms:title,dcterms:identifier

Enhance the server API to log additional information like 'intent' of a call, which should be logged, and investigate if logs can be automatically turned into HTML sequence diagrams

Aim is to be able to better understand/document the sequence of API calls involved in achieving any particular objective
Also enhance the examples to provide these intents
Sequence diagrams like https://bramp.github.io/js-sequence-diagrams/
Add annotations with the intents - also filter out routine/boring headers to reduce clutter, but show body content except for HTML, javascript and images/icons.
Pretty much every sequence will start with a prelude of login/discovery for the project/component/configuration then the actual application requests, e.g. get the structure of a module, will just be the last one or two events on the sequence diagram - is it possible to somehow make the prelude stuff condensed but you can expand it if you want to?

Error after upgrade to DOORS NG 7.0.2 oslcquery with a 400 response code and ClassCastException

Hello:
After implementing the DOORS NG 7.02 upgrade I encountered this error: [ClassCastException: com.ibm.rdm.service.views.model.impl.View incompatible with com.ibm.rdm.service.artifacts.model.IArtifact] at 292th page as shown at the bottom of this log file. Prior to upgrade the query works fine with DOORS NG 6.x.x

First line from the log file:

2023-10-10 17:04:54,930 [TRACE|elmclient.utils] COMMANDLINE: oslcquery -p RM_PROJECT -F "s" -r Requirement -s "s" -O output\RM_PROJECT_artifacts.csv --pagesize 200 -R -N -L INFO -U REDACTED -P REDACTED

--- 488000+ lines ---

Last lines from the log file:

INTENT: Retrieve 292th page of OSLC query results

GET https://rm01.redacted.com:9443/rm/views?oslc.pageSize=200&oslc.paging=true&oslc.prefix=dcterms%3D%3Chttp%3A%2F%2Fpurl.org---REDACTED---&page=292
Accept: application/rdf+xml
Accept-Encoding: gzip, deflate
Cache-Control: no-store, max-age=0
Connection: keep-alive
Cookie: jfs-oauth-access-token0=REDACTEDTOKEN; jfs-oauth-access_token-secret0=REDACTEDTOKEN; JAZZ_AUTH_TOKEN=REDACTEDTOKEN; JSESSIONID=REDACTEDTOKEN; LtpaToken2=REDACTEDTOKEN
OSLC-Core-Version: 2.0
Referer: https://rm01.redacted.com:9443/rm/web
User-Agent: python-requests/2.31.0
X-Requested-With: XMLHttpRequest

2023-10-10 17:16:43,817 [TRACE|elmclient.httpops]
WIRE: response ----- 400

Response: 400
Connection: close
Content-Encoding: gzip
Content-Language: en-US
Content-Type: application/rdf+xml; charset=UTF-8
Date: Wed, 11 Oct 2023 00:16:41 GMT
Set-Cookie: JAZZ_AUTH_TOKEN=REDACTEDTOKEN; Path=/rm; Secure; HttpOnly; SameSite=None
Strict-Transport-Security: max-age=31536000
Transfer-Encoding: chunked
Vary: Accept-Encoding
Warning: 110 - "Automatically cached! Response is Stale."
X-Powered-By: Servlet/3.0
X-com-ibm-team-Trace-Identifier: 136C1829
cache-control: public
expires: Wed, 18 Oct 2023 00:16:41 -0000

::::::::::@
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:err="http://jazz.net/xmlns/prod/jazz/foundation/1.0/">
rdf:Description
<err:detailedMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Propagated exception; original message [ClassCastException: com.ibm.rdm.service.views.model.impl.View incompatible with com.ibm.rdm.service.artifacts.model.IArtifact]</err:detailedMessage>
<err:errorMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Bad Request</err:errorMessage>
<err:errorStatus rdf:datatype="http://www.w3.org/2001/XMLSchema#long"
>400</err:errorStatus>
</rdf:Description>
</rdf:RDF>

----------@

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<!

2023-10-10 17:16:43,817 [TRACE|elmclient.httpops] HTTPError 400 Client Error: Bad Request for url: https://rm01.redacted.com:9443/rm/views?oslc.pageSize=200&oslc.paging=true&oslc.prefix=dcterms%3D%3Chttp%3A%2F%2Fpurl.org---REDACTED---&page=292
2023-10-10 17:16:43,817 [TRACE|elmclient.httpops] WIRE: handle content-style auth redirect
2023-10-10 17:16:43,817 [ERROR|elmclient.httpops] Exception on executing request. URL: https://rm01.redacted.com:9443/rm/views?oslc.pageSize=200&oslc.paging=true&oslc.prefix=dcterms%3D%3Chttp%3A%2F%2Fpurl.org---REDACTED---&page=292, 400, <rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:err="http://jazz.net/xmlns/prod/jazz/foundation/1.0/">
rdf:Description
<err:detailedMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Propagated exception; original message [ClassCastException: com.ibm.rdm.service.views.model.impl.View incompatible with com.ibm.rdm.service.artifacts.model.IArtifact]</err:detailedMessage>
<err:errorMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Bad Request</err:errorMessage>
<err:errorStatus rdf:datatype="http://www.w3.org/2001/XMLSchema#long"
>400</err:errorStatus>
</rdf:Description>
</rdf:RDF>

Thank you

load_components_and_configurations fails when trying to get all components and configurations from one QM project area

Hi there,

First of all - I am not an OSLC expert, just a user who wants to do something with this package.

Currently I am a user which has access only to some particular QM project areas (not all project areas).
So when trying to find a specific component (which is accessible to me), I use this simple code snipped:

import elmclient.server as elmserver
...

obj_server = elmserver.JazzTeamServer(str_host, str_user, str_pass, verifysslcerts=False, jtsappstring="jts:jts", appstring="qm", cachingcontrol=0)
obj_app = obj_server.find_app('qm', ok_to_create=True)
obj_project = obj_app.find_project(str_project_area_name)
obj_component = obj_project.find_local_component(str_component_name)

Which throws the following error:

Traceback (most recent call last):
  File "c:\XXX\.venv\Lib\site-packages\elmclient\_qm.py", line 296, in find_local_component
    self.load_components_and_configurations()
  File "c:\XXX\.venv\Lib\site-packages\elmclient\_qm.py", line 112, in load_components_and_configurations
    crx = self.execute_get_xml( components_uri, intent="Retrieve component definition" )
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\XXX\.venv\Lib\site-packages\elmclient\httpops.py", line 139, in execute_get_xml
    response = request.execute( **kwargs )
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\XXX\.venv\Lib\site-packages\elmclient\httpops.py", line 334, in execute
    return self._execute_request( no_error_log=no_error_log, close=close, **kwargs )
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\XXX\.venv\Lib\site-packages\elmclient\httpops.py", line 343, in _execute_request
    result = self._execute_one_request_with_login( no_error_log=no_error_log, close=close, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\XXX\.venv\Lib\site-packages\elmclient\httpops.py", line 624, in _execute_one_request_with_login
    response.raise_for_status()
  File "c:\XXX\.venv\Lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://host/qm/oslc_config/resources/com.ibm.team.vvc.Component

When trying to access the URL https://host/qm/oslc_config/resources/com.ibm.team.vvc.Component using my web browser, I get the following error message displayed:

The user "XXX" does not have permission to read item "_xxxxxxxxxxxxxxxxxxxxxxxx", which has the "ProjectArea" item type.

It seems that it querys all components of all project areas - including those which are not accessible for me - for some reason.
Does anybody know how to evade this issue?

Different results whith same osclquery

Hello,
I am experiencing some issues when requesting same oslcquery to Doors NG.

The type of request i am using is like the following:

oslcquery -J https://my.server:port -U usr -P pwd -A rm,gc,jts:jts23 -p "Project Name" -C "Module_Name" -F "Local Stream Name" -q "oslc:instanceShape in ['Artifact type 1','Arctifact Type 2']" -O "data/projName/reqs.csv" -s "dcterms:identifier,oslc:instanceShape,rm:module" -W

Also tried without any query(-q) just to check if it was the problem:

oslcquery -J https://my.server:port -U usr -P pwd -A rm,gc,jts:jts23 -p "Project Name" -C "Module_Name" -F "Local Stream Name" -O "data/projName/reqs.csv" -s "dcterms:identifier,oslc:instanceShape,rm:module" -W

But the result is the same:

We get different results by executing the same request (with no changes applied in doors side).
For instance we execute it once and get the correct result (e.g. 7500 requirements), and 5 minutes later do it again and only get 7000 requirements.

I don't know if is there any limitations on how many results the oslcquery can return at once. But it seems strange that sometimes it works pretty fine but other times it misses some results.

I would really appreciate if you have any suggestion on how could I proceed to avoid this behaviour.

Thanks a lot in advance!

Regards,
Sergio

Query for all qm components on a GC

Hello!

We are trying to get all testcases from a qm project.
This project is configuration management enabled and has more than one components. All this components are contributing to a GC.

We are trying to query to this qm project with the GC project and GC stream, and the query works but only extracts the testcases from the default component.

Do you know if is there a way to extract test cases from all the components or any workaround to deal with this?

Thanks a lot in advance!

How to select a ReqIF definition

I have a question: how to specify for reqif_io a certain ReqIF definition, if there more than one, for export. Currently all definitions will be exported.

Error during query with satisfies links

Hello,
I have been using the oslc query for retrieving artifacts attribute from DNG /rm.

When using global configuration, I am executing a query to retrieve artifacts with links of type 'Satisfies'.
To do that I execute a query as the following one:

oslcquery -J https://my.server:port -U usr -P pwd -A rm,gc,jts:jts23 -p "req_project_name" -E "gc_project_name" -G "gc_stream" -O data/proj/SWreqs.csv -s "dcterms:identifier,oslc:instanceShape,rm:module,Satisfies" -W --pagesize 0 

With this, I am getting the following error:

Version 0.19.0
querying for gc config **proj** V3.0
Loading DN shapes: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 70/70 [00:12<00:00,  5.62 results/s]
Preparing Query
Query completed in 1 page(s)
Processing       :  55%|█████████████████████████████████████████████████████████████████████████████████████████▊                                                                         | 14796/26837 [00:47<07:03, 28.44 results/s]T
raceback (most recent call last):
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\Scripts\oslcquery.exe\__main__.py", line 7, in <module>
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\examples\oslcquery.py", line 808, in main
    do_oslc_query(sys.argv[1:])
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\examples\oslcquery.py", line 569, in do_oslc_query
    results = queryon.do_complex_query( args.resourcetype, querystring=args.query, searchterms=args.searchterms, select=args.select, isnulls=args.null, isnotnulls=args.value
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\oslcqueryapi.py", line 214, in do_complex_query
    remappedvalue.append(self.resolve_uri_to_name(lv))
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\_project.py", line 307, in resolve_uri_to_name
    if ( result := self.app_resolve_uri_to_name(uri) ) is None:
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\_rm.py", line 751, in app_resolve_uri_to_name
    result = self.resource_id_from_uri(uri)
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\_rm.py", line 797, in resource_id_from_uri
    resource_xml = self.execute_get_rdf_xml(reluri=uri, intent="Retrieve type RDF to get its id (dcterms:identifier)")
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\httpops.py", line 152, in execute_get_rdf_xml
    response = request.execute( **kwargs )
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\httpops.py", line 317, in execute
    return self._execute_request( no_error_log=no_error_log, close=close, **kwargs )
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\httpops.py", line 326, in _execute_request
    result = self._execute_one_request_with_login( no_error_log=no_error_log, close=close, **kwargs)
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\elmclient\httpops.py", line 531, in _execute_one_request_with_login
    response.raise_for_status()
  File "C:\Users\sergio.marcos2\.conda\envs\kpienv\lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: **base_url**/rm/resources/BI_Jj_gc4qEEe2X94hUXsF8pw?oslc_config.context=**base_url**%3A9443%2Fgc%2Fconfiguration%
2F71
Processing       :  55%|█████████████████████████████████████████████████████████████████████████████████████████▎                                                                        | 14796/26837 [00:47<00:38, 311.40 results/s]

And actually, after checking, is true that with the given URI(if I check at navigator) there is no artifact with that URI UUID among that global configuration.
So checking further, in previous gc streams it was and I detected the id(and the requirement itself) is in the stream I'm looking for, but the URI UUID seems that changed between gc streams. So it is something like this status:

At GC stream 2.0 -> ID: 356124 || UUID: BI_Jj_gc4qEEe2X94hUXsF8pw
At GC stream 3.0 -> ID: 356124 || UUID: BI_XBPSkwM_Ee6lBfMIxW_p0w

I do not know, if for this change at the uri UUID it might be problems when quering, I have cheked the link at doors and it is okey, even tried removing link and adding again.

So I'd like to ask if you think this could be the reason why this error is appearing or you would know an approach to solve it, as when the error appears no results gets generated, so I cannot check for other values if are being properly processed(I guess yes), But I used similar queries for other projects and worked pretty fine.

--
Also, I am not querying here by any artifact type because the ones i need to look for, has same name in different components of the project but some of them have URIs defined and others not and I cannot change as some are already baselined (Querying by the artifact type name within global configuration do not return the needed result).

Thanks a lot in advance.
Regards!

New User: How to use OSLCQuery correctly

I am trying to follow the github article: https://github.com/IBM/ELM-Python-Client/blob/master/elmclient/examples/OSLCQUERY.md

to use OSLCQuery on our SaaS Test Manager.

My query (run from a command prompt) :
oslcquery -J "https://telent-dng.clm.ibmcloud.com/" -A qm -p "TPCMS inc RemoteSecure" -P "xxx*" -U "xxx" -s dcterms:title -O qmresults.csv

returns the unhelpful response:
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\daves\anaconda3\Scripts\oslcquery.exe__main
.py", line 7, in
File "C:\Users\daves\anaconda3\Lib\site-packages\elmclient\examples\oslcquery.py", line 814, in main
do_oslc_query(sys.argv[1:])
File "C:\Users\daves\anaconda3\Lib\site-packages\elmclient\examples\oslcquery.py", line 359, in do_oslc_query
c = p.find_local_component(args.component)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\daves\anaconda3\Lib\site-packages\elmclient_qm.py", line 291, in find_local_component
self.load_components_and_configurations()
File "C:\Users\daves\anaconda3\Lib\site-packages\elmclient_qm.py", line 72, in load_components_and_configurations
assert compuri is not None, "compuri is None"
^^^^^^^^^^^^^^^^^^^
AssertionError: compuri is None
Version 0.23.0
Warning - project 'TPCMS inc RemoteSecure' is opt-in but you didn't specify a component or a global configuration - using default component 'TPCMS inc RemoteSecure' - if this component name doesn't exist you'll get an error message

so far as I can tell our SaaS v7.0.2 Project is opt-in, but even adding '-C 8371' produces no .csv file,
what am I doing wrong?
David Sedar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.