Parameter 'layers' can not be empty.

I don't get the image when running a test. I only get this. sorry....first time user...

Please see opengeospatial/cite#56

We want to include other OGC tests in the ETF Inspirational Validator and we do not know where to find the references (dependencies) to the Team Engine Suist tests, could I get help?

Currently the validator adds test for WFS 2.0 adding the dependencies:

<con:property>
      <con:name>etf.supported.test**.object.type.ids**</con:name>
      <con:value>EID9981e87e-d642-43b3-ad5f-e77469075e74</con:value>
    </con:property>
    <con:property>
      <con:name>etf.**dependency.ids**</con:name>
      <con:value>EID95a1b6fc-2b55-3d43-9502-</con:value>
    </con:property>

Where can we find these identifiers (ids) in the case of WMS, or WMTS?

thanks!

Manual: https://github.com/opengeospatial/cite/wiki/How-to-run-tests-for-test-suites-with-SoapUI

Example how these tests were created for ets-wfs20 and teamengine: opengeospatial/teamengine#318 (comment)

Hint: Please keep the tests as close as possible to the ets-wfs20 tests. As far as I see, the SoapUI project can be copied from ets-wfs20 and just minor adjustments have to be done.

Currently, REST URI of test suite is [TEAM_ENGINE]/rest/suites/wms/ which is equal to WMS 1.1 test suite.

Example: See Beta environment: http://cite.opengeospatial.org/te2/rest/suites/wms/

REST URI of test suite shall contain major and minor version of specification: [TEAM_ENGINE]/rest/suites/wms13/

Parameter basic=basic is required to execute the basic test.
If the parameter is missing only the 'data-independent' conformance class is executed.

Behaviour should be the same as in the Web GUI.

Applies to integration and system tests.

SoapUI project must be updated as version was removed from URI to test suites of REST API (also see opengeospatial/teamengine#261).
Please also update Maven configuration and set version of SoapUI Maven Plugin to 5.5.0.

See updated documentation: https://github.com/opengeospatial/cite/wiki/How-to-run-tests-for-test-suites-with-SoapUI

Example, how this was solved for TEAM Engine tests: opengeospatial/teamengine#400

There seems to be a problem, I get this error:

Clients may request TRANSPARENT=TRUE on a layer that is opaque.
A sequence of more than one item is not allowed as the first argument of string() (, )

In the code I found this line:

<xsl:variable te:loc="3178,file:/mnt/testsuite/1er/ctl/1430827765737/getmap.xml" name="layer"
                       select="string(wms:Capability//wms:Layer[wms:Name and functions:boolean-as-integer(@opaque)=1][1]/wms:Name)"/>

I use this service:

https://cardo.cottbus.de/net3/public/ogc.ashx?nodeId=643&Service=WMS&Request=GetCapabilities&

If I test following XPath I get 2 hits: WMS_Capabilities//Layer[Name and @opaque = 1][1]/Name
If I use braces I get the expected one hit: (WMS_Capabilities//Layer[Name and @opaque = 1])[1]/Name

Could you check this please.
Greetings Stefan

Test getmap:transparent-default type Mandatory default result Passed (s0002/d711e387_1/d1794e56_1/d1794e339_1)

Assertion: When a GetMap request is made with no TRANSPARENT parameter and a FORMAT that supports transparency over a layer that is not opaque, then the response contains no transparent pixels.

Context:
Label:
Value: WMS_Capabilities element

Request d1794e5681_1:
Method: get
URL: http://myserver/wms?&BbOx=-0.0025,-0.0025,0.0025,0.0025&ReQuEsT=GetMap&StYlEs=&LaYeRs=map_map_lakes&VeRsIoN=1.3.0&FoRmAt=image%2Fpng&CrS=CRS%3A84&HeIgHt=100100&WiDtH=100100
Response from parser parsers:ImageParser:

Messages from parser parsers:ImageParser:
     No image handlers available for the data stream. null

Result: Failed

the OpenGIS WMS implementation specification here http://portal.opengeospatial.org/files/?artifact_id=14416 on heading H.1 provides a sample XML capabilities URL which is valid. the URL is here: http://schemas.opengis.net/wms/1.3.0/capabilities_1_3_0.xml when you paste this url in the OGC testing validator online, the test shows that this fails with 6 errors. Some of the errors are that there are no images returned with errors, the getCapabilities request made without a version number fails, getCapabilities made for version 1.3.0 fails, request for version 100.0.0 fails, request for version 0.0.0 fails, etc.
Moreover, although, a message states that the server can handle reserved characters in http get urls properly there is more errors for escaped chars, basic_elements and data-independent failure errors.
I also used the online validator tool to test some getCapabilities URL for existing certified companies and their URLs also fail in the OGS testing site.

My concern is that I get the same 6 errors as the sample valid OGC XML in the link above but for my own URL. I would like to know if there is a bug with the online validator that it fails all the time no matter what URL is being used. Is there a sample valid getCapabilities XML URL that I can try to test the validator to see if it passes at least one URL?

Could I also get a validation if the online OGC validator is capable of handling https urls? I read the closed issues about this topic but didn't get a concrete yes or no answer, it looks some https work some don't, what are the requirements for the https to work?

Some organizations are trying to run profiles, like the WMS DGIWG profile available here:

https://svn.opengeospatial.org/ogc-projects/cite/scripts/wms/profiles/dgiwg/ed1.3/trunk/

When trying to run the script via command line using the the DGIWG profile test as the source, the error message “no test suite found” is presented and nothing runs – no dialog box appears or anything.

Reported here: http://lists.opengeospatial.org/pipermail/cite-forum/2016-January/001572.html

The WMS 1.3.0 specification (OGC 06-042) section 6.3.3 states:

An Online Resource URL intended for HTTP GET requests is in fact only a URL prefix to which additional parameters are appended in order to construct a valid Operation request. A URL prefix is defined in accordance with IETF RFC 2396 as a string including, in order, the scheme (“http” or “https”), Internet Protocol hostname or numeric address, optional port number, path, mandatory question mark “?”, and optional string comprising one or more server-specific parameters ending in an ampersand “&”.

However, WMS 1.3.0 CITE tests don't seem to expect this, since it blindly adds an ampersand even if the prefix already ends in a question mark or ampersand.

For example, "https://demo.cubewerx.com/cubewerx/cubeserv/cite_wms130?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.3.0" advertises the following HTTP GET URL prefix:

https://demo.cubewerx.com/cubewerx/cubeserv/cite_wms130?

The CITE tests respond by making requests like the following:

https://demo.cubewerx.com/cubewerx/cubeserv/cite_wms130?&FoRmAt=text%2Fxml&ReQuEsT=GetCapabilities&VeRsIoN=1.3.0&SeRvIcE=WMS

(Note the "?&".) I'm not even sure if this is valid HTTP(s) URL syntax.

I have downloaded the Team Engine and Test Scripts and have run the WMS 1.3.0 compliance tests (r9 and r10) locally against our test server.

Two tests: bbox-outside-crs and invalid-crs are failing with the same error, I assume that this is related to the interpretation of the capabilities document.

I include the logs for the failing tests below and the capabilities document returned by our server.

<log>
<starttest local-name="bbox-outside-crs" prefix="getmap" namespace-uri="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/getmap.xml" type="Mandatory" defaultResult="1" path="s0049/d3652e372_1/d1990e29_1/d1990e105_1" file="C:\team_engine_4\TE_BASE\work\_team_engine_4_TE_BASE_scripts_wms-1.3.0-r10_ctl_\getmap\getmap$bbox-outside-crs.test">
<assertion>When a request contains a Bounding Box that falls outside of the defined range of the CRS, then the server returns empty content for the area.</assertion>
<params xmlns:cgf="http://www.opengis.net/cite/geometry"
        xmlns:getmap="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/getmap.xml"
        xmlns:ogc="http://www.opengis.net/ogc"
        xmlns:xsd="http://www.w3.org/2001/XMLSchema"
        xmlns:citef="http://www.opengis.net/cite/functions"
        xmlns:xlink="http://www.w3.org/1999/xlink"
        xmlns:saxon="http://saxon.sf.net/"
        xmlns:gml="http://www.opengis.net/gml"
        xmlns:xs="http://www.w3.org/2001/XMLSchema"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:wms="http://www.opengis.net/wms"
        xmlns:cdf="http://www.opengis.net/cite/data"
        xmlns:ccf="http://www.opengis.net/cite/complex"
        xmlns:ctl="http://www.occamlab.com/ctl"
        xmlns:functions="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/functions.xml"
        xmlns:tec="java:com.occamlab.te.TECore"
        xmlns:gen="java:com.occamlab.te.Generator"
        xmlns:tems="java:com.occamlab.te.web.MonitorServlet"
        xmlns:parsers="http://www.occamlab.com/te/parsers"
        xmlns:te="http://www.occamlab.com/te"/>
<context label=""><value>WMS_Capabilities element</value></context>
</starttest>
<exception><![CDATA[A sequence of more than one item is not allowed as the first argument of string() (<Name/>, <Name/>) ]]></exception>
<endtest result="6"/>
</log>

<log>
<starttest local-name="invalid-crs" prefix="getmap" namespace-uri="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/getmap.xml" type="Mandatory" defaultResult="1" path="s0049/d3652e372_1/d1990e39_1/d1990e146_1" file="C:\team_engine_4\TE_BASE\work\_team_engine_4_TE_BASE_scripts_wms-1.3.0-r10_ctl_\getmap\getmap$invalid-crs.test">
<assertion>When the CRS parameter in a GetMap request contains a CRS that is not valid for any of the layers being requested, then the server shall throw a Service Exception (code=InvalidCRS).</assertion>
<params xmlns:cgf="http://www.opengis.net/cite/geometry"
        xmlns:getmap="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/getmap.xml"
        xmlns:ogc="http://www.opengis.net/ogc"
        xmlns:xsd="http://www.w3.org/2001/XMLSchema"
        xmlns:citef="http://www.opengis.net/cite/functions"
        xmlns:xlink="http://www.w3.org/1999/xlink"
        xmlns:saxon="http://saxon.sf.net/"
        xmlns:gml="http://www.opengis.net/gml"
        xmlns:xs="http://www.w3.org/2001/XMLSchema"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:wms="http://www.opengis.net/wms"
        xmlns:cdf="http://www.opengis.net/cite/data"
        xmlns:ccf="http://www.opengis.net/cite/complex"
        xmlns:ctl="http://www.occamlab.com/ctl"
        xmlns:functions="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/functions.xml"
        xmlns:tec="java:com.occamlab.te.TECore"
        xmlns:gen="java:com.occamlab.te.Generator"
        xmlns:tems="java:com.occamlab.te.web.MonitorServlet"
        xmlns:parsers="http://www.occamlab.com/te/parsers"
        xmlns:te="http://www.occamlab.com/te"/>
<context label=""><value>WMS_Capabilities element</value></context>
</starttest>
<exception><![CDATA[A sequence of more than one item is not allowed as the first argument of string() (<Name/>, <Name/>) ]]></exception>
<endtest result="6"/>
</log>

<?xml version="1.0" encoding="UTF-8"?>
<WMS_Capabilities xmlns="http://www.opengis.net/wms" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.3.0" xsi:schemaLocation="http://www.opengis.net/wms http://schemas.opengis.net/wms/1.3.0/capabilities_1_3_0.xsd">
<Service>
  <Name>WMS</Name>
  <Title>Smallworld GeoSpatial Server WMS.</Title>
  <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
</Service>
<Capability>
  <Request>
    <GetCapabilities>
      <Format>text/xml</Format>
      <DCPType>
        <HTTP>
          <Get>
            <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
          </Get>
        </HTTP>
      </DCPType>
    </GetCapabilities>
    <GetMap>
      <Format>image/png</Format>
      <DCPType>
        <HTTP>
          <Get>
            <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
          </Get>
        </HTTP>
      </DCPType>
    </GetMap>
    <GetFeatureInfo>
      <Format>text/xml</Format><Format>application/json</Format>
      <DCPType>
        <HTTP>
          <Get>
            <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
          </Get>
        </HTTP>
      </DCPType>
    </GetFeatureInfo>
  </Request>
  <Exception>
    <Format>XML</Format>
  </Exception>      
  <Layer queryable="1"><Name>Smallworld</Name>
    <Title>Smallworld generic layer</Title>
    <CRS>urn:ogc:def:crs:EPSG:4326</CRS><CRS>urn:ogc:def:crs:EPSG:4326</CRS><CRS>urn:ogc:def:crs:EPSG:54003</CRS><CRS>CRS:84</CRS>
   <EX_GeographicBoundingBox><westBoundLongitude>-180.0</westBoundLongitude><eastBoundLongitude>180.0</eastBoundLongitude><southBoundLatitude>-90.0</southBoundLatitude><northBoundLatitude>90.0</northBoundLatitude></EX_GeographicBoundingBox>     
    <BoundingBox CRS="urn:ogc:def:crs:EPSG:4326" minx="-90.0" miny="-180.0" maxx="90.0" maxy="180.0" /><BoundingBox CRS="CRS:84" minx="-180.0" miny="-90.0" maxx="180.0" maxy="90.0" /><BoundingBox CRS="urn:ogc:def:crs:EPSG:54003" minx="-2.0037508342789244E7" miny="-1.4691480769173065E7" maxx="2.0037508342789244E7" maxy="1.4691480769173063E7" />
    <Style><Name>Default</Name><Title>Default</Title></Style><Style><Name>Simple</Name><Title>Simple</Title></Style><Style><Name>Plotting</Name><Title>Plotting</Title></Style><Style><Name>Outline_2</Name><Title>Outline_2</Title></Style><Style><Name>Outline_1</Name><Title>Outline_1</Title></Style> 
    <Layer queryable="1"><Name>cite:BasicPolygons</Name><Title>cite:BasicPolygons</Title></Layer><Layer queryable="1"><Name>cite:Bridges</Name><Title>cite:Bridges</Title></Layer><Layer queryable="1"><Name>cite:Buildings</Name><Title>cite:Buildings</Title></Layer><Layer queryable="1"><Name>cite:DividedRoutes</Name><Title>cite:DividedRoutes</Title></Layer><Layer queryable="1"><Name>cite:Forests</Name><Title>cite:Forests</Title></Layer><Layer queryable="1"><Name>cite:Lakes</Name><Title>cite:Lakes</Title></Layer><Layer queryable="1"><Name>cite:MapNeatline</Name><Title>cite:MapNeatline</Title></Layer><Layer queryable="1"><Name>cite:NamedPlaces</Name><Title>cite:NamedPlaces</Title></Layer><Layer queryable="1"><Name>cite:Ponds</Name><Title>cite:Ponds</Title></Layer><Layer queryable="1"><Name>cite:RoadSegments</Name><Title>cite:RoadSegments</Title></Layer><Layer queryable="1"><Name>cite:Streams</Name><Title>cite:Streams</Title></Layer><Layer queryable="1"><Name>parks</Name><Title>parks</Title></Layer><Layer queryable="1"><Name>Rest Theme</Name><Title>Rest Theme</Title></Layer>
  </Layer>
</Capability>
</WMS_Capabilities>

Hi,

I'm trying to run ets-wms13 on QGIS Server but when I run tests, I have the next issue described in the logfile:

Suite wms-1.3.0:ets-wms13-auto Failed
[Fatal Error] log.xml:42:33: An invalid XML character (Unicode: 0x10) was found in the element content of the document.
[Fatal Error] report_logs.xml:1:9704: Error attempting to parse XML file (href='/srv/tomcat/teamengine/temp/caea4ef4-c62e-44e2-9d51-9999b5d1df32/d1e17333_1/d1e17415_1/d1e3456_1/d1e3506_1/log.xml').
Mar 06, 2017 9:44:26 AM com.sun.jersey.spi.container.ContainerResponse logException
SEVERE: Mapped exception to response: 500 (Internal Server Error)

Versions currently used:

• teamengine: 4.10
• ets-wms13: 1.20
• Ubuntu 16.04.1 LTS

Some scripts are available here to build a Docker container provisioned with Ansible with Teamengine, ets-wms13 and QGIS Server : https://github.com/pblottiere/docker-qgis-ogc-cite.

So I think you can easily reproduce the error.

Thank you for your work!

Service: http://forums.igenesys.com/iGenVision/cite/wms?service=WMS&request=GetCapabilities&version=1.3.0

Error:

Test Name : getmap:invalid-crs type Mandatory
net.sf.saxon.s9api.SaxonApiException: Error in call to extension function {public synchronized void com.occamlab.te.TECore.callTest(net.sf.saxon.expr.XPathContext,java.lang.String,java.lang.String,net.sf.saxon.om.NodeInfo,java.lang.String) throws java.lang.Exception}: Exception in extension function java.util.NoSuchElementException
	at net.sf.saxon.s9api.XsltTransformer.transform(XsltTransformer.java:192)
	at com.occamlab.te.TECore.executeTemplate(TECore.java:576)
	at com.occamlab.te.TECore.executeTest(TECore.java:787)
	at com.occamlab.te.TECore.callTest(TECore.java:989)
	at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)

In the result of the currrent session for example: test WCS 2.0.1 and click on the test case, it will show the CTL of this test case, like this http://cite.opengeospatial.org/teamengine/listings/srv_local_teamengine-prod_TE_BASE_scripts_wcs_2.0.1_ctl_ext_crs.html#wcs2:crs

But for WMS 1.3, it will throw exception e.g:
http://cite.opengeospatial.org/teamengine/listings/worksrv_local_teamengine-prod_TE_BASE_scripts_wms_1.3.0_ctl_main.xml.html#getcapabilities:dims-time

`message /teamengine/listings/worksrv_local_teamengine-prod_TE_BASE_scripts_wms_1.3.0_ctl_main.xml.html

description The requested resource is not available.`

This issue occurred only when we stop the test execution in the middle, the following tags are missing in log file (TE_BASE\users\s0034\log.xml) :

1.The overall result i.e. <endtest result="1"/> element is missing.

2.The end tag </tag> is missing.

Attached log file: s0034.zip

The issues link is pointing to:
http://cite.opengeospatial.org/te2/about/wms/1.3.0/site

And should be:
https://github.com/opengeospatial/ets-wms13/issues

The getfeatureinfo:each-queryable-layer tests pass an empty CRS string when testing "https://demo.cubewerx.com/cubewerx/cubeserv/cite_wms130?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.3.0". E.g., it sends:

https://demo.cubewerx.com/cubewerx/cubeserv/cite_wms130?&LaYeRs=citedata_wms_130.Autos&WiDtH=100&HeIgHt=100&I=50&BbOx=0,0,1,1&CrS=&ReQuEsT=GetFeatureInfo&InFo_fOrMaT=application%2Fgml%2Bxml%3B%20version%3D%223.2%22&J=50&StYlEs=&FoRmAt=image%2Fpng&QuErY_LaYeRs=citedata_wms_130.Autos&VeRsIoN=1.3.0

Note the Crs= parameter. The response to this request is an exception report complaining about this, which of course causes the test to fail.

Develop an ETS that allows to test only WMS that publishes rasters.

See related email: http://lists.opengeospatial.org/pipermail/cite-forum/2011-February/000436.html

Reason: some products specialize in raster handling and does not support vector input.

The test needs to be updated to better provide information about the grouping of the conformance class and determination about which ones are the basic conformance classes needed for certification.

Changes that are required are described here:

https://github.com/opengeospatial/cite/wiki/Encoding-Conformance-Classes

I have downloaded the Team Engine and Test Scripts and have run the WMS 1.3.0 compliance tests (r9 and r10) locally against our test server. There are a number of tests failures. The tests fail for a variety of reasons but all have a common factor: in each request the CRS parameter is set to “NoCRSForLayer”
The failing tests are:

bbox-direct
crs-direct
each-style
first-style-invalid
second-style-invalid
styles-direct
styles-inherited
styles-some-default
three-styles
two-styles

The log from the failing bbox-direct test case is included below.

I wonder if this could be because crs, bounding box and style information is only associated with the top level layer and is inherited by all the other layers. This maybe is related to issue 885.

The capabilities document that the server returns is included below.

The log from the failing bbox-direct test case

<log>
<starttest local-name="bbox-direct" prefix="getmap" namespace-uri="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/getmap.xml" type="Mandatory" defaultResult="1" path="s0049/d3652e372_1/d1990e29_1/d1990e79_1" file="C:\team_engine_4\TE_BASE\work\_team_engine_4_TE_BASE_scripts_wms-1.3.0-r10_ctl_\getmap\getmap$bbox-direct.test">
<assertion>When a GetMap request contains a layer whose BoundingBox is contained directly in the layer, then the response is valid.</assertion>
<params xmlns:cgf="http://www.opengis.net/cite/geometry"
        xmlns:getmap="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/getmap.xml"
        xmlns:ogc="http://www.opengis.net/ogc"
        xmlns:xsd="http://www.w3.org/2001/XMLSchema"
        xmlns:citef="http://www.opengis.net/cite/functions"
        xmlns:xlink="http://www.w3.org/1999/xlink"
        xmlns:saxon="http://saxon.sf.net/"
        xmlns:gml="http://www.opengis.net/gml"
        xmlns:xs="http://www.w3.org/2001/XMLSchema"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:wms="http://www.opengis.net/wms"
        xmlns:cdf="http://www.opengis.net/cite/data"
        xmlns:ccf="http://www.opengis.net/cite/complex"
        xmlns:ctl="http://www.occamlab.com/ctl"
        xmlns:functions="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/functions.xml"
        xmlns:tec="java:com.occamlab.te.TECore"
        xmlns:gen="java:com.occamlab.te.Generator"
        xmlns:tems="java:com.occamlab.te.web.MonitorServlet"
        xmlns:parsers="http://www.occamlab.com/te/parsers"
        xmlns:te="http://www.occamlab.com/te"/>
<context label=""><value>WMS_Capabilities element</value></context>
</starttest>
<request id="d1990e420_1">
<ctl:request xmlns:ctl="http://www.occamlab.com/ctl" xmlns:cgf="http://www.opengis.net/cite/geometry" xmlns:getmap="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/getmap.xml" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:citef="http://www.opengis.net/cite/functions" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:saxon="http://saxon.sf.net/" xmlns:gml="http://www.opengis.net/gml" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wms="http://www.opengis.net/wms" xmlns:cdf="http://www.opengis.net/cite/data" xmlns:ccf="http://www.opengis.net/cite/complex" xmlns:functions="https://cite.opengeospatial.org/wms-1.3.0/src/ctl/functions.xml" xmlns:tec="java:com.occamlab.te.TECore" xmlns:gen="java:com.occamlab.te.Generator" xmlns:tems="java:com.occamlab.te.web.MonitorServlet" xmlns:parsers="http://www.occamlab.com/te/parsers" xmlns:te="http://www.occamlab.com/te"><ctl:url>http://3.69.209.27:8080/gssogc/ogc?</ctl:url><ctl:method>get</ctl:method><ctl:param name="LaYeRs">Smallworld</ctl:param><ctl:param name="BbOx">0,0,1,1</ctl:param><ctl:param name="WiDtH">200</ctl:param><ctl:param name="CrS">NoCRSForLayer</ctl:param><ctl:param name="ReQuEsT">GetMap</ctl:param><ctl:param name="VeRsIoN">1.3.0</ctl:param><ctl:param name="HeIgHt">200</ctl:param><ctl:param name="StYlEs"/><ctl:param name="FoRmAt">image%2Fpng</ctl:param><parsers:HTTPParser><parsers:parse><parsers:NullParser/></parsers:parse></parsers:HTTPParser></ctl:request>
<response><parser local-name="HTTPParser" namespace-uri="http://www.occamlab.com/te/parsers" prefix="parsers"/><content><response><status code="200" protocol="HTTP/1.1">OK</status><headers><header name="Server">Apache-Coyote/1.1</header><header name="X-Powered-By">Servlet/3.0; JBossAS-6</header><header name="Set-Cookie">JSESSIONID=0B855DF2FB7C7A216595653492EC5048; Path=/gssogc</header><header name="Content-Type">text/xml;charset=UTF-8</header><header name="Transfer-Encoding">chunked</header><header name="Date">Mon, 20 Apr 2015 11:27:12 GMT</header></headers><content/><!--Response received in [748] milliseconds--></response></content></response>
</request>
<endtest result="6"/>
</log>

The capabilities document that the server returns.

<?xml version="1.0" encoding="UTF-8"?>
<WMS_Capabilities xmlns="http://www.opengis.net/wms" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.3.0" xsi:schemaLocation="http://www.opengis.net/wms http://schemas.opengis.net/wms/1.3.0/capabilities_1_3_0.xsd">
<Service>
  <Name>WMS</Name>
  <Title>Smallworld GeoSpatial Server WMS.</Title>
  <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
</Service>
<Capability>
  <Request>
    <GetCapabilities>
      <Format>text/xml</Format>
      <DCPType>
        <HTTP>
          <Get>
            <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
          </Get>
        </HTTP>
      </DCPType>
    </GetCapabilities>
    <GetMap>
      <Format>image/png</Format>
      <DCPType>
        <HTTP>
          <Get>
            <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
          </Get>
        </HTTP>
      </DCPType>
    </GetMap>
    <GetFeatureInfo>
      <Format>text/xml</Format><Format>application/json</Format>
      <DCPType>
        <HTTP>
          <Get>
            <OnlineResource xlink:type="simple" xlink:href="http://3.69.209.27:8080/gssogc/ogc?" />
          </Get>
        </HTTP>
      </DCPType>
    </GetFeatureInfo>
  </Request>
  <Exception>
    <Format>XML</Format>
  </Exception>      
  <Layer queryable="1"><Name>Smallworld</Name>
    <Title>Smallworld generic layer</Title>
    <CRS>urn:ogc:def:crs:EPSG:4326</CRS><CRS>urn:ogc:def:crs:EPSG:4326</CRS><CRS>urn:ogc:def:crs:EPSG:54003</CRS><CRS>CRS:84</CRS>
   <EX_GeographicBoundingBox><westBoundLongitude>-180.0</westBoundLongitude><eastBoundLongitude>180.0</eastBoundLongitude><southBoundLatitude>-90.0</southBoundLatitude><northBoundLatitude>90.0</northBoundLatitude></EX_GeographicBoundingBox>     
    <BoundingBox CRS="urn:ogc:def:crs:EPSG:4326" minx="-90.0" miny="-180.0" maxx="90.0" maxy="180.0" /><BoundingBox CRS="CRS:84" minx="-180.0" miny="-90.0" maxx="180.0" maxy="90.0" /><BoundingBox CRS="urn:ogc:def:crs:EPSG:54003" minx="-2.0037508342789244E7" miny="-1.4691480769173065E7" maxx="2.0037508342789244E7" maxy="1.4691480769173063E7" />
    <Style><Name>Default</Name><Title>Default</Title></Style><Style><Name>Simple</Name><Title>Simple</Title></Style><Style><Name>Plotting</Name><Title>Plotting</Title></Style><Style><Name>Outline_2</Name><Title>Outline_2</Title></Style><Style><Name>Outline_1</Name><Title>Outline_1</Title></Style> 
    <Layer queryable="1"><Name>cite:BasicPolygons</Name><Title>cite:BasicPolygons</Title></Layer><Layer queryable="1"><Name>cite:Bridges</Name><Title>cite:Bridges</Title></Layer><Layer queryable="1"><Name>cite:Buildings</Name><Title>cite:Buildings</Title></Layer><Layer queryable="1"><Name>cite:DividedRoutes</Name><Title>cite:DividedRoutes</Title></Layer><Layer queryable="1"><Name>cite:Forests</Name><Title>cite:Forests</Title></Layer><Layer queryable="1"><Name>cite:Lakes</Name><Title>cite:Lakes</Title></Layer><Layer queryable="1"><Name>cite:MapNeatline</Name><Title>cite:MapNeatline</Title></Layer><Layer queryable="1"><Name>cite:NamedPlaces</Name><Title>cite:NamedPlaces</Title></Layer><Layer queryable="1"><Name>cite:Ponds</Name><Title>cite:Ponds</Title></Layer><Layer queryable="1"><Name>cite:RoadSegments</Name><Title>cite:RoadSegments</Title></Layer><Layer queryable="1"><Name>cite:Streams</Name><Title>cite:Streams</Title></Layer><Layer queryable="1"><Name>parks</Name><Title>parks</Title></Layer><Layer queryable="1"><Name>Rest Theme</Name><Title>Rest Theme</Title></Layer>
  </Layer>
</Capability>
</WMS_Capabilities>

Assertion: For format "image/png8", when the FORMAT parameter is set to that format, the MIME type of the response matches that format.

Issue: When the requested image format is image/png8 the test expects the response header to contain image/png8 . The server returns image/png because image/png8 is not a valid Content-Type. This makes the server fail the test.

The media type as per iana is here: https://www.iana.org/assignments/media-types/image/png.

The test should not expect the media type header to be image/png8 or image/png24 as these are not valid iana media types. It should just be image/png

The main:time test tries to test the multipleValues and nearestValue attributes of the dimension tag using the following code:

<xsl:if test="functions:boolean-as-integer($dim/@multipleValues) != 1 or functions:boolean($dim/@multipleValues) != 'true' ">
                  <ctl:message>Error: The time dimension for the cite:Autos layer must be defined with multipleValues="true".</ctl:message>
                  <ctl:fail/>
               </xsl:if>
               <xsl:if test="functions:boolean-as-integer($dim/@nearestValue) != 1 or functions:boolean($dim/@nearestValue) != 'true' ">
                  <ctl:message>Error: The time dimension for the cite:Autos layer must be defined with nearestValue="true".</ctl:message>
                  <ctl:fail/>
</xsl:if>

This is incorrect as there is no boolean function, and also redundant as the boolean-as-integer function tests both for numbers and true/false string values.

Running the test causes the following error:

Error in call to extension function {public synchronized void com.occamlab.te.TECore.callTest(net.sf.saxon.expr.XPathContext,java.lang.String,java.lang.String,net.sf.saxon.om.NodeInfo,java.lang.String) throws java.lang.Exception}: Exception in extension function java.lang.Exception: Fatal Error: XPath syntax error at char 99 on line 71 in {...dim/@multipleValues) != 'tr...}:
Cannot find a matching 1-argument function named {https://cite.opengeospatial.org/wms-1.3.0/src/ctl/functions.xml}boolean(); SystemID: file:/srv/local/teamengine-te2/TE_BASE/work/srv_local_teamengine-te2_TE_BASE_scripts_wms_1.3.0_ctl_/main/main$time.test; Line#: 71; Column#: -1
Fatal Error: XPath syntax error at char 95 on line 78 in {...($dim/@nearestValue) != 'tr...}:
Cannot find a matching 1-argument function named {https://cite.opengeospatial.org/wms-1.3.0/src/ctl/functions.xml}boolean(); SystemID: file:/srv/local/teamengine-te2/TE_BASE/work/srv_local_teamengine-te2_TE_BASE_scripts_wms_1.3.0_ctl_/main/main$time.test; Line#: 78; Column#: -1
Failed to compile stylesheet. 2 errors detected.

Issue 1: The specification talks bout 2 conformance classes: WMS Basic and Queryable WMS. We should refractor the test so the root parent in the CTL only has two children "WMS Basic" and "Queryable WMS".

Issue 2: We have these options in the test:

• Raster ELEVATION
• Vector ELEVATION
• TIME

Are all these a conformance class or?

Team Engine 4.6.
Specification: WMS – 1.3.0 [1.16]
Service: http://ws.cadcorp.com/ogctest/ogctest.exe?

I don’t understand why test 4.2.1 (getcapabilities:normative-schema) failed. I believe our service returns a correct XML schema document.

from CITE-Forum

The WMS 1.3.0 CITE test suite only allows the boolean attributes of the element to have the values "0" and "1". However, the more canonical forms of "false" and "true" are allowed by both the specification and the schema.

The following tests fail if a WMS 1.3.0 capabilities document legitimately returns "false"/"true" values for these attributes:
main:std-data-queryable
basic_elements:extra-GetFeatureInfo-param
getfeatureinfo:main

I believe this bug arose because the WMS 1.3.0 test suite evolved from the WMS 1.1.1 test suite, and in WMS 1.1.1 only "0" and "1" are legal values.

During WMS server 1.3.0 compliance testing of http://demo.luciad.com:8080/ComplianceLightspeedOGC2020/wms, I encountered 2 failures related to the handling of GetFeatureInfo requests; this happened with the tests "feature_count_default" and "feature_count_1". While investigating these failures, I noticed the following:

• both tests seem to include a GetFeatureInfo location (50,50) that does not align with the geometry of the used basic polygon layer. The requested map looks like http://demo.luciad.com:8080/ComplianceLightspeedOGC2020/wms?FoRmAt=image%2Fjpeg&InFo_fOrMaT=text%2Fplain&CrS=CRS:84&I=50&HeIgHt=100&J=50&ReQuEsT=GetMap&StYlEs=&QuErY_LaYeRs=cite:BasicPolygons&VeRsIoN=1.3.0&LaYeRs=cite:BasicPolygons&BbOx=-2,2,2,6&WiDtH=100 On this image, 50,50 is in an empty area. Could this be a bug? The other WMS GetMap request tests with visual tests are all ok, so I assume the rendered maps are fine.
• the response to the requests sent by both tests is empty (because of the previous bullet), but this triggers a test failure: Error in call to extension function {public org.w3c.dom.NodeList com.occamlab.te.TECore.request(org.w3c.dom.Document,java.lang.String) throws java.lang.Throwable}: Exception in extension function java.lang.reflect.InvocationTargetException

The images that were previewed by the user running the test, shall be saved to session folder of user

Related to: opengeospatial/ideas#119

Adjust release notes
https://github.com/opengeospatial/ets-wms13/blob/master/src/main/site/markdown/relnotes.txt

• do proper markdown syntax
• edit the revision notes
• update link from index.md

Adjust configuration
opengeospatial/cite#24

Specification OGC 06-042 describes in chapter 7.3.3.1 VERSION that the version parameter of a GetMap request is mandatory.
Chapter 7.3.4 GetMap response states that an exception shall be returned (HTTP status code is not clear for me).

A service which is accepting a GetMap request without version parameter is currently passing the test suite.

Also, the ATS does not describe such a test: https://opengeospatial.github.io/ets-wms13/wms-1_3_0-ats.html

The logo URL reported by our WMS 1.3.0 server is an HTTPS URL. I see nothing in the specification that forbids this. However, it causes the getcapabilities:logourls test to fail with the following error:

Error in call to extension function {public org.w3c.dom.NodeList com.occamlab.te.TECore.request(org.w3c.dom.Document,java.lang.String) throws java.lang.Throwable}: Exception in extension function java.lang.reflect.InvocationTargetException
Result: Failed

If I replace the URL with an HTTP url, the test passes.

The test results cannot be displayed due to malformed markup. The root cause is

org.xml.sax.SAXParseException; lineNumber: 28; columnNumber: 38; Character reference "&#26" is an invalid XML character.

It occurs when an XML entity is expected in response but a non-XML representation (i.e. an image) was received instead. It has been observed for tests that expect an exception report, such as bbox-minx-eq-maxx and bbox-miny-eq-maxy.

I noticed an issue with the optional test main:time (not sure if this also affects the elevation tests). If the values for nearestValue and multipleValues is set to "true" the test throws an exception:

Error in call to extension function {public synchronized void com.occamlab.te.TECore.callTest(net.sf.saxon.expr.XPathContext,java.lang.String,java.lang.String,net.sf.saxon.om.NodeInfo,java.lang.String) throws java.lang.Exception}: Exception in extension function java.lang.RuntimeException: net.sf.saxon.s9api.SaxonApiException: Cannot convert string "true" to a double

Setting them to "1" instead works, but as the schema defines these values as booleans shouldn't both variants be ok?

The test log includes the following error message:

A sequence of more than one item is not allowed as the first argument of
string() (<Name/>, <Name/>, ...)

This occurs when attempting to identify a non-queryable layer using the
following expression:

    string(wms:Capability//wms:Layer[wms:Name and functions:boolean-as-integer(@queryable) = 0]/wms:Name)

There may be more than one non-queryable layer in the capabilities document,
yielding a sequence.

[Originally reported in the OGC CITE issue tracker: issue #950](https://portal
.opengeospatial.org/?m=projects&a=view&project_id=85&tab=5&act=details&issue_i
d=950)
Opened: 2014-06-05 12:39:52, last_updated: 2014-06-05 13:01:50

Test execution with Web GUI does not work any more due to the commits regarding the REST interface.

When a test session is started via GUI, the dialogue for the URL is skipped and thus the tests are executed without URL.

This leads to a failing main function:

Test main:main type Mandatory default result Passed (s0078)

Assertion: The implementation under test complies with the WMS 1.3.0 specification.

Request d229e16096_1:
   Method: get
   URL: 
Error in call to extension function {public org.w3c.dom.NodeList com.occamlab.te.TECore.request(org.w3c.dom.Document,java.lang.String) throws java.lang.Throwable}: Exception in extension function java.net.MalformedURLException: no protocol: 
Result: Failed

When testing my WMS for OGC conformity using the testdataset, I do not
see the "large diamond". But the reason seems to be a bug in the testengine:

src does not send the mandatory SERVICE parameter:

I guess this is a bug in the r10 of WMS 1.3 testsuite in team engine 4.0.5

Marco

Originally reported by Fischer, Jean-Michel 2013-10-30
Closed 2014-05-06 05:23:48
https://portal.opengeospatial.org/services/srv_public_issues.php?call=viewIssue&issue_id=885

Original Issue Details:
The parameter CrS is empty for some tests.

The SUT defines the CRS in the parent layer:

http://services.interactive-instruments.de/cite2013wfs/cite/cgi-bin/wms/wms/wms?request=GetCapabilities&version=1.3

Maybe it has something to do with issue 783?

The empty CRS-code appears in the following tests:

Test getmap:bbox-inherited
Test getmap:styles-some-default
Test getmap:first-style-invalid
Test getmap:second-style-invalid
Test getmap:each-style
Test getmap:styles-direct
Test getmap:each-layer

Currently 4.10 is referenced.
Current release is 5.3.
Wait until 5.4 is released.

The test: getmap:bbox-pixel-interpretation fails.
The Request contains width and height as follows:

&HeIgHt=7&CrS=CRS:84&BbOx=0.0016,-0.0012,0.0026,-0.0005&WiDtH=10&

the resulting Image from our implementation meets this criteria.

in the sorce of the test it says:

 <parsers:ImageParser>  
   <parsers:model value=\""RGB\"">  
      <parsers:subimage x=\""0\"" y=\""0\"" width=\""10\"" height=\""1\"">  
         <parsers:count bands=\""RGB\"" sample=\""0xffffff\""/>  
      </parsers:subimage>  
      <parsers:subimage x=\""0\"" y=\""6\"" width=\""10\"" height=\""1\"">  
         <parsers:count bands=\""RGB\"" sample=\""0xffffff\""/>  
      </parsers:subimage>  
      <parsers:subimage x=\""0\"" y=\""1\"" width=\""1\"" height=\""5\"">  
         <parsers:count bands=\""RGB\"" sample=\""0xffffff\""/>  
      </parsers:subimage>  
      <parsers:subimage x=\""9\"" y=\""1\"" width=\""1\"" height=\""5\"">  
         <parsers:count bands=\""RGB\"" sample=\""0xffffff\""/>  
      </parsers:subimage>  
   </parsers:model>  
 </parsers:ImageParser>  

The returned image looks like this (screensot in Gimp):

http://services.interactive-instruments.de/cite2013wfs/simpledemo/wms-resources/wms.png

Why is this test failing?

Originally reported in the OGC CITE issue tracker: issue #953

Entered by: Fischer, Jean-Michel

Opened: 2014-07-08 11:55:08 Last Updated: 2014-07-08 12:04:06

The getmap:invalid-crs test is intended to elicit an exception report by submitting a GetMap request where the CRS parameter value is "UndefinedCRS". However, if the SUT incorrectly responds with a non-XML entity (i.e. an image) it is still serialized to the test results file, thereby producing malformed XML which cannot be parsed when a report is generated.

A ServletException is thrown (status code 500), with root cause SAXParseException: 'Character reference "&#26" is an invalid XML character.'

#35

1. Need to add the conformance class configuration attributes in the conformance class test to represent the conformance classes in the EARL report.

For example: <ctl:test name="getcapabilities:main" isConformanceClass="true" isBasic="false">

"isBasic": Represent the basic conformance class. Its value is 'true' if that conformance class is basic otherwise is 'false'.

2. Update the testName with well-formatted name:

e.g. : <ctl:call-test name="getmap:main">
  <ctl:with-param name="exhaustive" select="$exhaustive"/>
  </ctl:call-test>

  <ctl:call-test name="getfeatureinfo:main"/>

  <ctl:call-test name="dims:main"/>

If you observed the above code snippet the name of the tests are the same and the prefixes are different. After execution of the test, only the name (prefix: name) appears in the log. In the above case, all the tests have the same name prefix i.e. "main". We can't recognize the test which belongs to by just using the name.

So we need good test names which we can use to represent the better HTML report.

3. We need all the conformance classes in the root test.

Following is the ctl sampleTest which represents the conformance class:

sampleTest.zip

getmap:bbox-pixel-interpretation

Request : http://localhost:19090/maps?&StYlEs=&HeIgHt=7&CrS=CRS:84&BbOx=0.0016,-0.0012,0.0026,-0.0005&WiDtH=10&FoRmAt=image%2Fpng&ReQuEsT=GetMap&VeRsIoN=1.3.0&LaYeRs=cite:Lakes

Assertion: The server interprets the bounding box as going around the "outside" of the pixels of the image rather than through the centers of the border pixels.

If it were possible to invoke a CTL test suite using a RESTful API (as can be done with a TestNG-based suite) it would be much easier to hide the details about how a test suite is implemented and to facilitate interworking with test suites developed for WMS profiles (e.g. DGIWG, NSG).

The teamengine-spi-ctl module provides a CtlExecutor that can be used by a suite-specific controller. This is the same mechanism used by a TestNG suite. A profile ETS could then simply invoke the base ETS in this manner and merge the results.

Please see: How to create Docker Images of test suites

Hello,

some of the WMS 1.1/1.3 tests fail because the request-url does not follow the standard in encoding reserved characters.

For example in the “wms:wmsops-getmap-params-exceptions-2”-test there is a parameter “FoRmAt” with a missing encoding of the whitespace-character in the value:
“...&FoRmAt=image/png; mode=24bit&...”

It should be: “...&FoRmAt=image/png;%20mode=24bit&...“

see https://en.wikipedia.org/wiki/Percent-encoding

Spaces must be encoded for servers to accept them as normal characters in values, otherwise they identify the term after the whitespace as request-method (normally “HTTP/1.1”). So the interpretion of the parameters end at this point and all other subsequent parameters will not be parsed, which leads to false-errors.

Output of response should be EARL.

This affects test each-crs.

I publish a WMS service, and test it with the online Team Engine, however, I can't get the image.
By checking the requests, I find that the parameter LAYERS passed to the getMap operation is empty.
you can test it with the service:
https://ags.geoq.cn/arcgis/services/test/CiteWMS/MapServer/WMSServer?request=GetCapabilities&service=WMS&version=1.3.0
Why the value of the LAYERS is empty?