Git Product home page Git Product logo

redfish-service-validator's People

Contributors

abhirupseal avatar aheynig-mkl avatar arnewiebalck avatar billdodd avatar edtanous avatar gtmills avatar haitaoso avatar hancjiang avatar inguin avatar inva-raritan avatar jautor avatar jordanchw avatar krellan avatar mmasque avatar mraineri avatar pboyd04 avatar pwvancil avatar samerhaj avatar shinchley avatar tomasg2012 avatar zbigniewj-se avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redfish-service-validator's Issues

Proper validation for additional properties

Validation for properties that appear additional in the Json Payload requires checking for the annotation Odata.AdditionalProperties in the appropriate schema, and the tool currently does not handle this.

Additional properties may also occur with incorrect type verification/expectation or incorrect capitalization, whether or not that should be made apparent.

Tool should use later errata version of schema if found, replacing earlier version

$metadata should not have to carry every errata version of the schema, just the latest errata supported.
Here is section of $metadata
<edmx:Reference Uri="/redfish/v1/schema/Resource_v1.xml">
<edmx:Include Namespace="Resource"/
<edmx:Include Namespace="Resource.v1_0_3"/>
<edmx:Include Namespace="Resource.v1_1_2"/>
<edmx:Include Namespace="Resource.v1_2_1"/>
<edmx:Include Namespace="Resource.v1_3_0"/>
</edmx:Reference>

Here is error from tool

ERROR - Reference in a Schema not in metadata, this may not be compatible with ServiceMode
ERROR - References missing in metadata: ['Resource.v1_1_0']

My proposal - references to v1_1_x should use v1_1_y so long as y > x

[Refactor] Having trouble with line 90 - check Content-Type

can you see what is wrong? Not case is it?

C:\python35\lib\site-packages\requests\packages\urllib3\connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
Something went wrong: 'content-type'
Traceback (most recent call last):
  File "RedfishServiceValidator.py", line 90, in callResourceURI
    if 'application/json' in response.headers['content-type']:
  File "C:\python35\lib\site-packages\requests\structures.py", line 56, in __getitem__
    return self._store[key.lower()][1]
KeyError: 'content-type'

Traceback (most recent call last):
  File "RedfishServiceValidator.py", line 721, in validateURI
    SchemaSoup, refDict, SchemaFullType, 'entitytype')
  File "RedfishServiceValidator.py", line 210, in getTypeDetails
    for element in innerschema.find_all(tagType, attrs={'name': SchemaType}):
AttributeError: 'NoneType' object has no attribute 'find_all'

Validation has failed.

```*************here are the actual headers captured by chrome***************
HTTP/1.1 200 OK
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-UA-Compatible: IE=11
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Type: application/json
Transfer-Encoding: chunked
Date: Tue, 09 Feb 2016 22:26:09 GMT
Server: lighttpd/1.4.39

Not parsing right version of schema?

Regards the test attached to the previous schema, why would I get this error

EthernetInterface.v1_0_0.EthernetInterface:VLANs: Could not finish compliance check on this property: KeyError, 'Resource.v1_0_0'

When the odata.type is #EthernetInterface.v1_2_0.EthernetInterface

There are lots of similar. Does this tool is not using v1_2_0?

Tool must support both forms for odata.context

Redfish supports the use of both OData-defined formats for the @odata.context values, the tool currently supports the "Resource Type" format, but does not support the "Resource Path" format. Both are valid Redfish and are in use. While we recommend the Resource Type format (because it's simpler), implementations are free to use either.

KeyError: 'uri'

Can you explain what is wrong? Could it be my subdirectory with schema is not right name?

G:_Redfish_git\Redfish-Service-Validator-Test\Redfish-Service-Validator>py -3 RedfishServiceValidator.py
RedfishServiceValidator Config details: (True, 'https://192.168.2.15', 'admin', 'metadata', False, False)

*** ServiceRoot, /redfish/v1
Starting new HTTPS connection (1): 192.168.2.15
C:\python35\lib\site-packages\requests\packages\urllib3\connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
@odata.context
@odata.id
@odata.type
@Redfish.Copyright
#ServiceRoot.1.0.0.ServiceRoot
Starting new HTTPS connection (1): 192.168.2.15
C:\python35\lib\site-packages\requests\packages\urllib3\connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
Traceback (most recent call last):
File "RedfishServiceValidator.py", line 809, in
success, counts, results = validateURI('/redfish/v1', 'ServiceRoot')
File "RedfishServiceValidator.py", line 703, in validateURI
refDict = getReferenceDetails(SchemaSoup)
File "RedfishServiceValidator.py", line 174, in getReferenceDetails
refDict[item['namespace']] = (item['namespace'], ref['uri'])
File "C:\python35\lib\site-packages\bs4\element.py", line 997, in getitem
return self.attrs[key]
KeyError: 'uri'

ComplianceLog_04_24_2017_191927.txt

Determine accuracy of tool's counts and accesses

Tool as it stands offers counts of the number of properties and which of it fails, skips, or passes. When testing comes around after refactor, a little bit of research into what is a "property" vs what is a check, what else is what, etc. Especially determine if absolutely all properties are checked, without somehow escaping early due to numerous exceptions.

The only determinate rule is that OEM is skipped, but whether or not it reflects in the total property count of this tool is to be determined.

Schema file and version should be determined by @odata.type, not @odata.context

Related to issue #63

The schema file related to a payload should be determined using the @odata.type , not the @odata.context value. The Context does not contain schema version information, which will be important for proper validation. And determining the proper schema file is considerably more difficult if the implementation uses the ResourcePath style for @odata.context.

Unexpected Error in Manager Actions: list

Reviewing output I see these error -
ERROR - #Manager.ForceFailover: action not Found
ERROR - #Manager.ModifyRedundancySet: action not Found

These 2 actions are indeed not available in the page but should the test tool note this as an error?

Python crash for missing reference in $metadata

Python exception in this case -
Server Page has this "@odata.type": "#Storage.v1_1_1.Storage"
Highest reference in $metadata is <edmx:Include Namespace="Storage.v1_1_0"/>

This is clearly an implementation error in system. However besides the exception there was no clear hint in the log file. A reasonable message printed in log would save user lots of time during system debug.

At least that is my recommendation.

Also - the error is not seen when LocalOnlyMode = True. Appears in this mode the $metadata is bypassed. That is why error slipped past engineer.

Paste of the exception:
File ".\RedfishServiceValidator.py", line 940, in <module> success, counts, results = validateURITree('/redfish/v1', 'ServiceRoot') File ".\RedfishServiceValidator.py", line 774, in validateURITree linkURI, uriName + ' -> ' + linkName, allLinks=allLinks) File ".\RedfishServiceValidator.py", line 774, in validateURITree linkURI, uriName + ' -> ' + linkName, allLinks=allLinks) File ".\RedfishServiceValidator.py", line 774, in validateURITree linkURI, uriName + ' -> ' + linkName, allLinks=allLinks) File ".\RedfishServiceValidator.py", line 758, in validateURITree validateSingleURI(URI, uriName, expectedType, expectedSchema, expectedJson) File ".\RedfishServiceValidator.py", line 838, in validateSingleURI success, SchemaSoup = getSchemaDetails( SchemaNamespace, SchemaURI=SchemaURI) File ".\RedfishServiceValidator.py", line 124, in getSchemaDetails refLink = getReferenceDetails(soup).get(getNamespace(frag))[1] TypeError: 'NoneType' object is not subscriptable

Log:
DEBUG - True, OrderedDict([('@Redfish.Copyright', 'Copyright (c) 2015-2017, Insyde Software Corp. All Rights Reserved.'), ('@odata.context', '/redfish/v1/$metadata#StorageCollection.StorageCollection'), ('@odata.id', '/redfish/v1/Systems/BQWT64900010/Storage'), ('@odata.type', '#StorageCollection.StorageCollection'), ('Name', 'Storage Collection'), ('[email protected]', 1), ('Members', [OrderedDict([('@odata.id', '/redfish/v1/Systems/BQWT64900010/Storage/1')])])]), 200 INFO - #StorageCollection.StorageCollection DEBUG - StorageCollection /redfish/v1/$metadata#StorageCollection.StorageCollection DEBUG - dont chkauth DEBUG - callingResourceURI: /redfish/v1/$metadata#StorageCollection.StorageCollection DEBUG - 200, [200], {'X-XSS-Protection': '1; mode=block', 'Transfer-Encoding': 'chunked', 'Date': 'Fri, 26 Feb 2016 23:52:23 GMT', 'Content-Type': 'application/xml', 'X-Frame-Options': 'SAMEORIGIN', 'X-UA-Compatible': 'IE=11', 'Strict-Transport-Security': 'max-age=31536000; includeSubdomains; preload', 'Server': 'lighttpd/1.4.39'}

HTML Output Very Wide

In recent tests I see very wide HTML in log file. Lots of spaces. Wide Columns. More than 3 screens wide.

Anyone else seeing this?

Is there some algorithm for determining the column widths?

Existing XML schema cannot be found.

I am running your tool against Redfish API exposed by reference Intel's implementation of RSD.
Script is reporting following problem:

sudo python3 RedfishServiceValidator.py
System Info: Abc
RedfishServiceValidator Config details: ('https://localhost:8443', 'user:myUser', './SchemaFiles/metadata', 'no CheckCert', 'Attempt for Online Schema')
Start time: 05/25/17 - 09:30:48

*** ServiceRoot, /redfish/v1
/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
#ServiceRoot.v1_1_1.ServiceRoot
/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
SchemaURI missing reference link: /redfish/v1/$metadata ServiceRoot
SchemaURI unsuccessful: /redfish/v1/$metadata
validateURI: No schema XML for #ServiceRoot.v1_1_1.ServiceRoot ServiceRoot
Elapsed time: 0:00:00
1
Counter({'passGet': 1, 'failSchema': 1})
Validation has failed: 1 problems found

I am using following configuration:

[Information]
Updated = May 8, 2017
Description = Redfish Service-Schema Conformance Tool 0.91

[SystemInformation]
TargetIP = localhost:8443
UserName = myUser
Password = myPassword
SystemInfo = Abc
[Options]
MetadataFilePath = ./SchemaFiles/metadata
UseSSL = True
LocalOnlyMode = False
CertificateCheck = Off
Session_UserName = Test1
Session_Password = TestPwd1

I have verified metadata exposed by my service and it seems that mentioned type is correctly defined there. Service root of my service looks like:

{

    "@odata.context": "/redfish/v1/$metadata#ServiceRoot",
    "@odata.id": "/redfish/v1",
    "@odata.type": "#ServiceRoot.v1_1_1.ServiceRoot",
    "Id": "ServiceRoot",
    "Name": "Service root",
    "Description": "POD Manager Service root",
    "RedfishVersion": "1.0.2",
    "UUID": "24b00548-eb4e-40c6-84c9-0064525837ac"
   ...
}

Metadata exposed under /redfish/v1/$metadata#ServiceRoot

<?xml version="1.0" encoding="UTF-8"?><edmx:Edmx xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx" Version="4.0">
  <edmx:Reference Uri="/redfish/v1/metadata/Org.OData.Core.V1.xml">
    <edmx:Include Alias="OData" Namespace="Org.OData.Core.V1"/>
  </edmx:Reference>

  <edmx:Reference Uri="/redfish/v1/metadata/ServiceRoot.xml">
    <edmx:Include Namespace="ServiceRoot.v1_1_1"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/IntelRackScaleOem.xml">
    <edmx:Include Namespace="Intel.Oem"/>
  </edmx:Reference>

  <edmx:DataServices>
    <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="Service">
      <EntityContainer Extends="ServiceRoot.v1_1_1.ServiceContainer" Name="Service">
        <Annotation String="Default Entity Container" Term="OData.Description"/>
      </EntityContainer>
    </Schema>
  </edmx:DataServices>
</edmx:Edmx>

Metadata exposed under /redfish/v1/metadata/ServiceRoot.xml:

<?xml version="1.0" encoding="UTF-8"?><!-- Copyright 2014-2015 Distributed Management Task Force, Inc. (DMTF). All rights reserved.--><edmx:Edmx xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx" Version="4.0">

  <edmx:Reference Uri="/redfish/v1/metadata/Org.OData.Core.V1.xml">
    <edmx:Include Alias="OData" Namespace="Org.OData.Core.V1"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/Resource.xml">
    <edmx:Include Namespace="Resource"/>
    <edmx:Include Namespace="Resource.v1_0_0"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/RedfishExtensions.xml">
    <edmx:Include Alias="Redfish" Namespace="RedfishExtensions.v1_0_0"/>
    <edmx:Include Alias="Validation" Namespace="Validation.v1_0_0"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/EventService.xml">
    <edmx:Include Namespace="EventService"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/AccountService.xml">
    <edmx:Include Namespace="AccountService"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/ComputerSystemCollection.xml">
    <edmx:Include Namespace="ComputerSystemCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/ComposedNodeCollection.xml">
    <edmx:Include Namespace="ComposedNodeCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/ChassisCollection.xml">
    <edmx:Include Namespace="ChassisCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/ManagerCollection.xml">
    <edmx:Include Namespace="ManagerCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/TaskService.xml">
    <edmx:Include Namespace="TaskService"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/SessionService.xml">
    <edmx:Include Namespace="SessionService"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/Message.xml">
    <edmx:Include Namespace="Message"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/JsonSchemaFileCollection.xml">
    <edmx:Include Namespace="JsonSchemaFileCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/MessageRegistryFileCollection.xml">
    <edmx:Include Namespace="MessageRegistryFileCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/SessionCollection.xml">
    <edmx:Include Namespace="SessionCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/StorageServiceCollection.xml">
    <edmx:Include Namespace="StorageServiceCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/EthernetSwitchCollection.xml">
    <edmx:Include Namespace="EthernetSwitchCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/FabricCollection.xml">
    <edmx:Include Namespace="FabricCollection"/>
  </edmx:Reference>
  <edmx:Reference Uri="/redfish/v1/metadata/TelemetryService.xml">
    <edmx:Include Namespace="TelemetryService"/>
  </edmx:Reference>


  <edmx:DataServices>

    <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="ServiceRoot">

      <EntityType Abstract="true" BaseType="Resource.v1_0_0.Resource" Name="ServiceRoot">
        <Annotation String="This resource represents the root of the Redfish service, located at the \redfish\v1 URI.  As a hypermedia API, all other resources accessible through the Redfish interface on this device are linked directly or indirectly from the Service Root." Term="OData.Description"/>
        <Annotation String="This object represents the root Redfish service.  All values for resources described by this schema shall comply to the requirements as described in the Redfish specification." Term="OData.LongDescription"/>
      </EntityType>
    </Schema>

    <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="ServiceRoot.v1_0_0">

      <EntityContainer Name="ServiceContainer">
        <Singleton Name="Service" Type="ServiceRoot.ServiceRoot"/>
        <Singleton Name="Systems" Type="ComputerSystemCollection.ComputerSystemCollection"/>
        <Singleton Name="Nodes" Type="ComposedNodeCollection.ComposedNodeCollection"/>
        <Singleton Name="Chassis" Type="ChassisCollection.ChassisCollection"/>
        <Singleton Name="Managers" Type="ManagerCollection.ManagerCollection"/>
        <Singleton Name="Tasks" Type="TaskService.TaskService"/>
        <Singleton Name="AccountService" Type="AccountService.AccountService"/>
        <Singleton Name="SessionService" Type="SessionService.SessionService"/>
        <Singleton Name="EventService" Type="EventService.EventService"/>
        <Singleton Name="Registries" Type="MessageRegistryFileCollection.MessageRegistryFileCollection"/>
        <Singleton Name="JsonSchemas" Type="JsonSchemaFileCollection.JsonSchemaFileCollection"/>
        <Singleton Name="Sessions" Type="SessionCollection.SessionCollection"/>
        <Singleton Name="Services" Type="StorageServiceCollection.StorageServiceCollection"/>
      </EntityContainer>

      <EntityType BaseType="ServiceRoot.ServiceRoot" Name="ServiceRoot">
        <Annotation String="This object represents the root Redfish service." Term="OData.Description"/>
        <Annotation String="This object represents the root Redfish service.  All values for resources described by this schema shall comply to the requirements as described in the Redfish specification." Term="OData.LongDescription"/>
        <Property Name="RedfishVersion" Nullable="false" Type="Edm.String">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="The version of the Redfish service" Term="OData.Description"/>
          <Annotation String="The value of this string shall represent the version of the Redfish service.  The format of this string shall be of the format majorversion.minorversion.errata in compliance with Protocol Version section of the Redfish specification. " Term="OData.LongDescription"/>
          <Annotation String="^\d+\.\d+\.\d+$" Term="Validation.Pattern"/>
        </Property>
        <Property Name="UUID" Type="Resource.UUID">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="Unique identifier for a service instance. When SSDP is used, this value should be an exact match of the UUID value returned in a 200OK from an SSDP M-SEARCH request during discovery. " Term="OData.Description"/>
          <Annotation String="The value of this string shall represent the id of the Redfish service instance.  The format of this string shall be a 32-byte value in the form 8-4-4-4-12.  If SSDP is used, this value shall be an exact match of the UUID value returned in a 200OK from an SSDP M-SEARCH request during discovery. RFC4122 describes methods that can be used to create a UUID value. The value should be considered to be opaque. Client software should only treat the overall value as a universally unique identifier and should not interpret any sub-fields within the UUID." Term="OData.LongDescription"/>
        </Property>
        <NavigationProperty Name="Systems" Nullable="false" Type="ComputerSystemCollection.ComputerSystemCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection of Systems." Term="OData.Description"/>
          <Annotation String="This object shall only contain a reference to a collection of resources that comply to the Systems schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="Nodes" Type="ComposedNodeCollection.ComposedNodeCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection of Nodes." Term="OData.Description"/>
          <Annotation String="This object shall only contain a reference to a collection of resources that comply to the Nodes schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="Chassis" Nullable="false" Type="ChassisCollection.ChassisCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection of Chassis." Term="OData.Description"/>
          <Annotation String="This object shall only contain a reference to a collection of resources that comply to the Chassis schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="Managers" Nullable="false" Type="ManagerCollection.ManagerCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection of Managers." Term="OData.Description"/>
          <Annotation String="This object shall only contain a reference to a collection of resources that comply to the Managers schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="Tasks" Nullable="false" Type="TaskService.TaskService">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to the Task Service." Term="OData.Description"/>
          <Annotation String="The classes structure shall only contain a reference to a resource that complies to the TaskService schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="SessionService" Nullable="false" Type="SessionService.SessionService">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to the Sessions Service." Term="OData.Description"/>
          <Annotation String="The classes structure shall only contain a reference to a resource that complies to the SessionService schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="AccountService" Nullable="false" Type="AccountService.AccountService">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to the Account Service." Term="OData.Description"/>
          <Annotation String="The classes structure shall only contain a reference to a resource that complies to the AccountService schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="EventService" Nullable="false" Type="EventService.EventService">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to the EventService." Term="OData.Description"/>
          <Annotation String="The classes structure shall only contain a reference to a resource that complies to the EventService schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="Registries" Nullable="false" Type="MessageRegistryFileCollection.MessageRegistryFileCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection of Registries." Term="OData.Description"/>
          <Annotation String="This object shall contain a reference to Message Registry." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="JsonSchemas" Nullable="false" Type="JsonSchemaFileCollection.JsonSchemaFileCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection of Json-Schema files." Term="OData.Description"/>
          <Annotation String="This object shall only contain a reference to a collection of resources that comply to the SchemaFile schema where the files are Json-Schema files." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="EthernetSwitches" Type="EthernetSwitchCollection.EthernetSwitchCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection Ethernet Switches." Term="OData.Description"/>
          <Annotation String="The classes structure shall only contain a reference to a resource that complies to the Ethernet Switch schema." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <NavigationProperty Name="Services" Type="StorageServiceCollection.StorageServiceCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="This is a link to a collection of Storage Services." Term="OData.Description"/>
          <Annotation String="The classes structure shall only contain a reference to a resource that complies to the StorageService schema" Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
        <Property Name="Links" Nullable="false" Type="ServiceRoot.v1_0_0.Links">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="Contains references to other resources that are related to this resource." Term="OData.Description"/>
          <Annotation String="The Links property, as described by the Redfish Specification, shall contain references to resources that are related to, but not contained by (subordinate to), this resource." Term="OData.LongDescription"/>
          <Annotation Term="Redfish.Required"/>
        </Property>
      </EntityType>

      <ComplexType BaseType="Resource.Links" Name="Links">
        <NavigationProperty Name="Sessions" Nullable="false" Type="SessionCollection.SessionCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="Link to a collection of Sessions" Term="OData.Description"/>
          <Annotation String="This property shall contain the link to a collection of Sessions." Term="OData.LongDescription"/>
          <Annotation Term="OData.AutoExpandReferences"/>
        </NavigationProperty>
      </ComplexType>

    </Schema>
    <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="ServiceRoot.v1_0_2">
      <EntityType BaseType="ServiceRoot.v1_0_0.ServiceRoot" Name="ServiceRoot"/>
    </Schema>
    <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="ServiceRoot.v1_0_3">
      <EntityType BaseType="ServiceRoot.v1_0_2.ServiceRoot" Name="ServiceRoot"/>
    </Schema>
    <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="ServiceRoot.v1_1_0">
      <EntityType BaseType="ServiceRoot.v1_0_2.ServiceRoot" Name="ServiceRoot">
        <NavigationProperty ContainsTarget="true" Name="Fabrics" Nullable="false" Type="FabricCollection.FabricCollection">
          <Annotation EnumMember="OData.Permission/Read" Term="OData.Permissions"/>
          <Annotation String="A link to a collection of all fabric entities." Term="OData.Description"/>
          <Annotation String="The referenced collection shall contain references to all Fabric instances." Term="OData.LongDescription"/>
        </NavigationProperty>
      </EntityType>
    </Schema>

    <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="ServiceRoot.v1_1_1">
      <EntityContainer Extends="ServiceRoot.v1_0_0.ServiceContainer" Name="ServiceContainer">
        <Singleton Name="Fabrics" Type="FabricCollection.FabricCollection"/>
        <Singleton Name="TelemetryService" Type="TelemetryService.TelemetryService"/>
      </EntityContainer>

      <EntityType BaseType="ServiceRoot.v1_1_0.ServiceRoot" Name="ServiceRoot"/>
    </Schema>

  </edmx:DataServices>
</edmx:Edmx>

It seems that script doesn't work correctly for XML tags containing colon, for example: <edmx:reference>, following call: refs = soup.find_all('edmx:reference') doesn't find requested tags.

original script doesn't handle pound sign in URL

Official mockups fail on testing due to Fan and Sensor references using pound signs that aren't handled, not by the server or by the validator. Since HTTP responses do not respond to pound signs in URLS as they are for jumping around html docs, this seems to be an expectation that the server returns something but gives a 404.

I/O Error when logging errors from validateSingleURI function

In RedfishServiceValidator.py, when trying to log the errors in "validateSingleURI" function, we get I/O operation failed due to error handler missing. This is using version pulled 8/2/2017. Need to add rsvLogger.removeHandler(errh) before returns. This causes the program to terminate on the error.

--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.5/logging/__init__.py", line 982, in emit
    stream.write(msg)
ValueError: I/O operation on closed file
Call stack:
  File "RedfishServiceValidator.py", line 811, in <module>
    sys.exit(main(sys.argv))
  File "RedfishServiceValidator.py", line 754, in main
    rsvLogger.error('{} {} errors in {}'.format(innerCounts[countType], countType, results[item][0].split(' ')[0]))
Message: '1 exceptionPropCompliance errors in /redfish/v1/TaskService'
Arguments: ()
Traceback (most recent call last):
  File "RedfishServiceValidator.py", line 811, in <module>
    sys.exit(main(sys.argv))
  File "RedfishServiceValidator.py", line 754, in main
    rsvLogger.error('{} {} errors in {}'.format(innerCounts[countType], countType, results[item][0].split(' ')[0]))
AttributeError: 'NoneType' object has no attribute 'split'

workaround I did:
Add rsvLogger.removeHandler(errh) before each return in validateSingleURI function.

diff --git a/RedfishServiceValidator.py b/RedfishServiceValidator.py
index 7b02c33..df0ec2d 100644
--- a/RedfishServiceValidator.py
+++ b/RedfishServiceValidator.py
@@ -529,12 +529,16 @@ def validateSingleURI(URI, uriName='', expectedType=None, expectedSchema=None, e
             success = False
             results[uriName] = (URI, success, counts, messages,
                                 errorMessages, None, None)
+            # EH
+            rsvLogger.removeHandler(errh)
             return False, counts, results, None, None
     except Exception as e:
         counts['exceptionResource'] += 1
         success = False
         results[uriName] = (URI, success, counts, messages,
                             errorMessages, None, None)
+        # EH
+        rsvLogger.removeHandler(errh)
         return False, counts, results, None, None
     counts['passGet'] += 1
     results[uriName] = (str(URI) + ' ({}s)'.format(propResourceObj.rtime), success, counts, messages, errorMessages, propResourceObj.context, propResourceObj.typeobj.fulltype)

Following odata.id URI in array member within LogEntryCollection.Members

The odata.id within LogEntryCollection.Members array entry is not a #frag but a real URI. Looking at current output I do not see a check of the page referenced by this URI.

Is confusing because the full URI is referenced in the HTML heading.
But the Required Member error I see comes from the LogEntry object included in the array.
The Required Member is missing from the array entry but not the page pointed to by the array entry.

Am I reading the output correctly?

This used to work, I think the changes related to arrays like Temperature which have dummy #frag URI broke this case.

RSV tool does not accumulate type definitions discovered in processed xml files for use in child pages, relies solely on $metatdata supplemented by the page being validated

Based on some tests here it appears that the tool assumes that $metadata page list is the only source of Schemas files to process. However if the basic schemas are reviewed one discovers a number of edmx:Reference at the top of every schema xml files. ComputerSystem_v1.xml for example has 20 Reference definitions each including Uri and Namespace.

My issue = What should the Service-Validator do with these references inside of fetched xml schema?
Can these locally override the Uri of a schema also referenced in $metadata?
Can $metadata leave out schemas that are referenced in one of the files that are referenced by $metadata? Should the schema references add to the schema list?

The spec is not totally clear to me -
Redfish Services shall expose a metadata document describing the service at the "/redfish/v1/$metadata" resource. This metadata document describes the resources available at the root, and references additional metadata documents describing the full set of resource types exposed by the service.

Current version requires all referenced schema be present. Should it be more forgiving?

The service root will reference schema that are not used in a particular implementation. Currently the design of the tool assigns an error to this circumstance.
Example, recent ServiceRoot xml references Swordfish schemas so the program requires access to Swordfish xml locally even in a system that does not supply Swordfish services.

Is this correct?

What is the Redfish rule for local-supplied schema?
A. Must supply xml for every Schema referenced by any xml? Since we start with Service Root, in practical terms, this is every xml schema in the Redfish/Swordfish universe.
or
B. Must supply xml that describes every page that is present and used. Namely, every Schema referenced by odata.type in a real page.

Runtime check for incompatible bs4 version?

During the Plugfest, the beautifulsoup4 version issue hit a few people. It's hard for a user to diagnose this problem.

Is it possible to do an explicit check for the incompatible version and issue a helpful warning?
Would also apply to Reference Checker, Interop Validator and maybe other tools.

Actions not verified for existence

Since an Action is not a property of an EntityType, it is not considered expected by the tool for the JsonPayload, thus treating it as an additional property by default.

Currently, it ignores this issue, but the solution would require gathering Actions that are "IsBound" in the getTypeDetails stage when grabbing the schema.

Getting Unexpected Error on Boot.BootSourceOverrideMode

@odata.type is #ComputerSystem.v1_3_0.ComputerSystem
But getting this error
ERROR - Boot.BootSourceOverrideMode: Appears to be an extra property (check inheritance or casing?)
This is property added in v1.1.0
Above this in log is
INFO - ComputerSystem.v1_0_0.Boot:UefiTargetBootSourceOverride
Is it possible the tool is using Boot from 1.0.0 schema instead of 1.3.0?
This test is done with schema from dmtf.org
ComplianceLog_05_26_2017_102019.zip

Regex validation fails on valid 'RedfishVersion' in ServiceRoot

Note that the tool is using pre-loaded regex values for UUID and date/time values.

The regex validation pattern pulled from CSDL for "RedfishVersion" in ServiceRoot appears to be correct, and validates using online regex checkers to a value of "1.0.0". But the tool marks this value as a malformed string failure (regex mismatch). Tested on HPE ProLiant Gen9, multiple Dell implementations return identical or similar correct string values ("1.0.0", "1.0.2").

This may be caused by regex string conversion upon reading the CSDL - similar to an issued opened in SPMF regarding the conversion of CSDL strings to JSON schema.

github summary incorrect

The github summary line is incorrect. Tool has been updated to Python 3

Current = The Redfish Service Validator is a python2.7 tool for checking conformance of any "device" with a Redfish service interface against Redfish CSDL schema

Program Fails with latest version of beautifulsoup4

Setting up a new computer for a new person.

  1. Downloaded and installed Python 3.6.1 (been using Python 3.5)
  2. did pip3 install -r requirements.txt with no errors
    The system under test does not have valid trusted certificate
    $metadata page looks good using Chrome

Can you tell - Is this Some problem with Python?

---Jeff

`G:_Redfish_git\Redfish-Service-Validator-public\Redfish-Service-Validator>python RedfishServiceValidator.py
System Info: S2600WFO
RedfishServiceValidator Config details: ('https://192.168.100.209', 'user:root', 'metadata', 'no CheckCert', 'Attempt for Online Schema')
Start time: 06/12/17 - 15:22:01

*** ServiceRoot, /redfish/v1
C:\Python36\lib\site-packages\urllib3\connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
C:\Python36\lib\site-packages\urllib3\connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
SchemaURI missing reference link: /redfish/v1/$metadata ServiceRoot.ServiceRoot
SchemaURI unsuccessful: /redfish/v1/$metadata
File not found metadata/ServiceRoot_v1.xml:
Traceback (most recent call last):
File "RedfishServiceValidator.py", line 740, in
sys.exit(main(sys.argv))
File "RedfishServiceValidator.py", line 632, in main
success, counts, results, xlinks = validateURITree('/redfish/v1', 'ServiceRoot')
File "RedfishServiceValidator.py", line 568, in validateURITree
validateSingleURI(URI, uriName, expectedType, expectedSchema, expectedJson, parent)
File "RedfishServiceValidator.py", line 483, in validateSingleURI
propResourceObj = rst.ResourceObj(uriName, URI, expectedType, expectedSchema, expectedJson, parent)
File "G:_Redfish_git\Redfish-Service-Validator-public\Redfish-Service-Validator\traverseService.py", line 297, in init
SchemaFullType, SchemaType)
NameError: name 'SchemaFullType' is not defined`

Why does tool fetch URI with embedded #

I see the tool fetch URI with embedded #
DEBUG - callingResourceURI: /redfish/v1/Chassis/H88493-003-1638MSL/Thermal%23/Temperatures/0

Seems not required because this fetch will return /redfish/v1/Chassis/H88493-003-1638MSL/Thermal page which is already in memory?

Tool is incompatible with beautifulsoup4 version 4.6.0

An environment with a version of this library above 4.5.3 seems to break a piece of the code for a reason currently undiagnosed in getReferenceDetails. It should either be fixed or written in the documentation/requirements of which version is needed.

Json @ Annotations not considered for verification.

Determine whether or not Schema validation is required for certain circumstances of annotations. There is issues for verifying @OData items automatically through their schema file, but @RedFish and other extra annotations require grabbing their Schema file and verifying its typing.

Offering Session Authentification might yield faster execution for users

Reading the source code it seems the tool uses requests.get with auth parameter which is HTTP Basic authentication. This is certainly compliant with the spec.

But if the tool could be upgraded to optionally use session authentication I would predict tool would complete the tests more quickly. And test this important Redfish authentication feature in the process.

I suggest the basic/session method be selectable in config.ini.

I see the one-time investment of programmer time potentially yielding significant time saved by users waiting for the tests to complete.

We should discuss. Reply with opinions.

Frags '#' are to be ignored by service, must be client handled

In all situations with '#', it is expected that they be ignored by the service and thus not sent towards the server, and instead by the client. It is undetermined if this as an Odata compliance expectation, seems unlikely.

To close this issue, remove all expectations above. It would be wise to also implement the client side parsing of pound signs.

No-Internet Mode - SPMF confirms ServiceMode not behaving as expected

Hi Tomas,

Long discussion of Redfish use in no-Internet shop during Tuesday meeting. Some notes here - https://github.com/DMTF/spmf/issues/2118 . This issue is not closed but I am establishing this issue under validator as a discussion spot for no-Internet Validator test.

My take-away - SPMF consensus is Validator's ServiceMode implementation is not currently enforced correctly. When ServiceMode is True, and the Validator encounters a Uri that is off-system, this Uri should be ignored. Not an error but not used. They want the intelligent client to depend upon $metadata Uri only.

Current Insyde test system has modified $metadata with no off-system references. All of the schema in the $metadata are available local. But the Validator goes crazy in no-Internet test with multiple exceptions because there are multiple Uri="http://redfish.dmtf.org/schemas/v1/..." references inside the local schemas.

SPMF does not want to require implementers to edit every schema file. If it is possible, they want the client to be smart. And when client is configured for ServiceMode it should find everything local.

At this time I am not sure how they will recommend client get the 3 odata schema locally.
http://docs.oasis-open.org/odata/odata/v4.0/errata03/csd01/complete/vocabularies/Org.*****
But it seems most likely that $metadata will have include entry to deliver local Uri to client.

If you have comments and concerns Mike and I can take these back to larger group.

Tool Terminates unexpectedly after getting error message back from Service under test

This happens with ServiceMode True or False.
This implantation does not locally cache Base Message File.
Here is the end of the log. There is no exception or other indication why tool exited.
HTML file is not created.

INFO - #StorageCollection.StorageCollection, Counter({'passGet': 1, 'skipOptional': 1, 'failProp': 1, 'err.Collection(Storage.Storage)': 1, 'skipOem': 1, 'pass': 1})
DEBUG - OrderedDict([('Members.Storage#0', ('/redfish/v1/Systems/BQWT64900010/Storage/1', False, 'Storage.Storage', '/redfish/v1/schema/Storage_v1.xml', OrderedDict([('@odata.id', '/redfish/v1/Systems/BQWT64900010/Storage/1')])))])
INFO -
*** ServiceRoot -> Systems.Systems -> Members.ComputerSystem#0 -> Storage.Storage -> Members.Storage#0, /redfish/v1/Systems/BQWT64900010/Storage/1
DEBUG -
*** None, False, False
DEBUG - True, OrderedDict([('@odata.type', '/redfish/v1/$metadata#Message.1.0.0.Message'), ('MessageId', 'Base.1.0.ResourceInStandby'), ('Message', 'The request could not be performed because the resource is in standby.'), ('Severity', 'Critical'), ('Resolution', 'Ensure that the resource is in the correct power state and resubmit the request.')]), 200

[feature] Travis Ci friendly

Ensure programmatic testing for Travis Ci.

There's excessive text output and an HTML log that inconvenient for testing, trim the fat and test it personally on TravisCi.

Many "Not Found in Type Inheritance" Errors Reported

After update to latest pull, there are errors flagged that I do not understand.
This test has $metadata that pulls all schema from DMTF. (LocalOnlyMode = False)
Earlier version reports different errors.
metadata.zip
logs.zip
zip file created with 7zip

First Error (also note typo in message - inheritance)

ServiceRoot.v1_0_0.ServiceRoot:SessionService: Expected Entity type ('SessionService.SessionService', None), but not found in type inhertiance ['SessionService.v1_1_1.SessionService']
ServiceRoot.v1_0_0.ServiceRoot:AccountService: Expected Entity type ('AccountService.AccountService', None), but not found in type inhertiance ['AccountService.v1_1_0.AccountService']

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.