Git Product home page Git Product logo

ocm-api-model's Introduction

OpenShift cluster manager API model

Introduction

This project contains the specification of the OpenShift cluster manager API, also known as the model.

Concepts

The specification of the API is written using a DSL (the model language) similar to Go.

The specification is split into multiple model files with extension .model inside the model directory of this project. The location of the model files within sub-directories is important as it indicates which service and version a type or resource belongs to. For example, the clusters_mgmt/v1 sub-directory corresponds to version 1 of the clusters management service.

Data types are represented by class, struct and enum blocks.

For example, the Cluster type defining the cluster concept should be located in a file within the clusters_mgmt/v1 directory, should be named cluster_type.model and should contain something like this:

// Definition of an _OpenShift_ cluster.
class Cluster {
	// Name of the cluster.
	Name String

	// Link to the _flavour_ that was used to create the cluster.
	link Flavour Flavour

	...
}

Classes and structs contain attributes defined by an attribute name followed by the type of that attribute. In the above example there are two attributes defined: Name of type string and Flavour of type Flavour.

In the model language attributes are written using CamelCase because that is what is typically used in Go, which is the language that the model language tries to be close to. But the actual JSON representation uses snake_case. These are some examples of what is the correspondence between attribute names in the model and in JSON:

Model JSON
ID id
HREF href
Cluster cluster
AccessKeyID access_key_id

The main difference between classes and structs is that classes are intended to represent objects that have an identity. In practice that means that classes have built-in ID and HREF attributes. The ID attribute will contain the unique identifier of the object, and the HREF attribute will contain the location of the object in the server. For example, when retrieving a particular cluster from the server the returned JSON document will be like this:

{
    "id": "123",
    "href": "/api/clusters_mgmt/v1/clusters/123",
    ...
}

Resources are represented by resource blocks.

For example, the Clusters resource defining the collection of clusters should be located in a file within the clusters_mgmt/v1 directory, should be named clusters_resource.model and should contain something like this:

// Manages the collection of clusters.
resource Clusters {
	// Retrieves the list of clusters.
	method List {
		...
	}

	// Provision a new cluster and add it to the collection of clusters.
	method Add {
		...
	}

	...
}

Resource methods are represented as nested method blocks.

For example, the method that retrieves the list of clusters is defined in a nested List method block inside the resource block for the clusters resource:

// Retrieves the list of clusters.
method List {
	// Index of the requested page, where one corresponds to the first page.
	in out Page Integer

	// Maximum number of items that will be contained in the returned page.
	in out Size Integer

	// Search criteria.
	in Search String

	// Order criteria.
	in Order String

	// Total number of items of the collection that match the search criteria,
	// regardless of the size of the page.
	out Total Integer

	// Retrieved list of clusters.
	out Items []Cluster
}

Methods have parameters defined by their direction (in or out), their name and their type. In the above example there are four input parameters (named Page, Size, Search and Order) and four output parameter (named Page, Size, Total and Items).

  • "List" conceptual method is implemented by HTTP GET method, with in parameters represented in the URL as HTTP query parameters (with names converted from CamelCase to snake_case).

    cluster_mgmt API supports an alternative way to List — as HTTP POST method, with a method=get query parameter and in paramaters sent as fields of a JSON request body (again, with snake_case names).

    Out parameters become top-level fields in the JSON response body (again, with snake_case names). An additional kind field is added automatically (set to the type's name + a "List" suffix), it should not be included in the method block.

  • "Get" is regular HTTP GET method, declared with single out parameter representing the JSON response body.

  • "Add" is performed by HTTP POST method, declared with single in out parameter representing the JSON request body — as well as the response body.

  • "Delete" is performed by HTTP DELETE method, with no parameters.

In addition to methods resources also have locators, defined by nested locator blocks. Locators represent the relationships between resources. For example, the resource that manages the collection of clusters knows how to locate the resource that manages a specific cluster. That is represented in the model language with a locator block like this:

// Returns a reference to the service that manages an specific cluster.
locator Cluster {
	target Cluster
	variable ID
}

All resource locators have a name and a target. The target is defined using the target keyword and the name of the resource.

There are two kinds of resource locators: with and without variable.

Locators with variable are intended for collections, where location a sub-resource resource requires specifying the identifier of that object that is managed by that sub-resource. For example, to locate the sub-resource that manages a specific cluster it is necessary to provide the identifier of that cluster. That identifier is the variable. These kind of locators are defined using the variable keyword and the name of the variable, like in the previous example.

Locators without variable are intended for cases where no additional information is needed to identify the sub-resource. For example, the locator for the credentials sub-resource of a cluster can be defined like this:

// Reference to the resource that manages the credentials of the cluster.
locator Credentials {
	target Credentials
}

Locators also define the URL structure of the API: the path component of the URL of a particular resource is constructed concatenating the names/variables of the locators that are in the chain of locators from the root to that resource. For example, to get to the Credentials resource of cluster with identifier 123 the chain starts at the root resource of the clusters management service, it continues with the Clusters locator to find the clusters collection, then the Cluster locator with variable 123 to get the cluster resource and finally the Credentials locator to get to the credentials resource:

Root -> Clusters -> Cluster(123) -> Credentials

Each link in that chain of locators is translated into an URL path segment using the following rules:

  • The root resource corresponds to the root of the service/version. For example, for version 1 of the clusters management service that would be /api/clusters_mgmt/v1.

  • Locators without variables correspond to URL segments named like the locator, but using snake_case instead of CamelCase. For example, for the first link in the above example the URL path segment would be clusters.

  • Locators with variables correspond to URL segments that contain the value of the variable. For example, for the second link in the above example the URL path segment would be 123.

Taking these rules into account the complete URL path for the above example would be the following:

/api/clusters_mgmt/v1/clusters/123/credentials

Documentation

The Go language supports adding documentation in the code itself, using the documentation comments. These comments start with // and appear immediately before the documented item. The model language uses the same kind of documentation comments. For example, the Cluster type can be documented like this:

// Definition of an _OpenShift_ cluster.
//
// The `cloud_provider` attribute is a reference to the cloud provider. When a
// cluster is retrieved it will be a link to the cloud provider, containing only
// the kind, id and href attributes:
//
// ```json
// {
//   "cloud_provider": {
//     "kind": "CloudProviderLink",
//     "id": "123",
//     "href": "/api/clusters_mgmt/v1/cloud_providers/123"
//   }
// }
// ```
//
// When a cluster is created this is optional, and if used it should contain the
// identifier of the cloud provider to use:
//
// ```
// {
//   "cloud_provider": {
//     "id": "123",
//   }
// }
// ```
//
// If not included, then the cluster will be created using the default cloud
// provider, which is currently Amazon Web Services.
//
// The region attribute is mandatory when a cluster is created.
//
// The `aws.access_key_id`, `aws.secret_access_key` and `dns.base_domain`
// attributes are mandatory when creation a cluster with your own Amazon Web
// Services account.
class Cluster {
	...
}

Unlike Go the format of this documentation isn't plain text, but Markdown.

Attributes of types, methods of resources and parameters of methods can all be documented in a similar way, just placing documentation comment before the definition of the item. For example, to document the Search parameter of the List method of the Clusters resource the following documentation comment could be used:

// Search criteria.
//
// The syntax of this parameter is similar to the syntax of the _where_ clause of a
// SQL statement, but using the names of the attributes of the cluster instead of
// the names of the columns of a table. For example, in order to retrieve all the
// clusters with a name starting with `my` in the `us-east-1` region the value
// should be:
//
// ```sql
// name like 'my%' and region.id = 'us-east-1'
// ```
//
// If the parameter isn't provided, or if the value is empty, then all the
// clusters that the user has permission to see will be returned.
in Search String

This documentation is used to automatically generate OpenAPI reference documentation.

ocm-api-model's People

Contributors

andreadecorte avatar cben avatar ciaranroche avatar cristianoveiga avatar davidleerh avatar djberg96 avatar etabak avatar gdbranco avatar gshilin-sdb avatar igoihman avatar jhernand avatar lucasponce avatar machi1990 avatar marcolan018 avatar miguelsorianod avatar nimrodshn avatar oriadler avatar osherdp avatar pvasant avatar rawsyntax avatar renan-campos avatar samira-barouti avatar tbrisker avatar tirthct avatar tiwillia avatar tylercreller avatar tzvatot avatar vkareh avatar zgalor avatar zvikorn avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ocm-api-model's Issues

metadata for never/optional/always fields on POST/PATCH/GET

background

Historically, both in openapi and in backend Go code, CS & AMS used same types to represent:

  1. Fields you must send vs. can omit in POST body when creating resource.
  2. Fields you'll get back in POST response. Normally same as:
  3. Fields you will always GET vs. optional. This depends on lifecycle stage and also on cluster type in complicated ways... (the best case is omitting whole sub-structs e.g. "aws": {...})
    I think for practical client-development purposes, we mostly
  4. Fields you can PATCHall are optional here (and Go type marks all pointers to detect omission vs. zero value after de-serialization); some are read-only and can't be patched (but Go type includes them so they can be de-serialized so validator can return error if you sent them).
  5. "kind": "FooLink" fields are not present at all but only link to other resource with kind, id, and href. In CS at least, these still reuse the api.Foo Go structs, forcing all but these 3 fields to be optional pointers.
    (This was partly motivated by making it easier for Go to sometimes inline related resources in same place vs. linking them. Backends did add a few ?fetchFoo=true params like that, but they are small minority.)

☹️ Problem: The combination of all these, is that practically nothing is required in openapi

And for consumers of the API, nearly all fields being optional conveys zero information.

This is a hard problem to attack!
I don't know if it's realistic to move the needle on actual type level, but it's systematically lacking documentation UI would be glad to get even in the form of comments, for a start 🙏

Proposed first step:

At least in metamodel where we make up the rules, we can add formalized notations for all this.

Defining unrelated models e.g. PostCluster vs. PatchCluster here would be a mistake IMHO. It's valuable to know it's same resource with same fields, they just behave differently under different verbs.

  • 1. POST body: in resource definitions, we have in option. But I don't think this says anything about POST vs. PATCH bodies, and it doesn't exist for class files... I think POST needs 3 distinct behaviors:

    • @Add(required) — it's an error if you omit this field.
    • @Add(optional) — default if not specified? probably doesn't need syntax.
    • @Add(never) — it's an error if you send this field (e.g. metrics).

    Do we need support for separate APIs taking different subsets of same type? iirc AMS has 2 endpoints for creating cluster vs. register offline cluster, but that's rare. Let's ignore for now but keep in mind future possibility of more "verbs".

  • 2. can ignore, assume POST response == GET response.

  • 3. GET: in resource definitions, we have out option. But I think GET needs 3 distinct behaviors.

    • @Get(required) — it's an error if you omit this.
    • @Get(optional) — default if not specified? probably doesn't need syntax.
    • @Get(never) — it's an error if you send it (e.g. metrics).

    Is there value in separating @Get vs @List annotations? Not initially.

  • 4. PATCH: nothing is required, so I think needs 2 behaviors:

    • @Patch(optional)
    • @Patch(never) — it's an error if you send it (e.g. metrics).

    I'm not sure what should be default. Frequently you can PATCH a field iff you can POST it, but sometimes it's only day-1 or only day-2 (and tends to get relaxed over time).

  • 5. We already have link notation e.g. link Region CloudRegion.

The syntaxes proposed above are "strawman" suggestions. I'm not even sure they should use annotation @ style.

Future possibilities given such metadata

out of scope here, just mentioning to show this could be used in various ways...

  • openapi: append this data to description

  • openapi: emit custom x-... fields

    • UI typescript: generate separate types from each openapi type, per these custom fields?
  • openapi: emit separate types for different scenarios?? [disruptive to existing consumers, including AMS code generation]

  • backend: auto-generate validations of required/never fields under various verbs.
    Either as code, or as custom Go struct tags that can be introspected at runtime.

openapi: listing collections don't document returning a `kind` field

I notice none of the method List { blocks contain an out Kind String parameter.
And indeed the generated openapi documents kind for individual objects corresponding to method Get eg. /api/clusters_mgmt/v1/clusters/{cluster_id} but not for collection lists e.g. /api/clusters_mgmt/v1/clusters

@nimrodshn @jhernand I can send a PR to add those, will that be enough?

I see generated server code has special logic to return correct Kind, including link vs full object.
Will it automatically also return kind for lists?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.