I agree that we should not only define DTOs, but also APIs. One thing that felt quite good was the use of utoipa, which generated OpenAPI specs on the fly.

from trustify.

+1 on utoipa. We (specifically I) need to get better at not just copy/pasting, and laying in enough Schema stuff to make them robust.

from trustify.

Hi Guys, as the definition of the DTOs are related to the definition of the rest endpoints I started drafting some ideas around our DTOs and APIs. These are ideas and not a mandate. I've started drafting the UI already and I started in alphabetical order so ADVISORIES are the first abstraction to tackle.

We need to organize ourselves in term of where we define the rest apis since this is the contract between UI and backend. So to me is very important to move things forward in the UI and the api contract is a key component.

• Should we create an openapi.json file somewhere and start from there? where can we edit the same file all together?
• Should you guys define the rest apis on your own?
• Should I define the rest apis on my own?
• Any ideas around this so we can be fast at what we are doing?

We need to tackle this piece of work as fast as we can, what would be the fastest way for you guys? @bobmcwhirter @jcrossley3 @ctron @dejanb @mrizzi ?

Advisories

Endpoints

Search advisories

• GET /advisories?limit=10&offset=0&sort=desc:field. Search advisories that match query criteria.
  • Expected response: body = an array of Advisory. A header x-total to represent the total number of elements of the search
  • limit: the max number of elements the body should contain
  • offset: where the index for extracting data should start
  • sort: defined as direction:field_name. E.g. sort=asc:severity, sort=desc:severity. Any other way to describe sorting is also fine.
  • In regards to filters. We can use a single QueryParameter to express the whole filter. E.g. q=.... However, let's not invent a new query language here. Let's use common and long practiced patterns: ${field_name}${operator}${field_value} that's it, no more complexity added. E.g. filter1=xyz&filter2=abc&filter3=foo (single equality). In cases where a single filter need to express multiple conditions then those multiple conditions must be expressed within that particular filter. E.g. filter1=X|Y|Z (OR conditions), filter1=X,Y,Z (AND conditions), filter1>=99 (comparison), filter1=~my_text (like searches).
    • queryParameter=${field_name}${operator}${field_value}
    • operator="=" | "!=" | "~" | ">" | ">=" | "<" | "<="

The query system should be discussed I think. But as long as we keep things simple, easy to read, and easy to construct, and mainly should serve our specific use cases, then everyone will be happy.

AdvisoryDTO {
  id: string;
  severity: "low" | "moderate" | "important" | "critical";
  revision_date: string;
  vulnerabilities: {
     low: number,
     moderate: number,
     important: number,
     critical: number,
 };
 metadata: {
    title: string;
    publisher: {
      name: string;
      namespace: string;
    };
    tracking: {
      status: string;
    };
    related_products: {};
  };
}

Get single advisory by ID

• GET /advisories/{id}
  Should return an Advisory object

Get Advisory's vulnerabilities

• GET /advisories/{id}/vulnerabilities
  • I do not expect to have thousands of vulnerabilities per single advisory so NO query system needed. This endpoint should return ALL items (vulnerabilities).

AdvisoryVulnerabilityDTO {
  id: string;
  title: string;
  discovery_date: string;
  release_date: string;
  revision_date: string;
  score: number;
  cwe: number;
}

Get Advisory source file

• GET /advisories/{id}/source

from trustify.

However, let's not invent a new query language here.

I've the impression that what follows after that sentence does exactly that. Why don't we re-use what we have today? Why not focus on the pain points that we identified, which is not the way how we express a query, but the implementation side of this? I really have the feeling we completely lost track of what's important.

from trustify.

However, let's not invent a new query language here.

I've the impression that what follows after that sentence does exactly that. Why don't we re-use what we have today? Why not focus on the pain points that we identified, which is not the way how we express a query, but the implementation side of this? I really have the feeling we completely lost track of what's important.

• That's why I mentioned in my comment These are ideas and not a mandate
• I also mentioned Should you guys define the rest apis on your own? in regards of defining the contract. If you guys want to do it 100% on your on then that's fine. I just need confirmation and a date when you will hand over to me the openapi.json file
• The big design problem we have at the moment is that the advisories and sbom endpoints are returning raw data to the UI (zero use of DTOs and zero abstraction in the backend), I would rather fetch the data directly from s3 and it would not make a difference. I strongly believe this is a serious and bad thing. That's why I'm suggesting to define DTOs, unless we are happy to continue exposing advisory|sbom raw json files through our rest apis
• About the query system. I wrote The query system should be discussed I think. I'm not imposing but trying to open a discussion. If we don't want to have a discussion and keep things as they are, then fine we just need to say it. in my opinion, technically the current query system is bad, a query system designed for use case A but applied to use case B (designed for Github unique filtersearch input but applied to Red Hat multiple filters system), adds unnecessary complexity. An app is not just the UI but the APIs that will be exposed to whoever wants to consume them. But happy to keep things as they are if we decide so. I'm not imposing things here.

We need agreements about the definition of the endpoints guys ASAP. I'm just trying to figure out what will work the best, making a balance between "correct way of doing things" and "speed we need to finish this".

So who will create the openapi,yml file?

from trustify.

An idea came to my mind: would it be fair to say "let's keep 100%" the same endpoints and the same/exact body/headers (same expected responses) compared to the Trustification current set of of endpoints? I mean, no changes (literally zero changes) to the API definitions. Would that work for everyone? I'm getting pessimistic about reaching agreements, so keeping things as they are would be a good step1?. Then we could see when step2 (whatever it means) might come?

from trustify.

However, let's not invent a new query language here.

I've the impression that what follows after that sentence does exactly that. Why don't we re-use what we have today? Why not focus on the pain points that we identified, which is not the way how we express a query, but the implementation side of this? I really have the feeling we completely lost track of what's important.

Just to have my $0.02 on the record...

I'm not opposed to simplifying our query language. I like github's syntax, and I like what Carlos has proposed. What we currently have is neither, mostly due to the in:... qualifier, I think. Without some UI widgets to construct queries for me, I'd be lost. And the text-loving, server-side developer in me kinda hates depending on those widgets. 😜

I concede it's a lower priority than other pain points, but I'd like to keep it on the table.

from trustify.

Closing this issue as I believe we have more granular issues to tackle specific Models and rest endpoints

from trustify.