The current implementation of build_json requires that no exceptions be thrown while the response is being built. Until now, the only resolution to this I could think of was building to an intermediary IO and copying to context.response after it's built. That would be a big performance hit in a crucial spot. However, talking another look at my implementation I was thinking that there is one alternative, although it is less than ideal....suggestions are welcome, but I'd recommend we discuss this in a separate PR, so that I can also begin work on other routes while we work out the details.

def build_json(response : IO)
  error : Exception? = nil
  JSON.build response do |json|
    json.object do
      json.field "data" do
        json.object do
          yield json
        rescue e
          error = e
        end
      end
      if error
        json.field "errors" do
          ErrorResponse.new(error).errors.to_json json
        end
      end
    end
  end
end

I realize this would require specifying a partial succesful response, but there's no way to rewind an HTTP::Server::Response because it's streamed out to the client on the fly.

Originally posted by @dscottboggs in #10

For the package routes, I (think) I successfully avoided cases which raise, so that's an option as well, but it's...fragile, and I don't like it, especially not long-term

Since the beginning of the project, I have been using the convention of returning an JSON object which has at its top-level some meaningful keys which our client can look for (I.E. {"errors": ..., "data": ...}). The "errors" key is always present -- in the case of no errors it is an empty list. The reason for it being a list is so that we can express to the user (or the author of a client library) a more meaningful error, by allowing the server to note a list of errors rather than a single one.

However, the built-in error handling solutions in Crystal (raising exceptions) and in Kemal (exceptions, halt) are ideally designed to stop the execution of the context upon an error, without going on. My initial idea was to store a list of errors on HTTP::Server::Context, but after looking through the overrides Kemal offers on Context, I thought of this solution.

Now that I think about it though, neither solution works at all. The result would look something like...

{
  "data": { "some": "data" }
}
{
  "errors": ["an error", "Not Found"]
}

... which isn't even valid JSON.

Maybe if we check if context.response is empty (hasn't yet been written to) before writing to it in the error handler. Idk, that still doesn't solve the issue of what to do inside Actions.throw_error and the deny_access! macro.

Validations

This issue is here to note the necessary steps we need to take to validate any and all received data from the API before passing it through to the library. A high-level discussion of implementation may be appropriate, but details should be worked out in a PR that will later be linked to this issue. This is an important security step (especially since many parameters translate directly to file-paths!) and we should be careful to be well-organized and methodical in our implementation of this feature.

Types of validations

• Reject parameters containing characters that aren't relevant to the value expected to be received.
• Reject parameters that do not fit within an expected range of values. For example, when building a package, we could check that the package id is one of the known source names.
• Size of received parameters
• Path traversal issues (potentially covered by the first validation, depending on the parameter and how it is used)

Parameters that need validation

Path parameters

TODO

Query Parameters

TODO

Feel free to edit this summary as needed to update to-do lists, documentation, etc.

To allow editing user permissions from the web UI or other end-user application.

• POST /user
  JSON body: user's name and group membership.
  Accepted Keys: [ "name", "groups" ]
  Returns: API key.
• PUT /users
  JSON body: selectors and modifiers for users.
  Accepted Keys: [ "match_name", "match_groups", "api_key", "new_name", "add_groups", "remove_groups" ]
• PUT /user edit the current user
  JSON Body: modifiers for the user
  Accepted Keys: [ "new_name", "add_groups", "remove_groups" ]
• DELETE /users
  JSON body: of selectors for users
  Accepted Keys: [ "match_name", "match_groups", "api_key" ]
• GET /user
  Returns: information about the currently signed in user
• GET /users
  JSON body: selectors
  Returns: JSON-formatted list of user information
• PUT /user/rekey
  Returns: A new API key for the currently signed-in user.
• PUT /users/rekey
  JSON body: selectors
  Returns: the new user's API key.
  Error: if more than one user was selected.
• POST /groups
  JSON body: the group
• PUT /groups/:id/route
  JSON body: a new route to add to the group
• PUT /groups/:id/param
  JSON body: a mapping of a path to a list of globs to add to those available on that path
• DELETE /groups/:id/param
  JSON body: a group mapping a path to a list of globs to remove
• DELETE /groups/:id
  Deletes the group

• running crystal spec hangs indefinitely. Ideally, it should work out of the box.
• a hash in spec/permissions.json changes, this is a bit annoying. It may be linked to the upper command, so adding to .gitignore isn't a proper solution.

Inspired by HashiCorp Vault policies.

This allows to finely manage API route access, using patterns and permission composition.
The default permission is to deny the access to the resource.

This implementation isn't optimized, we can cache the roles in to the user struct in the future.

In the future we can even allow/disallow parameters like ?host= or ?port=

require "json"

struct Document
  include JSON::Serializable
  getter groups : Array(Group)
  getter users : Array(User)

  struct User
    include JSON::Serializable
    getter name : String,
      id : String,
      group_ids : Set(String)
  end

  struct Group
    include JSON::Serializable
    getter name : String,
      id : String,
      description : String,
      path_policies : Hash(String, PathPolicy)

    def path_permissions(path : String, &block : Set(PathPolicy::Permission) ->)
      path_policies.each do |path_key, policy|
        if File.match? path_key, path
          yield policy.permissions
        end
      end
    end

    struct PathPolicy
      include JSON::Serializable
      getter permissions : Set(Permission)

      enum Permission
        Create
        Read
        Update
        Delete

        def self.new(permission : String)
          {% begin %}
          case permission
            {% for const in @type.constants %}
          when {{const.downcase}} then {{const}}
            {% end %}
          else raise "invalid permission: #{permission}"
          end
          {% end %}
        end

        def to_json(json : JSON::Builder)
          {% begin %}
          case self
            {% for const in @type.constants %}
          when {{const}} then json.string "{{const.downcase}}"
            {% end %}
          else raise "unkown permission: #{value}"
          end
          {% end %}
        end
      end
    end
  end

  def path_permissions(user : String, path : String) : Set(Group::PathPolicy::Permission)?
    puts path
    permissions = nil

    if user = @users.find &.name.== "Tom"
      @groups.each do |group|
        if user.group_ids.includes? group.id
          group.path_permissions path do |group_permissions|
            if permissions
              permissions = permissions & group_permissions
            else
              permissions = group_permissions
            end
          end
        end
      end
    end
    if !permissions || permissions.try &.empty?
      puts "access to `#{path}` denied to #{user}"
      return
    end
    puts permissions
  end
end

json = <<-JSON
{ "users" : [
    {
       "name": "Tom",
       "id": "092",
       "group_ids": ["123"]
    }
  ],
  "groups": [
     {
      "name": "User",
      "id": "123",
      "description": "Default user group",
      "path_policies": {
        "/**": {
          "permissions": ["create", "read", "update", "delete"]
        },
        "/app/**": {
          "permissions": ["read", "update"]
        },
        "/app/*-private": {
          "permissions": []
        },
        "/app/dppm": {
          "permissions": []
        }
      }
    }
  ]
}
JSON

doc = Document.from_json json

doc.path_permissions "Tom", "/app"
doc.path_permissions "Tom", "/app/dppm"
doc.path_permissions "Tom", "/app/new-app"
doc.path_permissions "Tom", "/app/new-private"

Output:

/app
Set{Create, Read, Update, Delete}
/app/dppm
access to `/app/dppm` denied to Document::User(@name="Tom", @id="092", @group_ids=Set{"123"})
/app/new-app
Set{Read, Update}
/app/new-private
access to `/app/new-private` denied to Document::User(@name="Tom", @id="092", @group_ids=Set{"123"})

If you wouldn't mind, I could use a hand setting up the config file/CLI/internal config structure to match up with the general style of DPPM and integrate with it. See my update to cli.cr in a402837d.