https://pypi.org/project/radon/

want to add this to the CI tooling as an additional check

• By leaving the server evaluation logic intact, we have an escape hatch if needed
• Would need to add the evaluation logic itself to the CLI as well as the logic for creating evaluation objects and sending them to the server
• CLI needs to retrieve any missing objects from the server
• This includes the ability to dry-run all resource types

This will be complete when:

• we can dry-evaluate all resources on command
• edit multiple resource files locally, run the evaluate on the local copy
• we can run an evaluate without sending the evaluate object to the server using the -dry flag

Test cases:

• all resources exist locally
• none resources exist locally
• resources exist in multiple repos

Need to go through and rewrite/update the following things once issue #82 and #81 and #80 and #79 are complete:

• Deployment Guide
• Tutorial Guide
• Landing Readme
• Getting started documentation

Not all of these might need changes but at least a few of them will for sure

Need to clean up and simplify the Readme and point it to the proper docs

also need to update the docs to have a section about how to create/submit PRs

also should add a pronunciation guide for Fides :)

add a section in the docs about how we plan work

seems related to something in the caching/persistence, I know we ran into this before, maybe its related to the recent caching changes? i vaguely remember there being some

Currently, the FidesAPI will automatically sanitize certain inputs (such as the fidesKey and dataset field names) without surfacing that change to the user

This led to some weird behaviour during testing with fidesctl and after discussing with @stevenbenjamin , the solution is to throw errors instead of silently sanitizing inputs. otherwise, a manifest file can show a different name/fidesKey than what is actually being used by the API. This will also make it more clear to the user what is and isn't acceptable for different field types

This module will sit within the fidesctl folder (next to cli and core as a submodule) and contain the following logic:

1. parsing the resource files
2. validating the resource files (their types, constraints, etc.)
3. the pydantic models for each resource
4. update how configuration is handled (use pydantic basesettings, allow a default dir to be configured)

This issue will be complete when:

• we can pip install fidesctl separately
• import to another project (solon)

need to get these things thoroughly tested as they're major features that shouldn't be broken without the developer knowing

Now that the concept of a config exists in fidesctl, it needs to be leveraged more efficiently.

The server parameter should be converted to a config value and the config itself should probably be converted to a more concrete object instead of just a dictionary? there are definitely ways to get the code to interact with the config in a more elegant manner

this can be a pydantic model described in config.py to make it easier to extend, reuse loading methods, etc.

Need to rewrite the Scala FidesAPI in Python via FastAPI

For storing e.g. login and credentials. As a part of this it would be nice to have a "fidesctl login" command which stores these, similar to what applications like AWS cli do.

We've worked on many revisions of the language and structure for the "default" taxonomies; the current working versions are all here: https://docs.google.com/spreadsheets/d/1uP-McSq8cGXwpheHk55ZfLpIpizpj7XS8CnaD73wErY/edit. We'd like to get these checked into the repo now (as well as in the separate https://github.com/ethyca/Fides-Privacy-Taxonomy/ repo) so we can continue working with some external parties to both visualize and review the contents.

Within the linked spreadsheet, the first three tabs are marked:

1. Fides Data Uses
2. Fides Data Subjects
3. Fides Data Qualifiers

The sheet itself is a WIP but is reasonably stable now so it's ready to get pulled into the repo and made part of fideslang and similar. A couple implementation notes on the "Data Use" tab though:

• When converting Data Uses, use the "Proposed" column for the new name (and generate a new fides key to match)
• Ensure that the Data Uses taxonomy is preserved; in the sheet, the parent is tracked using the prior "name" column, not the new "Proposed" name, so just be careful
• Leave all the descriptions for the Data Uses blank as these need further editing & review before use

Tutorial accessibility

• Tutorial is still quite unapproachable; I really don't think it should start with a policy object (do that last). I'd recommend it gets cut into at least three steps to (1) define & apply your first system, (2) define & apply your first dataset, and (3) define & apply your first policy
• Tutorial might be a lot easier to follow and document if there was a /demo folder that shipped with the source that was already structured with the example YAML files and you could follow along- e.g. /demo/step_1_manifests, /demo/step_2_manifests, etc.
• Tutorial would be way more approachable with some diagrams and explanation of "why" I want to do any of these things. I think couching it in an example app (like a fictional eCommerce site selling items) would make it more approachable

Resources is all around a better term for what we're trying to describe, can also do some other renaming along with this

• Fides objects -> Fides resources
• show cli command changed to list or ls
• Update CLI docstrings to be more descriptive and helpful

It's too tedious to manually manage model/API documentation. While some of this will come with FastAPI for free there might still be come code in the core/lang that is in need of generated docs. It would be great to have these as part of our docs site

For the FastAPI stuff (includes models) -> https://pypi.org/project/mkdocs-render-swagger-plugin/#description

With the Beta coming up we need to make sure that we have a solid docs page up and running, its much simpler to do it here than have people spin up the docs in the repo for feedback

There is a feature built into our docs tool that can publish to github pages, this should be investigated and integrated into CI

We need a way to get people into Fides without needing a server at all. This command would do it!

Not sure if i'm sold on verify, maybe validate? not sure, but the gist is that the command should leverage the Fideslang module's parsing functions to create a local taxonomy without making any calls to the server. This would then lay the foundation for doing local-only evaluations with zero calls home to a server

• apply should allow for a dry-run
• apply should have an --include-systems-registries option

The versioning isn't quite working properly within fidsectl, and the docker tags are kind of meaningless. Need to clean this up and make them more usable in time for the go live push

This also includes the following requirements:

• Remove some server-side logic i.e. evaluations, dry-evaluations, so that generally the primary purpose of FidesAPI is to manage CRUD operations
• Remove the concept of "id's", instead allowing the fides_key to be the primary key for all resource types
• rename field names to be pythonic (snake_case) instead of camelCase
• An evaluations endpoint that accepts Evaluation objects from fidesctl, and evaluate updated on the fidesctl to send it
• Policy objects need to have a systems field that specifies which policies apply to which systems

Need to add Flask, a webserver command to Fidesctl, and a basic index page that pops up when visiting the webserver

A few of us have tried heading to the fides repo and getting started lately and I think it's pretty confusing. Holistically, it looks like we really need to step back and trim a lot of the docs to help smooth this out, but here are a few concrete suggestions, organized into a few themes...

Docs Consistency

• The "Quick guide" in the README is too dense to actually work- it dives immediately into manifests, classifiers, datasets, etc. This should likely just get removed in favour of a clearer tutorial
• The link to the tutorial docs in the README is a 404
• The README in the /docs site references an old setup step instead of make docs-serve
• Once I find the tutorial (https://github.com/ethyca/fides/blob/main/docs/fides/docs/tutorial.md) it does take me to the "getting started" guide for Docker, but I think you should consider just inlining that into the tutorial

Getting started errors

• The first command in the getting started with docker guide is make cli, but when I ran this from a clean repo it fails with org.flywaydb.core.api.exception.FlywayValidateException: Validate failed: Migrations have failed validation. After I thought about it a bit, I ran make init-db (which succeeded) and then make cli again
• Somehow, Kelly and I did manage to trigger build errors like "this project requires java 11" but I'm not sure exactly what command triggered that
• There seems to be no caching of the sbt update command, so on a slow internet connection like mine this means make cli takes a few minutes each time...

Deployment

• I realize now there is no "Deployment" guide for how someone would use this in their system without building from source- i.e. pull the docker images for CLI and Server, deploy those and start using it in their project. This is the primary use case we actually want, so what would that look like?

Without this feature it can't really be used in automated CI

Can be done once #82 is done!

the evaluate function needs to get updated so that it sends the Evaluate object it creates to the server

fidesapi.sql_models will also need to get updated with an Evaluation model (this will auto-generate the endpoints)

@ThomasLaPiana stumbled on this on Fri, but wanted to see if there's any way to not use explicit root passwords in our env files? i'm no expert, but...we are about to open this up to friends/family and don't want to have egg-on-face.

Need to get proper docs up and running, we've elected to use Mkdocs

Rebuild approval output so that it is cleaner and more comprehensible.

• add names to system declarations so that we can use those in approval output
• document all types of checks we're doing in the doc site

Need to be able to dry run manifests against what exists in the server to show what would change if those manifests were applied

a lot of this logic already exists in apply.py and could be abstracted out a bit into other functions within apply.py or maybe even an entirely new module

also should add a "--dry" flag to "apply" so that users can load the resource files and confirm they're at least locally valid

I'm doing some testing and it seems like even though I'm defining a specific dataset field, the privacy declaration is still looking at the other classifiers and not just at the classifiers within the specified field

@stevenbenjamin can you confirm this is the current behaviour? if so, do you object to updating it so that it only cares about the classifiers within the specified fields?

Mock tests can lead to situations where tests are passing but aren't representative of reality. Need to replace any instances of mocked tests with full integration tests

The current make help leaves a lot to be desired - it says "Under construction" 😄

I have a common snippet I use in Makefiles to effectively "annotate" the targets with some docs, it works like this:

# This target auto-generates a helpdoc by parsing the Makefile itself for special "##" comments
# See https://gist.github.com/prwhite/8168133#gistcomment-3291344
.PHONY: help
help: ## show this help
	@egrep '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

All you need to do for that to work is write your targets in the format of:

<target>: ## <docstring>

The egrep + sort + awk magic does the rest. I'm not sure how well supported this would be on non-UNIX systems (ie Windows) though, since it does rely on those being available.

Some points that still need to get covered before we should consider it "ready":

1. Talk about why Fides matters
2. Give a concrete example

Currently, a small inconsistency is that the privacy rules use the terms "approve" and "reject", while the evaluations themselves use the terms "pass" and "fail". We should settle on a single term to use for the sake of consistency

Currently the evaluate command accepts a fides_key as a parameter and passes it to the evaluate.py module but it doesn't actually do anything...

The evaluate function needs to be updated so that it will only evaluate the policy that matches that fides_key. This can be found in the fidesctl.core.evaluate module in the evaluate function at the bottom.

One of the core components of Fides is the ability for it to run as part of CI and the greater "SDLC". For this to be possible, Fides needs to be aware of when its running on main/master and when its running in a branch. My proposed solution for this is to add a new field to all database objects called branch or something similar, and have Fidesctl inject the name of the git branch into that field when sending objects to the server and also filtering for that field when querying the server for objects.

Needs to be an easier way to do this than just in the CLI, should be able to browse all of the objects in the server via the UI

https://github.com/python-versioneer/python-versioneer

Overview

For this issue, I'm going to provide as many notes as possible on the current docs site - specifically as if I was reading up about Fides for the first time. I'll use the structure of the docs themselves to leave comments here, since I didn't do this on the original PR 🙃

Home

Overall, I'd recommend we combine "Home" & "What Is Fides?". The very first paragraph of the docs should explain what the project is, who it's for, and give an example of how it helps you as an engineer. We are doing this twice in the docs right now and neither of them completely work independently. See React and Airflow as two examples of pretty good ones.

Once we combine the two sections, the general layout would be:

• What is Fides?
• Diagram
• Quick Example
• Core Components (Systems, Datasets, Registries, Policies) - this would be the descriptions from "What Is Fides?", plus the YAML examples from "Home".

Last feedback here is that the Example used it too complicated - we should focus on a really minimal example of a System, Dataset, and Policy.

License

Looks good

Getting Started

What does fidesctl ping do? It's not described why I should run that

Minor, but between the two setup methods they don't both start with "clone the Fides repo"

What is Fides?

This is a good section - see the comments in "Home" for how I think we should combine the two and lead with it

Tutorial

This is pretty lightweight, I'd expect to see example manifests / policies / etc. described in here. I'd be hard-pressed to use this tutorial to get started, I want to see example commands and more of a 1-2-3 guide to getting started with my first system+dataset+policy+evaluation, I think.

I wrote an imaginary Getting Started guide a few months ago that could be inspiration to help structure this more.

Fides Objects

To me, I'd expect to see the YAML syntax fully documented here, e.g. for each model all the supported keys and what they do (e.g. system.declarations.dataCategories). If this kind of documentation is better served by the API though (for example) we could link to the API docs.

The "Declarations" description is hard to understand when you read it, because it relies on the definitions from "Privacy Classifiers" section first, which is at the bottom. We'll likely need to reorder this.

The "Privacy Classifiers" needs a lot more explanation. What's a data category? What's a data use? Why do I care? What values can I use? We should also be citing ISO 19004 here, because that's what gives these initial definitions a bunch of merit and weight.

NOTE: I'm noticing here an API quirk that's going to bother me... why is it dataUse and not dataUseCategory?

Overall: this section is pretty good and it's explaining the concepts through each object and examples. I think we should try to reorganize them slightly by ensuring the description follows a consistent pattern like:

• What is the object used for?
• What are the fields/arguments/options/etc.?
• 2-3 Example Uses

So for example, take the dataset object:

### Dataset
In Fides, `datasets` are used to declare what kind of data is stored in a given database, by annotating individual database columns with an accurate `data category`. A single dataset can declare one or more tables, each of which should list all the fields in that table and annotations for each. This allows you to easily determine exactly where personal data is stored in your databases, since you've gone through the process of annotating them!

Structure:
* `organizationId` (required): ID of the organization this dataset belongs to
* `fidesKey` (required): a unique...
* ...

(you get the idea)

### Fidesctl
TODO

### Fides Server
TODO

### Deployment
TODO

### Contributing
TODO

Everything is standardized as JSON within the CLI (even though manifests are described as YAML, they get converted), is it worth tearing the YAML support out? If it is a significant amount of double code that needs to get maintained/updated, it's probably not worth keeping

After the huge #86, there is still some work to be done here!

• Consider a dataset's privacy data when evaluating a system that declares a relationship to it
• the function that finds FidesKeys should be recursive so that it can find nested references
• evaluations currently don't consider the hierarchy of type, only if there is a strict match. i.e. identified data is more dangerous than anonymized data