See if your favorite IP addresses are In Da Haus!!!
- About
- Getting Started
- A Note About The Data
- Deployment
- Usage
- k8s
- Tests
- Built Using
- TODO
- Regrets
- Author
- Acknowledgments
indahaus is the exciting new graphql api that stores a users DNS blacklist queries for fast retrieval
You'll need go 1.16+ installed to run and test locally.
For building and running the docker container you'll need docker.
For running in a local kubernetes cluster you can use my favorite, kind, install instructions here.
Then you'll need kubectl.
Finally you'll need helm, whew, got all that?
To run locally:
make migrate
make run
To create and run a docker container:
make docker-build
make docker-run
For k8s instructions see below.
There are some other handy command to aid in local dev, like
# tidy up the go.mod file
make tidy
# reset the db
make resetdb
# run the tests
make test
Clone the repo:
git clone https://github.com/shaneu/indahaus.git
cd indahaus
Download the dependencies:
go mod download
Migrate the db
make migrate
Build (or run) the binary:
make build
# or
make run
and you're off the races.
> AUTH_PASSWORD=***** AUTH_USERNAME=***** PORT=8080 make run
go run cmd/api/main.go
API: 2021/05/22 16:02:02.985390 main.go:79: main: Application initializing: version "develop"
API: 2021/05/22 16:02:02.985514 main.go:84: main: Initializing database support
API: 2021/05/22 16:02:02.985711 main.go:106: main: Debug Listening :4000/debug/vars
API: 2021/05/22 16:02:02.985979 main.go:127: main: Api listening on :8080
A single IP address can have multiple results, for example the IP 103.35.191.44 has three results, 127.0.0.3, 127.0.0.4, 127.0.0.2.
I have decided to store all three as a comma separated string so the user can see any codes that may apply to the IP address they enqueued.
Conversely, when an address has no codes the user will receive null
.
Codes that represent an error from the spamhaus API, their equivalent of a 400, will not be stored. In other words, if the code received is 127.255.255.255 meaning an excessive number of queries, that information is useful to us as the developers, but not the user.
A possible future state of the app would be to have the response_code field return a slice of items that might contain the code and a human readable message. We could have a table of response codes with a FK relationship.
If you have all the perquisites installed you can run:
make up
which will build the docker image, bring up the kind cluster, load the image into kind and install the helm chart. To interact locally run
# terminal 1
make port-forward-debug
# terminal 2
make port-forward-api
which will forward both the debug vars port 4000 and the api port 8080.
To view debug/metrics info you can use your favorite HTTP API tool such as postman, curl, or the good ole browser and visit http://localhost:4000/debug/vars
To interact with the graphql api you'll need pass a basic auth header.
To bring everything down, including the kind cluster, run:
make down
Some useful dev commands:
make update-api
will rebuild container, load it into kind, and redeploy the pod. This is handy when you've made a change
and want to test it.
make test
Runs the test suites
This application takes a layered architecture approach.
At the bottom layer are the packages in the pkg
dir. These are the kinds of packages that could be ripped out and moved
into any project that needs them. They contain no business logic or cross cutting concerns like logging and expose a simple api.
The next layer is our business logic/data layer, in the internal
dir. Packages in this layer can require packages from the pkg
dir, but not visa versa.
Finally we have our application layer where our graphql and our rest endpoints live. The cmd
dir is where our binaries live, our our case
we have two, the main app and a thin admin app that does some useful things like migrating our database. Ideally the graph
dir should be nested in
cmd/api
but gqlgen seems happier when it isn't.
- errors - Provides a nice way to capture errors along with their context
- viper - Elegant project configuration that allows us to define sane default parameters and override them with env vars
- go-sqlite3 - The only sqlite driver that is included in and has passed the go driver compatibility test suite doc
- sqlx - provides some nice extensions beyond what is delivered by the built in sql library while still being fully interface compliant. Will be useful if we decide to change from sqlite to something like postgres
- echo - A minimalist web framework we're using for it's routing and its auth and panic recover middlewares
- uuid - Generates our trace IDs and db IDs
- gqlgen - Takes a lot of the boiler plate out of creating a graphql api all while providing a high level of type safety
- go-cmp - For doing easy comparisons between fields in our tests
- Add support for tracing and metrics collection.
- Integration tests: go has amazing built in support for running integration tests using the httptest package
- Install a migration framework to allow us to update and roll back or database schema
- Improved error handling: We should create a subset of trusted errors or a custom error to respond to the user with without leaking information about our system
These are things I would have liked to have done/been able to do differently.
Right now requests are being handled by 2 different ServeHTTP methods - the echo framework and the gqlgen framework. This means that my centralized error handling/reporting is split into two - I have to have one for the graphql requests and one for the three REST routes. Now, the argument could be made that those 3 REST routes, /liveness, /readiness, and /debug/vars are not the business deliverable here and so elegant error handling and the like isn't critical, and I could easily be persuaded. What if tomorrow though a client comes and says I will give you 1 billion dollars if you write a rest endpoint for this service that I can use because Acme Corp doesn't use graphql? I mean, a billion dollars, that's a lotta green. I dug through the gqlgen source quite a bit and they don't expose a way to have it simply manage the graphql query execution and let the consumer choose the transport, which would have been nice.
- My dog Chaos
- My cats Judy and Eris
- My wife Nina for being super supportive and taking over a bunch of the chores while I worked on this