Golang Restful APIs in a cup, and ready to serve!
- Go-Rest framework for service controllers and tag-based configurations
- Popular WebServers (like Apache,IIS) for hierarchical configuration model
- Simplicity & Modularity
- Developer Productivity
- High Configurability
- Low learning curve
- Easy Versioning
- Pluggable Modules (using golang middleware)
- High Performance
- Preference for Json (over xml)
- Service Controllers to define endpoints (modular and organized code)
- Powerful Configuration Model. Supports 4 levels:
- at server level, programmatically (these are inherited by all endpoints)
- at service controller level, declaratively using golang tags (these are inherited by all contained apis)
- at service controller level, programmatically
- at api or endpoint level, declaratively (these override inherited configurations)
- Versioning
- Multiple versions are supported easily by
- defining at service controller level (inherited by all internal endpoints)
- overriding for each endpoint specifically
- Multiple versions are supported easily by
- Caching
- Database Binding
- CRUD endpoints
- Limited ad-hoc querying
- Stubbing
- If there are code/project dependencies on your api service, you can simply write a stub (sample output) in an external file and publish this mock api quickly before writing actual business logic
- Working with Queues |pending
- Proxying or Wrapping around existing APIs |pending
- Middleware support (by defining modules)
- Logging (using middleware)
##Lets explore these features
First define a service controller in your project that supports a GET response (aqua.GetApi as its type). Note that the controller defined as a struct must anonymously include aqua.RestService.
type HelloService struct {
aqua.RestService
world aqua.GET
}
Now implement a method corresponding to 'world' field after uppercasing the first letter. To start off, the method can return a string (more on this later).
func (me *HelloService) World() string {
return "Hello World"
}
Now setup your main function to run the Aqua rest server
server := aqua.NewRestServer()
server.AddService(&HelloService{})
server.Run()
Now open your browser window, and hit http://localhost:8090/hello/world
Sure, just change the function signature and you are good to go.
func (me *HelloService) World(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "Hello There!")
}
####Q: I want to change the url from /hello/world to /hello/moon. Do I need to change the method names?
The service urls are derived from url tags. If none are specified then it defaults to the method name. So you can simply introduce the tag as follows.
type HelloService struct {
aqua.RestService
world aqua.GET `url:"moon"`
}
Q: What if I need to return both Hello World, and Hello There as different versions of the same GET api?
Simply add both the methods, but specify versions in field tags.
type HelloService struct {
aqua.RestService
world aqua.GET `version:"1.0" url:"moon"`
worldNew aqua.GET `version:"1.1" url:"moon"`
}
func (me *HelloService) World() string {
return "Hello World"
}
func (me *HelloService) WorldNew(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "Hello There!")
}
Now you can hit:
http://localhost:8090/v1.0/hello/moon and
http://localhost:8090/v1.1/hello/moon to see the difference.
There are 3 out-of-box setting available, that help you customize URLs.
- prefix
- root
- url
We have already seen how 'url' works.
To change the root directory (hello), you can use the root tag at each service level, or more simply at the service controller level as demonstrated below:
type HelloService struct {
aqua.RestService `root:"this-is-the"`
world aqua.GET `version:"1.0" url:"moon"`
worldNew aqua.GET `version:"1.1" url:"moon"`
}
With this change, your api endpoints are now working as:
http://localhost:8090/v1.0/this-is-the/moon and
http://localhost:8090/v1.1/this-is-the/moon
You can also use the 'prefix' field. This part comes in before version information in the final constructed endpoint url
type HelloService struct {
aqua.RestService `root:"this-is-the" prefix:"sunshine"`
world aqua.GET `version:"1.0" url:"moon"`
worldNew aqua.GET `version:"1.1" url:"moon"`
}
So with this prefix now set, our end points would become:
http://localhost:8090/sunshine/v1.0/this-is-the/moon
http://localhost:8090/sunshine/v1.1/this-is-the/moon
Also note that, all there of these properties (url, root and prefix) can contain any number of slashes. So if you change the url to:
type HelloService struct {
aqua.RestService `root:"this-is-the" prefix:"sunshine"`
world aqua.GET `version:"1.0" url:"/good/old/moon"`
}
Then you get the final url as:
http://localhost:8090/sushine/v1.0/this-is-the/good/old/moon.
Yes, Gorilla mux is used internally. So to define url parameters, we'll need to follow Gorilla mux conventions. We'll get to those in a moment
By default an "aqua" route is setup:
- /aqua/ping returns "pong" if the server is running
- /aqua/status returns version, go runtime memory information
- /aqua/time returns current server time
It's 8090. You can change it though as follows:
server := aqua.NewRestServer()
server.AddService(&HelloService{})
server.Port = 5432;
server.Run()
type CatalogService struct {
aqua.RestService `root:"catalog" prefix:"mycompany"`
getProduct aqua.GET `version:"1.0" url:"product"`
}
If you setup a catalog service as shown above then out of box you can use version capability as shown below
- GET call to http://localhost:8090/mycompany/v1.0/catalog/product
- GET call to http://localhost:8090/mycompany/catalog/product
- pass a request header "Accept": "application/vnd.api+json;version=1.0"
- -or-
- pass a request header "Accept": "application/vnd.api-v1.0+json"
Note: If you want to customize the media type, you can do so.
type CatalogService struct {
aqua.RestService `root:"catalog" prefix:"mycompany"`
getProduct aqua.GetApi `vendor:"vnd.myorg.myfunc.api" version:"1.0" url:"product"`
}
Basis this, the required Accept header will be need to changed to following:
- "Accept" header : "application/vnd.myorg.myfunc.api+json;version=1.0"
- -or-
- "Accept" header : "application/vnd.myorg.myfunc.api-v1.0+json"
Its simple, you add an input variable to your implementation method of type aqua.Aide (a helper class) This variable gives you access to the Request object, and also has some helper methods as shown below:
type HelloService struct {
aqua.RestService
world aqua.GET
}
func (me *HelloService) World(j aqua.Aide) string {
j.LoadVars()
return "Hello " + j.QueryVars["country"]
}
Now, just hit the url: http://localhost:8090/hello/world?country=Singapore
You start by defining the url with the appropriate dynamic variable as per the guidelines of Gorilla mux.
type HelloService struct {
aqua.RestService
world aqua.GET `url:"/country/{c}"`
}
Then you just read this value in the associated method. Note: Aqua currently supports passing int and string parameters.
func (me *HelloService) World(c string) string {
return "Hello " + c
}
Now, you can hit http://localhost:8090/hello/country/Brazil
In case you are reading an integer value, then you can define strict logic in url to only match numbers using a regular expression:
type HelloService struct {
aqua.RestService
world aqua.GET `url:"/country/{c}"`
capital aqua.GET `url:/capital/{cap:[0-9]+}`
}
Q: Can you explain how the configuration model works? Will I need to define attributes at each endpoint level?
Aqua has a powerful configuration model that works at 4 levels:
- Server (programmatically)
- Service controller (declaratively)
- Service controller (programmatically)
- Endpoint (declaratively)
Lets look at each of them in detail
If you define any configuration at the server level, then it is inherited by all the Service controllers and all the contained services automatically.
server := aqua.NewRestServer()
// Note:
server.Prefix = "myapis"
// Prefix value is inherited by everything on this server!!
server.AddService(&HelloService{})
server.AddService(&HolaService{})
server.Run()
We added two service controllers to the server above - HelloService and HolaService. Let's assume that all the contained services need to begin with words 'Hello' and 'Hola' respectively.
To achive this, we specify the 'root' variable at the top level by defining it agains the RestServer.
type HelloService struct {
aqua.RestService `root:"Hello"`
service1 aqua.GET
service2 aqua.GET
..
serviceN aqua.GET
}
This ensures that all services in this now inherit the root value of "Hello"
Last but not the least, you can specify a value at a service endpoint. You can do so by configuring at the api level as shown below. Note that these values will override the inherited values.
type HelloService struct {
aqua.RestService `root:"Hello"`
service1 aqua.GET `root:"Hiya"` //Hiya overrides Hello
service2 aqua.GET
..
serviceN aqua.GET
}
Tag | Usage |
---|---|
prefix, pre | Url prefix as in: http://abc.com/[prefix]/v1/root/url |
root | Url root as in: http://abc.com/prefix/v1/[root]/url |
url | Url path as in in: http://abc.com/prefix/v1/root/[url] |
version, ver | Url version as in: http://abc.com/prefix/v[1]/root/url |
vendor, vnd | |
modules, mods | Sequence of module names (or middlewares) that the request goes through |
cache | The name of cache provider to use |
ttl | Duration to cache (e.g. 5s or 10m) |
stub | Relative or absolute path to the file containing the mock stub |
wrap | Wrapping other/3rd party rest services |
Multiple cache provider's can be added to RestServer via its AddCache method. The arguments are the unique name of the cache and an implementation of Cacher interface. Note: this interface is defined in the sister library aero/cache
server := aqua.NewRestServer()
server.AddCache("mycache", <implementation of aero.cache.Cacher>)
server.Run()
This interface is defined as:
// package aero.cache
type Cacher interface {
Set(key string, data []byte, expireIn time.Duration)
Get(key string) ([]byte, error)
}
"aero.cache" also defined multiple implementations of this interface, namely:
- Memcache implementation
- Redis implementation
- In memory cache (recommended for dev boxes)
- Debug Wrapper to log reads and writes to text files (for debugging)
Let's utilize the memcache implementation.
server := aqua.NewRestServer()
server.AddCache("mycache", cache.NewMemcache("127.0.0.1", 11211))
server.AddCache("remote", cache.NewMemcache("123.234.012.23", 11211))
server.Cache = "mycache" // default for all endpoints
server.Run()
Now all we need to do to use this cache is to set the "ttl" tag. So lets look at some services
type CatalogService struct {
RestService
getProduct GET `url:"product/{id}" ttl:"5m"`
getSeller GET `url:"seller/{id}" ttl:"15m" cache:"remote"`
}
Thats it. You are good to go!
Service http://localhost:8090/catalog/product/{id} inherts cache as "mycache" from server. All calls to different invocations will get cached for 5min (as set in ttl).
Service http://localhost:8090/catalog/seller/{id} overrides the server cache and sets its cache store to be "remote". So, all invocations will be saved in this remote memcache instance for a rolling 15min duration.
Current limitations:
- only available for GET calls (others will be added)
- not supported when using standard http handler (w http.ResponseWriter, r *http.Request)
Note: the cache key is the the unique url of the request (including the query string parameters)
Modules allow you to harness the power of Golang middleware. A module is a function that returns an anonymous function that consumes an http.Handler and returns another http.Handler. The signature of the module would be:
func SomeFunc(<params_if_needed>) func(http.Handler) http.Handler {
// code
}
Say, we want to log all those calls that are taking more then 1 sec to return. So we write this module as:
func LogSlowCalls() func(http.Handler) http.Handler {
file := "/path"
f, err := os.OpenFile(file, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
// error handling..
}
l := log.New(f, "", log.LstdFlags)
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
next.ServeHTTP(w, r)
dur := time.Since(start).Seconds()
if dur >= 1 {
l.Printf("%s %s %.3f", r.Method, r.RequestURI, time.Since(start).Seconds())
}
})
}
}
We can also pass duration to log and file path as parameters to LogSlowCalls function to make it more customizable. Now lets add this module to our server.
server := aqua.NewRestServer()
server.AddModule("slowLog", LogSlowCalls("/tmp/file.log", 1))
server.Run()
// And to use it in our service, we just pass it to the tag
type CatalogService struct {
RestService
getProduct GET `url:"product/{id}" module:"slowLog"`
getSeller GET `url:"seller/{id}"` module:"slowLog, module2, module3"
}
Note:
- slowLog is set for getProduct endpoint. Any request that takes more than 1 sec will be logged.
- getSeller endpoint has multiple modules set in a comma seprated manner. All these will be invoked in the sequence slowLog (first) → module2 → module3 (last) → and finally the method "GetProduct".
- Aqua uses github.com/carbocation/interpose for method chaining internally to setup these golang middleware.
Just a few:
- A slow logger, ModSlowLog, that takes millisec precision as an input
- An access logger, ModAccessLog
Yes. Aqua makes is possible to create mock api stubs using external files. You can specify an associated file using the "stub" tag as shown below.
type MockService struct {
RestService
yetToCode GET `stub:"samples/some.json"`
}
// And then run it
server := aqua.NewRestServer()
server.AddService(&MockService{})
server.Run()
Now when you invoke http://localhost:8090/mock/yet-to-code then the contents of "samples/some.json" file are read and returned. Note that the returned code may not necessarily be json data. It can be anything.
Aqua searches for file in both:
- executable directory, and
- working directory
Also, you can specify file path using both:
- relative file syntax, and
- absolute file path syntax
If the file is not found then Aqua returns 400 status code.
When using such mock stubs, you don't need to define any methods for your endpoints.
In order to improve developer productivity Aqua supports basic database operations. This functionality is tied to the popular GORM https://github.com/jinzhu/gorm project. Let us see how we can set this up, for a "user" table
First, we define the model (as per GORM specs):
type User struct {
id int `gorm:"primary_key"`
username string
name string
}
Second, we add an endpoint of type CrudApi (instead of Post or Get etc)
type AutoService struct {
RestService
users CRUD
}
Third, in the service method, we define the function to return a CrudApi struct address.
func (s *AutoService) Users() CRUD {
return CRUD {
Model: func() (interface{}, interface{}) {
return &User{}, nil
}
}
}
Basically in the CrudApi struct we define the gorm model to use as an address to return in the Model() function.
Now let's test the ready-made endpoints, by hitting:
- GET to http://localhost:8090/auto/users/123
- POST to http://localhost:8090/auto/users such that they body payload contains a json like:
{
id: 1234,
username: "jdoe",
name: "John Doe"
}
- PUT to http://localhost:8090/auto/users/345 such that the body payload contains a json to update user with id 345
{
username: "jbrown",
name: "Jason Browne"
}
- DELETE to http://localhost:8090/auto/users/567
That's it. You write a function to return a CrudApi object and you get 4 CRUD methods out of the box.
By default, AQUA uses the default master database (as specified in you yaml file). If you want to override it then you can do so easily, as shown below:
func (s *AutoService) Users() CRUD {
return CRUD {
Engine: "mysql",
Conn: "your-connection-string",
Model: func() (interface{}, interface{}) {
return &User{}, nil
}
}
}
While fully generic ad-hoc querying (that may include joins over many tables) is not yet supported, AQUA does support limited querying to a specific model.
In other words if your query is of this type, then you can run it out of box with AQUA.
SELECT * FROM <model_table> WHERE <conditions>
Continuing from the previous example, let us modify the CrudApi return to include Models() method:
func (s *AutoService) Users() CRUD {
return CRUD {
// Add 2nd return (slice of your models)
Model: func() (interface{}, interface{}) {
return &User{}, &[]User{}
}
}
}
This will give you two additional endpoints:
- POST @ http://localhost/auto/users/!
- POST @ http://localhost/auto/users/$
Let us see each of these in detail.
POST @ http://localhost/auto/users/!
This endpoint takes SQL where cluase as Raw Body and returns a json array of matching users. So, we can the Body as:
id in (1,2,3,4,5) OR username like "j%"
When executed, the final query becomes:
SELECT * FROM users WHERE id in (1,2,3,4,5) OR username like "j%"
POST @ http://localhost/auto/users/$
This endpoint takes parameterized inputs. You specify a json as Raw Body with "where" and "params" keys as shown below:
{
"where" : "username like ? or name like ?",
"params" : [ "j%", "Tim%" ],
"limit" : 100,
"offset" : 25,
"order" " ["username", "name desc"]
}
When executed, the final query becomes:
SELECT * FROM users WHERE username like "j%" or name like "Tim%" order by username, name desc limit 100 offset 25
The output is same, a json array.
Talking to NoSQL systems is planned but not implemented yet.
Q: If I wanted to switch out gorm (default ORMapping tool used), and switch to a different one then can that be achieved?
Yes, this can be done. At this time however, only GORM is supported.
There are a number of good reasons why you would want to wrap any existing Rest apis, say:
- to enable caching
- to setup logging / monitoring
- to modify/manipulate responses or headers
Aqua supports this using the "wrap" tag configuration
type WrapperService struct {
RestService
MyDoSomething GetApi `wrap:"http://abc.com/do/someting" ttl:"25m"`
}
[TBD]
You can add a header through a middleware. But this must be done before the call to next.ServeHTTP(..). Any header additions or response body modifications post this will have no effect.
func ModModify() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Custom-A", "a1") /* this works */
next.ServeHTTP(w, r)
w.Header().Set("X-Custom-B", "b1") /* no effect */
})
}
}
To conditionally add headers or modify response basis the body, you need to use httptest.ResponseRecorder. There is already a built in middleware that gives this functionality - it is named ModRecorder(). Let us see how to use it, with a complete example.
func ModModify() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Custom-A", "a1")
next.ServeHTTP(w, r)
w.Header().Set("X-Custom-B", "b1")
w.Write([]byte("appending text to response body"))
})
}
}
Now we add this to our server along with ModRecorder:
s := aqua.NewRestServer()
s.AddModule("chg", ModModify())
s.AddModule("rec", aqua.ModRecorder())
s....
s.Run()
Now invoke both these middleware in your rest call.
type HelloService struct {
aqua.RestService
world aqua.GetApi `mods:"rec,chg"`
}
This will first call ModRecorder middleware which will setup a httptest.ResponseRecorder and pass it on. The next middleware, ModModify can now change headers before and after the next.ServeHTTP call. It is also able to modify the response body now.