Git Product home page Git Product logo

gcf.cr's Introduction

gcf.cr

CircleCI

GCF provides managed execution of crystal language code within Google Cloud Functions. GCF compiles your crystal code statically using the crystallang/crystal docker image or optionally using your local crystal installation (if it is capable of static compilation) via the --local option. It then bundles your compiled crystal code in a thin node.js wrapper function and deploys it to GCP using the options you specify. An API is also provided for writing to the console, throwing errors, and returning the final response.

Installation

  1. set up a Google Cloud Platform account if you don't have one already and create an initial project
  2. install the gcloud sdk if you haven't already
  3. log in to gcloud locally via gcloud init if you haven't already
  4. install docker (if you haven't already)
  5. set up docker to not require sudo
  6. start the docker daemon (e.g. sudo systemctl start docker) if it isn't already running
  7. clone the repo git clone [email protected]:sam0x17/gcf.cr.git
  8. run ./setup. This will compile and install a gcf binary in /usr/bin.

If you plan to use docker-based static compilation (default option), you don't need to install crystal on your system as long as you have a statically compiled gcf binary. You can use the build_static script included in the repo to build a static binary for gcf using docker. That said, having crystal locally installed will make it easier to write tests.

Getting Started

All cloud functions should consist of a crystal app project (created via crystal init app) where the main project file (e.g src/my_project.cr) meets the requirements outlined below.

Add the following to your shard.yml file and run shards install:

# shard.yml
...
dependencies:
  gcf:
    github: sam0x17/gcf.cr
    branch: master

Create a class that inherits from GCF::CloudFunction and defines a run method that accepts an argument of type JSON::Any, typically named params:

# src/example.cr
require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    # your code here
  end
end

Once you have a setup like this, you can use the various API functions from within your run method, which makes up the body of your cloud function. The available API functions are listed below. Note that methods like send, send_file, and redirect stop execution when they are run, meaning any code after these methods will not run unless it was already defined in an at_exit block. If you do not call one of these methods in your function, your function will run until it times out.

Once you are done writing your function, you can deploy it using gcf --deploy.

Crystal API

A crystal-based API is provided for communicating with the Google Cloud Function host process so you can do things like log to the console redirect the browser, or send textual or file-based data to the browser. The API is a thin layer on top of the underlying ExpressJS API used by Google Cloud Functions, and uses a combination of inter-process communication and files to send data to/from the host process.

The most basic requirement, as outlined in the getting started section, is that you create a class that inherits from GCF::CloudFunction and provide a definition for the run(params : JSON::Any) method.

params

The run method of your crystal function will provide the HTTP params object in the form of a JSON::Any object named params. You can access values in the params object as you would a hash:

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.log params
    send "color: #{params["color"]?}"
  end
end

If you send the http parameter color with the value of red, the function will return "color: red" with a 200 status code and you will get the following console output: console image

console.log(msg)

Logs whatever you pass it to the GCF console with info priority. Equivalent to using console.log in JavaScript. msg is interpolated so non-strings may be passed in.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.log "some info here"
  end
end

console.warn(msg)

Logs whatever you pass it to the GCF console with warn priority. Equivalent to using console.warn in JavaScript. msg is interpolated so non-strings may be passed in.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.warn "woah, warning"
  end
end

console.error(msg)

Logs whatever you pass it to the GCF console with error priority. Equivalent to using console.error in JavaScript. msg is interpolated so non-strings may be passed in.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.error "GASP! an error"
  end
end

send(content)

An alias for send(200, content), where 200 is the HTTP OK/ready status code.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send "OK, done executing"
  end
end

send(status : Int, content)

Sends the interpolated version of content as output to the browser with an HTTP status code of status, and stops execution of the cloud function. This is forwarded to req.send in ExpressJS.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send 200, "<h1>YO</h1>"
  end
end

redirect(url : String)

An alias for redirect false, url, since temporary redirects are usually preferable when used with cloud functions.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    redirect "https://google.com"
  end
end

redirect(permanent : Bool, url : String)

Redirects the browser to the specified url. If permanent is true, it will do a 301 redirect. If permanent is false, it will do a 302 redirect.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    redirect true, "https://google.com"
  end
end

send_file(path : String)

An alias for send_file 200, path, since typically you will only want to send file content with a status code of 200.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send_file "van_gogh.jpg"
  end
end

send_file(status : Int, path : String)

Sends the file at the specified path to the browser with an HTTP status code of status (to write files from crystal in a cloud function, you need to write to something in /tmp).

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send_file 200, "van_gogh.jpg"
  end
end

Note on puts

If you call puts directly from within a cloud function's run method, this gets mapped to console.log. This does not apply to puts calls that are made indirectly (e.g. calling code outside of this class), so the contents of these puts calls will not be handled correctly and lead to undefined behavior.

Note on exceptions

Right now exceptions work locally but not in a deployed function. We are working on this but please feel free to take a look at #1 and help out if you have any ideas. From what we can tell, all execution stops the second an exception is thrown, even from within a try-catch block, when a function is executing on GCP. For now we are logging a generic message stating that an error occurred, however we are unable to retrieve the error stacktrace or name (locally we are able to do this).

Tests / Specs

You can test your cloud function with specs the same way you would any conventional crystal-based app or library. See gcf_spec.cr for examples of how to test for particular functions outputs, redirects, etc. Also make sure to set GCF.test_mode = true in your spec_helper.cr file before any specs are loaded or the development server will attempt to run before your specs can run.

Example spec_helper.cr file:

# spec/spec_helper.cr
GCF.test_mode = true

require "spec"
require "../src/my_project.cr"

Development Mode / Test Server

A built-in test server is provided (based on Kemal) that allows you to simulate requests to your (HTTP-triggered) cloud functions on your local machine. Simply compile and run your cloud function e.g. crystal run src/*.cr and Kemal will automatically start the test server on port 8080. You can access it in a web browser by going to http://localhost:8080/, which will trigger your cloud function and load the result as if you just visited the HTTP trigger URL for the function. Note that while send_file/send/redirect methods normally stop execution of your function once they execute, this is not the case in the test server, though this is something we are trying to find a workaround for.

GET and POST params that are sent to the test server are automatically loaded into the params of your cloud function when it is invoked.

gcf_test$ crystal run src/gcf_test.cr
[development] Kemal is ready to lead at http://0.0.0.0:8080
2018-07-27 04:04:06 -04:00 200 GET / 375.0µs
{"color" => "red"}
2018-07-27 04:04:13 -04:00 200 GET /?color=red 1.41ms

Deploying

Note that GCF expects your crystal function to follow the directory structure imposed by crystal init app, in that all of your crystal code should reside in project_name/src/. During compilation, GCF uses the src/*.cr glob to compile all crystal files in the src directory.

Note also that GCF will automatically consult gcloud to discover the current GCP project id if one isn't specified.

Below you can find some basic usage examples fo rcommon use cases. For full usage information, please see the output of gcf --help.

Compile the current directory using the docker image and deploy as a function named after the current directory (default):

gcf --deploy

Specifying the source directory, static compilation using the local crystal installation, the function name, the memory capacity of the deployed function, and the google project ID respectively.

gcf --deploy --source /home/sam/proj --local --name hello-world --memory 2GB --project cool-project

Or using shorthand:

gcf -d -s /home/sam/proj -l -n hello-world -m 2GB -p cool-project

TODO

  1. attribute API so we don't need command line params
  2. fix exceptions logging bug (#1)
  3. more testing

gcf.cr's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

gcf.cr's Issues

RFC: gcloud attribute API

Right now it is necessary to use the command line parameters and/or information obtained directly from gcloud to determine the function memory, trigger type, function name, gcp project ID, etc. Now that we have a crystal-based API, we could better specify this information directly within the cloud function source code. I'm thinking something like this:

require "gcf"

class MyFunction < GCF::CloudFunction
  function_memory "256 MB"
  project_id "some_project"
  trigger_type :http

  def run(params : JSON::Any)
    # your function code here
  end
end

etc.

Thoughts?

exception handling (and thus exception logging) does not work on deployed functions

Right now our exception logging feature is broken because we get different behavior locally vs on a deployed cloud function. Specifically, if you look at the comment out code in the exec method in cloud_function.cr, you will see that we run the entire cloud function in a try-catch statement, and then log any exceptions that occur to the console.exception logger mechanism. The problem is when we activate this try-catch statement, no code ever runs on deployed functions after the exception is thrown (the catch statement does not execute). When we do the same thing locally, this does not happen. If anyone has any insight we would love to fix this issue, as we can easily have full stackdriver support for crystal exceptions.

support new regions (us-east1, asia-northeast1, europe-west1)

Google Cloud Functions has added support for three new regions in addition to us-central1, so the total list now includes Iowa (us-central1), South Carolina (us-east1), Tokyo (asia-northeast1), and Belgium (europe-west1).

We need to update gcf.cr's command-line parsing options to allow for these additional regions, as right now any region but us-central1 is rejected.

local/development mode testing

Need to create an HTTP server that will serve the response of send_file and send. Bonus points for a params input window like Google Cloud Platform has.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.