Offer support for websockets using cloudrun behind the scenes similar to chalice websockets.

@app.on_ws_connect()
def connect(event):
    print('New connection: %s' % event.connection_id)


@app.on_ws_message()
def message(event):
    app.websocket_api.send(event.connection_id, 'I got your message!')
    print('%s: %s' % (event.connection_id, event.body))


@app.on_ws_disconnect()
def disconnect(event):
    print('%s disconnected' % event.connection_id)

• Note this would require dockerizing the goblet app and deploying to cloudrun

Got error when running goblet local when config.json was empty.

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Incorrect argument. Make sure you set the local param in your Goblet class and that it matches the arg used in goblet local```

would be nice to have a cli command goblet export-env to export env variables for local testing

Hello,

I noticed that the bucket used to store Cloud Function archives contains all the versions of all cloud functions deployed in the project. The number of objets is quite big after a couple of dev/try/test deployments.

Feature request :

• add an optional argument to goblet deploy to delete the previous version of the archive in Cloud Storage
• add an optional argument to goblet destroy to delete the archive of the function in Cloud Storage

Thanks !

add cli option to only deploy function and skipping other resources

would be nice to skip patching a function if no code changes were made from what was currently deployed.

uploaded hash can be retrieved from an api call headers

https://github.com/premisedata/cloud-custodian/blob/master/tools/c7n_gcp/c7n_gcp/mu.py#L153

Hello,

I created a topic in PubSub called mytopic and I tried to create a function triggered by a message in this topic. The deployment is successful but after I publish a message, I get a TypeError: dbt_handler() takes 1 positional argument but 2 were given. Here is the code of my function and the error traceback :

Function code

from goblet import Goblet

app = Goblet(function_name="goblet-pubsub")


@app.topic('mytopic')
def main_handler(data):
    app.log.info(data)
    return 

Error traceback

2021-11-11T15:20:13.051Z goblet-pubsub-topic-mytopic6gbe9k87mz0j 
Exception on / [POST] Traceback (most recent call last): 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app response
    = self.full_dispatch_request() 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request 
    rv = self.handle_user_exception(e) 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request 
    rv = self.dispatch_request() 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request 
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/functions_framework/__init__.py", line 171, in view_func 
    function(data, context) TypeError: main_handler() takes 1 positional argument but 2 were given

Exception on / [POST] Traceback (most recent call last): 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app response 
    = self.full_dispatch_request() 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request 
    rv = self.handle_user_exception(e) 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request 
    rv = self.dispatch_request() 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request 
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) 
File "/layers/google.python.pip/pip/lib/python3.7/site-packages/functions_framework/__init__.py", line 171, in view_func 
    function(data, context) 

TypeError: main_handler() takes 1 positional argument but 2 were given

What am I supposed to provided to main_handler ?

Thank you !

look into using grpc instead/together with openapispec

https://cloud.google.com/api-gateway/docs/grpc-overview

use gcp functions framework to deploy goblet function locally for testing

https://github.com/GoogleCloudPlatform/functions-framework-python

goblet init missing jsonify import

Hello

I start to have a lot of environment variables hard-coded in the config.json ; some are more sensitive than the others 🔒 . Would it be possible to allow the use of environment variable, and interpolate on run time, like the GIT_TOKEN and EXAMPLE_ENV_VAR here :

{
    "cloudfunction": {
        "runtime":"python37",
        "environmentVariables": {
            "KEY":"$EXAMPLE_ENV_VAR"
        },
        "buildEnvironmentVariables": {
            "GIT_TOKEN":"$GIT_TOKEN"
        },
        "availableMemoryMb": 256,
        "timeout": "30s"
    }
}

This could be particularly useful for local development, CI/CD and a light solution while waiting for the powerful [Secret Manager] Secret Manager Integration #58.

I would like to remove the request.headers.get('X-Envoy-Original-Path') which denotes a request coming from api gateway vs the cloudfunction being directly called via http. The one point that they overlap is for the route '/' so will just need to add logic for that route.

if request.path and request.headers.get('X-Envoy-Original-Path'):
            return "route"
return 'http'

A built in integration with Secret Manager to inject secrets into cloudfunctions. Currently the options are to supply env variable, or to write the secret manager logic yourself.

example config.json

"secrets": {
  "GH_TOKEN": projects/{project_id}/secrets/GH_TOKEN/versions/2",
  "OTHER_TOKEN"
}

with default being projects/{project_id}/secrets/{secret_name}/versions/latest

then in your code you would use config.secrets to access

from goblet.config import Config

config = Config()
@app.route("/reveal")
def reveal_secret():
    return config.secrets.GH_TOKEN

getting error while deploying to us-east1 . working in other regions. seems to be a recent change just to that region. following up with gcp whether it is a bug and/or will be deployed out to other regions

The function defined in file /workspace/main.py as app needs to be of type function. Got: invalid type <class 'goblet.app.Goblet'>

workaround would be to define a new entrypoint and call goblet app inside. this would require the user to set the entrypoint since now it is automatically set by goblet during deploy.

def entrypoint(request, context=None):
   app(request,context)

http cloudfunction should only be created for routes and scheduled jobs and not if only pubsub topics are selected.

go through and remove unused dependencies or move some to optional

Api gateway allows for different backends such as cloudrun, apigateway, compute engine or gke in addition to cloudfunctions, which should be configurable in goblet.

@app.route_backend("/cloudrun/{id}", CLOUDRUN_URL)
def cloudrun(id: string) -> string:
    return # wouldn't be called since in cloudrun, but needed for type_hints 


@app.route_backend("/app_engine/{id}", APP_ENGINE_URL)
def app_engine(id: string) -> string:
    return # wouldn't be called since in app engine, but needed for type_hints 

@app.route("/cloudfunction/{id}")
def regular_cloudfunction(id: string) -> string:
    return id  

could also be following where types returns the input and output types needed to create the openapi spec correctly.

app.route_backend("/app_engine/{id}", APP_ENGINE_URL, types)

goblet should automatically add security section with any defined securitydefitions. config.json should be able to override it as well if a user wants to add scopes. Security should also be able to be added to individual routes

https://cloud.google.com/pubsub/docs/reference/rest

Current the default service account is added to the gateway. should have a gateway or api config option in config.json where service account can be passed in and other options such as labels. needs to be added when creating the api config

be able to deploy different versions of the api.
would be able to define the different environments in config.json in the .goblet folder

goblet deploy --dev
goblet deploy --prod

Hello,

I am working on a simple use case : I have a cloud function that accepts a 'URl' parameter, calls that URl (which is a public API endpoint) and saves the data in a JSON file stored in Cloud Storage. I want to be able to set multiple cloud scheduler to call different endpoints on different schedules, but I only need 1 function.

Question : My guess is that @app.schedule('5 * * * *') creates both a Cloud Scheduler and a Cloud Function. Is there a possibility to only create a (or multiple) Cloud Scheduler and define parameters to use in the call to the Cloud Function ?

Add CLI command goblet logs that writes output logs from the corresponding cloudfunction

setup code coverage github checks now that test coverage is over 75%

https://docs.codecov.io/docs/quick-start

currently there is a some logic for deploying cloudfunctions in deployer, but it is also needed in some of the resource classes as well. might be good to move it all into one cloudfunction class.

functions that should be included

• zip
• upload
• create
• destroy
• patch
• check_diff

googleapiclient.errors.HttpError: HttpError 400 when requesting "Resource" has nested resources

for some reason there is some delay between deleting the config and then trying to delete the api so there is this error for nested resource.

you can currently just run the command again since the config resource is deleted in the first command. I will work on a fix going forward.

api gateway is now in ga version 1 as of jan 1 2021.

ner region is available us-east1

https://cloud.google.com/api-gateway/docs/release-notes

https://cloud.google.com/api-gateway/docs/reference/rest

We should not have to set env vars in order to deploy the app.

the problem is we import the app, which triggers the file to be run, causing issues.
https://github.com/anovis/goblet/blob/master/goblet/utils.py#L37

can try continue on exceptions, since we just want to get the goblet app itself.

But an issue is if the exception occurs before the app is initiated

Hello,

I'm asking to add retryConfig and attemptDeadline to @app.scheduler() as seen in HttpTarget.

Thank you in advance ! 🙏🏽

Add CLI command goblet metrics that writes metrics from the corresponding cloudfunction

Add api documentation on goblet functions and modules. Can use sphinx-autodoc

Will further help with #53

cli command to deploy the same functions in multiple regions

goblet deploy --regions region1,region2,region3 --project x

use api equivalent gcloud app describe --project PROJECT to get the default app engine region since this is the only region allowed in a project for cloud schedulers

error was
"Location must equal us-east1 because the App Engine app that is associated with this project is located in us-east1. Creating and editing resources in non-App Engine region is not allowed."

got AttributeError: 'NoneType' object has no attribute 'encode' in goblet package this is because GOOGLE_PROJECT env variable is not defined. a better error handling needs to occur. goblet package also shouldnt require this env

currently displayName is an optional field that is not being set so the gateway id is being created randomly.

https://cloud.google.com/api-gateway/docs/reference/rest/v1beta/projects.locations.gateways#Gateway

have a section for iam bindings in config.json

https://cloud.google.com/functions/docs/reference/rest/v1/projects.locations.functions/setIamPolicy

{
  "cloudfunction":{
      ....
   },
  "bindings": [
    {
      "role": "roles/resourcemanager.organizationAdmin",
      "members": [
        "user:[email protected]",
        "group:[email protected]",
        "domain:google.com",
        "serviceAccount:[email protected]"
      ]
    }
  ]
}

generate a boilerplate app with routes based on an openapispec file.

goblet generate -f openapispec.yml

/pets:
  get:
    description: Returns all pets from the system that the user has access to
    produces:
    - application/json
    responses:
      '200':
        description: A list of pets.
        schema:
          type: array
          items:
            $ref: '#/definitions/pet'

would yield in main.py

@app.route('/pets')
def pets() -> List[pet]
    raiseNotImplimented('/pets')

currently there is an error for the package and openapi commands if you do not use gcloud auth. also should have better error handling for those.

I noticed while checking the source code that pretty much nothing has a comment.
With some functions as they're quite big or do a lot of things it's definitely worth to add more comments.

currently support for requestbody is for openapispec 3, but it should be for openapi spec2, which is in parameters.

https://swagger.io/docs/specification/2-0/describing-request-body/

allow goblet to deploy a regular http cloudfunction without apigateway.

app = Goblet()

@app.http()
def http_endpoint(event):
    return "http_endpoint"

Add CORS support. Base off of chalice cors

create a goblet github action to deploy a goblet resources

similar to this chalice action

Hello,

Is it possible to define multiple cloud functions in the .goblet/config.json ?
I plan to create multiple cloud functions using goblet and I don't want to maintain them in separate folders/projects.
Thank you in advance.

Config parameters are usually stored in .ini or .txt files. Can we allow .ini and .txt files atleast?

https://github.com/anovis/goblet/blob/61a8ce2e4e1ff8d3967d79437573843f5c88c389/goblet/deploy.py#L157

Hello again ~

Is this expected from deployer? Earlier I didn't encounter this issue.

Trigger different functions based on header values similar to pubsub attributes. Allows for routing basic http requests without having to deploy pull apigateway

@app.http(headers={"X-GitHub-Event"})
def github(request):
return 

or

@app.http(headers={"X-GitHub-Event":"issues"})
def issues(request):
   return issues

@app.http(headers={"X-GitHub-Event":"pull_requests"})
def prs(request):
   return prs

would be nice to have a cli command to init all required pieces to deploy a function which would be an example main.py, requirements.txt with goblet-gcp incuded and a .goblet folder with a sample config.json

goblet init or goblet init NAME

Currently we set the filter on the function itself, but it would be better to set the filter on the subscription itself. Unfortunately, gcp only allows this by manually creating the subscription first and using a http function endpoint. we should have this as an option

https://stackoverflow.com/questions/66363652/how-to-set-pubsub-message-filter-while-creating-cloud-function-with-triggertopic

Make the argument to goblet local optional and set default to local

goblet local local -> goblet local