Git Product home page Git Product logo

fission / environments Goto Github PK

View Code? Open in Web Editor NEW
27.0 27.0 47.0 1.48 MB

A set of language environments for Fission

Home Page: https://fission.io/environments/

License: Apache License 2.0

Dockerfile 4.84% Shell 15.21% Go 9.64% C# 34.02% Roff 0.30% Java 5.97% JavaScript 10.34% Perl 0.75% PHP 2.44% Python 7.80% Ruby 2.96% HTML 1.75% Makefile 3.83% CSS 0.16%
environments fission functions hacktoberfest lambda-functions

environments's Introduction


Fission: Serverless Functions for Kubernetes

Fission Licence Fission Releases go.dev reference Go Report Card Fission contributors Commit Activity
Fission website Fission slack Fission twitter GitHub Repo stars


Fission is a fast serverless framework for Kubernetes with a focus on developer productivity and high performance.

Fission operates on just the code: Docker and Kubernetes are abstracted away under normal operation, though you can use both to extend Fission if you want to.

Fission is extensible to any language; the core is written in Go, and language-specific parts are isolated in something called environments (more below). Fission currently supports NodeJS, Python, Ruby, Go, PHP, Bash, and any Linux executable, with more languages coming soon.

Table of Contents

Performance: 100msec cold start

Fission maintains a pool of "warm" containers that each contain a small dynamic loader. When a function is first called, i.e. "cold-started", a running container is chosen and the function is loaded. This pool is what makes Fission fast: cold-start latencies are typically about 100msec.

Kubernetes is the right place for Serverless

We're built on Kubernetes because we think any non-trivial app will use a combination of serverless functions and more conventional microservices, and Kubernetes is a great framework to bring these together seamlessly.

Building on Kubernetes also means that anything you do for operations on your Kubernetes cluster — such as monitoring or log aggregation — also helps with ops on your Fission deployment.

Getting Started

  # Add the stock NodeJS env to your Fission deployment
  $ fission env create --name nodejs --image fission/node-env

  # Create a function with a javascript one-liner that prints "hello world"
  $ fission function create --name hello --env nodejs --code https://raw.githubusercontent.com/fission/examples/master/nodejs/hello.js

  # Run the function.  This takes about 100msec the first time.
  $ fission function test --name hello
  Hello, world!

Learn More

Contributing

Check out the contributing guide.

Who is using Fission?

Sponsors

The following companies, organizations, and individuals support Fission's ongoing maintenance and development. If you are using/contributing to Fission, we would be happy to list you here, please raise a Pull request.

InfraCloud Srcmesh

License

Fission is licensed under the Apache License 2.0 - see the LICENSE file for details

environments's People

Contributors

agiwalpooja20-zz avatar albertolopezbenito avatar blackfly19 avatar brendanstennett avatar dependabot[bot] avatar dgoujard avatar erwinvaneyk avatar fishman avatar garyyeap avatar girishg4t avatar joalmeid avatar ktrance avatar life1347 avatar littlefox94 avatar nouney avatar pindar avatar rapitable avatar realharshthakur avatar robertherhold avatar sahil-lakhwani avatar sanketsudake avatar sbfaulkner avatar smruthi2187 avatar snyk-bot avatar soamvasani avatar suramya-shah avatar therahulbhati avatar tosone avatar vishal-biyani avatar xiekeyang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

environments's Issues

Sync up python/gcc versions in official docker images fission/python-builder fission/python-env

Fission/Kubernetes version
Fission version 1.11.2

$ fission version
client:
  fission/core:
    BuildDate: "2020-10-20T14:43:45Z"
    GitCommit: 52508e19343ef93da4ce8960908c5569f44da225
    Version: 1.11.2
server:
  fission/core:
    BuildDate: "2020-10-20T14:43:45Z"
    GitCommit: 52508e19343ef93da4ce8960908c5569f44da225
    Version: 1.11.2
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"e19964183377d0ec2052d1f1fa930c4d7575bd50", GitTreeState:"clean", BuildDate:"2020-08-26T14:30:33Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:32:58Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes platform (e.g. Google Kubernetes Engine)
Bare metal

Describe the bug

fission/python-builder is built off of alpine:3.10, while fission/python-env is built from alpine:latest.

A simple fix would be to set the version of alpine that fission/python-env is built and make sure they're sync'd up, or multiple python 3 version tags supported, going forward.

When installing some python modules(numpy), this causes incompatability between the package and the environment.

$ docker run -it --entrypoint python3 fission/python-builder -c "import sys; print(sys.version);"
3.7.7 (default, Jul 19 2020, 03:57:54)
[GCC 8.3.0]

$ docker run -it --entrypoint python3 fission/python-env -c "import sys; print(sys.version);"
3.8.5 (default, Jul 20 2020, 23:11:29)
[GCC 9.3.0]

To Reproduce

See above or install and use numpy or pandas in a python package by using requirements.txt/build.sh in the package as per documentation.

Expected result
numpy works as expected

Actual result
importing numpy throws the following error

"Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
    https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
  * The Python version is: Python3.8 from \"/usr/bin/python3\"
  * The NumPy version is: \"1.19.4\"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
"

Official support for non-alpine environments

PROBLEM:
Fission uses Alpine images as the base for all of its function containers.
This is a good choice for most functions, but when dealing with more esoteric use cases, Alpine has too many restrictions built in

CONTEXT:
We created a Fission function in our project to check TLS certificates.
We had lots of problems installing the adequate dependencies in Python and Go environments.

We tried building custom images with Debian buster, but they seem to conflict with Fission builder image.

SOLUTION:
Create Fission images for every language environment with a "bulkier" docker distribution (Debian buster comes to mind)

[node-env] body parser for text/plain

The NodeJs Express framework attempts to parse the body of any incoming request to JSON. So in case of a missing or unsupported content-type such as text/plain, Express will not be able to convert the body to JSON. It seems to basically ignore body.

To reproduce, take any NodeJs Fission function that uses the body of the request, and force the content-type of the request to be text/plain: curl -d 'Hello world!' -H 'Content-Type: text/plain' $FISSION_ROUTER/fission-function/<fn>. The result is that the request.body is an empty object.

It seems that modifying the express environment's bodyparser could resolve this (but I haven't looked into it)

Misleading .net name

dotnet 2.0, for us, "dotneters", is .net 2.0 (from 2005).

For clarity, please, use the term .net standard 2 (which is different from .net core 2.0 and different from .net 2.0).

Great project, BTW.

Better environment management: environment repository, auto-update

Originally, environments were simply a Docker image, but over time, environments have gotten a little more complex. Some of the complexity is from "static" properties, like the builder image, port numbers, etc.; then there are runtime settings like poolSize; and also deployment-specific properties, like the namespace to run functions in, the security context, other configuration, etc.

There ought to be a way to simplify the management of environments within a Fission setup. Things we could do:

  • Create some repository of environments. This would include the images and any other properties that are "static" and apply across ~all deployments.
  • Enable people to discover and add environments from the repository into their cluster. Maybe something like fission env import etc.
  • Check imported environments for security updates regulary, and alert the user or even auto-update the environment
  • Allow specification of runtime properties (poolsize, cpu requests/limits etc) in conjunction with the properties of imported environment.

Binary environment does not process large POST bodies

Fission/Kubernetes version

$ fission version
client:
  fission/core:
    BuildDate: "2021-06-21T10:38:48Z"
    GitCommit: d06ad04341377bec6b8c905a440956b7d659d8fb
    Version: 1.13.1
server:
  fission/core:
    BuildDate: "2021-06-21T09:13:11Z"
    GitCommit: 401cbd597b63f5646e3e4be7df38870fdd617485
    Version: 1.13.1

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.3", GitCommit:"ca643a4d1f7bfe34773c74f79527be4afd95bf39", GitTreeState:"clean", BuildDate:"2021-07-15T21:04:39Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.8", GitCommit:"5575935422cc1cf5169dfc8847cb587aa47bac5a", GitTreeState:"clean", BuildDate:"2021-06-16T12:53:07Z", GoVersion:"go1.15.13", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes platform (e.g. Google Kubernetes Engine)
OpenStack + Rancher

Describe the bug

Sending Post request to fission functions in a binary environment with bodies above a certain size causes no response to be returned.

To Reproduce

Follow the example guide for binary environments to deploy hello.sh https://github.com/fission/environments/tree/master/binary/examples
hello.sh:

#!/bin/sh

echo "Hello World!"

Steps:

fission env create --name binary-env --image fission/binary-env

fission function create --name hello --env binary-env --code hello.sh

fission route create --method GET --url /hello --function hello

curl http://$FISSION_ROUTER/hello

returns Hello World!

Now change the route to POST.

# Might need to run 'fission route list' to get the route name
fission route update --function hello --method POST --url /hello --name $ROUTE_NAME

Running

curl http://$FISSION_ROUTER/hello -XPOST
# or
curl http://$FISSION_ROUTER/hello -XPOST -d "Test"

both return Hello World!
So far everything works as expected.
The hello.sh script makes no use of the POST body so it should not matter what is sent.
However when running

curl http://$FISSION_ROUTER/hello --data-binary @PATH_TO_SOME_FILE

Where the file is above a certain size, curl will timeout and return: (52) Empty reply from server

I tested with a 49 KB file which works and a 72 KB file which does not work.

Expected result

I expect fission binary environment to process POST bodies up to at least 2 MB.

In the above described case it should return Hello World! for the 72 KB file.

Actual result
Curl times out and prints:
cur: (52) Empty reply from server

Screenshots/Dump file

$ fission support dump

If needed I can attach this after removing sensitive information.

Additional context

Checking function logs with

fission function log --name hello

before and after invoking the function with a 72KB POST body shows new log entries so it reaches the function but somewhere on its way back the Hello World! response is lost.

Rust environment support

Is your feature request related to a problem? Please describe.
We would like to enabled Rust language as one of the fission environments.

Describe the solution you'd like
As a Fission user I should be able to run/deploy functions written using Rust language.

Describe alternatives you've considered

Additional context

Health Check for JVM issues

Currently, the JVM environment health checks do not indicate the real health of JVM. We need to modify the health check so that actual issues such as OutOfMemoryExceptions etc. cause the health checks to fail and Kubernetes to restart the pods.

Python: Provide more image flavours

Is your feature request related to a problem? Please describe.

Trying to install albumentations results on an installation error on scipy (see https://stackoverflow.com/questions/59968493/scipy-error-in-python3-8-alpine3-11-no-lapack-blas-resources-found)

Describe the solution you'd like

More image flavours, such as python-env-3.10-buster, based on python:3.10-buster. These are bigger images but allow installation of more complex dependencies

Describe alternatives you've considered

Provide documentation on how to create/extend environments and builders (see #275 (comment)).

Additional context

arm (not 64) support

While it's great to see linux/arm support in Fission itself (per fission/fission#1421), I note at the moment, though, would be arm (not arm64) users find their Fission install coming to an abrupt stop when they realize that not one of the environment or builder containers are available in linux/arm.

Could we get linux/arm added to the official environment images? Seems like it should just need linux/arm added to the rules.mk in the appropriate place; as I recall, all the groundwork for multiarch has already been done. Did a bit of it myself.

Environments repo CI-CD

The currently merged CI-CD work in #22 needs a few more enhancements:

  • Add missing environments to the workflow & Skaffold so that they are picked up
  • Run integration test from Fission for a given environment.
  • Create a release workflow for each environment - which can build and push images when a new PR is merged

binary-env and compiled binaries / example hello.go

Fission/Kubernetes version

$ fission --version
client:
  fission/core:
    BuildDate: "2021-11-23T09:29:28Z"
    GitCommit: 5349a591
    Version: v1.15.0
server:
  fission/core:
    BuildDate: "2021-11-23T09:29:28Z"
    GitCommit: 5349a591
    Version: v1.15.0

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.3", GitCommit:"ca643a4d1f7bfe34773c74f79527be4afd95bf39", GitTreeState:"clean", BuildDate:"2021-07-15T21:04:39Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.7-3+7700880a5c71e2", GitCommit:"7700880a5c71e25c44491ef5c7d7fb30527d8337", GitTreeState:"clean", BuildDate:"2021-11-17T22:02:47Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes platform (microk8s)

Describe the bug
I tried the binary-env hello.go example from binary hello.go example. As is it described there, before I created the binary environment with the comand fission env create --name binary-env --image fission/binary-env taken from requirements binary hello.go example. With this command automatically the image tagged with latest will then used.

When I test the function with fission fn test --name hello-go you can find in the logs the message

poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 2021/12/07 15:21:00 Starting binary function execution
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 2021/12/07 15:21:00 stderr: /userfunc/user: line 1: �ELF�: not found
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env /userfunc/user: line 2: syntax error: unexpected "("
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 2021/12/07 15:21:00 Error: failed to wait for subprocess: exit status 2

The latest tagged binary-env image is 1.32.1. Then I test older versions of the binary-env images. What I found out is
the the version 1.11.2 is working as expected and the latest one and print out Hello World! and the env-variables. All versions later than 1.11.2 produce the error above.

To Reproduce

Follow the example linked above, try different binary-env versions.

Expected result
cli:

fission fn test --name hello-go
Hello World!
Environment: [REQUEST_METHOD=GET REQUEST_URI=/ CONTENT_LENGTH=0 HTTP_X-FISSION-FUNCTION-RESOURCEVERSION=29214 HTTP_CONNECTION=close HTTP_ACCEPT-ENCODING=gzip HTTP_X-FISSION-FULL-URL=/fission-function/hello-go HTTP_X-FISSION-FUNCTION-NAMESPACE=default HTTP_X-FORWARDED-FOR=127.0.0.1 HTTP_X-FORWARDED-HOST=127.0.0.1:38277 HTTP_TRACEPARENT=00-bf5756fbca878321c8cd8bd84797f8b1-d3e80979f713d847-01 HTTP_X-FISSION-FUNCTION-NAME=hello-go HTTP_X-FISSION-FUNCTION-UID=bb81ed9c-14b9-4990-a5ea-562e8d981ab6 HTTP_USER-AGENT=Go-http-client/1.1 HTTP_FORWARDED=host=127.0.0.1:38277;]%        

Actual result
logs:

poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 2021/12/07 15:21:00 Starting binary function execution
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 2021/12/07 15:21:00 stderr: /userfunc/user: line 1: �ELF�: not found
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env /userfunc/user: line 2: syntax error: unexpected "("
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 
poolmgr-binary-env-default-25878-54f65bfbbb-q6nqk binary-env 2021/12/07 15:21:00 Error: failed to wait for subprocess: exit status 2

cli:

 fission fn test --name hello-go
Error: Error calling function hello-go: 500; Please try again or fix the error: failed to wait for subprocess: exit status 2
Error: Error getting function logs from controller: error getting logs from controller, status code: '500'. Try to get logs from log database.
Error querying logs: Internal error - 502 Bad Gateway
Options:
....

Python environment: entrypoint fails when module exists with conflicting name

To reproduce:

  • use a package in the requirements.txt (e.g. redis)
  • Name one of the files in the user function directory also redis
  • When creating the function with an entrypoint redis.main the imported redis module will be used and crashes the function execution.

Although this can be considered a user error, it is very opaque. This might be an annoying issue especially in larger code bases with many module dependencies. I am not sure if we want and how we can fix this issue.

Currently the workaround is obviously to avoid naming conflicts when naming the user files that can have entrypoints.

F# support

I see that it is already C# support, but it would be great to extend it to a decent language like F# on the dotnet platform :)

JVM-11 Environment mis configurations

Describe the bug
When booting up there are several problems with the docker file build as per the master branch. The two main issues are

  • server.portnumber parameter isn't being passed properly. This was copied from the non-jersey file so just needs to be fixed
  • There's a missing dependency in the POM as this was removed from the more recent jdk-11

To Reproduce

Look at the deploy logs. Sorry but I've solved it now locally so can confirm that the dependency being added fixes the POD build.

Merge request with update is pending and I'll post it to this issue.

Custom response code for binary environment

I read through the binary environment source and I found out that a function cannot return anything other than 200 and 500. Can you provide support for custom return codes, for example by setting an ENV variable?

Uniform testing of environments

Currently environments differ in the exact interface they implement. For example, some environments return a 500 vs. a 400 for an already specialized environment. There are probably other, more subtle differences that we don't know about yet, such as the way headers and query params are handled.

There should be a uniform method for verifying that environments correspond to a specific environment specification, as any variation in the environments can be a root for hard to debug problems, which would require intricate knowledge of the environment implementation details.

Specification should describe endpoints, HTTP methods, errors/codes, how to handle params (body, headers, query), ect. It might be an idea to write a swagger spec as a way to document this specification (although I do not know exactly to what detail Swagger describes the HTTP specification)

As a bonus, with the uniform testing tooling, we could include some simple, automated benchmarking of the environments, to provide some insight in how the environments compare for a similar user-function.

Tl;dr needed:

  • Write down the, currently implicit, environment spec
  • Document environment spec in a detailed, non-ambiguous way (e.g. using swagger)
  • Find or build tooling to test the environments in a language-agnostic way (an approach could be to test against the Docker containers)
  • Create a uniform test for all languages (e.g. using a simple test function that uses all the main features of the spec such as reading the body, headers and query params).

fyi @life1347 @soamvasani

Parent POM/Archetype for JVM environment

The current model of copying the POM file of the example is good but we need to improve the experience by having an archetype or parent POM for Fission Java example.

Create New Environment - Python/FastApi

Hi Team,

The python env uses Flask, but after reading about FastApi it suites better our needs.

But in order to use it, we need to create a new env. I've already read some environments code, python should to be complete, but perl seams to be just the bare minimum to work.

There is any documentation how to create a new env? What must and should be implemented?

Cheers,
Flip

WASM functions

Fission should add wasm as an environment runtime. This will allow multiple unrelated functions to be loaded into the same container. This will reduce cold-start times by a couple orders of magnitude. It will also allow each function invocation to be isolated from all other function invocations (a property you don't get today because we have to run multiple invocations of the same function within the same container).

Fastly open sourced lucet, an open source project compiler + runtime. This should give us pretty much everything we need, the remaining work is to write a thing that glues the compiler and runtime into our environment interface.

The compilation would go: {rust,c,AssemblyScript} -> wasm -> loadable binary. The loadable binary would get loaded and run by the lucent runtime.

dockerignore for examples and tests

From within the environment directories, we need to ignore examples and test/s directory so that they don't become part of the environment image.

Java later versions support and images

Currently, the Java version tested and officially supported in Fission is Java8. It might be good to test and publish images for the latest versions of Java - 11, 12, 13, etc.

JVM & Java Environment Re-Design and support for latest JDK

Many Fission users are using JVM/Java environment but the JDK/JRE version currently in use is quite old. There are other shortcomings also listed below which need a redesign of the JVM environment with the latest versions of JDK and other features. This issue is to list current limitations, future expectations, and to document future design before implementation.

Related Issues:

  1. Currently, the Fission library is in snapshot repo and should be in release version for broader usage: fission/fission#1579

  2. Health check for JVM specifically are not very useful as of now: https://github.com/fission/fission/issues/926

  3. Because of Spring and other dependencies, the JVM is a bit heavier than it should be: https://github.com/fission/fission/issues/832

  4. There were issues parsing XML: fission/fission#1505

  5. An easier way to use Fission POM: https://github.com/fission/fission/issues/833

Future environment expectation

  1. Newer versions of JDK/JRE supported: https://github.com/fission/fission/issues/1485

OOM killed environment should reload function after restart

Describe the bug

Parent Bug fission/fission#1874

If a pod is OOMKilled and restarted, the requests going to that pod typically do not work and start throwing 502. This was observed in a customer's environment with a custom python env which was using Debian image. It is not clear if this is a fetcher or env or executor issue but needs to be investigated and fixed.

To Reproduce

fission/fission#1874 (comment)

Reproducing issue with pool manager:

  1. Create kind k8s cluster & deploy fission
  2. Create env
fission env create --name python --image fission/python-env --mincpu 40 --maxcpu 80   --minmemory 64 --maxmemory 96  --poolsize 1 --version 3
  1. Create code file
$ cat code.py
def main():
    a = []
    while True:
        a.append(' ' * 10**6) #This line will create a OOM in seconds
    return "Done"
  1. Create fn
$ fission fn create --name oom-fn --env python --code code.py 
$ fission env pods --name oom-fn
  1. On making request we can see function pod is restarted, where as no restart on fetcher pod.
fetcher/0.log
python/0.log
python/1.log

From a fission perspective, the fn runtime server(python container) would be assumed as specialized.
But from when the fn runtime server restarted, it's not specialized. When a request is made to the env server we would get an error.

I am planning to try out adding a signal handler in the python server to check we can detect exit signals before OOM.

Additional context

fission/fission#1874 (comment)

What happens when container within pods get killed due to OOM ?

  • Container is given signal SIGKILL and container is forced to exit.
  • Container is stopped immediately and restarted to get rid consumed memory.
  • Any process typically can capture signals like SIGTERM or SIGINT, and act on it for cleanup. But in case of SIGKILL we can't really capture it.
  • Unless we are using something like proper monitoring or preoomkiller-controller, we cant identify oom situation in advance.
  • In case of fission function pod, typically we have two containers function runtime container and fetcher container. Typically function runtime container would hit OOM so it gets restarted.

Why we get 502 after function runtime container restart ?

  • Before container gets killed due to OOM, it is speciallized container generally, i.e, function is loaded into container and it is serving requests.
  • Fission currently doesnt have mechanism to continuously scan and identify if pod got restarted due OOM.
  • Restart window is very small so Fission doesnt even come to know if pod is restarted.
  • Once pod is specialized fission assume pod is able to serve function until gets cleaned up/deleted.
  • After container restart, Fission assumes pod is specialized and keeps throwing incoming requests at it. Since function is not loaded after container restart, its generic container and request fails.

Possible approaches

  • a. Identify OOM situation in advance and fix it by reducing memory usage of the pod. Or use something like veritical pod autoscaler to increase limits for the particular pod.
  • b. After restart, identify pod restarted due to OOM and get it out of the service.
  • c. After restart, restore state in function pod with specialization and make it ready to serve the function.

a. seems to be more effort on the user side. Where b. would be lot additional scanning on Fission side on continuous basis just to identify OOM scenario.
Both a. b. try to solve build solution at Fission level where as c. is something could be implemented at environment level.

c. seems easier to get quickly working in the environment.
I did a quick POC for this and after restart of function runtime was respecialized.

Please check PR for more details.
#213

The same fix needs to be implemented across environments.

  • python | #213
  • go |
  • nodejs |
  • java |

Update node version

Is your feature request related to a problem? Please describe.
The current default NodeJS version is set to v8.17.0. This version is way outdated. The support for this version stopped over a year ago (2019-12-31).

Describe the solution you'd like
It would make sense to update the default version in fission/node-env to 14. This is the stable, recommended, LTS version of node. Additionally we could also offer v12 (Maintenance LTS) and v16 (current version)

Describe alternatives you've considered
I know it's possible to create your own image. I do believe that the default version should at least be supported and safe though. :)

Additional context
https://nodejs.org/en/about/releases/

Creating multiple python versions

**Describe something that is unclear or not documented and needs a documentation update
Documentation on how to setup multiple python version, especially for newer versions such as python.

Node.js: runtime image dependencies available to functions?

https://docs.fission.io/docs/languages/nodejs/#modifying-the-nodejs-runtime-images says that dependencies added in the runtime image package.json will be available to the function. But in my testing they are not. Confirming that I docker built and pushed with the updated package.json and updated my fission environment to use the custom runtime image. I'm using node:12 as my base image.

I was able to get it to work with npm install --global, but then you also have to set ENV NODE_PATH /usr/local/lib/node_modules, since apparently globally installed packages are not available as modules by default (only cli).

see also https://fissionio.slack.com/archives/C3LUX6BBP/p1611941610031500 and preceding message outside the threads

client:
  fission/core:
    BuildDate: "2020-10-20T14:43:45Z"
    GitCommit: 52508e19343ef93da4ce8960908c5569f44da225
    Version: 1.11.2
server:
  fission/core:
    BuildDate: "2020-10-20T14:43:45Z"
    GitCommit: 52508e19343ef93da4ce8960908c5569f44da225
    Version: 1.11.2

Optimisations in JVM environment

The current JVM environment has some room for optimisation such as excluding some dependencies, pre-fetching certain dependencies and improving the cold start etc.

node-builder needs ability to add npm token via env

Is your feature request related to a problem? Please describe.

Currently node-builder is not able to install private npm packages.

Describe the solution you'd like

The builder should take two environment variables. The registry and the authtoken.

npm set //<registry>/:_authToken $TOKEN

Describe alternatives you've considered

It's currently unclear to me whether the example above covers all use cases, but it certainly should cover npmjs.net and github package repositories:

https://stackoverflow.com/questions/23460980/how-to-set-npm-credentials-using-npm-login-without-reading-from-stdin

Implement v2 interface for environment & environment docs

Currently, fission environment has two interfaces: v1, v2

  • v1

    • Support to load a function from one single file.
  • v2

    • Support to load the function from one file or from the directory
    • Support --entrypoint to specify which function need to be loaded in.
    • Support pkg builder to build pkg from a source package

Following is the interface implementation status

Environment v1 v2
binary v v
go v v
jvm (language run on jvm) v v
python v v
nodejs v v
dotnet20 v v
php7 v v
ruby v v
perl v x
dotnet v x

To support v2 interface, the environment needs to implement:

  • API exposed in url /v2/specialize with following request json body.

https://github.com/fission/fission/blob/ec495fbd1d5b10c4f04bab63a61f62d07c34c3e5/types.go#L72-L90

  • Environment package builder

The job for environment builder is to build deploy package from a source package. To implement it, all you need is one Dockerfile that contains necessary build environment and one default build script.

There are two build environment variable will be accessible during the build process

  1. $SRC_PKG: The path to the source package.
  2. $DEPLOY_PKG : The path to store final artifact (a.k.a deploy package)

For more information of default build script and builder dockerfile, please refer:

JVM environment function interface

It would nice if Function interface can have generics automatically inferred. This helps with spring automatic conversion of I as input object and O as an output object.

import io.fission.Context;

public interface Function<I, O> {

    org.springframework.http.ResponseEntity<O> call(org.springframework.http.RequestEntity<I> req, Context context);

php-env only answers one request

Fission/Kubernetes version

$ fission version
client:
  fission/core:
    BuildDate: "2021-08-03T11:34:27Z"
    GitCommit: 9b5eb069d69d47d22ecc0270bf8619857ea3f19f
    Version: 1.14.1
server:
  fission/core:
    BuildDate: "2021-08-03T09:53:20Z"
    GitCommit: 9b5eb069d69d47d22ecc0270bf8619857ea3f19f
    Version: 1.14.1

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.3+k3s1", GitCommit:"1d1f220fbee9cdeb5416b76b707dde8c231121f2", GitTreeState:"clean", BuildDate:"2021-07-22T20:52:14Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes platform (e.g. Google Kubernetes Engine)
k3s
Describe the bug

When I create a PHP environment like so:

$ fission env create --name php73 --builder "fission/php-builder" --version=2 --image "fission/php-env"

and deploy a basic function to it:

$ cat hello.php
<?php
echo "Hello from PHP";
$logger->warning("Hello logger");

$ fission fn create --name "phpnew" --code hello.php --method GET --env php73 

and then try to run it multiple times:

  $ for i in {1..100}
> do
> fission fn test --name phpnew
> sleep 1 && echo $i
> done

You can see in the output that the function does not get called cleanly everytime and often does not respond with the "Hello from PHP" as expected:

Hello from PHP1
2
3
4
5
6
7
8
9
10
11
Hello from PHP12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Hello from PHP32
33
^C

To Reproduce

Install fission via Helm (without Prometheus) as provided in the Official Documentation. Then do above steps.

Expected result

I want to make sure that every function call gets answered in a proper way. Thus, every call from above should be responded with "Hello from PHP".

Actual result

Only sporadically, the function gets called and I get a correct response.

Screenshots/Dump file

If needed, please tell me if you need a dump for this issue and what exact files are needed. I need to make sure that sensitive company informations are omitted.

Additional context

I noticed that everytime I get a correct response Hello from PHP, a new poolmgr Pod is being created. Maybe it's an issue that the PHP environment never closes the request properly and is only capable of answering exactly one request?

Edit: I've also tried it with your standard python-env and was not able to reproduce this issue there. It must have got to do something with that specific container image.

Where's the nodejs v12 env and builder image?

I'm looking at fission docker-hub and I see the latest version has node v8:
https://hub.docker.com/r/fission/node-env/tags?page=1&ordering=last_updated

I found that there's a close PR that updated the node-env + node-build to node v12:
fission/fission#1620

it's also said on milestone v1.11, but I don't see it anywhere (also in the master branch there's no environment folder anymore?)
Can I access the nodejs v12 images? or do I need to create my own?

Thanks!

ES module support for Node JS

Standards compliant way to load module is ES Modules not CommonJS in Node.

E.g. ESM Declaration

export default async function MyFunc(context) {
  return {
      status: 200,
      body: "hello, world!\n"
  };
}

ESM module use

import {MyFunc} from "my-module.mjs";

vs

CommonJS declaraion

module.exports = async function(context) {
    return {
        status: 200,
        body: "hello, world!\n"
    };
}

CommonJS use

const myModule = require("my-modules.cjs);

Is there a way to make NODE's ESM's imports / module declarations work in Fission?

Tried it locally and it did not work. Could not find any mentions/examples in your repos either

ESM is the recommended way to build Node JS code these days

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import

Docs on how to create new environments

I'm looking for documentation on how to deploy new languages to fission. I saw on the main readme that it is possible to extend/create environments but nothing further on how to go about it. Do any docs exist?

binary-env docker build is broken

On Docker-Desktop MacOSX Big Sur

If compiling from https://github.com/fission/environments/tree/master/binary, the runtime server errors with

# curl http://localhost:80/test-endpoint
Function error log: exit status 127
Function out log:

Steps to reproduce

# from fission/environments root
cd  binary
docker build -t test/binary-env .

mkdir -p /tmp/test-endpoint /tmp/test-endpoint/ls
cd /tmp/test-endpoint

cat <<EOT >> ls/entrypoint.sh
#!/bin/sh

ls
EOT

chmod +x ls/entrypoint.sh

fission spec init

fission env create --spec --name test-binary-env --image test/binary-env --builder fission/binary-builder

fission function create --spec --name test-endpoint --env test-binary-env --src "ls/*" --entrypoint entrypoint.sh

fission route create --spec --method GET --url /test-endpoint --function test-endpoint

fission spec validate

fission spec apply --wait

curl http://localhost:80/test-endpoint

I was able to resolve the error if I grab the binary-env base image from dockerhub and do a multistage build. Is the go server / dockerfile in the binary directory outdated?

FROM fission/binary-env

FROM alpine:3.5

WORKDIR /app

RUN apk update && apk add mysql mysql-client docker

COPY --from=0 /app/server /app/server

EXPOSE 8888
ENTRYPOINT ["./server"]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.