How can I extend this to Python 3.6 ? about sklearn-build-lambda HOT 16 OPEN

Thanks for the reply Ryan. Since my note, we tried getting everything to build with 3.6 on the Amazon Linux image, but ran into some problems. Interestingly we found this (https://hub.docker.com/r/vivantshen/awspy3-lambda-zip/), which has all of our needed libraries (SciKit 0.18.1, Pandas, Numpy) compiled in 3.6 for Lambda, so we are just using those pre-built libraries.

from sklearn-build-lambda.

Just in case someone else shows up here, you no longer need this.
AWS introduced layers to overcome this issue: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/

You can find some examples here: https://towardsdatascience.com/hosting-your-ml-model-on-aws-lambdas-api-gateway-part-1-9052e6b63b25

from sklearn-build-lambda.

This is not true @Strateus all the uncompressed lambda code + layers still cannot exceed the 250MB uncompressed limit. There is however an option for running lambda as a docker image, not sure about cold start times: https://docs.aws.amazon.com/lambda/latest/dg/images-create.html

from sklearn-build-lambda.

I am also looking to run SciKit in Python 3.6, and also to upgrade to SciKit 0.18.1. Ryan, any chance you are still actively supporting this, and could build those Lambda-compatible libraries and just upload the zip's to this repo?

from sklearn-build-lambda.

I'm still maintaining this - right now I haven't had the need to bump to 3.6. It doesn't look like the Amazon Linux container image has 3.6 yet, which is odd. That means we'd either have to compile a Py3.6, or rely on universal wheels. Can either of you give the current artifacts a try on Python 3.6?

from sklearn-build-lambda.

@awenzel can you please upload the zip file to you github account and share the link here it would be very helpful.
@ryansb Any update to use this for python3.6?

from sklearn-build-lambda.

@bhangaboy For some 3.6 options, see #12 and #16.

from sklearn-build-lambda.

@thunderfish24 Thank you for the reference. I am having trouble running these shell scripts in docker container through my windows if you can help me out with that please.

from sklearn-build-lambda.

@thunderfish24 I managed to solve the error and the script started running but updating yum ends up in error. gcc64-gfortran-6.4.1-1.45.amzn1.x86_64.rpm and atlas these package are not able to get installed

from sklearn-build-lambda.

@thunderfish24 and @awenzel if any of you managed to compile sklearn on python3.6 please share the zip with me [email protected]

from sklearn-build-lambda.

Wouldn’t this work for you?

https://github.com/thunderfish24/sklearn-build-lambda/blob/75c713d23107300370b16b134936b959f1f0f73b/lambda-package.zip

from sklearn-build-lambda.

@thunderfish24 Thanks this works for me

from sklearn-build-lambda.

This is a tremendous help! But... has anybody been successful adding pandas to this script and stripping it to be small enough for lambda?

from sklearn-build-lambda.

@teresagator did you tried running the shell script? if yes what is the size of the zip? If you haven't executed the script try changing the script with specifying installation of pandas then look at the zip.

from sklearn-build-lambda.

Lambda functions support python 3.6 now. Any chance for an update?

from sklearn-build-lambda.

Branch is open here - https://github.com/ryansb/sklearn-build-lambda/tree/python-3-test for porting to python 3. Feedback is welcome, see if it works for you & if not open an issue or PR.

from sklearn-build-lambda.