Comments (16)
Thanks for the reply Ryan. Since my note, we tried getting everything to build with 3.6 on the Amazon Linux image, but ran into some problems. Interestingly we found this (https://hub.docker.com/r/vivantshen/awspy3-lambda-zip/), which has all of our needed libraries (SciKit 0.18.1, Pandas, Numpy) compiled in 3.6 for Lambda, so we are just using those pre-built libraries.
from sklearn-build-lambda.
Just in case someone else shows up here, you no longer need this.
AWS introduced layers to overcome this issue: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/
You can find some examples here: https://towardsdatascience.com/hosting-your-ml-model-on-aws-lambdas-api-gateway-part-1-9052e6b63b25
from sklearn-build-lambda.
This is not true @Strateus all the uncompressed lambda code + layers still cannot exceed the 250MB uncompressed limit. There is however an option for running lambda as a docker image, not sure about cold start times: https://docs.aws.amazon.com/lambda/latest/dg/images-create.html
from sklearn-build-lambda.
I am also looking to run SciKit in Python 3.6, and also to upgrade to SciKit 0.18.1. Ryan, any chance you are still actively supporting this, and could build those Lambda-compatible libraries and just upload the zip's to this repo?
from sklearn-build-lambda.
I'm still maintaining this - right now I haven't had the need to bump to 3.6. It doesn't look like the Amazon Linux container image has 3.6 yet, which is odd. That means we'd either have to compile a Py3.6, or rely on universal wheels. Can either of you give the current artifacts a try on Python 3.6?
from sklearn-build-lambda.
@awenzel can you please upload the zip file to you github account and share the link here it would be very helpful.
@ryansb Any update to use this for python3.6?
from sklearn-build-lambda.
@bhangaboy For some 3.6 options, see #12 and #16.
from sklearn-build-lambda.
@thunderfish24 Thank you for the reference. I am having trouble running these shell scripts in docker container through my windows if you can help me out with that please.
from sklearn-build-lambda.
@thunderfish24 I managed to solve the error and the script started running but updating yum ends up in error. gcc64-gfortran-6.4.1-1.45.amzn1.x86_64.rpm and atlas these package are not able to get installed
from sklearn-build-lambda.
@thunderfish24 and @awenzel if any of you managed to compile sklearn on python3.6 please share the zip with me [email protected]
from sklearn-build-lambda.
Wouldn’t this work for you?
from sklearn-build-lambda.
@thunderfish24 Thanks this works for me
from sklearn-build-lambda.
This is a tremendous help! But... has anybody been successful adding pandas to this script and stripping it to be small enough for lambda?
from sklearn-build-lambda.
@teresagator did you tried running the shell script? if yes what is the size of the zip? If you haven't executed the script try changing the script with specifying installation of pandas then look at the zip.
from sklearn-build-lambda.
Lambda functions support python 3.6 now. Any chance for an update?
from sklearn-build-lambda.
Branch is open here - https://github.com/ryansb/sklearn-build-lambda/tree/python-3-test for porting to python 3. Feedback is welcome, see if it works for you & if not open an issue or PR.
from sklearn-build-lambda.
Related Issues (15)
- Strip failed: BFD: /home/ec2-user/sklearn_build/lib64/python2.7/site-packages/numpy/.libs/stV4bcuq: Not enough room for program headers, try linking with -N HOT 3
- $(pwd): HOT 2
- adding opencv 3.x to build? HOT 1
- windows
- /bin/bash: /outputs/build.sh: No such file or directory
- lib file HOT 1
- failure: repodata/filelists.sqlite.bz2 from amzn-main: [Errno 256] No more mirrors to try. HOT 1
- no such option: --use-wheel HOT 1
- OSError: libquadmath.so.0: cannot open shared object file: No such file or directory HOT 1
- Need to remove --use-wheel (this is deprecated) HOT 1
- Add to lambda-packages HOT 1
- removing test/doc files HOT 1
- how does this work HOT 6
- Yum not working HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sklearn-build-lambda.