Comments (5)
Thank you for reporting the error @SSJUSA. While UMAP claims to be pretty memory efficient (see their benchmark), it can still get pretty memory hungry!
To get the notebook running, I suggest switching to a faster embedding method. Here, TruncatedSVD should do the trick.
To use that, replace cell 4 with:
from sklearn.decomposition import TruncatedSVD
umap_embedder = TruncatedSVD()
umap_bow = umap_embedder.fit_transform(bag_of_words)
You will likely also need to change cell 8 to use TruncatedSVD
by replacing it with:
umap_embedder = TruncatedSVD()
umap_emb = umap_embedder.fit_transform(embeddings)
If that does not fix your issue, you can instead reduce the dataset size by selecting the first 2000 rows for example. To do so, add a new line in the first cell following the format_raw_df
call:
df = df.head(2000)
from ml-powered-applications.
from ml-powered-applications.
The error ValueError: Found array with 0 feature(s) (shape=(5676, 0)) while a minimum of 2 is required.
makes it seem like the embeddings
variable has rows of size 0. I think this may be a different issue than the previous one.
If you run embeddings.shape
in a notebook cell right before cell 8 should be able to see whether they are of the correct size. On the full dataset, the results should be (5676, 300)
. If you've run df = df.head(2000)
it should be (265, 300)
.
If they are not, try running train_author[train_author["is_question"]]["body_text"].shape
which should be the same number of rows (but only one dimension). So for the full dataset you should see (5676,)
and (265,)
for the partial one.
Finally, you may want to take another look at the spaCy docs and reinstall the large model by running python -m spacy download en_core_web_lg
from the command line.
from ml-powered-applications.
from ml-powered-applications.
This is resolved quickly by the author @hundredblocks . Thanks.
This is closed.
from ml-powered-applications.
Related Issues (10)
- /v1 does not work out of the box HOT 1
- ImportError: cannot import name 'joblib' from 'sklearn.externals', ModuleNotFoundError: No module named 'sklearn.externals.joblib' HOT 2
- ModuleNotFoundError: No module named 'sklearn.externals.joblib' on joblib.load HOT 1
- Can't run pip install -r requirements.txt HOT 5
- can't run FLASK_APP=app.py flask run HOT 1
- Pandas 0.24.2 not compatible with new versions of python HOT 1
- Issue with install requirements.txt HOT 5
- Licence? HOT 1
- Not an issue, But nice feature to have HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ml-powered-applications.