Comments (10)
Hello, very strange.
I re-tested the library on:
- my Mac (Catalina 10.15.2) with Boost-1.72.0 and Clang 11
- a Linux machine (Ubuntu 19.10) with Boost-1.67.0 and gcc 9.2.1
and everything works just fine...
from tongrams.
Could you please test it in a docker container?
I suspect that it is happening because of lesser RAM.
from tongrams.
It runs smoothly on my Mac too.
I also tried to run it on an AWS instance, getting the same runtime error there also.
from tongrams.
And what are the specifications of that AWS instance?
from tongrams.
I used an AWS Marketplace AMI Ubuntu 20.04.
I tried it on three instances m5ad.4xlarge
, m5ad.2xlarge
and m5.4xlarge
.
These instances have around 8 GB to 16 GB of RAM. Please Refer to this page for details.
from tongrams.
If you give me access to one of those systems, I can investigate by myself.
But as long as there is not a way to reproduce the error, there is nothing I can do right now.
from tongrams.
I cannot post the credentials publicly here.
Shall I send an email containing the private key and IP Address?
Please accept my connection request on linkedin https://www.linkedin.com/in/razdeeproychowdhury/
from tongrams.
Sure, thanks.
from tongrams.
Please check your email inbox
from tongrams.
The problem is that
sysconf(_SC_PAGESIZE) * sysconf(_SC_PHYS_PAGES)
returns the physical memory of the machine, but the ram that is left available
for the system is often less.
So doing
grams_counts_pool unigrams_pool(available_ram * 0.8);
(which allocates a pool of 80% of the available ram) solves the problem.
from tongrams.
Related Issues (18)
- Using Tongrams HOT 6
- Can't compile tongrams HOT 3
- format for vocabulary file HOT 1
- an error when I try python tongrams HOT 2
- how to use tongram in a class HOT 1
- Move as much of things out of headers as possible and make tongrams a shared library HOT 3
- Implement building ngrams storage via python HOT 4
- lookup() - Segmentation fault when ngram is not in data structure HOT 5
- Trying build_trie with arpa file. HOT 4
- Sequence is not sorted HOT 9
- Can't load MPH-based models in Python HOT 1
- sort_arpa can't work HOT 4
- Update external dependencies and use PTHash instead of EMPHF
- One master tool instead of many different executables
- Remove dependency from boost
- sort_grams - found the bug causing the exception HOT 4
- Compile fails on gcc 4.9, Debian Jessie HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tongrams.