Comments (19)
This works for me (BTW, does this mean you were able to compile?):
allow-unk: false
batch-size: 1
beam-size: 12
bpe:
- deen.bpe
bunch-size: 1
cpu-threads: 8
devices: [0]
gpu-threads: 0
n-best: false
no-debpe: false
normalize: false
relative-paths: true
return-alignment: false
scorers:
F0:
path: model.npz
type: Nematus
show-weights: false
softmax-filter:
[]
source-vocab:
- vocab.de.json
target-vocab: vocab.en.json
weights:
F0: 1
wipo: false
BTW: You can let amun create this config file by doing:
path/to/amunmt/bin/amun -m model.npz -s vocab.de.json -t vocab.en.json \
--bpe deen.bpe --cpu-threads 8 --gpu-threads 0 --dump-config \
--relative-paths > config.yml
from marian.
And for four models created with:
path/to/amunmt/bin/amun -m model-ens*.npz -s vocab.de.json -t vocab.en.json \
--bpe deen.bpe --cpu-threads 8 --gpu-threads 0 --dump-config \
--relative-paths > config.ens.yml
you get the following config:
allow-unk: false
batch-size: 1
beam-size: 12
bpe:
- deen.bpe
bunch-size: 1
cpu-threads: 8
devices: [0]
gpu-threads: 0
n-best: false
no-debpe: false
normalize: false
relative-paths: true
return-alignment: false
scorers:
F0:
path: model-ens1.npz
type: Nematus
F1:
path: model-ens2.npz
type: Nematus
F2:
path: model-ens3.npz
type: Nematus
F3:
path: model-ens4.npz
type: Nematus
show-weights: false
softmax-filter:
[]
source-vocab:
- vocab.de.json
target-vocab: vocab.en.json
weights:
F0: 1
F1: 1
F2: 1
F3: 1
wipo: false
from marian.
Hi @mehmedes
Can you pull the lastest version? What gcc version do you have? I tried to reproduce but it works on my machine. I downloaded en-de model (using scripts/download_models.py) and replaced with your config.
from marian.
I went back to 3a548b3 for testing.
I'll try your latest commit asap and let you know if it works. I'm using gcc 5.4.
Thank you so much for your invaluable input! Have a nice weekend!
from marian.
I've now been able to build the latest commit of amunmt.
When trying to create the config file, however, I still get the error:
Segmentation fault (core dumped)
Do you have any clues?
Thanks!
from marian.
Can you please post your entire config file using the markup described here https://help.github.com/articles/creating-and-highlighting-code-blocks/
from marian.
Sure. Thanks for the link. I was always wondering how to get the proper formatting in Github.
Amunmt creates an empty config file, stating Segmentation fault (core dumped). I've also tried to use the both configs above but still Segmentation fault (core dumped). Again, at the moment I'm not able to create a "real" config file. All I get is the error message with an empty file.
from marian.
So, just to make sure I understand. You tried one of the above commands I posted, exactly the way I wrote them, and you get a segfault when you do this?
Unfortunately I cannot reproduce this at the moment. One straw of hope: Can you remove your build directory and try the entire compilation once more from scratch?
Also, earlier you wrote that you were using a previous revision. Did that one work for you?
And a last thing to check. What happens if you call amun without any options, just ./bin/amun
? You should get an exception like this:
terminate called after throwing an instance of 'util::Exception'
what(): /home/marcin/amunmt/src/common/config.cpp:85 in void Validate(const YAML::Node&) threw util::Exception because `!config["scorers"] || config["scorers"].size() == 0'.
No scorers given in config file
Aborted (core dumped)
from marian.
I first tried the exact commands above on 3a548b3.
I got Segmentation fault (core dumped).
Then I tried to create a config file according to the instructions:
path/to/amunmt/bin/amun -m model.npz -s vocab.de.json -t vocab.en.json
--bpe deen.bpe --cpu-threads 8 --gpu-threads 0 --dump-config
--relative-paths > config.yml
path/to/amunmt/bin/amun -m model-ens*.npz -s vocab.de.json -t vocab.en.json
--bpe deen.bpe --cpu-threads 8 --gpu-threads 0 --dump-config
--relative-paths > config.ens.yml
Again, Segmentation fault (core dumped).
Then, I deleted the entire installation and set up the latest commit after your fix.
I first tried one of the above commands. Didn't work.
Lastly I tried creating a config file with the above instructions.
Again Segmentation fault (core dumped).
The only command that works is ./amun -help
from marian.
Interestingly, if I type ./bin/amun -help I get the help message.
If I type --help or -h, I get Segmentation fault (core dumped).
from marian.
What's your boost version? We use boost::options in amuNMT for handling this.
from marian.
Boost 1.54
from marian.
Would this core file be of any use?
core.zip
from marian.
I personally don't use core files. Being able to replicate your segfault is better so your model files, config file and the exact command you executed is better
from marian.
This is one of the commands I tried for creating a config file for the English-German WMT model. The model files are in the same folder I ran the command in.
./bin/amun -m model.npz -s vocab.en-json -t vocab.de.json --bpe ende.bpe --cpu-threads 8 --dump-config --relative-paths > config.yml
The output:
Segmentation fault (core dumped)
from marian.
Running that command is ok with me. However, I'm running on Ubuntu 16, boost 1.62, gcc 5.4
I'm trying to install a VM with your specs but it's quite difficult.
It may be better for you to update your OS
from marian.
I've just test on OSX Sierra + boost 1.54 (1.54 doesn't build on Ubuntu 16). It works fine
So the problem is likely to be in Ubuntu 14 with the custom gcc & cmake install.
from marian.
Alright. I'll update the OS and keep you updated!
from marian.
Everything runs on Ubuntu 16 smoothly. Thanks!
from marian.
Related Issues (20)
- Doc error in the explanation of --lr-decay-inv-sqrt?
- Linking CXX executable ../marian
- Not regular training : BLEU is brutally decreasing for several batches, before increasing again HOT 2
- running marian-server in secure fashion (wss) HOT 1
- Validation set is decoded multiple times for ngram level metrics
- undefined reference to `libiconv' HOT 1
- negative cost with negative weights
- Freezing layers/modules HOT 1
- when will update the doc HOT 1
- half float and other 3rd_party problems
- Error: Aborted from static To marian::fastopt_helpers::Convert<To, std::__cxx11::basic_string<char> HOT 1
- Unable to install Marian HOT 1
- What versions of CUDA & GCC are required to compile Marian-nmt? HOT 3
- Major / Minor changes
- Cost is nan When adding guided alignment HOT 1
- How to develop a C++ tokenizer for MarianMT in C++
- Get no output when using marian-decoder
- Marian's Sentencepiece Not Passing Case Encoding Command Through Marian HOT 1
- will marian continue to develop and update again? HOT 1
- Compilation errors due to warnings treated as errors
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from marian.