facebookresearch / symbolicmathematics Goto Github PK

What version of the dependencies (esp. NumPy and SymPy) have been used?
Thank you.

Is there any parameter which can restrict the power of the variables being generated for an equation?

According to https://github.com/facebookresearch/SymbolicMathematics/blob/master/src/envs/char_sp.py,

D[e][n] represents the number of different binary trees with n nodes that
can be generated from e empty nodes, using the following recursion:
D(0, n) = 0
D(1, n) = C_n (n-th Catalan number)
D(e, n) = D(e - 1, n + 1) - D(e - 2, n + 1)

I understand D(1,n) as number of different random binary tree with n node, but what is the meaning of e "empty node" in D(e,n)

I'm trying to evaluate the model, and I use the beam evaluation.
I find out that your answer in the third line is copied from the second line, which is the answer in dataset.
Your code which copies the answer is here:
beam_log = {}
for i in range(len(len1)):
src = idx_to_sp(env, x1[1:len1[i] - 1, i].tolist())
tgt = idx_to_sp(env, x2[1:len2[i] - 1, i].tolist())
if valid[i]:
beam_log[i] = {'src': src, 'tgt': tgt, 'hyps': [(tgt, None, True)]}
The 'hyps' should be the prediction by the model, but you use 'tgt' as the output.
Why not give the prediction in the program? And can you give the code which can output the answer?

I'd like to be able to compare equation similarity. I wonder if I can get this model to extract embeddings for 2 particular equation inputs.

Trying to use this the first time:

$ python main.py --export_data true

SLURM job: False
0 - Number of nodes: 1
0 - Node ID : 0
0 - Local rank : 0
0 - Global rank : 0
0 - World size : 1
0 - GPUs per node : 1
0 - Master : True
0 - Multi-node : False
0 - Multi-GPU : False
0 - Hostname : james-Bonobo-WS
INFO - 04/18/20 15:18:46 - 0:00:00 - ============ Initialized logger ============
INFO - 04/18/20 15:18:46 - 0:00:00 - accumulate_gradients: 1
amp: -1
attention_dropout: 0
balanced: False
batch_size: 32
beam_early_stopping: True
beam_eval: False
beam_length_penalty: 1
beam_size: 1
clean_prefix_expr: True
clip_grad_norm: 5
command: python main.py --export_data true --exp_id "imf0l5hfpl"
cpu: False
debug: False
debug_slurm: False
dropout: 0
dump_path: ./dumped/debug/imf0l5hfpl
emb_dim: 256
env_base_seed: 0
env_name: char_sp
epoch_size: 300000
eval_only: False
eval_verbose: 0
eval_verbose_print: False
exp_id: imf0l5hfpl
exp_name: debug
export_data: True
fp16: False
global_rank: 0
int_base: 10
is_master: True
is_slurm_job: False
leaf_probs: 0.75,0,0.25,0
local_rank: 0
master_port: -1
max_epoch: 100000
max_int: 10000
max_len: 512
max_ops: 10
max_ops_G: 4
multi_gpu: False
multi_node: False
n_coefficients: 0
n_dec_layers: 4
n_enc_layers: 4
n_gpu_per_node: 1
n_heads: 4
n_nodes: 1
n_variables: 1
node_id: 0
num_workers: 10
operators: add:2,sub:1
optimizer: adam,lr=0.0001
positive: False
precision: 10
reload_checkpoint:
reload_data:
reload_model:
reload_size: -1
rewrite_functions:
same_nb_ops_per_batch: False
save_periodic: 0
share_inout_emb: True
sinusoidal_embeddings: False
stopping_criterion:
tasks:
validation_metrics:
world_size: 1
INFO - 04/18/20 15:18:46 - 0:00:00 - The experiment will be stored in ./dumped/debug/imf0l5hfpl

INFO - 04/18/20 15:18:46 - 0:00:00 - Running command: python main.py --export_data true

WARNING - 04/18/20 15:18:46 - 0:00:00 - Signal handler installed.
INFO - 04/18/20 15:18:46 - 0:00:00 - Unary operators: []
INFO - 04/18/20 15:18:46 - 0:00:00 - Binary operators: ['add', 'sub']
INFO - 04/18/20 15:18:46 - 0:00:00 - words: {'': 0, '': 1, '': 2, '(': 3, ')': 4, '<SPECIAL_5>': 5, '<SPECIAL_6>': 6, '<SPECIAL_7>': 7, '<SPECIAL_8>': 8, '<SPECIAL_9>': 9, 'pi': 10, 'E': 11, '
x': 12, 'y': 13, 'z': 14, 't': 15, 'a0': 16, 'a1': 17, 'a2': 18, 'a3': 19, 'a4': 20, 'a5': 21, 'a6': 22, 'a7': 23, 'a8': 24, 'a9': 25, 'abs': 26, 'acos': 27, 'acosh': 28, 'acot': 29, 'acoth': 30, 'acsc':
31, 'acsch': 32, 'add': 33, 'asec': 34, 'asech': 35, 'asin': 36, 'asinh': 37, 'atan': 38, 'atanh': 39, 'cos': 40, 'cosh': 41, 'cot': 42, 'coth': 43, 'csc': 44, 'csch': 45, 'derivative': 46, 'div': 47, 'ex
p': 48, 'f': 49, 'g': 50, 'h': 51, 'inv': 52, 'ln': 53, 'mul': 54, 'pow': 55, 'pow2': 56, 'pow3': 57, 'pow4': 58, 'pow5': 59, 'rac': 60, 'sec': 61, 'sech': 62, 'sign': 63, 'sin': 64, 'sinh': 65, 'sqrt': 6
6, 'sub': 67, 'tan': 68, 'tanh': 69, 'I': 70, 'INT+': 71, 'INT-': 72, 'INT': 73, 'FLOAT': 74, '-': 75, '.': 76, '10^': 77, 'Y': 78, "Y'": 79, "Y''": 80, '0': 81, '1': 82, '2': 83, '3': 84, '4': 85, '5': 8
6, '6': 87, '7': 88, '8': 89, '9': 90}
INFO - 04/18/20 15:18:46 - 0:00:00 - 20001 possible leaves.
INFO - 04/18/20 15:18:46 - 0:00:00 - Checking expressions in [0.01, 0.1, 0.3, 0.5, 0.7, 0.9, 1.1, 2.1, 3.1, -0.01, -0.1, -0.3, -0.5, -0.7, -0.9, -1.1, -2.1, -3.1]
Traceback (most recent call last):
File "main.py", line 232, in
main(params)
File "main.py", line 167, in main
env = build_env(params)
File "/home/james/Data Science/SymbolicMathematics/src/envs/init.py", line 29, in build_env
assert len(tasks) == len(set(tasks)) > 0
AssertionError

• Instruction is as follows:
  python main.py --exp_name first_eval --eval_only true --reload_model "./fwd_bwd_ibp.pth" --reload_data "./prim_ibp.test" --beam_eval true --beam_size 10 --emb_dim 1024
  --n_enc_layers 6 --n_dec_layers 6 --n_heads 8 --dump_path ./dump

• The error record is as follows:
  Traceback (most recent call last):
  File "main.py", line 232, in
  main(params)
  File "main.py", line 156, in main
  logger = initialize_exp(params)
  File "D:\code\python\SymbolicMathematics\src\utils.py", line 57, in initialize_exp
  pickle.dump(params, open(os.path.join(params.dump_path, 'params.pkl'), 'wb'))
  FileNotFoundError: [Errno 2] No such file or directory: './dump\first_eval\5mqaq8pt0a\params.pkl'

hello, I want to run your code on a dual graphics machine, But I encountered the following problems：

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

Traceback (most recent call last):
File "main.py", line 256, in
check_model_params(params)
File "/public/home/pw/workspace/symbolicmathematics/SymbolicMathematics-master/src/model/init.py", line 27, in check_model_params
assert os.path.isfile(params.reload_model)
AssertionError
Traceback (most recent call last):
File "main.py", line 256, in
check_model_params(params)
File "/public/home/pw/workspace/symbolicmathematics/SymbolicMathematics-master/src/model/init.py", line 27, in check_model_params
assert os.path.isfile(params.reload_model)
AssertionError
Traceback (most recent call last):
File "/public/home/pw/anaconda3/envs/mathematics/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/public/home/pw/anaconda3/envs/mathematics/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/public/home/pw/anaconda3/envs/mathematics/lib/python3.7/site-packages/torch/distributed/launch.py", line 253, in
main()
File "/public/home/pw/anaconda3/envs/mathematics/lib/python3.7/site-packages/torch/distributed/launch.py", line 249, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/public/home/pw/anaconda3/envs/mathematics/bin/python', '-u', 'main.py', '--local_rank=1', '--exp_name', 'first_eval', '--eval_only', 'true', '--reload_model', 'fwd.pth', '--tasks', 'prim_fwd', '--reload_data', 'prim_fwd,prim_fwd.train,prim_fwd.valid,prim_fwd.test', '--emb_dim', '1024', '--n_enc_layers', '6', '--n_dec_layers', '6', '--n_heads', '8', '--beam_eval', 'true', '--beam_size', '10', '--beam_length_penalty', '1.0', '--beam_early_stopping', '1', '--eval_verbose', '1', '--eval_verbose_print', 'false']' returned non-zero exit status 1.

I run it completely on a single graphics card,My complete command is as follows

$NGPU = 2；
python -m torch.distributed.launch --nproc_per_node=$NGPU main.py --exp_name first_eval --eval_only true --reload_model "fwd.pth" --tasks "prim_fwd" --reload_data "prim_fwd,prim_fwd.train,prim_fwd.valid,prim_fwd.test" --emb_dim 1024 --n_enc_layers 6 --n_dec_layers 6 --n_heads 8 --beam_eval true --beam_size 10 --beam_length_penalty 1.0 --beam_early_stopping 1 --eval_verbose 1 --eval_verbose_print false

my Python version is 3.7.10, PyTorch version is 1.3.0 and torchversion is 0.4.1, Maybe my version of PyTorch is wrong or I need to modify the default parameter of local_rank?

Thank You.

Hi everyone !

I played a bit with the solver but there is something I tried but didn't succeed: Of course we can solve simple ODEs like f'+12=0 but can we solve f'+c=0 for c a fixed arbitrary real constant ? And in the latter case how to do so ?

Many thanks by advance for those of you that will spent some time on my question :)

Thanks for your great works!

I want to do evaluation for pre-trained models with beam size 10 on multiple gpus, because I cannot run evaluation on single gpu due to out of memory, though the gpu is TITAN X (Pascal) with 12GB of memory.

However, when I run evaluation on multiple gpus, it throws up the following error:

AttributeError: 'DistributedDataParallel' object has no attribute 'generate_beam'

How can I do in this situation?

Thanks!

Hello! Thank you so much for putting up this wonderful resource! I am a student trying to play with this code and when I am running python3 main.py --> and doing training, I end up getting the following error of DataLoader worker exiting unexpectedly? (attached image). Do you know what causes this error? Even when I restart the computer and try again, I eventually run into this same error.

Note: I don't think the DataLoader exit problem is related to the ImportError exception, because there were many such "ImportError exceptions" on previous training equations with no such DataLoader exit.

Many thanks, and have a good day! :)

Is this not finding my data? I downloaded and extracted the data and put it in the main.py directory, is this correct?

Hi, I'm currently playing around with the symbomath code and I keep running into this error with data generation.

Running the following code (the example generation code provided):
python main.py --export_data true --batch_size 32 --cpu true --exp_name prim_bwd_data --num_workers 20 --tasks prim_bwd --env_base_seed -1 --n_variables 1 --n_coefficients 0 --leaf_probs "0.75,0,0.25,0" --max_ops 15 --max_int 5 --positive true --max_len 512 --operators "add:10,sub:3,mul:10,div:5,sqrt:4,pow2:4,pow3:2,pow4:1,pow5:1,ln:4,exp:4,sin:4,cos:4,tan:4,asin:1,acos:1,atan:1,sinh:1,cosh:1,tanh:1,asinh:1,acosh:1,atanh:1"

Yields an error message telling me that './dumped/prim_bwd_data\48t7888vh8\params.pkl' doesn't exist. I can't manually create the file, since that 10-digit string of numbers constantly changes, so could someone explain why the code isn't correctly making a directory for this?

But that's not the issue - I can set a parameter to set a dump path, so the run code looks like this (all I did was add a --dump_path to it): python main.py --export_data true --dump_path C:\Users\Toby\PycharmProjects\PDE\venv\dumped --batch_size 32 --cpu true --exp_name prim_bwd_data --num_workers 20 --tasks prim_bwd --env_base_seed -1 --n_variables 1 --n_coefficients 0 --leaf_probs "0.75,0,0.25,0" --max_ops 15 --max_int 5 --positive true --max_len 512 --operators "add:10,sub:3,mul:10,div:5,sqrt:4,pow2:4,pow3:2,pow4:1,pow5:1,ln:4,exp:4,sin:4,cos:4,tan:4,asin:1,acos:1,atan:1,sinh:1,cosh:1,tanh:1,asinh:1,acosh:1,atanh:1"`

Doing this yields the following error message:
SLURM job: False
0 - Number of nodes: 1
0 - Node ID : 0
0 - Local rank : 0
0 - Global rank : 0
0 - World size : 1
0 - GPUs per node : 1
0 - Master : True
0 - Multi-node : False
0 - Multi-GPU : False
0 - Hostname : ChanPC
A subdirectory or file -p already exists.
Error occurred while processing: -p.
INFO - 06/25/20 14:02:53 - 0:00:00 - ============ Initialized logger ============
INFO - 06/25/20 14:02:53 - 0:00:00 - accumulate_gradients: 1
amp: -1
attention_dropout: 0
balanced: False
batch_size: 32
beam_early_stopping: True
beam_eval: False
beam_length_penalty: 1
beam_size: 1
clean_prefix_expr: True
clip_grad_norm: 5
command: python main.py --export_data true --dump_path 'C:\Users\Chan\PycharmProjects\PDE\venv\dumped' --batch_size 32 --cpu true --exp_name prim_bwd_data --num_workers 20 --tasks prim_bwd --env_base_seed '-1' --n_variables 1 --n_coefficients 0 --leaf_probs '0.75,0,0.25,0' --max_ops 15 --max_int 5 --positive true --max_len 512 --operators 'add:10,sub:3,mul:10,div:5,sqrt:4,pow2:4,pow3:2,pow4:1,pow5:1,ln:4,exp:4,sin:4,cos:4,tan:4,asin:1,acos:1,atan:1,sinh:1,cosh:1,tanh:1,asinh:1,acosh:1,atanh:1' --exp_id "9epgwherdq"
cpu: True
debug: False
debug_slurm: False
dropout: 0
dump_path: C:\Users\Chan\PycharmProjects\PDE\venv\dumped\prim_bwd_data\9epgwherdq
emb_dim: 256
env_base_seed: -1
env_name: char_sp
epoch_size: 300000
eval_only: False
eval_verbose: 0
eval_verbose_print: False
exp_id: 9epgwherdq
exp_name: prim_bwd_data
export_data: True
fp16: False
global_rank: 0
int_base: 10
is_master: True
is_slurm_job: False
leaf_probs: 0.75,0,0.25,0
local_rank: 0
master_port: -1
max_epoch: 100000
max_int: 5
max_len: 512
max_ops: 15
max_ops_G: 4
multi_gpu: False
multi_node: False
n_coefficients: 0
n_dec_layers: 4
n_enc_layers: 4
n_gpu_per_node: 1
n_heads: 4
n_nodes: 1
n_variables: 1
node_id: 0
num_workers: 20
operators: add:10,sub:3,mul:10,div:5,sqrt:4,pow2:4,pow3:2,pow4:1,pow5:1,ln:4,exp:4,sin:4,cos:4,tan:4,asin:1,acos:1,atan:1,sinh:1,cosh:1,tanh:1,asinh:1,acosh:1,atanh:1
optimizer: adam,lr=0.0001
positive: True
precision: 10
reload_checkpoint:
reload_data:
reload_model:
reload_size: -1
rewrite_functions:
same_nb_ops_per_batch: False
save_periodic: 0
share_inout_emb: True
sinusoidal_embeddings: False
stopping_criterion:
tasks: prim_bwd
validation_metrics:
world_size: 1
INFO - 06/25/20 14:02:53 - 0:00:00 - The experiment will be stored in C:\Users\Chan\PycharmProjects\PDE\venv\dumped\prim_bwd_data\9epgwherdq

INFO - 06/25/20 14:02:53 - 0:00:00 - Running command: python main.py --export_data true --dump_path 'C:\Users\Chan\PycharmProjects\PDE\venv\dumped' --batch_size 32 --cpu true --exp_name prim_bwd_data --num_workers 20 --tasks prim_bwd --env_base_seed '-1' --n_variables 1 --n_coefficients 0 --leaf_probs '0.75,0,0.25,0' --max_ops 15 --max_int 5 --positive true --max_len 512 --operators 'add:10,sub:3,mul:10,div:5,sqrt:4,pow2:4,pow3:2,pow4:1,pow5:1,ln:4,exp:4,sin:4,cos:4,tan:4,asin:1,acos:1,atan:1,sinh:1,cosh:1,tanh:1,asinh:1,acosh:1,atanh:1'

WARNING - 06/25/20 14:02:53 - 0:00:00 - Signal handler installed.
INFO - 06/25/20 14:02:53 - 0:00:00 - Unary operators: ['acos', 'acosh', 'asin', 'asinh', 'atan', 'atanh', 'cos', 'cosh', 'exp', 'ln', 'pow2', 'pow3', 'pow4', 'pow5', 'sin', 'sinh', 'sqrt', 'tan', 'tanh']
INFO - 06/25/20 14:02:53 - 0:00:00 - Binary operators: ['add', 'div', 'mul', 'sub']
INFO - 06/25/20 14:02:53 - 0:00:00 - words: {'': 0, '': 1, '': 2, '(': 3, ')': 4, '<SPECIAL_5>': 5, '<SPECIAL_6>': 6, '<SPECIAL_7>': 7, '<SPECIAL_8>': 8, '<SPECIAL_9>': 9, 'pi': 10, 'E': 11, 'x': 12, 'y': 13, 'z': 14, 't': 15, 'a0': 16, 'a1': 17, 'a2': 18, 'a3': 19, 'a4': 20, 'a5': 21, 'a6': 22, 'a7': 23, 'a8': 24, 'a9': 25, 'abs': 26, 'acos': 27, 'acosh': 28, 'acot': 29, 'acoth': 30, 'acsc': 31, 'acsch': 32, 'add': 33, 'asec': 34, 'asech': 35, 'asin': 36, 'asinh': 37, 'atan': 38, 'atanh': 39, 'cos': 40, 'cosh': 41, 'cot': 42, 'coth': 43, 'csc': 44, 'csch': 45, 'derivative': 46, 'div': 47, 'exp': 48, 'f': 49, 'g': 50, 'h': 51, 'inv': 52, 'ln': 53, 'mul': 54, 'pow': 55, 'pow2': 56, 'pow3': 57, 'pow4': 58, 'pow5': 59, 'rac': 60, 'sec': 61, 'sech': 62, 'sign': 63, 'sin': 64, 'sinh': 65, 'sqrt': 66, 'sub': 67, 'tan': 68, 'tanh': 69, 'I': 70, 'INT+': 71, 'INT-': 72, 'INT': 73, 'FLOAT': 74, '-': 75, '.': 76, '10^': 77, 'Y': 78, "Y'": 79, "Y''": 80, '0': 81, '1': 82, '2': 83, '3': 84, '4': 85, '5': 86, '6': 87, '7': 88, '8': 89, '9': 90}
INFO - 06/25/20 14:02:53 - 0:00:00 - 6 possible leaves.
INFO - 06/25/20 14:02:53 - 0:00:00 - Checking expressions in [0.01, 0.1, 0.3, 0.5, 0.7, 0.9, 1.1, 2.1, 3.1, -0.01, -0.1, -0.3, -0.5, -0.7, -0.9, -1.1, -2.1, -3.1]
INFO - 06/25/20 14:02:53 - 0:00:00 - Training tasks: prim_bwd
INFO - 06/25/20 14:02:53 - 0:00:00 - Number of parameters (encoder): 4231424
INFO - 06/25/20 14:02:53 - 0:00:00 - Number of parameters (decoder): 5286235
INFO - 06/25/20 14:02:53 - 0:00:00 - Found 177 parameters in model.
INFO - 06/25/20 14:02:53 - 0:00:00 - Optimizers: model
INFO - 06/25/20 14:02:53 - 0:00:00 - Data will be stored in prefix in: C:\Users\Chan\PycharmProjects\PDE\venv\dumped\prim_bwd_data\9epgwherdq\data.prefix ...
INFO - 06/25/20 14:02:53 - 0:00:00 - Data will be stored in infix in: C:\Users\Chan\PycharmProjects\PDE\venv\dumped\prim_bwd_data\9epgwherdq\data.infix ...
INFO - 06/25/20 14:02:53 - 0:00:00 - Creating train iterator for prim_bwd ...
Traceback (most recent call last):
File "main.py", line 225, in
main(params)
File "main.py", line 162, in main
trainer = Trainer(modules, env, params)
File "C:\Users\Chan\PycharmProjects\PDE\venv\src\trainer.py", line 140, in init
self.dataloader = {
File "C:\Users\Chan\PycharmProjects\PDE\venv\src\trainer.py", line 141, in
task: iter(self.env.create_train_iterator(task, params, self.data_path))
File "D:\Anaconda\envs\pde\lib\site-packages\torch\utils\data\dataloader.py", line 279, in iter
return _MultiProcessingDataLoaderIter(self)
File "D:\Anaconda\envs\pde\lib\site-packages\torch\utils\data\dataloader.py", line 719, in init
w.start()
File "D:\Anaconda\envs\pde\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "D:\Anaconda\envs\pde\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "D:\Anaconda\envs\pde\lib\multiprocessing\context.py", line 326, in _Popen
return Popen(process_obj)
File "D:\Anaconda\envs\pde\lib\multiprocessing\popen_spawn_win32.py", line 93, in init
reduction.dump(process_obj, to_child)
File "D:\Anaconda\envs\pde\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle f: attribute lookup f on main failed

(pde) C:\Users\Chan\PycharmProjects\PDE\venv>Traceback (most recent call last):
File "", line 1, in
File "D:\Anaconda\envs\pde\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "D:\Anaconda\envs\pde\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

train.log

To me, it looks like its trying to train at the same time or something, which is why it can't find an input (the pickle file doesn't exist yet). In the dump folder, there are the data.infix and prefix files, but they're empty.

Have I inputted the parameters wrong or am missing some kind of step? Any help would be much appreciated, as I am relatively new to coding. Thanks so much in advance!

I run this model on the ibp dataset and it gets killed when loading the training data.My GPU is 2070s and environment is pytorch1.3 devel. Does anybody know how to deal with the problem?
My command:
python main.py --exp_name first_train --fp16 true --amp 2 --tasks "prim_ibp" --reload_data "prim_ibp,prim_ibp.train,prim_ibp.valid,prim_ibp.test" --reload_size 40000000 --emb_dim 1024 --n_enc_layers 6 --n_dec_layers 6 --n_heads 8 --optimizer "adam,lr=0.0001" --batch_size 32 --epoch_size 300000 --validation_metrics valid_prim_fwd_acc
Reaction:INFO - 07/02/20 06:56:43 - 0:00:00 - The experiment will be stored in ./dumped/first_train/dgv5zq039m

INFO - 07/02/20 06:56:43 - 0:00:00 - Running command: python main.py --exp_name first_train --fp16 true --amp 2 --tasks prim_ibp --reload_data 'prim_ibp,prim_ibp.train,prim_ibp.valid,prim_ibp.test' --reload_size '-1' --emb_dim 128 --n_enc_layers 1 --n_dec_layers 1 --n_heads 1 --optimizer 'adam,lr=0.0001' --batch_size 8 --epoch_size 300000 --validation_metrics valid_prim_fwd_acc

WARNING - 07/02/20 06:56:43 - 0:00:00 - Signal handler installed.
INFO - 07/02/20 06:56:43 - 0:00:00 - Unary operators: []
INFO - 07/02/20 06:56:43 - 0:00:00 - Binary operators: ['add', 'sub']
INFO - 07/02/20 06:56:43 - 0:00:00 - words: {'': 0, '': 1, '': 2, '(': 3, ')': 4, '<SPECIAL_5>': 5, '<SPECIAL_6>': 6, '<SPECIAL_7>': 7, '<SPECIAL_8>': 8, '<SPECIAL_9>': 9, 'pi': 10, 'E': 11, 'x': 12, 'y': 13, 'z': 14, 't': 15, 'a0': 16, 'a1': 17, 'a2': 18, 'a3': 19, 'a4': 20, 'a5': 21, 'a6': 22, 'a7': 23, 'a8': 24, 'a9': 25, 'abs': 26, 'acos': 27, 'acosh': 28, 'acot': 29, 'acoth': 30, 'acsc': 31, 'acsch': 32, 'add': 33, 'asec': 34, 'asech': 35, 'asin': 36, 'asinh': 37, 'atan': 38, 'atanh': 39, 'cos': 40, 'cosh': 41, 'cot': 42, 'coth': 43, 'csc': 44, 'csch': 45, 'derivative': 46, 'div': 47, 'exp': 48, 'f': 49, 'g': 50, 'h': 51, 'inv': 52, 'ln': 53, 'mul': 54, 'pow': 55, 'pow2': 56, 'pow3': 57, 'pow4': 58, 'pow5': 59, 'rac': 60, 'sec': 61, 'sech': 62, 'sign': 63, 'sin': 64, 'sinh': 65, 'sqrt': 66, 'sub': 67, 'tan': 68, 'tanh': 69, 'I': 70, 'INT+': 71, 'INT-': 72, 'INT': 73, 'FLOAT': 74, '-': 75, '.': 76, '10^': 77, 'Y': 78, "Y'": 79, "Y''": 80, '0': 81, '1': 82, '2': 83, '3': 84, '4': 85, '5': 86, '6': 87, '7': 88, '8': 89, '9': 90}
INFO - 07/02/20 06:56:43 - 0:00:00 - 20001 possible leaves.
INFO - 07/02/20 06:56:43 - 0:00:00 - Checking expressions in [0.01, 0.1, 0.3, 0.5, 0.7, 0.9, 1.1, 2.1, 3.1, -0.01, -0.1, -0.3, -0.5, -0.7, -0.9, -1.1, -2.1, -3.1]
INFO - 07/02/20 06:56:43 - 0:00:00 - Training tasks: prim_ibp
INFO - 07/02/20 06:56:43 - 0:00:00 - Number of parameters (encoder): 734464
INFO - 07/02/20 06:56:43 - 0:00:00 - Number of parameters (decoder): 800859
INFO - 07/02/20 06:56:47 - 0:00:03 - Found 51 parameters in model.
INFO - 07/02/20 06:56:47 - 0:00:03 - Optimizers: model
Selected optimization level O2: FP16 training with FP32 batchnorm and FP32 master weights.

Defaults for this optimization level are:
enabled : True
opt_level : O2
cast_model_type : torch.float16
patch_torch_functions : False
keep_batchnorm_fp32 : True
master_weights : True
loss_scale : dynamic
Processing user overrides (additional kwargs that are not None)...
After processing overrides, optimization options are:
enabled : True
opt_level : O2
cast_model_type : torch.float16
patch_torch_functions : False
keep_batchnorm_fp32 : True
master_weights : True
loss_scale : dynamic
INFO - 07/02/20 06:56:47 - 0:00:03 - Creating train iterator for prim_ibp ...
INFO - 07/02/20 06:56:47 - 0:00:03 - Loading data from prim_ibp.train ...
Killed

I downloaded this repository containing codes, data sets, and models trained, and tried to run the commands in the ipython notebook given by Dr. Lample. But I get a bug that I cannot solve.
The first 10 Inputs in the ipython notebooks run well, but for the In [11] to Decode with beam search, there throw out an error:

_File "", line 109, in
_, _, beam = decoder.generate_beam(encoded, len1, beam_size=beam_size, length_penalty=1.0, early_stopping=1, max_len=200)

File "D:\LampleCharton2019\SymbolicMathematics-master\src\model\transformer.py", line 544, in generate_beam
cache[k] = (cache[k][0][beam_idx], cache[k][1][beam_idx])

RuntimeError: CUDA error: device-side assert triggered_

The environment in my computer is win10, anaconda3, python3.7.5, pytorch (gpu), torch.cuda.is_available() = true, two Nvidia quadro P4000, they work well in other programs.

What for example what does a line like that mean:
2|sub Y' add INT+ 4 mul INT+ 2 pow add x sqrt INT+ 3 INT- 1 add mul INT+ 2 ln add x sqrt INT+ 3 mul INT+ 4 x
What is the "sub Y' what is the operator "INT-" or "INT" is it just integer addition/subtraction? And why do you differenciate between that and sub/add? Also where do I get the training pairs, there is only one expression per line not two.

In the configuration of the script, there is this setting:
--leaf_probs "0.75,0,0.25,0" #leaf sampling probabilities

What is it?

We need a docker environment to quickly replicate the results. I would be interested in making one.

I have an intel graphics card.

To verify that your GPU is CUDA-capable, go to your distribution's equivalent of System Properties, or, from the command line, enter:
$ lspci | grep -i nvidia

The result is empty. Do you have suggestions for what parts of the code I can still run? Is running on AWS an option?

I am a beginner in this area, so the questions may be incorrect
When a model is trained, I want to use the model to complete a certain "prediction task", so I have some words to be "predicted".
The question is : are "eval_only" and "predicted" not the same thing? because i found in the "eval_only" related program that I still need to input the prediction results into the decoder, instead of inputting the results of the previous step prediction into the decoder as most tutorials say.
Looking forward to and thank you for your reply!

Hi everyone !

I need to use coefficients such as a_0 but I have some trouble to find out how to do so. Of course I changed n_coefficients from 0 to 1 but also the leaf probability but then I don't know how to declare my coefficient let's say a_0 many thanks by advance for those of you that can help me :)

Hello,

First of all thanks for your work and to share it !

I was wondering if there is a notebook about how to use the ODE's prediction instead of the integral prediction and if not where can I find instruction about how to do so ?

Thanks by advance :)

Best,

Arthur SERRES
student in applied mathematics.

Excuse me, could you tell me that the attn_mask for Masked Multi-Head Attention only use sequence mask but ignore the padding mask? To my konwledge, padding mask is necessay for eliminating the effect of padding.