mtensor / neural_sketch Goto Github PK

neural sketch project, currently in generative regex, list transformation (deepcoder), and text editing (robustfill) domains

Python 99.55% Shell 0.45%

neural_sketch's People

Contributors

Stargazers

Watchers

Forkers

stjordanis lcary shini-tm fpga-tom tymonxiong j-c-w tc-imba dotieuthien yichao-liang hejia-zhang danielrrr

neural_sketch's Issues

Missing "basegrammar.p" file.

Is there some code to generate a basegrammar.p file? There is a comment FixThis near the basegrammar variable. Can you help me with this?

Which section of the code is creating the "word.vocab" file?

And why did you hard code the vocabulary in the algolispPrimitives.py file? Any specific reason? If I change that section of the code to read from the word.vocab file, what all sections do I need to change.

Can't evaluate with evaluate_deepcoder.py

I pretrained the skech generator and dc synthesizer following the instructions with no problem, but got an error when trying evaluate the full model using python eval/evaluate_deepcoder.py.

More specifically I got EOFError: Ran out of input, with more details shown below. I'm using conda environment, and 1 strange thing is it says `No module named 'numpy' despite I have numpy installed in my environment.

Could you please let me know how I can make it run? I'm trying to experiment with this as a research project. Any help would be much appreciated! Thanks a lot!

$ python eval/evaluate_deepcoder.py
loading model with holes
data file: data/T3_test_data_new.p
/home/yliang/miniconda3/envs/sketch/lib/python3.7/site-packages/torch/nn/functional.py:1374: UserWarning: nn.functiona
l.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
25
[(lambda (lambda (COUNT isODD (DROP $1 (MAP SHR $0))))), (lambda (lambda (COUNT isODD (ZIPWITH MIN (DROP $1 $0) $0)))), (lambda (lambda (COUNT isPOS (ZIPWITH MAX (DROP $1 $0) <HOLE>)))), (lambda (lambda (COUNT isODD (DROP <HOLE> (MAP SHR $0))))), (lambda (lambda (COUNT isEVEN (MAP DIV3 (DROP $1 $0))))), (lambda (lambda (COUNT <HOLE> (MAP SHR (DROP $1 $0))))), (lambda (lambda (COUNT isODD (TAKE $1 (MAP SHR $0))))), (lambda (lambda (COUNT isODD (DROP $1 (FILTER isPOS $0))))), (lambda (lambda (COUNT isODD (FILTER isNEG (DROP $1 $0))))), (lambda (lambda (COUNT isEVEN (ZIPWITH MAX (DROP $1 $0) $0)))), (lambda (lambda (COUNT isEVEN (ZIPWITH MIN (DROP $1 $0) $0)))), (lambda (lambda (COUNT isODD (MAP SHR (DROP $1 $0))))), (lambda (lambda (COUNT isODD (DROP $1 (MAP DIV3 $0))))), (lambda (lambda (COUNT isPOS (DROP $1 (REVERSE $0))))), (lambda (lambda (COUNT <HOLE> (DROP $1 (FILTER isPOS $0))))), (lambda (lambda (COUNT isODD (SCANL1 MIN (DROP $1 $0))))), (lambda (lambda (COUNT isODD (ZIPWITH MIN $0 (DROP $1 $0))))), (lambda (lambda (COUNT isEVEN (MAP SHR (DROP $1 $0))))), (lambda (lambda (COUNT isODD (ZIPWITH MAX (DROP $1 $0) $0)))), (lambda (lambda (COUNT isNEG (DROP $1 (FILTER isEVEN $0))))), (lambda (lambda (COUNT isPOS (FILTER isODD (DROP $1 $0))))), (lambda (lambda (COUNT isPOS (DROP $1 (FILTER isODD $0))))), (lambda (lambda (COUNT isPOS (ZIPWITH MIN (DROP $1 $0) $0)))), (lambda (lambda (COUNT isODD (MAP DIV3 (DROP $1 $0))))), (lambda (lambda (COUNT isEVEN (DROP $1 (MAP SHR $0)))))]
Traceback (most recent call last):
  File "compiledDriver.py", line 17, in <module>
    request = pickle.load(sys.stdin.buffer)
  File "/home/yliang/miniconda3/envs/sketch/lib-python/3/pickle.py", line 1571, in _load
    encoding=encoding, errors=errors).load()
  File "/home/yliang/miniconda3/envs/sketch/lib-python/3/pickle.py", line 1070, in load
    dispatch[key[0]](self)
  File "/home/yliang/miniconda3/envs/sketch/lib-python/3/pickle.py", line 1358, in load_global
    klass = self.find_class(module, name)
  File "/home/yliang/miniconda3/envs/sketch/lib-python/3/pickle.py", line 1408, in find_class
    __import__(module, level=0)
ModuleNotFoundError: No module named 'numpy'
Traceback (most recent call last):
Traceback (most recent call last):
  File "eval/evaluate_deepcoder.py", line 195, in <module>
    results = evaluate_dataset(model, dataset, nSamples, mdl, max_to_check, dcModel=dcModel)
  File "eval/evaluate_deepcoder.py", line 137, in evaluate_dataset
    return {datum: list(evaluate_datum(i, datum, model, dcModel, nRepeats, mdl, max_to_check)) for i, datum in enumerate(dataset)}
  File "eval/evaluate_deepcoder.py", line 137, in <dictcomp>
    return {datum: list(evaluate_datum(i, datum, model, dcModel, nRepeats, mdl, max_to_check)) for i, datum in enumerate(dataset)}
  File "eval/evaluate_deepcoder.py", line 123, in evaluate_datum
    enum_results, n_checked, n_hit = pypy_enumerate(untorch(g), datum.tp, datum.IO, mdl, sketchtups, n_checked, n_hit, t, max_to_check)
  File "/home/yliang/neural_sketch/util/pypy_util.py", line 50, in pypy_enumerate
    return callCompiled(dc_enumerate, tp, IO, mdl, sketchtups, n_checked, n_hit, t, max_to_check)
  File "/home/yliang/neural_sketch/ec/utilities.py", line 325, in callCompiled
    success, result = pickle.load(p.stdout)
EOFError: Ran out of input

Missing Data Files

Hi, I tried to run the program, but encoutered problem in loading data data/DeepCoder_data/T2_A2_V512_L10_train.txt.

I downloaded deepcoder data from their github and the data format is T=2_train.json. May I know how can I transform it to the correct format?

Thank you.

Pre-Trained Weights

Hi,
Thank you for your fantastic work.

Can you please provide the pre-trained model for AlgoLisp Dataset

Calculation of accuracy and other queries

I have a few more queries:

When I am running the evaluation script it gives me a warning "WARNING: right now 'eval' gives correct 'eval' test set". What does this mean?
When printing the accuracy of the model at the end, do you only check if the generated code is compilable or do you check if for all inputs, of the corresponding instance, the output by the generated code is correct?
When I ran the code for main_supervised_algolisp.py --pretrain, it ran the pretraining code but automatically started the training process. Is this supposed to happen? Now that the training process has started do I have to again run the main_supervised_algolisp.py without the pretrain flag?
If I want to save the code generated for the test dataset, how should I proceed?
For the calculation of accuracy have you matched the generated code to the ground-truth value without compiling the code?
Based on our research we found that the theoretical maximum accuracy should be 89.9% in the normal dataset and you have reported a 90% accuracy for the same. In that case, your accuracy for the filtered dataset should be 100% since all the non-compilable code will be removed via filtering.

Can you clear these doubts?

Error with deepcoderModel.py

I ran the algolisp DC model with the algolisp dataset. It was running smoothly.
I tried running the algolisp DC model code with a new dataset. I took care of the format, file extension, etc. The following error occurred:

KeyError in symbolToIndex for the key 'Once'. I checked the word and it was present in word.vocab in the data/generated folder.
'Once' is the first word of the first instance in the training set. The training and dev data were loaded.

Could you help me in resolving this error?

Where exactly is "algolispPrimitives" defined?

There is no package by the name of algolispPrimitives and there is no class within the code. Could you help me with this?
For grammar, there is a package of the same name. Have you used that one or something else?

Trouble Getting Started

I am having trouble getting started with this project. Do I need to have Anaconda installed to run this?
What version of python do I need?
I just keep running into various dependency issues and I am not sure where to start

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.