dongjun-lee / text-summarization-tensorflow Goto Github PK

I have limited compute power on my laptop, it will be really useful t study the code locally if you can provide a small dataset using which I can run it locally on my laptop and debug the code.
Thanks
Shakti

It seems that to use BeamSearchDecoder forward_only must be set to True
model = Model(reversed_dict, article_max_len, summary_max_len, config, forward_only=True)

But loss and update are defined only when forward_only=False
So I get this error

 _, step, loss = sess.run([model.update, model.global_step, model.loss], feed_dict=train_feed_dict)
AttributeError: 'Model' object has no attribute 'update'

What will be the loss and update when forward_only=True
Any help would be greatly appreciated! Thank you!

How do we train this on gpu?

Hi, I just met this issue when I use the file2rouge in colab. The program is running more than 4 hours for the rouge assessment and still without any further feedback. Does anyone knows how to fix the issue? Many Thanks.

please give me idea of replacing unk from source document

when code run here::

prediction_output = [[reversed_dict[y] for y in x] for x in prediction[:, 0, :]]

    with open("result.txt", "a") as f:
        for line in prediction_output:
            summary = list()
            for word in line:
                if word == "</s>":
                    break
                if word not in summary:
                    summary.append(word)
            print(" ".join(summary), file=f)

print('Summaries are saved to "result.txt"...')

there is error:

KeyError Traceback (most recent call last)
in
27 print('prediction:', prediction.shape)
28
---> 29 prediction_output = [[reversed_dict[y] for y in x] for x in prediction[:, 0, :]]
30
31

in (.0)
27 print('prediction:', prediction.shape)
28
---> 29 prediction_output = [[reversed_dict[y] for y in x] for x in prediction[:, 0, :]]
30
31

in (.0)
27 print('prediction:', prediction.shape)
28
---> 29 prediction_output = [[reversed_dict[y] for y in x] for x in prediction[:, 0, :]]
30
31

KeyError: -1

What is the version numver of tensorfow and numpy used in this code? I got a lot errors when I test the code?
Thanks a lot！

Hi,could you please add a license under this repo? Thank you.

Hello,
I'm having some issues with preparing the data
Steps taken :

1. Clone repo
2. Unzip sample_data.zip
3. python train.py

This is the error I'm getting

Building dictionary...
Traceback (most recent call last):
  File "/Developer/Machine-Learning/Shared-folder/text-summarization-tensorflow/train.py", line 35, in <module>
    word_dict, reversed_dict, article_max_len, summary_max_len = build_dict("train", args.toy)
  File "/Developer/Machine-Learning/Shared-folder/text-summarization-tensorflow/utils.py", line 32, in build_dict
    train_article_list = get_text_list(train_article_path, toy)
  File "/Developer/Machine-Learning/Shared-folder/text-summarization-tensorflow/utils.py", line 25, in get_text_list
    return list(map(lambda x: clean_str(x.strip()), f.readlines()))
  File "/anaconda3/envs/deeplearning/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 5241: ordinal not in range(128)

Hi,

Only I have trouble with BasicLSTMCell?
I resolve this error using LSTMCell(num_units=self.num_hidden,name='basic_lstm_cell'). But, it's tru that I use num_units=self.num_hidden?

It seems that this code was developed against older version.
Apparently in tensorflow 2.4.0 there is no tensorflow.contrib module.

How long it took you to train the model and what are the specifications of your machine?

Well I am new to tensorflow and nlp.
Does it will make any difference if we use word2vec rather than GloVe and can we write the same model in Keras?

As of now its only reading small articles , how to increase the article size to a large one ?

Hi,
I'm getting a KeyError -1 error at line 36 of test.py when trying to run for the pretrained model.

File "test.py", line 36, in
prediction_output = list(map(lambda x: [reversed_dict[y] for y in x], prediction[:, 0, :]))
KeyError: -1

May i know why it is throwing this error?

When I tried to run the test file , it gives keyerror. Please help me.

anyone with the same issue?

File "C:\ProgramData\Anaconda3\lib\site-packages\gensim_init_.py", line 6, in
from gensim import parsing, matutils, interfaces, corpora, models, similarities, summarization, utils # noqa:F401
File "C:\ProgramData\Anaconda3\lib\site-packages\gensim\parsing_init_.py", line 4, in
from .preprocessing import (remove_stopwords, strip_punctuation, strip_punctuation2, # noqa:F401
File "C:\ProgramData\Anaconda3\lib\site-packages\gensim\parsing\preprocessing.py", line 40, in
from gensim import utils
File "C:\ProgramData\Anaconda3\lib\site-packages\gensim\utils.py", line 39, in
import scipy.sparse
ModuleNotFoundError: No module named 'scipy.sparse'

How can I run it on GPU?

The summary obtained in the result.txt has no relevance with the input text in valid.article.filter.txt
Is there anything else that I need to provide for this summarizer.

AttributeError: 'Model' object has no attribute 'update'

when i change the parameter forward_only as True ,then run train.py ,finally, i get the mistake,i don't know why,can you help me?

What is the base paper for this model? Can you please, give me your research paper link?

Hello, it is very beneficial to read your article. How should I change it if I want to use RNN encoder instead of BiRNN encoder? Thank you!

    with tf.name_scope("encoder"):
        fw_cells = [self.cell(self.num_hidden) for _ in range(self.num_layers)]
        bw_cells = [self.cell(self.num_hidden) for _ in range(self.num_layers)]
        fw_cells = [rnn.DropoutWrapper(cell) for cell in fw_cells]
        bw_cells = [rnn.DropoutWrapper(cell) for cell in bw_cells]

        encoder_outputs, encoder_state_fw, encoder_state_bw = tf.contrib.rnn.stack_bidirectional_dynamic_rnn(
            fw_cells, bw_cells, self.encoder_emb_inp,
            sequence_length=self.X_len, time_major=True, dtype=tf.float32)
        self.encoder_output = tf.concat(encoder_outputs, 2)
        encoder_state_c = tf.concat((encoder_state_fw[0].c, encoder_state_bw[0].c), 1)
        encoder_state_h = tf.concat((encoder_state_fw[0].h, encoder_state_bw[0].h), 1)
        self.encoder_state = rnn.LSTMStateTuple(c=encoder_state_c, h=encoder_state_h)

I have a saved model from previous training but when I start my training it starts from the beginning.

Hi,

When I run the code, I have next error:

Loading Glove vectors...
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<timed exec> in <module>()

<ipython-input-168-5fd352f14db2> in __init__(self, reversed_dict, article_max_len, summary_max_len, args, forward_only)
     35 
     36         with tf.name_scope("encoder"):
---> 37             fw_cells = [self.cell(self.num_hidden) for _ in range(self.num_layers)]
     38             bw_cells = [self.cell(self.num_hidden) for _ in range(self.num_layers)]
     39             fw_cells = [rnn.DropoutWrapper(cell) for cell in fw_cells]

<ipython-input-168-5fd352f14db2> in <listcomp>(.0)
     35 
     36         with tf.name_scope("encoder"):
---> 37             fw_cells = [self.cell(self.num_hidden) for _ in range(self.num_layers)]
     38             bw_cells = [self.cell(self.num_hidden) for _ in range(self.num_layers)]
     39             fw_cells = [rnn.DropoutWrapper(cell) for cell in fw_cells]

TypeError: __call__() missing 1 required positional argument: 'state'

Unfortunately, I can't resolve her.

Anyone care to comment training times for this model? CPU/GPU/TPU?
I'm thinking in creating a Google Cloud ML Engine package for this (For both training and serving)

Could you let me know that the results would look like when you use the CNN Dataset instead ? If there are pre-trained models for it, please let me know

What if I wanted to summarize larger text data using this summarizer? How can I do that?
And will it give larger summaries rather than just headlines?

It takes whole day still unfinished on my MBP,I don't have ideas of how long it will takes.

I am a newbie in deep learning. While self-studying seq2seq model, I try to modify this code so that it can be applicable to another language. However, I faced one critical issue that the number of my output data (in result.txt) generated is much shorter than the number of input data.

Here are some differences in my code.

1. tensorflow version 1.10.
2. pretrained fasttext embedding instead of glove.

I first want to make the overall process work then optimize or train the model with more data.
But, I got stuck in this issue for a couple days.
Wonder if you have any idea what can possibly cause above issue.

Hi,

Does anyone happen to have the model file saved. Training is taking very long time to run and I currently don't have GPU on my device.

hello, excuse me, I want to use it on chinese, and I don't want to use glove as the way of embedding, and I want to use TenCent_chinses_embedding, I want to change the code. Can you give me a advise that how to change the code. thanks very much!

Until how much epochs should I train the model? Because I have been used 5000 article training data and 2000 testing data. After 400 epochs I am not getting proper(It means not related to articles) summaries.

hi,

when I run train.py at _, step, loss = sess.run([model.update, model.global_step, model.loss],feed_dict=train_feed_dict) I have error ValueError: setting an array element with a sequence.
Can u help me how fix this?

Please anyone give me an idea about how to replace "unk" in the generated summary with the actual word from the article(source).

Hi.

I successfully tested a portuguese corpus I prepared and trained ( change line in utils.py for word in word_tokenize(sentence, language='portuguese'): ).

I'd like to have a frozen model in a single .pb file in order to serve within an API. I tried several approaches, like this: https://blog.metaflow.fr/tensorflow-how-to-freeze-a-model-and-serve-it-with-a-python-api-d4f3596b3adc

But unsuccessfully.

Would you consider providing some method to export a saved model? Or point me to the right direction?

Thanks!

How does one supply own sentences to this model to get summarized?
As far as I am seeing, the prediction uses the title as well as the sentence in the test set to create a new sentence. Is there any way to bypass the usage of titles and just feed sentences?

I am doing a research requiring extractive summarize. Is it possible to convert tjis to extraction summarize

How do you calculate loss during testing?

Any help would be greatly appreciated!! Thanks

Hi.

You actual code always trains from the beginning. But Tensorflow permits training after a certain model, or checkpoint, and this is really useful.

I added a few lines to my train.py. This does not deserve a Pull Request, I believe. So I just let the lines here for later use / adaptation /inspiration / ignoring.

Add an argument:
parser.add_argument("--reuse", action="store_true", help="Reuse previously trained model")

Check checkpoint and argument:
after
if not os.path.exists("saved_model"):
os.mkdir("saved_model")

add
else:
if args.reuse:
old_model_checkpoint_path = open('saved_model/checkpoint', 'r')
old_model_checkpoint_path = "".join(["saved_model/",old_model_checkpoint_path.read().splitlines()[0].split('"')[1] ])

Restore model to session:
near saver = tf.train.Saver(tf.global_variables())
add
if 'old_model_checkpoint_path' in globals():
print("Continuing from previous trained model " , old_model_checkpoint_path , "...")
saver.restore(sess, old_model_checkpoint_path )

How do you calculate accuracy at the end of each epoch?

Any help would be greatly appreciated!! Thanks

Could you please tell me on which data set was this pretrained model trained on ? I am a little confused

I have tried to run the files but after testing the result.txt file which is the final output is empty nothing is been written to the file.

Hello when I ran my code on tensorflow I am getting that error. Can you help me please?

> Traceback (most recent call last):
>   File "model.py", line 3, in <module>
>     from utils import get_init_embedding
>   File "../Desktop/text-summarization-tensorflow-master/utils.py", line 6, in <module>
>     from gensim.models.keyedvectors import KeyedVectors
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/gensim/__init__.py", line 6, in <module>
>     from gensim import parsing, matutils, interfaces, corpora, models, similarities, summarization, utils  # noqa:F401
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/gensim/corpora/__init__.py", line 14, in <module>
>     from .wikicorpus import WikiCorpus  # noqa:F401
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/gensim/corpora/wikicorpus.py", line 471, in <module>
>     class WikiCorpus(TextCorpus):
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/gensim/corpora/wikicorpus.py", line 504, in WikiCorpus
>     def __init__(self, fname, processes=None, lemmatize=utils.has_pattern(), dictionary=None,
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/gensim/utils.py", line 1529, in has_pattern
>     from pattern.en import parse  # noqa:F401
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/pattern/text/en/__init__.py", line 61, in <module>
>     from pattern.text.en.inflect import (
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/pattern/text/en/__init__.py", line 80, in <module>
>     from pattern.text.en import wordnet
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/pattern/text/en/wordnet/__init__.py", line 57, in <module>
>     nltk.data.find("corpora/" + token)
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/nltk/data.py", line 653, in find
>     return find(modified_name, paths)
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/nltk/data.py", line 639, in find
>     return ZipFilePathPointer(p, zipentry)
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/nltk/compat.py", line 221, in _decorator
>     return init_func(*args, **kwargs)
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/nltk/data.py", line 486, in __init__
>     zipfile = OpenOnDemandZipFile(os.path.abspath(zipfile))
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/nltk/compat.py", line 221, in _decorator
>     return init_func(*args, **kwargs)
>   File "../anaconda3/envs/tf/lib/python3.6/site-packages/nltk/data.py", line 1012, in __init__
>     zipfile.ZipFile.__init__(self, filename)
>   File "../anaconda3/envs/tf/lib/python3.6/zipfile.py", line 1108, in __init__
>     self._RealGetContents()
>   File "../anaconda3/envs/tf/lib/python3.6/zipfile.py", line 1175, in _RealGetContents
>     raise BadZipFile("File is not a zip file")
> zipfile.BadZipFile: File is not a zip file