Python3 didn't work at the end of pre-training And pre-model didn't save successfully

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): win10 x86
TensorFlow installed from (source or binary): anaconda
TensorFlow version: 1.14.0
Python version: 3.7.4

My problem

I downloaded the data files supported by you, when I began step4.2(Keywords Extraction and Genre Pattern Building), it always showed that 'gbk' can't decode \xxx in line xx. So I tried to change the read-file way in the source. Like this:
Before:

    fin = open(file_path, 'r')

After:

    fin = open(file_path, 'r',encoding='utf-8')

It works, errors have gone when I changed all the places where it gave a decoding error.

But at the end of pre-training, it showed that "Python3 didn't work" with a dialogue box. And the code showed at last are following, And there hasn't a fold named 'premodel' in WMPoetry\wm, can you help me?

...
...
不辞朝谒远                          # 唯要近慈恩                          # 唯有信慈亲                          #
epoch: 1, 8400/8607 97.607%, 0.282 s per iter
train loss: 3.806  ppl:44.96, l2 loss: 17243.167, lr:0.0010.
______________________
今朝复相见                          # 疑是葛仙翁                          # 留是后天翁                          #
epoch: 1, 8600/8607 99.930%, 0.284 s per iter
train loss: 3.773  ppl:43.52, l2 loss: 17536.656, lr:0.0010.
______________________
0.0010 to 0.0010
run validation...
validation epoch: 1  loss: 5.679  ppl: 292.55, l2 loss: 30140.641
saving model...
WARNING:tensorflow:*******************************************************
W0518 15:41:08.506486 15588 saver.py:1134] *******************************************************
WARNING:tensorflow:TensorFlow's V1 checkpoint format has been deprecated.
W0518 15:41:08.507480 15588 saver.py:1135] TensorFlow's V1 checkpoint format has been deprecated.
WARNING:tensorflow:Consider switching to the more efficient V2 format:
W0518 15:41:08.509477 15588 saver.py:1136] Consider switching to the more efficient V2 format:
WARNING:tensorflow:   `tf.train.Saver(write_version=tf.train.SaverDef.V2)`
W0518 15:41:08.510476 15588 saver.py:1137]    `tf.train.Saver(write_version=tf.train.SaverDef.V2)`
WARNING:tensorflow:now on by default.
W0518 15:41:08.511476 15588 saver.py:1138] now on by default.
WARNING:tensorflow:*******************************************************
W0518 15:41:08.512475 15588 saver.py:1139] *******************************************************
Windows fatal exception: access violation

Current thread 0x00003ce4 (most recent call first):
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\client\session.py", line 1429 in _call_tf_sessionrun
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\client\session.py", line 1341 in _run_fn
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\client\session.py", line 1356 in _do_call
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\client\session.py", line 1350 in _do_run
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\client\session.py", line 1173 in _run
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\client\session.py", line 950 in run
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\training\saver.py", line 1173 in save
  File "pretrain.py", line 244 in train
  File "pretrain.py", line 251 in main
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\absl\app.py", line 250 in _run_main
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\absl\app.py", line 299 in run
  File "C:\Users\Alice\Miniconda3\envs\tflearning\lib\site-packages\tensorflow\python\platform\app.py", line 40 in run
  File "pretrain.py", line 254 in <module>

mulword_map，poemyun这两个dict，在yun.py中使用到了，但是它们的构建部分没有找到？
得到每句的韵的规则为什么是先找two_words，然后再找three_words？
您的平声字典，仄声字典是从哪里找到资源的呢？还有FunctionWords.txt fchar.txt goodwords.txt这三个都是自己总结出来的还是采用了前人发布的？
关于绝句的四种句式，在GL.py中出现了很多种超出句式范围的情况为什么也能把它们划分进那四种里面？
tfidf或textrank是从另外一份绝句语料中训练得到的吗？还是就是corpus.txt？

关于generate_base.py的问题

您好！请问generate_base.py的228行~234行用的indices是否与原先不对应，这样用indices索引出的结果是没有意义的？因为indices是新的候选序列的下标列表。

关于tfidf.txt

您好，没太明白这个文件中每个词的tfidf是怎么来的。看tfidf的定义，不是要指定一篇文章，这篇文章中的某个词的重要度吗？但是这里的话就是一个词，一个tfidf。希望大佬能解释下，谢谢~

xiaoyuanyi / wmpoetry Goto Github PK

wmpoetry's People

Stargazers

Watchers

Forkers

wmpoetry's Issues

Python3 didn't work at the end of pre-training And pre-model didn't save successfully

System information

My problem

aaabbbccc

这个模型真的超级复杂代码逻辑看不懂^_^我都想写个pytorch版本

分词工具可以一并开源吗

请教预处理部分的几个问题

关于generate_base.py的问题

关于tfidf.txt

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent