hzy46 / tensorflow-time-series-examples Goto Github PK

Does the prediction fit for time series with an obvious linear trend plus some periodic term? Such as y=x+sin(x/50)+noise; It seems the Least Square (LS) can do much better than the TFTS in this case. I am not sure whether I use the TFTS correctly.

great code thanks
may you clarify :
will it work for multivariate time series prediction both regression and classification
1
where all values are continues values
weight height age target
1 56 160 34 1.2
2 77 170 54 3.5
3 87 167 43 0.7
4 55 198 72 0.5
5 88 176 32 2.3

2
or even will it work for multivariate time series where values are mixture of continues and categorical values
for example 2 dimensions have continues values and 3 dimensions are categorical values

color        weight     gender  height  age  target 

1 black 56 m 160 34 yes
2 white 77 f 170 54 no
3 yellow 87 m 167 43 yes
4 white 55 m 198 72 no
5 white 88 f 176 32 yes

Traceback (most recent call last):
File "D:/YINGYONG/pycharm/pycharmProfessional/project/venv/Lib/site-packages/tensorflow/contrib/timeseries/python/timeseries/state_management.py", line 23, in
from tensorflow.contrib.timeseries.python.timeseries import feature_keys
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib_init_.py", line 98, in
from tensorflow.contrib import timeseries
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib\timeseries_init_.py", line 40, in
from tensorflow.contrib.timeseries.python.timeseries import *
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib\timeseries\python_init_.py", line 22, in
from tensorflow.contrib.timeseries.python.timeseries import *
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib\timeseries\python\timeseries_init_.py", line 24, in
from tensorflow.contrib.timeseries.python.timeseries.ar_model import *
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib\timeseries\python\timeseries\ar_model.py", line 24, in
from tensorflow.contrib.timeseries.python.timeseries import model
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib\timeseries\python\timeseries\model.py", line 27, in
from tensorflow.contrib.timeseries.python.timeseries import estimators as ts_estimators
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib\timeseries\python\timeseries\estimators.py", line 27, in
from tensorflow.contrib.timeseries.python.timeseries import state_management
File "D:\YINGYONG\pycharm\pycharmProfessional\project\venv\lib\site-packages\tensorflow\contrib\timeseries\python\timeseries\state_management.py", line 25, in
from tensorflow.contrib.timeseries.python.timeseries.model import ModelOutputs
ImportError: cannot import name 'ModelOutputs'

Process finished with exit code 1

平台：ubuntu 16.04
tensorflow版本： tensorflow-gpu-1.4
报错如下

WARNING:tensorflow:Skipping summary for observed, must be a float, np.float32, np.int64, np.int32 or int.
WARNING:tensorflow:Skipping summary for times, must be a float, np.float32, np.int64, np.int32 or int.
WARNING:tensorflow:Skipping summary for mean, must be a float, np.float32, np.int64, np.int32 or int.
WARNING:tensorflow:Skipping summary for start_tuple, must be a float, np.float32, np.int64, np.int32 or int.
WARNING:tensorflow:Input graph does not contain a QueueRunner. That means predict yields forever. This is probably a mistake.
2018-06-09 16:57:18.216300: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1060, pci bus id: 0000:01:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from /tmp/tmpwvnv1vj8/model.ckpt-2000
2018-06-09 16:57:18.318936: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319003: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319028: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319078: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319125: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319148: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319168: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319191: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319238: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319260: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319306: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319343: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319377: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319399: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319419: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319441: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319461: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319483: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319502: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
2018-06-09 16:57:18.319518: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: Reached limit of 1
[[Node: limit_epochs_2/CountUpTo = CountUpToT=DT_INT64, _class=["loc:@limit_epochs_2/epochs"], limit=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Traceback (most recent call last):
File "/home/jarlan/PycharmProjects/time_serier/train_lstm.py", line 199, in
plt.savefig('predict_result.jpg')
File "/home/jarlan/.local/lib/python3.5/site-packages/matplotlib/pyplot.py", line 710, in savefig
res = fig.savefig(*args, **kwargs)
File "/home/jarlan/.local/lib/python3.5/site-packages/matplotlib/figure.py", line 2035, in savefig
self.canvas.print_figure(fname, **kwargs)
File "/home/jarlan/.local/lib/python3.5/site-packages/matplotlib/backend_bases.py", line 2263, in print_figure
**kwargs)
File "/home/jarlan/.local/lib/python3.5/site-packages/matplotlib/backends/backend_agg.py", line 584, in print_jpg
return background.save(filename_or_obj, format='jpeg', **options)
File "/usr/lib/python3/dist-packages/PIL/Image.py", line 1675, in save
save_handler(self, fp, filename)
File "/usr/lib/python3/dist-packages/PIL/JpegImagePlugin.py", line 708, in _save
ImageFile._save(im, fp, [("jpeg", (0, 0)+im.size, 0, rawmode)], bufsize)
File "/usr/lib/python3/dist-packages/PIL/ImageFile.py", line 480, in _save
e = Image._getencoder(im.mode, e, a, im.encoderconfig)
File "/usr/lib/python3/dist-packages/PIL/Image.py", line 431, in _getencoder
return encoder(mode, *args + extra)
TypeError: integer argument expected, got float

Hi,
i'm trying train_csv.py, but when i run i have the error in the object at this row

data = sess.run(data)

this is the console error

Traceback (most recent call last): File "timeSeries.py", line 50, in <module> tf.app.run() File "/Users/andreaspindox/Documents/Progetti/regressionTest/regressionTest/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run _sys.exit(main(_sys.argv[:1] + flags_passthrough)) File "timeSeries.py", line 20, in main data = sess.run(data) File "/Users/andreaspindox/Documents/Progetti/regressionTest/regressionTest/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 889, in run run_metadata_ptr) File "/Users/andreaspindox/Documents/Progetti/regressionTest/regressionTest/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1120, in _run feed_dict_tensor, options, run_metadata) File "/Users/andreaspindox/Documents/Progetti/regressionTest/regressionTest/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run options, run_metadata) File "/Users/andreaspindox/Documents/Progetti/regressionTest/regressionTest/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError

To export estimator m there are four steps:
1.Define estimator's features.

2.Create a feature config.

3.Build an export_input_fn suitable for use in serving.

4.Export the model using export_savedmodel().

I try to use ：
export_dir_base = "./serving_save_model"
feature_spec = {
'times': tf.placeholder(tf.float32, name='times')
}
serving_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)

estimator.export_savedmodel(export_dir_base, serving_input_fn)
But I encountered an error like this:

Traceback (most recent call last):
File "E:/MyProject/Py/tensorFlow_time_series_predict/train_lstm_multivariate.py", line 231, in
estimator.export_savedmodel(export_dir_base, serving_input_fn)
File "H:\ProgramFiles\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\estimator\estimator.py", line 504, in export_savedmodel
serving_input_receiver = serving_input_receiver_fn()
File "H:\ProgramFiles\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\estimator\export\export.py", line 142, in serving_input_receiver_fn
features = parsing_ops.parse_example(serialized_tf_example, feature_spec)
File "H:\ProgramFiles\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\parsing_ops.py", line 577, in parse_example
[VarLenFeature, SparseFeature, FixedLenFeature, FixedLenSequenceFeature])
File "H:\ProgramFiles\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\parsing_ops.py", line 291, in _features_to_raw_params
raise ValueError("Invalid feature %s:%s." % (key, feature))
ValueError: Invalid feature times:Tensor("times:0", dtype=float32).

How should I use it correctly?
Thanks so much！

from tensorflow.contrib.timeseries.python.timeseries.estimators import TimeSeriesRegressor 前面应该加’‘。

INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpzXWlhI
INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_task_type': 'worker', '_global_id_in_cluster': 0, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fc8d7933650>, '_evaluation_master': '', '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_service': None, '_num_ps_replicas': 0, '_tf_random_seed': None, '_master': '', '_device_fn': None, '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_model_dir': '/tmp/tmpzXWlhI', '_train_distribute': None, '_save_summary_steps': 100}
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpzXWlhI/model.ckpt.
ERROR:tensorflow:Model diverged with loss = NaN.

NanLossDuringTrainingErrorTraceback (most recent call last)
in ()
32 optimizer=tf.train.AdamOptimizer(0.001))
33
---> 34 estimator.train(input_fn=train_input_fn, steps=200)
35 evaluation_input_fn = tf.contrib.timeseries.WholeDatasetInputFn(reader)
36 evaluation = estimator.evaluate(input_fn=evaluation_input_fn, steps=1)

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.pyc in train(self, input_fn, hooks, steps, max_steps, saving_listeners)
364
365 saving_listeners = _check_listeners_type(saving_listeners)
--> 366 loss = self._train_model(input_fn, hooks, saving_listeners)
367 logging.info('Loss for final step: %s.', loss)
368 return self

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.pyc in _train_model(self, input_fn, hooks, saving_listeners)
1117 return self._train_model_distributed(input_fn, hooks, saving_listeners)
1118 else:
-> 1119 return self._train_model_default(input_fn, hooks, saving_listeners)
1120
1121 def _train_model_default(self, input_fn, hooks, saving_listeners):

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.pyc in _train_model_default(self, input_fn, hooks, saving_listeners)
1133 return self._train_with_estimator_spec(estimator_spec, worker_hooks,
1134 hooks, global_step_tensor,
-> 1135 saving_listeners)
1136
1137 def _train_model_distributed(self, input_fn, hooks, saving_listeners):

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.pyc in _train_with_estimator_spec(self, estimator_spec, worker_hooks, hooks, global_step_tensor, saving_listeners)
1334 loss = None
1335 while not mon_sess.should_stop():
-> 1336 _, loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss])
1337 return loss
1338

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.pyc in run(self, fetches, feed_dict, options, run_metadata)
575 feed_dict=feed_dict,
576 options=options,
--> 577 run_metadata=run_metadata)
578
579 def run_step_fn(self, step_fn):

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.pyc in run(self, fetches, feed_dict, options, run_metadata)
1051 feed_dict=feed_dict,
1052 options=options,
-> 1053 run_metadata=run_metadata)
1054 except _PREEMPTION_ERRORS as e:
1055 logging.info('An error was raised. This may be due to a preemption in '

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.pyc in run(self, *args, **kwargs)
1142 raise six.reraise(*original_exc_info)
1143 else:
-> 1144 raise six.reraise(*original_exc_info)
1145
1146

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.pyc in run(self, *args, **kwargs)
1127 def run(self, *args, **kwargs):
1128 try:
-> 1129 return self._sess.run(*args, **kwargs)
1130 except _PREEMPTION_ERRORS:
1131 raise

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.pyc in run(self, fetches, feed_dict, options, run_metadata)
1207 results=outputs[hook] if hook in outputs else None,
1208 options=options,
-> 1209 run_metadata=run_metadata))
1210 self._should_stop = self._should_stop or run_context.stop_requested
1211

/usr/local/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/basic_session_run_hooks.pyc in after_run(self, run_context, run_values)
633 if self._fail_on_nan_loss:
634 logging.error(failure_message)
--> 635 raise NanLossDuringTrainingError
636 else:
637 logging.warning(failure_message)

NanLossDuringTrainingError: NaN loss during training.

My understanding is that paramenters of neurals are shared in Multivariate prediction and they can learn some correlations between series. There is less training time in Multivariate prediction. I wonder if that's right. Could you please explain any basic principles of Multivariate prediction with LSTM or recommend related papers to me? Thank you.

TensorFlow 1.3.0

python train_lstm.py

Traceback (most recent call last):
File "train_lstm.py", line 173, in
estimator = ts_estimators.TimeSeriesRegressor(
AttributeError: 'module' object has no attribute 'TimeSeriesRegressor'

It seems TimeSeriesRegressor is in the master branch
in tensorflow/tensorflow/contrib/timeseries/python/timeseries/estimators.py

but the latest version 1.3.0 has it named _TimeSeriesRegressor

After I rename TimeSeriesRegressor to _TimeSeriesRegressor in train_lstm.py, I got this error

File "train_lstm.py", line 174, in
model=_LSTMModel(num_features=1, num_units=128),
TypeError: Can't instantiate abstract class _LSTMModel with abstract methods generate

Which version of TF do you use @hzy46 ?

The error information:
estimator = ts_estimators.TimeSeriesRegressor(
AttributeError: 'module' object has no attribute 'TimeSeriesRegressor'