dnguyengithub / multitaskais Goto Github PK

Hello, I run the python3 multitaskAIS.py --mode=train --logdir=./chkpt --bound=elbo --summarize_every=100 --latent_size=100 --batch_size=50 --num_samples=16 --learning_rate=0.0003 with the following problem

  File "...tensorflowpy36\lib\site-packages\tensorflow\python\client\session.py", line 300, in __init__
    fetch, allow_tensor=True, allow_operation=True))
  File "...tensorflowpy36\lib\site-packages\tensorflow\python\framework\ops.py", line 3475, in as_graph_element
    return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
  File "...tensorflowpy36\lib\site-packages\tensorflow\python\framework\ops.py", line 3567, in _as_graph_element_locked
    types_str))
TypeError: Can not convert a int64 into a Tensor or Operation.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/project/MultitaskAIS/python3/MultitaskAIS/multitaskAIS.py", line 55, in <module>
    runners.run_train(config)
  File "D:\project\MultitaskAIS\python3\MultitaskAIS\runners.py", line 327, in run_train
    log_step_count_steps=config.summarize_every) as sess:
  File "...tensorflowpy36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 508, in MonitoredTrainingSession
    stop_grace_period_secs=stop_grace_period_secs)
  File "...tensorflowpy36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 934, in __init__
    stop_grace_period_secs=stop_grace_period_secs)
  File "...tensorflowpy36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 648, in __init__
    self._sess = _RecoverableSession(self._coordinated_creator)
  File "...\tensorflowpy36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1122, in __init__
    _WrappedSession.__init__(self, self._create_session())
  File "...\tensorflowpy36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1127, in _create_session
    return self._sess_creator.create_session()
  File "...\tensorflowpy36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 812, in create_session
    hook.after_create_session(self.tf_sess, self.coord)
  File "C...\tensorflowpy36\lib\site-packages\tensorflow\python\training\basic_session_run_hooks.py", line 568, in after_create_session
    self._save(session, global_step)
  File "...\tensorflowpy36\lib\site-packages\tensorflow\python\training\basic_session_run_hooks.py", line 599, in _save
    self._get_saver().save(session, self._save_path, global_step=step)
  File "...\tensorflowpy36\lib\site-packages\tensorflow\python\training\saver.py", line 1143, in save
    global_step = training_util.global_step(sess, global_step)
  File ...\tensorflowpy36\lib\site-packages\tensorflow\python\training\training_util.py", line 69, in global_step
    return int(sess.run(global_step_tensor))
  File "...\tensorflowpy36\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
    run_metadata_ptr)
  File "...s\tensorflowpy36\lib\site-packages\tensorflow\python\client\session.py", line 1137, in _run
    self._graph, fetches, feed_dict_tensor, feed_handles=feed_handles)
  File "...\tensorflowpy36\lib\site-packages\tensorflow\python\client\session.py", line 471, in __init__
    self._fetch_mapper = _FetchMapper.for_fetch(fetches)
  File "...tensorflowpy36\lib\site-packages\tensorflow\python\client\session.py", line 271, in for_fetch
    return _ElementFetchMapper(fetches, contraction_fn)
  File ...\tensorflowpy36\lib\site-packages\tensorflow\python\client\session.py", line 304, in __init__
    (fetch, type(fetch), str(e)))
TypeError: Fetch argument 0 has invalid type <class 'numpy.int64'>, must be a string or Tensor. (Can not convert a int64 into a Tensor or Operation.)

, my environment is python3.6,tensorflow 1.13.1 is samilar with requirments.txt.

Look forward your reply.

NotFoundError (see above for traceback): Failed to create a NewWriteableFile: ./chkpt\elbo-ct_2017010203_10_20_train.pkl-data_dim-602-latent_size-100-batch_size-32\model.ckpt-0_tem
p_8225fa0c497740d988f4bb7823eb29d1/part-00000-of-00001.data-00000-of-00001.tempstate4670007497201137418 : The system cannot find the path specified.
; No such process

Hi guys,

When run python3 calculate_AIS_mean.py, it expose:

tensorflow.python.framework.errors_impl.NotFoundError: /users/local/dnguyen/Datasets/AIS_datasets/MarineC/MarineC_Jan2014_norm/MarineC_Jan2014_norm_train.pkl; No such file or directory

where can I get the pkl data? thanks.

@dnguyengithub, I'm trying to figure out how you are predicting trajectories when --mode traj_reconstruction.

• If I understand correctly there are two different prediction methods used. One for observed data, and one for hold-out sequence.
  • The reconstruction in the observed data, is simply a one-step ahead prediction based on the observed position (and other data) in a previous time step, t-1 (and hidden and latent variables from t-1).
  • The predictions made during the hold-out sample are prediction made multiple time steps into the future (depending on how far away they are from the last observed position). The predictions multiple time steps into the future is where the confusion arises.

If I understand correctly from your code you set the input array to zeros when you are in a holdout sample. Meaning the RNN depends on the hidden and latent state to make a prediction.
Why is the previous predicted position (and other data) not used as input for the next prediction? Can both methods be used to reconstruct trajectories over multiple time steps where data is missing? Is one approach better than the other?

Below is a summary of your code, to help me investigate how reconstructions are made. Most important is the runners.create_eval_graph function. Inside this function, the following snippet is where the distinction is made between regular and hold-out data. For regular data, the cur_inputs are used. When the data is supposed to be in the hold-out period, cur_inputs is changed to be equal to while_samples.

if config.missing_data:
     cur_inputs = tf.cond(tf.logical_and(t < max_seq_len - 6,t >= max_seq_len - 18),
         lambda: while_samples, 
         lambda: cur_inputs)

The while_samples are an input for the function while_step. When the while_step function is called, while_samples input is defined as init_targets. In its turn init_targets is a tuple of two tf.zeros arrays. Why is init_targets a tuple of target_sampled0 and target_true0 (the two tf.zeros arrays)? And not based on the previous predictions for example.

##############################
# runners.py
##############################
def create_eval_graph(inputs, targets, lengths, model, config):

	max_seq_len = tf.reduce_max(lengths)
	...
	target_sampled0 = tf.zeros(shape = [batch_size*num_samples, config.data_dim], dtype = tf.float32)
        target_true0 = tf.zeros(shape = [batch_size*num_samples, config.data_dim], dtype = tf.float32)
        init_targets = (target_sampled0, target_true0)
        def while_step(t, rnn_state, tas, accs, while_samples):
		...
		if config.missing_data:
                    cur_inputs = tf.cond(tf.logical_and(t < max_seq_len - 6,t >= max_seq_len - 18),
                                 lambda: while_samples, 
                                 lambda: cur_inputs)
        ...
	_, _, tas, accs, new_sample = tf.while_loop(while_predicate,
                                    while_step,
                                    loop_vars=(t0, init_states, tas, accs, init_targets),
                                    parallel_iterations=parallel_iterations,
                                    swap_memory=swap_memory)
    return track_sample, track_true, log_weights, ll_per_t, final_log_weights/tf.to_float(lengths), rnn_state_tf, rnn_latent_tf, rnn_out_tf

def create_dataset_and_model(config, split, shuffle, repeat):
    inputs, targets, lengths, mmsis, mean = datasets.create_AIS_dataset(...)
    ...
    return inputs, targets, mmsis, lengths, model

###############################
# multitaskAIS.py
###############################
if config.mode in ['save_outcomes','traj_reconstruction']:
	...
	inputs, targets, mmsis, lengths, model = runners.create_dataset_and_model(config, config.split, shuffle=False, repeat=False)
	if config.mode =='traj_reconstruction'
		config.missing_data = True

	track_sample, track_true, log_weights, ll_per_t, ll_acc,_,_,_\
            = runners.create_eval_graph(inputs, targets, lengths, model, config)
    ...
...
if ...
elif config.mode =='traj_reconstruction'
    ...
    save_dir = ...
    ...
    for d_i in tqdm(list(range(dataset_size))):
         tar, mmsi, dense_sample, ll_t, ll_tracks = sess.run([targets, mmsis, track_sample, ll_per_t, ll_acc])
         if len(tar) < config.min_duration:
            continue
        
        sparse_tar = np.nonzero(np.squeeze(tar))[1].reshape(-1,4)
        for d_i_sample in range(config.num_samples):
        	...
        	plt.plot(sparse_tar[-18:-6,1],sparse_tar[-18:-6,0],'ro')
        	...
        	logit_lat = np.argmax(dense_sample[:,d_i_sample,0:config.lat_bins], axis = 1)
        	logit_lon = np.argmax(dense_sample[:,d_i_sample,config.lat_bins:config.lat_bins+config.lon_bins], axis = 1) + config.lat_bins
        	plt.plot(logit_lon[1:],logit_lat[1:],'b')
        	plt.plot(logit_lon[-17:-5],logit_lat[-17:-5],'r')

In the interpolation function interpolate( ), defined in utils.py, COG and HEADING are interpolated as follows:

course_interp = (track[apos,COG] - track[bpos,COG] )*(dt_interp/dt_full) + track[bpos,COG]
heading_interp = (track[apos,HEADING] - track[bpos,HEADING])*(dt_interp/dt_full) + track[bpos,HEADING]

This updates the before position with a weighted difference between 'after' and 'before'.

The issue arises when the angle (in degrees) goes through north. E.g. when moving from 350 to 10: (10-350)*.5 + 350 = 180 (instead of 0). The above implementation goes the long way round the compass.

Hi @dnguyengithub ,

I'm trying to implement your proposed solution on another AIS dataset but I'm facing a few problems in the preprocessing steps:

• I'm having a hard time understanding this whole sampling process. In the paper you present the problem that we need regularly sampled data to train the model first. However, I am not sure I fully understand your proposed solution (i.e. using constant velocity model to create regularly time-sampled AIS tracks). Moreover, if I understand your solution correctly, it sounds rather odd to generate regularly sampled data in such an artificial manner. I would really appreciate some explanation on this topic.

• I'm a bit concerned about your train-test splitting strategy. In the code I can see that you split your data based on random permutation rather based on continuous time intervals. Doesn't it beat the purpose of learning sequences patterns from the past and check if they are valid in the future?

• In the paper you mention that you only chose tracks which the maximum time interval between two successive received AIS messages is 1 hour. However, in your code I see that you split the tracks using 2 hours interval. What is the correct interval value?

• In the paper you mention that you created regulary time-sampled AIS tracks at 10 minutes time scale. However, in your code I see that you used a 5 minutes sampling resolution. What is the correct interval value?

Any help with one or all of these questions will be much appreciated.

Thanks in advance.

How do you calculate the bin values for LAT, LON, SOG, and COG in the script calculate_AIS_mean.py?

Hello, @dnguyengithub I was trying to adapt this framework to a new dataset. While pre-processing I removed the ROT and NAV_STT and changed the length parameters wherever required accordingly to 7. I am getting an error when the reshape(-1,4) operation is applied for save_outcomes in line 116 specifically for the D["inp"] one If you have any leads can you tell me the reason?

Hi, Dear Teacher:
Firstly, please forgive me for my poor english. Then, I want to ask a question about the "Vessel type identification submodel".
As I am not understand the deep learning, so when I trying to run the "vessel_type_classification.py" program file, I find that the program need two pkl file which names are "MarineC_Jan2014_norm_train_rnn_state.pkl" and "MarineC_Jan2014_norm_test_rnn_state.pkl". I used the preprocessed "train.pkl" and "test.pkl" to instead them, but I am failed.
Are you mind provide the two files? Better if you could provide other solutions.

Thanks for your early reply.

Best Regards,
send by Seven

Hello,

Can you please provide the QGIS codes for conversion from raw format to AIS format?
Also if possible the scripts with libais?

@dnguyengithub, relating to #4 .
Within the file runners.py and the function create_eval_graph.
The function while_step is defined and called by tf.while_loop.

When it is called, one of the arguments passed is tas.

However, tas is not used inside while_step. Could it be that the following line inside while_step:

cur_inputs, cur_mask = nested.read_tas([inputs_ta, mask_ta], t)

should be changed to:

cur_inputs, cur_mask = nested.read_tas(tas, t)

???

Otherwise could you please explain why tas is not used. And inputs_ta, mask_ta are only defined once, outside of while_step?
Relating to #4 why they are changed to tf.zeros arrays when an observation belongs to the hold-out sample.

Hi, Dear Teacher:
Firstly, please forgive me for my poor english. Then, I want to ask a question about the "tensorflow 2.1.0 version".
The code is TensorFlow version 1, but how do I work with version 2?
Better if you could provide other solutions.

Thanks for your early reply.

Best Regards,
send by Gwanho

Hi,

I tried running the code in my system. Initially it threw the following error-

DuplicateFlagError: The flag 'log_dir' is defined twice. First from absl.logging, Second from flags_config. Description from first occurrence: directory to write logfiles

After that i tried to clear all the flags before each iterations.
Now it throws the followin error-
NotFoundError: NewRandomAccessFile failed to Create/Open: ./data/dataset8/dataset8_train.pkl : The system cannot find the path specified.
; No such process
Even I downloaded the file again and ran it . But the same error persists.

Would you please help me in getting this resolved

Hello,

First thank you for sharing your code, your paper is very interesting.
I went through your code and I could not seem to find where you deal with padded values.
I feel like the VRNN is being trained with these values so a bunch of tensor of size data_dim filled with 0. Sometimes in the same batch you may have some very large differences in term of sequence sizes, from 20 to 144 for instance.

Did I miss something or do we consider them as noisy data the VRNN has to deal with?
Thank you

Thanks, @dnguyengithub for answering all my queries patiently. In the file Contrario.py can you please tell me what's the numbers subtracting and dividing by 300 mean and how are they related to min lat, max lat, min long, max long. I understand that they are mapping indices properly but it's not very clear. Thank you.

@dnguyengithub we have a python 3 version of your code working. Let me know if you are interested in creating a python3 branch, then we can create a pull request.

Thank you @dnguyengithub, for providing the code of your paper.

When trying to reproduce your results using dataset8, the following questions arise:

1. What was the log-likelihood of your model on dataset8 (Ushant)?
   1. In the paper you indicate a log-likelihood of -6.5 for the Gulf of Mexico. For me the log-likelihood on the Ushant training seems to converge around -5 per timestep, but does not yield proper trajectories.
2. How long did you train for in updating steps and time?
   1. In the code you set the default for max_steps to int(1e9). Do you let it train until this limit or did you use early stopping?
   2. Furthermore, could you also give an indication of the training time (and hardware setup) that you had for both dataset8 and the marinecadastre data sets?

When trying to preprocess the MarineC data on Gulf of Mexico I came across the following:

dataset_preprocessing.py reads separate pickle files per zone, while the zones are grouped in the pickle file that is created by csv2pkl.py.

csv2pkl.py

• reads
  • Zone{ZZ}_2014_{MM}.csv
    • where MM (month) is hard-coded to 01
    • ZZ (zone) in [14,15,16]
• writes
  • "01_position.pkl"
    • is a list/dict containing filtered AIS broadcasts (grouped vessel's tracks)
    • where 01 resembles the first month.

dataset_preprocessing.py

• reads
  • "/Zone" + zone + "_2014_" + month + ".pkl"
    • month = "01"
    • zone in ['14','15','16']

@dnguyengithub thank you for sharing the code of your paper.
Would you be able to add a requirements.txt for the python enviroment?

Hello, Is there a PyTorch implementation of this?

Traceback (most recent call last):
File "multitaskAIS.py", line 66, in
from flags_config import config
File "C:\Users\Admin\Downloads\MultitaskAIS-master\MultitaskAIS-master\flags_config.py", line 195, in
config.onehot_lat_bins = math.ceil((config.lat_max-config.lat_min)/config.onehot_lat_reso)
File "C:\Users\Admin\anaconda3\envs\MultitaskAIS-master\lib\site-packages\tensorflow\python\platform\flags.py", line 84, in getattr
wrapped(_sys.argv)
File "C:\Users\Admin\anaconda3\envs\MultitaskAIS-master\lib\site-packages\absl\flags_flagvalues.py", line 670, in call
name, value, suggestions=suggestions)
absl.flags._exceptions.UnrecognizedFlagError: Unknown command line flag 'test_name'

data_preprocessing.py and flags_config.py define:
LAT_MIN = 26.5
LAT_MAX = 30.0
LON_MIN = -97.5
LON_MAX = -87

while csv2pkl.py uses:
LAT_MIN = 27
LAT_MAX = 30.0
LON_MIN = -98
LON_MAX = -88

Given the figures in the paper, I assume the bounds defined in data_preprocessing.py and flags_config.py are correct and csv2pkl.py needs to be adjusted?