geek-ai / irgan Goto Github PK

也就是生成https://github.com/geek-ai/irgan/tree/master/Question-Answer/model
里的文件的代码在哪？
也就是pre-trained.model文件如何生成？
多谢多谢多谢！
@LantaoYu @wabyking @wnzhang

I notice that in the code of item recommendations, both dynamic negative sampling and irgan generator is trained by a single user one iteration. In my perspective of view, this training strategy is same to setting the training batch size as 1, which is not quite weird for me. Could you please explain the reason why you choose the training strategy that way? Thanks.

Hi @wabyking @wabyking
I found there is an initial model in your github: dns_nn.model.
Can you tell me how to generate(train) it?
Another question is, can you give me some instructions about how to train IRGAN? Seems you just show the method to evaluate the stored models.

Thanks,
Yingjun

这个ndcg算的有问题把，没有考虑2，1这种关系啊

你好，我想了解实验环境，因为用python3，python2.7以及python2.5运行不兼容的地方太多，修改的有些吃力。
类似python的实验版本，TensorFlow的版本，还有是否用到了keras等工具以及具体的版本。
问的问题有些小白，望不吝赐教！

IndexError Traceback (most recent call last)
in
----> 1 vocab = build_vocab()
2 alist = read_alist()
3 raw = read_raw()

in build_vocab()
8 items = line.strip().split(' ')
9 for i in range(2, 3):
---> 10 words = items[i].split('_')
11 for word in words:
12 if not word in vocab:

IndexError: list index out of range

Hi, I've tried run you code about movielens-100k. But it is a really small dataset. When i try to train on Netflix with the same setting, it gives really bad performance and take a long time. Can you provide the training script of netflix?

When running ltr_gan_d_nn_g_nn.py, a problem occured:
"/irgan-master/ltr-gan/ltr-gan-pointwise/dis_model_pointwise_nn.py", line 40, in init
self.loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(self.pred_score, self.pred_data_label)
ValueError: Only call sigmoid_cross_entropy_with_logits with named arguments (labels=..., logits=..., ...)
I guess this is caused by changes in new version of Tensorflow. Can you confirm that?

In item_recommendation, in the training process of discriminator, it seems need to clear the cache of old file, since you use linechache in ut.get_batch_data.

for d_epoch in range(50):
    if d_epoch % 5 == 0:
        generate_for_d(sess, generator, DIS_TRAIN_FILE)
        # whether to clear cache for old file? It seems need to add below two line code.
        import linecache
        linecache.clearcache()
        train_size = ut.file_len(DIS_TRAIN_FILE)
    index = 1
    while True: 
        # ut.get_batch_data to generate a minibatch
        # train discriminator for this minbatch

in cf_gan.py, the code between 194~198 as follows:

pn = (1 - sample_lambda) * prob
pn[pos] += sample_lambda * 1.0 / len(pos)
sample = np.random.choice(np.arange(ITEM_NUM), 2 * len(pos), p=pn)

this way weights the pos sample of each user, which is not described in paper. and whether there exists some other sample way which can get same or even better results?
in addition, in some scenario, there might be more than millions user and 10k items.So, there is any rough evaluation of training time？

请教一下

1. 既然QA的G产生的NEG answer来自候选里面的根据CNN模型挑选的一部分。如果把全部的候选都给D做判别效果是否会更好？只是比较花时间。
   比如 从示例代码中，G的过程中，从100个里面用CNN识别后选出了5个，再给D用CNN训练这个5个。为什么不直接把这100个都拿去给D训练？（花的时间更少？）

看到代码中 generator loss 所需的 reward 有下面各种表达形式：
reward = 2 * (tf.sigmoid(...) - 0.5)
reward = tf.sigmoid(tf.maximum(0.0, 1.0 - (pos_score - neg_score)))
reward = tf.log(tf.sigmoid(neg_score - pos_score))

但是没有一处和paper中叙说的一样：
... the term **log(1+exp(f_φ(d, q_n))) acts as the reward ** for the policy
p_θ(d|q_n, r) taking an action d in the environment q_n [38].
In order to reduce variance during the REINFORCE learning, we
also replace the reward term log(1+exp(f_φ(d, q_n))) by its advantage
function: log(1+exp(f_phi(d, q_n))) + E_{d~p_θ(d|q_n, r)}log(1+exp(f_φ(d, q_n))) ...

为什么

感谢给出论文的示例代码. 我有几点疑惑. 如果有空儿希望能帮忙回答下,十分感谢.

关于 ltr-gan/ltr-gan-pointwise/ 目录下的 ltr_gan_d_nn_g_nn.py
变量 query_index_url 给定某个query时,对应的list应该是其对应的所有样本, 包括正负. 那么为什么, 在函数 generate_for_d 中采负样本的时候, 没有排除正样本? 也就是说可能会以一定的概率将正例采样成负例?

generate_for_d函数和generate_dns函数的关系是什么,为什么后者去除了正样本,而前者没有呢?

至于文件 Large_norm.txt. 为了了解MQ2008-semi这个数据集,我谷歌了一下,查到了以下两个网页,但是并没有查到上述文件.

1. http://www.bigdatalab.ac.cn/benchmark/bm/dd?data=MQ2008-semi
2. Introducing LETOR 4.0 Datasets

您好，请问为什么在paper里面没有正则项，但是在code里面看到有正则项呢？

Is the discriminator's goal to determine relevant vs. non-relevant query-document pairs, or true vs. generated query-document pairs? If the former is true, shouldn't the stopping criteria for IRGAN be the discriminator converging to 1 (relevant) everywhere, instead of 0.5? If the latter is true, shouldn't the generator perform positive sampling instead of negative sampling in order to fool the discriminator? Thanks in advance for the clarification!

Can you elaborate on this part of the code?
https://github.com/geek-ai/irgan/blob/master/ltr-gan/ltr-gan-pointwise/ltr_gan_d_nn_g_nn.py#L125-L129

I am trying to understand where the importance sampling came from - I do not see it mentioned anywhere in the papers for IRGAN, REINFORCE, GANs, RankNet, etc.

Is this just from Monte Carlo sampling? If so, can you provide a good reference?

Thanks!

After reading your code (item recommendation), I have several questions, and the first one maybe how you tune the best parameters. In the code you seems to pick the best one according to the test set. However, this will easily leads to overfitting.

Hello,

I have just read the code in Question-Answer/irgan.py and I have a question in line 264-268 in feed_dict. I think it should be:

generator.input_x_1: subsamples[:, 0]

instead of

generator.input_x_1: samples[:, 0]

Could you explain it ?

Thanks !

https://github.com/geek-ai/irgan/blob/master/Question-Answer/insurance_qa_data_helpers.py#L15

test1数据没有2, 3个item

怎么改，多谢多谢多谢

@LantaoYu @wabyking @wnzhang

Hello author, why do you use both train data and the test data when you pre-train your Discriminator part,i think it may lead to data-leak

There seems to be the following error that I am getting.

tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = 3440 is not in [0, 3231)
[[Node: output/embedding_lookup_2 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=> > ["loc:@embedding/embedding_W"], validate_indices=true, > _device="/job:localhost/replica:0/task:0/cpu:0"](embedding/embedding_W/read, _arg_input_x_3_0_2)]]

I have tried both Tensorflow version 1.3 with python 2 and Tensorflow 1.6 with python 3 (this one after making the necessary changes).

Any suggestions?

Thanks for sharing the code.
I wonder which version of tensorflow dis you use.
I have tensorflow-gpu (1.2.0), and running the baseline.py get the follow error,

load embeddings
Traceback (most recent call last):
  File "/home/hongzhi/wp/irgan/Question-Answer/baseline.py", line 229, in <module>
    main()
  File "/home/hongzhi/wp/irgan/Question-Answer/baseline.py", line 195, in main
    loss="pair")
  File "/home/hongzhi/wp/irgan/Question-Answer/Discriminator.py", line 15, in __init__
    learning_rate=learning_rate, embeddings=embeddings, loss=loss, trainable=trainable)
  File "/home/hongzhi/wp/irgan/Question-Answer/QACNN.py", line 70, in __init__
    q = self.getRepresentation(self.input_x_1)
  File "/home/hongzhi/wp/irgan/Question-Answer/QACNN.py", line 101, in getRepresentation
    pooled_reshape = tf.reshape(tf.concat(3, output), [-1, self.num_filters_total])
  File "/home/hongzhi/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1043, in concat
    dtype=dtypes.int32).get_shape(
  File "/home/hongzhi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 676, in convert_to_tensor
    as_ref=False)
  File "/home/hongzhi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 741, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/hongzhi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/hongzhi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/home/hongzhi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 374, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/home/hongzhi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

Hi,
IRGAN is a nice work. But the performance heavily depends on the choice of the sampling temperature (\tau in Eq.(10)). Does there any theoretical analysis or reference to this issue?

你好，这边在 CPU 上运行 QA 代码中的 baseline.py，报出 InvalidArgumentError 的错误，目前没有 GPU 的资源，不知道有什么方法可以暂时解决这个问题，感谢感谢：

First, thank you for excellent work.
I have some problems with it.
in dis_model.py 41 line:
self.pre_logits = tf.reduce_sum(tf.multiply(self.u_embedding, self.i_embedding), 1) + self.i_bias
I did not understand why use tf.multiply ?
I think use tf.matmul.
But your work about logits always use tf.multiply.
Please explain it. Thank you.

你好 我先问一下：采样的话必须用tf.gather这个函数吗。。直接让没有采到的那些样本乘于0行不行。。我试了一下好像直接乘与0会造成梯度为0的问题。。想问一下你们是不是也遇到过这个问题。 @

I encountered a error while trying to run the experiments of Question-Answering, I doubt it is the version of tensorflow that cause this error. Could you please tell me which version of tensorflow do you use. If tensorflow version is not the reason, could you please give me some advice on fixing this error. Thank!
Following is the Trace Back of this error:
Traceback (most recent call last):
File "baseline.py", line 226, in
main()
File "baseline.py", line 197, in main
loss="pair")
File "/gruntdata/public/tianxuan.jl/code/irgan/Question-Answer/Discriminator.py", line 12, in init
QACNN.init(self, sequence_length, batch_size,vocab_size, embedding_size,filter_sizes, num_filters, dropout_keep_prob=dropout_keep_prob,l2_reg_lambda=l2_reg_lambda,paras=paras,learning_rate=learning_rate,embeddings=embeddings,loss=loss,trainable=trainable)
File "/gruntdata/public/tianxuan.jl/code/irgan/Question-Answer/QACNN.py", line 70, in init
q =self.getRepresentation(self.input_x_1)
File "/gruntdata/public/tianxuan.jl/code/irgan/Question-Answer/QACNN.py", line 101, in getRepresentation
pooled_reshape = tf.reshape(tf.concat(3, output), [-1, self.num_filters_total])
File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1029, in concat
dtype=dtypes.int32).get_shape(
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 637, in convert_to_tensor
as_ref=False)
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 702, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 110, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 99, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

According to the paper, the objective of RL is to get a larger reward, if the output of the discriminative model gets larger, means that D see generated as a true one.
However, we found that there are some questions in the following code snippet :

self.gan_loss = -tf.reduce_mean(tf.log(self.i_prob) * self.reward) + self.lamda * ( tf.nn.l2_loss(self.u_embedding) + tf.nn.l2_loss(self.i_embedding) + tf.nn.l2_loss(self.i_bias))
Q1. Ignore the regulation, the tf.log(self.i_prob) should be negative and self.reward may be negative or positive, thus, when reward is positive the above function will minimize value of the reward, which is opposite to the RL design statement and concept of GAN.

Could you explain the reason why self.reward may be negative or positive?

Please correct me if there's anything wrong, thank you!

Would you mind telling me the Generative's loss function of Item Recommendation Method?thanks

In section 2.1 of the paper, the authors mention that the reward term is replaced with its advantage function. I have read the source code, but I still have some questions about the implementation of the variance reduction method.

1. how is it implemented in the code?
2. if the baseline function is set to a constant as I found (0.5 in the code ?), how this constant is obtained?
3. the parameters are updated with the training, should the baseline function be set to different values?
4. what if we just use simple policy gradient?

Look forward to your reply. Thanks.

I would like to ask why NDCG@5 is smaller than NDCG@10, because in terms of recommendation, in general, the bigger the topK, the bigger the NDCG.

我使用TF1.2.1，一些api的问题已经修改；但是在运行时遇到如下问题：
1、在CPU下运行baseline.py和irgan.py都会会报如下错误：

  
 InvalidArgumentError (see above for traceback): indices[0,11] = 3408 is not in [0, 3231)
[[Node: output_2/embedding_lookup_1 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@embedding_1/embedding_W"], validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](embedding_1/embedding_W, _arg_input_x_2_1_0_1)]]
   

2、在GPU（P40）下运行baseline.py报如下错误，感觉内存不够，缩小batch_size为50后可运行：

  
2018-04-17 19:35:50.616583: F tensorflow/core/framework/tensor_shape.cc:241] Check failed: size >= 0 (-6838710080 vs. 0)
   

3、在GPU（P40）下运行irgan.py仍然报如上错误，改小batch_size也不行，请问实验环境的配置要求多高？

谢谢。
@wabyking @LantaoYu @pangolulu

Hi,
After I and my superviser checked cafefully with your baseline.py, There are lots of issues happening. The description are the following:

1. in line 184, actually you did not provide the model mentioned.
2. in line 186, you did not claim Discriminator.Discriminator_pair_wise
3. so if I change the Discriminator.Discriminator_pair_wise into Discriminator only, error will appear in line 197 stating that "pair" is not recognized.

Could you help solve the above problems or provide a precise description?

I am looking forward to your response and thank you for your time,
Ryan

Hi~
In Question-Answer/irgan.py line 243, is there should be "sampled_index.remove(_index)" ? Or the generator.input_x_3 will sample both pos and neg, which will generate wrong neg samples for the discriminator.

In Question-Answer task, using the provided data. The vocabulary size of the provided pretrained QACNN model is 3231, but the vocabulary size calculated by "build_vocab" function is 3449.
This does not cause any error in tensorflow 0.12.0 of gpu version, however, it will report "index out of range" error when using a tensorflow 0.12.0 of cpu version.
According to tensorflow/tensorflow#5847. This is because when using tensorflow of cpu version, the "embedding_lookup" function will report error if any index is out of range in "embedding_lookup"function, but it will return zero vector for words whose index is out of range using tensorflow of gpu version. This is a small bug, since we usually use gpu to train deep model, and the words whose index out of range may be infrequent words, therefore, it does not have a great impact on model performace. It will be wonderful if it can be fixed.
Could you please provide a new pretrained QACNN model with correct vocabulary size? Or I can create a pull request to fix the bug in "build_vocab"?