yuyuz / metaqa Goto Github PK

First of all, thank you and your colleagues for your efforts in proposing the MetaQA dataset and making a huge contribution to subsequent research on knowledge graph question answering!

We recently discovered that some data in the MetaQA 1-hop test dataset have incomplete Groud Truth. The details are as follows:

For eg. qid 1444
Query: who acted in the movie [The 39 Steps]?
Ground Truth: ['Robert Donat', 'Madeleine Carroll']
But in KG, in addition to the above two actors, the movie The 39 Steps also has the following actors:

This error causes the accuracy rate to be falsely low when we use hit@1 as the criterion.
Attached are the 90 1-hop QA data that we have discovered so far that are suspected of this situation, for your reference.
1-hop_GT_missing.txt

Finally, thank you and your colleagues again for your hard work, and thank you for your precious time!

Hi,

I find no data under 3-hop/vanilla, 3-hop/ntm, 1-hop/vanilla and 1-hop/ntm folder.

Am I missing something in README?

I would like to follow this research direction to finally solve KBQA.

Thank, and great work!

In the implementation，how do you learn vector representation of the minimum subgraph?

Hello @yuyuz
I greatly appreciate it if you answer my question.
Each question has a topic entity which shows the main focus of the question. I would like to know which Entity Linking technique is used to identify the topic/main entities of MetaQA datasets.
I am looking forward to hearing from you.

Line 732 @ MetaQA/1-hop/vanilla/qa_dev.txt

Question 1:
what does [Harry Dean Stanton] star in

Answers:
Alien|Paris|Texas|Repo Man|Down Periscope|The Rose|Straight Time|Sonny|The Open Road|Man Trouble|One Magic Christmas|The Good Life|Twister|Fool for Love|Cockfighter

Corresponding facts:
Paris, Texas|starred_actors|Harry Dean Stanton
Paris, Texas|has_tags|harry dean stanton

Line 8795 @ MetaQA/1-hop/vanilla/qa_dev.txt

Question 2:
what movies can be described by [elizabeth taylor]

Answers:
Who's Afraid of Virginia Woolf?|Cat on a Hot Tin Roof|Giant|Cleopatra|A Place in the Sun|Suddenly|Last Summer|National Velvet|Reflections in a Golden Eye

Corresponding facts:
Suddenly, Last Summer|starred_actors|Elizabeth Taylor
Suddenly, Last Summer|has_tags|elizabeth taylor