imd,xgfs

Hey! I am a Research Scientist at Google Research, NYC. I did my Ph.D. at the University of Bonn. I am interested in scalable, principled methods for analyzing graph data.

You can find more about my work on Google Scholar and personal website. Follow me on Twitter to get the latest updates.

imd's People

Contributors

Stargazers

Watchers

imd's Issues

Different results on the same arrays

Hi, thanks for your good work.

I would like to try MSID for GANs evaluation, but found that the metric is extremely unstable. I just clone your repo and perform a simple experiment:

>>> import numpy as np
>>> from msid import msid_score
>>> x0 = np.random.randn(1000, 10)
>>> x1 = np.random.randn(1000, 10)
>>> for _ in range(20):
...     print('MSID(x0, x1)', msid_score(x0, x1))
...

MSID(x0, x1) 11.612343854772956
MSID(x0, x1) 7.671366682093675
MSID(x0, x1) 1.8117880712326395
MSID(x0, x1) 6.205967034975149
MSID(x0, x1) 1.9430385102291492
MSID(x0, x1) 2.467981390832042
MSID(x0, x1) 4.359253678580822
MSID(x0, x1) 5.705092418121339
MSID(x0, x1) 7.084854325912502
MSID(x0, x1) 8.925101261419211
MSID(x0, x1) 2.6563495105769963
MSID(x0, x1) 6.67076587871034
MSID(x0, x1) 0.9609276170219742
MSID(x0, x1) 4.134198699891847
MSID(x0, x1) 2.061919358106404
MSID(x0, x1) 5.4849779235186045
MSID(x0, x1) 2.5738576367295107
MSID(x0, x1) 3.597934029471011
MSID(x0, x1) 1.0966421686877845
MSID(x0, x1) 13.116242604321098

Next, I provide the results of MSID(x0, x0)

>>> for _ in range(20):
...     print('MSID(x0, x0)', msid_score(x0, x0))
...
MSID(x0, x0) 1.8842238243396114
MSID(x0, x0) 6.653959832884025
MSID(x0, x0) 2.896296044612713
MSID(x0, x0) 1.7874406866486243
MSID(x0, x0) 2.212118637843133
MSID(x0, x0) 5.352864291155722
MSID(x0, x0) 4.492301054567285
MSID(x0, x0) 1.3662656634830224
MSID(x0, x0) 2.2663591630199416
MSID(x0, x0) 4.5750399290303045
MSID(x0, x0) 4.094359241800621
MSID(x0, x0) 2.4488511702991795
MSID(x0, x0) 5.929584568192836
MSID(x0, x0) 7.591811322838174
MSID(x0, x0) 7.372357733571717
MSID(x0, x0) 5.6968201645123075
MSID(x0, x0) 1.4797792557903116
MSID(x0, x0) 1.1783656760547234
MSID(x0, x0) 7.6904604926511295
MSID(x0, x0) 5.483936755815125

I expect that a metric for GANs evaluation should: