karpathy / randomfun Goto Github PK

Very cool idea on using SVM for similarity search.

As an alternative, we could also use logistic regression with l1 penalty. Because of the induced sparsity, this gives us the benefit of only storing the subset of embedding dimensions that are relevant and so reduce storage and computation during inference.

Small nit: I think technically we would call these examples instances of Positive Unlabeled (PU) learning where our "negatives" are not labeled (but we assume some of them are positive matches against the query).

Hi,

In es.ipynb I noticed that your four Gaussian functions don't have braces around sigma**2, looks like this:

G1 = np.exp(-((X-mux)**2+(Y-muy)**2)/2.0*sigma**2)
instead of:
G1 = np.exp(-((X-mux)**2+(Y-muy)**2)/2.0*(sigma**2))

So you actually multiply by sigma**2 in the exponential function, instead of dividing as you would in a normal Gaussian function.

This doesn't have any consequence for the toy example, I'm just curious if it was intentional (there could be a reason of doing this that I'm not aware of)?