rate limit the API call about safebrowsing HOT 10 CLOSED

Hi, can you elaborate? Currently, the logic avoids any API calls over the network if the result is in the local cache.

Are you talking about a general rate-limiter for network traffic?

from safebrowsing.

I am talking about the network traffic in making the API call.
The scenario I am talking about is, if there are more than 10k unique partial hash hit and these are not in the local cache then we would make API call. At this point if the API key has a rate limit of 10K. It would exceed and get blocked.

from safebrowsing.

I see. In your usage of LookupURLs, are you passing in a batch of URLs every time you call it?

Before implementing any rate limiting logic, I would much rather see the TODO in that function be addressed. As it currently is, we make a network call for each URL, rather than batching them together.

On the other hand, if you have many goroutines calling LookupURLs independently with only a small handful of URLs, then some rate limiting mechanism may be needed.

from safebrowsing.

I am currently not batching the URLs to LookupURLs(which is on my TODO). Also I have a scenario where I have multiple goroutines/instance of the client running which would still need a mechanism for rate limiting across them.

from safebrowsing.

Batching of the URLs is going to be done any time soon or should I submit a PR?

from safebrowsing.

You're welcome to submit a PR (assuming you sign the CLA) and I can review it.

from safebrowsing.

I have a PR ready for the batching of the partial hashes. Can you please take a look at it @dsnet .

from safebrowsing.

@dsnet can you please let me know how to get this reviewed and merged in to master branch.

from safebrowsing.

Hi @rajnmithun, I apologize for the delay. I will try to take a look at this as soon as possible!

from safebrowsing.

@alexwoz thank you.

from safebrowsing.