markbaggett / apiify Goto Github PK

Hi,
Any ways to trigger the removal/expiry/deletion of a specific entry from the cache regardless of the cache size or the cache expiry date?

I am testing using Apiify to cache dynamic information and have been looking for a way to automatically remove the entry from the cache.

Let's say I am caching Mac address - IP address - Gateway based on DHCP logs and would like to delete the cached entry when the IP address is assigned to a different mac-address. Something like: http://{apiify_svr_ip}:{port}/delete/{key}? To force Apiify to go get a more up-to-date value next time http://{apiify_svr_ip}:{port}/{key} is called.

Thanks

I have been using this awesome tool to alltraceroute info the logs and it is working great exept for some performance issues. I would also like to use a second web server based on this tool to perform another type of lookup while listening to a different port.

I though cloning another copy and providing it with another command and port to listen to would do just that. But when trying to start the second web server, I get the following error:

Traceback (most recent call last):
  File "/opt/apiify_instance02/apiify.py", line 123, in <module>
    server = ThreadedApiIfy((config['local_address'], config['local_port']), apiify)
  File "/opt/apiify_instance02/apiify.py", line 117, in __init__
    http.server.HTTPServer.__init__(self, *args, **kwargs)
  File "/usr/lib/python3.8/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/usr/lib/python3.8/http/server.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/usr/lib/python3.8/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use

The Server address is the same with the first instance but the listening port is different. I have tried to change the address 0.0.0.0 but I get the same error. I have also tries binding varies different ports but it won't let me start the second web server while for first one is running.

Also, as far as performance goes, is there a way to customize the threads of this tool like it is possible to do with domain-stats? I find that this tool struggles to keep up with our requests volume and I was wondering if there was a way to assign it more resources.