osperlabs / logbeam Goto Github PK

I have installed the latest logbeam 1.0.1.

pip3.6 install logbeam --user

I tried to run the sample code mentioned on the pypi repo, However, I am getting import error.

I am running the sample code as python3.6 testlogbeam.py

The default configuration of logbeam that is passed to cwlogs is the following:

buffer_duration = 10000
batch_count = 10
batch_size = 1024 * 1024

Default AWS's configuration is:
buffer_duration = 5000
batch_count = 1000
batch_size = 32768

logbeam sets maximum batch_size but sets batch_count too low which may lead (and does in our case) to slow log queue processing and excessive memory usage. It may be beneficial to set default batch_count to at least 1000.

I have following code :

import logging from logbeam import CloudWatchLogsHandler cw_handler = CloudWatchLogsHandler( log_group_name='my_log_group', log_stream_name='my_log_stream', buffer_duration=10000, batch_count=10, batch_size=1048576 )

Caught exception: An error occurred (ResourceNotFoundException) when calling the PutLogEvents operation: The specified log group does not exist.

If you create a logbeam logging handler on a machine that has no Internet access, an error like the following occurs while the application closes:

Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.6.4_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/local/Cellar/python3/3.6.4_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/terris/science37/edc-export-service/edcexport/main.py", line 16, in
EDCExportService().start()
File "/Users/terris/science37/edc-export-service/edcexport/edcexport.py", line 75, in start
loop.run_forever()
File "/usr/local/Cellar/python3/3.6.4_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 421, in run_forever
self._run_once()
File "/usr/local/Cellar/python3/3.6.4_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 1395, in _run_once
event_list = self._selector.select(timeout)
File "/usr/local/Cellar/python3/3.6.4_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/selectors.py", line 577, in select
kev_list = self._kqueue.control(None, max_ev, timeout)
KeyboardInterrupt
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.6.4_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/logging/init.py", line 1944, in shutdown
h.close()
File "/Users/terris/venv/edc/lib/python3.6/site-packages/logbeam/init.py", line 160, in close
self.sink.shutdown()
AttributeError: 'CloudWatchLogsHandler' object has no attribute 'sink'

logbeam currently uses awscli-1.9.8. I removed awscli and installed the latest version (1.14.30). logbeam worked fine but perhaps there are issues that I am not aware of.