jschniper / gelf_logger Goto Github PK
View Code? Open in Web Editor NEWAn Elixir Logger backend for GELF
License: MIT License
An Elixir Logger backend for GELF
License: MIT License
We want to send all metadata to our server, without defining it first in a configuration.
The console Logger supports this option: metadata: :all
https://hexdocs.pm/logger/Logger.html
Could this be possible here too please?
I have gelf_logger setup as dependency as such {:gelf_logger, "~> 0.7.5"}
and my config/prod.exs
file is equivalent to the following:
use Mix.Config
# Do not print debug messages in production
config :logger,
level: :info,
backends: [:console, {Logger.Backends.Gelf, :gelf_logger}]
config :logger, :gelf_logger,
host: "gelf.internal.co",
port: "12201",
application: "my_app",
metadata: :all,
hostname: "my_app@${HOSTNAME}"
I've tried with the port both as an integer and a string. When I run my release PORT=8080 /opt/app/bin/my_app console
as it is starting up the following happens and it crashes:
=SUPERVISOR REPORT==== 26-Apr-2019::20:24:37.139709 ===
supervisor: {local,'Elixir.Logger.Supervisor'}
errorContext: start_error
reason: {'EXIT',
{{badmatch,
{error,
{{{case_clause,
{'EXIT',
{undef,
[{'Elixir.Logger.Backends.Gelf',init,
[{'Elixir.Logger.Backends.Gelf',gelf_logger}],
[]},
{gen_event,server_add_handler,4,
[{file,"gen_event.erl"},{line,473}]},
{gen_event,handle_msg,6,
[{file,"gen_event.erl"},{line,318}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,249}]}]}}},
[{'Elixir.Logger.Watcher',init,1,
[{file,"lib/logger/watcher.ex"},{line,28}]},
{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},
{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,249}]}]},
{child,undefined,
{'Elixir.Logger.Backends.Gelf',gelf_logger},
{'Elixir.Logger.Watcher',start_link,
[{'Elixir.Logger',
{'Elixir.Logger.Backends.Gelf',gelf_logger},
{'Elixir.Logger.Backends.Gelf',gelf_logger}}]},
transient,5000,worker,
['Elixir.Logger.Watcher']}}}},
[{'Elixir.Logger.BackendSupervisor','-start_link/1-fun-0-',2,
[{file,"lib/logger/backend_supervisor.ex"},{line,14}]},
{'Elixir.Enum','-reduce/3-lists^foldl/2-0-',3,
[{file,"lib/enum.ex"},{line,1940}]},
{'Elixir.Logger.BackendSupervisor',start_link,1,
[{file,"lib/logger/backend_supervisor.ex"},{line,13}]},
{supervisor,do_start_child_i,3,
[{file,"supervisor.erl"},{line,379}]},
{supervisor,do_start_child,2,
[{file,"supervisor.erl"},{line,365}]},
{supervisor,'-start_children/2-fun-0-',3,
[{file,"supervisor.erl"},{line,349}]},
{supervisor,children_map,4,
[{file,"supervisor.erl"},{line,1157}]},
{supervisor,init_children,2,
[{file,"supervisor.erl"},{line,315}]}]}}
offender: [{pid,undefined},
{id,'Elixir.Logger.BackendSupervisor'},
{mfargs,{'Elixir.Logger.BackendSupervisor',start_link,[[]]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]
=CRASH REPORT==== 26-Apr-2019::20:24:37.139267 ===
crasher:
initial call: Elixir.Logger.Watcher:init/1
pid: <0.1668.0>
registered_name: []
exception error: no case clause matching
{'EXIT',
{undef,
[{'Elixir.Logger.Backends.Gelf',init,
[{'Elixir.Logger.Backends.Gelf',
gelf_logger}],
[]},
{gen_event,server_add_handler,4,
[{file,"gen_event.erl"},{line,473}]},
{gen_event,handle_msg,6,
[{file,"gen_event.erl"},{line,318}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,249}]}]}}
in function 'Elixir.Logger.Watcher':init/1 (lib/logger/watcher.ex, line 28)
in call from gen_server:init_it/2 (gen_server.erl, line 374)
in call from gen_server:init_it/6 (gen_server.erl, line 342)
ancestors: ['Elixir.Logger.BackendSupervisor',
'Elixir.Logger.Supervisor',<0.1662.0>]
message_queue_len: 0
messages: []
links: [<0.1666.0>,<0.1664.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 610
stack_size: 27
reductions: 266
neighbours:
=ERROR REPORT==== 26-Apr-2019::20:24:37.140758 ===
** Generic server <0.1667.0> terminating
** Last message in was {gen_event_EXIT,'Elixir.Logger.Backends.Console',
shutdown}
** When Server state == {'Elixir.Logger','Elixir.Logger.Backends.Console'}
** Reason for termination ==
** {noproc,[{gen,do_call,4,[{file,"gen.erl"},{line,177}]},
{gen_event,rpc,2,[{file,"gen_event.erl"},{line,239}]},
{'Elixir.Logger.Watcher',terminate,2,
[{file,"lib/logger/watcher.ex"},
{line,90}]},
{gen_server,try_terminate,3,[{file,"gen_server.erl"},{line,673}]},
{gen_server,terminate,10,[{file,"gen_server.erl"},{line,858}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
=CRASH REPORT==== 26-Apr-2019::20:24:37.141115 ===
crasher:
initial call: Elixir.Logger.Watcher:init/1
pid: <0.1667.0>
registered_name: []
exception exit: noproc
in function gen:do_call/4 (gen.erl, line 177)
in call from gen_event:rpc/2 (gen_event.erl, line 239)
in call from 'Elixir.Logger.Watcher':terminate/2 (lib/logger/watcher.ex, line 90)
in call from gen_server:try_terminate/3 (gen_server.erl, line 673)
in call from gen_server:terminate/10 (gen_server.erl, line 858)
ancestors: ['Elixir.Logger.BackendSupervisor',
'Elixir.Logger.Supervisor',<0.1662.0>]
message_queue_len: 1
messages: [{'EXIT',<0.1666.0>,shutdown}]
links: []
dictionary: []
trap_exit: true
status: running
heap_size: 2586
stack_size: 27
reductions: 6576
neighbours:
=SUPERVISOR REPORT==== 26-Apr-2019::20:24:37.141966 ===
supervisor: {local,'Elixir.Logger.BackendSupervisor'}
errorContext: shutdown_error
reason: noproc
offender: [{pid,<0.1667.0>},
{id,'Elixir.Logger.Backends.Console'},
{mfargs,
{'Elixir.Logger.Watcher',start_link,
[{'Elixir.Logger','Elixir.Logger.Backends.Console',
console}]}},
{restart_type,transient},
{shutdown,5000},
{child_type,worker}]
=CRASH REPORT==== 26-Apr-2019::20:24:37.140851 ===
crasher:
initial call: application_master:init/4
pid: <0.1661.0>
registered_name: []
exception exit: {{shutdown,
{failed_to_start_child,
'Elixir.Logger.BackendSupervisor',
{'EXIT',
{{badmatch,
{error,
{{{case_clause,
{'EXIT',
{undef,
[{'Elixir.Logger.Backends.Gelf',init,
[{'Elixir.Logger.Backends.Gelf',
gelf_logger}],
[]},
{gen_event,server_add_handler,4,
[{file,"gen_event.erl"},{line,473}]},
{gen_event,handle_msg,6,
[{file,"gen_event.erl"},{line,318}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,249}]}]}}},
[{'Elixir.Logger.Watcher',init,1,
[{file,"lib/logger/watcher.ex"},{line,28}]},
{gen_server,init_it,2,
[{file,"gen_server.erl"},{line,374}]},
{gen_server,init_it,6,
[{file,"gen_server.erl"},{line,342}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,249}]}]},
{child,undefined,
{'Elixir.Logger.Backends.Gelf',gelf_logger},
{'Elixir.Logger.Watcher',start_link,
[{'Elixir.Logger',
{'Elixir.Logger.Backends.Gelf',gelf_logger},
{'Elixir.Logger.Backends.Gelf',gelf_logger}}]},
transient,5000,worker,
['Elixir.Logger.Watcher']}}}},
[{'Elixir.Logger.BackendSupervisor',
'-start_link/1-fun-0-',2,
[{file,"lib/logger/backend_supervisor.ex"},
{line,14}]},
{'Elixir.Enum','-reduce/3-lists^foldl/2-0-',3,
[{file,"lib/enum.ex"},{line,1940}]},
{'Elixir.Logger.BackendSupervisor',start_link,1,
[{file,"lib/logger/backend_supervisor.ex"},
{line,13}]},
{supervisor,do_start_child_i,3,
[{file,"supervisor.erl"},{line,379}]},
{supervisor,do_start_child,2,
[{file,"supervisor.erl"},{line,365}]},
{supervisor,'-start_children/2-fun-0-',3,
[{file,"supervisor.erl"},{line,349}]},
{supervisor,children_map,4,
[{file,"supervisor.erl"},{line,1157}]},
{supervisor,init_children,2,
[{file,"supervisor.erl"},{line,315}]}]}}}},
{'Elixir.Logger.App',start,[normal,[]]}}
in function application_master:init/4 (application_master.erl, line 138)
ancestors: [<0.1660.0>]
message_queue_len: 1
messages: [{'EXIT',<0.1662.0>,normal}]
links: [<0.1660.0>,<0.1599.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1598
stack_size: 27
reductions: 276
neighbours:
=INFO REPORT==== 26-Apr-2019::20:24:37.142650 ===
application: logger
exited: {{shutdown,
{failed_to_start_child,'Elixir.Logger.BackendSupervisor',
{'EXIT',
{{badmatch,
{error,
{{{case_clause,
{'EXIT',
{undef,
[{'Elixir.Logger.Backends.Gelf',init,
[{'Elixir.Logger.Backends.Gelf',gelf_logger}],
[]},
{gen_event,server_add_handler,4,
[{file,"gen_event.erl"},{line,473}]},
{gen_event,handle_msg,6,
[{file,"gen_event.erl"},{line,318}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,249}]}]}}},
[{'Elixir.Logger.Watcher',init,1,
[{file,"lib/logger/watcher.ex"},{line,28}]},
{gen_server,init_it,2,
[{file,"gen_server.erl"},{line,374}]},
{gen_server,init_it,6,
[{file,"gen_server.erl"},{line,342}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,249}]}]},
{child,undefined,
{'Elixir.Logger.Backends.Gelf',gelf_logger},
{'Elixir.Logger.Watcher',start_link,
[{'Elixir.Logger',
{'Elixir.Logger.Backends.Gelf',gelf_logger},
{'Elixir.Logger.Backends.Gelf',gelf_logger}}]},
transient,5000,worker,
['Elixir.Logger.Watcher']}}}},
[{'Elixir.Logger.BackendSupervisor','-start_link/1-fun-0-',
2,
[{file,"lib/logger/backend_supervisor.ex"},{line,14}]},
{'Elixir.Enum','-reduce/3-lists^foldl/2-0-',3,
[{file,"lib/enum.ex"},{line,1940}]},
{'Elixir.Logger.BackendSupervisor',start_link,1,
[{file,"lib/logger/backend_supervisor.ex"},{line,13}]},
{supervisor,do_start_child_i,3,
[{file,"supervisor.erl"},{line,379}]},
{supervisor,do_start_child,2,
[{file,"supervisor.erl"},{line,365}]},
{supervisor,'-start_children/2-fun-0-',3,
[{file,"supervisor.erl"},{line,349}]},
{supervisor,children_map,4,
[{file,"supervisor.erl"},{line,1157}]},
{supervisor,init_children,2,
[{file,"supervisor.erl"},{line,315}]}]}}}},
{'Elixir.Logger.App',start,[normal,[]]}}
type: permanent
{"Kernel pid terminated",application_controller,"{application_start_failure,logger,{{shutdown,{failed_to_start_child,'Elixir.Logger.BackendSupervisor',{'EXIT',{{badmatch,{error,{{{case_clause,{'EXIT',{undef,[{'Elixir.Logger.Backends.Gelf',init,[{'Elixir.Logger.Backends.Gelf',gelf_logger}],[]},{gen_event,server_add_handler,4,[{file,\"gen_event.erl\"},{line,473}]},{gen_event,handle_msg,6,[{file,\"gen_event.erl\"},{line,318}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,249}]}]}}},[{'Elixir.Logger.Watcher',init,1,[{file,\"lib/logger/watcher.ex\"},{line,28}]},{gen_server,init_it,2,[{file,\"gen_server.erl\"},{line,374}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,342}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,249}]}]},{child,undefined,{'Elixir.Logger.Backends.Gelf',gelf_logger},{'Elixir.Logger.Watcher',start_link,[{'Elixir.Logger',{'Elixir.Logger.Backends.Gelf',gelf_logger},{'Elixir.Logger.Backends.Gelf',gelf_logger}}]},transient,5000,worker,['Elixir.Logger.Watcher']}}}},[{'Elixir.Logger.BackendSupervisor','-start_link/1-fun-0-',2,[{file,\"lib/logger/backend_supervisor.ex\"},{line,14}]},{'Elixir.Enum','-reduce/3-lists^foldl/2-0-',3,[{file,\"lib/enum.ex\"},{line,1940}]},{'Elixir.Logger.BackendSupervisor',start_link,1,[{file,\"lib/logger/backend_supervisor.ex\"},{line,13}]},{supervisor,do_start_child_i,3,[{file,\"supervisor.erl\"},{line,379}]},{supervisor,do_start_child,2,[{file,\"supervisor.erl\"},{line,365}]},{supervisor,'-start_children/2-fun-0-',3,[{file,\"supervisor.erl\"},{line,349}]},{supervisor,children_map,4,[{file,\"supervisor.erl\"},{line,1157}]},{supervisor,init_children,2,[{file,\"supervisor.erl\"},{line,315}]}]}}}},{'Elixir.Logger.App',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,logger,{{shutdown,{failed_to_start_child,'Elixir.Logger.BackendSupervisor',{'EXIT',{{badmatch,{error,{{{case_clause,{'EXIT',{
I'm fairly new to the Erlang/Elixir world and not exactly sure how to decipher this. Is something misconfigured here or is it possible there there's a conflict with another dependency?
Hi
If you're using releases and want to pass in the port as an ENV variable, you'll likely be using the REPLACE_OS_VARS
approach
config :logger, :gelf_logger,
host: "${GRAYLOG_HOST}",
port: "${GRAYLOG_PORT}"
Problem with that is that the port gets interpreted as a string, which fails when calling the gen_udp
module.
It'd be nice if the library could handle casting the port to an integer before being used
I would expect that it just tries to connect but doesn't fail at all.
exited: {{shutdown,
{failed_to_start_child,'Elixir.Logger.Watcher',
{'EXIT',
{{badmatch,
{error,
{{{case_clause,
{'EXIT',
{{badmatch,{error,nxdomain}},
[{'Elixir.Logger.Backends.Gelf',configure,2,
[{file,"lib/gelf_logger.ex"},{line,105}]},
{'Elixir.Logger.Backends.Gelf',init,1,
[{file,"lib/gelf_logger.ex"},{line,72}]},
{gen_event,server_add_handler,4,
[{file,"gen_event.erl"},{line,429}]},
{gen_event,handle_msg,5,
[{file,"gen_event.erl"},{line,274}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,247}]}]}}},
[{'Elixir.Logger.Watcher',do_init,3,
[{file,"lib/logger/watcher.ex"},{line,78}]},
{gen_server,init_it,6,
[{file,"gen_server.erl"},{line,328}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,247}]}]},
{child,undefined,
{'Elixir.Logger.Watcher',
{'Elixir.Logger',
{'Elixir.Logger.Backends.Gelf',gelf_logger}}},
{'Elixir.Logger.Watcher',watcher,
['Elixir.Logger',
{'Elixir.Logger.Backends.Gelf',gelf_logger},
{'Elixir.Logger.Backends.Gelf',gelf_logger}]},
transient,5000,worker,
['Elixir.Logger.Watcher']}}}},
[{'Elixir.Logger.Watcher','-start_link/3-fun-0-',2,
[{file,"lib/logger/watcher.ex"},{line,16}]},
{'Elixir.Enum','-reduce/3-lists^foldl/2-0-',3,
[{file,"lib/enum.ex"},{line,1755}]},
{'Elixir.Logger.Watcher',start_link,3,
[{file,"lib/logger/watcher.ex"},{line,15}]},
{supervisor,do_start_child,2,
[{file,"supervisor.erl"},{line,365}]},
{supervisor,start_children,3,
[{file,"supervisor.erl"},{line,348}]},
{supervisor,init_children,2,
[{file,"supervisor.erl"},{line,314}]},
{gen_server,init_it,6,[{file,"gen_server.erl"},{line,328}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,247}]}]}}}},
{'Elixir.Logger.App',start,[normal,[]]}}
type: temporary
** (Mix) Could not start application logger: Logger.App.start(:normal, []) returned an error: shutdown: failed to start child: Logger.Watcher
** (EXIT) an exception was raised:
** (MatchError) no match of right hand side value: {:error, {{{:case_clause, {:EXIT, {{:badmatch, {:error, :nxdomain}}, [{Logger.Backends.Gelf, :configure, 2, [file: 'lib/gelf_logger.ex', line: 105]}, {Logger.Backends.Gelf, :init, 1, [file: 'lib/gelf_logger.ex', line: 72]}, {:gen_event, :server_add_handler, 4, [file: 'gen_event.erl', line: 429]}, {:gen_event, :handle_msg, 5, [file: 'gen_event.erl', line: 274]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 247]}]}}}, [{Logger.Watcher, :do_init, 3, [file: 'lib/logger/watcher.ex', line: 78]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 328]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 247]}]}, {:child, :undefined, {Logger.Watcher, {Logger, {Logger.Backends.Gelf, :gelf_logger}}}, {Logger.Watcher, :watcher, [Logger, {Logger.Backends.Gelf, :gelf_logger}, {Logger.Backends.Gelf, :gelf_logger}]}, :transient, 5000, :worker, [Logger.Watcher]}}}
(logger) lib/logger/watcher.ex:16: anonymous fn/2 in Logger.Watcher.start_link/3
(elixir) lib/enum.ex:1755: Enum."-reduce/3-lists^foldl/2-0-"/3
(logger) lib/logger/watcher.ex:15: Logger.Watcher.start_link/3
(stdlib) supervisor.erl:365: :supervisor.do_start_child/2
(stdlib) supervisor.erl:348: :supervisor.start_children/3
(stdlib) supervisor.erl:314: :supervisor.init_children/2
(stdlib) gen_server.erl:328: :gen_server.init_it/6
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
On several machines here at work, the UNIX Timestamp calculation seems to be incorrect. This is causing graylog to ignore the messages. Pulling the project down and using the erlang os module timestamp seems to fix the problem. Could we change timestamp to:
timestamp: :os.system_time(:seconds)
I am not sure what the issue is with the current way of calculating the timestamp but it seems we could save complexity by just using the already built in functionality.
When u use rpc to connect to a running elixir process and then execute any action that would cause
a log to be sent to gelf, the messages are not sent.
Steps to reproduce
1.Build a long living app called hello_world. Say a api based on phoenix.
2.mix release.
3.start your app with
/bin/hello_world daemon (or start)
4.from another console, you go
/bin hello_world rpc "require Logger; Logger.info("flavius was here")"
Expected result
flavius was here is logged to gelf
Actual result
flavius was here is not logged to gelf
The fix is easy and it is suggested at the end of this thread.
https://elixirforum.com/t/logging-from-within-remote-session/26323/5
Mainly just delete the handle_event at line 71.
Hi there,
I have Graylog running and verified it's receiving via an echo/nc line.
However, I don't receive any log statements out of my application.
I used the exact config as per Readme - how could I debug what the problem is?
I'm running the latest version of logstash via docker.
https://www.elastic.co/guide/en/logstash/current/docker.html
I'm running it via the "host" adapter (so logstash will start a GELF listener @ localhost:12201).
Logging via docker's native gelf log adapter works.
--log-driver=gelf --log-opt gelf-address=udp://localhost:12201
I have configured gelf_logger
to send messages to "localhost:12201" and all of the :gen_udp.send()
s are going through just fine.
In fact, I can see all of the data if I listen for it via nc -ul 127.0.0.1 12201
.
However, logstash is either not receiving or dropping the messages.
warning: Code.ensure_compiled?/1 is deprecated. Use Code.ensure_compiled/1 instead (see the proper disclaimers in its docs)
Pass in a domain name rather than an ip address for the host config parameter
host: "garylog.com",
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.