JSON log format about semantic_logger HOT 7 CLOSED

Hello,

I tried setting the log level to :json but I am still getting a partial JSON string including the timestamp, etc:

2020-04-23 00:24:25.366219 I [20657:63966080 logger.rb:41] Rack -- Started -- { :method => "POST", :path => "/sso/ValidateUserSessionWS", :ip => "127.0.0.1" }

unless Rails.env.test? || Rails.env.development?
SemanticLogger.add_appender(
file_name: 'log/staging_json.log',
formatter: :json,
filter: -> log { log.payload.try!(:[], :action) != 'health_check' },
level: :info
)
end

Any ideas?

@reidmorrison Thanks

from semantic_logger.

Not sure what you mean by prefix strings? Do you have an example?

The JSON format only adds a new line after each JSON entry so that it can be parsed by standard tools.
For example we use the JSON output file so that Splunk can read it at its leisure.

from semantic_logger.

The output is like this even if the format is JSON:

Apr 26 12:43:42 boo Semantic Logger[19701]: {"name":"Sidekiq","pid":19701,"thread":"51537380","level":"debug","level_index":1,"host":"foo.foo.foo.com","application":"Semantic Logger","message":"unique_digest : {\"class\"=\u003e\"FooClass\", \"queue\"=\u003e\"default\", \"unique_args\"=\u003e[\"1111111\", \"qq\", \"foo\", {:banks_to_retry=\u003e[22222]}]} into uniquejobs:fa3b01a2aea3bd396d30f39a65285ed0","timestamp":"2017-04-26T17:43:42.636755Z"}

I was expected it to be all JSON.

from semantic_logger.

We have this initializer that adds a second log file that is in JSON format so that Splunk can read it:

# Using a json log file with the splunk forwarder to rule out the possibility
# of a splunk outage holding up in flight processes.
unless Rails.env.test? || Rails.env.development?
  SemanticLogger.add_appender(
    file_name: 'log/splunk_json.log',
    formatter: :json,
    filter:    -> log { log.payload.try!(:[], :action) != 'health_check' },
    level:     :info
  )
end

We keep the human readable text file in case we need to debug directly on a server.

Note that we filter out the health check messages so that we don't fill spunk up with noise.

from semantic_logger.

Please re-open this ticket if further action is needed.

from semantic_logger.

Hi @reidmorrison, my rails app is hosted in AWS as an ECS task and I have configured a Splunk log driver for it. All the ECS logs go to Splunk. I configured the sematic logger to log to STDOUT (like below). However I think Splunk took it as a plain text and and not as JSON. I wasn't able to use Splunk's spath over that log.

config.semantic_logger.add_appender(io: STDOUT,
                                        level: config.log_level,
                                        formatter: :JSON)

However I will try your suggested solution but I have a question - How would Splunk know what log file to consider? Does the file_name in add_appender takes care of it by itself?

from semantic_logger.

Yes, the file_name tells the appender where to write the json log file.
Then the Splunk agent needs to be configured to read the from the same path as where you are writing your log file too.

from semantic_logger.