Git Product home page Git Product logo

Comments (2)

yurchenkosv avatar yurchenkosv commented on July 17, 2024 1

Решили проблему путем добавления фильтра в fluentd:

    <filter **>
      @type record_modifier
      whitelist_keys timestamp, nsec, source, namespace, host, pod_name, container_name, stream, labels.names, labels.values, string_fields.names, string_fields.values, number_fields.names, number_fields.values, boolean_fields.names, boolean_fields.values, null_fields.names 
    </filter>

from loghouse.

yurchenkosv avatar yurchenkosv commented on July 17, 2024

Сделали проверку, что это за ошибка, путем модификации скрипта insert_ch.sh
Получили следующий вывод в консоль:

2017-11-08 07:12:42 +0000 [warn]: suppressed same stacktrace
Code: 117. DB::Exception: Unknown field found while parsing JSONEachRow format: _SELINUX_CONTEXT
 2017-11-08 07:29:19 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-11-08 08:04:59 +0000 error_class="RuntimeError" error="command returns 29952: bash /usr/local/bin/insert_ch.sh /tmp/fluent-plugin-exec-20171108-7-11q1bjb" plugin_id="object:3f9d896aa8cc"
  2017-11-08 07:29:19 +0000 [warn]: suppressed same stacktrace

// Это получили после модификации скрипта
{"_SELINUX_CONTEXT":"system_u:system_r:kernel_t:s0","_SYSTEMD_INVOCATION_ID":"dabc3bc11ebc4f8db8fc4d98fb8b235e","timestamp":1503909212,"source":"docker","host":"kubernetes-worker1","stream":"stdout","string_fields.names":["log"],"string_fields.values":["time=\"2017-08-28T08:33:32.389934701Z\" level=info msg=\"API listen on /var/run/docker.sock\""]}

Code: 117. DB::Exception: Unknown field found while parsing JSONEachRow format: _SELINUX_CONTEXT
 2017-11-08 08:04:59 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-11-08 09:06:11 +0000 error_class="RuntimeError" error="command returns 29952: bash /usr/local/bin/insert_ch.sh /tmp/fluent-plugin-exec-20171108-7-1kudxtx" plugin_id="object:3f9d896aa8cc"
  2017-11-08 08:04:59 +0000 [warn]: suppressed same stacktrace

Судя по всему, скрипт пытается запихнуть в БД, данные, которые не соответствуют структуре данных в таблице, поскольку содержат дополнительные ключи _SELINUX_CONTEXT, _SYSTEMD_INVOCATION_ID

Ожидаемая структура данных:

timestamp       DateTime
nsec    UInt32
source  String
namespace       String
host    String
pod_name        String
container_name  String
stream  String
labels.names    Array(String)
labels.values   Array(String)
string_fields.names     Array(String)
string_fields.values    Array(String)
number_fields.names     Array(String)
number_fields.values    Array(Float64)
boolean_fields.names    Array(String)
boolean_fields.values   Array(Float64)
null_fields.names       Array(String)

Я не специалист во fluentd, но возможно, поможет модификация конфига в части обработки лог сообщения, как сделано здесь:

<filter kubelet.**>
  @type record_modifier
  <record>
    timestamp              ${time}
    source                 "kubelet"
    host                   ${record["_HOSTNAME"]}
    stream                 ${record["_TRANSPORT"]}
    string_fields.names    ${["log"]}
    string_fields.values   ${[record["MESSAGE"]]}
  </record>
  remove_keys _TRANSPORT, PRIORITY, SYSLOG_FACILITY, _UID, _GID, _CAP_EFFECTIVE, _SYSTEMD_SLICE, _BOOT_ID, _MACHINE_ID, _HOSTNAME, SYSLOG_IDENTIFIER, _PID, _COMM, _EXE, _CMDLINE, _SYSTEMD_CGROUP, _SYSTEMD_UNIT, MESSAGE
</filter>

Как видно из конфига, запись формируется из конкретных полей лог сообщения, а невалидные ключи удаляются.

from loghouse.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.