[2021/03/24 04:25:46] [debug] [bigquery:bigquery.0] created event channels: read=24 write=25
[2021/03/24 04:25:46] [ info] [output:bigquery:bigquery.0] project='<project-id>' dataset='staging' table='backend_stag_log_test'
[2021/03/24 04:25:46] [debug] [output:bigquery:bigquery.0] JWT signature:
xxxxxxx
[2021/03/24 04:25:46] [debug] [http_client] not using http_proxy for header
[2021/03/24 04:25:46] [debug] [http_client] header=POST /oauth2/v4/token HTTP/1.1
Host: www.googleapis.com
Content-Length: 735
Content-Type: application/x-www-form-urlencoded
[2021/03/24 04:25:46] [ info] [oauth2] HTTP Status=200
[2021/03/24 04:25:46] [debug] [oauth2] payload:
{"access_token":"xxxxxxxx","expires_in":3599,"token_type":"Bearer"}
[2021/03/24 04:25:46] [ info] [oauth2] access token from 'www.googleapis.com:443' retrieved
[2021/03/24 04:25:46] [debug] [router] match rule tail.0:bigquery.0
[2021/03/24 04:25:46] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2021/03/24 04:25:46] [ info] [sp] stream processor started
[2021/03/24 04:31:11] [debug] [input:tail:tail.0] 0 new files found on path '/var/log/containers/backend-stag-*.log'
[2021/03/24 04:31:12] [debug] [input:tail:tail.0] inode=199365869 events: IN_MODIFY
[2021/03/24 04:31:12] [debug] [task] created task=0x7f5df4237c80 id=0 OK
[2021/03/24 04:31:12] [debug] [http_client] not using http_proxy for header
[2021/03/24 04:31:12] [debug] [http_client] header=POST /bigquery/v2/projects/<project-id>/datasets/staging/tables/backend_stag_log_test/insertAll HTTP/1.1
Host: www.googleapis.com
Content-Length: 2107
User-Agent: Fluent-Bit
Content-Type: application/json
Authorization: Bearer ya29.c.xxxxxx-dDqtzdWqsI-9168zgOWPHzjjCKcOjAeMpmy9iIxYdCD4VtRcEALuqVJysJ8Xx-xxxx-no4zmiPUKnU
[2021/03/24 04:31:12] [debug] [output:bigquery:bigquery.0] HTTP Status=200
[2021/03/24 04:31:12] [debug] [socket] could not validate socket status for #44 (don't worry)
[2021/03/24 04:31:12] [debug] [out coro] cb_destroy coro_id=13
[2021/03/24 04:31:12] [debug] [task] destroy task=0x7f5df4237c80 (task_id=0)
I see that the fluent-bit have push log to BigQuery success (HTTP Status=200), but when I check the data on table of BigQuery, there is nothings there.
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.15", GitCommit:"73dd5c840662bb066a146d0871216333181f4b64", GitTreeState:"clean", BuildDate:"2021-01-13T13:22:41Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.9-eks-d1db3c", GitCommit:"d1db3c46e55f95d6a7d3e5578689371318f95ff9", GitTreeState:"clean", BuildDate:"2020-10-20T22:18:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
version.BuildInfo{Version:"v3.3.4", GitCommit:"a61ce5633af99708171414353ed49547cf05013d", GitTreeState:"clean", GoVersion:"go1.14.9"}
So, How can i continue to debug this situation? And do we have any guides to implement my purpose with fluent-bit, the official document is not enought right now.