VictoriaLogs 配置多租户和优化

简介

VictoriaLogs 通过 AccountID 和 ProjectID，区分不同租户。ID为32位整数，默认为 0:0 。

PS：类似ES不同的索引。

VictoriaLogs 系列文章：https://songxwn.com/tags/VictoriaLogs/

写入到VictoriaLogs 配置多租户

Filebeat 兼容写入ES 配置多租户

output.elasticsearch:
  hosts: ["http://localhost:9428/insert/elasticsearch/"]
  headers:
    AccountID: 12
    ProjectID: 34
  parameters:
    _msg_field: "message"
    _time_field: "@timestamp"
    _stream_fields: "host.name,log.file.path"

Logstash 兼容写入ES 配置多租户

output {
  elasticsearch {
    hosts => ["http://localhost:9428/insert/elasticsearch/"]
    custom_headers => {
        "AccountID" => "1"
        "ProjectID" => "2"
    }
    parameters => {
        "_msg_field" => "message"
        "_time_field" => "@timestamp"
        "_stream_fields" => "host.hostname,process.name"
    }
  }
}

Grafana 数据源配置多租户读取

配置文件

​

apiVersion: 1
datasources:
    # <string, required> Name of the VictoriaLogs datasource
    # displayed in Grafana panels and queries.
  - name: VictoriaLogs
    # <string, required> Sets the data source type.
    type: victoriametrics-logs-datasource
    # <string, required> Sets the access mode, either
    # proxy or direct (Server or Browser in the UI).
    access: proxy
    # <string> Sets URL for sending queries to VictoriaLogs server.
    # see https://docs.victoriametrics.com/victorialogs/querying/
    url: http://victorialogs:9428
    # <string> Sets the pre-selected datasource for new panels.
    # You can set only one default data source per organization.
    isDefault: true
    jsonData:
      AccountID: '9527'
      ProjectID: '9527'

Web UI

数据发送调优

​

压缩 - 节省网络带宽

​

output.elasticsearch:
  hosts: ["http://localhost:9428/insert/elasticsearch/"]
  parameters:
    _msg_field: "message"
    _time_field: "@timestamp"
    _stream_fields: "host.name,log.file.path"
  compression_level: 1

​

提高性能

​

output.elasticsearch:
  hosts: ["http://localhost:9428/insert/elasticsearch/"]
  parameters:
    _msg_field: "message"
    _time_field: "@timestamp"
    _stream_fields: "host.name,log.file.path"
  worker: 8
  bulk_max_size: 1000