Use Case
Customize the Document Metadata
The MongoDB output plugin uses "tags" as metadata, which then define the metadata in the root document. This means that to query the data, you need to specify all tags in the query (and Mongo's query syntax doesn't make this easy). Even if you exclude some tags or just make them fields, the query syntax still isn't great.
Could we add a configuration option that lets users define what tags to use as metadata, with all tags being the default?
[[outputs.mongodb]]
# Tags to use as root document metadata. Uses all tags by default.
metadata_keys = ["hostname", "region"]
This way the metadata is the root entry and Mongo will keep all related documents together so queries are optimized across shards. The main question would be—what happens to tags that aren't part of the metadata? Are they dropped? Included in the document as fields? etc.
Expected behavior
see above
Actual behavior
see above
Additional info
split from #18042
Use Case
Customize the Document Metadata
The MongoDB output plugin uses "tags" as metadata, which then define the metadata in the root document. This means that to query the data, you need to specify all tags in the query (and Mongo's query syntax doesn't make this easy). Even if you exclude some tags or just make them fields, the query syntax still isn't great.
Could we add a configuration option that lets users define what tags to use as metadata, with all tags being the default?
This way the metadata is the root entry and Mongo will keep all related documents together so queries are optimized across shards. The main question would be—what happens to tags that aren't part of the metadata? Are they dropped? Included in the document as fields? etc.
Expected behavior
see above
Actual behavior
see above
Additional info
split from #18042