Comments (5)
Pinging @elastic/es-storage-engine (Team:StorageEngine)
from elasticsearch.
Stepping back, mapping all attributes as dimensions seems problematic, since they're also used in tsid calculations etc. What's the reasoning behind storing all these IPs along with metric values? Do we anticipate filtering for a particular IP, as opposed to hostname etc? More so, how much value do we get by storing an array of IPs, instead of the first one as suggested, without additional info on what each IP represents?
from elasticsearch.
OTel's metrics data model define a time series by an entity consisting of several metadata properties, which includes all attributes.
We can't make a generic assumption that the time series can still be uniquely identified when we just store the first value of an attribute array.
I do agree that in the specific case of the host.ip
field, it may not always be an identifying attribute. But at this time, there's no concept of a non-identifying attribute in OpenTelemetry. This may change in the future and in that case we can store these non-identifying attributes in a namespace that is not a container for dimensions (time_series_dimension: false
). However, that doesn't change the fact that there can still be multi-valued identifying attributes. It's something that the OTel data model allows for but the TSDS data model doesn't, which creates some kind of impedance mismatch when trying to map OTel metrics to TSDS.
While I do think that we need to allow for multi-valued dimensions at some point, I also think that we can live with the workaround of storing a comma-separated string and having no support for automatically mapping IP fields as the ip
field type.
from elasticsearch.
Makes sense, thanks. While we do want to eventually support all corner cases, having concrete data about how much of an issue this is in practice can help us prioritize it.
from elasticsearch.
I don't have data on that at the moment but I don't expect this to be a critical issue. We'll probably learn more along the way and maybe users can chime in here when and if this becomes a bigger problem than anticipated. As long as we're aligned that we want to fix this eventually, I'm good with putting this on the shelf for the time being.
from elasticsearch.
Related Issues (20)
- Add Issuer to SAML WARN log events HOT 1
- Search preference docs render incorrectly HOT 4
- Formalize "known issues" docs concept HOT 2
- ESQL: operators and functions returning Infinite for Doubles HOT 2
- [CI] DataStreamsClientYamlTestSuiteIT test {p0=data_stream/190_failure_store_redirection/Ensure failure is redirected to correct failure store after a reroute processor} failing HOT 2
- [CI] SingleValueQueryTests class failing HOT 3
- [CI] DenseVectorFieldMapperTests testKnnHalfByteQuantizedHNSWVectorsFormat failing HOT 2
- [CI] DenseVectorFieldMapperTests testKnnQuantizedFlatVectorsFormat failing HOT 4
- [CI] DenseVectorFieldMapperTests testKnnQuantizedHNSWVectorsFormat failing HOT 1
- [CI] DenseVectorFieldMapperTests testKnnVectorsFormat failing HOT 1
- [CI] CompletionFieldMapperTests testPostingsFormat failing HOT 2
- [CI] ValueSourceReaderTypeConversionTests testLoadAllStatusAllInOnePage failing HOT 2
- Rename `logs` index mode to `logsdb` HOT 1
- Create a `metricsdb` alias for `time_series` and `tsdb` HOT 1
- [CI] TextSimilarityRankTests testRerankInferenceFailure failing HOT 4
- [CI] DockerTests test022InstallPluginsFromLocalArchive failing HOT 2
- ArrayIndexOutOfBoundsException in TDigest library HOT 1
- [CI] KnnSearchSingleNodeTests testKnnSearchAction failing HOT 4
- [CI] MlDistributedFailureIT testClusterWithTwoMlNodes_RunsDatafeed_GivenOriginalNodeGoesDown failing HOT 1
- Add docs for additional fields when level parameter is set to shards in nodes stats HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elasticsearch.