Comments (5)
Pinging @elastic/ml-core (Team:ML)
from elasticsearch.
Expected: a collection with size <2>
but: collection size was <1>
{
"checkpointing": {
"last": {
"checkpoint": "1",
"timestamp_millis": "1708495449698",
"time_upper_bound_millis": "1708495448698"
}
},
"node": {
"transport_address": "127.0.0.1:40149",
"name": "javaRestTest-1",
"attributes": {},
"id": "mkiRJ3gTS7CPfwTEIHwrvA",
"ephemeral_id": "sm-DQ1zAQWa_utYiFknKzA"
},
"stats": {
"pages_processed": "1",
"index_time_in_ms": "0",
"documents_deleted": "0",
"search_failures": "0",
"index_failures": "0",
"search_total": "1",
"processing_total": "1",
"delete_time_in_ms": "0",
"documents_indexed": "0",
"trigger_count": "2",
"documents_processed": "0",
"search_time_in_ms": "12",
"index_total": "0",
"exponential_avg_checkpoint_duration_ms": "53.0",
"exponential_avg_documents_processed": "0.0",
"processing_time_in_ms": "0",
"exponential_avg_documents_indexed": "0.0"
},
"health": {
"issues": [
{
"issue": "Privileges check failed",
"count": "1",
"details": "Cannot create transform [transform-permissions-defer-unattended] because user john_junior lacks the required permissions [transform-permissions-defer-unattended-dest:[create_index, index, read], transform-permissions-defer-unattended-index:[read, view_index_metadata]]",
"type": "privileges_check_failed"
}
],
"status": "red"
},
"id": "transform-permissions-defer-unattended",
"state": "started"
}
Looks like we were expecting a second issues
to exist, odd because we do have the call in the logs:
1> [2024-02-14T14:32:59,181][INFO ][o.e.x.t.i.TransformInsufficientPermissionsIT] [testTransformPermissionsDeferUnattendedNoDest] Transform audit: [2024-02-14T14:32:58.747Z] [transform-permissions-defer-unattended] [Created transform.] [javaRestTest-1]
1> [2024-02-14T14:32:59,182][INFO ][o.e.x.t.i.TransformInsufficientPermissionsIT] [testTransformPermissionsDeferUnattendedNoDest] Transform audit: [2024-02-14T14:32:59.016Z] [transform-permissions-defer-unattended] [Updated transform state to [STARTED].] [javaRestTest-1]
1> [2024-02-14T14:32:59,182][INFO ][o.e.x.t.i.TransformInsufficientPermissionsIT] [testTransformPermissionsDeferUnattendedNoDest] Transform audit: [2024-02-14T14:32:59.037Z] [transform-permissions-defer-unattended] [Transform encountered an exception: [Could not create destination index [transform-permissions-defer-unattended-dest] for transform [transform-permissions-defer-unattended]]; Will automatically retry [1/-1]] [javaRestTest-1]
And we see at least 2 iterations: "trigger_count": "2"
Can't repro with 100 iterations, will try again tomorrow =)
from elasticsearch.
Our audit log for this test
1> [2024-02-21T19:04:09,813][INFO ][o.e.x.t.i.TransformInsufficientPermissionsIT] [testTransformPermissionsDeferUnattendedNoDest] Transform audit: [2024-02-21T06:04:04.682Z] [transform-permissions-defer-unattended] [Transform encountered an exception: [Could not create destination index [transform-permissions-defer-unattended-dest] for transform [transform-permissions-defer-unattended]]; Will automatically retry [1/-1]] [javaRestTest-1]
Corresponding stack trace:
» [2024-02-21T06:04:04,663][ERROR][o.e.x.t.p.TransformIndex ] [javaRestTest-1] Could not create destination index [transform-permissions-defer-unattended-dest] for transform [transform-permissions-defer-unattended] org.elasticsearch.ElasticsearchSecurityException: action [indices:admin/create] is unauthorized for user [john_junior] with effective roles [transform_admin] on indices [transform-permissions-defer-unattended-dest], this action is granted by the index privileges [create_index,manage,all]
»
The next subsequent stack trace is here:
» [2024-02-21T06:04:09,729][WARN ][o.e.x.t.t.TransformIndexer] [javaRestTest-1] [transform-permissions-defer-unattended] unable to load progress information for task. org.elasticsearch.ElasticsearchSecurityException: action [indices:data/read/search] is unauthorized for user [john_junior] with effective roles [transform_admin] on indices [transform-permissions-defer-unattended-index], this action is granted by the index privileges [read,all]
Which shouldn't be run, we should be calling the failure handler every time because the destination index cannot be created.
Confirmed in tests that we can cycle from 1 to 2 then back to 1 issues in the health blob. We seem to be correctly logging the issue, but then incorrectly overwriting the issue?
from elasticsearch.
When we retry, this check now evaluates to true, so we do not reattempt creating the index, and the next permission check will not invoke the failure listener so the previous failure gets reset.
from elasticsearch.
When we retry, this check now evaluates to true, so we do not reattempt creating the index, and the next permission check will not invoke the failure listener so the previous failure gets reset.
Good catch, Pat!
from elasticsearch.
Related Issues (20)
- ESQL: API endpoint (or similar) to retrieve available language versions HOT 2
- ESQL: add usage metrics/telemetry to language version HOT 1
- Enable better ordering of ILM steps (allocate, migrate...) for more efficient resource usage HOT 1
- [ML] Associate text_expansion subsearch with parent search request HOT 1
- COUNT(*) after STATS BY should not return the number of documents HOT 4
- DROP of EVAL planning issues HOT 2
- Update dependency jars in docker image HOT 4
- [CI] ScaledFloatFieldMapperTests#testSyntheticSourceMany failing HOT 3
- [CI] task ':plugins:repository-hdfs:haHdfs2Fixture' failing HOT 2
- [CI] EsqlQueryRequestTests testKnownVersionIsValid failing HOT 1
- _terms_enum when using alias indices HOT 2
- [ML] Improve handling of annual time changes (daylight savings) in anomaly detection HOT 3
- [CI] TopNFunctionTests testToXContent failing HOT 4
- skip_unavailable changes from true to false when remote connection fails HOT 1
- [ES|QL] Add Base64 to text conversion inside ES|QL HOT 3
- Fetching _seq_no via `fields` results in 5xx error HOT 3
- [Transform] There is a lot common code between `single-node-tests` and `multi-node-tests` HOT 1
- [ML] Enable perDeploymentMemoryBytes to handle errornous configurations HOT 1
- ESQL: Prune duplicate groups hidden behind aliases HOT 1
- OldRepositoryAccessIT.testOldRepoAccess fails on Lucene Snapshot HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elasticsearch.