Comments (4)
Pinging @elastic/es-distributed (Team:Distributed)
from elasticsearch.
This is still happening: https://gradle-enterprise.elastic.co/s/qii4jqdjvu32u
from elasticsearch.
I was checking the failures and every time we have a mismatch of docs count it is due to something like:
[2024-03-13T13:33:54,641][WARN ][o.e.x.c.a.ShardFollowNodeTask] [follower3] shard follow task encounter non-retryable error
org.elasticsearch.index.IndexNotFoundException: no such index [index1]
at org.elasticsearch.cluster.metadata.Metadata.getIndexSafe(Metadata.java:1238) ~[elasticsearch-8.12.3-SNAPSHOT.jar:8.12.3-SNAPSHOT]
at org.elasticsearch.xpack.ccr.action.ShardFollowTasksExecutor$1.lambda$innerUpdateSettings$6(ShardFollowTasksExecutor.java:204) ~[main/:?]
[2024-03-13T13:33:54,643][WARN ][o.e.x.c.a.ShardFollowNodeTask] [follower3] shard follow task encounter non-retryable error
org.elasticsearch.index.IndexNotFoundException: no such index [index1]
at org.elasticsearch.cluster.metadata.Metadata.getIndexSafe(Metadata.java:1238) ~[elasticsearch-8.12.3-SNAPSHOT.jar:8.12.3-SNAPSHOT]
at org.elasticsearch.xpack.ccr.action.ShardFollowTasksExecutor$1.lambda$innerUpdateAliases$8(ShardFollowTasksExecutor.java:284) ~[main/:?]
[2024-03-13T13:33:54,645][WARN ][o.e.x.c.a.ShardFollowNodeTask] [follower3] shard follow task encounter non-retryable error
org.elasticsearch.index.IndexNotFoundException: no such index [index1]
at org.elasticsearch.cluster.metadata.Metadata.getIndexSafe(Metadata.java:1238) ~[elasticsearch-8.12.3-SNAPSHOT.jar:8.12.3-SNAPSHOT]
at org.elasticsearch.xpack.ccr.action.ShardFollowTasksExecutor$1.lambda$innerUpdateSettings$6(ShardFollowTasksExecutor.java:204) ~[main/:?]
[2024-03-13T13:33:54,654][WARN ][o.e.x.c.a.ShardFollowNodeTask] [follower3] shard follow task encounter non-retryable error
org.elasticsearch.index.IndexNotFoundException: no such index [index1]
at org.elasticsearch.cluster.metadata.Metadata.getIndexSafe(Metadata.java:1238) ~[elasticsearch-8.12.3-SNAPSHOT.jar:8.12.3-SNAPSHOT]
at org.elasticsearch.xpack.ccr.action.ShardFollowTasksExecutor$1.lambda$innerUpdateAliases$8(ShardFollowTasksExecutor.java:284) ~[main/:?]
There is nothing in leader logs indicating that index was deleted.
The leader node was restarting in meanwhile. Is there a way we could return incomplete/empty or a cluster state from a node with no master during this period of time?
from elasticsearch.
The failure is likely happening as we see the unrecovered cluster state that does not contain the index we follow.
This is a bug in ShardFollowNodeTask we need to fix.
from elasticsearch.
Related Issues (20)
- [ES|QL] Support date_period and time_duration in named parameters HOT 2
- Shrink Action in ILM creates a shrunk index with the same number of primary shards HOT 1
- [CI] ComparisonTests testSparseGaussianDistribution failing HOT 2
- [CI] RestEsqlIT failing in release builds HOT 3
- (API) Cluster Health report unassigned_primary_shards HOT 1
- vector search error when the target index is empty HOT 1
- Trial license inconsistencies between the doc and _license/start_trial API HOT 2
- [ESQL] Extend range of supported timespan literals HOT 2
- Support for dimension shrinking in dense vector fields HOT 1
- term condition does not work
- Support for bit precision in the Inference API text_embedding task HOT 2
- docker pull arm64v8/elasticsearch:8.15.0 results in no matching manifest for linux/arm/v8 in the manifest list entries HOT 1
- Better handling of node shutdown in SearchScrollAsyncAction HOT 2
- [CI] VerifierTests testUnsupportedAndMultiTypedFields failing HOT 2
- Improve inactive API key removal HOT 1
- Aggregator result equality easy to break HOT 1
- [CI] MixedClusterClientYamlTestSuiteIT class failing HOT 5
- [CI] MixedClusterClientYamlTestSuiteIT class failing HOT 4
- [CI] MixedClusterClientYamlTestSuiteIT test {p0=search.vectors/40_knn_search/kNN search in _knn_search endpoint} failing HOT 1
- [CI] MixedClusterClientYamlTestSuiteIT test {p0=search.vectors/40_knn_search/kNN search with filter in _knn_search endpoint} failing HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elasticsearch.