Comments (3)
The error you're encountering, org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: The rest server is too busy to write
, suggests that the HugeGraph server is experiencing high load, which is impacting its ability to process write requests efficiently. Here are targeted suggestions to address this issue:
-
Optimize HugeGraph and RocksDB Configurations: Adjust RocksDB settings like
max_background_jobs
,write_buffer_size
, andmax_open_files
to better match your hardware capabilities, potentially improving performance under load. -
Increase Iterator Pool Capacity: The error might be related to the capacity of the iterator pool. Increasing the
ITERATOR_POOL_CAPACITY
inRocksDBIteratorPool.java
could help manage load more effectively by allowing more iterators to be pooled concurrently. -
Enable Iterator Reusing: In
ReusedRocksIterator.java
, changingREUSING_ENABLED
fromfalse
totrue
can reduce the overhead associated with iterator creation and destruction, which is beneficial under high load scenarios. -
Benchmark Code Optimization: Review your benchmarking approach to ensure it doesn't overwhelm the server. Adjusting the rate of requests or the level of concurrency might prevent the server from becoming too busy to write.
-
Server Performance Monitoring: Utilize monitoring tools to identify performance bottlenecks. This can guide further optimizations, whether in configuration or hardware adjustments.
-
Update HugeGraph and RocksDB: Ensure you're using the latest versions as they may include relevant performance improvements or bug fixes.
For specific RocksDB configurations:
max_background_jobs
: Consider setting this to a higher value to allow more concurrent background processes.write_buffer_size
: Increasing this can help manage write workload spikes more effectively.max_open_files
: Setting this to-1
or a high number can reduce file operation overhead, aiding in performance under heavy load.
These adjustments are starting points. Monitoring and iterative tuning based on actual performance are crucial for achieving optimal configurations.
Sources
- hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBIteratorPool.java
- hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBOptions.java
About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
from incubator-hugegraph.
@dosu-bot could u help to find the relative context/issues & give some suggestion?
from incubator-hugegraph.
Due to the lack of activity, the current issue is marked as stale and will be closed after 20 days, any update will remove the stale label
from incubator-hugegraph.
Related Issues (20)
- 并发查询时 /gremlin接口返回的结果与预期不符合 HOT 1
- [Question] call rest-api in multi-users env (concurrency) HOT 6
- Optimizing Shortest Path Calculation HOT 6
- [Bug] migrating from version 0.11 to apache release 1.2.0, but start failed HOT 3
- [Summary] Dropping Support for Java 8: Compatibility Breaks Ahead
- [Question] 1.2.0 create graph in rest-api throws auth related permission HOT 7
- 创建属性的时候报错 HOT 2
- hugegraph集群应该怎么配置? HOT 2
- 集群部署hugegraph1.2.0 ,但是在主节点创建图同步失败,是因为什么原因造成的? HOT 5
- [Summary] Feedback on the trial of HugeGraph Distributed (pd-store) Version (BETA)
- [Bug] g.E().count() the result is not as expectedThe result is not as expected HOT 8
- 如何对Hugegraph-server中的Kneighbor算法的执行过程作profiling。分析算法执行过程中的各个阶段的内存和CPU占用。 HOT 7
- [Bug] gremlin example script execution error with RocksDB backend
- [Question] understanding Single Source Shortest Path and Traversal API within HugeGraph HOT 2
- [Question] Why we use relational/SQL based backend storage in GraphDB? HOT 5
- [Question] Error during serialization, serializer org.apache.hugegraph.backend.id.IdGenerator$StringId not found HOT 4
- [Question] There is no demo of how creating a graph through GraphAPI in test module and I had try client.post() failed. HOT 13
- [Question] How to check hugegraph node(role) status like master-worker? HOT 3
- [Question] When vertex strategy is CUSTOMIZE_UUID, is there need to pass the id value in json? HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from incubator-hugegraph.