Git Product home page Git Product logo

halin's Introduction

CircleCI

Halin Neo4j Monitoring

Halin is a Graph App for monitoring Neo4j.

As of June 2021, with the release of Neo4j 4.3.0 halin is now deprecated.

The software will continue to be available as a GraphApp and via the URL above; and if you currently depend on it, it will not break or disappear. Halin will not support all 4.3.0 features, and you may encounter incompatibilities when using Halin with this version. Halin will continue to support 3.5 <= Neo4j <= 4.2 as it always did.

Click here to run Halin!

Primary features:

  1. Performance monitoring (system load, memory usage)
  2. Checks your Neo4j configuration and finds problems, makes suggestions on how to improve.
  3. Works with both Neo4j Enterprise and Neo4j Community, clusters and stand-alone.
  4. User & Role Management

Deploying Halin on Your Own (e.g. behind a company firewall)

  1. yarn install to pull dependencies
  2. Edit package.json locally, and change the "homepage" field to the URL where Halin will be accessible when deployed. For example, if you're using an internal company site https://fileserver.mycompany.com you might set this to https://fileserver.mycompany.com/halin
  3. npm run build to build the finished HTML/CSS artifacts; this will produce a dist folder containing everything needed to be deployed.
  4. Copy everything in the dist folder to your file sharing server; in the example above that would be a halin directory hosted by fileserver.mycompany.com.

Halin's build chain already does all of these steps; the public version of Halin is hosted in an AWS S3 bucket. If you'd like to see a scripted version of how that was done, check the .circleci/config.yml file, and look for the "Deploy to S3" step.

Running Halin in Development Mode

  1. Clone the repo
  2. yarn install
  3. yarn start
  4. Navigate to http://localhost:3000/

Running in Docker

docker run -d -p 127.0.0.1:3000:3000 --rm -t mdavidallen/halin:latest --name halin

Open a browser to http://localhost:3000/

If you'd like to build the Halin docker container from source:

docker build -t halin:latest -f Dockerfile .

Running in Kubernetes

Note that the docker container above can be run just as any other docker container; but you must specify stdin: true in the container specification; if you do not, Halin may restart repeatedly.

Running as a GraphApp

  1. Inside of Neo4j Desktop, go to application settings, scroll all the way to the bottom, enable development mode
  2. For development app entry point, use http://localhost:3000/ (that's where yarn start will serve the app)
  3. For development app root path, enter /absolute/path/to/halin where you cloned the repo
  4. Finally, inside of desktop you'll see a special tile labeled "Development App 9.9.9". This will point to your running copy of Halin

FAQ

  1. Does it support Neo4j Enterprise or Community?

Both. Some features must be disabled for community (such as user management) because community does not support them. For a comparison of features between Neo4j Enterprise and Community, see this link

  1. Does Halin capture any data about my database?

Halin runs purely in your browser and does not share details of your database outside of your machine. Even when you generate a diagnostic package, this is not shared. Halin does use a JS error reporting library called Sentry to help catch errors and improve the software, but nothing about your configuration is sent back to me. You may optionally tick a box in the diagnostics pane to "opt-in" share your diagnostic package, which helps us improve Halin & Neo4j.

  1. What versions of Neo4j does Halin support?

Supported: 3.4.* and 3.5.*; causal cluster and stand-alone, enterprise and community.

Unsupported: Neo4j < 3.4, embedded deployments, and HA deployments.

  1. How does Halin work?

Halin uses pure cypher/bolt connections to nodes in your cluster, and uses Neo4j's existing management interfaces and queries, including things like JMX, to obtain all statistics. This approach will work on any Neo4j instance with zero modification or configuration.

If you'd like to see what kinds of queries Halin is running, click on the gear icon at the very bottom and you'll get a pop-up showing all of the queries and their response times.

  1. Do I have to run Halin as an admin user?

No, but some types of data (for example your configuration) are not available unless you are an administrator. If you run halin with a low privileged user, some features may not be available or work as expected. An admin user is highly recommended.

  1. Why doesn't (this feature) appear in Halin? Why can't I use User administration?

Depending on how your database is configured, some options are not available. For example, active running query monitoring requires procedures that are only available in Neo4j Enterprise, so if you're running Community this feature will be disabled. This is not due to a limitation of Halin, simply which functionality the Neo4j instance exposes.

Additionally, there are some special circumstances. For example if you use an LDAP authorization provider, Halin will not permit adding users because Neo4j itself is not the authority on the user set.

  1. Why is it called Halin?

The application is named for Rudolf Halin an influential German graph theorist, who came up with (among many other things) Halin graphs.

  1. How much load does Halin put on my cluster?

Halin polls your Neo4j cluster by sending a variety of queries concurrently to each member of the cluster. Generally, these queries are answered very quickly by Neo4j, requiring a few milliseconds per query. The total load that Halin will place on the average small database, for example one running in Neo4j Desktop, will generally be about 1% of its load, with larger clusters experiencing a small fraction of a percent of the concurrent queries Neo4j can handle, and CPU. Further, these queries are strictly read-only (unless you are creating users/roles) and generally require little heap. All in all, the load Halin places should be negligible.

Installing Halin in Neo4j Desktop

Neo4j Desktop minimum version 1.1.10 is required.

  1. Open Neo4j Desktop
  2. Click on the "Graph Applications" tile at left
  3. Scroll to the bottom where it says "Install Graph Application"
  4. Enter the URL https://neo.jfrog.io/neo/api/npm/npm/halin into the box
  5. Click install

That's it. As Neo4j Desktop is installed, and as new versions of Halin are available, you'll catch updates.

Screenshots

Halin Screenshot: Overview

Halin Screenshot: Cluster Member

Halin Screenshot: User Management

Halin Screenshot: Diagnostics

Serve in Neo4j Desktop env

To serve it as a graph application, read the docs for that in the Neo4j Desktop development pages.

halin's People

Contributors

moxious avatar wagnerjt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

halin's Issues

Thresholds for alerting

For example on the performance pane set a threshold above which you get a system notification

Neo4j Desktop app shows a blank screen

Hi,

When I used to install Halin in the Neo4j Desktop I get a blank screen (neo4j desktop version 1.1.12 on Mac). When I do development mode than I see that the urls in the index.html to js and css resources are not resolved.
changing "/static/css/main.f976c12b.css" to "static/css/main.f976c12b.css"
and "/static/js/main.3d31f8ef.js" to "static/js/main.3d31f8ef.js"
will give me an UI, however in the log there are still errors when a there are references to 'root' resources with '/'. I do not know if this is a desktop-electron issue or an application issue

Investigate permissions issue when user is not admin

ClusterNode.js line 144 -- when loading config at cluster startup process, if the user doesn't have the right permissions this can create "Permission Denied" errors. May need a pre-query to probe whether the right permissions are in place before regular bootstrapping.

Check if auth is enabled prior to listing users / roles

46 | return this.cypher('CALL dbms.security.listUsers()')
47 | .then(results => results.records)
48 | .then(records => {
49 | sentry.fine('Got users ', records);
50 | // Convert to dropdown option format.

and

68 | return this.cypher('CALL dbms.security.listRoles()')
69 | .then(results => results.records)
70 | .then(records => {
71 | // Convert to dropdown option format.
72 | const roles = records.map(rec => ({

I have checked CALL dbms.procedures() there is no listUsers or listRoles at my enterprise version of neo4j, I have tried to check if there is a replacement of that procedure at apoc, but there is none ๐Ÿ˜ž

Any suggestions ?

Kees Changes

  • Rename the "Data" tab underneath of the node to "Plugins" and orient it around which plugins are installed
  • Remove the node count widget. Kees' dbAnalyzer program does it better -- is a linkage here possible?

Add "other" column for Store File Sizes summary table.

The Store File Size table has the file sizes for nodes, relationships, properties, logical log, strings, and arrays. Those will not add up to the total size of the files in the db directory if there are any foreign files in there. e.g. somebody created a db from a tar file in the db directory and forgot to clean up the tar (like what I did)

image

Besides completeness, having an "foreign file(s)" warning column could help avoid the scenario where the database directory runs out of space.

Detect leader re-elections

Need another data feed that polls dbms.cluster.role() not only just at startup, and refreshes leader status in the top tab across leader re-elections.

Add Dockerfile / docker support

It'd be useful to have a dockerfile where you could pre-configure a deployed docker halin to monitor a a particular cluster, for example one in kubernetes.

Topology view widget

a suggestion from a parter
show a cluster topology, perhaps as graph or in another visual format that adds info to each node.

with status colors :)

Page cache sizing rule

Ideal scenario: page cache = store size + native index size.

Look at the page cache setting. If it's >= store size + native index, it's good.

If page cache is > 50% bigger than necessary, then maybe you're wasting memory, that could be a warning.

Include script for one-time sampling and diagnostic package generation

Use case:

  • Run a simple command line script (no browser) or docker container that connects to a cluster, extracts all information and generates a diagnostic package.
  • Inputs: env vars for hostname, username, password, port, encrypted
  • Outputs: diagnostic package JSON and optionally advisor results.

Advisor Results from Diagnostics

Hi @moxious,

Thank you for writing this tool. It is fantastic to be able to see the configuration of all members of the cluster. I like the Advisor Results that show up after the running the diagnostics. However, I have a question about one of the findings I saw:
Your page cache setting is too small considering your store size. Store size=3860898203502 bytes, page cache 196000000000 bytes (196g)
I believe 196g is not small at all. The RAM on the server is 256 GB. Any thoughts?

Thanks,
Vivek

In line graphs, mark the leader

Line graphs will often display hostnames or IP addresses. The top bar tells you which is the leader, but it's visually confusing to have to figure out which is the leader when you have a triple line stacked chart.

Need to put a star in the legend of the line graphs to indicate which color on the graph is the leader, or otherwise change the label ordering so that the leader is always displayed in the first consistent color.

Investigate db.stats.*

yes the built-in procedures db.stats* will allow for this... in that first start the collection through `call db.starts.clear('QUERIES'); call db.stats.collect('QUERIES'); .... ..... .... run your queries as your normally do the call db.stats.stop('QUERIES'); call db.stats.retrieve('QUERIES'); the result of the last cmd is output similar to which could then be subsequently parse

Troubleshoot connection errors on bad port / scheme

Users have been making several mistakes that yield to connection failures, with bad driver error messages.

  • Host name: entering something that includes http or https
  • Port: trying to connect on port 7474 or 7473 and not 7687

Browser compatibility: Object.values

TypeError: Object.values is not a function
  at getFeedsFor (data/HalinContext.js:52:23)
  at sampleFeeds (ClusterNodeTabHeader.js:14:43)
  at apply (ClusterNodeTabHeader.js:30:48)

Check for user permissions

Read-only users don't have access to some things, like being able to list configuration. Need a way to detect user role up-front, and then selectively disable components they can't use.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.