Comments (9)
@kayrus we have a project now to find a way to do this.
from hermes.
Hermes is the API in the process. In order of process
Openstack Service (nova/designate/etc) - Could do them, but requires a lot of updates to each openstack component
->
Openstack Audit Middleware https://github.com/sapcc/openstack-audit-middleware - Could do them, requires a consistent service to do lookups
->
Rabbitmq - No
->
Logstash - Bad idea, would require a service for lookups
->
Elasticsearch - No
->
Hermes Api - All attempts we've made in the past to do lookups for names has overloaded the identity service (identity lookups), and we were told to stop.
So the main question I have is, do we have, or can we(should we) build a lookup service that can handle a ton of lookups for all sorts of varying services.
That's the main impediment. From there, we can sort out a place for them.
from hermes.
Ok, then it means it's expensive. I cannot imagine that identity lookups have overloaded the identity service. Did the lookup logic use cache?
from hermes.
I wasn't involved, it may very well have been that the implementation was flawed. I recall there being a cache, but am unsure the specifics. If we can add lookups that don't overwhelm the identity service, that's reasonable.
from hermes.
You haven't provided your opinion on the client side lookup. Will it be a better alternative?
from hermes.
Ideally it should be in the audit process, and held in elasticsearch so that the data is the same in all locations. The client giving different answers to the dashboard feels unacceptable.
from hermes.
If we can add lookups that don't overwhelm the identity service, that's reasonable.
Do we need involve someone else to discuss this?
from hermes.
A POC for this is in process, and work should be done this sprint to implement a first run on it.
from hermes.
This is implemented via enrichments, if you have specific events you would like detail on, please request it to me. It is not done via the client, but through our enrichment process.
from hermes.
Related Issues (20)
- Migrate to ES cluster and increase data retention HOT 1
- Gorilla Mux is archived and no longer maintained. HOT 1
- observer name inconsistent
- Rate Limiting for Hermes API
- Sort isn't working on all fields HOT 1
- Limit for attributes is applied before the max-depth filter HOT 4
- Support OR and NOT conditions for the filter HOT 5
- Show project resources events made from the system scope HOT 2
- Date ranges via Dashboard no longer functional HOT 1
- Could not load attributes if no events HOT 1
- Hermes API crashes, when attributes are requested from an empty index
- Log user token details HOT 1
- Allow listing all events from a system (cloud admin) scope HOT 2
- Details to events API HOT 1
- Max Limit for ES Queries is limited to 10k. Can we increase? HOT 2
- Dependency Dashboard
- Document Client sending Audit Events HOT 1
- Add search over entire event HOT 2
- Replace Depricated olivere's golang es client HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hermes.