honeynet / ochi Goto Github PK

I agree this ended up to be much more complex from a UX point of view.
I wonder if a better approach would be to show on the main page a list with all the enabled queries and the user can define the boolean operator between them. I will create a sketch to make it a bit more clear what I mean.
We should park this ticket until we find a solution that is understandable to the user from a UX point of view.

We should introduce a tcp and udp filter to include and exclude by transport type.
This is currently not supported as the honeypot needs to indicate the transport type in the event payload.

Leaving this here as there are going to be people wanting to discuss this:

There is an obvious possibility of a sensor IP address to be leaked through Ochi. I have addressed this for simple cases in Glutton.

For everyone else: If you hook up your sensor to Ochi, you are responsible for making sure you are not leaking your IP address. We will not sanitize payload data in Ochi.

Create a new page with a statistics overview. Statistics consist of aggregations of:

• Count events by destination port
• Count events by query match

Statistics are stored client side. They only persist in the local storage.

In PR I implemented query editing using inline form. It would be nice to use modal dialog same as for config.

If the event comes from a sensor with valid UUID, accept the event

Currently, we have a fixed-size input field limiting usability when providing multiple filter rules.
Look at Greynoise input field for inspiration.

• On focus, expand the input field's width
• Add a clear or reset button to the expanded input field to remove all filters
• Add additional lines to the input field if the text exceeds the field length

We pass service configuration as parameters. We should use a config file instead. Create config file if it doesn't exist. Default config shoold have configuration for local testing.

Use github.com/hashicorp/hcl/v2 for the configuration.

The backend will have access to binaries for each architecture.
UUID of the sensor needs to be baked into binary. Use the following snippet to replace a placeholder sensor UUID in the binary:

// IndexReplace uses the optimized offset finding
func IndexReplace(b, old, new []byte) []byte {
	i := bytes.Index(b, old)
	if i == -1 {
		return b
	}
	copy(b[i:], new)
	return b
}

Currently, hot reloading is only available for svelte frontend. Ideally we can run golang backend with svelte frontend reloading both. To support that we need the following:

• google authentication not working on localhost:8080
• live reload go backend with air or something similar
• make svelte frontend fetch data from golang backend on different ports. I partially implemented that in #77.

The App.svelte has a lot of hard-coded layout glory. This makes is rather difficult to add new layout elements without breaking the current layout. Even adding new pages would be challenging. Here are some offenders:

Redo the layout allowing for:

• Dynamically resize the window
• Adding more elements (like separating the right column)
• Adding a new page (e.g. rule management, user profile...)

A GitHub workflow should be set up to test the formatting of PR and new pushes.

Outcome: Queries can be tagged

Query metadata:

• A tag to be applied.

Tags:

• Tags have a type and a value

Flow:

This issue depends on #6

It would be nice if the width of the component was dynamic in response to the current window width. When the window is too narrow, we can decrease width parameter passed to hexy.

Create a new page with statistics

• Events by sensor ID
• Events by global/public queries
• Events by port

There is an endpoint for statistics.
There is a websocket with statistics updates.

Add a confirmation modal.
Add a new endpoint DELETE /sensor/:uuid that removes the sensor.
Make sure that only owners can delete a sensor, otherwise return 404.

We introduce the term Queries for filters and labels.

Goals:

• Being able to filter events by more than just destination port.
• Instead of filtering events, apply a label to an event. E.g. assign a protocol to an event.
• When picking Apply, add the filter to a list of filters.
• User should be able to delete a filter.
• User should be able to edit a filter.
• On a new event, apply all filters and labels.

Approach:

• Consider wireshark https://wiki.wireshark.org/DisplayFilters as an option for the syntax .
• Consider waxeye for DSL https://waxeye-org.github.io/waxeye/index.html
• Consider using js-dsl for DSL https://venkatperi.github.io/js-dsl/
• Consider chevrotain (although I had some dependency issues, see dsl branch)

Branch:

• I started a little on this, branch here: https://github.com/glaslos/ochi/tree/dsl (some issues handling the library as dependency)

Example Queries:

• tcp.port eq 25 (shows only TCP traffic with destination port 25)
• ip.src eq 192.168.0.0/16 (show only traffic from IPs in this subnet)

Outcome: On-demand, create a permanent link to an event.

• Add a "Share" button on the Content component
• On click, store JSON to the database (BLOB) with a new UUID (UUID: string, user_id: string, event: blob)
• Create a route /events/<uuid> reusing the Content component, showing the event from the JSON blob.
• Use the go server for routing.

• Create a backend repository with sensors. ID should be UUID and it should have a name and owner ID.
• Create a new endpoint in the backend /sensors that returns the sensors by authenticated user ID
• Add a new page that lists all the sensors in a table

I need to provide a valid filter query and can't reset the filter by providing an empty query.

Add a config for Prettier. Initially should set the line length to something reasonable.

We use hexy to display the payload as hex and ASCII, but if you want to mark and copy multiple lines of the ascii you end up also copying hex. Modify hexy so the HTML output has the hex and ASCII in two separate columns instead of line by line.

Outcome: Stop following new events on scroll, resume with button

When scrolling in the events list, stop following new events. There should be a button at the bottom of the events list that resumes following events.

Currently the config modal code is in App.svelte but should be moved to Modal.svelte.

Section could be footer, header or grid element.

• Should have a link to this GitHub repository and a call to action to contribute: "Contribute to this project through GitHub".
• Should have the following copy: "Ochi - Honeypot Data Feed Interface".

We should stop Action if generate-check produces a different file.
Developers should always re-generate changes and check in all changes.

Before publishing to the websockets, truncate the sensor UUID to only the first part until the first -.
Remove truncating from the front-end code.

Add Apply button to each saved query implemented in #77.

Readability can be improved by adding custom css styles to alternating lines.

Endpoint returning a config for Glutton with http producer enabled and pointing to Ochi with sensor ID and secret.
Events from sensors with configs from this endpoint are by default private and only visible to the owner.

Add sensor ID to the event list/details. Probably a synonym and not the actual sensor ID.

Disable sensor configs sending invalid events.

Have a list of sensor configs per user. Users can CRUD sensor configs.

In #46 we introduced the initial version of filtering syntax using chevrotain. We need to improve the syntax by adding new features.

Some things to consider:

• Should we support operators like contains or matches?
• Should we support in operator?
• Should we support "unary" clause (for example tcp, ip.src, etc)?
• How can we filter by payload? #57

Currently we parse the event filter using chevrotain DSL definition when user types. When user clicks Apply, we parse the same filter again. We can do some optimization by persisting the result of filter parsing on input change and reusing it on pressing Apply button. Things to consider here is that persisted event might be out of date or invalid.

Ochi is participating as part of the Honeynet project in the Google Summer of Code 2023 (GSoC). Ochi is looking for engaged and motivated students to join us during 12 weeks of exciting collaboration. This is a very new project so there is a lot of potential to work on the foundation of it.

Ochi is a web interface for live Honeypot events to correlate, filter and annotate.
As part of the GSoC'23 we would like to add the ability to filter the events using a version of the Wireshark query syntax as a DSL. Queries should be persistent and the backend should calculate metrics how often a query is matched.

• #4
• #6
• #8

Furthermore the backend should support perma-links to events so they can be shared:

• #5

Primary skills required are HTML/CSS/JavaScript. Secondary would be Golang but this can also be covered by the mentor of the project.

• Add a Google sign-up/sign-in button to the to right corner
• Callback should go to the backend and create a user and return a session ID
• Use sqlite3 as a persistent database, a new db should be created if none found.
• Login user if already exists
• User receives a session/(jw?)token for future authentication.

Button opens modal with form for sensor name.
Create a new backend endpoint POST /sensors to create new sensors.
Sensor names are unique per user, not globally.
Confirming form creates a new sensor entry in the database with the chosen name.

Add a button to open configuration to the top bar.
On click open a model to configure:

• The number of messages in the view: Currently set to 50
• Switching between dial() and test()

This is crashing the app and I need to reload.

Add a button in the content section to download the raw payload.

This is a feature of chevrotain, documentation is here
I don't think we need to be concerned regarding the performance but have a look at the related section in the documentation.

Outcome: User can save and load their queries.

Metadata:

• Boolean if rules is active
• The actual query as DSL
• Description of the query

Flow:

• If the user is signed in and has an account:
• A save button added to the query input
• Save button opens a dialog asking for a name for the query, confirm saves the query
• Create a new page with a list of all saved queries
• From the query page, queries can be loaded and applied to future events

This issue depends on #4

Add a new endpoint GET /configs
Add a new endpoint GET /rules

Populate the config with the Ochi details.