emnudge / logpipe Goto Github PK
View Code? Open in Web Editor NEWInspect your logs
Home Page: https://logpipe.dev/
License: MIT License
Inspect your logs
Home Page: https://logpipe.dev/
License: MIT License
1h2m3s
and (12.31s)
are common for timing systems.
Let's add a new time
type for these.
Also should fix 2.332
(decimals)
We're currently doing some unconventional things Shoelace is getting in the way of our movement.
We need to take over control so that ArrowUp and ArrowDown are not intercepted.
We have a pretty bad perf situation with massive log inputs.
Our biggest obstacle is rendering. Querying itself seems performant, even on massive datasets. But rendering is where we choke.
This should link to the github, docs, and display the current version.
Silly me, just realized that all of our docs saying to use logpipe --port <number>
were wrong.
Right now node swallows these args, so we need to do logpipe -- --port <number>
which looks pretty annoying.
There should be a way to get around this. One way is to capture the arguments before passing it to node through bash.
Currently we do text-based filtering for tags.
This is done so we can re-use the text filter box while still supporting a helpful feature.
However, text-based filtering has some disadvantages. Here we see that Info
is selected, but both [INFO]
and Info
are filtered:
To fix this, we need to do 2 things:
"tag"
instead of raw texttype=tag
, but it needs to be something people wouldn't normally type.We can use subshells to pipe both stderr and stdout to logpipe.
If you want to separate these 2 output streams, this can be very helpful.
my_program > >(logpipe -t "stdout") 2> >(logpipe -t "stderr")
Also 2>&1 |
can be shortened to |&
in ZSH or newer versions of bash. I should add that as well to the docs.
Note
As a future idea, it might be helpful to allow using this to combine input streams while still preserving their origin.
i.e. we can have some logic which combines all input streams that pipe out to the same port.
If we tag the stream with some ID or title, the frontend can easily differentiate the input streams.
my_program > >(logpipe -t "stdout" -p 8080) 2> >(logpipe -t "stderr" -p 8080)
The current system will have them on different pages which may be less ideal
Our parsing logic does not properly remove all ansi escapes - particularly the reset when at the end of a message.
The extended tag container is still having trouble with some massive tags.
Another approach is capping the length of tags in the display. Maybe doing [tag name...]
if it's too long, but keeping the full tag in some data-key.
It's usually complete fine, but when using multi redirection such as in #44, we get a confusing output where it may be hard to tell which command started which port.
❯ my_program > >(logpipe) 2> >(logpipe)
Logs are displayed on http://localhost:55716
Logs are displayed on http://localhost:55717
We can fix this if the user supplies a title. Currently this won't resolve the problem, even if the user supplies a title.
logpipe will swallow all logs. We should document the use of tee
to place logs into a file before logpipe gets it.
Something like
my-program |& tee logs.txt | logpipe
Make a new view in the command palette that shows a "help" view with useful information from the readme.
It should include
As I keep adding features, it's become useful to check what version we're running.
I've had times where something was confusing me, but it turned out my installation was pulling from an old version.
Some users have mentioned that the automatic log entry is jarring, despite this being the current behavior of every terminal.
I fear adding an animation here will have implications for future perf work in virtual lists. However, it could be interesting to explore.
As I mentioned on the reddit thread about it make the video stand out more in the README. I didn't know to look for it and only your comment on reddit made me look.
Like Warp's feature, it could be nice to get a ping when the command finishes executing.
I don't have a windows machine to test, but currently we're only capturing Mac's Cmd+K.
Windows browsers remap ctrl+k to some browser search functionality, but enough websites (github, angular docs) still use this mapping that it should be fine. We just need to e.preventDefault()
.
I tried viewing some large logs (>1k lines), but after a few seconds of loading the website mostly stayed empty, and logged this:
Uncaught (in promise) InternalError: too much recursion
replacePath http://localhost:37647/highlight.js:155
highlightText http://localhost:37647/highlight.js:205
getLogEl http://localhost:37647/index.js:42
onmessage http://localhost:37647/index.js:86
onmessage http://localhost:37647/index.js:86
It also shows a line count, albeit incorrect, but no lines:
(Firefox 124.0.1, NixOS, Node v21.7.1)
Broken off from #16
We've added the context menu in #19. Now it's time to add the "jump to log" action.
This is similar to Discord where you can search for messages, but then view the surrounding context right after. There's often important information around the message that you won't see when performing a regular search.
Centering the log output makes placing an optional "settings" menu difficult.
Cramming everything into a command palette has been surprisingly effective for previous projects, as brutish as it sounds.
Possible features:
--title
flagThe last 2 are related to direct filtering. I don't know that the command palette is the best place to do it, but it can be a convenient area if no other ideas crop up.
Unrelated to the main project goals.
It was expressed that, being a web tool, we're in a good position to provide an easy view into the tool's behavior without asking people to install a package directly from git.
We can make a dedicated branch that we regularly rebase off of master. We can then either
index.js
file to take input from some functionThe 2nd option is more interesting since it lets us almost completely preserve the main code (which makes rebasing simpler), but there are some issues with service workers taking a second to register on the initial load, making for a poorer user experience.
We'll likely move to a non-networked version unless it becomes important to communicate that the demo uses SSE at some point in the future.
We can trivially get around problems by moving away from the current use of .innerHTML
to DOM mutations. This also let's us remove that glaring warning in the readme.
We need to parse files better.
Right now we only parse files if it ends in :\d+
, but we should also match
:\(\d+,\d+\)
I've written this tool with as much vanilla HTML as I can.
It's a bit unusual, but I do this:
That last bit is primarily for people who aren't web devs, because deviating from a framework actually makes the code a whole lot more difficult since you remove the "style guides" these frameworks make.
Luckily, there is probably a way to bring in component behavior without using a new framework.
If we make our "components" based on HTML pages, we can keep the idea of a "component" model still.
Take our command palette for example. Right now, it has its HTML inside of our index.html file. The JS for it is sizable so we put it in a separate command-palette.js
file. But the CSS is together with the rest of the style.css
data.
What if, we instead imported the command palette in our HTML as if it were another HTML file that we inlined.
We could have a directory
command-palette/
index.html
style.css
index.js
The HTML makes reference to local files and by inlining the HTML, we get the entire related component tree.
The JS file can still communicate with the rest of the codebase by importing utility files where we might store state. It's a single use component, but we get the benefits of keeping it in one place.
We currently do text-based filtering exclusively. This is limiting and we have the power to do something more interesting.
This stems from #25 where we described the need for a query language.
For something really basic, we can support
type=<classname>
in text boxes which run a queryselector on each log item for matching node types. This way we can re-use our syntax highlighter for the query system.
An initial issue I had with this approach is that it may limit us if someone has a log containing "type=" and wants to search for this content. We can initially support instead lp-type=
which is unlikely to clash with anything for now.
We can also fix the issue with people copy/pasting logs directly into the search box by creating a "search for logs like this" option to the context menu. Also a really cool idea that would be based off of the work in this feature.
This is gonna be really cool!
This is a cool project.
Just so you're aware, in the ancient times, sysadmins would use CCZE.
=> https://github.com/cornet/ccze
Add a version type for v<num>.<num>.<num>
or just any 3 numbers in general that are dot-separated.
Don't parse time if it's inside of a word. It should start and end on a word boundary.
Might want to also look into ANSI parsing not removing the first few bits of ANSI from certain messages.
One disadvantage of using the web browser is that there's no shell history to refer back to.
It may be helpful to allow saving the current log state. Syntax highlighting is automatic, so we only need to save timestamps and raw log chunks.
This feature forces us to start thinking about storage solutions. This can't be done in the browser since we change ports (and therefore origins), making it impossible to share data between sessions.
Sometimes it's helpful to copy/paste a log's text content.
Sometimes, you want to document some strange behavior and it's not as important for people to examine the text contents.
Screenshotting the view manually can be a bit messy. We can add a command palette option to automatically screenshot the current view.
I don't yet know exactly what this will look like, but it's an interesting idea. We could also embed the log's content into the png via a TXT
section. I've done this work in earlier projects, so it shouldn't be too difficult.
A few things are missing from the log filtering that I think would be a pretty big QoL feature
A lot of issues can be solved by simply using a much larger terminal space.
I like keeping it small for simplicity's sake, but I understand how this can introduce a big burden for some people.
We also may want to look into word-break CSS.
Allow the user to change their current theme using the command palette.
This also makes for a good follow up on allowing the user to save data using Node.js (so it sticks between hosts). Baby steps.
It is impossible to query for content which contains quotes at the moment. We should fix this.
Some commands use ansi escape parsing, but just color the entire input one color.
This is not very useful. It might help to differentiate the inputs form the rest in the terminal, but it's actively unhelpful with our automatic syntax highlighting.
We can do this in two ways. We should perhaps add both.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.