Windows 10
Microsoft Edge Chromium Version 85.0.564.41

E.g. see: https://logs.ivr.fi/channel/pajlada/range

Returned are messages from 2019-06-27 and 2019-07-27 so this is basically [now, now - 24h] ∪ [now - 30d, now - 31d] when it (I'm guessing) really should only show the last 24 hours.

querying by id like id:77829817 returns a 500 from the list API and doesn't load any logs

Attempting to build justlog returns the following errors:

# github.com/gempir/justlog/bot
bot/main.go:54:14: twitchClient.OnNewMessage undefined (type *twitch.Client has no field or method OnNewMessage)
bot/main.go:76:14: twitchClient.OnNewClearChatMessage undefined (type *twitch.Client has no field or method OnNewClearChatMessage)
# github.com/gempir/justlog/api
api/channel.go:119:23: assignment mismatch: 2 variables but 1 values
api/channel.go:187:23: assignment mismatch: 2 variables but 1 values
api/user.go:163:22: assignment mismatch: 2 variables but 1 values
api/user.go:236:23: assignment mismatch: 2 variables but 1 values
api/user.go:305:23: assignment mismatch: 2 variables but 1 values

show the display name in the frontend

We could query multiple backend instances of justlog. We just keep a list of every available justlog in the config.

The backend should handle the distribution part the frontend doesn't need to know too much about it. Maybe an option later to switch between different instances incase we have duplications of channels.

• So add field in configs for array of instances
• On startup and maybe periodically fetch /channels from each instance to find out logged channels
• When querying for a channel find out if we log that channel, if not query another instance

After the go-twitch-irc rework, all logs started showing blank lines where timeout seconds used to be.

API should be documented.

Swagger was a pain last time, maybe try something else or just atleast document it via Markdown like the Admin API.

The new randomquote reads multiple files for a lot of users, which is pretty inefficent. It should cache older months in redis or in memory elsewhere.

so there would be capability for an api request to disable mod action logging just before a large amount of bans is executed, and then re-enabling it once the bans are done

just the api verison of manually typing

!justlog messageType channelname 1
!justlog messageType channelname reset

https://logs-auth.ivr.fi/?channel=vadikus007&username=nicekcoc

Only visible when emotes are turned on

Hasing the js file would prevent overly long caches

Example: https://api.gempir.com/channel/forsen/user/skyggepikken

Downloads in the browser instead of showing as plain text, because the server returns Content-Type: application/octet-stream:

$ curl -L -i "https://api.gempir.com/channel/forsen/user/skyggepikken"
HTTP/2 303
date: Wed, 23 Jan 2019 13:08:09 GMT
content-length: 0
set-cookie: __cfduid=d3d080101ed7bdcd32f0f33a1bba3afea1548248889; expires=Thu, 23-Jan-20 13:08:09 GMT; path=/; domain=.gempir.com; HttpOnly
access-control-allow-origin: *
location: /channel/forsen/user/skyggepikken/2019/1
vary: Origin
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 49da7cc6bcdc2d7d-TXL

HTTP/2 200
date: Wed, 23 Jan 2019 13:08:09 GMT
content-type: application/octet-stream
content-length: 122
set-cookie: __cfduid=d3d080101ed7bdcd32f0f33a1bba3afea1548248889; expires=Thu, 23-Jan-20 13:08:09 GMT; path=/; domain=.gempir.com; HttpOnly
access-control-allow-origin: *
vary: Origin
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 49da7cc87e5c2d7d-TXL

[2019-01-23 12:57:02] #forsen skyggepikken: USERINFO
[2019-01-23 12:57:03] #forsen skyggepikken was timed out for 300:

In this example, there are some weird unicode characters in that first message:

Either implement a way to show a channel's logs for a given day on the current site or an easier way to access the txt version from the site.

Justlog tries to send all IDs in the config to helix at once regardless of the maximum of 100 IDs limit in helix, causing justlog not to join any channels at all on startup.

Deploys currently are annoying because they cause downtimes of like 1minute.
Current deploy:

• stop service
• scp file onto server
• start service

New deploy:

• scp file to server
• stop service
• symlink new file
• start service
• remove old file

Regarding these logs: https://api.gempir.com/channel/forsen/user/fourtfbot/2017/11

Visiting the link downloads a file called "11" for me (using Chrome on Windows).
This seems to be related to this Chromium issue: https://bugs.chromium.org/p/chromium/issues/detail?id=106150 and this StackOverflow post: https://stackoverflow.com/questions/9660514/content-type-of-text-plain-causes-browser-to-download-of-file

I spun up a small local test server and was able to mitigate the behaviour by setting the X-Content-Type-Options: nosniff header.

Adding ?type=raw at the end of a query would make the api just return a list of raw irc messages. useful for debugging stuff in chatterino. not 100% sure if type is the right query variable

Add /channels Endpoint to show all joined channels as usernames

• global ffz (just incase)
• channel ffz emote
• bttv emote
• sub emote

This could be solved by two ways:

1. Make a seamless deploy system
2. Allow the bot to join a new channel (and save the channel to the config) during runtime, using a chat command (e.g. !justlogjoin channelName)

or both! :)

There should be an option to only log privmessages or ban messages etc. for specific channels because some channels use banlists and ban a lot of users at once which is mostly unintersting to log.

Currently, when a user changes their name or deletes their account, its no longer possible to access their userlogs via the old name or their now-deleted login name.

I think it would be a good feature to keep a long-lived or even permanent database or cache of username -> ID mappings, and to refresh them periodically as they are used (e.g. when a user types in chat -> save the mapping as a fresh mapping. When a users logs are looked up, use the database if available and if the entry is not older than 1 day, an hour or w/e). If the entry is expired, fetch a fresh lookup and use the new user ID if twitch returns one (then update the DB too), otherwise use the stale one from the DB.

Not sure what volume of user ID -> username mappings that would be, but i imagine the lookup could improve performance quite drastically as well, since users who typed recently in chat would be cached, and those are most likely to be looked up. Not sure if redis supports this kind of data, or if a relational database fits better. (thats implementation detail :) )

• Vertical layout
• most recent top?
• Scroll down to load older months
• download as txt button (jump to txt)
• id:123 to query by id
• ffz
• bttv
• handle 404s

zgrep/grep over list of logs?

• API Endpoint to search over entire list of logs

New version of gempbot is running on justlog.gempir.com we should migrate the current logs to the api domain. Once we have the data migrated

https://discuss.dev.twitch.tv/t/requiring-oauth-for-helix-twitch-api-endpoints/23916

Helix will soon require more authentication when requesting userids.

Deadline: April 30, 2020

At the moment, you can only have the current month and one past month open. If I have December 2019 open and try to open September 2019, for example, December won't stay open.

Suggestion: make any number of months stay open at the same time.

RQ is currently not really random since it first randoms the years, then the month etc.

For example, these currently do not redirect:

https://api.gempir.com/channel/forsen/user/pajlada/

https://api.gempir.com/channel/forsen/

config file should be editable while the bot is running.

As a start we could have a command like !justlog cfg ...
And change a value or so.

We need join channel, part channel, specific message types for channels etc.

Prevent crawlers from indexing logs

Example how it was done previously:
8237afb

Example: https://api.gempir.com/channel/pajlada/user/swiftapples

Last logs are in october: https://api.gempir.com/channel/pajlada/user/swiftapples/2018/10
The first link redirects to january though!

Since there's no current search option (I saw it's on the to do list), have an option to disable embedded emotes, so that you can search for their specific text (for example, if you're searching for a specific moment when you remember what emote you've typed).

Example of URLs that do not work:
https://logs.ivr.fi/channel/Pajlada/user/Pajlada/2020/9 (Mixed case channelName, Mixed case userName)

Example of URLs that do work:
https://logs.ivr.fi/channel/Pajlada/user/pajlada/2020/9 (Mixed case channelName, lower case userName)
https://logs.ivr.fi/channel/pajlada/user/Pajlada/2020/9 (Lower case channelName, mixed case userName)
https://logs.ivr.fi/channel/pajlada/user/pajlada/2020/9 (Lower case channelName, lower case userName)

add options for json or plain text response specified by the Accept request-header field

Go 1.14

Incompatible with newest helix package

https://i.alazymeme.com/EmbarrassedDampImage.png
Some lines have " /> at the end of them

Emotes can lagg out the browser. There should be an option to turn them off.

Some channellogs (forsen e.g.) can grow so big that the request can time out before the server has the response ready for sending. I think it would be quite benefitial if justlog transformed the input line-by-line, writing the transformed lines to the http response as they are generated (streaming), so that the response can start being sent right away after the request is made, and not when the whole file has been transformed in memory.

https://api.gempir.com/channelid/11148817/user/77829817

is the endpoint for Ids, it should be .../userid/...

Also ah nice option would be that the normal API can parse userids aswell and mix match usernames and ids. Maybe check if user exists via twitch API and if not try it as userid.

I would like to be able to link to a specific part of a users log

Use case:

• Moderator goes to my bot's dashboard, looks up moderation actions for a user and sees who timed the user out. Dashboard example https://i.imgur.com/O8xkJ9S.png
• The moderator wants to know why the user was timed out, so he needs log contexts.
• If I could I have a "logs" button next to each moderation action that would link to your logs at the time he got timed out, that would be pretty neat

Parameters I can imagine sending:

• Channel ID, User ID, Timestamp in UTC

If it's easier, I would only be interested in messages before the timestamp. but getting more messages below it wouldn't be an issue, as long as it can be marked out properly where the last message before the timestamp exists

NaM

/channel/:channel/user/:user/:year/:month works fine, however...
/channel/:channel/user/:user/:year/:month/:day makes many 302's and after several redirects it throws me to this weird path: /channel/zneix/user/zneix/2020/8/20/2020/8

when an emoji is placed before an emote, the emote renders and then the first letter following the prefix is shown

I host an instance of justlog with a friend. We would both like to use justlog's commands and change channels on the go, so is it okay to turn config.admin into an Array of strings as config.admins intead?

i.e. resubs and subs, mainly.

https://dev.twitch.tv/docs/irc/tags#usernotice-twitch-tags

USERNOTICE commands are always bound to a user but don't always contain a separate user message - this is only if the message tag is present. I think USERNOTICEs should always be rendered regardless of whether a user message is present.

It would be nice to just reset the message types that are logged to the default option per command

I would like to have API endpoints for adding and removing channels from Justlog.

Proposed interface

Adding channels

POST /admin/channels/?id=...&id=...&id=...
Would add channels identified by ids into the config

Removing channels

DELETE /admin/channels/?id=...&id=...&id=...
Would delete channels identified by ids from the config, but not their logs.

When visiting /channels/:channel/user/:user, it redirects to the current year and month currently. I am suggesting a different endpoint (maybe /last) or to change the redirect behaviour so it directs to the last month the user typed, e.g. when calling that endpoint in January 2019 and the user's last log line was in November 2018, the endpoint would redirect to /channels/:channel/user/:user/2018/11.