Implement Text Labels in discord bot about open-assistant HOT 8 CLOSED

Would creating a bot that has Message Commands for each label work?

from open-assistant.

Would creating a bot that has Message Commands for each label work?

could work, yes. maybe it needs a bit of thinking when it comes to graded labels (each label can be between 0 and 1), but for binary ones this could totally work out of the box!

from open-assistant.

could work, yes. maybe it needs a bit of thinking when it comes to graded labels (each label can be between 0 and 1), but for binary ones this could totally work out of the box!

We could have the context menu open a modal with a select menu for the label and a text field for the label value.

If I start working on this, should I add it to the existing bot?

Also,

For context, I have been writing discord bots for 3-4 years. I looked around the existing bot code, and I'm not trying to be mean when I say this, but it looks like a mess.

• You are using prefix based commands (which is discouraged by discord) and are mixing manually parsing commands with discord.py handled ones. I would suggest using slash commands because they have better support from discord and are easier to use.
• discord.py was abandoned in the past so out of fear I switch all my bots over to hikari. IMO it's easier to develop with because it's actually typesafe (unlike discord.py). I've had a lot of issues in the past where discord.py would lie about its types which would lead to hard to track down errors.
• A lot of the feature logic is mixed with the bot logic

If you don't mind, I suggest switching away from discord.py to hikari. Given that the project is still fairly small, I would be willing to do a full rewrite if every one else who is working on the bot agrees.

Either way, I hope you can see how the current code is not welcoming to new development because it lacks documentation and structure, so a rewrite or restructure almost seems necessary. I want to contribute to this project, but in the current state I don't think I can.

from open-assistant.

thank you, that's extremely valuable feedback, and you're certainly not mean! I'd definitely support switching frameworks and / or improving the bot code. if you want and have capacity, you can have a go at it, very happy to incorporate expert opinions. if you'd like to delegate out the work, it would also be great if you made a few issues on the repo that break down the problems and suggest solutions, so other people can pick them up.

@andreaskoepf what do you think?

from open-assistant.

If I could get a list of features of the current bot (or the bot MVP) that would help a lot.

from open-assistant.

Hi @AlexanderHott .. nice to see a discord bot expert here & thanks for the valuable feedback! Sure, if you want to rewrite the bot using hikari that would be very helpful.

The main challenge for the bot is IMO beside the framework/impl details more the overall communication/workflow of the human feedback tasks. You almost certainly have a better overview than we over the bot-landscape and you may know what works and and what not. Could you ping me on the LAION discord server?

from open-assistant.

[will be copied for further discussion/planning to notion]

@AlexanderHott You probably already read the high level protocol documentation. In general a frontend (like the bot) can request a task at the backend. If a task of the requested type is available the backend sends it back for the human to provide input on/feedback for it. Depending on the task different interactions are necessary. A human feedback task can potentially consist of multiple steps.

Currently the following tasks are defined (neither complete nor final set):

summarize_story
rate_summary
initial_prompt
user_reply
assistant_reply
rank_initial_prompts
rank_user_replies
rank_assistant_replies

On of the critical decisions to make is how the users interact with the bot. Yannic was advocating for a group-interaction inside a channel that allows everyone to see posts of the other users (and potentially also interacting with them with emoji-reactions). Since different users will work at their own pace multiple tasks probably will have to be active at a given time and users should be able to request dedicated sessions. After prototyping some things I came up with the current structure of thread-based task-processing and self-destructing messages. Other earlier ideas looked much more cluttered and hard to follow. Different/better interaction concepts are up for debate.

Some requirements for MVP are:

• a minimal registration process with acceptance of terms of use for first-time users (e.g. privacy policy)
• configuration which languages the user is able to speak, or alternatively a way to switch to/select a single language for all subsequent feedback tasks that are presented to the user
• a tutorial with some examples that explains how the human feedback process works and which commands and options are available; giving some guidelines how good input for the bot looks like and what to flag/report when the user sees it
• template based fully customizable bot task messages
• support of tasks for the whole group inside a channel (probably task specific ui), discussion with @yk
• support of dedicated sessions requested by a user (e.g. either only for the user in the channel or via DM, user can specify type of task to work on)
• display of leader board and important news in the bot channel
• giving users a way to provide feedback to other user's messages and task-input messages (e.g. conversation-fragments) via reactions

Additional things:

• it is not clear how a group could work collaboratively on multi-step tasks, we could restrict group-tasks to single-step only [frontend would need a way to signal this to the backend]
• in dedicated user sessions: a way for the user to reject/skip a task (e.g. if the user doesn't have the required expertise)
• moderation commands: moderators must be able to disable submissions of malicious users and to prevent content that has been authored by them from being shown to others (e.g. manual exclusion) [requires backend protocol extensions]
• give feedback to users in a timely manner about the ranking of their submissions (gamification) - [requires backend protocol extensions]

from open-assistant.

@AlexanderHott You can see the current bot running in the test-discord here: https://discord.gg/DttpNT4p

from open-assistant.