Comments (7)
Hi, I would like to contribute to this task, but before this I have a couple of questions to ask:
- What do you consider to be a thread? From what I can see right now in the code a thread is only formed by a pair of ("task-started-post", "user-reply-post"). It feels like all messages of one conversation should share the same thread, is it correct?
- Do you consider only linear conversations? With parent post id one could construct a number of conversation variants branching from one or more intermediate posts.
Also are there any other docs describing the algorithm, data scheme and interaction protocol you would like to implement besides the readmes and high level protocol architecture article on Notion? This would help a lot in understanding :)
from open-assistant.
hey thanks so much for being here :)
What do you consider to be a thread? From what I can see right now in the code a thread is only formed by a pair of ("task-started-post", "user-reply-post"). It feels like all messages of one conversation should share the same thread, is it correct?
the idea (which might not yet be fully reflected in the code) is that the interactions of the system with the users result in tree-structures. At the root is a prompt, which a user has entered. We then send out this prompt as a task to multiple users, each of which is tasked to "play" the assistant and give a response. Given those responses, we can do two things: First, we can have other people rank the responses, which will be useful for training the reward model. Second, we can take each response and form a thread from (initial prompt, assistant response), and give each of these threads as a task to multiple users. Thus, each thread results in multiple replies from users (who are now "playing" the user). Again, we can rank those and also combine each (initial prompt, assistant reply, user reply) as one thread to continue the tasks.
So, in my view, a thread is a chain of messages between user and assistant. We build threads by going down one path of the tree we collect by people fulfilling their tasks. I hope that makes it a bit more clear and answers the second question: The result is always a linear conversation, but during data collection, we collect a tree.
Also are there any other docs describing the algorithm, data scheme and interaction protocol you would like to implement besides the readmes and high level protocol architecture article on Notion? This would help a lot in understanding :)
Not yet, no, but you're right it's important we improve this!
from open-assistant.
Hi there, thank you for an explanation, it helped a lot :)
I guess PostgreSQL is capable of recursive queries to walk over the posts of one thread using parent_id, but for now I'd suggest a much simpler way to sample a conversation:
- (DB) pick random thread_id (which is the root post id)
- (DB) select all posts with this thread_id
- (Backend) pick a random message
- (Backend) trace back the conversation line back from this post to the root using parent_id
Later it might also be useful to have a more customizable way to pick the last message of a conversation (the one we choose at 3.) instead of a random one.
I also have some suggestions for the data model:
- Currently ACK request generates a new post entry which is used to link subsequent interaction to the "task description message". I think work_package (aka task) is a better place for this link, it will keep posts clean and easy to work with. So at the time of interaction the task may be identified by (frontend_key, user_id, task_message_id). On the other hand post.frontend_post_id seems unnecessary in this case, I can't see other use for it yet.
- When the user replies to the task message (user_reply task) and a new post is inserted we need to link it to its thread and parent post. At this moment the information from the frontend is sufficient only to identify the task. Thus we need to link the task to its post of origin, parent_post_id should do the trick.
- Posts may need a flag to designate the end of this line of conversation. In this case we'll also need a way to determine whether this post is a valid end of a conversation, maybe a new task type.
- If the data is generated not only by users but by the RL agents as well then the posts may need to have a flag to distinguish between them. If not could you please clarify how
is_assistant
field is calculated when the conversation is formed in the backend? Should it just be an alternating pattern starting from the initial prompt?
As a bonus I've also updated the sequence diagram of the interaction between user and backend. Now it should better reflect what happens at the time of a request:
sequenceDiagram
par Task Creation
User DMs / Channel->>Frontend: Asks for a new task
Frontend ->> Backend: Requests new task
Backend ->> DB: Inserts new task and links it<br/>to the thread and the parent post
DB ->> Backend: Generates task_id
Backend ->> Frontend: Returns the task with type and payload
alt Frontend displays the task
Frontend ->> User DMs / Channel: Presents the task in a message and asks the user to interact
Frontend ->> Backend: Sends ACK with presented "task description message" ID
Backend ->> DB: Updates task with "task description message" ID<br/>and set "ack"=True
else Frontend failed to present the task
Frontend ->> Backend: Sends NACK for a given task
Backend ->> DB: Updates the task with "ack"=False
end
end
par Task Fulfillment
User DMs / Channel ->> Frontend: Interacts with the post (reply, rating, etc.)
Frontend ->> Backend: Posts interaction along with "task description message" ID
Backend ->> DB: Updates the task with "done"=True
Backend ->> DB: Inserts new post with the interaction data<br/>and links it to the parent post using the data from the task
Backend ->> Frontend: Sends "Task Done"
Frontend ->> User DMs / Channel: Replies "thank you"
end
There may be an insignificant theoretical race condition when the user responds before ACK has been delivered, but I think backend should back this request off and frontend should handle it.
This diagram is also available at https://www.notion.so/Open-Assistant-protocol-25a906a38efd488cac9414a6c8022e24
from open-assistant.
Currently ACK request generates a new post entry which is used to link subsequent interaction to the "task description message". I think work_package (aka task) is a better place for this link, it will keep posts clean and easy to work with. So at the time of interaction the task may be identified by (frontend_key, user_id, task_message_id). On the other hand post.frontend_post_id seems unnecessary in this case, I can't see other use for it yet.
Yes, you're right. The initial goal was to make the frontends as stateless as possible, but this seems less and less viable, so we'll have to live with a degree of state, i.e. for the frontend to at least be able to link some reply to the initial message of the task (could be done by traversing reply structures). The issue is that there might not always be the exact mapping of 1 task description <-> 1 message, but I guess when that breaks, we can think of a solution then.
When the user replies to the task message (user_reply task) and a new post is inserted we need to link it to its thread and parent post. At this moment the information from the frontend is sufficient only to identify the task. Thus we need to link the task to its post of origin, parent_post_id should do the trick.
Agree
Posts may need a flag to designate the end of this line of conversation. In this case we'll also need a way to determine whether this post is a valid end of a conversation, maybe a new task type.
I suggest we attach some "depth" level to posts, i.e. how far down the conversation they are, so always their parent's depth plus 1, then we can either have a hard cap on depth or we just sample in inverse proportion to depth or so.
If the data is generated not only by users but by the RL agents as well then the posts may need to have a flag to distinguish between them. If not could you please clarify how is_assistant field is calculated when the conversation is formed in the backend? Should it just be an alternating pattern starting from the initial prompt?
The current idea is that threads are turn-taking, but does not distinguish whether the system created one message or a human. I.e. is_assistant
just indicates that the message is written from the perspective of the assistant, but could still be fulfilled by a human worker. If it does not already exist, we could add to each interaction explicitly which user made that interaction, with null values indicating no user, i.e. the system itself.
Thanks a lot for updating the diagram, very good thinking!
from open-assistant.
- When the user replies to the task message (user_reply task) and a new post is inserted we need to link it to its thread and parent post. At this moment the information from the frontend is sufficient only to identify the task. Thus we need to link the task to its post of origin, parent_post_id should do the trick.
Currently I think the main problem is the task generation. When the backend receives the user's reply it adds it nicely as child-post in PromptRepository.store_text_reply()
. What is missing IMO is the original message in the database. When I designed the thread-tree-structure of the post-table my idea was that the source posts of a conversation would also reside in this tree so that the task would refer to them.
from open-assistant.
I guess PostgreSQL is capable of recursive queries to walk over the posts of one thread using parent_id, but for now I'd suggest a much simpler way to sample a conversation:
The posts table has a thread_id
column which is always the id of the root post of the thread. This should allow to fetch at thread-level and for most cases hopefully will not require recursive-queries.
from open-assistant.
@mjagkow I can quickly make some of your suggested changes. What is your state working on this? Are you on the OA discord?
from open-assistant.
Related Issues (20)
- when i click on start new message it doesn'[t click HOT 1
- Open AI doesn't work for me HOT 1
- Create a New Chat HOT 4
- Questions about what's going on with Open-Assistant? please watch. HOT 2
- Chat doesn't open HOT 1
- text to speech HOT 1
- Unable to create new chat HOT 6
- unable to create new chat HOT 2
- Dear ladies and gentlemen, I click on the button "create a chat" but it doesn't work at all. Could you please solve this problem and help me ? Best regards Ehsan Pazooki HOT 1
- Not able to access the chat dashboard HOT 1
- Open assistant registration error
- Dashboard not working in the official website. HOT 2
- /dashboard exit HOT 1
- chat frontend no longer active, fix readme HOT 2
- Can't open dashboard HOT 1
- Sign on malfunction HOT 1
- Not able to get to the dashboard HOT 1
- no puedo acceder a la bandeja de entrenamiento supervisado
- Potential Information Leakage
- https://open-assistant.io/ SSL cert is outdated by 28 days
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from open-assistant.