chrisimmel / calliope Goto Github PK

1. Define a data format to hold a story.
2. Create a "download story" feature (probably in Calliope Admin).
3. Create an "upload story" feature (also probably in Calliope Admin).

Create a story strategy that can be given a story in the form of a URL to a plain text file available online such as at Project Gutenberg (for example here). We will need:

• An algorithm for automatically dividing the text into frame-sized chunks. We would likely want to preprocess and cache this chunked text.
• When asked for a new frame, we would then pull a new frame of text from that cache (chunking some more if necessary), illustrate it, store it, and serve it.
• This ideally depends on: #7.

Presently, thumbnails for Thoth are computed for each story each time the Thoth homepage is loaded. This is expensive and unnecessary. Add a thumbnail_image foreign key to the Story model so this can be persisted and loaded quickly.

@mikalhart reports this to have happened when requesting an image for a rectangular Sparrow screen. If I recall correctly, the image had black bars introduced to pad a dimension to a multiple of 64, but also was larger than requested so was clipped by the device.

It must be possible to record a story in the database and play it back separately.

• start_frame_number and num_frames parameters will be added to the /frames/ request parameters. This will enable a client to ask for a specific range of frames from a story. (Maybe story_id as well?)

• To preserve existing behavior, when a client asks for N frames and no start frame is given, we will by default serve the next N frames of the story in progress, having kept track of the last frame previously served.

• It will be possible for a strategy to sometimes compute frames in advance (helpful if too much text comes back from the LLM, for instance).

• This suggests that the playback aspect should happen at a higher level in the stack. If there are already existing frames at the right point in the sequence, the story player just grabs and serves them. If not, it calls to the strategy to generate one or more new frames. If some other sparrow's request is already in the process of generating the requested frames, this request can just wait for the other to complete, and serve the result when it becomes available.

• This will aid in creating the desired behavior of a flock of sparrows seeing the same generated story together. The first sparrow to ask for a new frame N will trigger the generation and storage of the frame. Subsequent sparrows asking for frame N of the story will just get that same frame served instead from the database.

The current Calliope Admin is almost literally what comes out of the box when pointing the Piccolo Admin package at Callipe's database models. It's a great start, but we will want quite a lot more.

Some early thoughts (likely to be split off into their own issues):

• Custom validators to properly validate Pydantic/JSON fields.
• Custom forms to create a story from input text (or from a URL that refers to input text).
• See images on stories and story frames, more like Thoth.
• See/edit frames connected to a story (instead of each being viewable only as separate, unrelated tables).
• Custom actions to illustrate a story or frame.
• Add a merge story function.
• Copy/paste of frames.
• Apply correction and formatting rules to existing text.

• Request a frame only on demand (user tap or swipe).
• Enable scrolling back through frames.

Once we can play back precomputed stories, it will become interesting to be able to schedule their playback to begin at a designated date and time. We'd also like to be able to say they should repeat until further notice (or until the calendar dictates something else happen).

For example: "Repeat the following 10 frames in a loop the full day of December 17th."

We need to limit the text length generated for a single frame because we have limited display size. This is most important for hardware Sparrows, but is true even for Clio, which is sometimes run on mobile devices with small screens.

It is challenging to control the text length coming from the language models. If we pass a max_tokens value that is too small, this causes misbehavior from the models, including premature exit (resulting in poorer quality) and text truncation. Some other options:

1. Take only the head of the text, discarding the rest. (Partition on sentence or line boundaries, then take the maximum number of sentences that fit within the requested maximum text length. If even the first sentence is too long, truncate it on a word boundary. If even the first word is too long, just truncate it.
2. Same as above, but instead of discarding the unused portion, divide it into additional frames, and store them for later use.

2 seems preferable, but requires some enhancement to the way stories are generated and served.