☑️ Checkpoint Summarization Filter

A new filter for managing context use by summarizing previous parts of

the chat as the conversation continues. Designed for both general

chats and narrative/roleplay use. Work in progress.

Configuration

There are currently 4 settings:

Summarizer Model: The model used to summarize the conversation as the chat continues. This must be a base model.
Large Context Summarizer Model: If large context summarization is turned on, use this model for summarizing huge contexts.
Summarize Large Contexts: If enabled, the filter will attempt to load the entire context into the large summarizer model for creating an initial checkpoint of an existing conversation.
Wiggle Room: This is the amount of 'wiggle room' for estimating a context shift. This number is subtracted from `num_ctx` for the purposes of determining whether or not a context shift has occurred.

In general, you should only need to specify the summarizer model and

enable the filter on the OpenWebUI models that you want it to work on.

Or even enable it globally. The filter works best when used from a new

conversation, but it does have the (currently limited) ability to deal

with existing conversations.

When the filter detects a context shift in the conversation, it will summarize the pre-existing context.
After that, the summary is appended to the system prompt, and old messages before summarization are dropped.
When the filter detects the next context shift, this process is repeated, and a new summarization checkpoint is created.
And so on.

If the filter is used in an existing conversation, it will summarize

on the first time that it detects a context shift in the conversation:

If there are enough messages that the conversation is considered "big," and large context summarization is DISABLED, all but the last 4 messages will be dropped to form the summary.
If the conversation is considered "big," and large context summarization is ENABLED, then the large context model will be loaded to do the summarization, and the ENTIRE CONVERSATION will be given to it.

There are some basic commands the user can use to interact with the

filter in a conversation:

`!nuke`: Deletes all summary checkpoints in the chat, and the filter will attempt to summarize from scratch the next time it detects a context shift.

There are some limitations to be aware of:

If you enable large context summarization, you need to make sure your system is capable of loading and summarizing an entire conversation.
Handling of branching conversations and regenerated responses is currently rather messy. It will kind of work. There are some plans to improve this.
If large context summarization is disabled, pre-existing large conversations will only summarize the previous 4 messages when the first summarization is detected.
The filter only loads the most recent summary, and thus the AI might "forget" much older information.

‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗