This update brings several important improvements:
Repetition Control! This update adds fine-grained control to stop your characters from repeating the same phrases over and over again. Previously this is partly solved by enabling the Mirostat sampler. While Mirostat helps, this update provides a direct sampling method that targets fixing repetition.
You can turn it on by setting the "multiplier" to anything greater than 1 in the Advanced settings:
PDF Digest
Chat with your PDF in this update! Send your PDFs to Layla, and she can give you an informed response on the contents of the PDF by picking out relevant information within.
Phi-3 support! Microsoft has recently published both Llama3 and Phi-3 open source models, two best in class open source models for their weight. Both are now supported in Layla. You can download the official weights from Hugging Face and load them into Layla via the RMI app (Raw Model Instructions).
Download the Phi3 model here: https://huggingface.co/l3utterfly/phi-3-mini-128k-instruct-gguf
Executorch support!
Executorch is an experimental LLM inference engine that utilises the GPU and Neural engines of the latest smartphone processors. It runs Llama3 models at 5-10x the speed normally in Layla.
Enable Executorch by adding the experimental app in Layla:
Full Changelog
New features:
Phi-3 support!
PDF Digest app: chat with your PDF documents!
Executorch Inference Engine - highly experimental app for very high-end phones (improves Llama3 response times by 5x on high end phones such as S23+ Ultra)
Improvements:
added ability to create/save custom prompts in RMI app
prompts in RMI app now represents the raw prompts given to the LLM, so you have full control over the prompt format
LTM now injects the latest memory into new chats with a character
implementation of DRY sampler: contrasted with MiroStat, which uses a completely different sampling methodology in an effort to reduce repetitions, this sampler targets reducing repetition. Don't use with MiroStat. (Configure DRY in Advanced settings)
added new section in settings page to deal with model repetitions, allowing you to control parameters such as repetition penalty, range, DRY, etc.
Bug fixes:
unread messages are not deleted after opening them
fixed bug in backup feature
fixed bug where the delete button in chat history is hard to click
fixed bug where chat name was not updated after edit
fixed bug "failed to eval" when carrying out long conversations