Creating an image generation agent

In this article, we will explore how to create an image generation agent. This Agent will automatically generate an image after every message, giving you an immersive chatting experience.

The Agent will use your conversation context to generate an image!

Here's is how the Agent looks in action!

The idea is to have the LLM add a stable diffusion prompt after each message. We add a prompt in the character card telling the LLM to append a short description of the scene in <stable_diffusion_prompt></stable_diffusion_prompt> tags.

First, we create the Agent. This agent is very similar to our Roleplay Agent:

Here we use a simple grammar to structure the output to end with <stable_diffusion_prompt></stable_diffusion_prompt> tags.

The next step is to create or copy your own character. You need to do two things here. The first is to add a custom instruction in the Scenario section, telling the LLM to put scene description keywords in the stable diffusion prompt tags. You can get creative here: try telling the LLM to put character descriptions to focus on generating images of your character instead of scenes!

Then attach the Agent to your character in the Advanced tab just like before.

The last thing you need to do is to enable image generation in your Inference Settings. For more information, you can read here: https://www.layla-network.ai/post/how-to-enable-image-generation-in-layla

If you have a phone with a Snapdragon CPU, I strongly recommend you generate images with the NPU: https://www.layla-network.ai/post/layla-supports-generating-images-locally-using-the-npu. This means your images will take only a few seconds to generate after each message, so your flow of conversation is not interrupted!

Here is the Agent for you to import if you wish:

generate-image-agent.json

Download JSON • 655B