Enhance PromptRunner interface

[stefanks-px]

Hello,

I have a simple use case where I want to use the PromptRunner interface to

• Add a system message via template with substitutions
• Add a user message
• Convert the response to an object

I don't see how to do this with the Rendering or Creating builder. We need a strict separation of the system and user message to make use of prompt-caching. My workaround right now:

        String systemPrompt = templateRenderer.renderLoadedTemplate("ssystem-prompt",
                Map.of("ABC", "ABC")
        );
        List<Message> messages = Stream.concat(
                Stream.of(new SystemMessage(systemPrompt)),
                input.conversation().getMessages().stream()
        ).toList();

        return context.ai()
                .withLlm(llmOptions)
                .creating(ABC.class)
                .fromMessages(messages);

Is there an easier way to do this or is this still missing?

[igordayen]

@stefanks-px - you can cleanly separate the system and user messages, if that is what you are looking for:

PromptRunner runner = ai.withLlm(options)
.withSystemPrompt(systemPrompt)

then

runner.createObject(new UserMessage(....), ABC.class).

Thanks

[stefanks-px]

@igordayen
Yes this already works. But when the System prompt should come from a template, I have to manually render the template. My suggestion is just about convenience to have all integrated into the PromptRunner interface, if possible. For example

PromptRunner runner = ai.withLlm(options)
.withSystemPromptTemplate("template", MAP_WITH_SUBSTITUTIONS)
runner.createObject(new UserMessage(....), ABC.class).

[igordayen]

@stefanks-px - thank you for you suggestions, will analyze post 1.0.0 GA release. thanks again for helping with product enhancements.