What is difference between system prompt and query prompt wrapper?

[abrahimzaman360]

I'm a bit confused!

[talhasarit]

Hi @abrahimzaman360,

Basically system_prompt is used for assigning a role to the LLM. You can say something like 'You are an expert software engineer with at least 10 years of experience'. And continue with the DO's and DON'T you want in your LLM's response logic/structure.

The job of the query_wrapper_prompt is to wrap the context we fetch from the closest embeddings. We need to use 2 special parameters here: context_str and query_str. For example:

    "Context information is below.\n"
    "---------------------\n"
    "{context_str}\n"
    "---------------------\n"
    "Given the context information and not prior knowledge, "
    "answer the query in the style of a Shakespeare play.\n"
    "Query: {query_str}\n"
    "Answer: "

Hope it helps!

[abrahimzaman360]

Hi @abrahimzaman360,

Basically system_prompt is used for assigning a role to the LLM. You can say something like 'You are an expert software engineer with at least 10 years of experience'. And continue with the DO's and DON'T you want in your LLM's response logic/structure.

The job of the query_wrapper_prompt is to wrap the context we fetch from the closest embeddings. We need to use 2 special parameters here: context_str and query_str. For example:

    "Context information is below.\n"
    "---------------------\n"
    "{context_str}\n"
    "---------------------\n"
    "Given the context information and not prior knowledge, "
    "answer the query in the style of a Shakespeare play.\n"
    "Query: {query_str}\n"
    "Answer: "

Hope it helps!

Thanks

[abrahimzaman360]

@SeeknnDestroy how to improve it because it is taking a lot of time than usual for loading response!

[abrahimzaman360]

I have just one xlsx file, but taking 30-40 seconds to generate

[talhasarit]

can you please share a reproducible code snippets for me to better resolve the issue you are experiencing, also the size of the xlsx file?

[abrahimzaman360]

174 KB

[talhasarit]

@SeeknnDestroy how to improve it because it is taking a lot of time than usual for loading response!

You can use faster llm models like gpt-3.5-turbo, and your chunk size & similarity_top_k will also effect your pipeline execution time overall.

[abrahimzaman360]

Currently, I'm getting issues regarding '\n' everywhere in JSON response! I'm doing the response in the form of JSON string which will be JSON.parsed in TypeScript! I'm getting the correct response as I want in the Normal GPT Model but wanna train on a small kind of specific use case.