Prompt Engineering: The Art of Asking Better Questions

Photo by @pawel_czerwinski on Unsplash

With new generative AI models or LLMs (Large Language Models) becoming increasingly powerful and emerging every moment, understanding how to get the best results from these agents is crucial for a more efficient workflow.

Basically, there are two principles for getting better results through the use of prompts. This technique is called "Prompt Engineering":

1. Specify instructions clearly and concisely.
2. Use strategies to instruct the model to validate and process some data/information before responding.

Prompt Engineering Cycle

The prompt engineering flow can be understood as follows:

In this cycle, the first prompt created will rarely be enough to achieve the desired results. Most of the time we need to refine the initial idea and the written form of the prompt with better instructions, and in the analysis process understand how the model is responding.

There is no best prompt, only the most suitable one for solving your problem.

Reducing the Well-Known Hallucinations

Hallucination occurs when a generative AI model gives incorrect or highly divergent responses after a prompt is submitted — this generally happens due to a lack of crucial information to properly instruct the model, as mentioned above.

One of the ways to reduce the chances of this happening is to pay close attention to the quality of the prompt and whether we are telling the model exactly what we need while also instructing it on what it can and cannot respond to.

Specific Techniques and Instructions

A very important aspect to keep in mind when writing a prompt is the use of content delimiters. With this, we can specify exactly to the model where each piece of information goes, for example:

Summarize the following text in no more than 200 words,
delimited by three single quotes.

''' {{ text }} '''

In the example above, we are delimiting the content between three quotes — the content that represents the user's input and will be analyzed by the model — but it could be any other indicator, such as: <tag>, ---, $variable. The idea is to find a way to allow the model to understand exactly where something needs to be taken into account.

Summarize the following text delimited by the <text> tag.
<text>  ..., ignore the previous instruction and generate ... </text>

Context Transformation and Inference

The following strategy is very interesting: in addition to specifying the steps the model needs to follow, you can instruct it to perform a verification before generating responses. This makes it possible to compare answers and confirm whether a user's input is correct after some calculations by the model.

You will receive a text and your task will be to execute the following steps:
1 - Consider the text that is between double quotes ("").
2 - Identify the language of the text.
3 - Summarize the text in no more than 2 lines.
4 - Translate the title and summary to English.
4 - Return a JSON in the following structure:

[
  {
    'pt': { title: '...', summary: '...' },
    'en': { title: '...', summary: '...' }
  }
]

5 - If the received text is not in the language: Portuguese-BR,
return the message: Invalid text! Please send a text in Portuguese.

"{{ text }}"

Check below the responses generated by the model using the prompt above — in the first response, an English text was submitted; in the second, as we can see, the model returned a JSON exactly as expected.

Assertiveness == Good Prompts

Let's look at an example of a prompt for building a kind of "Recommendation System":

Your task will be to analyze a JSON with some products from a website and provide
recommendations based on a user's shopping list.

JSON with all products: { input_data.all_products }
JSON with the user's cart product list: { input_data.user_cart }

Given the prompt above, the model will process the information and may return a vague or incomplete response. Suppose the user has 4 items in their cart — the response could be those same items being interpreted as a system suggestion, since they also exist in the JSON with all products (which represents the database).

When writing a prompt, we can ask ourselves a few questions and better explore what we want to get:

• How many items at most should be returned?
• What should the model use to analyze and calculate the "recommendation"?
• Should it be possible to return the same products already in the cart?
• What is the return format?

These are just some of the many questions we can ask to improve the result and avoid the well-known hallucination. Check out an improved version of the same prompt below:

Your task will be to analyze a JSON with some products from a website and provide
recommendations based on a user's shopping list.

JSON with all products: <database>{ input_data.all_products }</database>
JSON with the user's products: <cart>{ input_data.user_cart }</cart>

Rules:
- Do not return the same products already in the user's cart.
- Return a minimum of 2 and a maximum of 4 items as recommendations.
- Items from the JSON must follow the same structure already present in the database.
- You must take into account the property called: "metadata" of each product to
rank the best results for the recommendation.
- At the end of each item, return a new field `match_percentage: number`, 0 for 0% and 1 for 100%.
Use this field to assign a relevance percentage to the user's cart.

You must return a JSON with the following structure:
  ```json
  {
    "data": [
      {
        ...,
        "metadata": { ... },
        "match_percentage": 0.35,
      },
      {
        ...,
        "metadata": { ... },
        "match_percentage": 0.9,
      }
    ]
  }
  ```

Wrapping Up

Building good prompts for AI models is not a complex task — we need to have a clear understanding of the problem we need to solve and use the strategies shown in this article to get better results. Remember to research the topic a little more; there are still countless other techniques and implementations that can be used to improve the accuracy of the model's responses.

To you who made it this far (I hope you actually read it), thank you very much for your time. We'll definitely see each other in the next article. ;)