Small Steps Towards Effective Prompt Engineering

This article will introduce the concept of prompt engineering, and more specifically, programmatic prompt engineering, and how it relates to building applications using Large Language Models (LLMs).

The topic is vast and this article will only scratch the surface. Our team is just beginning to explore prompt engineering, and my hope is to show that small changes to the LLM prompt can have a big impact on the accuracy and value of the results.

What We Built

For the purpose of expanding our knowledge of AI while building something useful, the Focused Labs special projects team focused on creating a “knowledge hub” for use by visitors to the Focused Labs website. Early iterations of the knowledge hub will be similar to a chatbot, but with a richer natural language interface than is typically seen. We like to think of it as an “intelligent search bar”.

Prompt Engineering

So what do we mean by prompt engineering? In its simplest form, it means anything the end user can do to steer the LLM to the desired results. Here’s a simple example.

What are the Focused Labs core values? 

Not bad, but as much as we personally value “Empathy & Collaboration”, it is not a core value. Next, let’s try

What are the three Focused Labs core values? 

Spot on. Obviously, this is a contrived example, since you need to know there are 3 core values before you ask the question. But it demonstrates how sensitive LLMs can be to the precise text of their prompts.

(Aside: adding three caused the model to format the response as a numbered list. Formatting is another common use case for prompt engineering.)

For the knowledge hub, we don’t want the user to need to know anything about Focused Labs before using the tool. Instead, we want programmatic prompt engineering, or what I like to call “prompt expansion”.

What Makes a Good Prompt?

1. Specificity. Tell the LLM as precisely as possible what you want it to do. Since for our use case, we can’t rely on the user to do this, we need to be clever with our programmatic-based engineering. So far, we have experimented with persona matching, aka “instructor-based” prompting, which we have done programmatically by asking the user to self-identify from a list of possible roles. (e.g. software engineer vs executive vs potential client). The user’s choice is then converted into a prompt enhancement as described in the next section.
   
   
2. Structure. Describe what you want the response to look like. This is known as “example-based prompting”. This can be simple formatting (e.g. list vs paragraph) but can also be quite complex, suggesting or mandating structure and even snippets of verbatim text you want the LLM to respond with.
3. Context. The LLM will produce better results if it remembers what you offered previously about yourself and the domain you are querying. You’ve probably seen this on e-commerce sites or customer service bots that engage you in a short conversation. All of your responses can be fed back to the LLM on subsequent queries.
4. And many others. There are some good examples here targeted to problem-solving and reasoning questions, such as “list the steps needed to remodel a kitchen”, which do not apply directly to our use case but are interesting to think about.

Querying with a Persona

We discovered early on that adding a phrase like “You are an expert software engineer” to queries can improve the results. This is known as instructor-based prompting. It steers the LLM towards a desired perspective when responding to your question.

You can do this yourself when interacting with ChatGPT as shown in this excellent introduction.

In our case, in the UI we asked the user to select 1 of 5 personas and programmatically added a phrase to each query.

For example, when the following question is asked with no persona: 

Q. What services does Focused Labs offer?

A. Focused Labs offers a range of services including agile workflow development, DevOps culture implementation, user-centered design, lean product development, software engineering, extreme programming, automation, performance monitoring, guidance on transitioning away from a top-down workflow, building trust with teams, building highly productive software teams in a traditional IT environment, … etc …

Pretty good content, but it’s one terribly dense run-on sentence. Now let’s ask again, but as a software engineer: 

Persona: Software Engineer

Q. What services does Focused Labs offer?

Expanded Prompt: You are an expert software engineer. What services does Focused Labs offer?