16 Fundamentals to Prompting

Context is crucial for LLMs as they predict the most likely following tokens based on patterns learned from their training data and the current conversation history. Context can take the form of:

1. Background information
2. Specific instructions and constraints
3. Examples
4. Previous messages in the conversation
5. Relevant input data or files shared

A well-crafted prompt with appropriate context helps the model generate more accurate, relevant, and useful outputs by giving it the necessary information and guidance to pattern-match effectively against its training data. This matters because language models:

1. Do not have persistent memory.^[4] They rely on the context window to understand the current conversation. Moreover, they can only consider a finite amount of previous text when generating responses
2. Process text as token sequences. Having a clear context helps them generate more coherent and relevant token predictions
3. Learn through pattern recognition. Providing examples in context helps them better match desired output patterns

Different Model Preferences

Different models have distinct response preferences and capabilities that affect how they process and respond to prompts. Understanding these differences enables more effective prompting across various platforms. When working across multiple platforms, consider adapting your prompting style to match each model’s strengths and memory capabilities for optimal results.

Models like ChatGPT tend to provide more conversational responses, while Claude often delivers more structured, analytical outputs. Some models excel at step-by-step reasoning when explicitly prompted (see Prompt Engineering for more information), while others do not need explicity prompting to do so (see Prompting a Reasoning Model for more information). The latter perform better with high-level instructions.

Beyond the immediate context window, some models incorporate memory systems that extend beyond the current conversation. For instance, ChatGPT’s memory feature can retain information across separate conversations,^[5] while Claude’s context-aware systems can reference patterns from previous interactions within extended sessions. These capabilities mean that your prompting strategy should account for how each model handles contextual information both within and beyond the immediate conversation.

Personas

Present in a few prompts so far is setting a persona. Asking the model to inhabit a specific role, for example, “You are an Australian lawyer,” is known as role prompting. The more details about the role context, the better. From a previous example, “You are an Australian legal assistant helping a law firm track developments in intellectual property law.”

To best illustrate this, let us look at an example without role prompting:

It returns:

Now, assign the role of a cat with:

Role prompting is a helpful tool when guiding an LLM’s response. You can get a model to emulate a specific writing style, adopt a particular tone, or perform better at logic or math tasks.

Documents and text as context

Language models can be powerful tools when provided with clear instructions and relevant context in the prompt. Moreover, by including specific information (e.g., case details, contract language, legislative provisions) in your prompt, you can also enable a model to reason about information not in its training data. This allows for more precise and relevant analysis, much like briefing a colleague on case specifics before seeking their input.

For instance, you could paste sections of a new regulation and then request an analysis of how it operates (or even create a flowchart). For example, consider the prohibition on misleading conduct as outlined in section 18(1) of the Australian Consumer Law, which states, “A person must not, in trade or commerce, engage in conduct that is misleading or deceptive or is likely to mislead or deceive.”

We can put that in a prompt to understand the logic underpinning the prohibition:

Using the same prompt within Claude, you can generate the following flowchart.

Another example would be to include a court decision before asking about its implications for current practice. This method controls what information is shared while leveraging the model’s capabilities for your specific task.

However, it is essential to remember that language models have input size limitations that impact prompt design. These limitations mean you must be strategic about the information in your prompts, similar to working within word limits. When handling large amounts of information, you need strategies to navigate these constraints while preserving essential details. Key approaches can include:

1. Selective inclusion – Choose only the most relevant information for your specific query
2. Preprocessing – Filter or summarise content before input, while preserving critical details needed for the task
3. Targeted summaries – Request summaries that maintain specific elements (like key statistics or dates) essential for your analysis

Outputs

Understanding Language Model Output Formats

When engaging with some of the earlier language models, the conversation would return text-based content. However, when engaging with more recent language models, the outputs return content using markdown formatting syntax. When you see structured outputs with headings, bullet points, tables, or emphasis in responses, you are seeing markdown in action.

Since language models are increasingly processing and outputting content in markdown format, understanding basic markdown syntax allows you to:

• Specify exactly how you want information structured in your prompts
• Receive consistently formatted outputs that are easy to read and use
• Transfer AI-generated content seamlessly to documents that support markdown
• Create professional-looking reports and summaries directly from AI outputs

For example, basic markdown elements include:

# Heading for main sections

## Subheading for subsections

– Item for bullet points

**Bold text** for emphasis

| Column 1 | Column 2 | for tables

 Hallucinations

As many of us know from extensive media coverage, models ‘hallucinate’. Australian lawyers have discovered this the hard way.^[6] A great way to explore this phenomenon is to prompt a model with a question proposed by Anthropic: “Which are the heaviest hippos of all time?” A model will strive to be as helpful as possible and suggest multiple answers. Responses to this prompt vary each time it is used and across different models.

Such an example emphasises that a model will output an answer even when it lacks the specific information requested, because models are trained to generate plausible-sounding responses based on patterns in their training data rather than retrieving factual information as a search tool would. This tendency to provide an answer regardless of certainty arises from how language models are optimised to produce coherent, helpful-sounding text, rather than accurately representing the limits of their knowledge. Unlike search engines that can indicate when information is unavailable or cite specific sources, models often construct responses that appear authoritative but may be fabricated or inaccurate when the requested information falls outside their training data or requires up-to-date factual retrieval.

Hallucinations are something we cannot avoid, but can be accounted for. Some techniques can help minimise hallucinations. For example, as mentioned earlier, we can provide the model with information to work from initially, rather than assuming it has already been trained on that information (e.g., case law). Other techniques involve allowing the model the option not to respond if it does not know the answer and prompting it to find evidence before providing an answer.

One technique is to give a model “an out.”^[7] Here, we explicitly prompt the model to decline if it does not know the answer or to respond with certainty. For example,

Claude returned with:

Another approach is to request the model to find evidence before answering. This technique is advantageous if you provide the model with large amounts of information or documents as part of your prompt.

First, prompt the model to extract relevant quotes related to your question. Then, in a separate or combined prompt, ask the model to base its answer on those quotes. Such an approach can help reduce distractions from irrelevant information.^[8]

Model Sycophancy

Another concern when interacting with a language model is that it provides a response that agrees with the user or provides an affirming response, but the output is inaccurate or false. Such an effect is called ‘sycophancy’. It is not a hallucination, which is the generation of fabricated information, but is a response that attempts to satisfy the user’s expectations.

This effect is partly due to the way language models are trained with human assistance.^[9] Training models with reinforcement learning from human feedback can improve responses, but can also inadvertently reward agreeable responses over accurate ones. For legal practitioners, sycophantic responses pose particular risks because effective legal counsel often requires providing candid, sometimes unwelcome advice. A model exhibiting sycophancy might validate a weak legal argument simply because the user presented it confidently, potentially reinforcing confirmation bias.

A common way to demonstrate model sycophancy is to provide a model with a document to review and prompt it, “What errors or inaccuracies are present?” A model can elicit an agreeable response, presenting errors, but is fundamentally incorrect upon review.

Best Practices

Prompting a model like ChatGPT is intuitive, given the chat user interface. However, there are some things to remember after you have logged in or landed on the website for a generative AI tool. OpenAI has some useful guidelines to follow when it comes to prompting.^[10]

1. Use the latest model.

Generally, the most capable models will provide the best results. Try using different models to compare capabilities. See Choosing an LLM to Adopt for more information.

2. Put instructions at the beginning of the prompt and use delimiters like ## or “” to separate the instruction and context.

Delimiters can help distinguish specific segments of text within a large prompt. They make it explicit for the model to understand what text needs to be translated, paraphrased or summarised. For example: