AI Models Cannot Read Your Mind
Writers, creative thinkers, we all spend much time and attention in researching our subject and double-checking our facts before publishing our work. For instance, to write my early books on prompt engineering, I spent over three years reading papers, experimenting, making connections, and finally developing a curriculum to teach prompt engineering.
In the beginning, I did the research traditionally: identify relevant and timely papers and books, print them out, read, underline, and make notes in the margins. For my latest book, Prompt Engineering Foundations, I turned to generative AI, testing out word patterns and techniques to create generative AI research assistants. I used two models, ChatGPT and Claude for this task.
Let’s step back for a moment. When people seek an answer from a model, they typically assume that they can just start asking the model questions spontaneously, such as “What foods did people eat during the Gilded Age?” and expect the model to provide accurate and cogent answers. Somehow the model is expected to infer your intentions. The model, however, no matter how advanced or how close to AGI, cannot read your mind. When you give it a general, vague question without context or stating your intentions or task requirements, you’ll get back a neutral, generic answer, which may or may not be helpful.
If spontaneous questioning is not the best way to use generative AI as a research assistant, what is the correct way?
It’s very straightforward and begins with carefully considering the task at hand and what it is you hope to achieve, what “experts” you need to accomplish the task, the level of expertise you bring to conversation, and what data the expert should and should not use. You also should have a plan for fact checking any facts that the model gives you. You need to clearly identify how you want the answers to appear and how specific you want the answers to be.
The machine learning research community has identified word patterns that when included in your prompt, will have various impacts on the output. For instance, to create a specialized AI research assistant that can truly be helpful, relevant word patterns include: Persona, Audience, Reflection, and Fact Check List. The technique, augmented prompting, should also be used to feed the model the information with which you want it to work.
Let’s look at an example. Here is the context: I am a writer who will be writing about a woman who was born after the end of the Civil War and lived her adult life during the Gilded Age in New York City. I don’t have information about what her day-to-day life looked like or what events might have affected her. I am not a historian and I have never visited New York City. Most of my notions about the era were developed through reading the novels of Edith Wharton.
What research expert do I need to help me with research? I need to consult with a historian who specializes in that period and place and is known for checking and double checking his/her facts. The expert also needs to have a deep understanding of the social situations surrounding the lives, limitations, and motivations of New Yorkers during that period.
What output do I want the model to provide? I want short, well-conceived paragraphs with concrete details and references to back up those details.
To set up the assistant I’ll use the following prompt that incorporates the word patterns mentioned above.
PROMPT: Please act as an expert historian who specializes in the social history of life in New York City after the end of the Civil War and through the early 1900s. I am not a historian and I have never been to New York City. Everything I know about this period and place is through reading Edith Wharton novels. I am writing a book about a woman who was born after the Civil War and lived a prosperous life throughout the Gilded Age. I have a few details about her, however, I don’t know anything about what her life might have looked like. You will be my research assistant who answers my questions thoughtfully in short paragraphs that include specific details and authentic references for those details.
——————————————
Pattern Key:
Persona pattern, Audience pattern, Task context, Output format, Reflection pattern
This is the beginning of what promises to be a fruitful conversation with a model.
When you initiate a conversation with a well-designed prompt using patterns, the outputs will be better aligned with your intentions and needs. Models still hallucinate and provide inaccurate answers, however, by requesting references and verifying those references and facts before accepting the responses from the model, you can mitigate this limitation.
Remember also, in conversing with AI models, be clear and specific and define the output that you want. Tell it what your intent is and what you want it to do for you.
The responses will always mirror your prompts. Use respectful words and it will respond likewise. However, if the model tries to take control of the situation by suggesting a next step, resist the temptation. Stay in control of what the model produces.
I’ll be happy to answer any questions you might have. Contact me through meripointbooks.com