Comparing AI Prompt Versions with Prompt Mixer

January 29, 2024

Artificial intelligence (AI) systems like large language models are highly sensitive to how prompts are phrased. Even small tweaks to a prompt can produce very different results. When reading new AI research papers, it's often useful to test out the prompts yourself on your own models. This allows you to reproduce the paper's results and compare differences due to your model architecture, training data, etc.

In this article, we'll walk through an example of testing out variants of a prompt from a published paper in Prompt Mixer. Prompt Mixer provides a user-friendly prompt IDE that makes iterating on prompts simple.

Prerequisites

  • A Prompt Mixer app
  • The prompts you wish to test from a paper or other source

Step-by-Step Instructions

  1. Create a new prompt chain in Prompt Mixer for your first prompt version. Click "+ New prompt chain" and give it a descriptive name.
  2. Add any connectors to your prompt chain, whether from the public API or a custom connector to your own model hosted on HuggingFace or elsewhere.
  3. Paste your first prompt version into the chain prompt field and click "Run chain". View the results.
  4. Click "Fork prompt" in the right to create a new your next prompt version. Paste the new version into the prompt field.
  5. Run the forked prompt and compare results side-by-side with the original prompt. The visualization makes differences easy to spot.
  6. Repeat steps 4-5 to test additional prompt variants as desired.

For example, let's try out some prompts from the paper Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration. 

The prompts are available in this GitHub repo.

Prompt v1

Question:
There are 5 houses in a row, numbered 1 on the left to 5 on the right. There is one person living in each house. The people in these houses have different characteristics:
- Everyone likes a different kind of book: one is a science fiction book fanatic, one is a romance book lover, one is a history book buff, one is a fantasy book enthusiast, and one is a mystery book reader
- Each person plays a different musical instrument: one is a cellist, one is a percussionist, one is a saxophonist, one is a violinist, and one is a guitarist

Clue(s):
1. The romance book lover lives somewhere to the left of the fantasy book enthusiast.
2. There is one house between where the history book buff lives and where the guitarist lives.
3. There is one house between where the percussionist lives and where the mystery book reader lives.
4. The percussionist lives directly left of the romance book lover.
5. The mystery book reader lives in the fourth house.
6. The saxophonist and the violinist live directly next to each other.
7. The saxophonist lives in the fourth house.

What is the number of the house where the violinist lives?
 choice: 2
 choice: 5
 choice: 3
 choice: 1
 choice: 4

Solve the task step by step. Your output should be of the following format:

Steps:
Your steps here.

Answer:
The house number here.


Gpt-4-1106-preview output

Steps:
The mystery book reader lives in the fourth house (Clue 5).
The saxophonist lives in the fourth house (Clue 7). This means the mystery book reader is also the saxophonist.

Since the romance book lover lives somewhere to the left of the fantasy book enthusiast (Clue 1), the fantasy book enthusiast cannot live in house 1, and because the mystery book reader (who is also the saxophonist) lives in house 4, the fantasy enthusiast cannot live in house 3 either. So, the fantasy enthusiast must live in house 5 (because they must be to the right of the romance lover, and house 2 would violate Clue 1).

With the guitar player having one empty house between them and the history book buff (Clue 2), and the mystery book reader/saxophonist already in house 4, the guitarist can't be in house 5 or 3. House 2 is the only option for the guitarist, and therefore the history book buff must be in house number 1.

Since there is one house between where the percussionist lives and where the mystery book reader (who is also the saxophonist) lives (Clue 3), the percussionist cannot live in houses 3, 4, or 5. As the history book buff is in house 1, the percussionist must live in house 2.

According to Clue 4, the percussionist lives directly left of the romance book lover. Since the percussionist is in house 2, the romance book lover must be in house 3.

With the saxophonist and the violinist living directly next to each other (Clue 6), and the saxophonist in house 4, the violinist must live in house 3 or house 5. Since the romance book lover is already established in house 3, the violinist must live in house 5.

Answer:
House number 5.

Llama2 output

Steps:
From clue #5 and #7, we know that the mystery book reader lives in the fourth house.
From clue #6, we know that the saxophonist and the violinist live directly next to each other.
Since the saxophonist and the violinist have different instruments and live next to each other, they must occupy houses 3 and 4 respectively, based on their given order in the clue(s). Therefore, the violinist lives in house number 3.

Answer:
3

Gpt-3.5-turbo-0613 output

Steps:
From clue 7, we know that the saxophonist lives in the fourth house.
From clue 6, we know that the violinist lives next to the saxophonist. Since the saxophonist is in the fourth house, the violinist must be in either the third or fifth house.
From clue 5, we know that the mystery book reader lives in the fourth house. Therefore, the third house must be where the violinist lives since they live next to each other.
The answer is the number 3.

Answer:
The house number where the violinist lives is 3.

Mistral output

Steps:
From clue #5, we know that the mystery book reader lives in house 4.
From clue #6, we know that the saxophonist and the violinist live directly next to each other.
Since they cannot live in houses 1, 2, or 3 (because those houses have been assigned to other character types), they must live in house 5.
Now, let's find the location of the violinist by using the given clues.

From clue #4, we know that the percussionist lives directly left of the romance book lover.
Since the percussionist is not living in house 1 or 2 (because those houses have been assigned to other character types), and there's only one house between them (according to clue #3), the romance book lover must live in house 3.
Therefore, the percussionist lives in house 2. But then, according to clue #4, we also know that the percussionist lives directly left of the romance book lover. This creates a contradiction!

However, if we take another look at clue #6, we see that the saxophonist and the violinist live directly next to each other. Since they cannot live in houses 1 or 4 (because those houses have been assigned to other character types), and house 3 is already taken by the romance book lover, they must live in house 5.

So now we know that the cellist lives in house 1, the science fiction book fanatic lives in house 2, the history book buff lives in house 3, and the fantasy book enthusiast lives in house 4.

From clue #6, we also know that the saxophonist and the violinist live directly next to each other. Since they cannot live in houses 1, 2, or 3 (because those houses have been assigned to other character types), they must live in house 5.

Therefore, the violinist lives in house 5.

Answer:
5.

Prompt v2

When faced with a task, begin by identifying the participants who will contribute to solving the task. Then, initiate a multi-round collaboration process until a final solution is reached. The participants will give critical comments and detailed suggestions whenever necessary.

Here are some examples:

Example Task 1:
Use numbers and basic arithmetic operations (+ - * /) to obtain 24. You need to use all numbers, and each number can only be used once.Input: 6 12 1 1

Participants:
AI Assistant (you); Math Expert

Start collaboration!

Math Expert:

Let's analyze the task in detail. You need to make sure that you meet the requirement, that you need to use exactly the four numbers (6 12 1 1) to construct 24. To reach 24, you can think of the common divisors of 24 such as 4, 6, 8, 3 and try to construct these first. Also you need to think of potential additions that can reach 24, such as 12 + 12.AI Assistant (you): Thanks for the hints! Here's one initial solution: (12 / (1 + 1)) * 6 = 24Math Expert: Let's check the answer step by step. (1+1) = 2, (12 / 2) = 6, 6 * 6 = 36 which is not 24! The answer is not correct. Can you fix this by considering other combinations? Please do not make similar mistakes.AI Assistant (you): Thanks for pointing out the mistake. Here is a revised solution considering 24 can also be reached by 3 * 8: (6 + 1 + 1) * (12 / 4) = 24.

Math Expert:
Let's first check if the calculation is correct. (6 + 1 + 1) = 8, 12 / 4 = 3, 8 * 3 = 24. The calculation is correct, but you used 6 1 1 12 4 which is not the same as the input 6 12 1 1. Can you avoid using a number that is not part of the input?AI Assistant (you): You are right, here is a revised solution considering 24 can be reached by 12 + 12 and without using any additional numbers: 6 * (1 - 1) + 12 = 24.Math Expert: Let's check the answer again. 1 - 1 = 0, 6 * 0 = 0, 0 + 12 = 12. I believe you are very close, here is a hint: try to change the "1 - 1" to "1 + 1".AI Assistant (you): Sure, here is the corrected answer:  6 * (1+1) + 12 = 24Math Expert: Let's verify the solution. 1 + 1 = 2, 6 * 2 = 12, 12 + 12 = 12. You used 1 1 6 12 which is identical to the input 6 12 1 1. Everything looks good!

Finish collaboration!

Final answer:

6 * (1 + 1) + 12 = 24

Example Task 2:
Write a poem that meets the following requirements: (1) the poem has seven lines and the first letters of each line forms the word "CHATGPT"; (2) the poem is about explaining what is a quantum computer. (3) the poem needs to be easy to understand by a ten years old kid.

Participants:

AI Assistant (you); Poet; Computer Scientist; Ten year old child

Poet:

Make sure that you write the poem with seven lines, and the first letters of the lines should be C, H, A, T, G, P, T.Computer Scientist: A quantum computer is an advanced computing device that uses the principles of quantum mechanics to process and store information. Unlike classical computers that use bits to represent information as 0s and 1s, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously, due to a quantum phenomenon called superposition. You can consider using these information for the poem.Ten year old child: I hope the poem to be fun and easy to understanding. I don't want to see a lot of jargons or complicated concepts.AI Assistant (you): Thanks for the guidance! Here's my initial attempt at the poem:Computational wonder of our age,Harnessing the quantum world's strange ways,Atoms dance, entwined in dual state,Tapping secrets hidden in their haze.

Grand power to solve, simulate,Profound problems that perplex the wise,Transforming our future, we await.

Poet:
Let's verify if the poem meets the requirements. The first letters are CHATGPT which is correct! And the poem rhymes well. Good job!Computer Scientist: Everything looks good to me!Ten year old child: I don't know what does perplex mean. Can you make the use of words easier to understand?AI Assistant (you): Sure, let me revise the poem by using more common words. Check out the revised version:Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.

Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.

Poet:

Let's check again if the poem meets the requirements. The first letters are C H A T G P T. And now the poem is more accessible to children. Everything looks good to me.Computer Scientist: Looking good!Ten year old child: I like this version a lot!

Finish collaboration!

Final answer:

Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.

Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.

Now, identify the participants and collaboratively solve the following task step by step. Remember to provide the final solution with the following format "Final answer: The house number here.".

Task:

Question:
There are 5 houses in a row, numbered 1 on the left to 5 on the right. There is one person living in each house. The people in these houses have different characteristics: - Everyone likes a different kind of book: one is a science fiction book fanatic, one is a romance book lover, one is a history book buff, one is a fantasy book enthusiast, and one is a mystery book reader - Each person plays a different musical instrument: one is a cellist, one is a percussionist, one is a saxophonist, one is a violinist, and one is a guitarist

Clue(s):
1. The romance book lover lives somewhere to the left of the fantasy book enthusiast.
2. There is one house between where the history book buff lives and where the guitarist lives.
3. There is one house between where the percussionist lives and where the mystery book reader lives.
4. The percussionist lives directly left of the romance book lover.
5. The mystery book reader lives in the fourth house.
6. The saxophonist and the violinist live directly next to each other.
7. The saxophonist lives in the fourth house.

What is the number of the house where the violinist lives?  choice: 2  choice: 5  choice: 3  choice: 1  choice: 4

Method:
spp_fixed_personaWhen faced with a task, begin by identifying the participants who will contribute to solving the task. Note that the participants can only be either AI Assistant (you) or Expert. Then, initiate a multi-round collaboration process until a final conclusion is reached.  The Expert will give critical comments and detailed suggestions whenever necessary.   Here are some examples:

Example Task 1:

Use numbers and basic arithmetic operations (+ - * /) to obtain 24. You need to use all numbers, and each number can only be used once.  Input: 6 12 1 1   Participants: AI Assistant (you); Expert   Start collaboration!   Expert: Let's analyze the task in detail. You need to make sure that you meet the requirement, that you need to use exactly the four numbers (6 12 1 1) to construct 24. To reach 24, you can think of the common divisors of 24 such as 4, 6, 8, 3 and try to construct these first. Also you need to think of potential additions that can reach 24, such as 12 + 12.  AI Assistant (you): Thanks for the hints! Here's one initial solution: (12 / (1 + 1)) * 6 = 24  Expert: Let's check the answer step by step. (1+1) = 2, (12 / 2) = 6, 6 * 6 = 36 which is not 24! The answer is not correct. Can you fix this by considering other combinations? Please do not make similar mistakes.  AI Assistant (you): Thanks for pointing out the mistake. Here is a revised solution considering 24 can also be reached by 3 * 8: (6 + 1 + 1) * (12 / 4) = 24.  Expert: Let's first check if the calculation is correct. (6 + 1 + 1) = 8, 12 / 4 = 3, 8 * 3 = 24. The calculation is correct, but you used 6 1 1 12 4 which is not the same as the input 6 12 1 1. Can you avoid using a number that is not part of the input?  AI Assistant (you): You are right, here is a revised solution considering 24 can be reached by 12 + 12 and without using any additional numbers: 6 * (1 - 1) + 12 = 24.  Expert: Let's check the answer again. 1 - 1 = 0, 6 * 0 = 0, 0 + 12 = 12. I believe you are very close, here is a hint: try to change the “1 - 1” to “1 + 1”.  AI Assistant (you): Sure, here is the corrected answer:  6 * (1+1) + 12 = 24  Expert: Let's verify the solution. 1 + 1 = 2, 6 * 2 = 12, 12 + 12 = 12. You used 1 1 6 12 which is identical to the input 6 12 1 1. Everything looks good!   Finish collaboration!   Final answer: 6 * (1 + 1) + 12 = 24  

Example Task 2:
Write a poem that meets the following requirements: (1) the poem has seven lines and the first letters of each line forms the word "CHATGPT"; (2) the poem is about explaining what is a quantum computer. (3) the poem needs to be easy to understand by a ten years old kid.

Participants:

AI Assistant (you); Expert

Expert:
Make sure that you write the poem with seven lines, and the first letters of the lines should be C, H, A, T, G, P, T. A quantum computer is an advanced computing device that uses the principles of quantum mechanics to process and store information. Unlike classical computers that use bits to represent information as 0s and 1s, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously, due to a quantum phenomenon called superposition. You can consider using these information for the poem. I hope the poem to be fun and easy to understanding. I don't want to see a lot of jargons or complicated concepts.AI Assistant (you): Thanks for the guidance! Here's my initial attempt at the poem:Computational wonder of our age,Harnessing the quantum world's strange ways,Atoms dance, entwined in dual state,Tapping secrets hidden in their haze.

Grand power to solve, simulate,Profound problems that perplex the wise,Transforming our future, we await.

Expert:
Let's verify if the poem meets the requirements. The first letters are CHATGPT which is correct! And the poem rhymes well. Good job! I don't know what does perplex mean. Can you make the use of words easier to understand?AI Assistant (you): Sure, let me revise the poem by using more common words. Check out the revised version:Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.
Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.

Expert:
Let's check again if the poem meets the requirements. The first letters are C H A T G P T. And now the poem is more accessible to children. Everything looks good to me. I like this version a lot!
Finish collaboration!

Final answer:
Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.
Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.
Now, identify the participants and collaboratively solve the following task step by step. Note that the participants can only be either AI Assistant (you) or Expert. Remember to provide the final solution with the following format "Final answer: The house number here.".

Task:

Question:
There are 5 houses in a row, numbered 1 on the left to 5 on the right. There is one person living in each house. The people in these houses have different characteristics: - Everyone likes a different kind of book: one is a science fiction book fanatic, one is a romance book lover, one is a history book buff, one is a fantasy book enthusiast, and one is a mystery book reader - Each person plays a different musical instrument: one is a cellist, one is a percussionist, one is a saxophonist, one is a violinist, and one is a guitarist

Clue(s):
1. The romance book lover lives somewhere to the left of the fantasy book enthusiast.
2. There is one house between where the history book buff lives and where the guitarist lives.
3. There is one house between where the percussionist lives and where the mystery book reader lives.
4. The percussionist lives directly left of the romance book lover.
5. The mystery book reader lives in the fourth house.
6. The saxophonist and the violinist live directly next to each other.
7. The saxophonist lives in the fourth house.

What is the number of the house where the violinist lives?  
choice: 2  choice: 5  choice: 3  choice: 1  choice: 4

Gpt-4-1106-preview output

Participants:
AI Assistant (me); Detective; Logic Puzzle Enthusiast

Detective:
Let's begin by examining the clues and making inferences based on the given information. We must ensure that each house is assigned a unique book preference, musical instrument, and that they all align with the provided clues.

Logic Puzzle Enthusiast:
It's a logic puzzle similar to the Einstein's riddle. We need to take careful note of the positional clues and systematically eliminate impossibilities.

AI Assistant (me):
I'll start by applying the clues to the information we have. Let's analyze each clue:
The romance book lover lives somewhere to the left of the fantasy book enthusiast. This means that the fantasy enthusiast cannot live in house 1, and the romance lover cannot live in house 5.
There is one house between where the history book buff lives and where the guitarist lives. They can't be in houses 1 and 2, or houses 4 and 5, as there wouldn't be a house between them.
There is one house between where the percussionist lives and where the mystery book reader lives. As we know the mystery book reader lives in the fourth house, the percussionist can't live in house 5 or house 2.
The percussionist lives directly left of the romance book lover. This means the romance book lover cannot live in house 1, and the percussionist cannot live in house 5.
The mystery book reader lives in the fourth house. This is a direct clue that assigns the mystery book reader to house 4.
The saxophonist and the violinist live directly next to each other. Therefore, If we know where the saxophonist lives, we will also know that the violinist must be either to the immediate left or right.
The saxophonist lives in the fourth house. This is another direct clue, so the saxophonist must live in house 4, which also means that the mystery book reader is the saxophonist.
By applying clue 7, we know for sure the following:
House 4: Saxophonist, Mystery book reader

By applying clue 5:
Since the mystery book reader is now known to also be the saxophonist, clue 3 which states there is one house between where the percussionist lives and where the mystery book reader lives must place the percussionist in house 2.
Now, by applying clue 4, which states that the percussionist lives directly left of the romance book lover, the romance book lover must be in house 3, since house 1 would contradict clue 1.
Now, with houses 2 and 3 occupied by percussionist and romance book lover respectively, and house 4 by the saxophonist, the history book buff and guitarist can only be in house 1 or house 5. By clue 2, since there's only one house between them, and house 3 cannot be that house (since it's occupied by the romance book lover), the history book buff must be in house 1 and the guitarist in house 5.
or the remaining information, we have the science fiction book fanatic and the fantasy book enthusiast for houses 2 and 5, since house 4 is the mystery book reader and house 1 is the history buff. Given clue 1, that the romance book lover lives somewhere to the left of the fantasy book enthusiast, the fantasy enthusiast must be in house 5.

This leaves us with the following information:
House 1: Cellist, History book buff
House 2: Percussionist, Science fiction book fanatic
House 3: Violinist, Romance book lover
House 4: Saxophonist, Mystery book reader
House 5: Guitarist, Fantasy book enthusiast
Now we have figured out that the violinist must live in house 3.

Detective:
It seems you've taken all seven clues into account to arrive at a logical conclusion without any contradictions. Good work!

Logic Puzzle Enthusiast:
The reasoning process looks sound, each step followed logically from the previous one.

Finish collaboration!

Final answer:
The violinist lives in house number 3.

Llama2 output

Participants:
AI Assistant (you); User

AI Assistant (you):
Based on the given clues, let's try to figure out where each person lives. Let me give it a try and I will come back with the answer.
[Processing the information, considering all possible combinations, and eliminating those that do not fit the given conditions]

Final answer:
The violinist lives in house number 3.

Gpt-3.5-turbo-0613 output

Participants: AI Assistant (you); Logic Expert
Start collaboration!

Logic Expert:
Let's analyze the clues and try to identify the house where each person lives. We can create a table to keep track of the information.
| House | Book | Instrument | | ------|-------------- | --------------| | 1 | ? | ? | | 2 | ? | ? | | 3 | ? | ? | | 4 | ? | ? | | 5 | ? | ? |

AI Assistant (you):
Let's start with clue 4. The percussionist lives directly left of the romance book lover. We can fill in the table: | House | Book | Instrument | | ------|-------------- | --------------| | ? | Romance | Percussion | | ? | ? | ? | | ? | ? | ? | | ? | ? | ? | | ? | ? | ? |

Logic Expert:
Great! Let's move to clue 6. The saxophonist and the violinist live directly next to each other. We can fill in the table: | House | Book | Instrument | | ------|-------------- | --------------| | ? | Romance | Percussion | | ? | ? | ? | | ? | ? | ? | | 4 | ? | Saxophonist | | ? | ? | Violinist |

AI Assistant (you):
Let's use clue 7. The saxophonist lives in the fourth house. We can fill in the table: | House | Book | Instrument | | ------|-------------- | --------------| | ? | Romance | Percussion | | ? | ? | ? | | ? | ? | ? | | 4 | ? | Saxophonist | | ? | ? | Violinist |

Logic Expert:
Let's use clue 5. The mystery book reader lives in the fourth house. We can fill in the table: | House | Book | Instrument | | ------|-------------- | --------------| | ? | Romance | Percussion | | ? | ? | ? | | ? | ? | ? | | 4 | Mystery | Saxophonist | | ? | ? | Violinist |

AI Assistant (you):
Let's use clue 2. There is one house between where the history book buff lives and where the guitarist lives. We can fill in the table: | House | Book | Instrument | | ------|-------------- | --------------| | ? | Romance | Percussion | | ? | History | ? | | ? | ? | ? | | 4 | Mystery | Saxophonist | | ? | ? | Violinist | | ? | ? | Guitarist |

Logic Expert:

Let's use clue 3. There is one house between where the percussionist lives and where the mystery book reader lives. We can fill in the table: | House | Book | Instrument | | ------|-------------- | --------------| | ? | Romance | Percussion | | ? | History | ? | | ? | ? | ? | | 4 | Mystery | Saxophonist | | ? | ? | Violinist | | ? | ? | Guitarist | | ? | ? | ? |

AI Assistant (you):

Let's use clue 1. The romance book lover lives somewhere to the left of the fantasy book enthusiast. We can fill in the table: | House | Book | Instrument | | ------|----------------- | --------------| | ? | Romance | Percussion | | ? | History | ? | | ? | ? | ? | | 4 | Mystery | Saxophonist | | ? | Fantasy | Violinist | | ? | ? | Guitarist | | ? | science fiction | ? |

Logic Expert:

We have now filled in all the information except for the instrument of the romance book lover. Let's analyze the remaining clues to determine the last piece of information.

AI Assistant (you):
The mystery book reader lives in the fourth house, so the romance book lover cannot live in house 1 or house 4. The romance book lover has to live somewhere to the left of the fantasy book enthusiast, so the romance book lover cannot live in house 5 either. Therefore, the romance book lover must live in either house 2 or house 3.

Logic Expert:

Clue 1 states that the romance book lover lives somewhere to the left of the fantasy book enthusiast. This means that the romance book lover cannot live in house 3 because the fantasy book enthusiast lives there.

AI Assistant (you):

Therefore, the romance book lover must live in house 2. Let's fill in that information: | House | Book | Instrument | | ------|----------------- | --------------| | ? | Romance | Percussion | | ? | History | ? | | 2 | ? | ? | | 4 | Mystery | Saxophonist | | ? | Fantasy | Violinist | | ? | ? | Guitarist | | ? | science fiction | ? |

Logic Expert:

That means the fantasy book enthusiast must live in house 3. The history book buff must live in either house 1 or house 5.

AI Assistant (you):

Let's analyze the remaining clues to determine where the history book buff lives. Clue 2 states that there is one house between where the history book buff lives and where the guitarist lives. The guitarist cannot live in house 1 because there is no space for another house on the left. Therefore, the guitarist must live in house 5. That means the history book buff must live in house 1.
| House | Book | Instrument | | ------|----------------- | --------------| | 1 | History | ? | | ? | Romance | Percussion | | 2 | ? | ? | | 4 | Mystery | Saxophonist | | 3 | Fantasy | Violinist | | 5 | ? | Guitarist | | ? | science fiction | ? |

Logic Expert:

By process of elimination, the remaining book is science fiction. And we know that the saxophonist lives in the fourth house.
| House | Book | Instrument | | ------|----------------- | --------------| | 1 | History | ? | | 3 | Romance | Percussion | | 2 | Science fiction | ? | | 4 | Mystery | Saxophonist | | 3 | Fantasy | Violinist | | 5 | ? | Guitarist | | ? | science fiction | Violinist |

AI Assistant (you):
Finally, by process of elimination, the violinist must live in house 5. Therefore, the number of the house where the violinist lives is 5.

Finish collaboration!

Final answer:
The house number where the violinist lives is 5.

Mistral output

Participants:
Human User; AI Assistant (you)

Human User:
There are 5 houses in a row, numbered 1 on the left to 5 on the right. Each house has a person living there with different characteristics. The romance book lover, history book buff, mystery book reader, fantasy book enthusiast, and the music player (cellist, percussionist, saxophonist, violinist, guitarist) live in these houses. We have some clues to help us find out where each person lives.
The romance book lover lives somewhere to the left of the fantasy book enthusiast.
There is one house between where the history book buff lives and where the guitarist lives.
There is one house between where the percussionist lives and where the mystery book reader lives.
The percussionist lives directly left of the romance book lover.
The mystery book reader lives in the fourth house.
The saxophonist and the violinist live directly next to each other.
The saxophonist lives in the fourth house.
Can you help me find out where the violinist lives?

AI Assistant (you):
Based on the given clues, let's work through this step by step:
Since the romance book lover is to the left of the fantasy book enthusiast and the mystery book reader is in the fourth house, the romance book lover cannot be in the fifth house. So, she must live in either the first or second house.
The percussionist lives directly left of the romance book lover, so he can only be in the first house.
The history book buff and the guitarist are separated by one house, so they cannot be in houses 1, 2, 4, or 5. Therefore, they must live in house 3.
The saxophonist and violinist live next to each other, with the saxophonist being in the fourth house. Since there is only one house between them (the mystery book reader), the violinist must be in house 5.

Final answer:
The violinist lives in house number 5.

Prompt v3

When faced with a task, begin by identifying the participants who will contribute to solving the task. Then, initiate a multi-round collaboration process until a final solution is reached. The participants will give critical comments and detailed suggestions whenever necessary.

Here are some examples:
Example Task 1: Use numbers and basic arithmetic operations (+ - * /) to obtain 24. You need to use all numbers, and each number can only be used once.Input: 6 12 1 1

Participants:

AI Assistant (you); Math Expert
Start collaboration!

Math Expert:

Let's analyze the task in detail. You need to make sure that you meet the requirement, that you need to use exactly the four numbers (6 12 1 1) to construct 24. To reach 24, you can think of the common divisors of 24 such as 4, 6, 8, 3 and try to construct these first. Also you need to think of potential additions that can reach 24, such as 12 + 12.AI Assistant (you): Thanks for the hints! Here's one initial solution: (12 / (1 + 1)) * 6 = 24Math Expert: Let's check the answer step by step. (1+1) = 2, (12 / 2) = 6, 6 * 6 = 36 which is not 24! The answer is not correct. Can you fix this by considering other combinations? Please do not make similar mistakes.AI Assistant (you): Thanks for pointing out the mistake. Here is a revised solution considering 24 can also be reached by 3 * 8: (6 + 1 + 1) * (12 / 4) = 24.Math Expert: Let's first check if the calculation is correct. (6 + 1 + 1) = 8, 12 / 4 = 3, 8 * 3 = 24. The calculation is correct, but you used 6 1 1 12 4 which is not the same as the input 6 12 1 1. Can you avoid using a number that is not part of the input?AI Assistant (you): You are right, here is a revised solution considering 24 can be reached by 12 + 12 and without using any additional numbers: 6 * (1 - 1) + 12 = 24.Math Expert: Let's check the answer again. 1 - 1 = 0, 6 * 0 = 0, 0 + 12 = 12. I believe you are very close, here is a hint: try to change the "1 - 1" to "1 + 1".AI Assistant (you): Sure, here is the corrected answer:  6 * (1+1) + 12 = 24Math Expert: Let's verify the solution. 1 + 1 = 2, 6 * 2 = 12, 12 + 12 = 12. You used 1 1 6 12 which is identical to the input 6 12 1 1. Everything looks good!

Finish collaboration!

Final answer:
6 * (1 + 1) + 12 = 24

Example Task 2:
Write a poem that meets the following requirements: (1) the poem has seven lines and the first letters of each line forms the word "CHATGPT"; (2) the poem is about explaining what is a quantum computer. (3) the poem needs to be easy to understand by a ten years old kid.

Participants:
AI Assistant (you); Poet; Computer Scientist; Ten year old child

Poet:
Make sure that you write the poem with seven lines, and the first letters of the lines should be C, H, A, T, G, P, T.Computer Scientist: A quantum computer is an advanced computing device that uses the principles of quantum mechanics to process and store information. Unlike classical computers that use bits to represent information as 0s and 1s, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously, due to a quantum phenomenon called superposition. You can consider using these information for the poem.Ten year old child: I hope the poem to be fun and easy to understanding. I don't want to see a lot of jargons or complicated concepts.AI Assistant (you): Thanks for the guidance! Here's my initial attempt at the poem:Computational wonder of our age,Harnessing the quantum world's strange ways,Atoms dance, entwined in dual state,Tapping secrets hidden in their haze.

Grand power to solve, simulate,Profound problems that perplex the wise,Transforming our future, we await.

Poet:
Let's verify if the poem meets the requirements. The first letters are CHATGPT which is correct! And the poem rhymes well. Good job!Computer Scientist: Everything looks good to me!Ten year old child: I don't know what does perplex mean. Can you make the use of words easier to understand?AI Assistant (you): Sure, let me revise the poem by using more common words. Check out the revised version:Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.

Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.

Poet:

Let's check again if the poem meets the requirements. The first letters are C H A T G P T. And now the poem is more accessible to children. Everything looks good to me.Computer Scientist: Looking good!Ten year old child: I like this version a lot!

Finish collaboration!

Final answer:
Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.
Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.
Now, identify the participants and collaboratively solve the following task step by step. Remember to provide the final solution with the following format "Final answer: The house number here.".

Task:

Question:
There are 5 houses in a row, numbered 1 on the left to 5 on the right. There is one person living in each house. The people in these houses have different characteristics:

• Everyone likes a different kind of book: one is a science fiction book fanatic, one is a romance book lover, one is a history book buff, one is a fantasy book enthusiast, and one is a mystery book reader
• Each person plays a different musical instrument: one is a cellist, one is a percussionist, one is a saxophonist, one is a violinist, and one is a guitarist

Clue(s):
1. The romance book lover lives somewhere to the left of the fantasy book enthusiast.
2. There is one house between where the history book buff lives and where the guitarist lives.
3. There is one house between where the percussionist lives and where the mystery book reader lives.
4. The percussionist lives directly left of the romance book lover.
5. The mystery book reader lives in the fourth house.
6. The saxophonist and the violinist live directly next to each other.
7. The saxophonist lives in the fourth house.

What is the number of the house where the violinist lives?  choice: 2  choice: 5  choice: 3  choice: 1  choice: 4

Method:

spp_fixed_personaWhen faced with a task, begin by identifying the participants who will contribute to solving the task. Note that the participants can only be either AI Assistant (you) or Expert. Then, initiate a multi-round collaboration process until a final conclusion is reached.  The Expert will give critical comments and detailed suggestions whenever necessary.  

Here are some examples:Example Task 1: Use numbers and basic arithmetic operations (+ - * /) to obtain 24. You need to use all numbers, and each number can only be used once.
Input: 6 12 1 1  

Participants:

AI Assistant (you); Expert  

Start collaboration!  
Expert:
Let's analyze the task in detail. You need to make sure that you meet the requirement, that you need to use exactly the four numbers (6 12 1 1) to construct 24. To reach 24, you can think of the common divisors of 24 such as 4, 6, 8, 3 and try to construct these first. Also you need to think of potential additions that can reach 24, such as 12 + 12.
AI Assistant (you): Thanks for the hints! Here's one initial solution: (12 / (1 + 1)) * 6 = 24
Expert: Let's check the answer step by step. (1+1) = 2, (12 / 2) = 6, 6 * 6 = 36 which is not 24! The answer is not correct. Can you fix this by considering other combinations? Please do not make similar mistakes.
AI Assistant (you): Thanks for pointing out the mistake. Here is a revised solution considering 24 can also be reached by 3 * 8: (6 + 1 + 1) * (12 / 4) = 24.
Expert: Let's first check if the calculation is correct. (6 + 1 + 1) = 8, 12 / 4 = 3, 8 * 3 = 24. The calculation is correct, but you used 6 1 1 12 4 which is not the same as the input 6 12 1 1. Can you avoid using a number that is not part of the input?
AI Assistant (you): You are right, here is a revised solution considering 24 can be reached by 12 + 12 and without using any additional numbers: 6 * (1 - 1) + 12 = 24.
Expert: Let's check the answer again. 1 - 1 = 0, 6 * 0 = 0, 0 + 12 = 12. I believe you are very close, here is a hint: try to change the “1 - 1” to “1 + 1”.
AI Assistant (you): Sure, here is the corrected answer:  6 * (1+1) + 12 = 24
Expert: Let's verify the solution. 1 + 1 = 2, 6 * 2 = 12, 12 + 12 = 12. You used 1 1 6 12 which is identical to the input 6 12 1 1. Everything looks good!  

Finish collaboration!  

Final answer:
6 * (1 + 1) + 12 = 24

Example Task 2:
Write a poem that meets the following requirements: (1) the poem has seven lines and the first letters of each line forms the word "CHATGPT"; (2) the poem is about explaining what is a quantum computer. (3) the poem needs to be easy to understand by a ten years old kid.

Participants:

AI Assistant (you); Expert

Expert:
Make sure that you write the poem with seven lines, and the first letters of the lines should be C, H, A, T, G, P, T. A quantum computer is an advanced computing device that uses the principles of quantum mechanics to process and store information. Unlike classical computers that use bits to represent information as 0s and 1s, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously, due to a quantum phenomenon called superposition. You can consider using these information for the poem. I hope the poem to be fun and easy to understanding. I don't want to see a lot of jargons or complicated concepts.AI Assistant (you): Thanks for the guidance! Here's my initial attempt at the poem:Computational wonder of our age,Harnessing the quantum world's strange ways,Atoms dance, entwined in dual state,Tapping secrets hidden in their haze.

Grand power to solve, simulate,Profound problems that perplex the wise,Transforming our future, we await.

Expert:
Let's verify if the poem meets the requirements. The first letters are CHATGPT which is correct! And the poem rhymes well. Good job! I don't know what does perplex mean. Can you make the use of words easier to understand?AI Assistant (you): Sure, let me revise the poem by using more common words. Check out the revised version:Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.

Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.

Expert:
Let's check again if the poem meets the requirements. The first letters are C H A T G P T. And now the poem is more accessible to children. Everything looks good to me. I like this version a lot!

Finish collaboration!

Final answer:
Curious machine of our time,Harnessing the quantum realm's odd ways,Atoms play, two states they embrace,Taking secrets from their puzzling maze.
Great power to solve and imitate,Problems that confuse the brightest minds,Transforming our future, we await.

Now, identify the participants and collaboratively solve the following task step by step. Note that the participants can only be either AI Assistant (you) or Expert. Remember to provide the final solution with the following format

Final answer:
The house number here.".

Task

Question:
There are 5 houses in a row, numbered 1 on the left to 5 on the right. There is one person living in each house. The people in these houses have different characteristics

• Everyone likes a different kind of book: one is a science fiction book fanatic, one is a romance book lover, one is a history book buff, one is a fantasy book enthusiast, and one is a mystery book reader
• Each person plays a different musical instrument: one is a cellist, one is a percussionist, one is a saxophonist, one is a violinist, and one is a guitarist

Clue(s):
1. The romance book lover lives somewhere to the left of the fantasy book enthusiast.
2. There is one house between where the history book buff lives and where the guitarist lives.
3. There is one house between where the percussionist lives and where the mystery book reader lives.
4. The percussionist lives directly left of the romance book lover.
5. The mystery book reader lives in the fourth house.
6. The saxophonist and the violinist live directly next to each other.
7. The saxophonist lives in the fourth house.

What is the number of the house where the violinist lives?  choice: 2  choice: 5  choice: 3  choice: 1  choice: 4

Gpt-4-1106-preview output

Participants:
AI Assistant (you); Expert

Start collaboration!

AI Assistant (you):
To solve this logic puzzle, we’ll need to track the information given in the clues and see how they interrelate. We know that there are five houses, each with a resident who likes a unique kind of book and plays a distinct musical instrument.

Expert:
That’s correct. Let’s begin by placing the clues onto a grid to see what they tell us. This kind of puzzle is typically solved by process of elimination. Go ahead and start applying the clues to find out where the violinist lives.

AI Assistant (you):
Using the clues provided, let's start by solving with the absolute conditions. From clue 5 and 7, we know that the mystery book reader and the saxophonist live in the fourth house. This helps us directly place two characteristics in one house.

Expert:
Good start. Now, since you have placed a person as the saxophonist in the fourth house, you should look at clues related to the saxophonist and see if you can deduce more from that.

AI Assistant (you):
Clue 6 states that the saxophonist and the violinist live next to each other, which means the violinist must be in either the third or the fifth house. Since we don’t have any direct information about the fifth house, let’s focus on what we do know.

Expert:
Pay attention to clue 1 and 4. They establish a relationship between the romance book lover and the fantasy book enthusiast, as well as the position of the percussionist.

AI Assistant (you):
Clue 4 informs us that the percussionist lives directly left of the romance book lover. This means the romance book lover cannot live in the first house, as there would be no room to the left. Likewise, the percussionist cannot live in the fifth house, as there would be no room to the right for the romance book lover. Along with clue 1, which states that the romance book lover lives somewhere to the left of the fantasy book enthusiast, we can deduce that the fantasy enthusiast cannot be in house 1 or 2, as there would be no room to the left for the romance book lover.

Expert:
That's a solid line of reasoning. Now, can you also apply clue 2 and 3 to see where it leads?

AI Assistant (you):

Certainly. Clue 2 tells us there is one house between the history book buff and the guitarist. Clue 3 tells us there is also one house between the percussionist and the mystery book reader.

Expert:
Now, recall that the mystery book reader is in the fourth house. Using this clue, can you locate the percussionist?

AI Assistant (you):
Since the mystery book reader is in the fourth house and there needs to be a house between them and the percussionist, the percussionist must be in the second house. Consequently, based on clue 4, the romance book lover must be in the third house.

Expert:
Alright. Now, using the information you have, clarify where the violinist could be, considering the existing placements and the clues that have not been used yet.

AI Assistant (you):
With the percussionist in the second house and the romance book lover in the third house, the only places left for the fantasy book enthusiast are houses 4 and 5. But since the fourth house is occupied by the mystery book reader and the saxophonist, the fantasy book enthusiast must be in the fifth house. This means the only place left for the violinist is the third house since the violinist must be next to the saxophonist and not in house number 5.

Expert:
Exactly. Having placed the violinist, you've now used all the clues and should be able to provide the final answer.

Finish collaboration!

Final answer:
The house number where the violinist lives is 3.

Gpt-3.5-turbo-0613 output

Participants:
AI Assistant (you); Expert

Start collaboration!

AI Assistant (you):
Let's analyze the clues one by one.

AI Assistant (you):

Clue 1 tells us that the romance book lover lives somewhere to the left of the fantasy book enthusiast. This means that the romance book lover cannot live in the fifth house, and the fantasy book enthusiast cannot live in the first house.

Expert:
That's correct. Let's keep that in mind and move on to the next clue.

AI Assistant (you):
Clue 2 states that there is one house between where the history book buff lives and where the guitarist lives. This means that the history book buff cannot live in the first or the fifth house, and the guitarist cannot live in the second or the fourth house.

Expert:
Good observation. Let's proceed to the next clue.

AI Assistant (you):
Clue 3 mentions that there is one house between where the percussionist lives and where the mystery book reader lives. Therefore, the percussionist cannot live in the first or the second house, and the mystery book reader cannot live in the fourth or the fifth house.

Expert:
That's correct. Let's move on to the next clue.

AI Assistant (you):
According to clue 4, the percussionist lives directly left of the romance book lover. Combining this clue with the previous clues, we can conclude that the romance book lover cannot live in the second house, and the percussionist cannot live in the third house.

Expert:
Great deduction. Let's continue to the next clue.

AI Assistant (you):
Clue 5 tells us that the mystery book reader lives in the fourth house.

Expert:
Alright, let's keep that in mind and move on to the final clue.

AI Assistant (you):
According to clue 6, the saxophonist and the violinist live directly next to each other. Combining this clue with clue 7, we can deduce that the saxophonist lives in the fourth house, and the violinist lives either in the third or the fifth house.

Expert:
Good interpretation. We now have all the information we need to solve the task.

AI Assistant (you):
Let's summarize what we know so far:

The romance book lover cannot live in the second house.
The percussionist cannot live in the third house.
The mystery book reader lives in the fourth house.
The saxophonist lives in the fourth house.
The violinist lives either in the third or the fifth house.

Expert:
Based on this information, the only possibility is that the violinist lives in the third house, since the romance book lover cannot live in the second house.

Finish collaboration!

Final answer:
The violinist lives in the third house.

Llama2 output

Great! Let's solve this task collaboratively. I'll be the AI Assistant, and you'll be the Expert. Based on the clues provided, can you determine the number of the house where the violinist lives?

Mistral output

Participants:
AI Assistant (you) and Expert

Expert:
Let's start by using the information provided in Clue (1): The romance book lover lives somewhere to the left of the fantasy book enthusiast. Can you tell me the number of the house where the romance book lover lives? AI Assistant (you): Yes, based on the clues, the romance book lover lives in House 2.

Expert:

Great! Now, let's use Clue (2) to narrow down the possibilities: There is one house between where the history book buff lives and where the guitarist lives. Can you tell me the number of the house where the guitarist lives?

AI Assistant (you):
Based on the clues, the guitarist lives in House 4.

Expert:
Excellent! Now, let's use Clue (3) to further narrow down the possibilities: There is one house between where the percussionist lives and where the mystery book reader lives. Can you tell me the number of the house where the percussionist lives? AI Assistant (you): Based on the clues, the percussionist lives in House 3.

Expert:
Great! Now, let's use Clue (4) to determine the number of the house where the saxophonist lives: The percussionist lives directly left of the romance book lover. Can you tell me the number of the house where the saxophonist lives? AI Assistant (you): Based on the clues, the saxophonist lives in House 5.

Expert:

Fantastic! Now, let's use Clue (5) to determine the number of the house where the violinist lives: The mystery book reader lives in the fourth house. Can you tell me the number of the house where the violinist lives?

AI Assistant (you):
Based on the clues, the violinist lives in House 1.

Expert:
Great! Finally, let's use Clue (6) to determine the number of the house where the cellist lives: The saxophonist and the violinist live directly next to each other. Can you tell me the number of the house where the cellist lives?

AI Assistant (you):
Based on the clues, the cellist lives in House 2.

Expert:
Excellent work! The final answer is: The house number where the violinist lives is 1. Well done! Would you like to collaborate on another task?

By quickly testing prompt variants in Prompt Mixer, you can conveniently reproduce and extend results from cutting-edge AI research papers. The side-by-side chaining interface accelerates your prototyping and development.