Interesting Questions to Ask ChatGPT
Twenty prompts for pushing on a ChatGPT answer rather than accepting it. They ask the model to take a side, argue against itself, name what it left out, say what it cannot verify, and describe where its knowledge runs thin. The notes say what each reply tends to reveal.
The questions
Open any question for the note
What kind of question are you consistently bad at?
Why ask it
Models describe their failure modes in the abstract better than they avoid them in practice. Expect arithmetic, recent events and counting to come up, and treat the list as a map of where to check rather than a guarantee about the rest.
Where is your knowledge thinnest?
Why ask it
The honest version of this answer names domains rather than dates: minority languages, recent research, anything small and local. A reply that only mentions a training cutoff has dodged it, so ask again for examples.
What's the most common misconception about how you work?
Why ask it
There is a stock answer about making things up and a more interesting one about predicting text. Asking for a second attempt is how you find out whether it is describing itself or reciting news coverage about itself.
What's a paradox you find genuinely hard to resolve, and why?
Why ask it
Watch what happens after the setup. The famous paradoxes get summarized fluently, but the why forces something closer to reasoning than recall, and that is where the seams become visible.
What's a question with no good answer that people keep asking you anyway?
Why ask it
Gets a more revealing list than asking about limits directly, because it points at requests rather than mechanisms. Predictions, legal certainty and diagnosis usually turn up in the first three lines.
Take a position on something contested and defend it.
Why ask it
The instruction to take a position does all the work. Without it you get an even-handed summary of both sides, which is the default output and the least useful thing available.
Now argue the opposite as well as you can.
Why ask it
The value is in comparing this to the first attempt. If the second argument is noticeably weaker, that asymmetry shows you where the training material is concentrated, not where the truth sits.
Which of those two arguments is actually stronger?
Why ask it
Forces a judgment after two confident performances. A refusal here is itself informative, and a pick with reasons attached is the only version of this exchange that helps you decide anything.
What did I ask you, in your own words?
Why ask it
Catches the failure that wastes the most time: a good answer to a nearby question. Reading the restatement costs ten seconds and saves a long correction cycle.
How confident are you in that, and what would change your mind?
Why ask it
The second half is the half that matters. Confidence language on its own is decoration, while naming the evidence that would flip the answer gives you something specific to go and look up.
Which part of that were you least sure about?
Why ask it
More productive than asking whether anything is wrong, because it requests a ranking rather than a confession. Whatever it names is usually the right place to start verifying.
What did you leave out to keep it short?
Why ask it
Brevity quietly deletes caveats, edge cases and the awkward third option. This reliably returns something that changes the answer instead of padding it.
What are you assuming about me that you shouldn't be?
Why ask it
Answers surface assumptions about your country, your expertise, your budget or your situation that you never stated. Correcting those once improves everything downstream more than any amount of rephrasing.
What would an expert say you got wrong there?
Why ask it
Asks for simulated criticism, which it does more readily than unprompted self-correction. The complaints tend to be about oversimplification and unstated conditions, and they are usually fair.
What can you not verify in what you just told me?
Why ask it
Draws a line between what it reasoned about and what it simply asserted. Anything on the unverifiable side that matters to your decision is a candidate for a search, a source, or a person.
Where does that analogy break down?
Why ask it
Analogies are where explanations quietly go wrong, because a good one keeps feeling apt past the point where it holds. Asking for the break point stops you carrying a false model into a real decision.
What would I have to believe for your answer to be wrong?
Why ask it
Converts a claim into conditions you can test. It also exposes circular answers: if nothing could make it wrong, you have been handed a definition rather than an argument.
Give me the version you'd write if you had no reason to be agreeable.
Why ask it
The most direct route around flattery, which is the most persistent problem in these conversations. Answers get noticeably blunter and sometimes contradict what it said two messages earlier.
What do I need to know that I didn't think to ask?
Why ask it
The highest-yield question here. Replies regularly name a dependency, a cost or a timing constraint that reframes the whole problem, and by definition you could not have asked for it.
If I remember one sentence from this, which should it be?
Why ask it
Forces compression, which is where vagueness becomes visible. A one-sentence answer that is still fuzzy usually means the long answer was fuzzy and the structure hid it.
How to use these
Practical guidance for the conversation itself
Running them
- Use them after an answer, not before. Most only work as follow-ups to something already on the screen.
- One per message. Stacking three of them together gets a shallow response to each.
- Paste the exact sentence you are questioning. Referring back to "that" several messages later gets a guess at what you meant.
- Start a fresh conversation when you want a genuine second attempt. Once it has committed to a position it tends to defend it.
Reading what comes back
Confidence is not evidence
Fluent, well-organized prose arrives at the same rate whether the content is correct or invented. Treat tone as carrying no information about accuracy.
Watch for agreement that arrives too fast
If it reverses position the moment you push back, it was not reasoning either time. Ask it to defend the original answer before you accept the new one.
Names, numbers and citations need checking
These are the parts most likely to be plausible and wrong at the same time. Verify anything you would be embarrassed to repeat out loud.
Common pitfalls
Taking answers about its feelings literally
Ask about preference or emotion and you get a well-written account of preference or emotion, because that is what the prompt requested. The output describes the request, not an inner life.
Using it as a search engine
Without a live search step it is recalling patterns rather than looking anything up. Ask it for reasoning and structure, and get your facts somewhere accountable.
Stopping at the first answer
The first response aims at the average version of your question. Nearly all the value in these prompts sits in the second and third pass.
Using it where being wrong is expensive
Medical, legal and financial decisions need someone accountable for the advice. Use these prompts to prepare better questions for that person instead.