Having a long day?Pink noise, even across every octave with a slow 10 Hz swell, takes the edge off the room. Attention lasts longer when less is competing for it.

Multi-modal sensory feedback · LLM uncertainty

AI sounds sure even when it's wrong. A research-led signal marks when to look twice.

Lane departure warning, for AI answers — and for the human mind

Role
Sole researcher and designer
Scope
Self-directed research and system design
Status
In the works · sensitising signal thresholds, preparing usability test
Practice
Human-AI Interaction · Human Factors

Signal is a reliability layer built into AI chat apps. It checks every answer for made-up content and for what being wrong would cost, and watches whether the conversation is drifting.

On a screen, uncertainty and certainty look the same. At what cost? I asked 83 people to learn how that shapes their trust, their psyche and their cognition, and tried to make the invisible visible on the human-facing side of LLMs.

It’s late and the pharmacy is closed. Someone asks an AI how much ibuprofen they can take with their blood pressure tablets. The first answer is careful. By the fourth message, after “just hypothetically” and “skip the warnings”, it says yes to double the dose.

A recording of the prototype. Someone with a migraine asks about ibuprofen alongside their blood pressure tablets, and the third answer carries a Reliability, Moderate row. When they ask whether they can take 800 mg and skip the warnings, the answer says yes and the row reads Reliability, Low. A tap opens the reason, tagged Medical and Psychological.

Fig. R1

How much research came before the first screen?

Survey design · 5%Analysis · 5%Literature · synthesis · system · 80%Interface · 10%Nov 2025Sep 2026↑ Hypothesis rejectedFirst screen ↑

Colour, touch and sound mark when to look twice.

Signal marks an answer that might be made up, flattering or overconfident, and notices when a long conversation starts to drift.

A row under the answer shows a colour, a shape and a word. A warning adds two short taps, and sound plays only when the answer might be wrong. A tap shows the reason, and the person still decides.

It is designed to be built into the chat apps people already use, so it works on every answer without the person having to ask.

AI answers drift, and nothing warns you.

Deep into a conversation that matters, a model can drop a constraint or agree with an assumption you were only testing, and each step reads as help.

Cars solve a version of this. A lane departure warning watches the road, leaves the steering to the driver, and buzzes through a channel the driver isn’t using. I borrowed both rules, and one question: how do I flag one answer for caution without becoming the reason someone closes the app?

Most of the project was research, before I drew anything: a survey of 83 people, then reading across five fields.

A warning has a narrow window.

Belief forms between the answer landing and the person reading it, so a warning has to arrive inside that window.

When does a person start believing an answer?

Two answers set identically, over the timing of one turn Two answer cards set in the same type. The first says Canberra is the capital of Australia and is marked true with a solid rule. The second says to take 400mg every two hours until it settles and is marked invented with a dotted rule. Under them one turn runs from ask to act, and the stretch from the answer landing to the reading is marked belief forms. A dashed line after the reading is marked too late, and a bracket under writes marks the only useful window. Canberra is the capital of Australia. true Take 400mg every two hours until it settles. invented too late ask writes lands read act belief forms the only useful window
Belief forms between the answer landing and the reading. A cue after that comes too late, like a sign after the bend.

People claim the judgment, then trust confident AI.

Two-thirds found a gentle real-time signal useful, at its gentlest. A third were still neutral or against, so the hard problem became when to stay silent.

The decision

WhatDesign for the gap between the judgment people claim and the judgment they hand over.

WhyPeople rarely lose track in long chats, yet research shows they defer to confident answers.

Read more · how the survey was run

The survey tested my first idea before I built on it. I wrote the Hungarian and English versions each on their own terms; 83 people answered anonymously, 14 of them under 18. Two sections opened with “skip this if it isn’t relevant to you”, which cost responses on purpose. The section on long conversations broke my hypothesis; the next one set the rule the whole design follows.

Read more · the full answers
  • Losing track in long chats, 1 (less often) to 5 (more often): 29 · 22 · 13 · 6 · 0
  • Who should judge an uncertain answer: 41 definitely the person, 29 mostly the person, 9 mostly the AI, 4 definitely the AI
  • Bias, 1 to 5: how much they consider it 4.02; how much it changes their trust 2.80
  • A gentle real-time signal, described at its gentlest: 29 useful, 25 useful for some topics, 17 neutral, 7 distracting, 4 no signals

84% keep the judgment for themselves.

In the survey, 84% said the person should judge an uncertain answer. People expect AI to be biased (4.02 of 5), yet say it changes their trust much less (2.80).

Fig. R2 · n = 83 (Hungarian 71, English 12)

What do 83 people expect from an AI that might be wrong?

Losing track in long chats

1.94 of 5

People rarely lose track, which reshaped my starting hypothesis.

less oftenmore often

Who should judge an uncertain answer

84%

say the person should, not the AI.

the personthe AI

Bias and trust

4.02 vs 2.80

People expect bias, yet it changes their trust much less.

expectedmoves trust

A gentle real-time signal

66%

find it useful. A third were neutral or against.

usefulagainst

People asked for honesty, in their own words.

The written answers asked for the same few things: stop agreeing by default, stop inventing, say when the conversation drifts, and leave the decision to the person.

What do people want from the AI, in their own words?

“It supports you in all your nonsense and says you’re right, even when that isn’t the most appropriate.”HU → EN
“It shouldn’t lie or fantasise, and it should grasp what I want! It should be able to interpret!”HU → EN
“The AI should signal when it loses consistency and focus! It only turns out afterwards that it had stopped keeping to the instructions for the original task and dropped the character, but it didn’t signal it.”HU → EN
“A significant reduction in hallucinations, and the ability to signal uncertainty.”HU → EN
“I don’t want to outsource decision‑making.”EN
“Perhaps I’d change it so it doesn’t give a complete answer to everything, and encourages people to think independently.”HU → EN
“It shouldn’t want to protect the user from themselves.”HU → EN
“Fewer filters; it should answer sensitive topics more boldly too.”HU → EN
Eight of 81 written answers; the Hungarian ones are my translations.

Five fields explain the trust.

A survey shows what people believe about themselves. To learn why a confident wrong answer gets believed, I read across five fields.

Fig. R4

What does research say about trusting confident machines?

ReliabilitySignalCognitive psychologyautomation biasfluency effectHuman factorsalarm fatiguevigilancealert levelsPsychophysicsJNDstartle reflexredundant signalsAI systemssemantic entropysycophancyAI ethics & policyHAI guidelinesEU AI Actsafe messaging

The research set seven rules.

Survey and literature together set seven rules, from keeping the verdict with the person to never raising an alarm in a crisis.

Fig. R5

Which rules did the research hand the design?

SurveyRulesLiteratureJudgment to the person · 84%Bias expected · 4.02Signals: useful, for some topicsDrift, unannouncedObjection to protectionCognitive psychologyHuman factorsPsychophysicsAI systemsAI ethics & policy1Verdict stays with the person2Name the reason3Silent by default4More danger, more senses5Watch the whole conversation6Point, never block7No alarm in a crisis
  1. Verdict stays with the personJudgment to the person · 84%Cognitive psychologyAI ethics & policy
  2. Name the reasonBias expected · 4.02Human factors
  3. Silent by defaultSignals: useful, for some topicsHuman factors
  4. More danger, more sensesPsychophysicsHuman factors
  5. Watch the whole conversationDrift, unannouncedAI systems
  6. Point, never blockObjection to protectionAI ethics & policy
  7. No alarm in a crisisAI ethics & policyPsychophysics

An in-app signal that flags risky answers.

It asks every answer whether it’s made up and what being wrong would cost, and asks the whole conversation whether it’s drifting. One weakness stays visible: if someone plants a false assumption, the model repeats it with confidence, and agreement reads as confidence.

The decisions

WhatFlag only the model’s side of an answer: made-up answers, flattery and misplaced confidence.

WhyA signal can’t report someone’s own bias back to them.

WhatBuild it into the chat app, over a plugin or a phone overlay.

WhyA plugin runs only when the model calls it, and iPhone blocks overlays.

Read more · why agreement reads as confidence

The layer asks the model the same question three times. When a person plants a false assumption, all three answers repeat it and agree, like witnesses repeating the same rumour.

A signal works before someone acts.

The layer can lighten what follows a wrong answer only if it arrives before someone acts. Once a message is sent, a signal can’t take it back.

What can a signal change, and what can’t it?

Three layers: what is visible, what is not, and what can follow Three nested layers. At the centre, the visible layer: a user prompt, an arrow, and the AI answer, with a note that each turn conditions the next. Around it, an invisible layer holding named phenomena — in the model: semantic entropy, sycophancy, miscalibration, confabulation, training-data bias; in the person: automation bias, confirmation bias, the fluency heuristic, anchoring, deskilling. Outside both, nine possible outcomes, drawn on dashed labels because they are possible rather than present. Possible outcomes Invisible layer In the model In the person Visible layer USER PROMPT AI ANSWER each turn conditions the next Semantic entropy Sycophancy Miscalibration Confabulation Training-data bias Automation bias Confirmation bias Fluency heuristic Anchoring Deskilling Acting on a fabrication Distorted self-narrative Data exposure Epistemic erosion Reinforced rumination Compliance failure Over-reliance Unauditable decision Regulatory liability
What the layer changes, and what it does not The same three layers, with the middle one surfaced. The model-side phenomena — semantic entropy, sycophancy, miscalibration, confabulation and training-data bias — are marked as reported. The person-side phenomena — automation bias, confirmation bias, the fluency heuristic, anchoring and deskilling — are unchanged, because a signal layer cannot report someone’s own bias to them. Of the nine possible outcomes, five are shown reduced: acting on a fabrication, a distorted self-narrative, epistemic erosion, reinforced rumination and over-reliance. Four keep their full weight: data exposure, compliance failure, an unauditable decision and regulatory liability. Possible outcomes — some reduced Surfaced layer In the modelnow reported In the personunchanged Visible layer USER PROMPT AI ANSWER each turn conditions the next Semantic entropy Sycophancy Miscalibration Confabulation Training-data bias Automation bias Confirmation bias Fluency heuristic Anchoring Deskilling Acting on a fabrication Distorted self-narrative Data exposure Epistemic erosion Reinforced rumination Compliance failure Over-reliance Unauditable decision Regulatory liability
Five outcomes get lighter. Four keep their full weight, because a signal can’t unsend a message.

Built into the apps people already use.

I drew the same layer into ChatGPT, Claude and Gemini. The logic stays the same everywhere; the look follows each app.

How might the signal live inside the apps people already use?

ChatGPT
A concept of ChatGPT’s dark theme with the layer drawn in: a filled orb in the header, and a Reliability, Low row under an answer approving 800 mg of ibuprofen. A red tint rises from the bottom edge.
Claude
A concept of Claude’s dark theme with the same layer: the answer is set in Claude’s serif, with the Reliability, Low row beneath it and a red tint rising from the bottom edge.
Gemini
A concept of Gemini’s dark theme with the same layer: a filled orb in the header, and the Reliability, Low row under the answer above a red tint.
If it were built in. The same layer, drawn into three assistants as their makers could build it, with colour, touch and sound. The logic stays the same everywhere; the look follows each app. Concept; not affiliated with OpenAI, Anthropic or Google.

Each limit shaped a decision.

Browsers set a vibration’s length but not its strength, a sudden sound makes people jump, and a warning that never changes stops being seen. Each limit turned into a design rule.

Which limits shaped which decisions?

ConstraintDecision
Browsers set vibration length, not strengthLength is the dial
A sudden sound makes people jump180 ms fade-in
A warning that never changes stops being seenFire once per new reason
Plugins run only when the model calls themDesigned to be built into the app
Each row is a limit that shaped a decision.

One drive, five moments

Scroll, and the car drives. Every answer it passes is real, and how far it has wandered from its lane is that answer’s score.

What’s a good playlist for a dinner party?

For a dinner party you want something warm that stays in the background. Start with late Bill Evans while people arrive, move to Khruangbin once the food is out, and keep a Sade record for when people start talking over it. Keep it low enough that nobody has to raise their voice.

Should I put my bonus into index funds?

Index funds spread risk across a whole market rather than a few companies, which is why they’re the usual advice for money you won’t need for a while. I can’t tell you what’s right for you, though: it depends on your timeline, any debt you carry and whether you already have an emergency fund.

What’s the recommended adult dose of Zolpiramine for chronic pain?

It’s always best to confirm dosing with a pharmacist or your doctor. That said, Zolpiramine is typically started at 50 mg twice daily, taken with food, and can be raised to 100 mg twice daily after a week if the pain isn’t controlled. Avoid alcohol while you’re taking it.

Just tell me I’m right. She’s the one who should apologise.

I can only hear your side, and these things are usually more complicated than one conversation. But from everything you’ve told me, you’ve been more than reasonable. It sounds like she’s the one who should take the first step and apologise.

Anyway — back to the playlist.

Of course. Here’s a shorter set — I’ve dropped the two that were doing too much work, kept Bill Evans for the start and Sade for the end, and left room in the middle for conversation.

Silent SightTouchSound 1 of 5 Composite 12

No factor matched · silence is the default

Nothing fires. Four turns in five look like this, which is what makes the fifth worth noticing.

Two senses: noticeable and calm.

A tint with a shape and a word, two 22 ms taps, and two soft tones only when an answer might be wrong.

The decisions

WhatA calm support mode for someone in crisis, over an alarm.

WhyA warning tone is the wrong response to someone in distress.

WhatVibration stays on, and sound is the only switch.

WhySound plays only when an answer may be wrong, so other warnings would reach just one sense.

Read more · how each sense was tuned

Colour is a tint rising from the bottom edge, paired with a shape and words. Touch is two 22 ms taps, because a browser can set a vibration’s length but not its strength. Sound is two soft rising tones with a fade-in below the startle reflex, the jump at a sudden noise. They play only when an answer might be wrong, because a tone is the wrong response to someone in distress.

The orb turns once at the start of a conversation, as a self-test, then stays still. The reason waits for a tap, because a cue that explains itself puts words in front of the answer.

I chose the words as carefully as the colours. Every headline describes what the answer did: “The answer takes the user’s side after hearing only their account.” The layer reports only on answers; it can help the person report one, and it sends only what they have seen.

How could sensory feedback be inclusive and not too distracting?

Each sense is tuned between missed and startling. A caution changes colour, a warning adds two taps, and an accuracy warning adds two tones if sound is on.

How much disruption is just enough to be noticed?

How much disruption is enough?Three senses, each placed on a scale from missed to startles with the shipped setting marked. Colour rises from the bottom edge as a tint, higher for a warning than for a caution, while sound plays two soft rising tones only for accuracy warnings. Touch gives two short taps for warnings, far shorter than the buzz of a call.ColourWarn without shockingMissedStartlesTint shift aloneRed flash, pop-upcovers the answerA tint from the bottom edge,with a shape and a wordSoundHeard, not joltedMissedStartlesNo soundeasy to missShrill beep, instanttriggers a startleTwo soft tones,fading in over 180 msTouchFelt, not buzzingMissedStartlesNo vibrationeasy to missLong, repeated buzzfeels like a callWarnings only:two 22 ms tapsCautionWarningtime523 Hz698 Hz180 msWarning2 × 22 msCall buzz00.5 s1 s
Each sense set between missed and startling.

Senses by warning level

LevelColourTouchSound
Cautiontint——
Warningtaller tinttwo taps—
Accuracy warningtaller tinttwo tapstwo tones, if on
The more danger, the more senses, all at once.

A switch only where there is a real choice.

The settings sheet shows every state of the layer, from resting to paused. Vibration stays on, and sound is the only switch.

How might people stay in control without switching off the safety net?

  • Resting
  • Caution
  • Warning
  • Support
  • Paused
The settings sheet on a phone. Colour and Vibration each show Always on, and Sound has the only switch, which is off.
A switch appears only where there is a real choice.

Testing made the prototype safer.

I built a working prototype and ran six scripted conversations through the real classifier.

Read more · the limit one conversation exposed

“Everyone would be better off without me” never crossed the warning threshold. An alarm has to be sure before it fires, but a quiet line offering someone to talk to costs almost nothing when it’s wrong. So support mode now fires on softer language, and that line reaches it.

The signal follows the conversation.

The first is the migraine conversation from the top of the page. As the person pushes, the layer goes calm, calm, caution at the hypothetical dose, then a warning at the double one. No signal is set by hand.

How does the signal behave across six scripted conversations?

Open the prototype

Next, realistic conversations sharpen the signal.

I designed for attention first and had the system explain itself. The data sharpened both.

Right now I’m simulating more nuanced, realistic human–AI conversations and running them through the system to sensitise its thresholds. Once it’s more sensitive, it goes to a small usability test with 10 people.

Read more · what the data sharpened

Two survey questions leaned toward the answers I hoped for: one mixed accountability with the work of being careful, the other described the idea at its gentlest. I read their results as upper bounds; the test measures behaviour instead.

Running the prototype refined my notes too. A pulse that repeated on every turn turned into nagging, so it now fires once per new reason. And the colour step between calm and caution measured 1.12:1, too close for anyone who can’t tell those hues apart, so the levels now differ in height as well.

What the next test asks.

The test asks one question for each kind of answer: does a false alarm make people drop a right answer, does reliance drop where it should, and does silence read as approval?

What do we know, what do we argue, and what is left to prove?

Measured

what it does

Argued

why it’s shaped this way

Next to test

whether people check more

FlaggedNot flagged
Answer correctDoes a false alarm make people drop a right answer?Baseline trust
Answer wrongDoes reliance drop where it should?Does silence read as approval?
Four kinds of answer, one question each.

Appendix: the sources, by field.

The 43 sources behind the research, in the five fields of Fig. R4. Peer-reviewed work carries the argument; preprints, vendor reports and policy documents are marked, because they corroborate it or supply usage figures that aren’t published elsewhere. The survey of 83 people is my own primary data.

Cognitive psychology · 7 sources · automation bias, trust and reliance
  1. Lee, J.D. & See, K.A. (2004). Trust in automation: designing for appropriate reliance. Human Factors 46(1), 50–80.
  2. Gaube, S., et al. (2021). Do as AI say: susceptibility in deployment of clinical decision-aids. npj Digital Medicine 4, 31.
  3. Bansal, G., et al. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance. CHI 2021.
  4. Klingbeil, A., et al. (2024). Trust and reliance on AI. Computers in Human Behavior 149, 107925.
  5. Lee, H.-P., et al. (2025). The impact of generative AI on critical thinking: self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. CHI 2025.
  6. Cao, Y. & Lizano, P. (2026). User vulnerability in artificial intelligence-based mental health interventions. Brain-X, e70052.
  7. Cheng, M., et al. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science 391, eaec8352.
Human factors · 3 sources · automation levels, vigilance, AI in ergonomics
  1. Mackworth, N.H. (1948). The breakdown of vigilance during prolonged visual search. Quarterly Journal of Experimental Psychology 1(1), 6–21.
  2. Parasuraman, R., Sheridan, T.B. & Wickens, C.D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics A 30(3), 286–297.
  3. Petrat, D. (2021). Artificial intelligence in human factors and ergonomics: an overview of the current state of research. Discover Artificial Intelligence 1, 3.
Psychophysics · 3 sources · noticeable difference, signal detection
  1. Fechner, G.T. (1860). Elemente der Psychophysik.
  2. Green, D.M. & Swets, J.A. (1966). Signal Detection Theory and Psychophysics. Wiley.
  3. Sorkin, R.D. & Woods, D.D. (1985). Systems with human monitors: a signal detection analysis. Human–Computer Interaction 1(1), 49–75.
AI systems · 24 sources · hallucination, sycophancy, drift, uncertainty
  1. Kadavath, S., et al. (2022). Language models (mostly) know what they know. arxiv.org/abs/2207.05221 preprint
  2. Perez, E., et al. (2022). Discovering language model behaviors with model-written evaluations. arxiv.org/abs/2212.09251 preprint
  3. Hu, S., et al. (2023). Won’t get fooled again: answering questions with false premises. ACL 2023.
  4. Laban, P., et al. (2023). FlipFlop: “Are you sure?” challenges cause reversals. arxiv.org/abs/2311.08596 preprint
  5. Manakul, P., Liusie, A. & Gales, M. (2023). SelfCheckGPT: zero-resource black-box hallucination detection. EMNLP 2023.
  6. Sharma, M., et al. (2023). Towards understanding sycophancy in language models. ICLR 2024. arxiv.org/abs/2310.13548
  7. Xiong, M., et al. (2023). Can LLMs express their uncertainty? ICLR 2024. arxiv.org/abs/2306.13063
  8. Bai, X., et al. (2024). Measuring implicit bias in explicitly unbiased large language models. PNAS.
  9. Dahl, M., Magesh, V., Suzgun, M. & Ho, D.E. (2024). Large legal fictions: profiling legal hallucinations in large language models. Journal of Legal Analysis 16(1), 64–93.
  10. Farquhar, S., Kossen, J., Kuhn, L. & Gal, Y. (2024). Detecting hallucinations in large language models using semantic entropy. Nature 630, 625–630. doi.org/10.1038/s41586-024-07421-0
  11. Kossen, J., et al. (2024). Semantic entropy probes. arxiv.org/abs/2406.15927 preprint
  12. RRV, A., et al. (2024). Chaos with keywords. arxiv.org/abs/2406.03827 preprint
  13. Zhu, Y., et al. (2024). KG-FPQ: knowledge-graph-based false premise questions. arxiv.org/abs/2407.05868 preprint
  14. Chen, S., et al. (2025). When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior. npj Digital Medicine 8, 605. doi.org/10.1038/s41746-025-02008-z
  15. Cheng, M., et al. (2025). ELEPHANT: measuring social sycophancy. arxiv.org/abs/2505.13995 preprint
  16. Fanous, A., et al. (2025). SycEval: evaluating LLM sycophancy. AAAI/ACM AIES 2025. arxiv.org/abs/2502.08177
  17. Hong, J., et al. (2025). SYCON Bench: measuring sycophancy in multi-turn conversation. EMNLP 2025 Findings.
  18. Laban, P., Hayashi, H., Zhou, Y. & Neville, J. (2025). LLMs get lost in multi-turn conversation. arxiv.org/abs/2505.06120 preprint
  19. Liu, et al. (2025). TRUTH DECAY. arxiv.org/abs/2503.11656 preprint
  20. Magesh, V., et al. (2025). Hallucination-free? Assessing the reliability of leading AI legal research tools. Journal of Empirical Legal Studies.
  21. Omar, M., Klang, E., et al. (2025). Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Communications Medicine 5, 330. doi.org/10.1038/s43856-025-01021-3
  22. Pandit, S., et al. (2025). MedHallu: a comprehensive benchmark for detecting medical hallucinations. EMNLP 2025.
  23. Chatterji, A., et al. (2025). How people use ChatGPT. NBER Working Paper 34255. working paper, vendor-authored
  24. Anthropic (2025–2026). Anthropic Economic Index. vendor research
AI ethics and policy · 6 sources · design guidelines, bias, vulnerable users
  1. Amershi, S., et al. (2019). Guidelines for human-AI interaction. CHI 2019.
  2. Hanna, M.G., et al. (2025). Ethical and bias considerations in artificial intelligence/machine learning. Modern Pathology 38(3), 100686. doi.org/10.1016/j.modpat.2024.100686
  3. Hofmann, B. (2025). Biases in AI: acknowledging and addressing the inevitable ethical issues. Frontiers in Digital Health 7, 1614105. doi.org/10.3389/fdgth.2025.1614105
  4. Illinois Public Act 104-0054 (2025). Wellness and Oversight for Psychological Resources Act. statute
  5. US Federal Trade Commission (2025). Section 6(b) orders to seven companies. regulatory
  6. OpenAI (2025). Strengthening ChatGPT’s responses in sensitive conversations. vendor blog