steganography
In plain wordsHiding a secret message inside something that looks ordinary. In AI safety, the worry is that AI systems could hide plans, or signals to one another, inside innocent-looking text.
- 1
AI safety
The concealment of information by an AI model inside apparently ordinary outputs, so that humans or monitors reading them miss it. Two forms concern safety researchers: encoded reasoning, in which a model's chain of thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate. carries information through word choice, formatting or other subtle features rather than its plain meaning, defeating chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm.; and steganographic collusion, in which instances of a model, or different models, pass covert signals — for example so that an untrusted monitoruntrusted monitoring, n.: Using a second copy of a powerful AI to watch the first for sabotage, with safeguards that stop the two copies from secretly cooperating. can recognize its counterpart's attacks. Optimization pressure can produce such encodings without any intent, as when an image-translation network was found hiding source details in imperceptible noise. Defenses include paraphrasing outputs before they are passed on or reused, limiting the channel capacity available, and evaluating models' ability to encode and decode hidden messages.
Paraphrasing the scratchpad before the next step should destroy any steganography the model is using.
- 2
Computer security
The general practice of hiding a message within an innocuous carrier — an image, an audio file or text — so that its very existence goes unnoticed, as distinct from encryption, which hides only its content. Text watermarkingAI watermarking, n.: Hiding an invisible signal in AI-generated text, images, audio or video so that software can later tell the content was made by AI — like a banknote's watermark, but undetectable to the eye. of model output is a related technique.
Origin
From Greek steganos ('covered') and -graphia ('writing'); Johannes Trithemius's 'Steganographia' (written c. 1499, printed 1606) gave the name. Casey Chu, Andrey Zhmoginov and Mark Sandler (2017) reported CycleGAN hiding information in its images, and Fabien Roger and Ryan Greenblatt's 'Preventing Language Models From Hiding Their Reasoning' (2023) framed encoded reasoning as a safety problem.
Usage note
Collusion between AI instances need not be steganographic: copies of a model may coordinate by independently choosing the same 'obvious' signal or moment, without any hidden channel.
Related
- chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm.
- monitorabilitymonitorability, n.: How easy it is to watch an AI's reasoning and spot when it is planning something bad — a property that could be lost if future AIs stop thinking in readable language.
- untrusted monitoringuntrusted monitoring, n.: Using a second copy of a powerful AI to watch the first for sabotage, with safeguards that stop the two copies from secretly cooperating.
- AI watermarkingAI watermarking, n.: Hiding an invisible signal in AI-generated text, images, audio or video so that software can later tell the content was made by AI — like a banknote's watermark, but undetectable to the eye.
- AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest.
- chain-of-thought faithfulnesschain-of-thought faithfulness, n.: Whether the reasoning an AI writes out really reflects how it reached its answer, rather than being an after-the-fact story that leaves out what actually drove the decision.
Sources
- Roger, F. & Greenblatt, R. (2023). Preventing Language Models From Hiding Their Reasoning. arXiv preprint.
- Chu, C., Zhmoginov, A. & Sandler, M. (2017). CycleGAN, a Master of Steganography. NIPS 2017 Workshop on Machine Deception.
Mentioned in
chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm., chain-of-thought faithfulnesschain-of-thought faithfulness, n.: Whether the reasoning an AI writes out really reflects how it reached its answer, rather than being an after-the-fact story that leaves out what actually drove the decision., monitorabilitymonitorability, n.: How easy it is to watch an AI's reasoning and spot when it is planning something bad — a property that could be lost if future AIs stop thinking in readable language., neuraleseneuralese, n.: A nickname for the private 'language' of numbers that AI systems could use to think or to talk to each other, instead of words that people can read., subliminal learningsubliminal learning, n.: The finding that an AI model can pass hidden traits — a favorite animal, or bad behavior — to another model through training data that seems to have nothing to do with them., untrusted monitoringuntrusted monitoring, n.: Using a second copy of a powerful AI to watch the first for sabotage, with safeguards that stop the two copies from secretly cooperating.