The Emulation Test: Can This Shell Add Two Numbers?

Do You Know You're a Honeypot? — Part 2

Classification: MEDIUM · Confidence: HIGH (first-party command capture)

The previous letter's honeypot checks all shared a weakness: they looked for the trap by name, and a name is the cheapest thing in the world to change. Rename the default user, move the install directory, and the entire roll-call comes back empty, having learned nothing. This letter is about the operators who do not rely on names — who ask instead whether the shell they landed in can behave like a real one. That question cannot be answered away by renaming a file, because it does not probe what the environment is called. It probes what the environment can do, and a thing that only pretends to be a shell fails questions a real shell answers without thinking. The simplest of those questions is the one this letter is named for: can you add two numbers?

The same discipline as the previous letter applies. What follows describes the tests attackers run — long-documented techniques for distinguishing a real shell from an emulated one — and deliberately does not state how any particular sensor responds to them. The subject is the probe, not the answer.

echo $((1337+1337))

The specimen is a single line, and it is famous: echo $((1337+1337)). To a real shell this is trivial. Bash implements arithmetic expansion — the $(( )) construct — as a core feature, so a genuine shell evaluates the expression and prints 2674. To a certain kind of honeypot it is not trivial at all. A medium-interaction honeypot does not run a real shell; it emulates one, recognizing commands and returning canned or synthesized responses. If its emulator implements echo by printing back the argument it was given but does not implement arithmetic expansion, then echo $((1337+1337)) prints the literal string $((1337+1337)) — or an error — rather than 2674. One line, and the answer discriminates a real shell from a shallow imitation of one.

The choice of 1337 is not arbitrary either; it is craft. It is a distinctive, memorable number — "leet" — whose doubling, 2674, appears nowhere else a checker might confuse it with, so the verdict is unmistakable in a stream of output: either the exact string 2674 is present, meaning the shell truly evaluated the expression, or it is not, meaning the shell only pretended to. The operator has picked an input whose correct answer is a fingerprint of correctness itself. This is the single most economical honeypot test in the corpus: seven characters of arithmetic that ask the environment to prove it is a computer and not a tape recording.

Why behavior cannot be renamed away

The deep reason this tier of check matters more than the name checks is a difference in kind between metadata and capability. The name of a user, the path of a directory, the string a process reports — these are metadata: labels the honeypot attaches to itself and can reassign at will, at no cost, in a second. Arithmetic evaluation is not metadata. It is a capability — a thing the shell must actually be able to perform. And here is the crux: you cannot fake a capability by relabeling. The only way to make an emulated shell correctly answer echo $((1337+1337)) is to implement arithmetic expansion in the emulator — to actually build the feature. Faking the behavior and having the behavior are the same thing.

That is why behavioral detection is a fundamentally stronger technique than signature detection, and why it changes the game for the defender. A name check pressures the honeypot to be anonymous — to hide its tells. A behavior check pressures the honeypot to be real — to actually possess the capabilities a real shell has. The first is cheap to satisfy: rename things. The second is expensive, because satisfying it means implementing, one at a time, every behavior an attacker might test, until the emulator has been pushed so far toward completeness that it is approaching a real shell anyway. The behavioral test does not ask the honeypot to hide better. It asks the honeypot to be better, and being better has no shortcut.

The liveness conjunction

A second specimen tests capability in aggregate rather than one feature at a time: uname -m && df / >/dev/null 2>&1 && which systemctl && echo GOOD. Read the logic. It runs uname -m, and only if that succeeds runs df /, and only if that succeeds checks that systemctl exists, and only if all three succeed prints GOOD. It is a conjunction — a chain of && — so the single word GOOD at the end is a summary verdict: every one of these basic tools is present and working here.

A real machine passes trivially; uname, df, and systemctl are unremarkable and all work. A shallow environment — an emulator that implements some commands but not others, a stripped sandbox, a container missing systemctl — fails one link in the chain, the conjunction short-circuits, and GOOD never appears. The attacker gets one bit that answers "are the fundamentals all real here?" without having to read and interpret four separate outputs. It is the behavioral analogue of the roll-call's FOUND/ABSENT: a compound test compressed to a single legible token, built for an automated caller to branch on. And it is harder to fake than the roll-call, because passing it requires not the absence of honeypot names but the presence of working system tools — again, a capability, not a label.

Reading the error, not the answer

A third technique probes services the way the arch-oracle's ===SHELL_BEHAVIOR=== clause probed the shell: by asking a question whose failure is diagnostic. sshd -V is the example. The SSH daemon has no -V option; a real sshd, invoked that way, prints a specific usage error to standard error, in a specific format, with a specific exit code. An emulated environment that does not faithfully reproduce the real daemon's error behavior for an unknown flag responds differently — a generic message, the wrong message, or the honeypot's tell-tale SSHD NOT_FOUND. As with running a command known not to exist, the attacker is not interested in success. It is interested in whether the failure has the exact shape a real system's failure would have.

This is the subtlest form of the behavioral test, and the hardest to fake, because it requires the emulator to be correct not only about what real tools do but about how they break. A convincing success is achievable by implementing the happy path; a convincing failure requires implementing the error paths too — the usage messages, the exit codes, the exact wording of the complaint a real binary makes about a flag it does not recognize. Almost no emulator bothers to get the error behavior of every tool exactly right for every malformed invocation, because the error paths are numerous, unglamorous, and invisible to ordinary use. The attacker who probes them is exploiting the gap between implementing what a tool does and implementing what it does when misused — and that gap is where emulation is thinnest.

Running a real tool is itself a test

There is a fourth, less obvious behavioral probe hiding in plain sight, and it connects this series back to the reconnaissance-layer letter. One captured command opens with version=7.1.0; bash_version=${BASH_VERSINFO[0]:-5}; shopt -s eval_unsafe_arith; … read -rd '' config <<'EOF'; print_info() { … } — the preamble of neofetch, the system-information tool the recon letter found weaponized for hardware appraisal. But that preamble doubles as a shell-realness test, whether or not the operator intended it to, because running neofetch at all exercises a dense stack of advanced bash features: parameter defaults (${VAR:-default}), shopt option-setting, a quoted heredoc (<<'EOF'), function definitions, arrays, and arithmetic. An emulator that does not implement all of them cannot run the tool to completion; it chokes partway, or produces garbage, and the operator learns — from the failure to produce a clean system-info banner — that the shell is not real.

This is a category worth naming: the complex-tool-as-probe. You do not need a dedicated honeypot test if you simply run a real, feature-hungry program and check whether it works. A tool that exercises enough of the shell becomes, for free, a realness test, because completing it requires the shell to genuinely be one. So the neofetch drop the recon letter read as hardware appraisal is, simultaneously, a behavioral honeypot check: an operator who gets back a well-formed neofetch banner has learned both what the hardware is and that the shell running the tool is real enough to run it. Two questions, one command — which is exactly the efficiency a productized post-exploitation kit rewards, and a reminder that the same artifact can serve reconnaissance and evasion at once.

The endless ladder

Where the name check is a one-time exchange — the defender renames phil, and that specific check is dead forever — the behavioral check is an endless ladder, and that is its defining property. When echo $((1337+1337)) became a known honeypot tell, the emulators that cared implemented arithmetic expansion and began returning 2674. So the checkers moved to a harder expression the emulators had not yet implemented — nested expansions, parameter substitutions, process substitutions, the long tail of shell features. The emulators implemented those too. The checkers moved again. There is always a deeper feature the emulator has not yet built, because a real shell's behavior is effectively infinite and an emulator's is finite, and the frontier between them is exactly where the next test lives.

This is the arms race the whole series circles, seen at its purest. A signature race has an end: there are only so many default artifacts, and once they are all renamed, the signature check is exhausted. A behavioral race has no end, because the defender can never finish implementing "a real shell" — every feature added closes one test and reveals the next, and the only way to win the behavioral race completely is to stop emulating and run an actual shell. The behavioral test does not merely pressure the honeypot to improve; it pressures it toward a specific, expensive destination: abandon emulation entirely, and be a real, contained system. Which is precisely the choice the next rung of the ladder — the tests that ask not whether the shell is real but whether the machine is — forces all the way.

The distinction the tests exploit: emulation has a bottom

Underneath every probe in this letter is one structural fact about honeypots that the attackers are exploiting: the difference between a medium-interaction honeypot, which emulates, and a high-interaction one, which is a real system placed under containment and observation. Emulation is cheaper, safer, and easier to deploy — the honeypot cannot actually be used to attack others because it cannot actually run anything — but it has a bottom: it implements a finite repertoire of behaviors, and every behavioral test is a probe for the edge of that repertoire. A real contained system has no such bottom; it does arithmetic and runs systemctl and errors correctly on sshd -V because it genuinely is a system, not a model of one.

The behavioral tests, then, are really a single question asked many ways: am I in a model, or in the thing itself? And the honeypot operator's defense against them is the uncomfortable one the ladder keeps pointing at — that the only complete answer to "prove you are not a model" is to not be a model, to pay the cost and containment burden of a real system so that there is no repertoire to reach the edge of. Every behavioral probe is the attacker pressing on that choice, testing whether the defender took the cheap emulated path and can therefore be caught at the bottom of its repertoire, or the expensive real path and cannot. The arithmetic question is small; the choice it probes is the largest one a honeypot operator makes.

The realism paradox

The behavioral ladder leads somewhere philosophically uncomfortable for the defender, and it is worth stating because it bounds how good any honeypot can be. If the only way to pass every behavioral test is to actually possess every behavior a real system has, then a honeypot that perfectly resists behavioral detection is indistinguishable from a real vulnerable machine — and a machine indistinguishable from a real vulnerable machine mostly is one. The logical endpoint of winning the behavioral race is to offer the attacker something almost exactly as capable, and therefore almost exactly as dangerous, as the real target they were hunting. Perfect realism and perfect safety pull in opposite directions.

That is the paradox at the heart of high-interaction honeypots. To catch the behavior of sophisticated attackers, you must let them operate in an environment convincing enough to fool their behavioral checks; but an environment convincing enough to fool their checks is, by construction, an environment real enough to run their payloads — to mine, to scan, to spread — which is the very outcome the containment exists to prevent. The honeypot operator is therefore always trading realism against risk: every behavior added to defeat a detection test is a genuine capability handed to whoever is standing in the trap, and the operator's containment has to be good enough to allow the capability while denying the harm. The arithmetic test looks trivial — add two numbers — but the ladder it stands on climbs to this: the better a honeypot answers "prove you are real," the more real, and the more dangerous, it has to actually become. There is no free realism. Every convincing behavior is a real one, and a real capability in an attacker's hands is exactly what the honeypot was built to study and to contain at the same time — a tension the behavioral testers are, with each new probe, quietly tightening.

Defense: the probes are a conformance suite

The gift here is even more direct than in the name-check letter. Each behavioral probe is not just a tell to eliminate; it is a specification of a behavior the emulator must implement to pass. echo $((1337+1337)) specifies "implement arithmetic expansion." The liveness conjunction specifies "have these tools present and working." The sshd -V probe specifies "reproduce the real daemon's error behavior for unknown flags." Collected, the attackers' behavioral probes form a conformance test suite for a honeypot's shell emulation — written, refined, and prioritized by the people most motivated to find where the emulation breaks.

A honeypot operator who runs these probes against their own sensor learns exactly where its emulation falls short of a real shell, in the order attackers actually test, and can close each gap or make the deliberate decision that the gap is acceptable. And the operator who watches attackers run these probes learns which behaviors the current threat population checks — a live, evolving conformance suite that updates itself every time an operator invents a new test. The behavioral race has no permanent victory, but it has a permanent asset for whichever side reads the other's probes, and the honeypot, by definition, reads them all. The attacker asks the shell to add two numbers to see if it is real; the defender who overhears the question learns that arithmetic is now table stakes, and moves on to the harder features before the attackers do.

Bounds and limits

DOCUMENTED, first-party: the arithmetic probe echo $((1337+1337)); the liveness conjunction uname -m && df / >/dev/null 2>&1 && which systemctl && echo GOOD; the sshd -V error-behavior probe; and bash-feature probes (BASH_VERSINFO, shopt -s eval_unsafe_arith, nocasematch) — all captured in the command log.

INFERRED, high confidence: that these are behavioral honeypot/sandbox-detection tests (running an arithmetic expression whose answer the operator already knows, chaining basic-tool checks into a single GOOD verdict, and probing a service with a flag it does not have are behaviors with no operational purpose except to test whether the environment is a real shell); that the arithmetic test specifically targets the emulation gap that $(( )) historically exposed in medium-interaction honeypots. INFERRED, general: that behavioral detection is stronger than signature detection because it probes capability rather than metadata (faking a capability requires implementing it).

BOUNDED — deliberately: this letter does not state how the capturing sensor responded to any of these probes; whether a given honeypot passes or fails echo $((1337+1337)) is exactly the emulation detail the test is designed to extract, and there is no reason to publish it. The historical fact that arithmetic expansion once distinguished emulated shells is stated about emulation in general, not about any specific deployment. And the "endless ladder" is a structural claim about emulation versus reality, not a measurement of how far up the ladder the current attacker population has climbed — the corpus shows the tests being run, not the frontier's exact height.

None of these bounds change the finding. A tier of attacker does not ask what the trap is named; it asks what the trap can do, with tests a rename cannot defeat — the arithmetic a real shell evaluates and a shallow emulator echoes, the tool-chain a real machine passes and a stripped one fails, the error a real daemon makes and a model gets wrong. Each is a probe for the bottom of the emulation's repertoire, and each pushes the honeypot toward the same expensive destination: stop modeling a system and be one. The signature race ends when the names are changed; the behavioral race ends only when the model becomes the machine. The next letter follows the checks up their final rung — from "is this shell real?" to "is this a real, valuable machine, or an empty room built to hold me?"

This is Part Two of Do You Know You're a Honeypot? Part One found attackers checking the trap by name; this part found them checking whether it can behave like a real shell — a test no rename defeats and no finite emulator finally passes. Part Three climbs to the last rung: the checks that probe the hardware beneath the shell and the emptiness of a machine with no real secrets to steal.