extra/esperanto/README.md
A DBMS-agnostic blind-extraction engine that speaks one "esperanto" at any SQL back-end, driven by nothing but a boolean oracle.
Ordinary blind SQL injection retrieval needs to know the back-end first: sqlmap fingerprints the
DBMS, loads that dialect's queries.xml row, and extracts with it. When the DBMS cannot be
identified - an unknown or heavily-firewalled target, an exotic engine, a fork that answers unlike
its parent - that pipeline dead-ends.
Esperanto removes the prerequisite. Given only a boolean oracle - a callable
oracle(condition) -> bool that reports whether an arbitrary SQL boolean expression holds at the
target - it discovers the dialect from scratch (string concatenation, substring / length /
character-code functions, the usable comparison operator, the catalog surface) and then extracts
data character-by-character, without ever being told which DBMS is on the other end.
The candidate SQL forms it probes are distilled from sqlmap's own data/xml/queries.xml across all
supported DBMSes: rather than fingerprint-then-load-one-dialect, it reuses that accumulated knowledge
as a single language it tries to speak at any backend. The engine is self-contained (no sqlmap
imports) and runs unmodified on Python 2.7 and Python 3.
Inside sqlmap, via the --esperanto switch. sqlmap detects the boolean-blind injection as usual,
then hands the engine its own oracle (checkBooleanExpression, which rides the full request /
comparison / WAF stack) instead of fingerprinting. The normal enumeration switches then work against
an unidentified back-end:
python sqlmap.py -u 'http://host/vuln?id=1' --esperanto --banner --tables --dump -T users
Standalone, straight from this directory (no sqlmap, no PYTHONPATH):
python run.py -u 'http://host/vuln?id=1*' --tables --dump -T users
python run.py -u 'http://host/vuln?id=1*' --string <true-marker> --dump -T users -C uname,pass
python run.py --self-test
The * (or an explicit [INFERENCE]) marks the injection point; without one it defaults to the end
of the URL. The true/false oracle is taken from --string / --code, or
auto-calibrated from the response when none is given.
Discovery is a set of capability ladders. For each primitive it tries the known SQL forms in turn and keeps the first that a small set of probes proves correct, so a backend that lacks - or a WAF that filters - one form falls through to the next rather than failing:
| Primitive | Ladder (first that works wins) |
|---|---|
| Concatenation | ||, CONCAT(), +, & |
| Substring | SUBSTR / SUBSTRING / MID, else LEFT/RIGHT composition |
| Length | a length function, else derived from the substring's end |
| Character read | code point (ASCII/UNICODE/ORD/...), collation-forced byte order, hex, ordinal, = scan |
| Comparison | >, else BETWEEN, else an operator-free ordered form (SIGN/ABS/LEAST/...), else order-free IN() |
| No-substring floor | LIKE / GLOB / SIMILAR TO pattern matching |
Each candidate is accepted only against a semantic check (for example, the comparison ladder is
validated across a signed truth table, not two positive samples), so a rewriting layer that turns
> into >= is rejected rather than silently mis-reading every count and length.
Discovery produces a Dialect; identify() then fuses catalog family, required dual-table and
version-banner evidence into a best-guess product. Enumeration walks catalogs by keyset
(MIN(name) WHERE name > prev) with no dialect row-limiter, and dump() reconstructs rows by a
discovered primary/unique key, else a physical row-id, else the row's own value.
from extra.esperanto import Esperanto
esp = Esperanto(oracle) # oracle(condition_str) -> bool
esp.discover() # ladder out the dialect
esp.identify() # best-guess product + evidence
esp.extract("(SELECT ...)") # one scalar, char-by-char
esp.dump("users", schema="app") # {columns, rows, complete, exact, keyed_by}
esp.enumerate("table") # keyset catalog walk
True / False for an observed result; a
transport or observation failure must raise (or return a non-boolean), which the engine treats as
undecided rather than a definitive False. A wrong-dialect or unsupported probe is expected to be
reported as False by the oracle itself.buildHandler() is the sqlmap adapter (--esperanto), a dbmsHandler that maps sqlmap's
enumeration calls onto the engine. sqlmap-core imports are deferred inside it so the rest of the
package stays dependency-free.strategy() freezes the discovered dialect into an immutable InferenceStrategy of pure render_*
methods (no oracle, no loop), and hostExtract() is a reference host loop that extracts using only
a strategy plus an oracle.Every recovered value carries an Integrity classification so a caller can tell "we visited every
position" apart from "the bytes are provably the source's":
| State | Meaning |
|---|---|
EXACT | proven byte-identical to the source (or a proven NULL) |
WHOLE_BUT_AMBIGUOUS | every position read, but case/accent is uncertain (collation-dependent comparison, no byte-exact primitive) |
TRUNCATED | a bounded prefix; the source continues |
UNRESOLVED | a character could not be recovered |
FAILED | could not observe or verify |
The engine is designed to fail closed: it never manufactures a False from an unobservable probe,
never silently substitutes or drops a character, distinguishes NULL from empty string, and reports
a dump along two independent axes - complete (coverage: every row) and exact (content: the values
are byte-faithful). EXACT is claimed only when a byte-faithful witness backs the value.
This is a last-resort engine for targets a normal fingerprint cannot reach, not a replacement for sqlmap's native retrieval. Known boundaries:
database.schema.table via a
sys.schemas join) is not yet modelled, so schema scoping on those engines is best-effort.latin1 is Windows-1252, Oracle WE8MSWIN1252 too),
resolving the common cases definitively. A column whose character set differs from the database
default, or a set outside the mapped list, decodes best-effort and is marked non-exact rather
than asserted.InferenceStrategy / hostExtract() drive the character
comparison modes under an ordered comparator; they do not yet drive the pattern-only or
membership-only modes, and the frozen field set does not yet carry every discovered semantic.Where a value cannot be recovered exactly, the engine surfaces its Integrity rather than presenting
it as trustworthy.
python run.py --self-test
runs the engine against an in-memory SQLite oracle across every compare mode plus identification,
byte extraction, noisy-oracle quorum voting, the fail-closed integrity invariants and the frozen
strategy hand-off. The regression corpus lives in tests/test_esperanto.py.
Part of sqlmap. See the top-level LICENSE for copying permission.