docs/superpowers/plans/2026-08-11-user-variable-literal-tracking.md
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Add opt-in, ParserSQL-backed tracking and replay of literal MySQL user-defined-variable assignments so supported SET @name = literal statements remain multiplexable without leaking state between frontend sessions.
Architecture: ParserSQL supplies lossless user-variable and literal AST nodes, full-input coverage, and read/write usage classification. ProxySQL keeps bounded, symmetric user-variable maps on frontend and backend MySQL_Connection objects, stages client assignments until the original backend SET succeeds, includes the maps in pool selection, and replays deterministic SET batches after ordinary session-variable synchronization. Anything outside the proven subset follows the current connection-bound fallback.
Tech Stack: C++17, ParserSQL recursive-descent parser and GoogleTest, ProxySQL MySQL session state machine, MariaDB Connector/C async APIs, GNU Make, TAP, nlohmann JSON, SpookyHash, OpenSSL RAND_bytes, Prometheus counters.
mysql-user_variable_tracking, default 0, with the initial accepted range 0..1. Do not make it a boolean; later integer modes are reserved for different semantics.1 accepts new assignments only when mysql-set_parser_algorithm=3 or mysql-query_processor_parser=1. Existing tracked sessions drain safely when either the feature or its parser prerequisite is disabled at runtime.sql_mode, character_set_client, character_set_connection, collation_connection, SET NAMES, or SET CHARACTER SET, synchronize the old state first and take the existing connection-bound fallback before applying that context change. Do the same if backend session tracking reports one of those context changes. This preserves value/type/charset/collation correctness without evaluating literals in ProxySQL.PROXYSQL INTERNAL SESSION. Diagnostics expose only counts, byte totals, and one process-keyed aggregate fingerprint.doc/agents/common-mistakes.md: validate the changed ParserSQL through the real ProxySQL adapter and the legacy SET-parser comparison tests before creating the ParserSQL tag. Produce one ParserSQL release and one downstream version bump.blogs/ tree or docs/superpowers/plans/2026-08-11-gtid-from-ok-packets.md currently present in the ProxySQL working tree.Files:
include/sql_parser/token.hinclude/sql_parser/common.hinclude/sql_parser/tokenizer.hinclude/sql_parser/ast.hinclude/sql_parser/expression_parser.hinclude/sql_parser/set_parser.hinclude/sql_parser/parse_result.hinclude/sql_parser/parser.hinclude/sql_parser/user_variable.hsrc/sql_parser/parser.cpptests/test_tokenizer.cpptests/test_expression.cpptests/test_set.cpptests/test_user_variable.cppMakefileinclude/MySQL_User_Variables.hinclude/Query_Processor_ParserSQL.hlib/Query_Processor_ParserSQL.cpptest/tap/tests/unit/parsersql_unit-t.cpptest/tap/tests/setparser_parsersql_test.cppInterfaces:
Append these ParserSQL enum members at the end of their respective enums:
// TokenType, appended after every existing token.
TK_USER_VARIABLE,
TK_HEX_LITERAL,
TK_BIT_LITERAL,
// NodeType, appended after every existing node.
NODE_USER_VARIABLE,
NODE_LITERAL_HEX,
NODE_LITERAL_BIT,
Extend tokens and AST nodes with an exact source span while keeping text / value() semantics unchanged for existing emitters:
struct Token {
TokenType type = TokenType::TK_EOF;
StringRef text;
StringRef source;
uint32_t offset = 0;
};
struct AstNode {
AstNode* first_child;
AstNode* next_sibling;
const char* value_ptr;
const char* source_ptr;
uint32_t value_len;
uint32_t source_len;
NodeType type;
uint16_t flags;
StringRef value() const;
StringRef source() const;
void set_value(StringRef ref);
void set_source(StringRef ref);
void add_child(AstNode* child);
};
AstNode* make_node_from_token(
Arena& arena, NodeType type, const Token& token, uint16_t flags = 0);
static_assert(sizeof(AstNode) == 48, "AstNode layout changed unexpectedly");
Add strict parse metadata and the upstream usage API:
struct ParseResult {
enum Status : uint8_t { OK = 0, PARTIAL, ERROR };
Status status = ERROR;
StmtType stmt_type = StmtType::UNKNOWN;
AstNode* ast = nullptr;
ErrorInfo error;
StringRef remaining;
bool full_input = false;
bool has_user_variables = false;
// existing fields remain unchanged
};
enum class UserVariableUsage : uint8_t {
NO_USER_VARIABLE,
READ_ONLY,
UNSAFE_OR_UNKNOWN
};
UserVariableUsage classify_mysql_user_variable_usage(const ParseResult& result);
Define the downstream typed contract in include/MySQL_User_Variables.h exactly once so both the adapter and connection state can use it without a circular dependency:
enum class UserVariableSetStatus : uint8_t {
NOT_USER_VARIABLE_SET,
SUPPORTED,
UNSUPPORTED,
PARSE_ERROR
};
enum class UserVariableLiteralKind : uint8_t {
STRING,
INTEGER,
DECIMAL,
HEXADECIMAL,
BIT,
NULL_VALUE
};
enum class UserVariableUsage : uint8_t {
NO_USER_VARIABLE,
READ_ONLY,
UNSAFE_OR_UNKNOWN
};
struct UserVariableAssignment {
std::string canonical_name;
std::string replay_target;
std::string raw_literal;
UserVariableLiteralKind kind;
uint64_t hash;
};
struct UserVariableSetAnalysis {
UserVariableSetStatus status { UserVariableSetStatus::NOT_USER_VARIABLE_SET };
std::vector<UserVariableAssignment> assignments;
};
UserVariableSetAnalysis parsersql_analyze_user_variable_set_mysql(
const char* query, size_t query_length);
UserVariableUsage parsersql_classify_user_variable_usage_mysql(
const char* query, size_t query_length);
git -C /data/rene/ParserSQL fetch origin --tags
git -C /data/rene/ParserSQL worktree add \
/data/rene/ParserSQL-user-variable-literals \
-b feature/user-variable-literals v1.0.10
git -C /data/rene/ParserSQL-user-variable-literals status --short
Expected: the new ParserSQL worktree is clean and based on v1.0.10. Leave the unrelated untracked .claude/ and third_party/libpg_query/ in /data/rene/ParserSQL untouched.
Add focused GoogleTests covering all of these cases:
1, 1.25, .25, 1., 1e3, 1.2E-3, 0xCAFE, X'CAFE', 0b101, B'101', single/double-quoted strings, and NULL;@plain, @with.dot, @with$dollar, @'quoted-name', @"quoted-name", and @`quoted-name` represented as NODE_USER_VARIABLE, with decoded value and exact replayable source;+ and - retaining a NODE_UNARY_OP source span, while parentheses and operators remain expression nodes;full_input=true only for EOF or one optional trailing semicolon followed by EOF; trailing commas, tokens, or a second statement are false;NO_USER_VARIABLE for SELECT 1 and SELECT '@x', READ_ONLY for SELECT @x / supported predicates that read @x, and UNSAFE_OR_UNKNOWN for SET @x=1, SELECT @x:=1, SELECT id INTO @x FROM test.uv_source, calls/functions/placeholders/subqueries containing @x, malformed input, and incomplete parsing.Register tests/test_user_variable.cpp in TEST_SRCS, then run:
make -C /data/rene/ParserSQL-user-variable-literals test
Expected: new tests fail because the tokenizer lacks these literal/user-variable types, source spans, full-input status, and usage classification.
Implement these rules in ParserSQL:
TK_USER_VARIABLE consumes a whole MySQL user variable beginning at @, but @@ remains TK_DOUBLE_AT. Unquoted names accept alphanumeric bytes plus ., _, and $. Quoted names accept MySQL string/backtick forms and reject missing closing delimiters.@... bytes in source. SetParser / ExpressionParser decode unquoted names and doubled-delimiter quoted names into arena storage, enforce the 64-byte decoded-name limit, and build NODE_USER_VARIABLE with decoded value() plus exact source(). Backslash-containing quoted names retain lossless source but are not eligible for downstream tracking.scan_number() supports fixed and exponent forms without accepting a bare dot or incomplete exponent.0x... / 0b... and X'...' / B'...' receive distinct literal token and node types. Reject characters outside each base lexically; leave server-specific semantic validation to the backend.+ and - both produce NODE_UNARY_OP and span from the sign through the operand.Parser::parse() copies tokenizer_.has_user_variables() into the result. scan_to_end() sets full_input only when the grammar stopped at EOF or at a single trailing semicolon followed by EOF; it records the first unconsumed token in remaining otherwise.classify_mysql_user_variable_usage() traverses the AST with an explicit read-context allowlist. Any NODE_USER_VARIABLE below NODE_VAR_TARGET or NODE_INTO_CLAUSE is a write. A user-variable query containing NODE_FUNCTION_CALL, NODE_CALL_STMT, NODE_DO_STMT, NODE_PLACEHOLDER, NODE_SUBQUERY, or an unknown ancestor shape is unsafe. Direct SELECT items and ordinary unary/binary predicate expressions are read-only. A parse with has_user_variables but without OK, full_input, or an AST is unsafe.Run the upstream suite and grammar build:
make -C /data/rene/ParserSQL-user-variable-literals clean
make -C /data/rene/ParserSQL-user-variable-literals test
make -C /data/rene/ParserSQL-user-variable-literals build-corpus-test
Expected: all ParserSQL tests pass and corpus_test builds without warnings or enum-index regressions.
In parsersql_unit-t.cpp, test the typed API with exact assertions for:
UNSUPPORTED, while backslashes in RHS string literals remain supported because ProxySQL stores/replays their raw source without decoding;UNSUPPORTED and PARSE_ERROR;@ inside a string/comment.In setparser_parsersql_test.cpp, add regressions proving the existing lossy system-variable SET adapter still returns the same maps after the AST/source changes.
Build the upstream library, overlay only generated build products and changed headers into ProxySQL's ignored extracted dependency, then run the downstream tests:
make -C /data/rene/ParserSQL-user-variable-literals lib
cp -a /data/rene/ParserSQL-user-variable-literals/include/sql_parser/. \
deps/parsersql/parsersql/include/sql_parser/
cp /data/rene/ParserSQL-user-variable-literals/libsqlparser.a \
deps/parsersql/parsersql/libsqlparser.a
make -C lib clean
make -j4 debug
make -C test/tap/tests/unit parsersql_unit-t
./test/tap/tests/unit/parsersql_unit-t
make -C test/tap/tests setparser_parsersql_test setparser_test setparser_test2 setparser_test3
./test/tap/tests/setparser_parsersql_test
./test/tap/tests/setparser_test
./test/tap/tests/setparser_test2
./test/tap/tests/setparser_test3
Expected before adapter implementation: the ProxySQL unit test fails to compile because the typed functions do not exist.
Implement parsersql_analyze_user_variable_set_mysql() as an all-or-nothing AST walk:
PARSE_ERROR for non-OK or non-full input;NOT_USER_VARIABLE_SET for non-SET input or a SET without any user-variable target;UNSUPPORTED for mixed targets, malformed assignment shape, any RHS outside the approved literal/unary forms, or an invalid target;SUPPORTED only after every assignment is converted.Use the decoded NODE_USER_VARIABLE::value() for ASCII-lowercased canonical_name, the validated target source() for replay_target, and the literal/unary source() for raw_literal. Reject a quoted target source containing a backslash so canonical identity is independent of session SQL mode. Compute the fast entry hash over a length-delimited tuple of kind, target, and literal with SpookyHash::Hash64; retain exact strings for collision-safe comparison.
Map the upstream classifier enum one-for-one in parsersql_classify_user_variable_usage_mysql(). Reset the thread-local parser only after all source spans have been copied into owning std::string values.
Rerun every command from Step 4, plus:
git -C /data/rene/ParserSQL-user-variable-literals diff --check
git diff --check -- \
include/MySQL_User_Variables.h \
include/Query_Processor_ParserSQL.h \
lib/Query_Processor_ParserSQL.cpp \
test/tap/tests/unit/parsersql_unit-t.cpp \
test/tap/tests/setparser_parsersql_test.cpp
Expected: upstream tests, ProxySQL typed adapter tests, the ParserSQL SET comparison, and all three legacy SET parsers pass.
git -C /data/rene/ParserSQL-user-variable-literals add \
include/sql_parser/token.h \
include/sql_parser/common.h \
include/sql_parser/tokenizer.h \
include/sql_parser/ast.h \
include/sql_parser/expression_parser.h \
include/sql_parser/set_parser.h \
include/sql_parser/parse_result.h \
include/sql_parser/parser.h \
include/sql_parser/user_variable.h \
src/sql_parser/parser.cpp \
tests/test_tokenizer.cpp \
tests/test_expression.cpp \
tests/test_set.cpp \
tests/test_user_variable.cpp \
Makefile
git -C /data/rene/ParserSQL-user-variable-literals commit \
-m "feat: expose lossless MySQL user variables"
Leave the ProxySQL adapter/test changes uncommitted until the official archive is installed in Task 2; that keeps every ProxySQL commit buildable from a clean checkout.
Files:
Delete: deps/parsersql/parsersql-1.0.10.tar.gz
Create: deps/parsersql/parsersql-1.0.11.tar.gz
Modify symlink: deps/parsersql/parsersql
Modify: deps/parsersql/README.md
Include the already-tested downstream files from Task 1:
include/MySQL_User_Variables.h, include/Query_Processor_ParserSQL.h,
lib/Query_Processor_ParserSQL.cpp,
test/tap/tests/unit/parsersql_unit-t.cpp, and
test/tap/tests/setparser_parsersql_test.cpp
Step 1: Tag the validated upstream commit locally and create the canonical archive
git -C /data/rene/ParserSQL-user-variable-literals status --short
git -C /data/rene/ParserSQL-user-variable-literals tag -a v1.0.11 \
-m "ParserSQL 1.0.11"
git -C /data/rene/ParserSQL-user-variable-literals archive \
--format=tar.gz \
--prefix=ParserSQL-1.0.11/ \
v1.0.11 \
-o /tmp/parsersql-1.0.11.tar.gz
sha256sum /tmp/parsersql-1.0.11.tar.gz
Expected: upstream status is clean before tagging, and the archive is generated from the validated tag. Do not push the branch or tag unless the user separately authorizes that external write.
git rm deps/parsersql/parsersql-1.0.10.tar.gz
cp /tmp/parsersql-1.0.11.tar.gz deps/parsersql/parsersql-1.0.11.tar.gz
ln -sfn parsersql-1.0.11 deps/parsersql/parsersql
Update the audit history in deps/parsersql/README.md with one v1.0.11 entry describing lossless literal/user-variable nodes, full-input coverage, and usage classification.
make -C deps parsersql/parsersql/libsqlparser.a
make -C lib clean
make -j4 debug
make -C test/tap/tests/unit parsersql_unit-t
./test/tap/tests/unit/parsersql_unit-t
make -C test/tap/tests setparser_parsersql_test setparser_test setparser_test2 setparser_test3
./test/tap/tests/setparser_parsersql_test
./test/tap/tests/setparser_test
./test/tap/tests/setparser_test2
./test/tap/tests/setparser_test3
Expected: the official archive gives the same green result as the pre-tag overlay.
git add \
deps/parsersql/parsersql-1.0.11.tar.gz \
deps/parsersql/parsersql \
deps/parsersql/README.md \
include/MySQL_User_Variables.h \
include/Query_Processor_ParserSQL.h \
lib/Query_Processor_ParserSQL.cpp \
test/tap/tests/unit/parsersql_unit-t.cpp \
test/tap/tests/setparser_parsersql_test.cpp
git commit -m "vendor: add ParserSQL user-variable support"
Files:
include/MySQL_User_Variables.hlib/MySQL_User_Variables.cpplib/Makefiletest/tap/tests/unit/mysql_user_variables_unit-t.cpptest/tap/tests/unit/Makefiletest/tap/groups/groups.jsonInterfaces:
struct MySQL_User_Variable_Entry {
std::string replay_target;
std::string raw_literal;
UserVariableLiteralKind kind;
uint64_t hash;
size_t stored_bytes() const;
bool exactly_equals(const MySQL_User_Variable_Entry& other) const;
};
enum class MySQL_User_Variable_Apply_Result : uint8_t {
OK,
VARIABLE_LIMIT,
BYTE_LIMIT
};
struct MySQL_User_Variable_Replay_Batch {
std::string sql;
std::vector<UserVariableAssignment> assignments;
};
enum class MySQL_User_Variable_Replay_Status : uint8_t {
OK,
ASSIGNMENT_TOO_LARGE
};
struct MySQL_User_Variable_Replay_Plan {
MySQL_User_Variable_Replay_Status status;
std::vector<MySQL_User_Variable_Replay_Batch> batches;
};
class MySQL_User_Variable_State {
public:
static constexpr size_t kMaxVariables = 128;
static constexpr size_t kMaxStoredBytes = 64 * 1024;
MySQL_User_Variable_Apply_Result stage(
const std::vector<UserVariableAssignment>& assignments,
MySQL_User_Variable_State& staged) const;
void apply(const std::vector<UserVariableAssignment>& assignments);
void clear();
size_t size() const;
size_t stored_bytes() const;
bool has_names_absent_from(const MySQL_User_Variable_State& desired) const;
unsigned int count_matches(
const MySQL_User_Variable_State& desired,
unsigned int& not_matching) const;
MySQL_User_Variable_Replay_Plan build_replay_plan(
const MySQL_User_Variable_State& actual,
size_t max_query_bytes) const;
std::string diagnostic_fingerprint() const;
};
Register mysql_user_variables_unit-t in UNIT_TESTS and unit-tests-g1. Test:
hash; exactly_equals() and matching must still report a mismatch;max_query_bytes, raw target/literal preservation, already-equal entries skipped, and a single over-limit assignment returning ASSIGNMENT_TOO_LARGE without a partial plan;make -C test/tap/tests/unit mysql_user_variables_unit-t
Expected: compilation fails because the state class and implementation do not exist.
Use std::map<std::string, MySQL_User_Variable_Entry> for deterministic order. stage() must copy the current state, apply assignments sequentially to the copy, check distinct-name and stored-byte limits after replacement accounting, and leave both the caller and output unchanged on failure. apply() is only called after a successful stage() or successful backend command.
Replay SQL concatenates only ParserSQL-validated targets and raw literals, without injecting spaces into their syntax. For example:
SET @browser_lang='en-US',@browser_timezone='GMT+2'
Initialize two process-random 64-bit SpookyHash seeds once with RAND_bytes. Hash a deterministic length-delimited serialization of every canonical name, target, kind, and literal to produce the single aggregate diagnostic fingerprint. If RAND_bytes fails, log one warning and return an empty fingerprint so callers omit the field rather than expose an unkeyed value.
Add MySQL_User_Variables.oo to _OBJ_CXX and run:
make -C lib obj/MySQL_User_Variables.oo
make -C test/tap/tests/unit mysql_user_variables_unit-t
./test/tap/tests/unit/mysql_user_variables_unit-t
Expected: all state and replay-builder tests pass.
git add \
include/MySQL_User_Variables.h \
lib/MySQL_User_Variables.cpp \
lib/Makefile \
test/tap/tests/unit/mysql_user_variables_unit-t.cpp \
test/tap/tests/unit/Makefile \
test/tap/groups/groups.json
git commit -m "feat: add bounded MySQL user-variable state"
Files:
Modify: include/MySQL_Thread.h
Modify: include/proxysql_structs.h
Modify: lib/MySQL_Thread.cpp
Modify: lib/Admin_FlushVariables.cpp
Modify: test/tap/tests/unit/mysql_variables_unit-t.cpp
Step 1: Write failing variable-registration tests
Extend mysql_variables_unit-t.cpp to assert:
user_variable_tracking appears in get_variables_list();0;set_variable("user_variable_tracking", "1") succeeds and reads back as 1;-1 and 2 are rejected and preserve the prior value.make -C test/tap/tests/unit mysql_variables_unit-t
./test/tap/tests/unit/mysql_variables_unit-t
Expected: registration/default assertions fail.
Add:
// MySQL_Threads_Handler::variables
int user_variable_tracking;
// thread-local declaration and extern
__thread int mysql_thread___user_variable_tracking;
Register user_variable_tracking, initialize it to 0, add it to VariablesPointers_int with range 0..1, and refresh it beside query_processor_parser / set_parser_algorithm.
In LOAD MYSQL VARIABLES TO RUNTIME, emit one warning when the loaded values satisfy:
user_variable_tracking == 1 &&
set_parser_algorithm != 3 &&
query_processor_parser != 1
The warning must name all three fully qualified variables, explain that tracking remains inactive, and must not rewrite either parser setting.
Rerun the unit test and compile the admin path:
make -C test/tap/tests/unit mysql_variables_unit-t
./test/tap/tests/unit/mysql_variables_unit-t
make -C lib obj/Admin_FlushVariables.oo obj/MySQL_Thread.oo
Expected: validation passes and the admin/thread objects compile.
git add \
include/MySQL_Thread.h \
include/proxysql_structs.h \
lib/MySQL_Thread.cpp \
lib/Admin_FlushVariables.cpp \
test/tap/tests/unit/mysql_variables_unit-t.cpp
git commit -m "feat: add MySQL user-variable tracking mode"
Files:
Modify: include/mysql_connection.h
Modify: lib/mysql_connection.cpp
Modify: lib/mysql_data_stream.cpp
Modify: test/tap/tests/unit/mysql_user_variables_unit-t.cpp
Step 1: Add failing connection-state tests
Extend the state unit test with small MySQL_Connection fixtures to verify:
requires_CHANGE_USER() is true when the backend has a user-variable name absent from the frontend;number_of_matching_session_variables();not_matching and do not count as matches;reset() clears user variables along with ordinary variables;count, stored_bytes, and optional fingerprint, never the source strings.make -C test/tap/tests/unit mysql_user_variables_unit-t
./test/tap/tests/unit/mysql_user_variables_unit-t
Expected: the new connection assertions fail.
Add this public member to MySQL_Connection:
MySQL_User_Variable_State user_variables;
Extend requires_CHANGE_USER() with backend-extra-name detection. Extend number_of_matching_session_variables() with exact user-variable match/mismatch accounting. Clear the map in MySQL_Connection::reset(); this automatically covers successful backend COM_CHANGE_USER, reset algorithms, frontend COM_RESET_CONNECTION, frontend COM_CHANGE_USER, and disconnect teardown because those paths already call reset().
Add aggregate JSON under:
conn.user_variables
backends[].conn.user_variables
with count, stored_bytes, and fingerprint only. Omit fingerprint if keyed initialization failed.
make -C lib obj/mysql_connection.oo obj/mysql_data_stream.oo
make -C test/tap/tests/unit mysql_user_variables_unit-t
./test/tap/tests/unit/mysql_user_variables_unit-t
Expected: connection matching, reset, and redaction tests pass.
git add \
include/mysql_connection.h \
lib/mysql_connection.cpp \
lib/mysql_data_stream.cpp \
test/tap/tests/unit/mysql_user_variables_unit-t.cpp
git commit -m "feat: match pooled connections by user variables"
Files:
include/MySQL_Thread.hlib/MySQL_Thread.cpptest/tap/tests/unit/statistics_unit-t.cppCounters:
User_variable_assignments_tracked
User_variable_replay_commands
User_variable_replay_failures
User_variable_fallback_unsupported
User_variable_fallback_limits
proxysql_mysql_user_variable_assignments_tracked_total
proxysql_mysql_user_variable_replay_commands_total
proxysql_mysql_user_variable_replay_failures_total
proxysql_mysql_user_variable_fallback_unsupported_total
proxysql_mysql_user_variable_fallback_limits_total
Extend statistics_unit-t.cpp to assert the five new status names and Prometheus metric descriptors are registered exactly once.
make -C test/tap/tests/unit statistics_unit-t
./test/tap/tests/unit/statistics_unit-t
Expected: all five names are absent.
Append five MySQL_Thread_status_variable members before MY_st_var_END, append five p_th_counter::metric values before SIZE_, and add the one-to-one mappings/descriptions in lib/MySQL_Thread.cpp. Keep assignments-count semantics separate from SET-command semantics: a four-target successful SET increments assignments by four, while each internal replay batch increments replay commands by one.
make -C lib obj/MySQL_Thread.oo
make -C test/tap/tests/unit statistics_unit-t
./test/tap/tests/unit/statistics_unit-t
Expected: stats registration passes.
git add include/MySQL_Thread.h lib/MySQL_Thread.cpp \
test/tap/tests/unit/statistics_unit-t.cpp
git commit -m "feat: expose MySQL user-variable tracking counters"
Files:
include/proxysql_structs.hinclude/MySQL_Session.hlib/MySQL_Session.cpptest/tap/tests/unit/mysql_user_variables_unit-t.cppSession additions:
// Append to session_status.
SETTING_USER_VARIABLES,
std::vector<MySQL_User_Variable_Replay_Batch> user_variable_replay_batches;
size_t user_variable_replay_batch_index { 0 };
bool handler_again___verify_backend_user_variables(MySQL_Connection* myconn);
bool handler_again___status_SETTING_USER_VARIABLES(int* rc);
Factor the non-I/O completion decision into a small function in MySQL_User_Variables.h/.cpp and test:
SETTING_USER_VARIABLES;FAIL_CLIENT_QUERY_AND_RETIRE_BACKEND, and leaves frontend desired state intact.make -C test/tap/tests/unit mysql_user_variables_unit-t
./test/tap/tests/unit/mysql_user_variables_unit-t
Expected: replay transition tests fail until the helper exists.
Call handler_again___verify_backend_user_variables(myconn) immediately after handler_again___verify_multiple_variables(myconn) and before the client query is sent. It must:
myconn->options.max_allowed_pkt and ProxySQL's query packet framing;SETTING_USER_VARIABLES when work exists;This preserves the required order: user/schema, autocommit and regular session variables including charset/sql_mode, then user variables, then the client query.
Dispatch SETTING_USER_VARIABLES from handler_again___multiple_statuses(). Send each batch through async_send_simple_command() using the batch's exact SQL. On OK, apply that batch's assignments to the backend map and increment st_var_user_variable_replay_commands; then send the next batch or resume the saved client-query status.
On server or client-library error:
st_var_user_variable_replay_failures;RequestEnd() once;Treat ASSIGNMENT_TOO_LARGE from the planner as the same replay failure disposition. Emit the fixed debug reason REPLAY_FAILURE without serializing batch SQL, targets, literals, or hashes.
make -C lib obj/MySQL_Session.oo
make -C test/tap/tests/unit mysql_user_variables_unit-t
./test/tap/tests/unit/mysql_user_variables_unit-t
Expected: the session object compiles and every replay success/failure decision test passes.
git add \
include/proxysql_structs.h \
include/MySQL_Session.h \
include/MySQL_User_Variables.h \
lib/MySQL_Session.cpp \
lib/MySQL_User_Variables.cpp \
test/tap/tests/unit/mysql_user_variables_unit-t.cpp
git commit -m "feat: replay tracked user variables on backends"
Files:
include/MySQL_Session.hinclude/mysql_connection.hlib/MySQL_Session.cpplib/mysql_connection.cpptest/tap/tests/unit/parsersql_unit-t.cppSession query state:
std::optional<UserVariableSetAnalysis> pending_user_variable_set;
bool current_query_user_variable_safe { false };
bool user_variable_tracking_latched { false };
Extend parsersql_unit-t.cpp with a pure policy helper test matrix proving that tracking new assignments requires all of:
1;COM_QUERY, not prepare/execute;Also assert resource preflight happens against a staged post-SET map and never mutates committed state.
make -C test/tap/tests/unit parsersql_unit-t
./test/tap/tests/unit/parsersql_unit-t
Expected: policy helpers are missing.
At the start of the existing SET branch in handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_qpo():
parsersql_analyze_user_variable_set_mysql() on CurrentQuery.QueryPointer and QueryLength;SUPPORTED, stage the post-SET frontend state. On success, retain the analysis in pending_user_variable_set, mark the query safe, and forward the original packet without passing it through the old system-variable walker;st_var_user_variable_fallback_limits, synchronize prior tracked state through Task 7, forward the original SET, and call the existing unable_to_parse_set_statement() fallback;UNSUPPORTED or PARSE_ERROR for a user-variable SET, increment st_var_user_variable_fallback_unsupported, preserve the existing digest/raw warning behavior, and use unable_to_parse_set_statement();NOT_USER_VARIABLE_SET, continue unchanged through the existing system-variable SET parser.Mixed user/system SETs must enter step 5 before any ordinary frontend variable is mutated.
Add fixed debug reasons for PARSER_PREREQUISITE_MISSING, UNSUPPORTED_AST, and RESOURCE_LIMIT. They may follow the existing parse-failure logging choice of raw query versus digest, but must not introduce any new value/name logging independent of that existing policy.
In the rc==0 query-completion path, before RequestEnd(myds) can return the backend to the pool:
status == PROCESSING_QUERY and pending_user_variable_set exists, apply its assignments atomically to both client_myds->myconn->user_variables and myconn->user_variables;user_variable_tracking_latched=true on the first commit;st_var_user_variable_assignments_tracked by assignment count;In every rc==-1, retry, disconnect, RequestEnd() error, and session reset() path, discard pending state without changing either map.
Change the status API to:
void ProcessQueryAndSetStatusFlags(
char* query_digest_text,
bool user_variable_usage_is_safe);
Pass current_query_user_variable_safe from RequestEnd(). The new boolean suppresses only ProcessQueryAndSetStatusFlags_UserVariables(); query-rule multiplex=0/1, warnings, temporary tables, savepoints, locks, prepared statements, and every other status classifier remain unchanged.
Clear the per-query safety boolean after RequestEnd().
make -C lib obj/MySQL_Session.oo obj/mysql_connection.oo
make -C test/tap/tests/unit parsersql_unit-t
./test/tap/tests/unit/parsersql_unit-t
Expected: supported SETs take the staging path, the code compiles, and parser/policy tests pass.
git add \
include/MySQL_Session.h \
include/mysql_connection.h \
lib/MySQL_Session.cpp \
lib/mysql_connection.cpp \
test/tap/tests/unit/parsersql_unit-t.cpp
git commit -m "feat: commit literal user-variable SETs on backend OK"
Files:
Modify: include/MySQL_Session.h
Modify: include/Query_Processor_ParserSQL.h
Modify: lib/MySQL_Session.cpp
Modify: lib/Query_Processor_ParserSQL.cpp
Modify: test/tap/tests/unit/parsersql_unit-t.cpp
Step 1: Add failing query-disposition tests
Test a pure disposition helper with these inputs:
@: do not call ParserSQL and mark the UDV status path safe;@ only inside string/comment: classifier returns no user variable and remains safe;@x: safe and multiplexable after synchronization;SET @x=1 supported: handled by Task 8;SELECT @x:=1, SELECT id INTO @x FROM test.uv_source, function/call AST shapes containing a real user-variable occurrence, partial/multi-statements, and prepared SETs: unsafe fallback;sql_mode, client/connection charset, connection collation, NAMES, or CHARACTER SET synchronize the old user-variable state and bind before changing interpretation context;make -C test/tap/tests/unit parsersql_unit-t
./test/tap/tests/unit/parsersql_unit-t
Expected: runtime/latch cases fail until the disposition helper is added.
@ fast gate and ParserSQL usage classificationFor plain COM_QUERY not already handled as a supported SET:
memchr(query, '@', query_length) as the only no-parse fast gate;@ exists, leave routing unchanged and mark UDV classification safe;@ exists and either new tracking is active or user_variable_tracking_latched is true, call parsersql_classify_user_variable_usage_mysql();NO_USER_VARIABLE and READ_ONLY safe; Task 7 will synchronize any desired map before the query;UNSAFE_OR_UNKNOWN, synchronize prior state, invoke the existing connection-bound/hostgroup-lock policy while honoring explicit qpo->multiplex, and leave the legacy UDV status classifier unsuppressed.Do not apply the safe-read relaxation to prepared statement prepare/execute paths; the initial implementation tracks text protocol only.
Use two predicates:
bool accepts_new_user_variable_assignments() const;
bool must_classify_and_sync_user_variables() const;
The first requires current mode/prerequisite and an unbound session. The second is true when the first is true or user_variable_tracking_latched is true. Never clear the latch on runtime variable refresh; clear it only through session reset() / change-user / disconnect lifecycle.
Once an unsafe query causes connection-bound fallback, stop updating the tracked map. The bound backend remains authoritative, while any previously tracked map stays available only for the already-selected backend's prior synchronization and aggregate diagnostics.
Add this adapter predicate using the same strict full-input ParserSQL AST:
bool parsersql_set_changes_user_variable_replay_context_mysql(
const char* query, size_t query_length);
It returns true for SET assignments affecting sql_mode, character_set_client, character_set_connection, or collation_connection, plus SET NAMES and SET CHARACTER SET nodes. When the frontend map is nonempty, check this predicate before the legacy SET parser mutates ordinary frontend state. Synchronize the old map, forward the original context-changing SET through the existing fallback, and bind the selected backend; do not update the frontend user-variable map afterward.
In handler_rc0_Process_Variables(), if backend session tracking reports any of those context variables changed while the frontend user-variable map is nonempty, set the existing connection-bound/user-variable status before RequestEnd() so the current backend cannot return to the pool. This handles context mutation hidden inside backend-side code while leaving hidden user-variable writes themselves documented as unsupported.
make -C lib obj/MySQL_Session.oo
make -C test/tap/tests/unit parsersql_unit-t
./test/tap/tests/unit/parsersql_unit-t
Expected: all disposition, runtime-disable, and latch tests pass.
git add \
include/MySQL_Session.h \
include/Query_Processor_ParserSQL.h \
lib/MySQL_Session.cpp \
lib/Query_Processor_ParserSQL.cpp \
test/tap/tests/unit/parsersql_unit-t.cpp
git commit -m "feat: classify user-variable reads and unsafe uses"
Files:
Create: test/tap/tests/mysql-user-variable-tracking-t.cpp
Modify: test/tap/groups/groups.json
Step 1: Write the failing TAP happy-path fixture
Create a dedicated test that:
mysql-user_variable_tracking, mysql-set_parser_algorithm, mysql-query_processor_parser, and mysql-set_query_lock_on_hostgroup;runtime_mysql_servers, mirrors it into temporary hostgroups 18110 and 18111, and installs comment-scoped query rules for /* uv_hg_a */ and /* uv_hg_b */ for the test user only;1, SET parser to 3, full-query parser to 0, hostgroup locking to 1, and loads MySQL variables/rules/servers;SELECT;0x/X'', 0b/B'', and NULL;MYSQL_FIELD::type, HEX(@name), CHARSET(@name), COLLATION(@name), and COERCIBILITY(@name) after each replay-sensitive hostgroup switch;test.proxysql_uv_read() that returns @browser_lang, routes SELECT test.proxysql_uv_read()—which contains no @ byte—to a different hostgroup, and proves tracked state is still synchronized for backend-side readers;CONNECTION_ID() values are observed, and verifies every tracked value survives each switch;$REGULAR_INFRA_DATADIR/proxysql.log to EOF before the reported SET and asserts no new Unable to parse unknown SET query record contains that statement;PROXYSQL INTERNAL SESSION reports the expected frontend count/byte total/fingerprint and contains none of the variable names or values;locked_on_hostgroup == -1 and that no backend acquires status.user_variable=true for supported SET/read traffic;@name IS NULL, proving pooled backend state is reset rather than leaked;test.proxysql_uv_read, removes only hostgroups 18110/18111 and test-comment query rules, restores saved variables, and loads runtime state even when an assertion fails.Register the test in legacy-g4, mariadb10-galera-g4, mysql-multiplexing=false-g4, mysql84-g4, mysql90-g4, mysql95-g4, and set_parser_algorithm_3-g1.
make -C test/tap/tests mysql-user-variable-tracking-t
./test/tap/tests/mysql-user-variable-tracking-t
Expected before the completed feature: the reported SET locks the session or tracked values disappear/leak while changing hostgroups.
Trace failures to parser analysis, commit-on-OK, pool matching, normal-variable-before-UDV ordering, or replay. Do not loosen syntax classification or make the test reuse a single backend connection merely to pass.
Rerun:
make -j4 debug
make -C test/tap/tests mysql-user-variable-tracking-t
./test/tap/tests/mysql-user-variable-tracking-t
Expected: the exact client statement remains multiplexable, survives hostgroup changes, and does not contaminate another frontend.
git add \
test/tap/tests/mysql-user-variable-tracking-t.cpp \
test/tap/groups/groups.json \
include/MySQL_Session.h \
lib/MySQL_Session.cpp \
include/mysql_connection.h \
lib/mysql_connection.cpp
git commit -m "test: cover multiplexed literal user variables"
Files:
Modify: test/tap/tests/mysql-user-variable-tracking-t.cpp
Modify when a defect is exposed: lib/MySQL_Session.cpp
Modify when a defect is exposed: lib/mysql_connection.cpp
Modify when a defect is exposed: lib/MySQL_User_Variables.cpp
Step 1: Add failing negative and lifecycle TAP phases
Use a fresh frontend connection per connection-bound case and add assertions for:
SELECT @x:=..., and SELECT ... INTO @x retain the current fallback and lock under the default policy;SELECT @x, comparisons using @x, and @ inside a string/comment do not lock;SET @context_value='A\\n', changing sql_mode or SET NAMES binds the session before the change and preserves HEX(@context_value), CHARSET(@context_value), and COLLATION(@context_value) on the authoritative backend rather than replaying under the new context;1 without either ParserSQL prerequisite is inactive and locks the reported SET; changing parser configuration must not be forced by the load;$REGULAR_INFRA_DATADIR/proxysql.log after the misconfigured LOAD MYSQL VARIABLES TO RUNTIME and names mysql-user_variable_tracking, mysql-set_parser_algorithm, and mysql-query_processor_parser without printing the SET value;mysql-query_processor_parser=1) activates tracking while mysql-set_parser_algorithm=2, proving both documented prerequisite alternatives;0 or moving both parser settings away from ParserSQL still lets that existing session read/synchronize prior state across both hostgroups, while a new SET on it falls back and a fresh session gets current disabled behavior;multiplex=0 remains authoritative for a supported read/SET, and an explicit multiplex=1 retains the existing override behavior for an unsafe use;mysql-set_query_lock_on_hostgroup=0 retains the pre-2.0.6 connection-status fallback for unsafe user-variable use rather than silently using mode-1 tracking;mysql_reset_connection() clears frontend diagnostics and makes subsequent SELECT @x IS NULL true;mysql_change_user() to the same fixture credentials clears the state and value;Also force a replay failure at the pure replay-completion seam from Task 7 in the unit test and assert the action retires the backend and fails the pending query; do not add a production runtime failpoint solely for TAP.
make -C test/tap/tests mysql-user-variable-tracking-t
./test/tap/tests/mysql-user-variable-tracking-t
make -C test/tap/tests/unit mysql_user_variables_unit-t
./test/tap/tests/unit/mysql_user_variables_unit-t
Expected: at least runtime drain, reset, or error accounting fails before final lifecycle wiring is complete.
Ensure pending_user_variable_set, replay queue/index, current-query safety, and the tracking latch are cleared in MySQL_Session::reset(). Ensure backend map changes occur only after successful original SET/replay commands. Verify connection destroy paths cannot return a partially replayed backend to the pool.
Keep the documented limitation intact: writes hidden inside procedures/functions/triggers are not detected. Do not claim correctness for those paths and do not add heuristic SQL scanning.
Rerun both commands from Step 1 until green.
git add \
test/tap/tests/mysql-user-variable-tracking-t.cpp \
test/tap/tests/unit/mysql_user_variables_unit-t.cpp \
include/MySQL_Session.h \
lib/MySQL_Session.cpp \
lib/mysql_connection.cpp \
lib/MySQL_User_Variables.cpp
git commit -m "test: cover user-variable fallback and lifecycle"
Files:
Create: doc/mysql-user-variable-tracking.md
Modify: deps/parsersql/README.md only if the final archive audit needs a correction
Verify: every production/test file changed above
Step 1: Write operator-facing documentation
Document:
integer modes, default/range, and both ParserSQL prerequisites;
the exact supported literals and all-or-nothing SET rule;
initial SET forwarding and commit-on-OK behavior;
frontend/backend maps, resource limits, pool reset on backend extras, replay order, and batching;
read-only versus unsafe usage behavior;
runtime-disable/prerequisite drain semantics;
COM reset/change-user/disconnect clearing;
all five counters and aggregate-only internal-session diagnostics;
the explicit hidden-backend-write limitation for stored procedures, functions, and triggers;
the connection-bound safeguard when interpretation context changes after tracked assignments;
a deployment example using the reported browser metadata statement.
Step 2: Run focused upstream and downstream verification from clean build products
make -C /data/rene/ParserSQL-user-variable-literals clean
make -C /data/rene/ParserSQL-user-variable-literals test
make -C /data/rene/ParserSQL-user-variable-literals build-corpus-test
make -C deps parsersql/parsersql/libsqlparser.a
make -C lib clean
make -j4 debug
make -C test/tap/tests/unit \
parsersql_unit-t \
mysql_user_variables_unit-t \
mysql_variables_unit-t \
statistics_unit-t
./test/tap/tests/unit/parsersql_unit-t
./test/tap/tests/unit/mysql_user_variables_unit-t
./test/tap/tests/unit/mysql_variables_unit-t
./test/tap/tests/unit/statistics_unit-t
make -C test/tap/tests \
setparser_parsersql_test \
setparser_test \
setparser_test2 \
setparser_test3 \
mysql-user-variable-tracking-t
./test/tap/tests/setparser_parsersql_test
./test/tap/tests/setparser_test
./test/tap/tests/setparser_test2
./test/tap/tests/setparser_test3
./test/tap/tests/mysql-user-variable-tracking-t
Expected: every command exits zero. Investigate any failure; do not label it baseline or flaky without a deterministic root cause.
make -C test/tap/tests \
reg_test_3327-process_query_set_status_flags-t \
test_com_reset_connection_com_change_user-t \
set_testing-t \
set_testing-multi-t \
test_filtered_set_statements-t
./test/tap/tests/reg_test_3327-process_query_set_status_flags-t
./test/tap/tests/test_com_reset_connection_com_change_user-t
./test/tap/tests/set_testing-t
./test/tap/tests/set_testing-multi-t
./test/tap/tests/test_filtered_set_statements-t
Expected: existing SET status flags, reset/change-user behavior, ParserSQL SET mode, and legacy filtered SET handling remain green.
git diff --check 309ca545f
git status --short
git diff 309ca545f -- \
include/MySQL_User_Variables.h \
lib/MySQL_User_Variables.cpp \
lib/MySQL_Session.cpp \
lib/mysql_connection.cpp \
test/tap/tests/mysql-user-variable-tracking-t.cpp \
doc/mysql-user-variable-tracking.md | \
rg -n "TODO|FIXME|placeholder|not implemented"
git diff 309ca545f -- \
lib/mysql_data_stream.cpp \
lib/mysql_connection.cpp \
lib/MySQL_Session.cpp | \
rg -n "canonical_name|replay_target|raw_literal|\.hash"
Review the last search manually: uses in internal state/replay are expected; no logging or JSON serialization of those fields is allowed. Confirm enum types match across ParserSQL and ProxySQL adapters, byte limits use size_t, query lengths never narrow unsafely, and every pending-state error path clears exactly once.
If Step 2, 3, or 4 requires a production correction, return to the owning task's focused test, rerun it, and commit that correction before the documentation commit. Do not hide production changes inside a docs-only commit.
git add doc/mysql-user-variable-tracking.md
git commit -m "docs: explain literal user-variable tracking"
Do not include unrelated working-tree files in this or any earlier commit.