What Is an AST
An Abstract Syntax Tree is a hierarchical data structure that represents the syntactic structure of source code. Every node in the tree corresponds to a syntactic construct in the language: a function declaration, a variable assignment, a binary operation, a string literal, and so on. The tree captures the relationships between these constructs, such as which statements belong inside which function body, or which expressions are operands of which operator. Consider this simple Lua snippet:Chunk node containing two Statement children. The first is a LocalStatement with an Identifier named x and a value that is a BinaryExpression with operator +, left operand NumberLiteral{1}, and right operand NumberLiteral{2}. The second is a CallExpression with a NameExpression callee print and a single argument that is a NameExpression referencing x.
This tree representation is fundamentally different from the raw text. The AST knows that x in print(x) is a variable reference, not a string or keyword. It knows that 1 + 2 is an addition operation with two numeric operands. It knows the scope boundaries created by local declarations. This semantic awareness is what makes AST transforms so powerful.
Why AST-Based Obfuscation Beats Text Replacement
Text-based obfuscation using regular expressions or string replacement is brittle and error-prone. AST-based obfuscation is precise, scope-aware, and guaranteed to produce syntactically valid output.Text Replacement
- Breaks on multi-line strings containing keywords
- Cannot distinguish variables from string literals
- Ignores scope boundaries
- Fails on comments and whitespace edge cases
- Produces invalid syntax when reordering statements
AST Transforms
- Operates on semantic structure, not characters
- Knows the difference between identifiers and strings
- Full scope and upvalue analysis
- Comments and formatting are handled by the serializer
- Every transformation preserves syntactic correctness
x to a would break this code:
"the variable a is unused", corrupting the program’s behavior. An AST-based renamer knows that x inside the string literal is a StringLiteral node, not a NameExpression, and leaves it untouched.
The Lua AST Structure
Lua and Luau parsers produce ASTs with a well-defined set of node types. Understanding these node types is essential for writing custom transforms.Top-Level and Statement Nodes
The root of every Lua AST is aChunk node, which contains a list of statements. Each statement is one of the following types:
Expression Nodes
Expressions appear as values on the right-hand side of assignments, as conditions in control flow, as arguments to functions, and in many other positions. The main expression node types are:Annotated Code Sample
Here is a slightly more complex function with prose annotations mapping each syntactic element to its AST node type:How a Parser Produces the AST
Parsing is a two-phase process: lexing (tokenization) followed by syntactic analysis.Phase 1: Lexing
The lexer scans the raw source text character by character and produces a stream of tokens. Each token has a type and a value. For the inputlocal x = 1 + 2, the lexer produces:
Phase 2: Parsing
The parser consumes the token stream and builds the AST using a recursive-descent algorithm. It matches token sequences against the grammar rules of Lua. Forlocal x = 1 + 2, the parser performs roughly these steps:
- See
Keyword("local")— begin aLocalStatement - Read
Identifier("x")— add to the names list - See
Operator("=")— expect value expressions next - Read
Number("1")— create aNumberLiteralnode - See
Operator("+")— create aBinaryExpressionwith the previousNumberLiteralas left operand - Read
Number("2")— set as the right operand of theBinaryExpression - Finish the
LocalStatementwith names["x"]and values[BinaryExpression]
JSON-Like Pseudo-AST
Here is a simplified JSON-like representation of the AST for a small Lua function:The AST Visitor Pattern
The visitor pattern is the standard technique for traversing and transforming an AST. A visitor is a function that receives a node, performs some action, and recursively visits the node’s children.Counting String Literals
Here is pseudocode for a visitor that counts all string literals in an AST:Chunk node, and it walks the entire tree, incrementing the counter every time it encounters a StringLiteral.
Renaming Local Variables
Here is a more complex visitor that renames all variables declared inLocalStatement nodes:
LocalStatement, it generates a new random name for each declared identifier and records the mapping. When it sees a NameExpression, it looks up the name in the scope table and applies the rename if found.
Real-world renamers need proper scope analysis to handle shadowing and upvalues correctly. The pseudocode above is simplified. See AST Transformations for a complete scope-aware renaming implementation.
AST Serialization
After you transform the AST, you must convert it back into valid Lua source text. This process is called serialization, pretty-printing, or code generation. A serializer traverses the AST depth-first and emits source text for each node type:- A
LocalStatementwith names[a, b]and values[1, 2]emitslocal a, b = 1, 2 - A
BinaryExpressionwith operator+emits the left operand, then+, then the right operand - A
FunctionExpressionemitsfunction, then parameters in parentheses, then the body, thenend
AST vs. CST
What is the difference between an AST and a CST?
What is the difference between an AST and a CST?
A Concrete Syntax Tree (CST) includes every token from the source file, including whitespace, comments, and parentheses that may be semantically redundant. A CST preserves the exact surface structure of the code.An Abstract Syntax Tree (AST) strips away unneeded syntax and represents only the semantic structure. Parentheses around expressions are folded into the tree structure. Whitespace and comments are typically discarded or stored as metadata.Most obfuscation tools work on ASTs because the goal is to transform semantics, not preserve formatting. However, some tools use CSTs when they need fine-grained control over output formatting or when they want to preserve comments during minification.
Next Steps
Now that you understand what an AST is and how parsers produce it, you can explore the specific transformations you can apply:AST Transformations
Learn scope-aware renaming, expression splitting, statement reordering, and function inlining.
AST Tools
Discover the parsers and libraries available for Lua and Luau.
Variable Renaming
Dive deeper into identifier renaming strategies.
