Skip to main content
No single obfuscation technique is sufficient against a motivated reverse engineer. Real-world protection uses multiple layers, each defending against a different attack vector. By combining source-level transforms, AST manipulation, bytecode obfuscation, virtual machine protection, and runtime defenses, you create a pipeline where an attacker must defeat every layer to recover your original logic. This page maps techniques to attack surfaces, presents a recommended layering order, explains interaction pitfalls, and provides a practical protection level framework.

​
The Two Attack Surfaces

Obfuscation must defend against two distinct approaches an analyst can take:

​
Static Analysis

Static analysis reads the file without executing it. Tools include decompilers, AST parsers, grep, and bytecode disassemblers. The analyst examines the file on disk.

​
Dynamic Analysis

Dynamic analysis runs the file under a debugger, instrumented VM, or hook framework. The analyst observes behavior at runtime to extract secrets or reconstruct logic.

​
Technique Mapping

A complete pipeline includes at least one technique from each layer. The order in which you apply transformations matters. Some transforms create patterns that later transforms should hide. Others expand code size in ways that make subsequent steps unnecessarily expensive.
1

Start with Original Source

Begin with clean, well-tested source code. Obfuscation should be the final build step, applied only after all functionality is verified.
2

Apply AST-Level Transforms

Perform scope-aware renaming, expression decomposition, and statement reordering at the AST level. These restructure the program without changing semantics and provide the foundation for later obfuscation.
3

Encode Sensitive Strings

Apply XOR or byte-array encoding to all sensitive strings. This hides plaintext secrets from static inspection. The encoder itself should use randomized keys per build.
4

Obfuscate Numeric Constants

Replace numeric literals with opaque expressions. For example, replace 42 with (17 * 3) - 9 or a lookup in a precomputed constant table.
5

Flatten Control Flow

Apply control flow flattening with opaque predicates. Replace structured control flow with state-machine dispatch loops guarded by always-true or always-false conditions that confuse decompilers.
6

Inject Junk Code and Identifier Flooding

Add dead code branches and flood the namespace with dummy identifiers. Do this after control flow flattening so the junk is also flattened and harder to distinguish from real logic.
7

Wrap Sensitive Sections in a Custom VM

For the most sensitive logic (license checks, cryptographic keys), compile to a custom bytecode and embed a register-based VM interpreter in Lua. The analyst must reverse the VM instruction set before they can analyze the protected section.
8

Compile to Bytecode and Strip Debug Info

Compile the final Lua source to bytecode and strip debug symbols. This removes line numbers, local variable names, and upvalue names from the bytecode, making decompilation harder.
9

Add Anti-Debug and Sandbox Detection

Inject runtime checks for debuggers, hooks, and sandboxed environments into the final output. These must use standard library names that have not been renamed.
10

Test the Output

Run your full test suite against the obfuscated output. Obfuscation can introduce subtle bugs, especially when control flow flattening reorders statements or when opaque predicates have edge-case behavior.

​
Layer Interaction and Ordering Pitfalls

Applying transformations in the wrong order reduces effectiveness or inflates output size unnecessarily.

​
Junk Code Before Control Flow Flattening

If you inject junk code before flattening, the flattener treats every junk branch as real code and generates additional dispatch states for it. This produces larger output than necessary. Inject junk after flattening, or ensure your flattener can mark dead branches.

​
String Encoding Before Renaming

If your string encoder introduces identifiable patterns (for example, decode_string("\x41\x42")), the variable renamer will also rename the decoder function name, which may make the pattern harder to spot. However, if the encoder uses global names for helper functions, rename those globals first so the encoded strings do not reference obfuscated names.

​
Anti-Debug After All Other Transforms

Anti-debug checks call standard library functions like debug.gethook, os.clock, and tostring. If you rename identifiers globally before injecting anti-debug code, these checks will fail because they reference the renamed functions. Always inject anti-debug and sandbox detection code after renaming, or exempt standard library names from renaming.

​
Choosing the Right Level of Protection

Not every script needs maximum protection. Higher levels increase file size, reduce performance, and complicate debugging. Match the protection level to the sensitivity of the logic and the threat model.

​
Level 1: Basic

Variable renaming and string encoding are sufficient to stop casual copying and pasting. A determined analyst can still read the logic, but it is no longer immediately understandable. This level is appropriate for open-source scripts where you want to deter direct theft but not prevent reverse engineering.

​
Level 2: Moderate

Adding control flow flattening, junk code, and constant obfuscation defeats most automated decompilers. Tools that rely on pattern matching or structured control flow reconstruction will produce garbled output. A skilled human can still reverse this, but it requires significant manual effort.

​
Level 3: Strong

AST-level transforms restructure the program at a deeper level than source transforms. Bytecode stripping removes the metadata that decompilers use to reconstruct variable names and line mappings. Anti-debug checks deter dynamic analysis. This level is appropriate for protecting license validation and authentication code in commercial products.

​
Level 4: Maximum

VM virtualization replaces sensitive logic with custom bytecode executed by an embedded interpreter. Environment spoofing and sandbox detection add runtime defenses against dynamic analysis. This level can make a script ten times larger and five times slower. Use it only for the most sensitive sections of code, not for the entire application.
Profile your obfuscated script at each level. VM virtualization combined with control flow flattening and junk code can make a script ten times larger and five times slower. Benchmark critical paths and tune the obfuscation parameters. Consider protecting only the most sensitive functions with maximum obfuscation while leaving the rest at a lower level.