Security teams depend on binary analysis for tasks such as incident response, threat intelligence, and auditing third-party software. Two commonly used capabilities accelerate almost all of this work: identifying known components, and understanding how those components are used. This is usually achieved by matching common library functions and turning raw disassembly into readable pseudocode.
Function matching has traditionally relied on byte-level and signature-based techniques, including hashing, fuzzy hashing, and pattern signatures such as FLIRT. These work well when a function is untouched, but small changes, such as a different compiler flag or a minor patch can shift the underlying bytes enough to break a match, even without any change to the function's behaviour.
Classical approaches to decompilation, such as those implemented in Ghidra and Hex-Rays’s decompilers, rely on rule-based transformations and refinement passes to reconstruct pseudo code from assembly. Such approaches handle common patterns well, but often produce output that is technically correct yet hard to read, or that breaks down on heavily optimised or obfuscated binaries the rules weren't designed for.
Machine learning offers a different lever on both problems: rather than hand-coding what "the same function" or "readable code" looks like, models can learn this directly from data, generalising across the compiler noise and structural variation that breaks rule-based tools. This is the premise behind two models we've built at RevEng.AI: WilBERT for function matching, and Ventris for decompiling binaries into readable, structured source code. This post gives an overview of our latest model releases under Mega Bite, our new suite of specialized AI models for binary analysis , comparing our results against current state-of-the-art.
Ventris is our Specialised Language Model (SLM) trained from scratch to recover source code from software binaries. It incorporates six-months of improvements to our previous BinNet style binary-analysis methodology, training techniques, and the scale and quality of our datasets.
Neural decompilation takes a fundamentally different approach from traditional decompilation. Traditional decompilers use hand-crafted rules and heuristics to reconstruct higher-level code from a binary. Although highly capable, they often produce verbose or unnatural output that can be difficult for an analyst to interpret. A neural decompiler instead learns from examples of compiled binaries and their original, human-written source code. Given enough high-quality examples, a sufficiently capable neural network can learn to generate human-like source code directly from the binary. Readability alone, however, is not sufficient: a useful decompiler must also preserve the behaviour of the original program. Our objective with Ventris is therefore to produce code that is both natural to read and semantically faithful to the binary.
Frontier language models have also become surprisingly capable at decompilation of small assembly sequences. Nevertheless, even the largest general-purpose models remain imperfect, and their inference costs can make real-world binary-wide analysis impractical—particularly when binaries contain many large functions. Our approach is to use similar techniques for training frontier AI models and apply them to smaller models built specifically for decompilation. These small language models can decompile every function in a binary quickly and economically, while producing a readable and reliable source code suitable for subsequent analysis. This also creates a useful foundation for automated program analysis.
Evaluating a decompiler is a difficult and comparatively under-explored task. Neural decompilation makes two longstanding problems particularly important:Firstly, a function is rarely self-contained. It may depend on user-defined types, global variables, declarations, or other context elsewhere in the program. A semantically correct decompilation may therefore fail to compile when evaluated in isolation.Secondly, determining whether two implementations are semantically equivalent is difficult. A function can be written correctly in many different ways, and a useful evaluation should distinguish a nearly correct result from one that bears little resemblance to the original behaviour. Most reverse engineers will have seen first hand that pseudocode structure of decompiled functions can completely change compared to the original source code.
One early benchmark for neural decompilation is HumanEval-Decompile. It contains 164 programming problems converted to C and supplemented with hand-written tests. Each function is compiled, its assembly is passed to the decompiler, and the generated source is substituted back into the program. The program is then recompiled and tested.
This provides a useful end-to-end measure, but it has two important limitations: Results are binary, either every test passes, or the sample fails. A subtle error therefore receives the same score as a fundamentally incorrect implementation. Conversely, a verbose or unreadable function receives full credit if it passes the tests.Secondly, the benchmark does not closely resemble real-world binaries. It uses only standard C libraries, contains no user-defined types, and has an average function length of approximately 15 lines.
Despite these limitations, HumanEval-Decompile remains a useful signal. Below, we compare Ventris with our previous model, LLM4Decompile, and several frontier language models.
Table 1: Evaluation of Ventris on the HumanEval dataset.
Ventris 1.0 improves substantially over our previous model across the evaluated configurations. It also outperforms the larger LLM4Decompile model in this evaluation, despite LLM4Decompile operating on Ghidra pseudocode.
Frontier models can achieve comparable results when given large reasoning budgets, but matching Ventris’s performance costs approximately 300–600 times more per function. We were unable to evaluate Opus 5 or Fable 5 because their cyber-safety controls declined the decompilation requests.
DecBench
DecBench is a newer open-source benchmark designed to address some of the limitations of earlier evaluations. It measures structural similarity using edit distance between control-flow graphs and also evaluates recompilation byte matching and type correctness.
Ventris does not yet support every architecture included in DecBench, so we cannot report a directly comparable result for the complete benchmark. Instead, we evaluated the intersection containing only the binaries and functions supported by Ventris.
Table 2: Evaluation of Ventris on the DecBench dataset (excluding ARM binaries).
On this subset, Ventris outperforms the evaluated traditional decompilers in every sample setting. The latest frontier models perform slightly better on the small sample set, but their average inference cost per function is 1000x higher. That cost is also why their published results are limited to the smaller evaluation set.
There is a further complication when evaluating frontier models. In several cases, Claude appears to recognize a function from its assembly and reproduces the original source verbatim, including variable names and comments. This is obviously impossible to recover, and suggests that some benchmark samples—or closely related source—may have appeared in the model’s training data.
It is difficult to determine the extent of this effect or to evaluate frontier models exclusively on genuinely unseen binaries. The same caveat applies to our own training corpus: some benchmark binaries, or closely related projects, may be present. In the future, benchmark results should therefore be interpreted alongside evaluations whose provenance and holdout boundaries can be controlled more closely.
All the presented benchmarks only provide a weak signal of actual semantic integrity. HumanEval does not capture truly challenging function samples, DecBench does not attempt to execute the recompiled artifacts or verify their behavior otherwise, and even more recent approaches, such as ExeBench are lacking: It relies on synthetic, function-signature-derived input/output pair verification and is in almost all cases not honest to the true execution context or application-specifics under which a target function is usually executed.
To improve on this frontier, we are developing an internal benchmark called VeriDecomp. It is based on ideas from DecompileBench and is designed to preserve the external context needed to compile and execute each evaluated function. During the build process, we use OSS-Fuzz harnesses to retain the declarations, types, globals, and other context required by each function. We also collect a set of valid fuzzing inputs. The generated function can then be transplanted into the original program, recompiled, and executed using the same inputs as the groundtruth implementation, retaining full program semantics.
This allows us to evaluate semantic equivalence on functions drawn from real software rather than isolated programming exercises. Functions in the current VeriDecomp dataset average 26 lines of code, compared with 15 in HumanEval-Decompile, and have an average cyclomatic complexity of 6.6, compared with 5.5.
In our initial evaluation, Ventris has a recompilation rate of 65.8%, and of those functions that recompile, 82.2% behave equivalently to the groundtruth implementation on the benchmark inputs.
These results remain preliminary. Some outputs that fail to compile may still be approximately correct but incompatible with limitations in the current harness. Passing every available input also provides evidence of equivalence rather than a formal proof.
We are continuing to expand VeriDecomp, improve its harnesses, and evaluate additional traditional decompilers and frontier language models. We expect this work to provide a more realistic picture of where neural decompilation succeeds, and where further progress is still needed. We are also planning to release a version of Ventris in the coming months that provides wider architecture support, which will allow us to compare like-to-like with the other decompilers in decbench.
WilBERT, our function similarity model, represents months of R&D driven improvements over our previous model series. WilBERT underpins a number of capabilities in the RevEng.AI product suite, including software composition analysis, malware attribution, and the identification of known code across otherwise unrelated binaries. These capabilities require us to identify semantically equivalent or similar functions across different binaries, compilers, CPU architectures, and optimisation levels, and to do so against indexes potentially containing hundreds of millions or billions of functions. Traditional approaches, such as Ghidra's BSim and MCRIT, are designed to scale efficiently to large collections, but typically rely on engineered features or measures of structural and syntactic similarity, which can become less reliable as compilation choices increasingly change the representation of the underlying code. Heavier, pairwise and differential approaches can provide stronger, more robust measures of similarity, but applying such techniques directly to large-scale search would require comparing a query against a substantial portion of the corpus, making them impractical for our use case without upfront filtering/selection. Learned embeddings provide a useful middle ground: they allow us to learn representations that capture semantic similarity while representing each function as a dense fixed-size vector that can be efficiently indexed and searched using approximate nearest-neighbour (ANN) techniques. Thus, allowing us to query very large corpora efficiently and producing a small set of candidate matches. .
To train WilBERT, we follow a two-stage approach. We first train a BERT based embedding model to learn a representation of individual functions from their own semantics. This provides the overall model with a meaningful representation for each function before introducing inter-procedural information. We then train a further component to refine the representation of a target function using interprocedural information. This allows the model to first learn what characterises semantically equivalent functions locally, before incorporating wider context information from the binary to further distinguish between otherwise similar functions.
Consistent with prior work, we evaluated WilBERT on a number of retrieval tasks and datasets. The primary method for assessing the efficacy of a model of this kind is to evaluate its hit rate against a large index of embeddings, i.e., its ability to retrieve a semantically equivalent function for a given query function within the top N most similar matches. Clearly, the larger N is, the more candidates a downstream process must sift through in order to identify useful matches. A strong model should therefore not only allow us to retrieve the correct function within a reasonably small candidate set, but also rank the most relevant matches as highly as possible.
To capture both properties, we report hit rate at N=1,5,50, together with mean reciprocal rank (MRR) over the top 50 candidates. Hit@N measures the proportion of queries for which a valid match is retrieved within the first N candidates, while MRR additionally captures where the first valid match occurs within that set, rewarding models that consistently place relevant results towards the top of the ranking.
We construct groundtruth pairs from compiled instances of the same function, spanning different binaries, optimisation levels, and architectures, as identified through project metadata and function symbols. Before evaluation, we filter samples to remove exact duplicate disassembly, which would otherwise artificially inflate or distort retrieval results.
Table 3: Evaluation of WilBERT on Linux and Windows binaries.
The Linux evaluation has an index size of 34.5k and we are querying with 34.6k samples, while the Windows evaluation has an index size of 100k and we are querying with an equivalent number of samples. On a combined Linux/Windows dataset WilBERT achieves 0.848 Hit@1 compared with 0.169 for Snowflake Arctic Embed L V2, an enterprise-grade embedding model with approximately seven times as many parameters as our model.
Table 4: Evaluation of WilBERT and Snowflake Arctic Embed L V2 on a combined dataset of Windows and Linux binaries.
Evaluating cross-architecture retrieval provides an especially useful measure of whether the model is learning architecture-independent properties of a function (semantic properties) rather than similarity between particular instruction sequences (syntactic properties). We define cross-architecture as the ability to identify a stripped binary function in a target CPU architecture e.g. AArch64 by knowing its source binary function in a different architecture e.g. x86. To that end, in Table 5, we evaluate on a combined cross-platform and cross-architecture dataset of x86, x86-64, and AArch64 binaries (index size of 22.2k and 33.5k for cross and same architecture respectively, with an equivalent number of query samples).
Table 5: Evaluation of WilBERT and Snowflake Arctic Embed L V2 on a combined dataset of Windows and Linux binaries of multiple architectures (x86, x86-64, AArch64).
On the cross-architecture dataset, WilBERT’s performance remains mostly stable across architectures, with Hit@1 falling by less than three percentage points, from 0.833 for same-architecture retrieval to 0.804 cross-architecture. By comparison, the (much) larger Snowflake model falls from 0.382 to 0.016. This suggests that WilBERT's embeddings capture semantic properties of each function's representation, rather than purely syntactic properties.
Table 6: Evaluation of WilBERT compared to Ghidra BSim and IDA Flirt on a dataset of 38,270 AArch64 Linux function pairs
Table 6 provides a further comparison of WilBERT against Ghidra BSim and IDA FLIRT where WilBERT outperforms both tools comfortably in a retrieval task. In particular, the poor results from IDA FLIRT reflect that it performs byte level matching and therefore is not designed for matching across different levels of optimisation, whereas Ghidra refuses to process certain samples if the binary graph is insufficient or the sample too short.
WilBERT and Ventris represent two complementary approaches to binary analysis in the absence of source code: learned matching in place of byte-level signatures, and learned decompilation in place of fixed transformation rules. Across our evaluations, both models achieve competitive or superior performance relative to commercially available tools, at significantly lower cost than modern LLM-based approaches.
Looking ahead, we have set ourselves a number of ambitious goals that push these models further for automated program analysis of potentially obfuscated code. We’re also working on expanding language coverage to Rust and Go, and supporting additional architectures such as MIPS, PowerPC, and RISC-V. We look forward to sharing further results as this work progresses.
We use essential cookies for security and site functionality, plus optional tracking cookies to understand how you use our site and improve it. Privacy policy.