llms.txt won’t save your developers documentation, it's accelerating broken code snippets

Developer experience is no longer just a developer-only problem, it’s a problem for AI agents too.  Everyone in Developer Experience is rushing to ship llms.txt files and “Copy as a Markdown” buttons. At Adyen, we did it too, we stripped out unnecessary HTML, exposed raw markdown, and made documentation scraping easy for developers and large-language models with a single click.

It's progress, but AI agents also rely on quality context to get the job done. When that context is code, it is now even more important to keep that code up-to-date. Our SDK automation allows us to ship newer SDKs faster, but it also makes our SDK code snippets change faster over time. The cost of outdated code snippets has increased, along with the speed of discovering them.  Today, we are tackling this fundamental piece, something that every Developer Experience team struggles with: keeping code snippets up-to-date over a longer period of time.

Why give Large Language Models a chance at hallucinating?

Kwok He Chu

Developer Advocate at Adyen

Our Developer Experience team looked at the state of the code snippets, optimizing documentation for LLMs meant balancing out the following three priories:

1. Token efficiency: reducing file size and compressing information efficiently as models become more token-efficient.

2. Completeness: Including context-dependent fields and missing fields.

3. Accuracy: Enforcing versioning and snippets that compile in their respective language.

We chose to prioritize accuracy first, tackle completeness second, and leave the token efficiency for last. Why? Because humans and AI agents ‘fail’ differently. When a human developer hits a broken code snippet, they spend a chunk of their time checking the SDK source code. When an AI agent hits the resources indexed in llms.txt with broken code snippets, it assumes that syntax is correct, injects it in its context window, and will treat that as a fact in subsequent turns.

To understand the scale of effort, we analyzed the state of our code snippets across both docs.adyen.com and our API explorer.

Before our update in September 2026, we maintained 8,965 code snippets on our documentation site and 2,112 code snippets on the API Explorer.  We spotted two gaps that we wanted to tackle: 

1. 47.6% of documentation pages were missing code snippets in at least one supported language.

2. 183 different SDK versions coexisted across the site. While these older snippets compiled against older versions of our SDKs, almost none* compiled against our latest SDK releases. *Note: We set an overly strict standard for quality here. If a snippet has invalid import, unresolved namespaces, or incorrect field casing (e.g. camelCase or PascalCase), we’d mark it as broken. Every snippet had to become fully “copy-paste”-ready.

On the developer-usage side, between January and September 2026,  we logged over 10,000 clicks on our code snippet language selector, with nearly half  (4,800+) of those clicks on the “Copy Code”-button. That sent us a clear signal: developers are selecting their preferred coding language and copying their code snippets somewhere, most likely straight into their IDEs.

After our update, in October 2026, we updated our code snippets to align with our latest SDKs, while doubling our coverage in eight languages. 

1. Documentation: 8,965 → 17,709 code snippets (+97%)

2. API Explorer: 2,112 → 3,736 code snippets (+77%)

This update couldn’t have happened without a proper deterministic validation tool in place: a compiler. You cannot pray that a piece of code works… At least, I hope not. That’s why we prioritized validation first.

To ensure every snippet compiles, our validator CLI takes our code snippets across eight languages, and runs the native compiler that can perform two checks:

1. A static build-time analysis (ran by passing a --build-only flag): does the code compile against our latest SDK? Are namespaces, imports, and field types correctly formatted?  There are two outcomes:

   • A successful compilation

   • An unsuccessful compilation

2. A dynamic run-time analysis (ran by passing a --execute flag): does the snippet compile and successfully send an API request to the Adyen platform?  There are three outcomes:

   • A successful 2XX response from Adyen

   • An unsuccessful 3XX, 4XX, 5XX response from Adyen

   • A generic exception from the SDK, due to misconfiguration

Let’s take a look at a build error, taken from the log files that the validation CLI outputs.

        ---
        Build errors       
---
[1] ./src/snippets/paymentsetuprequest-049.java:23: 

error: cannot find symbol
PaymentSetupRequest paymentSetupRequest = new PaymentSetupRequest()
^
  symbol:   class __PaymentSetupRequest__
  location: class Snippet
./src/snippets/paymentsetuprequest-049.java:28: 
error: package PaymentSetupRequest does not exist
  .channel(PaymentSetupRequest.ChannelEnum.ANDROID)

  > Successful compilations: 0/1
  > 
  > Build errors: 1
  > 
  > Adyen.exceptions: 0
  > 
  > Generic exceptions: 0

      

First, the compiler throws a cannot find symbol error for PaymentSetupRequest, followed immediately by package PaymentSetupRequest does not exist when resolving PaymentSetupRequest.ChannelEnum.ANDROID. These static compilation failures typically stem from missing or moved  import statements. We mark it as a build error.

Let’s take another look at an Adyen exception error:

        ---------------------------
      Adyen exception      
---------------------------
[1] ./src/api-explorer-snippets/patch-transactionRules-transactionRuleId-updateTransactionRuleStatus.java 
Exception in thread "main" __ApiException__{error=class ApiError {
    invalidFields: null
    additionalData: null
    errorCode: __00_401__
    errorType: https://docs.adyen.com/errors/unauthorized
    message: null
    pspReference: null
    status: __401__
    title: Unauthorized
}, statusCode=401, message=null ErrorCode: __00_401__, responseHeaders={Content-Type=[application/json]}, responseBody={"type":"https:\/\/docs.adyen.com\/errors\/unauthorized","title":"__Unauthorized__","status":__401__,"errorCode":"__00_401__"}}
	at com.adyen.service.resource.Resource.executeRequest(Resource.java:173)
	at com.adyen.service.resource.Resource.request(Resource.java:118)
	at com.adyen.service.resource.Resource.request(Resource.java:94)
	at com.adyen.service.balanceplatform.TransactionRulesApi.updateTransactionRule(TransactionRulesApi.java:209)
	at main.Snippet.main(__patch-transactionRules-transactionRuleId-updateTransactionRuleStatus.java__:49)

  > Successful compilations: 1/1
  > 
  > Build errors: 0
  > 
  > Adyen.exceptions: 1
  > 
  > Generic exceptions: 0

      

The Adyen exception was logged, and it showed that the snippet compiled and executed successfully, reaching the Adyen platform API boundary, but failed at runtime with a 401 Unauthorized (error code 00_401). This happens when the endpoint requires a distinct API key configured with specific role permissions that this particular key does not have.

Looking at the two checks, you’ll notice that an exception is not always considered a failure. Example: a method_not_allowed response, would still count as a successful compilation. The validator can also catch generic exceptions before the request reaches the Adyen endpoint. This is typically caused by incorrectly configured classes/object instantiation.

This leaves us with the following validation loop:

• Find: Locates target code files by extension (.cs, .php, .rb, etc.).

• Replace: 

  • Swaps documentation placeholders (e.g. returnUrl, currencies) with test values

  • Swaps  API Keys, MerchantAccount names to authenticate the request against the Adyen TEST environment. 

• Wrap: 

  • Wraps code snippet in an async or main() function. 

  • Adds language-specific imports if needed.

• Run: compiles by calling the native compiler, interpreter, or type checker.

•  Evaluate: Classifies output into build errors, Adyen exceptions, or generic exceptions.

All that’s left is running it, here are the results:

We’ve seen the output, now let’s tackle the input next.

If you work backward from developer expectations, figuring out how to translate snippets from raw JSON can feel pretty straightforward on paper, until you account for language specific conventions, or changing SDK structure. cURL-requests are convenient to write or even automate to the documentation, but developers don’t write curl requests in their IDEs. We want every snippet to be 100% copy and paste-ready in an IDE, so developers don’t have to guess what the SDK equivalent should look like, while for LLMs, it can trust the snippet and treat it as a fact.

To see how this works in-practice, let’s go from from the desired output to the input, taking the following java snippet as an example:

        ```
// Adyen Java API Library v38.0.0 to latest
import com.adyen.Client;
import com.adyen.Config;
import com.adyen.enums.Environment;
import com.adyen.model.*;
import com.adyen.service.*;
import java.math.*;
import com.adyen.model.checkout.*;
import java.time.OffsetDateTime;
import java.util.*;
import com.adyen.model.RequestOptions;
import com.adyen.service.checkout.*;

Config config = new Config()
   .apiKey("YOUR_ADYEN_API_KEY")
   .environment(Environment.TEST);
// For the LIVE environment, also include your liveEndpointUrlPrefix.
Client client = new Client(config);

// Create the request object(s)
Amount amount = new Amount()
 .currency("USD")
 .value(420L);

CreateCheckoutSessionRequest createCheckoutSessionRequest = new CreateCheckoutSessionRequest()
 .reference("YOUR_MERCHANT_REFERENCE")
 .amount(amount)
 .merchantAccount("YOUR_ADYEN_MERCHANT_ACCOUNT")
 .countryCode("US")
 .channel(CreateCheckoutSessionRequest.ChannelEnum.WEB)
 .shopperEmail("shopper@adyen.com")
 .returnUrl("https://adyen.com/checkout");

// Send the request
PaymentsApi service = new PaymentsApi(client);
CreateCheckoutSessionResponse response = service.sessions(createCheckoutSessionRequest, new RequestOptions().idempotencyKey("UUID"));
```
      

The desired output is a fully functionally correct snippet containing the client configuration, client instantiation, request models, parameters, and complete import statements.

The input is a curl request: an endpoint, followed by some headers and a JSON payload, this is also what you can find in our OpenAPI Specifications, or manually written by our technical writing team.

        ```
curl -X POST 'https://checkout-test.adyen.com/v72/sessions' \
 -H 'Content-Type: application/json' \
 -H 'X-API-Key: YOUR_ADYEN_API_KEY' \
 -d '{
   "merchantAccount": "YOUR_ADYEN_MERCHANT_ACCOUNT",
   "amount": {
     "value": 420,
     "currency": "USD"
   },
   "countryCode": "US",
   "channel": "Web",
   "returnUrl": "https://adyen.com/checkout",
   "reference": "YOUR_MERCHANT_REFERENCE",
   "shopperEmail": "shopper@adyen.com"
 }'
```
      

So, how do we go from input to output?

Years ago, we built an internal CLI tool that relied on strict template rules, regex pattern matching, casing conversions (camelCase, PascalCase), and OpenAPI specifications. That’s what got us most of the way there, but as our SDKs evolved, edge cases started popping up: forwarded HTTP headers, query parameters, complex configurations, and language-specific idioms broke our static template assumptions. It worked well for simple endpoints. 

We looked at the two main approaches, to design a better translation tool:

1. The string-template replacement approach: Using OpenAPI specifications with generator templates via SDK automation. 

2. The Abstract Syntax Tree (AST) traversal approach: Parsing generated SDK libraries directly into an Abstract Syntax Tree (AST) to build a reverse JSON symbol map for O(1) lookups. This allows the translation tool to resolve the exact model classes, types, and method signatures from the compiled SDK source code.

We transitioned from string templates to AST traversal. By parsing the compiled SDK source code directly, our pipeline extracts the structural layout of every class, method, argument, and namespace. For example, when parsing CreatePermitRequest in our Java SDK v41.1.1, the compiler generates the following AST representation (*simplified below):

        ```
AST
├── PackageDeclaration (name: "com.adyen.model.recurring")
└── ClassOrInterfaceDeclaration (name: "CreatePermitRequest", public)
    │
    ├── FieldDeclaration (name: "merchantAccount", type: "String", private)
    ├── FieldDeclaration (name: "permits", type: "List<Permit>", private)
    │
    ├── MethodDeclaration (name: "merchantAccount")
    │   ├── ReturnType: ClassType ("CreatePermitRequest")
    │   ├── Parameter (name: "merchantAccount", type: "String")
    │   └── Body: BlockStmt
    │       ├── ExpressionStmt
    │       │   └── AssignExpr (operator: ASSIGN)
    │       │       ├── Target: FieldAccessExpr ("this.merchantAccount")
    │       │       └── Value: SimpleName ("merchantAccount")
    │       └── ReturnStmt
    │           └── ThisExpr ("this")
    │
    └── MethodDeclaration (name: "addPermitsItem")
        ├── ReturnType: ClassType ("CreatePermitRequest")
        ├── Parameter (name: "permitsItem", type: "com.adyen.model.recurring.Permit")
        └── Body: BlockStmt
            ├── IfStmt (condition: "this.permits == null")
            │   └── Then: AssignExpr ("this.permits = new ArrayList<>()")
            ├── ExpressionStmt
            │   └── MethodCallExpr ("this.permits.add")
            │       └── Argument: SimpleName ("permitsItem")
            └── ReturnStmt
                └── ThisExpr ("this")
```
      

We then transform this AST into a lightweight JSON, that’s also humanly readable.

        ```
{
  "library-name": "adyen-java-api-library"
  "library-version": "v41.1.1",
  "com.adyen.model.recurring.CreatePermitRequest": {
    "name": "CreatePermitRequest",
    "namespace": "com.adyen.model.recurring",
    "functions": {
      "merchantAccount(merchantAccount)": {
        "arguments": [{"name": "merchantAccount", "class": "String"}]
      },
      "getMerchantAccount()": {
        "arguments": []
      },
      "setMerchantAccount(merchantAccount)": {
        "arguments": [{"name": "merchantAccount", "class": "String"}]
      },
      "permits(permits)": {
        "arguments": [{"name": "permits", "class": "java.util.List<Permit>"}]
      },
      "addPermitsItem(permitsItem)": {
        "arguments": [{"name": "permitsItem", "class": "com.adyen.model.recurring.Permit"}]
      },
      "getPermits()": {
        "arguments": []
      },
      "setPermits(permits)": {
        "arguments": [{"name": "permits", "class": "java.util.List<Permit>"}]
      },
      "recurringDetailReference(recurringDetailReference)": {
        "arguments": [{"name": "recurringDetailReference", "class": "String"}]
      },
      "getRecurringDetailReference()": {
        "arguments": []
      },
      "setRecurringDetailReference(recurringDetailReference)": {
        "arguments": [{"name": "recurringDetailReference", "class": "String"}]
      },
      "shopperReference(shopperReference)": {
        "arguments": [{"name": "shopperReference", "class": "String"}]
      },
      "getShopperReference()": {
        "arguments": []
      },
      "setShopperReference(shopperReference)": {
        "arguments": [{"name": "shopperReference", "class": "String"}]
      },
      "setIncludeNullValues(includeNullValues)": {
        "arguments": [{"name": "includeNullValues", "class": "boolean"}]
      },
      "equals(o)": {
        "arguments": [{"name": "o", "class": "Object"}]
      },
      "toString()": {
        "arguments": []
      },
      "toJson()": {
        "arguments": []
      }
    }
  }
}
```
      

The result: instead of maintaining regex rules to do string transformations, whether it's camelCasing, or PascalCasing, we can do the look-ups when translating the JSON payload. If you ask our translation tool: "Does CreatePermitRequest exist in v41.1.1 of our Java SDK, and what is its exact method signature?" We can just call a function that performs an O(1) lookup to confirm the class namespace, argument types, or builder methods directly from the compiled SDK. If the JSON payload doesn’t contain the key, it simply doesn’t exist.

This update is part of our commitment to building structured infrastructure for AI-driven development. Standardization is the key to this transition. We are following the same philosophy as our Model Context Protocol (MCP) efforts: standardizing how applications provide context to LLMs to support secure, and efficient integrations.

Our Developer Experience team will continue to address these foundations in order to make our documentation and API explorer more AI-ready than it was before. When AI agents find broken code in documentation, it treats those invalid syntax patterns as facts in their context window, especially when shipped as part of llms.txt. We addressed this challenge across 17,000+ code snippets by prioritizing accuracy first, completeness second, and token efficiency last. We started with validation and upgraded our translation tools, hitting a 92% pass rate.

By serving standardized, structured, verified context directly into AI agents and developer IDEs, we eliminate hallucinated methods, invalid namespaces and outdated fields. To explore our full suite of AI-ready resources, download the complete  document mirror (.zip).

We are excited to see how this changes the way you build integrations with the Adyen platform, so stay tuned for more updates as we continue to share what we’re doing in the (Agent) Developer Experience space.