API-first at Adyen

APIs have been playing a critical role in the expansion of the Fintech landscape, especially with Open Banking regulations changing the way businesses and consumers interact with financial data. The new workflows require open but secure systems that can provide a new range of services and products, accessible in a standard, modular and consistent way.

At Adyen the journey started several years ago and it has been going through various phases and iterations, geared around the opportunities and challenges of the payment world (take a look atBecoming API-First in the Payment Industryon YouTube), and resulting in different layers (internal and external) and families (payments, banking, platform) of REST APIs sitting on top of a large dynamic platform.

The outcome is a company-wide culture whereAPIs are products, teams are the ownersand everyone plays a part in shaping the API strategy.

...we will look at whatAPI-first means at Adyen, the API capabilities and strategy, and why we have adopted the OpenAPI specification.

TheStrawhecker Group(TSG), a globally recognized analytics, intelligence, and consulting firm in the payments industry, publishes annually theBest of Breed API Reportranking the API experience of payment gateways.

The API assessment is performed by developers with different levels of experience and payment knowledge. It looks at various aspects: functionality, documentation, integration experience. We are proud that our effort over the years has been recognised with the 2022 “Overall API Assessment” award.

The feedback is valuable and the award is nice, but we are looking beyond that. We have made APIs a key asset of our platform while embracing the API-first philosophy.

API-first is aproduct-centric visionwhere APIs are not just a technical solution, but are instead designed, managed and evolved as products. APIs are not limited at integrating producers and consumers, but they establish the foundations of a modern and robust platform.

From artifacts merely created by developers for developers, APIs become the end product of the work of a cross-functional team that involves product owners, designers, developers, security experts, etc..

APIs, just like any other product, have their own lifecycle:

They start with an idea (inception) which turns into a plan with stakeholders, goals and priorities. Throughout the design and development the teams might use different tools and methodology (agile works well), and deliver a working product that evolves and improves overtime but might also eventually retire.

The benefits of an API-first vision are everywhere. Teams develop better building blocks that fit within a larger context (without the need to understand the full enterprise architecture). They deploy faster and more frequently (ownership translates into confidence). They deliver services that are, by design, readily available to other applications.

APIs as products have a positive impact on thedeveloper experience, an essential feature for B2B platforms. This is why they tend to adopt open standards and think seriously about consistency. We focus on good documentation and never underestimate security and governance.

The mindset behind the shift from ‘API-as-tools’ to ‘API-as-products’ is the real deal. The commitment, effort and long-term strategy are the result of aculture forged by the peopleat all levels of the organization. At Adyen we see the full support from the executive-level that cascades down to leaders and teams. And, while it is important that everyone understands and embraces the API-first vision, there are key individuals (API champions) and groups (API Review group) that enable that vision through initiatives, engagement and knowledge sharing.

In theWhat is an API-First companyblog post by Postman Chief Evangelist we recognise several of the Adyen traits. But again we love to do things in our own unobvious way.

Governance sets the tone but doesn’t take all decisions. The API Review group recruits API stakeholders and champions to establish best practices and help teams to make good decisions (typically at design time) but does not dictate how and when things get done. We have seen many times that things work out the best when teams come together to discuss and collaborate rather than following pre-established patterns.

Discussions trigger decisions that are recorded in the internal ticketing and task management tool: this allows us to have specific tracks defined on what the design of APIs should take into consideration, who is involved and how progress is monitored.

Open standards (OpenAPI, REST, JSON) play a pivotal role, butour tech stack is open source or built in-house: we have created our own frameworks to empower development teams so they can iterate fast and efficiently in the context of our platform and products.

Documentation rules but it must also change fast and often. We have created theAPI Explorerto provide exhaustive documentation of the APIs together with JSON payload samples and additional resources.

We love open source: the OpenAPI specification files are published on Github. Our SDKs (Java, .Net, Node, Python, etc..) are also open source and developed for and with the communities of developers using them.

APIs are grouped in categories based on the product they serve.

All APIs implement versioning using a version suffix in the endpoint URL. This suffix has the following format: "vXX", where XX is the version number.

For example:

Consistent versioning is amustin the API-first approach: an API always requires a new version when breaking changes are introduced. We complement this with clear documentation, release announcements and communication with the merchants.

Deprecation is another essential practice that we use to flag in advance when API functionalities are going to be removed. It is consistently applied both to the OpenAPI specifications and the documentation while the deprecated elements are deleted only after one or more versions.

The OpenAPI specification marks, for example, the deprecated attribute (i.e. `mobilePhone`) and usesextensionsto add extra information that will help developers to understand more about the deprecation (is which version has the attribute been deprecated, helper message to suggest what to do).

We are fond of open standards but we do love good tools too.

We adoptedStoplightfor its design and documentation capabilities: the tool works nicely when creating new APIs from scratch. At that point the focus is on the design (not on source code) and Spotlight simplifies the process of converting the API needs and requirements into a design.

Postmanis the most loved API tool out there, and we see clearly why: API calls can be easily tested, organized in shareable collections and configured with variables (i.e. run on different environments). The creation of Postman collections from an OpenAPI file is very powerful: the specifications are imported and converted as Postman collections, shared with the internal teams and we are now working on publishing them externally for all developers working with the platform.

We useSpectral(by Stoplight) to verify the correctness of the OpenAPI files and their consistency with the API Style guide. It is nicely embedded in the CI/CD pipeline to lint the API specifications as they get generated.

Last but not least theOpenAPI generatorwhich has a significant impact in the maintenance of our SDKs. The tool, with over 13k stars on Github, allows the generation of API client libraries, server stubs, documentation and configuration automatically given an OpenAPI specification file.

TheOpenAPIspecification defines an open standard for describing REST APIs. Originally created and known as Swagger, it was donated in 2015 to the Linux Foundation to become the common language for API documentation. The standard has contributed to the creation of an API ecosystem which is now used by many developers and most API-driven organizations.

We haveadopted OpenAPIwith the main goal toimprove the developer experience. In doing that we have achieved many other benefits: our development pipeline streamlines the process in the most efficient way (still improving and tweaking as we speak).

The OpenAPI specifications are generated from the internal source code repositories, validated (lint) and published on ourAdyen OpenAPI Github repository. This pipeline makes sure the specifications are always “in sync” with the development (and refactoring) of the APIs without being afraid of creating discrepancies between code and documentation.

The API specs trigger different workflows and usage:

We built our ownAPI Explorerto deliver the best experience for developers working with our platform. The API Explorer ingests the OpenAPI specifications, enriched withcustom properties, to create an up-to-date, interactive and comprehensive environment for working with all our (public) APIs.

Every endpoint and model is described, code snippets are available, sample request and response payloads can be analyzed.

The OpenAPI files can be imported in different tools: For our own internal use case, we use Postman and it has been working awesome for us. The Postman collections are shared internally with our teams and are made available to anyone who works with the APIs.

Importing in SwaggerUI or Spotlight is also interesting as developers might prefer different tools or have other use cases.

The AdyenGithub repositoryis the home of all our open source SDKs. You can find the client libraries for all main languages (Java, .Net. Node, Go, Python, etc..) here that allow developers to integrate with the Adyen platform(instead of working directly with the REST APIs - which is still possible).

The libraries evolve fast and frequently. The main challenge is to reflect the API changes (new or updated endpoints, modifications to the models) in a consistent manner (no discrepancies) and pretty quickly (new API features must be promptly available in the SDKs).

Leveraging on theOpenAPI Code Generatorwe have implemented a pipeline forupdating the client libraries based on the OpenAPI specifications. When a new API version is released the OpenAPI file is published on Github. At that point the source code of the libraries gets updated (adding and modifying endpoints and models) using the Code Generator feature together with our custom templates.

OpenAPI codegen feature comes with a long list of generators for client-side and server-side code, and sometimes multiple frameworks are supported for the same language (i.e. for Java, think of Spring, RestEasy, Jersey, Apache HTTP client and others). Additionally each generator is bundled with a default implementation that will generate a working version of the client or server code you need.

The default implementation is defined using Mustache templates, a logic-less templating framework used to generate output in a certain structure and format. The templates, during the transformation from the OpenAPI file to source code, define how the method signatures are created, how documentation and comments are included, which implementation (method body) is provided, etc..

TheMustache templatescan be fetched from the OpenAPI Generator Github repository and customized to fit the organization and team needs. In the API clients we make sure certain HTTP headers are always supplied and the Adyen libraries are imported to reuse existing components.

Obviously there is more to it: next to the automated code generation, which saves lots of time and spares mistakes, there is other work to be considered. Adding unit testing is fundamental to keep the client libraries healthy and release new version with confidence. Opportunities for refactoring are endless and removing unused code is also a very important step of the process.

We have developed a large and deep API experience, both in the lifecycle management and in the technical implementation, that we want to share with developers and open source communities. We know that there is still room for improvement and a lot to learn, so we want to connect and engage with other API fans.

Are you an API guru? Or do you have a good story to share or questions to ask? Reach out to us on Twitter@AdyenDevs!