Tech stories
Apache Airflow at Adyen: Our journey and challenges to achieve reliability at scale
By Natasha Shroff, Data Engineer; and Jorrick Sleijster, Data Platform Engineer.
In this blog post, we will share how we work towards reliability at scale. We answer the following questions that cover the challenges we have encountered on our journey:
Setting up a scalable multi-tenant Airflow setup
Preparing for the worst: Handling machine failures
Handling task priority
Maximizing user productivity
Fresh insights, straight to your inbox
By submitting your information you confirm that you have read Adyen's Privacy Policy and agree to the use of your data in all Adyen communications.