Senior Data Engineer

OAG is the essential intelligence partner transforming global travel. For nearly a century, we've been the foundation the travel industry builds on, trusted, proven, essential. Today, we're going beyond. We combine unmatched data quality, quantity and access with breakthrough technology and deep industry expertise to deliver what the industry truly needs: not just more data, but better answers. From real-time operational intelligence to strategic decision-making, we power the entire travel ecosystem, from airlines and airports to technology companies and financial institutions. 

We are actively embedding AI throughout our intelligence layer, combining nearly a century of proprietary data with the latest technology to deliver answers that were simply not possible before. We exist because travel moves the world, and we move travel. By transforming complexity into clarity and questions into answers, we give our partners the intelligence they need to thrive in an evolving world. For more information visit www.oag.com 

ABOUT THE ROLE: 

You will join a team of software and data engineers modernizing OAG’s core Flight Status data platform. You will play a key role in building high-throughput, near real-time ingestion and processing pipelines that handle global flight status data using Azure Databricks and Delta Lake. As OAG actively embeds AI across its intelligence layer and adopts modern agentic development practices, you will leverage AI-assisted tools to accelerate delivery, improve system observability, and support AI-driven data products.

WHAT YOU’LL BE DOING: 

• Deliver Real-Time Data Pipelines: Build, scale, and maintain high-volume batch and Structured Streaming pipelines to ingest multi-source flight data into the Lakehouse platform.
• Drive Engineering Standards: Implement reusable data patterns, automated data quality controls, and observability to ensure data consistency, accuracy, and reliability.
• Optimize Platform Performance: Monitor and tune compute resources, streaming triggers, and storage for cost efficiency, low latency, and high scalability.   
• Automate Workflows & DevOps: Design automated job orchestration using Databricks Workflows and maintain CI/CD pipelines for reliable deployments.
• Leverage Modern AI Workflows: Integrate AI-assisted tools into daily engineering practices to boost productivity, edge-case testing, and pipeline automation.
• Collaborate & Lead: Work closely with product, analytics, and platform teams to define data contracts, review code, and mentor engineers across the domain.
  
  

WHAT WE’RE LOOKING FOR: 

• Senior Data Engineering Experience: Strong track record building and scaling production-grade streaming and batch ETL/ELT data pipelines.
• Databricks & Governance Mastery: Hands-on proficiency with Azure Databricks, Apache Spark (PySpark), Structured Streaming, Delta Lake, and Unity Catalog (for fine-grained access control, data governance, and lineage).
• Advanced Pipeline Design: Deep understanding of modern data patterns, including Medallion architecture, Change Data Capture (CDC), SCD Type 1/2, MERGE/upsert strategies, and schema evolution.
• Orchestration & Streaming: Practical experience building job dependencies with Databricks Workflows and integrating event streaming platforms (e.g., Azure Event Hubs, Kafka).
• DevOps & Observability Focus: Strong experience with automated testing, CI/CD tools (Azure DevOps or GitHub Actions), and data observability practices.
• AI Tooling Adoption: Active experience using AI coding assistants (e.g., GitHub Copilot, Cursor) to accelerate development while maintaining high code quality and ownership.
• Ownership & Collaboration: Strong analytical skills, a results-oriented mindset, and excellent technical communication skills.
  
  

IF YOU HAVE THE FOLLOWING THAT WOULD BE GREAT: 

• Databricks AI/ML Ecosystem: Experience with MLflow, Databricks AI Functions, or vector search in streaming/data workflows.
• Specialized Data Quality Tools: Hands-on experience with dedicated data quality and anomaly detection frameworks (e.g., Great Expectations, Soda).
• Infrastructure as Code (IaC): Practical experience with Terraform for cloud resource management.
• Industry Certifications: Databricks Certified Data Engineer Professional or Microsoft Certified: Azure Data Engineer Associate.
  
  

WE WILL OFFER YOU: 

🚀            A progressive, non-hierarchical culture where innovation drives everything we build

🏡            Flexible working arrangements that allow remote work or office attendance

💰            An attractive compensation and benefits package, including private health insurance from day one of   joining, a company bonus scheme and company pension scheme

🏖️            22 days holiday, growing with each year of service, volunteering days and an additional day off during your birthday month

👪            Inclusive family benefits designed to support you through parenthood and fertility journeys

📚            Annual learning and development budget

🌍            Work remotely anywhere in the world for up to 1 month per year

🍕            Free lunch and snacks provided in the office.

Monthly salary range from 4583 EUR to 8000 EUR gross (exact compensation may vary based on skills, experience and competencies).

OAG is committed to creating an inclusive workplace where everyone feels valued and able to perform at their best. 

We welcome applications from people of all backgrounds and experiences. If you require reasonable adjustments at any stage of the recruitment process, please let us know — we’re happy to have an open conversation about how we can support you.