Volver a los puestos de trabajo
Data Engineering - Databricks Sr.
Madrid, Spain
NEORIS, now part of EPAM, is a Digital Accelerator helping companies step into the future. With more than 20 years of experience as the trusted Digital Partner of some of the world’s most recognized brands, we are a global team of 4,000+ professionals across 11 countries. Our multicultural and startup-driven culture fosters innovation, continuous learning, and the creation of high-impact solutions for our clients.
We are looking for: Senior Data Engineer – Azure Databricks
Main Responsibilities:
- Design, develop, and optimize end-to-end ETL/ELT data pipelines using Azure Databricks and Spark (SQL, PySpark).
- Build scalable ingestion processes for structured and semi-structured data from various sources including APIs, databases, and file systems.
- Implement complex data transformation logic and publish clean datasets to Azure Data Lake or other cloud storage for analytics consumption.
- Leverage Databricks Workflows and automation tools (e.g., Apache Airflow) for orchestrating and scheduling pipeline execution.
- Ensure best practices for performance tuning, including partitioning, caching, and efficient Spark configurations.
- Implement robust data governance and security practices using Unity Catalog and Azure-native capabilities, including encryption and RBAC.
Requirements:
Mandatory:
- +5 years of proven experience as a Data Engineer working with large-scale data processing in cloud environments.
- Expertise in building and optimizing data pipelines using Azure Databricks, Spark (SQL & PySpark), and Databricks Workflows.
- Strong hands-on experience with Azure Data Lake, Blob Storage, and Azure Synapse Analytics.
- Solid understanding of data governance practices and tools, including Databricks Unity Catalog.
- Demonstrated experience in performance tuning and optimization of Spark jobs.
- Working knowledge of infrastructure as code tools like Terraform and monitoring using Azure Monitor.
- English proficiency (B2 or above).
Nice to Have:
- Experience with Delta Lake for managing data versioning and incremental data loads.
- Familiarity with CI/CD pipelines and DevOps practices in Azure environments.
- Relevant certifications in Microsoft Azure or Databricks.
- Experience with Apache Airflow or similar orchestration tools.
We offer:
- Permanent contract with a competitive salary.
- Flexible work model with remote work possibilities.
- Personalized career plan and continuous learning (certifications, English training, etc.).
- Participation in stable, technically advanced projects.
- Flexible schedule and work-life balance culture.
- Social benefits tailored to your needs.
Visit us at www.neoris.com or connect with us on LinkedIn, Facebook, Twitter, and Instagram: @NEORIS
Mercedes Manzano
#LI-MM4
Crear una alerta de empleo
¿Le interesa desarrollar su carrera en NEORIS? Reciba futuras oportunidades directamente en su correo electrónico.
Solicitar este puesto
*
indica un campo obligatorio