Описание

Ryanair Labs is hiring a Data Engineer in Madrid to design and maintain batch and real-time data pipelines. The role uses Spark, Airflow, dbt, Databricks, Kafka, and AWS to build and operate a governed data lake. Responsibilities also include monitoring, CI/CD, infrastructure support, and collaboration with data science teams.

Responsibilities

Design and develop automated ELT processes ensuring a high level of data quality and reliability, using tools such as Spark, Apache Airflow, DBT, Python, Databricks, etc.
Build and maintain batch and real-time data processing using Apache Kafka and AWS services like SNS, SQS, and Kinesis, supporting microservice architectures.
Maintain the data lake in AWS and Databricks, building well-governed, query-oriented data models for efficient consumption.
Implement monitoring and observability for the pipelines you work on.
Collaborate cross-functionally, assisting other teams in identifying, collecting, and processing valuable information for diverse business use cases.
Support the Data Science team by helping ensure stable, scalable environments for model deployment and experimentation.
Build and maintain CICD pipelines, following best practices for automatic deployment and consistency.
Help administer and maintain deployed infrastructure.
Follow and contribute to engineering standards and best practices across the Data Engineering team.
Deliver work on time, managing your own priorities in line with team commitments.

Requirements

3+ years in data engineering or related data processing roles.
Strong experience working with varied data formats and sources (JSON, CSV, Parquet, APIs, multiple DB engines).
Hands-on experience with data modelling in Big Data platforms (e.g. Databricks, Glue, Snowflake).
Familiarity with event-driven architectures and AWS.
At least 2 years of hands-on experience with AWS, especially: Core: IAM, S3, EC2, VPC; Data Services: EMR, Lambda, Batch, SNS, SQS, DynamoDB, Glue, Athena.
Ability to set up and maintain monitoring, alerting, and debugging solutions for your pipelines.
Experience collaborating with Data Science teams to support production-grade pipelines.
Solid understanding of computing systems (OS, memory, networks, etc.).
Proficient in data analysis and interpretation, with a good grasp of statistics and data quality methods.
Strong SQL and performance tuning skills.
Good knowledge of Big Data technologies, Spark, and object storage systems.
Proficiency in Python (and optionally Scala), with strong coding and testing practices.
Familiarity with Docker.
Airflow (or similar) for orchestration.
Good understanding of IaaC and CICD principles.
Dashboard and storytelling experience for data visualization.
Strong communication skills.
Proactive mindset and collaborative team spirit.
Comfortable working within an agile team setup.

Nice to have

Kubernetes experience is a plus.