Datamatics Technologies 7 дней назад
Senior Data Engineer / Data Pipeline Architect
- Stockholm, Sweden
- Гибрид
- Опыт от 5 лет
- Полная занятость
PythonKafkaAirflowSparkDBTBigQuery
Описание
Responsibilities
- Design and drive the architecture, migrating pipelines incrementally while ensuring existing outputs remain unchanged.
- Establish sustainable engineering processes for testing, code reviews, incident management, and releases.
- Combine architectural leadership with hands-on engineering using Airflow, Spark, dbt, BigQuery, and Python.
- Collaborate closely with and mentor a Data Engineer within the team, ensuring long-term ownership, reliability, and independent maintenance of the migrated pipelines.
Requirements
- Minimum 5 years of relevant professional experience (5–10 years overall).
- Deep hands-on experience with Airflow, Spark, dbt, BigQuery, and Python.
- Proven experience technically leading at least one complete data pipeline migration from start to finish.
- Experience refactoring production data pipelines while verifying that their outputs remain unchanged.
- Experience establishing and implementing engineering processes for testing, code reviews, and incident management.
- Demonstrated mentoring experience with measurable outcomes.
- Minimum Bachelor's degree (or equivalent).
Nice to have
- Experience with encryption at rest, data retention, or data deletion compliance (e.g., GDPR).
- Experience with envelope encryption and key management solutions (e.g., GCP KMS, AEAD functions, or Tink).
- Knowledge of BigQuery column-level security, policy tags, and Data Catalog.
- Experience with Config Connector (KCC) for managing GCP resources through Kubernetes.
- Experience with event-driven architectures using Kafka.
We offer
- 6 months (Extendable)
- Full-time
- Hybrid (Travel Required)
Hiring process
- Apply