Datamatics Technologies 7 дней назад

Senior Data Engineer / Data Pipeline Architect

  • Stockholm, Sweden
  • Гибрид
  • Опыт от 5 лет
  • Полная занятость
PythonKafkaAirflowSparkDBTBigQuery

Описание

Responsibilities

  • Design and drive the architecture, migrating pipelines incrementally while ensuring existing outputs remain unchanged.
  • Establish sustainable engineering processes for testing, code reviews, incident management, and releases.
  • Combine architectural leadership with hands-on engineering using Airflow, Spark, dbt, BigQuery, and Python.
  • Collaborate closely with and mentor a Data Engineer within the team, ensuring long-term ownership, reliability, and independent maintenance of the migrated pipelines.

Requirements

  • Minimum 5 years of relevant professional experience (5–10 years overall).
  • Deep hands-on experience with Airflow, Spark, dbt, BigQuery, and Python.
  • Proven experience technically leading at least one complete data pipeline migration from start to finish.
  • Experience refactoring production data pipelines while verifying that their outputs remain unchanged.
  • Experience establishing and implementing engineering processes for testing, code reviews, and incident management.
  • Demonstrated mentoring experience with measurable outcomes.
  • Minimum Bachelor's degree (or equivalent).

Nice to have

  • Experience with encryption at rest, data retention, or data deletion compliance (e.g., GDPR).
  • Experience with envelope encryption and key management solutions (e.g., GCP KMS, AEAD functions, or Tink).
  • Knowledge of BigQuery column-level security, policy tags, and Data Catalog.
  • Experience with Config Connector (KCC) for managing GCP resources through Kubernetes.
  • Experience with event-driven architectures using Kafka.

We offer

  • 6 months (Extendable)
  • Full-time
  • Hybrid (Travel Required)

Hiring process

  • Apply