Назад к вакансиям

Senior Data Engineer

qlubBengaluru и ещё 1 городУдалённоот 5 лет
PythonSQLPostgreSQLCI/CDClickHouseAirflowGCPDBTBigQuery

Описание

Responsibilities

  • Own how data gets in: CDC from our transactional databases, SaaS sources, application events, and third-party APIs.
  • Handle the unglamorous parts well: backfills, retries, schema changes, late-arriving data.
  • Own the dbt project end to end: staging through marts, tests, docs, incremental models, and the naming and contract conventions everyone follows.
  • Keep BigQuery fast and cheap: partitioning, clustering, query tuning, and cost monitoring.
  • Serve low-latency analytics from an OLAP store such as ClickHouse: table design, sorting keys, materialized views, aggregation strategy.
  • Look after the GCP estate around the warehouse: Cloud Storage, Cloud Run services, IAM, and the occasional read from sources hosted elsewhere.
  • Run production Airflow: scheduling, dependencies, retries, alerting, and the operational habits that go with it.
  • Put freshness and quality checks in place, and fix root causes when they fire rather than restarting the job.
  • Set up CI/CD and tests for data code so changes ship safely.
  • Find where LLMs and agents genuinely help (data discovery, quality monitoring, troubleshooting), and ship those.
  • Review code, mentor junior and mid-level engineers, and write down the decisions and runbooks the team will need later.

Requirements

  • 5+ years in data or backend engineering, most of it on production systems.
  • Strong Python, with years of writing and debugging production pipelines and services.
  • Advanced SQL: window functions, complex transformations, and knowing why a query is slow.
  • 2+ years of production dbt experience, including incremental models, macros, tests and docs.
  • Experience running production Airflow DAGs, including backfills, monitoring and troubleshooting when things break.
  • Strong BigQuery skills: modelling, partitioning, clustering, and query and cost tuning.
  • Good knowledge of GCP, including Cloud Storage, IAM and general cloud data infrastructure.
  • Solid PostgreSQL experience: indexing, query performance and incremental extraction.
  • Experience with stream or distributed processing, such as Apache Flink or similar.
  • Comfortable working on a Linux VM: Bash, cron, Docker, reading logs, SSH and basic networking.
  • Strong engineering practice with Git, CI/CD, tests and code review, writing code other people can maintain.

Nice to have

  • Experience with ClickHouse or production OLAP: tuning, materialized views and cluster operations.
  • Experience with managed ETL/ELT platforms such as Airbyte or Fivetran.
  • Experience with Datastream for CDC and database replication.
  • Experience with Kafka and Debezium for event streaming and change data capture.
  • Experience with Kubernetes and Terraform.
  • Experience with BI tools such as Tableau or Power BI, and a feel for what makes dashboards slow.
  • Experience with LLM APIs, RAG and agent frameworks.
  • Experience with data observability tools such as Soda, Great Expectations or Monte Carlo.
  • Experience with AWS (EC2, S3, RDS).
  • A CS or engineering degree, though a strong track record matters more.

We offer

  • Real ownership over products that impact millions of diners and thousands of restaurants globally
  • The opportunity to solve challenging problems in payments, hospitality, and customer experience
  • A fast-moving environment where great ideas quickly become reality
  • Close collaboration with experienced operators, builders, and industry experts
  • A chance to leave a visible mark on one of the fastest-growing restaurant technology companies globally