Описание

This is a Senior Data Engineer position on a Data Platform team at inDrive, focusing on building and operating large-scale batch and streaming data infrastructure for analytics, machine learning, and CDC delivery. The role involves working with cutting-edge cloud technologies like GCP, AWS, BigQuery, Databricks, and Kubernetes.

Responsibilities

Build and operate batch and streaming ingestion into a layered BigQuery DWH (raw

ODS data marts) using Airflow, Debezium CDC over Kafka with protobuf, Pub/Sub, and Dataflow

Integrate external data sources end-to-end, including marketing platforms (GA4, AppsFlyer, TikTok/Meta/Google Ads), payment providers, S3 buckets, and third-party APIs
Engineer the data platform in Python, including custom Airflow operators and connectors, Kafka Connect on Strimzi (K8s), Cloud Functions, and API integrations
Build CI/CD and change-management tooling for BigQuery, such as GitHub-based test-and-approval flows, SQL migration engines (Liquibase/Flyway/Bytebase), sandbox validation, backup and rollback
Own reliability and correctness of pipelines: idempotency, deduplication, late-data handling, backfill and replay, freshness monitoring and alerting; write integration and unit tests
Drive data governance and compliance, including ITGC-compliant change management for BigQuery, IAM and least-privilege access, PII policy tags and DLP, Unity Catalog on Databricks, column-level lineage (OpenMetadata/Dataplex), and disaster-recovery planning
Build internal data tools and platform services for agentic workflows with data, such as Streamlit apps, Slack bots, LLM-based agents, and MCP servers
Support analysts and business teams with data requests, fostering data-driven decision-making
Contribute to system design and architecture with the development team

Requirements

Strong practical Python skills for clean, well-structured, and tested code for services, tooling, and data pipelines
Solid software design skills (OOP, modularity, design patterns)
Experience building and operating services in a cloud environment (GCP, AWS or similar), including CI/CD, containerization, monitoring and alerting
Familiarity with Kubernetes and Terraform
Hands-on experience with DWH-related tasks (BigQuery or another cloud warehouse) and confident working SQL
Clear communication with non-engineering stakeholders for data requests from analysts and business teams
Demonstrated ability to take ownership of technologies or services and proactively contribute ideas to the team

Nice to have

Advanced SQL skills: complex queries, window functions, partitioning, clustering, and cost optimization
Experience building reliable pipelines around CDC (e.g., Debezium): idempotency, schema evolution, backfills, and reconciliation
Analytical data modeling skills: table grain, facts vs dimensions, slowly changing dimensions, and metric definitions
Experience with stream processing frameworks such as Flink or Apache Beam/Dataflow
Exposure to data governance and audit compliance (ITGC/SOX), Databricks Unity Catalog, or lineage/catalog tooling (OpenMetadata, Dataplex)
Interest in building LLM-based agents and AI tooling for data

We offer

Help challenge injustice by creating fair choices for millions of people across 47 countries
Develop professional skills with access to mentoring, career consulting, and learning programs
Collaborate with teams around the world and gain international experience through the Global Talent Exchange Program
Engage in company-wide challenges, awards, sports activities, employee-led social impact and volunteering projects
Work alongside people who take initiative, speak openly, and challenge themselves to grow
Improve language skills through co-financed courses and internal speaking clubs
Final benefits may vary depending on the location