Accel Club вчера
Senior Data Engineer
- Удалённо
- Опыт от 5 лет
- Полная занятость
PythonSQLE-commerceAirflowPlaywrightDBTBigQueryApache BeamAmazon SP-APIGoogle Cloud Dataflow
Описание
Responsibilities
- Build and maintain integrations with marketplace, advertising and business systems, including Amazon SP-API, Amazon Ads, TikTok and Odoo.
- Develop Apache Beam pipelines on Google Cloud Dataflow to process source data into BigQuery.
- Build and maintain Airflow workflows for pipeline orchestration.
- Develop dbt models supporting financial reporting, advertising and inventory planning.
- Maintain Playwright-based scraping and report collection, parsing results into validated datasets and adapting to changes in source pages and formats.
- Improve data quality through testing, reconciliation, freshness monitoring and actionable alerts.
- Investigate discrepancies across source systems, warehouse models and business reports.
- Deliver safe migrations and backfills while improving pipeline performance and cloud costs.
Requirements
- Strong production experience with Python and SQL.
- Hands-on experience developing and operating Apache Beam pipelines on Google Cloud Dataflow.
- Practical experience with BigQuery, including data modelling, query performance and cost optimisation.
- Experience building and operating Airflow workflows.
- Experience developing and maintaining dbt models.
- Strong understanding of incremental processing, deduplication, late-arriving data and safe reprocessing.
- Careful reasoning about dataset grain, joins, metric definitions and historical revisions. You can demonstrate how you verify data completeness and accuracy.
- Independent ownership of production pipelines, including troubleshooting and incident resolution.
- Effective use of AI coding tools, with responsibility for reviewing and verifying their output.
- Clear communication with technical and business colleagues, with English sufficient for daily collaboration.
Nice to have
- Amazon SP-API, Amazon Ads API or Marketing Stream.
- Playwright-based browser automation, scraping and structured data extraction.
- E-commerce, advertising, financial or inventory data.
- Odoo integrations, PostgreSQL or change data capture.
- Terraform/Terragrunt or AWS messaging services.
We offer
- Удалёнка (EMEA)
Hiring process
- Откликнуться на вакансию