MARGO 4 часа назад

Senior Data Engineer

180 – 200 ₽в час
  • Удалённо
  • Опыт от 6 лет
PythonSQLData EngineeringAzurePower BITableauApache SparkDatabricksApache AirflowData WarehouseAzure Data FactoryUnity Catalog

Описание

Responsibilities

  • Hands-on Data Product development
  • Implement scalable, reliable and reusable data pipelines using Azure Data Factory and Apache Airflow.
  • Develop data transformations in Azure Databricks (PySpark / SQL) to create/extend data products following the Medallion Architecture layers: Bronze (raw ingestion), Silver (cleansed & conformed), Gold (business-ready data products)
  • Optimize performance, cost, and reliability of data workloads.
  • Raise important questions during business maturation in order to detect risks before development stage.
  • Technical Leadership
  • Participate in architecture reviews and technical decision-making.
  • Promote engineering excellence through code reviews and design reviews.
  • Collaborate with Architects, Product Owners in data product technical maturation.
  • Support & Operations (Run Team Collaboration)
  • Provide support to the Run/Operations team for production pipelines and data products for complex topics.
  • Troubleshoot and resolve data incidents, failures, and performance issues
  • Participate in incident management, root cause analysis, and post-mortems.
  • Ensure proper handover, documentation, and operational readiness of developed solutions.
  • Contribute to improving runbooks, monitoring, and operational processes.

Requirements

  • 6+ years of experience in data engineering
  • Strong technical expertise with: Python (PySpark), SQL (advanced level)
  • Hands-on experience with Azure-based Data Platform including: Databricks, Unity Catalog, Apache Airflow, Azure Data Factory
  • Reporting and BI tools (Power BI, Tableau or similar)
  • Solid understanding of data modelling fundamentals, main architectures and design patterns: Data Warehouse, Data Lake and Data Lakehouse architectures, Medallion architecture, Metadata-driven architecture, ETL/ELT design patterns
  • Understanding of: Partitioning, optimization, and performance, Batch and streaming pipelines

We offer

  • 180 – 200 PLN per hour