CareeRoom вчера

Lead Data Platform Engineer

от 400 000 ₽в месяц
  • Москва
  • Гибрид
  • Полная занятость
PythonKubernetesGitLab CIFastAPIData EngineeringAirflowleadData PlatformCephбиоинформатика

Описание

Responsibilities

  • Design a scalable, reliable, and performant architecture for a unified data platform
  • Develop and maintain the platform's technical roadmap, synchronized with business goals and researchers' needs
  • Analyze and ensure the scalability, reliability, and performance of key components
  • Design and implement systems for orchestrating multi-stage data processing pipelines
  • Develop key platform components: control panel, orchestrators, agents, API, SDK
  • Create and develop self-service tools for data engineers and analysts
  • Ensure multi-tenancy, access control, and audit logging
  • Ensure data integrity, reliability, and security at all stages
  • Optimize the use of computing and storage resources (capacity planning, performance benchmarking)
  • Develop observability: monitoring, APM, alerting
  • Formulate and implement technical standards for the platform team
  • Systematize development processes, unify the stack and tools
  • Maintain technical documentation for platform services and architectural solutions

Requirements

  • Experience in designing horizontally scalable, fault-tolerant data-intensive systems on bare-metal/VM and/or in Kubernetes
  • Understanding of architectural trade-offs and the ability to choose solutions for specific loads
  • Strong proficiency in Python
  • Experience developing backend services and APIs using FastAPI
  • Experience with pipeline orchestrators: Airflow, Dagster, and others
  • Understanding of HTC/HPC systems
  • Experience with distributed storage systems: Ceph (CephFS), S3-compatible storage
  • Understanding of object and file storage for data-intensive workloads
  • Experience building CI/CD pipelines in GitLab
  • Ability to make and justify system architectural decisions
  • Experience in formulating technical standards: architectural principles, code review, engineering documentation, operational practices

Nice to have

  • Experience developing in Go
  • Deep understanding of Kubernetes: workloads, operators, network policies, security
  • Experience with schedulers like Slurm
  • Experience with Singularity/Apptainer containers
  • Experience in the biotechnology/bioinformatics domain and understanding of workload specifics

We offer

  • Formal employment according to the Labor Code of the Russian Federation
  • Competitive salary, discussed individually based on skills
  • Work in a rapidly growing, stable organization that is not afraid of crisis
  • A team of professionals at the intersection of advanced IT and science
  • Modern equipment for workstations
  • Opportunities for professional growth and training
  • 5/2 schedule, office in Moscow, hybrid possible after the probation period

Hiring process

  • Respond with a CV to ••• with the subject DATA