Ecowind 7 дней назад

Backend (Python) / Data Infrastructure Engineer

  • Worldwide
  • Удалённо
  • Полная занятость
PythonSQLPostgreSQLClickHouseDistributed systemsFastAPIremoteseniorSnowflakeMiddleData Infrastructureworldwide

Описание

Responsibilities

  • Develop and maintain backend services and APIs for interacting with different databases and query engines.
  • Build and improve integrations with Snowflake, DuckDB, Trino, ClickHouse, BigQuery, and other systems.
  • Develop mechanisms for SQL query routing, execution, and optimization across different query engines.
  • Work with SQL parsing, logical query plans, and SQL dialect transformations.
  • Ensure data consistency and result accuracy when executing queries across different systems.
  • Work with data storage technologies, including Apache Iceberg.
  • Develop automated benchmarking tools to test and compare different query engines in terms of execution speed, throughput, and cost.
  • Improve the platform's reliability, scalability, and performance.

Requirements

  • Practical experience in Software Engineering or Backend Development.
  • Commercial development experience with Python; experience with Rust is also a plus.
  • Experience working with databases, data warehouses, or analytical platforms.
  • Understanding of SQL and database architecture.
  • Hands-on experience with distributed systems.
  • Understanding of SQL query optimization principles and database performance tuning.
  • Understanding of ETL processes and data modeling.
  • Experience developing APIs, integrations, or connectors for working with different systems.

Nice to have

  • Experience with Snowflake, BigQuery, Databricks, DuckDB, Trino, or ClickHouse.
  • Experience with SQLGlot, Apache Calcite, Substrait, or SQL parsers.
  • Understanding of Query Federation, Query Routing, and Query Pushdown.
  • Experience with JDBC/ODBC, database connectors, or data migration between different systems.
  • Knowledge of Apache Iceberg, Delta Lake, or Apache Hudi.
  • Experience with data consistency validation and cross-database result comparison.
  • Contributions to open-source projects in the Data Infrastructure domain.
  • Experience working in early-stage startups.

We offer

  • Employment: Full-time
  • Work Format: Remote