Ecowind 7 дней назад
Backend (Python) / Data Infrastructure Engineer
- Worldwide
- Удалённо
- Полная занятость
PythonSQLPostgreSQLClickHouseDistributed systemsFastAPIremoteseniorSnowflakeMiddleData Infrastructureworldwide
Описание
Responsibilities
- Develop and maintain backend services and APIs for interacting with different databases and query engines.
- Build and improve integrations with Snowflake, DuckDB, Trino, ClickHouse, BigQuery, and other systems.
- Develop mechanisms for SQL query routing, execution, and optimization across different query engines.
- Work with SQL parsing, logical query plans, and SQL dialect transformations.
- Ensure data consistency and result accuracy when executing queries across different systems.
- Work with data storage technologies, including Apache Iceberg.
- Develop automated benchmarking tools to test and compare different query engines in terms of execution speed, throughput, and cost.
- Improve the platform's reliability, scalability, and performance.
Requirements
- Practical experience in Software Engineering or Backend Development.
- Commercial development experience with Python; experience with Rust is also a plus.
- Experience working with databases, data warehouses, or analytical platforms.
- Understanding of SQL and database architecture.
- Hands-on experience with distributed systems.
- Understanding of SQL query optimization principles and database performance tuning.
- Understanding of ETL processes and data modeling.
- Experience developing APIs, integrations, or connectors for working with different systems.
Nice to have
- Experience with Snowflake, BigQuery, Databricks, DuckDB, Trino, or ClickHouse.
- Experience with SQLGlot, Apache Calcite, Substrait, or SQL parsers.
- Understanding of Query Federation, Query Routing, and Query Pushdown.
- Experience with JDBC/ODBC, database connectors, or data migration between different systems.
- Knowledge of Apache Iceberg, Delta Lake, or Apache Hudi.
- Experience with data consistency validation and cross-database result comparison.
- Contributions to open-source projects in the Data Infrastructure domain.
- Experience working in early-stage startups.
We offer
- Employment: Full-time
- Work Format: Remote