Silverfort 4 часа назад
Senior Data Engineer
- Tel Aviv, Israel
- Опыт от 3 лет
PythonETLData EngineeringSparkApache SparkDatabricksIcebergLakehouseStreamingDelta LakeObject Storage
Описание
Responsibilities
- Collaborate as a strong team player within a dynamic, cross-functional environment
- Design, develop, and maintain scalable data models, Lakehouse architectures, pipelines, and ETL processes
- Enhance data workflows to support efficient real-time and batch processing
- Work closely with cross-functional teams to understand data requirements and deliver impactful solutions
- Stay up to date with the latest data engineering technologies and best practices, continuously improving our data platform
Requirements
- 6+ years of development experience, including at least 3 years as a Data Engineer
- Experience with distributed computing frameworks (e.g., Spark, Flink, EMR)
- Experience with Iceberg / Delta Lake / Databricks or similar technologies
- Experience designing scalable data storage solutions over object storage (structured and semi-structured data)
- Hands-on experience building data pipelines and ingestion systems (batch and/or streaming)
- Proficiency in Python and PySpark
- Experience in streaming systems and real-time data processing - an advantage
- Experience optimizing data processing performance for cost and efficiency - an advantage