Silverfort 4 часа назад

Senior Data Engineer

  • Tel Aviv, Israel
  • Опыт от 3 лет
PythonETLData EngineeringSparkApache SparkDatabricksIcebergLakehouseStreamingDelta LakeObject Storage

Описание

Responsibilities

  • Collaborate as a strong team player within a dynamic, cross-functional environment
  • Design, develop, and maintain scalable data models, Lakehouse architectures, pipelines, and ETL processes
  • Enhance data workflows to support efficient real-time and batch processing
  • Work closely with cross-functional teams to understand data requirements and deliver impactful solutions
  • Stay up to date with the latest data engineering technologies and best practices, continuously improving our data platform

Requirements

  • 6+ years of development experience, including at least 3 years as a Data Engineer
  • Experience with distributed computing frameworks (e.g., Spark, Flink, EMR)
  • Experience with Iceberg / Delta Lake / Databricks or similar technologies
  • Experience designing scalable data storage solutions over object storage (structured and semi-structured data)
  • Hands-on experience building data pipelines and ingestion systems (batch and/or streaming)
  • Proficiency in Python and PySpark
  • Experience in streaming systems and real-time data processing - an advantage
  • Experience optimizing data processing performance for cost and efficiency - an advantage