Data Engineer Cloud Google Cloud Platform / Amazon Web Services
TalanBordeaux и ещё 2 городаГибридот 4 летПолная занятость
PythonSQLKafkaAWSETLGCPSparkDatabricksData WarehouseNiFi
Описание
Responsibilities
- Design, develop and maintain robust, scalable and industrialized data pipelines.
- Develop, transform, manipulate and optimize data using Python and SQL.
- Implement distributed processing of large data volumes with Apache Spark, PySpark and Databricks.
- Contribute to streaming and real-time processing architectures around Apache Kafka and Apache NiFi.
- Work on cloud data and Data Warehouse environments, notably AWS Glue and Redshift or GCP BigQuery.
- Participate in the construction of modern data platforms, such as data warehouse, data lake or lakehouse, according to client needs.
- Ensure the quality, performance, reliability and security of data throughout its lifecycle.
Requirements
- Hold a degree in computer science at Bac+5 level.
- Have at least 4 years of experience in data platform design and maintenance projects.
- Be comfortable communicating with business stakeholders, understanding their needs and proposing pragmatic, robust solutions adapted to client challenges.
- Master Python and SQL for developing, transforming, manipulating and optimizing data.
- Have good experience with Big Data environments, particularly Apache Spark, PySpark and Databricks for distributed data processing.
- Be familiar with streaming or real-time processing architectures, with Apache Kafka and/or Apache NiFi.
- Have experience working on cloud data or Data Warehouse environments, ideally GCP BigQuery and AWS Glue, Redshift.
- Know the principles of ETL/ELT, data quality, performance optimization and processing industrialization.
Nice to have
- Knowledge in data governance and Domain Driven Design (DDD).
We offer
- Modern offices in Bordeaux near the city center.
- Hybrid organization between teleworking and on-site presence depending on client projects.
- A wide range of training (catalog of over 1,500 courses).
- Health insurance covered at 80%.
- Possibility of returning from maternity leave at 4/5ths without salary loss for 6 months.
Hiring process
- First discussion with Quentin, recruiter, to present the position and understand your career plan.
- Joint interview with the Data & IA team manager and a technical consultant.