Описание

This Principal Data Engineering role at Western Digital focuses on building ML-ready scientific data pipelines, feature engineering, data quality, versioning, lineage, and MLOps data infrastructure, dealing with experimental measurement data from precision product development instruments.

Responsibilities

Build and maintain reliable, versioned feature engineering pipelines that transform raw engineering, sensor, and operational data into structured ML-ready feature sets, delivered to specification
Design and operate data quality checks covering completeness, schema consistency, statistical distribution stability, and label accuracy across all AI training datasets
Alert the ML team when data quality degrades before it impacts model training
Collaborate with the team who performs final downstream validation
Build and maintain training data versioning and lineage tracking, ensuring full reproducibility of all model training runs and early alerting when deployment data diverges from training distributions
Implement and maintain agreed data contracts between upstream data producers and downstream ML consumers, following governance standards established with guidance from ML Engineer
Establish access control and retention practices for all AI data assets
Contribute to real-time sensor data ingestion pipelines under technical direction, progressively developing operational ownership over 6–12 months
Build pipeline infrastructure to operationalize synthetic data generation workstreams, including ingestion, storage, and versioning infrastructure with generative model methodology provided
Build and maintain the training dataset registry, feature store, and model input validation, tightly integrated with the AI platform (AWS Kubernetes, PortKey, Agent Gateway, LangFuse, AWS Guardrails, Elastic Search etc.)

Requirements

Bachelor's or Master's degree in AI, Computer Science, Data Engineering, Electrical Engineering, Applied Mathematics, or related field; AI major preferred; strong data engineering fundamentals required
At least 2-3 years demonstrated project experience building end-to-end data pipelines (academic, personal, or internship contexts) is the primary evaluation criterion
Solid and demonstrable Python, SQL, and pipeline design fundamentals through project evidence
Strong proficiency in Python for primary pipeline development
Strong proficiency in SQL, including complex queries, window functions, and data transformation logic
Experience with scalable batch pipeline architecture, reliability, schema management, fault tolerance
Experience with pipeline orchestration (Airflow, Prefect, AWS Glue Jobs)
Understanding of data quality principles: completeness checks, schema validation, distribution stability monitoring
Experience with data versioning and lineage to ensure reproducibility of training data and ability to trace data origin and transformations
Basic understanding of how data pipelines connect to ML model training
Awareness that data quality and pipeline design affect model performance downstream, specifically awareness of risks like train/test data leakage and label quality impact on model accuracy

Nice to have

Experience with feature store (Feast, Tecton)
Experience with synthetic data generation (VAE, GAN, diffusion models)
Experience with Data Built Tool (DBT)
Experience with Data Load Tool DLT
Experience with annotation platform integration (Label Studio, CVAT)
Experience with MES / LIMS system integration
Experience with active learning data loop design
Experience with data lakehouse (Iceberg, Dremio, AWS Glue, AWS Lake Formation) / Redshift

We offer

Western Digital is committed to offering opportunities to applicants with disabilities and ensuring all candidates can successfully navigate careers website and hiring process
Western Digital is committed to an inclusive environment where every individual can thrive through a sense of belonging, respect and contribution

Hiring process

Contact Western Digital via email to advise of accommodation requests, including a description of the specific accommodation, job title, and requisition number