Staff Infrastructure Engineer - Ingestion

Описание

Shopify is hiring a Staff Infrastructure Engineer for its data ingestion platform. The role focuses on designing, operating, and optimizing large-scale batch and streaming ingestion pipelines from operational databases and Kafka into analytics data stores. It includes cost optimization, reliability work, cross-team collaboration, mentoring, and on-call support.

Responsibilities

• Architect, optimize, and own Shopify's production ingestion platform
• Leverage technologies like Kafka, BigQuery, GCP, Dataflow, Terraform, and Kubernetes to deliver state-of-the-art performance and efficiency across both database and stream ingestion pipelines
• Partner with database platform, streaming platform, governance, and downstream analytics teams to seamlessly land, validate, and scale data products powering our platform
• Drive cost optimization and system efficiency
• Lead deep technical investigations across upstream and downstream data-store internals (indexing, replication, load patterns) to apply rigorous solutions
• Contribute to technical strategy and culture for the ingestion team, represent the team across Shopify, champion projects, mentor engineers across all levels, and collaborate with senior engineers across Data Platform
• Participate in the team's on-call rotation, supporting production reliability for ingestion systems

Requirements

• Proven, hands-on expertise in building and operating large-scale data infrastructure, with measurable wins in reliability, throughput, or cost
• Deep experience close to data stores
• Strong software engineering skills (Python, Java, and/or other relevant languages) with a robust systems and distributed computing mindset
• Demonstrated leadership in architecting or operating reliable data pipelines at scale
• Track record of cross-functional impact: working closely with database teams, streaming platform engineers, infra, and downstream data consumers to deliver production results
• Advanced understanding of streaming systems and the trade-offs between latency, throughput, and cost

Nice to have

• Experience with ingestion / streaming frameworks (Kafka Connect, Debezium, Beam, Flink, etc.) as part of a broader data infrastructure practice
• Experience optimizing pipelines across a variety of source systems (MySQL, distributed SQL like Yugabyte/Spanner/CockroachDB)
• Experience with modern data warehouse and lakehouse patterns (BigQuery, Snowflake, Iceberg)
• Prior experience in e-commerce, large-scale product infrastructure, or globally distributed data workloads
• Experience maintaining Scala services
• Fluency with AI coding tools (Claude Code, Cursor, etc.) as part of your daily workflow

Hiring process

• Be prepared to interview within the week
• The goal is to complete the entire interview loop within 30 days
• You will be expected to complete a live pair programming session, come prepared with your own IDE