Senior Software Engineer, Data Engineering

Описание

The Data team at Todyl owns the systems that turn high-volume security telemetry into something the rest of the platform and our partners can act on, designing and operating ingestion pipelines, storage, and search and analytics layer that power detection, investigation, and business intelligence.

Responsibilities

• Design and implement data ingestion pipelines that pull from many sources through Kafka-compatible streaming into Quickwit and ClickHouse for real-time analytics, search, and visualization, optimized for scalability, reliability, and fault tolerance under high volume and velocity
• Own complex features and projects end to end, from an ambiguous problem statement through to long-term maintenance, breaking that ambiguity into achievable milestones and estimating large epics with reasonable accuracy
• Design and tune the storage layer: index and table design, partitioning, retention, and query performance, weighing trade-offs across cost, scale, security, and maintainability, and documenting designs the team will reference long after they ship
• Build monitoring, alerting, and debugging into what you design so we catch correctness and freshness issues early, and measurably reduce how long it takes us to detect and resolve them
• Participate in the team's on-call rotation, resolve most issues independently, and follow up on incidents and the toil they expose rather than leaving that work unfinished
• Partner with the AI/ML and product teams at the design phase of the systems you own rather than reviewing them the week before launch, and serve fast, correct queries to the teams that build on our data
• Mentor engineers on the team through pairing, design partnership, and code review, and lead thorough design reviews
• Proactively reduce technical debt across multiple projects a year and take initiative on important work nobody has assigned
• Treat code review and testing as quality levers, not checkboxes
• Design inherently testable systems with comprehensive coverage, catch testing gaps in other people's designs, and watch dashboards and logs to verify your own changes after they ship
• Run retrospectives with genuine follow-through, document the patterns you find, and build onboarding material that makes the next person faster

Requirements

• 5+ years of software engineering, with 2+ years building and operating data infrastructure in production
• Production experience with a streaming platform: Kafka, Redpanda, Kinesis or a similar log-based system, having built or operated producers and consumers, and dealt with consumer lag
• Production experience with at least one search or analytics store: ClickHouse, Elasticsearch, OpenSearch or Quickwit, having designed indexes or tables and tuned queries that were too slow
• Strong Go or Python, and strong SQL
• Demonstrated experience owning complex data features or projects end to end, through to production and long-term maintenance
• Experience with high data volume, and the ability to explain the cost and performance trade-offs you made
• Recognized as the go-to expert in their technical domain, designs complex features with minimal rework, and debugs the hardest issues their team faces
• Strong sense of ownership and stays with the work through stabilization, not just through a successful deploy, owning what they build through its full lifecycle
• Makes sound architecture decisions for multi-sprint work, evaluates several approaches before choosing a direction, and explicitly balances technical debt against delivery speed with clear reasoning
• Designs for failure modes and graceful degradation, anticipates significant scaling needs, and builds monitoring, alerting, and debugging tools into their designs rather than adding them later
• Anticipates problems before they happen and proposes solutions proactively, thinking roughly 6 months ahead
• Mentors other engineers with real, visible improvement in their skills, and leads thorough design reviews
• Communicates technical concepts clearly to both engineers and non-technical stakeholders, and quantifies the business impact of their technical work
• Actively uses AI tooling in their day-to-day work
• Thrives in a fast-paced startup environment and is comfortable adapting existing systems to fast-changing requirements
• Expected to actively use AI tools, including tools like Claude, to accelerate development, reduce toil, and solve data engineering problems more quickly

Nice to have

• Quickwit specifically
• Columnar formats and object storage (Parquet, Iceberg, S3)
• Log shippers such as Vector or Fluent Bit
• Observability for data systems (Grafana, Prometheus)
• Multi-tenant SaaS, AWS, and Linux
• Security telemetry or SIEM experience

We offer

• Comprehensive benefits including medical, dental, and vision coverage
• Health savings and flexible spending accounts (HSA/FSA)
• Life insurance
• Short- and long-term disability
• Access to on-demand healthcare and telehealth services
• Employee Assistance Program (EAP)
• Flexible PTO in addition to 13 company holidays
• 401(k)
• Generous parental leave programs
• The salary range for this position is $170,000-$190,000, plus equity
• The actual annual salary for this role will depend on each candidate's experience, qualifications, and work location, with most new hires placed near the midpoint of the posted range to ensure fairness and consistency across our team
• Based in Denver, CO, with 3 days per week in our office