Назад к вакансиям

Senior Software Engineer - Husky Storage

192 000 – 240 000 $в год
DatadogBoston и ещё 2 городаГибрид
JavaKubernetesKafkaGoDistributed systemsRustS3IcebergParquetData fusionArrowStorage Engines

Описание

Senior software engineer role on Datadog's Husky distributed petabyte-scale event storage team.

Responsibilities

  • Drive Husky to the next level of adoption by migrating our storage format to open source Parquet and re-imagine compaction on a format-agnostic, DataFusion-based substrate
  • Drive concrete cost-reduction work: block-level inverted index compression, combining the database with external objects at the row level, and storage tiering/caching
  • Scale our solution to enable customers to manage event versioning at scale, using temporal-database techniques
  • Play an influential role in operating and deciding how to best operate our service going forward
  • Work across the write path (with Ingestion) and toward a cleaner Iceberg-based handoff to ADP/Lakehouse for batch/analytical workloads

Requirements

  • Enthusiastic and experienced with large-scale distributed systems and/or storage engines
  • Comfortable with improving and operating large distributed systems that customers rely on for critical business functions
  • Strong fundamentals in columnar/file storage formats (Parquet, ORC, Arrow) or comparable systems

Nice to have

  • Experience with Go, Java, or Rust
  • Experience with compaction or compression/encoding schemes (RLE, dictionary, delta, gorilla)
  • Experience with object storage (S3/GCS/Azure Blob) at scale, including caching and cost optimization
  • Experience with Kafka and stream processing
  • Experience with Kubernetes

We offer

  • New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
  • Continuous professional development, product training, and career pathing
  • Intradepartmental mentor and buddy program for in-house networking
  • An inclusive company culture, ability to join our Community Guilds (Datadog employee resource groups)
  • Free, global mental health benefits for employees and dependents age 6+
  • Competitive salary and equity package, and may include variable compensation
  • Wide range of best in class, comprehensive and inclusive employee benefits including healthcare, dental, parental planning, and mental health benefits, a 401(k) plan and match, paid time off, fitness reimbursements, and a discounted employee stock purchase plan
  • Salary: $192,000 - $240,000 USD per year