Staff Backend Engineer (Ingestion Platform)

About the role

The Idukki ingestion platform is how every Instagram post, TikTok clip, X mention and YouTube Short reaches a customer dashboard. You own the pipeline end-to-end: connector lifecycle, rate-limit handling, dedupe, enrichment, durability, and the SLOs we publish to customers.

What you'll do

• Own ingestion SLOs: < 60s p95 from post → dashboard, 99.99% durability, < 0.1% dedupe miss-rate
• Scale the platform from 12M to 40M posts/day without a re-architecture
• Lead 3-engineer squad on the connector lifecycle (rate-limit drift, deprecations, capacity)
• Partner with AI/ML on the enrichment pipeline — auto-tagging, brand-safety, embeddings
• Run incident command for ingestion-side P0s; write the post-mortem

What you'll need

• 8+ years backend / distributed systems with at least one stint owning a > 1B-event/day pipeline
• Postgres tuning at scale (partitioning, logical replication, vacuum strategies)
• ClickHouse or another columnar warehouse in production
• Kafka / Kinesis / similar — you can talk about exactly-once delivery and back-pressure with conviction
• Demonstrable on-call leadership