Large-Scale Backfill Planner
Data Engineering
Design a safe, restartable backfill that does not overwhelm production systems or violate SLAs.
Curated AI prompts for data engineering, software engineering, and AI engineering.
Data Engineering
Design a safe, restartable backfill that does not overwhelm production systems or violate SLAs.
Data Engineering
Size a streaming platform for peak traffic, ordering, lag, and failure recovery, with formulas you can defend and a load-test plan.
Data Engineering
Determine whether a streaming system delivers real end-to-end exactly-once outcomes, or only a framework marketing claim.
Data Engineering
Diagnose stragglers, spills, OOMs, and expensive shuffles in large Spark jobs with concrete next checks.
Data Engineering
Fix lakehouse small-file amplification while preserving ingestion latency, query scan cost, and safe concurrent writers.
Data Engineering
Diagnose why a warehouse query suddenly got slower or more expensive, with likely causes, checks to run, and practical remediations.
Data Engineering
Design auditable reconciliation controls for revenue, payments, or ledger pipelines, without silently netting unexplained gaps.
Data Engineering
Turn vague data reliability expectations into measurable SLOs, error budgets, and burn-rate alerts consumers can use.
Data Engineering
Find sensitive-data exposure and design least-privilege controls across a data platform.
Data Engineering
Build and test recovery plans for regional outages, corruption, and credential compromise.