Act as a staff data engineer designing a production extractor for a third-party SaaS API. Vendor docs are incomplete until proven. Prefer checkpointed, replayable pulls over a giant in-memory dump.
API and resources: [CRM, billing, ads, support, or named product]
Auth and tenancy: [OAuth, key, per-account]
Pagination style: [cursor, page/offset, time window, link header]
Rate limits and burst behavior: [known numbers or unknown]
Change detection: [updated_at, events, full snapshot only]
Volume and freshness SLA: [values]
PII and residency constraints: [details]
Current extractor notes or code:
[Paste]
Deliver:
1. Extraction unit of work (account, resource, time window) with a durable checkpoint
2. Pagination strategy that cannot skip or loop when a page mutates mid-pull
3. Rate-limit, backoff, and quota-budget policy including 429/5xx classes
4. Cursor reset, clock skew, and overlapping-window recovery
5. Idempotent landing-zone writes (raw JSON + extract metadata)
6. Schema-drift detection for additive, type-change, and missing-field cases
7. Secret handling, token refresh, and least-privilege scopes
8. Validation: record counts vs vendor totals, hash samples, freshness
9. Backfill vs incremental runbook with stop-the-line thresholds
10. Failure taxonomy: retryable, skip-with-alert, and poison-record quarantine
If the vendor has no stable updated_at, say so and design overlapping snapshots plus a merge key. Do not pretend offset pagination is safe on a mutating collection.