Buy versus build for ingestion, decided per connector

Ingestion debates go wrong by being singular: "should we buy or build our ingestion" assumes ingestion is one thing. An estate is typically thirty connectors, and the correct answer differs across them, connector by connector.

Buy the long tail. The fifteen SaaS sources (CRM, ads, support, HR) change their APIs on their schedule, not yours. A managed connector service's entire business is chasing those changes; your engineer's business is not. Managed connectors from services like Airbyte or its commercial peers turn API churn into a subscription line. The math fails only at volume: metered pricing on your largest sources deserves its own scrutiny, because row-based billing on a firehose is how ingestion becomes the second biggest platform line item.

Build the two that carry the business. The operational database replication and the domain-specific feed with your revenue in it deserve first-party engineering: log-based CDC for the database, a owned pipeline for the feed. These need guarantees a generic connector cannot promise (ordering, recovery behavior, schema evolution on your terms), and when they break at 2 a.m., you want the fixing hands to be the owning hands.

Price maintenance, not construction. A built connector costs a sprint to write and a percentage of an engineer forever. The buy side's equivalent is the subscription plus the occasional migration when the vendor deprecates something. In assessments we put both numbers on one page per connector; the decision usually makes itself, and it is rarely unanimous across the estate.

Negotiate the bought half like the infrastructure it is. Row volume grows, and per-row pricing grows with it, so the connector bill that was a rounding error at signature becomes a line item by renewal. Ask for volume tiers in writing, confirm which schema changes count as billable resyncs, and check the deprecation policy for the specific connectors you depend on rather than the platform in general. Vendors publish sunset lists; read the last two years of them and note whether anything you use has relatives on it.

The failure modes are symmetric: all-build teams run connector hospices, all-buy teams discover their revenue feed is a black box with a status page. Split the estate on differentiation, and review the split yearly.