What is change data capture (CDC), and why do data teams use it?

What the interviewer wants to hear

• CDC captures inserts, updates and deletes from a source database
• Log-based CDC reads the transaction log, such as the binlog or WAL
• It avoids heavy full extracts and catches hard deletes
• Changes usually flow through Kafka into the lake or warehouse
• Target applies changes with MERGE, keeping order per key

Model answer

Change data capture means capturing every change made to a source database, inserts, updates and deletes, and streaming those changes to other systems. The most reliable form is log-based CDC, where a tool like Debezium reads the database transaction log, such as the MySQL binlog or the PostgreSQL WAL. Compared with nightly full extracts or polling on updated_at, it puts little load on the source, gives near real-time data and captures hard deletes that polling would miss. Typically the change events go into Kafka topics keyed by primary key, and a downstream job applies them to a lake or warehouse table with a MERGE, using the log position or a sequence number to keep changes in order per key. Teams use it to keep analytics tables fresh, feed caches and search indexes, and replicate data without touching application code.

More easy Data Engineer questions

• Describe a data pipeline you built or would build, from source to dashboard.
• What is the difference between an OLTP database and an OLAP data warehouse?
• Explain a star schema and why it is used in data warehouses.
• What data quality checks would you add to a pipeline?
• Why use an orchestration tool like Airflow instead of cron jobs?
• What is a slowly changing dimension, and how do SCD Types 1, 2 and 3 differ?
• Why would you store analytics data as Parquet rather than CSV or JSON?
• Explain Kafka topics, partitions, offsets and consumer groups to a new teammate.
• What does dbt do in a modern data stack, and what are its main features?