Your main nightly Airflow DAG failed at 3 a.m. and dashboards are due at 8 a.m. Walk me through how you handle it.
What the interviewer wants to hear
- Check the failed task logs and identify the root cause first
- Tell stakeholders early about the delay and impact
- Fix or work around the cause, then clear and rerun from the failed task
- Rely on idempotent tasks so reruns do not duplicate data
- Validate the output with row counts and quality checks before announcing
- Write a short post-mortem and add alerts or tests to prevent a repeat
Model answer
First I open the failed task in the Airflow UI and read its logs to find the real cause: a source outage, a schema change, bad data, credentials or resources. If the 8 a.m. deadline is at risk, I post early in the data channel saying which dashboards are affected and when I expect an update. Then I fix the cause, for example by adjusting a query for a renamed column or waiting for the source to recover, and clear the failed task so it and its downstream tasks rerun. Our tasks overwrite their date partition or MERGE on a key, so reruns are safe. Before saying it is fixed, I check row counts, freshness and key metrics against previous days. Afterwards I write a brief post-mortem and add whatever was missing, like a contract test, a source freshness alert or retries for transient errors.
Can you answer it under time pressure?
Take a timed medium round and get scored on every answer.
More medium Data Engineer questions
- Your source system sometimes sends the same event twice. How do you handle duplicates in your pipeline?
- A dbt model that used to take 5 minutes now takes 2 hours. How do you troubleshoot it?
- A stakeholder asks for a "real-time" dashboard. How do you decide whether to build streaming?
- Your Spark job fails with executor out-of-memory errors. What do you check and change?
- How would you design an incremental load for a large orders table that also receives updates to old orders?
- How would you build a pipeline that maintains an SCD Type 2 customer dimension from a daily source extract?
- When would you choose a lakehouse with Delta Lake or Iceberg over a cloud warehouse like Snowflake or BigQuery, or use both?
- A Spark job that used to take 20 minutes now takes 2 hours. How do you investigate?
- Product engineers keep breaking your pipelines by changing database columns without warning. How would you fix this?