Medium Data Engineer questions
34 questions · 10 points each for a correct first answer.
- 1.Tracking customer address historydata-modeling, scdmedium
- 2.Incremental load with updated_atincremental-loads, cdcmedium
- 3.What causes a shufflespark, performancemedium
- 4.Joining with a small lookup tablespark, joinsmedium
- 5.Ordering in Kafkakafka, streamingmedium
- 6.Upserting into a target tablesql, incremental-loadsmedium
- 7.Why use ref() in dbtdbt, orchestrationmedium
- 8.Handling late eventsstreaming, event-timemedium
- 9.Keeping the latest row per keysql, window-functions, deduplicationmedium
- 10.RANK and DENSE_RANK with tiessql, window-functionsmedium
- 11.NOT IN with a NULL in the listsql, null-handlingmedium
- 12.Day-over-day change with LAGsql, window-functionsmedium
- 13.WHERE filter after a LEFT JOINsql, joinsmedium
- 14.Counting orders for every customersql, joins, aggregationmedium
- 15.Table state after a MERGEsql, merge, upsertmedium
- 16.Revenue doubled after a joinsql, joins, data-modelingmedium
- 17.Employees above their team averagesql, ctemedium
- 18.Keeping just the previous valuescd, data-modelingmedium
- 19.Data Vault building blocksdata-modeling, data-vaultmedium
- 20.Log-based CDC advantagescdcmedium
- 21.Duplicates from at-least-once deliverykafka, idempotency, delivery-semanticsmedium
- 22.Surprise runs from catchupairflow, backfillsmedium
- 23.When a daily Airflow run startsairflow, schedulingmedium
- 24.When to cache in Sparkspark, performancemedium
- 25.repartition vs coalescespark, partitioningmedium
- 26.Eight consumers, six partitionskafka, scalingmedium
- 27.How Parquet skips dataparquet, file-formats, performancemedium
- 28.Undoing a bad loaddelta-lake, time-travelmedium
- 29.LIMIT and BigQuery costbigquery, costmedium
- 30.Idle Snowflake warehouse costsnowflake, costmedium
- 31.Filtering new rows in dbtdbt, incremental-loadsmedium
- 32.Detecting stale source datadbt, data-quality, monitoringmedium
- 33.What a data contract doesdata-contracts, data-qualitymedium
- 34.Rerunning a half-failed loadidempotency, data-pipelinesmedium