Hard Data Engineer questions
32 questions · 20 points each for a correct first answer.
- 1.One task takes foreverspark, performancehard
- 2.Exactly-once end to endkafka, reliabilityhard
- 3.Millions of tiny filesdata-lake, performancehard
- 4.Safe backfillsorchestration, reliabilityhard
- 5.Applying CDC eventscdc, incremental-loadshard
- 6.Cutting warehouse scan costperformance, cost-optimizationhard
- 7.Upstream renamed a columndata-quality, data-contractshard
- 8.Why table formats existdata-lake, lakehousehard
- 9.Running total with tied datessql, window-functionshard
- 10.LAST_VALUE returns the current rowsql, window-functionshard
- 11.Longest login streaksql, window-functionshard
- 12.FULL OUTER JOIN row countsql, joins, reconciliationhard
- 13.GROUP BY with NULL keyssql, null-handling, aggregationhard
- 14.Recursive CTE stopping pointsql, ctehard
- 15.Double counting in an SCD2 joinscd, sql, data-modelinghard
- 16.Which late event is keptstreaming, watermarks, sparkhard
- 17.Committing offsets too earlykafka, delivery-semanticshard
- 18.Adding partitions to a keyed topickafka, orderinghard
- 19.A slow Python UDF in PySparkspark, python, performancehard
- 20.Distinct users on billions of rowssql, performance, warehousinghard
- 21.Iceberg hidden partitioningiceberg, partitioninghard
- 22.VACUUM and time traveldelta-lake, time-travel, governancehard
- 23.Two writers, one lakehouse tablelakehouse, concurrencyhard
- 24.Copy-on-write vs merge-on-readhudi, lakehousehard
- 25.Clustering for two filter columnslakehouse, performance, clusteringhard
- 26.Choosing a Redshift DISTKEYredshift, performancehard
- 27.Rows missed by a strict watermarkincremental-loads, late-data, idempotencyhard
- 28.New column in a dbt incremental modeldbt, schema-evolutionhard
- 29.Hashing emails is not anonymisinggovernance, pii, securityhard
- 30.Overwrite deleted the whole tablespark, backfills, idempotencyhard
- 31.Which day does an event belong totime-zones, partitioninghard
- 32.Sensors blocking every workerairflow, orchestration, performancehard