Market Data Cleaning Notes

Market data cleaning is not a boring preprocessing step. It is part of the research problem.

Bad timestamps, missing prices, duplicated rows, split adjustments, survivorship bias, and inconsistent symbol histories can all change the conclusion of a strategy test.

My preferred workflow is to document every data assumption, keep raw and cleaned data separate, and write validation checks before building models on top of the data.