Data leakage

Data leakage is when information that wouldn’t be available at prediction time sneaks into the data a model learns from, so it looks far better in testing than it will ever be in real use. A classic example: predicting whether a patient has a disease using a feature like “was prescribed the treatment”, which only exists because the diagnosis was already made. Leakage also happens more quietly, such as normalizing data using statistics from the whole dataset before splitting it, or letting records from the same customer land in both training and test sets. The warning sign is accuracy that seems too good to be true. It usually is.