A row of data that appears more than once in a dataset, often due to system errors, data merging, or repeated imports.
Duplicate records inflate counts, distort averages, and produce incorrect analysis results. They are one of the most common data quality problems and can be surprisingly difficult to detect when duplicates are not exact copies — for example, the same person appearing with slightly different name spellings or address formats.
Example: A dataset of registered voters is compiled from two regional systems. The same person appears twice — once as "Jean Tremblay, 123 Rue Main" and once as "J. Tremblay, 123 Main St." These are duplicates, but a simple exact-match check would not catch them. Deduplication logic must account for variations.
Detecting and removing duplicates is a core step in data cleaning. The approach depends on whether duplicates are exact matches or fuzzy matches requiring more sophisticated comparison.