The process of identifying and correcting errors, inconsistencies, and missing values in a dataset to improve its quality and reliability.
Data cleaning — also called data cleansing or data scrubbing — addresses problems that make data unreliable: duplicate records, missing values, formatting inconsistencies, invalid entries, and outdated information. It is a systematic process, not a one-time fix — data quality degrades over time and requires ongoing attention. A systematic, repeatable cleaning process is called a data refinery.
Common cleaning operations include: removing or merging duplicate records, standardizing formats (dates, addresses, codes), correcting spelling errors, filling or flagging missing values, and removing records that fail validation rules.
Example: A dataset of business licences collected over 10 years contains inconsistent address formats ("123 Main St", "123 Main Street", "123 main st."), duplicate entries from system migrations, and records with null postal codes. A data cleaning process standardizes all addresses to a consistent format, deduplicates records using fuzzy matching, and flags null postal codes for manual review.
Related Terms
- Data Quality
- Data Validation
- Duplicate Record
- Data Standardization
- Null Value
- Data Deduplication
- ETL