Glossary

Data cleansing

What is Data cleansing?

Data cleansing is the detection and correction of incorrect, duplicate or inconsistent entries in data sets. Typical steps include standardising spellings and date formats, removing superfluous spaces and merging duplicates such as 'Smith Ltd' and 'Smith Limited'. Clean data is a prerequisite for reliable analyses, mailings and AI applications.

How KI.ordo helps

A KI.ordo template cleans lists, flags possible duplicates and missing details, and produces a change log.

Learn more →

Matching templates

Clean up a listStandardises spellings, spaces, capitalisation and date formats, flags possible duplicates and missing details, and logs every change.View template →

Topics

data quality
duplicates
data maintenance

Try KI.ordo

Agents, flows and templates in one workspace – free 24-hour demo.

Start demo

← All terms

Related terms

Data validation

Data validation checks whether data is complete, correctly formatted and plausible before it is processed further. Typical checks cover mandatory fields, the format of email addresses, postcodes or dates, and logical contradictions – such as a delivery date before the order date. This catches errors early instead of letting them carry over into reports, invoices or AI analyses.

data quality
checks
+1

Template

Fuzzy matching

Fuzzy matching finds entries that are similar but not exactly identical – for example because of typos, abbreviations or different spellings. It relies on similarity measures such as the Levenshtein distance, which counts how many characters need to be changed. A typical use is finding duplicates in address and customer data: ‘Smith Ltd, 5 High St’ and ‘Smith Limited, 5 High Street’ are recognised as probably the same record.

Data cleansing
Duplicates
+1

Template

Garbage in, garbage out (GIGO)

‘Garbage in, garbage out’ describes the principle that a system can only produce results as good as the data it receives. This applies doubly to AI: incorrect, outdated or one-sided training or knowledge data leads to wrong or biased answers. A chatbot that draws on an outdated price list will quote outdated prices. Looking after data is therefore an ongoing task in every AI project.

Data quality
Principle
+1

Structured data

Structured data follows a fixed schema with clearly named fields, such as tables, databases or JSON objects containing details like name, price and date. Machines can read, filter and process it reliably – unlike free text or images. On websites, schema.org markup describes products, events or FAQs, for example, so that search engines and AI systems understand the content unambiguously. AI can also extract structured data from unstructured text.

data
JSON
+1