Data reconciliation in the narrow sense is the comparison of electronic data solely to detect the differences. In a broader sense, this includes the alignment of data sets with each other. After that, each of the storage media contains the same updated data.
A well-known example is the comparison of address data and appointments between a handheld computer (or PDA) and a (main) computer. The comparison (more precisely the alignment or synchronization) is usually carried out via a fixed (i.e. wired) connection (such as USB) or wirelessly (for example via infrared, Bluetooth, WLAN, etc.). For such a data comparison between two devices to be possible at all, both must agree on a defined protocol, i.e. the type and format of the data must be known to both. In practice, a so-called replication program is also used for this purpose – but (auxiliary) applications are also possible with the help of which data can also be mirrored (such as robocopy).
Data matching is used in the industry to increase the availability and data integrity of a system. Examples of this would be data mirroring between databases to maintain operation in the event of a failure of one or some servers, or an overplay of the data every few hours, where the technical effort is lower. In the second case, you can use the log files, which are constantly overplayed and the last transferred data status on the main computer after a restart, to restore the last state in some time. However, such an immediate switch to another computer is no longer possible.
---
Data reconciliation is also a technique with a corrective goal such as correction of measurement errors which are random errors in nature. The main assumption in such a case is that there are no systematic errors in the set of measurements. During data migration, errors can be made out of practical reasons such as network dropouts leading to data being left in an invalid state. These may lead to a range of issues:
- Missing records
- Missing values
- Incorrect values
- Duplicated records
- Badly formatted values
- Broken relationships
Data reconciliation can mathematically be expressed as an optimization problem. Some data migration solutions provide data prototyping and comparative reconciliation functionality.
