Check data quality
Question and objective
Does an unusual result reflect the production process or an acquisition error? Systematic checks prevent teams from basing actions on distorted metrics.
Implementation in Bytefabrik
Use the event preview during connection setup, dataset schema and event views, and tables and charts. For custom AI Pipelines, the test function provides an additional check. These functions support validation; they do not automatically confirm operational completeness.
Best practices: check at three points
- After connection: Are the machine, value, and timestamp of a known event correct?
- After transformation: Does the output match the operational rule, including edge cases?
- After storage: Are the expected time range, event count, and values available historically?
| Check | Typical error |
|---|---|
| Time | Seconds are interpreted as milliseconds; the time zone is missing. |
| Quantity | Repeated events are counted as additional parts. |
| Completeness | Missing messages appear as zero values. |
| Identity | Reused part identifiers link unrelated operations. |
| Unit | Values with different scales are analyzed together. |
What to consider
Timestamps do not necessarily increase in arrival order: delays and multiple sources can reorder events. What matters is handling this behavior explicitly.
Identical measurements are not automatically duplicates. With periodic acquisition, a stable value can legitimately occur repeatedly. Likewise, a data gap does not by itself indicate machine downtime.
Document known limitations and compare against real events again after changes. This makes data validation repeatable and reduces subsequent troubleshooting effort.