Can I read Towards Explainable Automated Data Quality Enhancement without Domain Knowledge on EtoBox?

Towards Explainable Automated Data Quality Enhancement without Domain Knowledge by Sarr, Djibril is a scholarly article available to read on EtoBox.

What is Towards Explainable Automated Data Quality Enhancement without Domain Knowledge about?

In the era of big data, ensuring the quality of datasets has become increasingly crucial across various domains. We propose a comprehensive framework designed to automatically assess and rectify data quality issues in any given dataset, regardless of its specific content, focusing on both textual and numerical data. Our primary objective is to address three fundamental types of defects: absence, redundancy, and incoherence. At the heart of our approach lies a rigorous demand for both explainability and interpretability, ensuring that the rationale behind the identification and correction of data anomalies is transparent and understandable. To achieve this, we adopt a hybrid approach that integrates statistical methods with machine learning algorithms. Indeed, by leveraging statistical techniques alongside machine learning, we strike a balance between accuracy and explainability, enabling users to trust and comprehend the assessment process. Acknowledging the challenges associated with automating the data quality assessment process, particularly in terms of time efficiency and accuracy, we adopt a pragmatic strategy, employing resource-intensive algorithms only when necessary, while

Author

Sarr, Djibril

Published

2024

Language

EN