Hoyoung Byeon, AI Model & Data QA Senior Researcher
Introduction of ADQ
As explained in the previous publication on the blog, domestic artificial intelligence data quality verification is in its early stages and there is no standardized method. Although there are a variety of labeling tools developed by AI data set establishment companies, most of them focus on data labeling, automation of labeling and work management of labellers.
Therefore, it is time to secure a lighter and more convenient tool to manage procedures and tasks solely for data verification and support statistics required after verification.
For this reason, Testworks which has accumulated AI data quality verification expertise for many years, has launched the beta version of ADQ, third party verification tool to provide quality data to train AI continuously.
Even though ADQ is at its beta version, it was developed to reflect the difficulties in the field of business for companies and institutions that need to perform artificial intelligence data quality verification. With ADQ, it is possible to gain business productivity associated with data verification.
Launch of ADQ beta version
In this beta version, we offer four key features and eight specific features.
▶ 4 major features
1) It is a tool that enables quality verification of training data according to the quality verification process.
After setting a reliable number of validations based on confidence levels and sample errors, you can use the validation tool to facilitate a series of processes, including sampling for validation, division of tasks for efficient validation, and assignment of validation workers.
2) A verification team can be formed for each project and the verification process can be carried out smoothly.
The administrator (&asset) checks the progress and errors of the verification workers to improve the quality of the verification work and basically provide information to help build a training dataset.
3) It provides convenient operations for verification operations
Verification can be performed as convenient as possible from the user’s point of view, such as displaying a screen for error objects, displaying object types, and selecting multiple objects with errors to check for errors at once.
4) It provides quality verification results in the form of report output.
It is possible to download and view the results of a quality verification operation as a report-type file. This can be utilized if data-related evidence is required, or when performing third-party quality verification.
▶ 8 specific functions
Description on ADQ specific features
Let’s look at the features provided by ADQ beta version in more details.
1. Project status dashboard
It shows screen that makes it easy to identify overall situation by graphically handling the project progress, verification data domain distribution, inspector status, annotation type distribution.
2. Data format conversion for labeling results file
Data type conversion is possible to load result files with different data formats into ADQ. We currently support CVAT xml, COCO json, and Pascal VOC xml forms, and we will support other forms in the future. As mentioned in Serial 1, we will provide customized format conversion services for format conversions that are not supported by the current tool, rather than support for all the resulting file types used in the industry due to the lack of recommended standard guides.
3. Data sampling split work and upload
Quality verification is typically done by sampling the entire set of deployment data. If you sample and proceed with one work, one person will proceed due to the nature of the work, and if the amount of samples is large, the work efficiency will naturally decrease. To increase the efficiency of this management, ADQ divides the project into multiple tasks and allows workers to proceed simultaneously. Once the partitioned jobs are uploaded to the server, you can start the actual jobs.
4. Multiverification project management and job management
Multiple projects can be verified at the same time, and the status of verification task assignment and verification progress can be monitored by each worker.
5. Labeling error setting on the verification work screen and error target setting
It is possible to set error settings for each object with a screen that performs a substantial verification operation. In addition, you can easily identify the error objects that you worked on by object type.
It is possible to identify the total number of errors and type of error vehicles. It is also possible to go directly to the page with error objects.
6. Labeling data analysis
It provides not only quality verification results but also analysis results on label distribution and object size distribution.
7. Checking and reporting verification results (accuracy of the meaning)
It is possible to check the progress and verification results of the semantic accuracy verification project, and it also provides the output of the final verification results report for the verification dataset.
8. Management of authority of administrator, verifier, operator
For the efficiency of verification operations, ADQ has three privileges that restrict the use of menus and features.
1) Operator – Quality Verification Operations
2) Inspector – Quality Verification Operations Inspections
3) Administrator – Create user accounts, create and manage projects, view validation results
The current beta version only provides operator authority.
Features as a professional quality validation tool
Finally, the features of the data professional quality verification tool ADQ are summarized below.
It is expected that ADQ will help reduce the time and cost required to build high-quality training datasets for artificial intelligence models by verifying errors and providing error analysis information. Good quality validation specialists will eventually lead to an expansion of data quality validation that determines the quality of the final AI service.
Testworks specializes in building artificial intelligence training datasets and quality verification, uses internally developed tools to increase quality verification efficiency and productivity, and continues to enhance additional features in terms of usability.
It is a proven tool that is used directly internally, and we want to provide it as a service to the outside world to help improve data quality. In the next series, we will introduce the current status of overseas artificial intelligence data verification and actual efficiency improvement according to ADQ use.