VAD Input and Artifacts
Data Preparation
Prepare your training data
Prepare normal (or identical) image data with the same shape (1024x1024, 3 channels) in the 1. png, .jpg shape. 2. Prepare a tabular ground truth file consisting of the image path and the correct answer label (all of which are normal, so write them in the same type). 3. If you have some abnormal (or a few other types) image data, you can include it in the training data to create a more accurate AI model. 4. Prepare the ground truth file in the following form. 5. When preparing a ground truth file, it is not necessary to uniform labels for images as long as they have multiple types. 6. If there is data that you want to verify, enter the item name in the 'train_validation_column' argument, and the image corresponding to train will be used for training, etc. Images can be used for verification to check performance. (Even if there is only a normal image, only the performance cannot be derived, but it is trained normally.) 7. Prepare the ground truth file and the image file by putting them in the same folder. 8. For inference data, prepare a ground truth file as well as training data, or prepare the image you want to infer in one folder. (For inference data, if you don't have a Ground truth file, it looks at the path where the image was saved and generates the file internally.) 9. Even though the training data type is composed of several types, it is classified into only two types: normal type and abnormal type.
Training dataset example
| label | image_path | train_val_split |
|---|---|---|
| good | ./image0.png | train |
| bad | ./image1.png | valid |
| good | ./image2.png | train |
| good | ./good/image0.png | train |
| good | ./good/image1.png | train |
| good | ./good/image0.jpg | train |
| good | ./good/image1.jpg | train |
Example input data directory structure
/project_folder/sample_data
└ inference
└ 10479_0.png
└ 1080_7.png
└ 10735_7.png
└ train
└ train.csv
└ abnormal
└ 10019_0.png
└ 10024_0.png
└ 10097_0.png
└ good
└ 10395_7.png
└ 10034_7.png
└ 10852_7.png
Data Requirements
Essential Requirements
The input data must meet the following conditions:
| index | item | spec. |
|---|---|---|
| 1 | Enough Normal or Single-Type Images | 100~50000 |
| 2 | Ground Truth File Item Name Form Compliance (label, image_path) | Yes |
| 3 | Fixed number of channels: 3 or 1 channel (gray) | Yes |
| 4 | Resolution | 32x32 ~ 1920x1920 pixels |
| 5 | Is the inference interval for new images greater than 3 seconds | Yes |
| 6 | Is the time interval required for learning to be more than 6 hours | Yes |
- See Data Examples
Additional Requirements
Conditions to ensure minimum performance. Even if the following conditions are not met, the algorithm will still work, but the performance has not been confirmed
| index | item | spec. |
|---|---|---|
| 1 | Save the image name as a unique name regardless of type | Yes |
| 2 | Obtaining a small number of anomalies to verify AI performance | 10 pieces per type |
| 3 | Resolution | 224x224 ~ 1024x1024 pixels |
| 4 | Region Of Interests (ROI) Size | 15 x 15 pixels or larger at 224 x 224 |
| 5 | Fixed product position/orientation/distance from camera | Tolerance: Rotation +/- 3 degrees or less, translation 1 pixel or less at 224x224 |
| 6 | Image must be in focus | Yes |
| 7 | Image measurement environment: Maintain the same and consistent environment as much as possible during training/verification/operation (brightness, illumination, background, etc.) | Yes |
Artifacts
Training/inference will produce the following outputs:
model config
/project_folder/.workspace/vad/model_artifacts/config
└ config
└ input
└ config.json
└ readiness
└ config.json
└ train
└ config.json
└ fastflow_model.ckpt
└ model_lite.pt
└ model.pickle
└ best_model_path.json
Train pipeline
/project_folder/.workspace/vad/history/latest/train
└ dataset
└ abnormal
└ 1.png
└ 2.png
└ ...
└ good
└ a.png
└ b.png
└ ...
└ train.csv
└ output
└ output.csv
└ score
└ train_summary.yaml
└ extra_output
└ images
└ abnormal_1.png
└ abnormal_2.png
└ ...
└ threshold_test_result.csv
└ threshold_test_result.png
└ validation_classificaion_report.json
└ validation_confusion_matrix.csv
└ validation_score.csv
└ log
└ pipeline.log
└ process.log
Inference pipeline
/project_folder/.workspace/vad/history/latest/inference
└ log
└ pipeline.log
└ process.log
└ dataset
└ 1.png
└ 2.png
└ ...
└ score
└ inference_summary.yaml
└ output
└ output.csv
└ 1.jpg
The detailed description of each deliverable is as follows.
{model_name}_model.ckpt
This is a model file that has been trained.
output.csv
It is a tabular file in which the results of inferences and probabilities are recorded.
validation_confusion_matrix.csv
A confusion matrix file that stores the results of the performance of the validation data in the training pipeline. It is only stored if there is abnormal data in the validation data.
validation_score.csv
A file that records performance on validation data from a training pipeline. Accuracy, precision, recall, F1-score, and AUROC are recorded.
{imagefilename}.png
This is an image where an abnormal area is shown. For the training pipeline, save for incorrectly inferred images and all abnormal images. For inference pipelines, the output folder stores the original images for retraining.
inference_summary.yaml
A summary file of the inference results. It is used by edge conductors and consists of score, result, probability, and so on.
threshold_test_result.csv
This is a table of performance changes for validation data when the threshold for anomaly detection is changed.
threshold_test_result.png
This graph of performance change when the threshold for anomaly detection is changed.
VAD Version: 3.0.0