To create a custom defect classification system, follow this structured approach using deep learning with TensorFlow/Keras. This example focuses on image-based defect classification, but the principles apply to other data types (e.g., text, sensors).

Step 1: Data Preparation

1 Dataset Structure

Organize your data as follows:

dataset/
├── train/
│   ├── defect_type_1/
│   │   ├── img001.jpg
│   │   └── img002.jpg
│   ├── defect_type_2/
│   │   └── ...
│   └── non_defect/
│       └── ...
├── validation/
│   ├── defect_type_1/
│   └── ...
└── test/
    ├── defect_type_1/
    └── ...

2 Data Loading & Preprocessing

import tensorflow as tf
IMG_SIZE = (224, 224)
BATCH_SIZE = 32
# Load datasets
train_ds = tf.keras.utils.image_dataset_from_directory(
    'dataset/train',
    image_size=IMG_SIZE,
    batch_size=BATCH_SIZE,
    label_mode='categorical'  # For multi-class
)
val_ds = tf.keras.utils.image_dataset_from_directory(
    'dataset/validation',
    image_size=IMG_SIZE,
    batch_size=BATCH_SIZE,
    label_mode='categorical'
)
# Optimize data loading
AUTOTUNE = tf.data.AUTOTUNE
train_ds = train_ds.prefetch(buffer_size=AUTOTUNE)
val_ds = val_ds.prefetch(buffer_size=AUTOTUNE)

3 Data Augmentation (Optional but recommended for small datasets)

data_augmentation = tf.keras.Sequential([
    tf.keras.layers.RandomFlip("horizontal"),
    tf.keras.layers.RandomRotation(0.1),
    tf.keras.layers.RandomZoom(0.1),
])

Step 2: Model Architecture

Use transfer learning with a pre-trained model (e.g., EfficientNet) for better performance with limited data.

from tensorflow.keras.applications import EfficientNetB0
from tensorflow.keras import layers, models
# Load pre-trained model (without top layers)
base_model = EfficientNetB0(
    weights='imagenet',
    include_top=False,
    input_shape=(224, 224, 3)
)
base_model.trainable = False  # Freeze initial layers
# Build custom classifier
model = models.Sequential([
    layers.Rescaling(1./255),  # Normalize pixel values
    data_augmentation,         # Apply augmentation
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dropout(0.5),       # Reduce overfitting
    layers.Dense(256, activation='relu'),
    layers.Dense(num_classes, activation='softmax')  # num_classes = number of defect types
])

Step 3: Model Compilation

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

Step 4: Training

history = model.fit(
    train_ds,
    validation_data=val_ds,
    epochs=20,
    callbacks=[
        tf.keras.callbacks.EarlyStopping(patience=3, restore_best_weights=True),
        tf.keras.callbacks.ReduceLROnPlateau(factor=0.1, patience=2)
    ]
)

Step 5: Evaluation

1 Test Set Evaluation

test_ds = tf.keras.utils.image_dataset_from_directory(
    'dataset/test',
    image_size=IMG_SIZE,
    batch_size=BATCH_SIZE,
    label_mode='categorical'
)
test_loss, test_acc = model.evaluate(test_ds)
print(f"Test Accuracy: {test_acc:.2f}")

2 Confusion Matrix & Metrics

import numpy as np
from sklearn.metrics import classification_report, confusion_matrix
# Get predictions
y_true = []
y_pred = []
for images, labels in test_ds:
    preds = model.predict(images)
    y_true.extend(np.argmax(labels, axis=1))
    y_pred.extend(np.argmax(preds, axis=1))
# Print metrics
print(classification_report(y_true, y_pred, target_names=class_names))
print(confusion_matrix(y_true, y_pred))

Step 6: Deployment

1 Save the Model

model.save('defect_classifier.h5')

2 Inference Script

import tensorflow as tf
# Load model
model = tf.keras.models.load_model('defect_classifier.h5')
# Preprocess new image
def preprocess_image(image_path):
    img = tf.keras.preprocessing.image.load_img(image_path, target_size=(224, 224))
    img_array = tf.keras.preprocessing.image.img_to_array(img)
    img_array = tf.expand_dims(img_array, 0)  # Create batch
    return img_array / 255.0
# Predict
image_path = 'new_defect.jpg'
processed_img = preprocess_image(image_path)
prediction = model.predict(processed_img)
class_idx = np.argmax(prediction)
print(f"Predicted Class: {class_names[class_idx]}")

Key Considerations

1. Class Imbalance: Use oversampling/undersampling or class weights if defect types are uneven.
2. Hyperparameter Tuning: Adjust learning rate, dropout, and layers using keras-tuner.
3. Edge Cases: Add "unknown" class to handle unseen defects.
4. Real-time Deployment: Use TensorFlow Lite for mobile/edge devices or Flask/Django for web APIs.
5. Continuous Monitoring: Track model drift and retrain periodically.

Alternative Approaches

• For Tabular Data: Use XGBoost, LightGBM, or MLPs.
• For Text Defects: Use BERT/RoBERTa transformers.
• For Video/Time Series: Use CNN+LSTM or 3D CNNs.

By following these steps, you can build a robust defect classification system tailored to your specific use case. Adjust preprocessing, model architecture, and evaluation metrics based on your data characteristics.