Colab Note-Object Detection -Pot Hole Detection with TinyYOLOv4
Training TinyYOLOv4 using a Pot Hole Dataset
Introduction
In this note, we implement the tiny version of YOLOv4 for training our public [Pot Hole Dataset] which consists of 665 images(https://public.roboflow.com/object-detection/pothole), YOLOv4 tiny.
We also recommend reading our blog post on Training YOLOv4 on custom data side by side.
We will take the following steps to implement YOLOv4 on our custom data:
- Configure our GPU environment on Google Colab
- Install the Darknet YOLOv4 training environment
- Download our custom dataset for YOLOv4 and set up directories
- Configure a custom YOLOv4 training config file for Darknet
- Train our custom YOLOv4 object detector
- Reload YOLOv4 trained weights and make inference on test images
When you are done you will have a custom detector that you can use. It will make inference like this:
Reach out for support
If you run into any hurdles on your own data set or just want to share some cool results in your own domain, reach out!
Configuring cuDNN on Colab for YOLOv4
# CUDA: Let’s check that Nvidia CUDA drivers are already pre-installed and which version is it.
!/usr/local/cuda/bin/nvcc — version
# We need to install the correct cuDNN according to this output
#take a look at the kind of GPU we have
!nvidia-smi
my output:
Sun Sep 11 14:01:34 2022
NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2
# This cell ensures you have the correct architecture for your respective GPU
# If you command is not found, look through these GPUs, find the respective
# GPU and add them to the archTypes dictionary
# Tesla V100
# ARCH= -gencode arch=compute_70,code=[sm_70,compute_70]
# Tesla K80
# ARCH= -gencode arch=compute_37,code=sm_37
# GeForce RTX 2080 Ti, RTX 2080, RTX 2070, Quadro RTX 8000, Quadro RTX 6000, Quadro RTX 5000, Tesla T4, XNOR Tensor Cores
# ARCH= -gencode arch=compute_75,code=[sm_75,compute_75]
# Jetson XAVIER
# ARCH= -gencode arch=compute_72,code=[sm_72,compute_72]
# GTX 1080, GTX 1070, GTX 1060, GTX 1050, GTX 1030, Titan Xp, Tesla P40, Tesla P4
# ARCH= -gencode arch=compute_61,code=sm_61
# GP100/Tesla P100 — DGX-1
# ARCH= -gencode arch=compute_60,code=sm_60
# For Jetson TX1, Tegra X1, DRIVE CX, DRIVE PX — uncomment:
# ARCH= -gencode arch=compute_53,code=[sm_53,compute_53]
# For Jetson Tx2 or Drive-PX2 uncomment:
# ARCH= -gencode arch=compute_62,code=[sm_62,compute_62]
import os
os.environ[‘GPU_TYPE’] = str(os.popen(‘nvidia-smi — query-gpu=name — format=csv,noheader’).read())
def getGPUArch(argument):
try:
argument = argument.strip()
# All Colab GPUs
archTypes = {
“Tesla V100-SXM2–16GB”: “-gencode arch=compute_70,code=[sm_70,compute_70]”,
“Tesla K80”: “-gencode arch=compute_37,code=sm_37”,
“Tesla T4”: “-gencode arch=compute_75,code=[sm_75,compute_75]”,
“Tesla P40”: “-gencode arch=compute_61,code=sm_61”,
“Tesla P4”: “-gencode arch=compute_61,code=sm_61”,
“Tesla P100-PCIE-16GB”: “-gencode arch=compute_60,code=sm_60”
}
return archTypes[argument]
except KeyError:
return “GPU must be added to GPU Commands”
os.environ[‘ARCH_VALUE’] = getGPUArch(os.environ[‘GPU_TYPE’])
print(“GPU Type: “ + os.environ[‘GPU_TYPE’])
print(“ARCH Value: “ + os.environ[‘ARCH_VALUE’])
my output is:
GPU Type: Tesla T4 ARCH Value: -gencode arch=compute_75,code=[sm_75,compute_75]
Installing Darknet for YOLOv4 on Colab
%cd /content/
%rm -rf darknet
#we clone the fork of darknet maintained by roboflow
#small changes have been made to configure darknet for training
!git clone https://github.com/roboflow-ai/darknet.git
next cell…
#install environment from the Makefile
%cd /content/darknet/
# compute_37, sm_37 for Tesla K80
# compute_75, sm_75 for Tesla T4
# !sed -i ‘s/ARCH= -gencode arch=compute_60,code=sm_60/ARCH= -gencode arch=compute_75,code=sm_75/g’ Makefile
#install environment from the Makefile
#note if you are on Colab Pro this works on a P100 GPU
#if you are on Colab free, you may need to change the Makefile for the K80 GPU
#this goes for any GPU, you need to change the Makefile to inform darknet which GPU you are running on.
!sed -i ‘s/OPENCV=0/OPENCV=1/g’ Makefile
!sed -i ‘s/GPU=0/GPU=1/g’ Makefile
!sed -i ‘s/CUDNN=0/CUDNN=1/g’ Makefile
!sed -i “s/ARCH= -gencode arch=compute_60,code=sm_60/ARCH= ${ARCH_VALUE}/g” Makefile
!make
output: skip
#download the newly released yolov4-tiny weights
%cd /content/darknet
!wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.weights
!wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.conv.29
Set up Custom Dataset for YOLOv4
Set up Custom Dataset for YOLOv4
We’ll use Roboflow to convert our dataset from any format to the YOLO Darknet format.
- To do so, create a free Roboflow account.
- Upload your images and their annotations (in any format: VOC XML, COCO JSON, TensorFlow CSV, etc).
- Apply preprocessing and augmentation steps you may like. We recommend at least
auto-orientand aresizeto 416x416. Generate your dataset. - Export your dataset in the YOLO Darknet format.
- Copy your download link, and paste it below.
See our blog post for greater detail.
In this example, I used the open source BCCD Dataset. (You can fork it to your Roboflow account to follow along.)
!wget https://moderncomputervision.s3.eu-west-2.amazonaws.com/Pothole.v1-raw.darknet.zip
!unzip -q Pothole.v1-raw.darknet.zip
next cell…
#if you already have YOLO darknet format, you can skip this step
#otherwise we recommend formatting in Roboflow
%cd /content/darknet
!curl -L “https://public.roboflow.com/ds/I2ZXTaUHUY?key=BBFNVcFack" > roboflow.zip; unzip roboflow.zip; rm roboflow.zip
#Set up training file directories for custom dataset
%cd /content/darknet/
%cp train/_darknet.labels data/obj.names
%mkdir data/obj
#copy image and labels
%cp train/*.jpg data/obj/
%cp valid/*.jpg data/obj/
%cp train/*.txt data/obj/
%cp valid/*.txt data/obj/
with open(‘data/obj.data’, ‘w’) as out:
out.write(‘classes = 3\n’)
out.write(‘train = data/train.txt\n’)
out.write(‘valid = data/valid.txt\n’)
out.write(‘names = data/obj.names\n’)
out.write(‘backup = backup/’)
#write train file (just the image list)
import os
with open(‘data/train.txt’, ‘w’) as out:
for img in [f for f in os.listdir(‘train’) if f.endswith(‘jpg’)]:
out.write(‘data/obj/’ + img + ‘\n’)
#write the valid file (just the image list)
import os
with open(‘data/valid.txt’, ‘w’) as out:
for img in [f for f in os.listdir(‘valid’) if f.endswith(‘jpg’)]:
out.write(‘data/obj/’ + img + ‘\n’)
Write Custom Training Config for YOLOv4
#we build config dynamically based on number of classes
#we build iteratively from base config files. This is the same file shape as cfg/yolo-obj.cfg
def file_len(fname):
with open(fname) as f:
for i, l in enumerate(f):
pass
return i + 1
num_classes = file_len(‘train/_darknet.labels’)
max_batches = num_classes*2000
steps1 = .8 * max_batches
steps2 = .9 * max_batches
steps_str = str(steps1)+’,’+str(steps2)
num_filters = (num_classes + 5) * 3
print(“writing config for a custom YOLOv4 detector detecting number of classes: “ + str(num_classes))
#Instructions from the darknet repo
#change line max_batches to (classes*2000 but not less than number of training images, and not less than 6000), f.e. max_batches=6000 if you train for 3 classes
#change line steps to 80% and 90% of max_batches, f.e. steps=4800,5400
if os.path.exists(‘./cfg/custom-yolov4-tiny-detector.cfg’): os.remove(‘./cfg/custom-yolov4-tiny-detector.cfg’)
#customize iPython writefile so we can write variables
from IPython.core.magic import register_line_cell_magic
@register_line_cell_magic
def writetemplate(line, cell):
with open(line, ‘w’) as f:
f.write(cell.format(**globals()))
my output:
writing config for a custom YOLOv4 detector detecting number of classes: 1
%%writetemplate ./cfg/custom-yolov4-tiny-detector.cfg
[net]
# Testing
#batch=1
#subdivisions=1
# Training
batch=64
subdivisions=24
width=416
height=416
channels=3
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1
learning_rate=0.00261
burn_in=1000
max_batches = {max_batches}
policy=steps
steps={steps_str}
scales=.1,.1
[convolutional]
batch_normalize=1
filters=32
size=3
stride=2
pad=1
activation=leaky
[convolutional]
batch_normalize=1
filters=64
size=3
stride=2
pad=1
activation=leaky
[convolutional]
batch_normalize=1
filters=64
size=3
stride=1
pad=1
activation=leaky
[route]
layers=-1
groups=2
group_id=1
[convolutional]
batch_normalize=1
filters=32
size=3
stride=1
pad=1
activation=leaky
[convolutional]
batch_normalize=1
filters=32
size=3
stride=1
pad=1
activation=leaky
[route]
layers = -1,-2
[convolutional]
batch_normalize=1
filters=64
size=1
stride=1
pad=1
activation=leaky
[route]
layers = -6,-1
[maxpool]
size=2
stride=2
[convolutional]
batch_normalize=1
filters=128
size=3
stride=1
pad=1
activation=leaky
[route]
layers=-1
groups=2
group_id=1
[convolutional]
batch_normalize=1
filters=64
size=3
stride=1
pad=1
activation=leaky
[convolutional]
batch_normalize=1
filters=64
size=3
stride=1
pad=1
activation=leaky
[route]
layers = -1,-2
[convolutional]
batch_normalize=1
filters=128
size=1
stride=1
pad=1
activation=leaky
[route]
layers = -6,-1
[maxpool]
size=2
stride=2
[convolutional]
batch_normalize=1
filters=256
size=3
stride=1
pad=1
activation=leaky
[route]
layers=-1
groups=2
group_id=1
[convolutional]
batch_normalize=1
filters=128
size=3
stride=1
pad=1
activation=leaky
[convolutional]
batch_normalize=1
filters=128
size=3
stride=1
pad=1
activation=leaky
[route]
layers = -1,-2
[convolutional]
batch_normalize=1
filters=256
size=1
stride=1
pad=1
activation=leaky
[route]
layers = -6,-1
[maxpool]
size=2
stride=2
[convolutional]
batch_normalize=1
filters=512
size=3
stride=1
pad=1
activation=leaky
##################################
[convolutional]
batch_normalize=1
filters=256
size=1
stride=1
pad=1
activation=leaky
[convolutional]
batch_normalize=1
filters=512
size=3
stride=1
pad=1
activation=leaky
[convolutional]
size=1
stride=1
pad=1
filters={num_filters}
activation=linear
[yolo]
mask = 3,4,5
anchors = 10,14, 23,27, 37,58, 81,82, 135,169, 344,319
classes={num_classes}
num=6
jitter=.3
scale_x_y = 1.05
cls_normalizer=1.0
iou_normalizer=0.07
iou_loss=ciou
ignore_thresh = .7
truth_thresh = 1
random=0
nms_kind=greedynms
beta_nms=0.6
[route]
layers = -4
[convolutional]
batch_normalize=1
filters=128
size=1
stride=1
pad=1
activation=leaky
[upsample]
stride=2
[route]
layers = -1, 23
[convolutional]
batch_normalize=1
filters=256
size=3
stride=1
pad=1
activation=leaky
[convolutional]
size=1
stride=1
pad=1
filters={num_filters}
activation=linear
[yolo]
mask = 1,2,3
anchors = 10,14, 23,27, 37,58, 81,82, 135,169, 344,319
classes={num_classes}
num=6
jitter=.3
scale_x_y = 1.05
cls_normalizer=1.0
iou_normalizer=0.07
iou_loss=ciou
ignore_thresh = .7
truth_thresh = 1
random=0
nms_kind=greedynms
beta_nms=0.6
next cell…
#here is the file that was just written.
#you may consider adjusting certain things
#like the number of subdivisions 64 runs faster but Colab GPU may not be big enough
#if Colab GPU memory is too small, you will need to adjust subdivisions to 16
%cat cfg/custom-yolov4-tiny-detector.cfg
print my config information:
jitter=.3 scale_x_y = 1.05 cls_normalizer=1.0 iou_normalizer=0.07 iou_loss=ciou ignore_thresh = .7 truth_thresh = 1 random=0 nms_kind=greedynms beta_nms=0.6 [route] layers = -4 [convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=leaky [upsample] stride=2 [route] layers = -1, 23 [convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=leaky [convolutional] size=1 stride=1 pad=1 filters=18 activation=linear [yolo] mask = 1,2,3 anchors = 10,14, 23,27, 37,58, 81,82, 135,169, 344,319 classes=1 num=6 jitter=.3 scale_x_y = 1.05 cls_normalizer=1.0 iou_normalizer=0.07 iou_loss=ciou ignore_thresh = .7 truth_thresh = 1 random=0 nms_kind=greedynms beta_nms=0.6
Train Custom YOLOv4 Detector
!./darknet detector train data/obj.data cfg/custom-yolov4-tiny-detector.cfg yolov4-tiny.conv.29 -dont_show -map
#If you get CUDA out of memory adjust subdivisions above!
#adjust max batches down for shorter training above
my output:
mean_average_precision (mAP@0.5) = 0.721922 New best mAP! Saving weights to backup//custom-yolov4-tiny-detector_best.weights Saving weights to backup//custom-yolov4-tiny-detector_2000.weights Saving weights to backup//custom-yolov4-tiny-detector_last.weights Saving weights to backup//custom-yolov4-tiny-detector_final.weights
Infer Custom Objects with Saved YOLOv4 Weights
#define utility function
def imShow(path):
import cv2
import matplotlib.pyplot as plt
%matplotlib inline
image = cv2.imread(path)
height, width = image.shape[:2]
resized_image = cv2.resize(image,(3*width, 3*height), interpolation = cv2.INTER_CUBIC)
fig = plt.gcf()
fig.set_size_inches(18, 10)
plt.axis(“off”)
#plt.rcParams[‘figure.figsize’] = [10, 5]
plt.imshow(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB))
plt.show()
#check if weigths have saved yet
#backup houses the last weights for our detector
#(file yolo-obj_last.weights will be saved to the build\darknet\x64\backup\ for each 100 iterations)
#(file yolo-obj_xxxx.weights will be saved to the build\darknet\x64\backup\ for each 1000 iterations)
#After training is complete — get result yolo-obj_final.weights from path build\darknet\x64\bac
!ls backup
#if it is empty you haven’t trained for long enough yet, you need to train for at least 100 iterations
my output:
custom-yolov4-tiny-detector_1000.weights
custom-yolov4-tiny-detector_2000.weights
custom-yolov4-tiny-detector_best.weights
custom-yolov4-tiny-detector_final.weights
custom-yolov4-tiny-detector_last.weights
#coco.names is hardcoded somewhere in the detector
%cp data/obj.names data/coco.names
#/test has images that we can test our detector on
test_images = [f for f in os.listdir(‘test’) if f.endswith(‘.jpg’)]
import random
img_path = “test/” + random.choice(test_images);
#test out our detector!
!./darknet detect cfg/custom-yolov4-tiny-detector.cfg backup/custom-yolov4-tiny-detector_best.weights {img_path} -dont-show
imShow(‘/content/darknet/predictions.jpg’)
my output:
CUDA-version: 11010 (11020), cuDNN: 7.6.5, GPU count: 1
OpenCV version: 3.2.0
compute_capability = 750, cudnn_half = 0
net.optimized_memory = 0
mini_batch = 1, batch = 24, time_steps = 1, train = 0
layer filters size/strd(dil) input output
0 conv 32 3 x 3/ 2 416 x 416 x 3 -> 208 x 208 x 32 0.075 BF
1 conv 64 3 x 3/ 2 208 x 208 x 32 -> 104 x 104 x 64 0.399 BF
2 conv 64 3 x 3/ 1 104 x 104 x 64 -> 104 x 104 x 64 0.797 BF
3 route 2 1/2 -> 104 x 104 x 32
4 conv 32 3 x 3/ 1 104 x 104 x 32 -> 104 x 104 x 32 0.199 BF
5 conv 32 3 x 3/ 1 104 x 104 x 32 -> 104 x 104 x 32 0.199 BF
6 route 5 4 -> 104 x 104 x 64
7 conv 64 1 x 1/ 1 104 x 104 x 64 -> 104 x 104 x 64 0.089 BF
8 route 2 7 -> 104 x 104 x 128
9 max 2x 2/ 2 104 x 104 x 128 -> 52 x 52 x 128 0.001 BF
10 conv 128 3 x 3/ 1 52 x 52 x 128 -> 52 x 52 x 128 0.797 BF
11 route 10 1/2 -> 52 x 52 x 64
12 conv 64 3 x 3/ 1 52 x 52 x 64 -> 52 x 52 x 64 0.199 BF
13 conv 64 3 x 3/ 1 52 x 52 x 64 -> 52 x 52 x 64 0.199 BF
14 route 13 12 -> 52 x 52 x 128
15 conv 128 1 x 1/ 1 52 x 52 x 128 -> 52 x 52 x 128 0.089 BF
16 route 10 15 -> 52 x 52 x 256
17 max 2x 2/ 2 52 x 52 x 256 -> 26 x 26 x 256 0.001 BF
18 conv 256 3 x 3/ 1 26 x 26 x 256 -> 26 x 26 x 256 0.797 BF
19 route 18 1/2 -> 26 x 26 x 128
20 conv 128 3 x 3/ 1 26 x 26 x 128 -> 26 x 26 x 128 0.199 BF
21 conv 128 3 x 3/ 1 26 x 26 x 128 -> 26 x 26 x 128 0.199 BF
22 route 21 20 -> 26 x 26 x 256
23 conv 256 1 x 1/ 1 26 x 26 x 256 -> 26 x 26 x 256 0.089 BF
24 route 18 23 -> 26 x 26 x 512
25 max 2x 2/ 2 26 x 26 x 512 -> 13 x 13 x 512 0.000 BF
26 conv 512 3 x 3/ 1 13 x 13 x 512 -> 13 x 13 x 512 0.797 BF
27 conv 256 1 x 1/ 1 13 x 13 x 512 -> 13 x 13 x 256 0.044 BF
28 conv 512 3 x 3/ 1 13 x 13 x 256 -> 13 x 13 x 512 0.399 BF
29 conv 18 1 x 1/ 1 13 x 13 x 512 -> 13 x 13 x 18 0.003 BF
30 yolo
[yolo] params: iou loss: ciou (4), iou_norm: 0.07, cls_norm: 1.00, scale_x_y: 1.05
nms_kind: greedynms (1), beta = 0.600000
31 route 27 -> 13 x 13 x 256
32 conv 128 1 x 1/ 1 13 x 13 x 256 -> 13 x 13 x 128 0.011 BF
33 upsample 2x 13 x 13 x 128 -> 26 x 26 x 128
34 route 33 23 -> 26 x 26 x 384
35 conv 256 3 x 3/ 1 26 x 26 x 384 -> 26 x 26 x 256 1.196 BF
36 conv 18 1 x 1/ 1 26 x 26 x 256 -> 26 x 26 x 18 0.006 BF
37 yolo
[yolo] params: iou loss: ciou (4), iou_norm: 0.07, cls_norm: 1.00, scale_x_y: 1.05
nms_kind: greedynms (1), beta = 0.600000
Total BFLOPS 6.787
avg_outputs = 299663
Allocate additional workspace_size = 26.22 MB
Loading weights from backup/custom-yolov4-tiny-detector_best.weights...
seen 64, trained: 96 K-images (1 Kilo-batches_64)
Done! Loaded 38 layers from weights-file
test/img-576_jpg.rf.f6cd32a51b0c518b58ff750ecab687d1.jpg: Predicted in 5.150000 milli-seconds.
pothole: 90%
Unable to init server: Could not connect: Connection refused
(predictions:4463): Gtk-WARNING **: 14:20:51.624: cannot open display:
Done to detect!!
