开始-感知-目标追踪

Perception - Object Tracking

This page provides a hands-on example for Object Tracking in Perception. It uses the ByteTrack model for 2D tracking of vehicles and pedestrians.

Given belowModel OverviewandCode walkthrough。

Model Overview

1. Model overview:

Bytetrack is an algorithm for multi-object tracking (MOT, Multi Object Tracking) tasks, in the paper《ByteTrack: Multi-Object Tracking by Associating Every Detection Box》It mainly achieves efficient multi-target tracking by processing the target frames generated by the detector. It solves the problems of the traditional MOT algorithm when dealing with low-confidence targets and improves tracking accuracy. The core idea of Bytetrack is to track all detection boxes, including high-confidence and low-confidence targets, which avoids missing low-confidence targets, thus improving the overall tracking performance.

In multi-object tracking, the input is traffic-flow video and the output is an ID label for each detected object.

2. Model Architecture:

Object detection and assignment: input video frames first pass through an object detector to obtain detection boxes, which can be sorted by confidence.
Initial association: use a Kalman Filter and the Hungarian Algorithm for initial tracking matches. High-confidence boxes are first matched with historical tracks and assigned to existing tracked objects.
Low-confidence objects: ByteTrack does not simply discard low-confidence boxes. For unmatched tracks, it further checks and attempts to match low-confidence boxes, capturing objects previously missed by the detector and making tracking more complete.
State update: matched objects update their states, while unmatched objects enter a potential-object list. If they remain unmatched in later frames, they are eventually discarded and marked as lost.

3. Implementation:

Data preprocessing: split a video sequence into an ordered image sequence at a selected frame rate, or read it directly with OpenCV.
Model selection: choose an appropriate object-tracking model and provide its configuration file.
Visualization: Object Tracking inference is fast enough for real-time display and can also be processed offline to generate a video.

Code walkthrough

0. Preparation:

System requirements: Windows/Linux with Anaconda or Miniconda installed.
An NVIDIA discrete GPU with CUDA, cuDNN, and PyTorch is required. On Windows, install Visual Studio before CUDA.
The example uses video frames from Traffic Footage downloaded from YouTube. The demo folder is already included, so no additional download is required.

1. Environment Setup:

Copy to Clipboard

2. Data Preprocessing:

Copy to Clipboard

video_path: storage path of traffic flow video files

3. Model Selection:

Copy to Clipboard

model_name: Use the name of the model. If it cannot be searched locally, it will be automatically downloaded online and may need to be executed twice.

4. Result Visualization:

Copy to Clipboard

# Loop through the video frames
while cap.isOpened():
    # Read a frame from the video
    success, frame = cap.read()

if success:
        # Run YOLO tracking on the frame, persisting tracks between frames
        results = model.track(frame, persist=True)

# Get the boxes and track IDs
        boxes = results[0].boxes.xywh.cpu()

if results[0].boxes.id is None:
            track_ids = []
        else:
            track_ids = results[0].boxes.id.int().cpu().tolist()

# Visualize the results on the frame

annotated_frame = results[0].plot()

# Plot the tracks
        for box, track_id in zip(boxes, track_ids):
            x, y, w, h = box
            track = track_history[track_id]
            track.append((float(x), float(y)))  # x, y center point
            if len(track) > 30:  # retain 90 tracks for 90 frames
                track.pop(0)

# Draw the tracking lines
            points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))
            cv2.polylines(annotated_frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

# Display the annotated frame
        cv2.imshow("Object Tracking", annotated_frame)

videoWriter.write(annotated_frame)

# Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        # Break the loop if the end of the video is reached
        break

window_name: visualization window name
thickness: line thickness for trajectory visualization

Complete Demo

Copy to Clipboard

from collections import defaultdict

import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"

import cv2
import numpy as np

from ultralytics import YOLO

# Load the YOLOv10 model
model = YOLO("yolov10b.pt")

# Open the video file
video_path = "demo/demo.mp4"
cap = cv2.VideoCapture(video_path)

# Store the track history
track_history = defaultdict(lambda: [])

fps = cap.get(cv2.CAP_PROP_FPS)
size = (
    int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
    int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
)

videoWriter = cv2.VideoWriter('output.mp4', cv2.VideoWriter.fourcc('m', 'p', '4', 'v'), fps, size)

# Loop through the video frames
while cap.isOpened():
    # Read a frame from the video
    success, frame = cap.read()

if success:
        # Run Object tracking on the frame, persisting tracks between frames
        results = model.track(frame, persist=True)

# Get the boxes and track IDs
        boxes = results[0].boxes.xywh.cpu()

if results[0].boxes.id is None:
            track_ids = []
        else:
            track_ids = results[0].boxes.id.int().cpu().tolist()

# Visualize the results on the frame

annotated_frame = results[0].plot()

# Display the annotated frame
        cv2.imshow("Object Tracking", annotated_frame)

videoWriter.write(annotated_frame)

# Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        # Break the loop if the end of the video is reached
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()
videoWriter.release()

Copy to Clipboard