Detect, then track
Detection finds a target in a broad search space. Tracking follows an already located target through a smaller change. Combining them can reduce work per frame while retaining a way to recover after the target is lost.
Detect a textured plane
Section titled “Detect a textured plane”Extract ORB or SIFT descriptors from the template and current frame. Match compatible descriptor types, reject ambiguous correspondences and fit a homography with a robust method. Check both the number and spatial distribution of inliers before accepting the pose.
Eight tentative matches include two inconsistent pairs, shown in orange.
A transform proposed from an unsuitable sample does not align the plane. RANSAC tries additional small samples.
Project each reference point and compare it with its observed partner. Orange arrows expose the two large residuals; mint rings mark agreeing pairs.
The accepted mapping bends a rectangular grid into perspective. Every grid intersection uses the same projective transform as the inlier points.
Refine the mapping using the accepted correspondences.
Illustrative example. The stages explain the method; they are not a live OpenCV execution.
Follow the pose
Section titled “Follow the pose”Track good points with pyramidal Lucas-Kanade and re-estimate geometry from surviving correspondences. Alternatively, refine an already close warp with ECC when photometric alignment fits the target. Keep the last successful estimate as the next frame’s initialization.
Compare the same structure before and after a coordinate transformation.
Warp the current image using an initial transform.
Measure normalized photometric agreement and its local gradient.
The mint structure now coincides with the reference in this illustrative convergence. Real alignment can stop at a local optimum.
Update the warp parameters until the stopping criterion is met.
Illustrative example. The stages explain the method; they are not a live OpenCV execution.
Decide when to detect again
Section titled “Decide when to detect again”When confidence remains adequate, track the next frame. When tracking fails, support collapses or the warp becomes implausible, return to detection. Thresholds need to be calibrated against your footage; no universal inlier count establishes the right object.
Keep detector output, tracking status and application confidence separate. A tracker returning a box is not proof that it still contains the intended target.