内定者懇親会のためゼミは欠席します。

1.learn the framework of yolo-v8

2. learn the seam attention mechanism（https://arxiv.org/abs/2208.02019）

3. Try to come up with a better data fusion framework

4.Evaluation mechanisms are learning

面接があるので今週のゼミはお休みします。

The details of some long-term trackers.

1 . SiamX: An Efficient Long-term Tracker Using Cross-level Feature Correlation and Adaptive Tracking Scheme.

The key is “ADAPTIVE TRACKING SCHEME”.

（1）Momentum Compensation.

Exploit the concept “fast motion” to judge whether the target object is lost.

“If the target displacements between consecutive frames exceeds target sizes, it considers the target object is at a fast-moving state. To avoid targets leaving the search regions, the search center drifts in the direction of momentum:”

conclusion： Fake paper. Its codes lack the long-term tracker.

2. Combining complementary trackers for enhanced long-term visual object tracking.

Running two trackers.

But we can use its score’s method to re-detect.

3. GUSOT: Green and Unsupervised Single Object Tracking for Long Video Sequences

if  s1(f∗, x1) > s1(f∗, x2) and s2(f∗, x1) ≤ s2(f∗, x2) :

re-detect else: continue.

Key: motion residual. The key is “UHP-SOT”

4. High-Performance Long-Term Tracking with Meta-Updater

(1) appearance model (lstm)

(2) re-detection( the flag of DiMP ? )

Conclusion: Another fake paper. The most important point is DIMP !

5. UHP-SOT: An Unsupervised High-Performance Single Object Tracker(2017)

Methods: It has three trackers:

(1) Trajectories-based box prediction ( principal component analysis)

(2) Background motion modeling ( optical flow)

(3) Appearance model (normal tracker)

6. Object Tracking Using Background Subtraction and Motion Estimation in MPEG Videos (2005)

Key: Using four corner to compute the motion of background(Optical flow).

7. Fast Object Tracking Using Adaptive Block Matching(2005)

Key: Exploiting ‘Mode filter’ in order to straighten up noisy vectors (Optical flow) and thus eliminate this problem.

1. Finish paper "Corn plant phenotypic parameters extraction algorithm based on keypoint detection and stereo images"
2. Change my paper "FCOS_Stereo"

The long term object tracker requires the tracker to be able to retrieve lost targets. So I want to predict the possible locations where the target might appear based on the historical motion trajectory of the object.  Trajectory prediction requires the camera motion and current object tracking method can’t provide camera information,such as camera pose or motion.

In order to get depth map and camera poses, I am reading papers about slam with monocula camera, involving unsupervised learning.

1. Future Person Localization in First-Person Videos

Purpose:  predicting future locations of people observed in first-person videos.

key point :a)  ego-motion  b) Scales of the target person. c) KCF for tracking d) feature concatenating.

evaluation: Excellent introductory work. But how to get ego-motion information?

2. Unsupervised Learning of Depth and Ego-Motion from Video

Purpose: Presenting an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences.

Key point: a) Visual synthesis b)  unsupervised learning.

Evaluation: Nice paper. But it still need camera intrinsics.

3. Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras

Purpose: Presenting a novel method for simultaneously learning depth, egomotion, object motion, and camera intrinsics from monocular videos, using only consistency across neighboring video frames as a supervision signal.

Key opint: a) Generating camera intrinsics.

Evaluation: Nice paper. Rrovide code. But it may be too slow.

SOT Summary

The work summary for the last week and this week is as follows: Reviewed the progress and main contributions of single object tracking algorithms over the years (2010~2023), summarizing the respective advantages of several representative algorithms. In conjunction with single object tracking, I explored the development history of 2D to 3D detection and tracking.

I learned how to use Inkscape to draw image with format of .svg and output results without distortion.

Center-based

The main contribution of CenterPoint lies in its adoption of a center-based detection head, which simplifies the process of dealing with object positions by not considering their intrinsic orientations. This significantly reduces the search space of the object detector, especially beneficial when vehicles drive on straight roads. Traditional anchor-based methods struggle to fit axis-aligned bounding boxes to rotated objects during critical maneuvers like a left turn.

Method

CenterPoint proposes representing, detecting, and tracking 3D objects as points. By regressing to 3D bounding boxes directly at the center point without voting, this method uses a single positive cell for each object alongside a keypoint estimation loss. A two-stage 3D detector with a Lidar-based backbone network identifies the centers of objects and their attributes, with a second stage refining all estimates.

Input

3D object detection aims to predict three-dimensional rotated bounding boxes. CenterPoint directly regresses to 3D bounding boxes through features at the center point without voting. This method employs a single positive cell for each object and utilizes a keypoint estimation loss. The two-stage detector extracts sparse features of 5 surface center points from the intermediate feature map.

Center-based Detection Head

Initially, six tasks are generated in the code, with the nuScenes dataset comprising 10 categories. Four tasks have two categories each. The separation of regression for ‘car’ and ‘pedestrian’ is due to the relatively large difference in size, allowing both to achieve higher accuracy more easily. Grouping ‘pedestrian’ and ‘traffic cone’ in the same task is because their sizes from a Bird’s Eye View (BEV) are similar, which can avoid generating two different categories of targets at the same position. Every task has the same head, and each head has a similar structure, including predictions for offset, height, dimensions, rotation angle, velocity, and a heatmap center.

Loss

• Regression loss for dimensions, offset, height, rotation: The loss is calculated based on the differences between the predicted annotation boxes and the target boxes. The localization loss sums up all these component losses, and the total loss is a weighted sum of heatmap loss and localization loss.

3D Object Tracking

Many 2D tracking algorithms can directly track 3D objects out of the box. However, dedicated 3D trackers based on 3D Kalman filters still have an advantage as they better exploit the three-dimensional motion in a scene.