tensorflowではモデルの読み出しができなかったため、kerasを使用したところ読み出しに成功した。先週は、1匹分のモデルを作成し、識別を実行したが、犬の画像を猫と判別してしまったため、プログラムを見直す必要がある。

大ゼミの発表のためのスライド作成をしました。

遠い人でも、認識できるように超解像について調べています。最初は一枚の画像を超解像してみようと思います。 RealESRGANというモデルが超解像するときに画像のぼやけが少なそうなので試してみます。一枚の画像からでは入力情報が足りなそうな場合は、難しそうですが一定時間の動画から複数枚の画像に分けて入力して超解像してみます。

京セラ、NTT西日本、村田製作所に応募

申し訳ありませんが、自動車免許取得のため本日のゼミは欠席します。

意外と早く終わったので出れそうです。

First run the CenterPoint (3-D MOT) algorithm successfully

It took me about 6 days to establish the complete conda envoriment and run the Python algorithm of CenterPoint in my system (Ubuntu 20.04.6).

There is still a problem that the DCN part can be compiled, but it won’t run normally with my torch (1.12.0), an older version should be applied, but it  will cause some other problems.

Problems records

1. not implemented for CPU ONLY build.

The CenterPoint contains two third-part libs, git clone cannot download both of them. They should be cloned desperately. And then, re-adding the CUDA path, CenterPoint path and third-part lib path to .bashrc. Definitely, one of libs can be installed by pip without compiling…… After that, this problem was solved.

2. RuntimeError:

———————————–File“/home/omaqzy/PycharmProjects/CenterPoint/det3d/ops/dcn/deform_conv.py”, line 93, in backward   cur_im2col_step)

RuntimeError: view size is not compatible with input tensor’s size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(…) instead.

————————————–

This is the problem about DCN. Replaced the AT_CHECK with TORCH_CHECK in file deform_conv_cuda.cpp and deform_pool_cuda.cpp, so that the DCN can be compiled. When I run it to train the model, I will face the above problem.

The train result:

NEXT… …

Solve the above problem.

Learn the 3D-Detection and try to modify the algorithm.

This Week

This three version code can be seen in my gitlab

• I first coded my ideal by a numpy version, but the time is very long. Because I extract numbers from the GPU to the CPU, analyze and send back to the GPU … Continue Reading ››
  

This week I am reading  some papers about long term object tracking  and downloading  related tracking datasets. Compared to  normal object tracking datasets, the long term object datasets lack sources and are hard to download. Everything needs to start from scratch.

I want to find a accurate method to evaluate whether the tracked target is absent . Most of paper exploit  confience score to judge the suituation of tracking performence but I think it is not realiable.  The tragectory predictor is expected to correct the tracker. But only one paper adopts this way, using a distribution to guess a location.

And the metrics of long term tracking  are F1-score, Precision and Recall,which differs from single object tracking. But these papers did not test on the same benchmark, which make me diffcult to evaluate the trackers. (The most usual benchmark are VOT2020LT. )

The specific analysis of  papers are as follows:

1. Combining complementary trackers for enhanced long-term visual object tracking

The paper’s purpose:

Cmbining the capabilities of baseline trackers in the context of long-term visual object tracking.

Contributions:

(1) Proposing a strategy which can perceive whether the two trackers are following the target object through an online learned deep verification model to  select the best performing tracker as well as it corrects their performance when failing.

Personal Evaluation:

The long term tracker usually own the ability to switch the local tracker and global tracker.This paper may propose a novel way.

Details:

There are two trackers to run, namely Stark and SuperDiMP.

Stark can predict a more accurate bounding box.

SuperDiMP can output a more score to judge whether target is lost.

The two trackers are running together and the paper propose a policy to select which tracker is better in the t th frame.

The judgement are decided by confidence score and the verifier.

The verifer are trained offline and updated by image patch online.

Deep evaluation: Very simple design and easy to reproduce. The key technology is a simple deep binary classification. The training data can be collected from training process. But the paper did not provide code and I don’t know if it really work.

The  flow chart are as follows:

 

 

2. Target-Aware Tracking with Long-term Context Attention

The paper’s purpose:

(1) Unlike siamese tracker,  exploiting contextual information.

(2) Coping with large appearance changes, rapid target movement, and attraction from similar objects.

Contributions:

(1) Proposing a LCA (transformer ) which uses the target state from the previous frame to exclude the interference of similar objects and complex backgrounds.

(2) Proposing a concise and efficient online updating approach based on classification confidence to select highquality templates with very low computation burden.

Personal Evaluation:

A normal tracker with new transformer.

 

Details:

The first contribution rely on the Swin transformer.

The template update mechanisms is key contributions. It exploits the continuous confidence score. The normal mean of T templates are  not enough and it use the threshold and penalty.

Deep evalutaions:

Its contributions is too simple and easy to reproce.

The paper provide the code and I should read it.

 

3. Multi-Template Temporal Siamese Network for Long-Term
Object Tracking

The paper’s purpose:

(1) Avoiding defects of  siamese tracker.

(2) Coping with target appearance changes and similar objects .

Contributions:

(1) Learning the path history by a bag of templates

(2) Projecting a potential future target location in a next frame.

Personal Evaluation: Competitor! Consensus of ideas. Worth reading.

Details:

Using the confidence score of RPN head to update templates. The first mechanism exploit  confidence score and  weights to compute the similarity.

The second mechanism is to predict possible bbox but it is a passing. It should be the most important part.

The second mechanism exploit the latent bbox and similarity to compute a reliability score.

Deep evaluation: Normal paper. But it provide the code. So I can read the code the find the part of predicting bbox.

 

 

4. SiamX: An Efficient Long-term Tracker Using Cross-level Feature
Correlation and Adaptive Tracking Scheme

The paper’s purpose:

(1) Improving siamese tracker.

(2) large variance, presence of distractors, fast motion, or target disappearing and the like .

Contributions:

(1) Exploiting cross-level Siamese features to learn robust
correlations between the target template and search regions.

(2) Proposing inference strategies to prevent tracking loss
and realize fast target re-localization.

Personal Evaluation: Normal. I want to know how to re-locate. Worth reading.

Details: The key contributions is to compute the probability
distribution of the target and maintain a heat-map. The heatmap is only used to judge if the target is loss. I dont know how to  search the most possible region.

Deep evalutions: No code. The key contributions seems not reliable.

 

5. ‘Skimming-Perusal’ Tracking: A Framework for Real-Time and Robust Long-term Tracking

The paper’s purpose:

(1) Traditional long-term tracker are limited.

(2) To find the missing target object and accerlate the speed of global search.

Contributions:

(1) Determining whether the tracked object being present or absent, and then chooses the tracking strategies of local search or global search respectively in the next frame.

(2) Speeding up the image-wide global search,
a novel skimming module is designed to efficiently choose
the most possible regions from a large number of sliding
windows.

Personal Evaluation: The global search  is useful. Worth reading.

Details: The key contributions is that the skimming global seach.

Another contributions  is to judge whether the target is loss.

Its approach expoits the similarity between template and candidate bboxes to judge whether the target is loss.

Deep evalution: The judge mechanism is rough but efficient. It provide the code. I want to check the skimming global search.

I am reading  some novel papers about long term object tracking in 2023.

Compare to  normal object tracking, the long term object tracking must have the ability to find the missing target object. The papers are as follows:

1. “MFT: Long-Term Tracking of Every Pixel”

The paper’s purpose:

Solving the challenges, such as dense,  pixel-level, long-term tracking.

Contributions:

(1) The approach exploits optical flows estimated not only between consecutive frames, but also for pairs of frames at logarithmically spaced intervals.

(2) It then selects the most reliable sequence of flows on the basis of estimates of its geometric accuracy and the probability of occlusion, both provided by a pre-trained CNN.

Personal Evaluation:

The paper exploits a new routine, which is worth reading.

2. Combining complementary trackers for enhanced long-term visual object tracking

The paper’s purpose:

Cmbining the capabilities of baseline trackers in the context of long-term visual object tracking.

Contributions:

(1) Proposing a strategy which can perceive whether the two trackers are following the target object through an online learned deep verification model to  select the best performing tracker as well as it corrects their performance when failing.

Personal Evaluation:

The long term tracker usually own the ability to switch the local tracker and global tracker.This paper may propose a novel way.

3. Target-Aware Tracking with Long-term Context Attention

The paper’s purpose:

(1) Unlike siamese tracker,  exploiting contextual information.

(2) Coping with large appearance changes, rapid target movement, and attraction from similar objects.

Contributions:

(1) Proposing a LCA (transformer ) which uses the target state from the previous frame to exclude the interference of similar objects and complex backgrounds.

Personal Evaluation:

A normal tracker with new transformer.

4. Multi-Template Temporal Siamese Network for Long-Term
Object Tracking

The paper’s purpose:

(1) Avoiding defects of  siamese tracker.

(2) Coping with target appearance changes and similar objects .

Contributions:

(1) Learning the path history by a bag of templates

(2) Projecting a potential future target location in a next frame.

Personal Evaluation: Competitor! Consensus of ideas.Worth reading.

5. SiamX: An Efficient Long-term Tracker Using Cross-level Feature
Correlation and Adaptive Tracking Scheme

The paper’s purpose:

(1) Improving siamese tracker.

(2) large variance, presence of distractors, fast motion, or target disappearing and the like .

Contributions:

(1) Exploiting cross-level Siamese features to learn robust
correlations between the target template and search regions.

(2) Proposing inference strategies to prevent tracking loss
and realize fast target re-localization.

Personal Evaluation: Normal. I want to know how to re-locate. Worth reading.

6. ‘Skimming-Perusal’ Tracking: A Framework for Real-Time and Robust Long-term Tracking

The paper’s purpose:

(1) Traditional long-term tracker are limited.

(2) To find the missing target object and accerlate the speed of global search.

Contributions:

(1) Determining whether the tracked object being present or absent, and then chooses the tracking strategies of local search or global search respectively in the next frame.

(2) Speeding up the image-wide global search,
a novel skimming module is designed to efficiently choose
the most possible regions from a large number of sliding
windows.

Personal Evaluation: The global search  is useful. Worth reading.