Visualization of trajectories

[Joker-hang]

Hello, your job is very good! I would like to ask how to convert world coordinates into pixel coordinates in the visualization of dataset testing. I hope to receive your reply

[cocoon2wong]

Hi @Joker-hang !
Thank you for your questions and sorry for the late reply!

In our work, we compute this transformation matrix by labeling several (about 10 pedestrians is enough) pixel positions corresponding to pedestrians in the video and recording their true positions one by one, and then by least squares, since we found that there is almost very little lens aberration in these scenes and the shooting angle is fixed.

Denote their real-world positions as $\mathbf{p}^i$ and the observed pixel positions as $\mathbf{p}_p^i$, we have
$\hat{\mathbf{p}}_p^i = \mathbf{W}_p \mathbf{p}^i + \mathbf{b}_p.$
Here, $\mathbf{W}_p \in \mathbb{R}^{1 \times 2}$ and $\mathbf{b}_p \in \mathbb{R}^{1 \times 2}$ are the weights and bias matrices used in the coordinate transform. They are optimized by minimizing the following loss function:
$\mathcal{L} \left(\hat{\mathbf{p}}_p^i,\mathbf{p}_p^i\right) = \left\Vert\hat{\mathbf{p}}_p^i -\mathbf{p}_p^i\right\Vert^2.$

We can solve the following equation to get compute $\mathbf{W}_p$ and $\mathbf{b}_p$:
$\frac{\mathcal{L}}{\partial \mathbf{W}_p} = 0$.

Take the zara1 as an example, our computation results are saved in

Vertical/datasets/subsets/zara1.plist

Lines 23 to 29 in 178866c

	<key>weights</key>
	<array>
	<real>-42.54748107</real>
	<real>580.5664891</real>
	<real>47.29369894</real>
	<real>3.196071003</real>
	</array>

When visualizing results, other real-world positions can be transformed via

Vertical/codes/training/__vis.py

Lines 130 to 134 in 178866c

	if len(weights) == 4:
	result = np.column_stack([
	weights[2] * r.T[1] + weights[3],
	weights[0] * r.T[0] + weights[1],
	]).astype(np.int32)

Here, weights is the four results, w_x, w_y, b_x, b_y. This way, we can visualize these results in pixel coordinates.

I hope these can help you!
Conghao Wong