Skip to content

Modifications to generate a rotated new view, but the driving trajectory will deflect, how to resolve this? #41

Description

@gama123123

May I ask if there is a modification plan for rotating the camera 15 degrees around the z-axis? I made this modification in evaluate.py based on another person's answer(in issues).
evaluate.py:

        for idx, viewpoint in enumerate(tqdm(config['cameras'])):
            angle = np.radians(15)
            rotation_z = np.array([
                [np.cos(angle), -np.sin(angle), 0],
                [np.sin(angle),  np.cos(angle), 0],
                [0,             0,            1]
            ], dtype=np.float32)
            viewpoint.R = rotation_z @ viewpoint.R  
            viewpoint.timestamp += 0.05
            viewpoint.update()

cameras.py:

def update(self):
    self.world_view_transform = torch.tensor(getWorld2View2(self.R, self.T, self.trans, self.scale)).transpose(0, 1).cuda()
    if self.cx is not None:
        self.FoVx = 2 * math.atan(0.5*self.image_width / self.fx)
        self.FoVy = 2 * math.atan(0.5*self.image_height / self.fy)
        self.projection_matrix = getProjectionMatrixCenterShift(self.znear, self.zfar, self.cx, self.cy, self.fx, self.fy,
                                                                self.image_width, self.image_height).transpose(0, 1).cuda()
    else:
        self.cx = self.image_width / 2
        self.cy = self.image_height / 2
        self.fx = self.image_width / (2 * np.tan(self.FoVx * 0.5))
        self.fy = self.image_height / (2 * np.tan(self.FoVy * 0.5))
        self.projection_matrix = getProjectionMatrix(znear=self.znear, zfar=self.zfar, fovX=self.FoVx,
                                                     fovY=self.FoVy).transpose(0, 1).cuda()
    self.full_proj_transform = (
        self.world_view_transform.unsqueeze(0).bmm(self.projection_matrix.unsqueeze(0))).squeeze(0)
    print("before rotation:", self.camera_center)
    self.camera_center = self.world_view_transform.inverse()[3, :3]
    print("After rotation:", self.camera_center)
    self.c2w = self.world_view_transform.transpose(0, 1).inverse()
    self.grid = kornia.utils.create_meshgrid(self.image_height, self.image_width, normalized_coordinates=False, device='cuda')[0]

However, the result is that the vehicle's trajectory is also deviated by 15°, causing it to head towards the sidewalk into someone else's house. How should I resolve this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions