Training with '3.train.ipynb' does not yield good results.

[zifeiyu-tan]

I collected my own data using '1. collect_data.ipynb', only changing the seed to 1 and leaving all other parameters unchanged. Then, I trained the model for 200,000 steps with '3.train.ipynb'. However, the resulting model performs poorly, and the robotic arm is constantly twitching.
Can you tell me why?

[JinLong-Qin]

I collected my own data using '1. collect_data.ipynb', only changing the seed to 1 and leaving all other parameters unchanged. Then, I trained the model for 200,000 steps with '3.train.ipynb'. However, the resulting model performs poorly, and the robotic arm is constantly twitching. Can you tell me why?

Hi, we encountered the same issue.

We hypothesise that during evaluation in 3.train.ipynb, the model was fed a sequence of static images stored in the dataset.

However, when deployed in the virtual environment in 4.deploy.ipynb, the images fed to the model for inference were captured by the robotic arm at 20Hz intervals.

This implies that during the inference process, the robotic arm will continue to move in real time. By the time the model outputs its movement strategy, the arm has already reached its next position.

This causes cumulative error, resulting in persistent jitter observed in the virtual environment.

[zifeiyu-tan]

I collected my own data using '1. collect_data.ipynb', only changing the seed to 1 and leaving all other parameters unchanged. Then, I trained the model for 200,000 steps with '3.train.ipynb'. However, the resulting model performs poorly, and the robotic arm is constantly twitching. Can you tell me why?

Hi, we encountered the same issue.

We hypothesise that during evaluation in 3.train.ipynb, the model was fed a sequence of static images stored in the dataset.

However, when deployed in the virtual environment in 4.deploy.ipynb, the images fed to the model for inference were captured by the robotic arm at 20Hz intervals.

This implies that during the inference process, the robotic arm will continue to move in real time. By the time the model outputs its movement strategy, the arm has already reached its next position.

This causes cumulative error, resulting in persistent jitter observed in the virtual environment.

Thank you for your reply. When I use the author's original model, the aforementioned issue does not occur—the robot arm does not jitter and completes the task normally. Additionally, it seems that the frequency (Hz) was indeed not set during training.

[JinLong-Qin]

I collected my own data using '1. collect_data.ipynb', only changing the seed to 1 and leaving all other parameters unchanged. Then, I trained the model for 200,000 steps with '3.train.ipynb'. However, the resulting model performs poorly, and the robotic arm is constantly twitching. Can you tell me why?

Hi, we encountered the same issue.
We hypothesise that during evaluation in 3.train.ipynb, the model was fed a sequence of static images stored in the dataset.
However, when deployed in the virtual environment in 4.deploy.ipynb, the images fed to the model for inference were captured by the robotic arm at 20Hz intervals.
This implies that during the inference process, the robotic arm will continue to move in real time. By the time the model outputs its movement strategy, the arm has already reached its next position.
This causes cumulative error, resulting in persistent jitter observed in the virtual environment.

Thank you for your reply. When I use the author's original model, the aforementioned issue does not occur—the robot arm does not jitter and completes the task normally. Additionally, it seems that the frequency (Hz) was indeed not set during training.

Are you referring to the original task involving moving the cup? Have you attempted replacing the cup with another object to construct a short-range task involving moving a different item?

We plan to validate the short-range task of moving a single non-cup object to determine whether the entire project can operate successfully (as we haven't actually achieved any meaningful results in this project so far). If so, we intend to explore extending the project to other long-range tasks.

[zifeiyu-tan]

I collected my own data using '1. collect_data.ipynb', only changing the seed to 1 and leaving all other parameters unchanged. Then, I trained the model for 200,000 steps with '3.train.ipynb'. However, the resulting model performs poorly, and the robotic arm is constantly twitching. Can you tell me why?

Hi, we encountered the same issue.
We hypothesise that during evaluation in 3.train.ipynb, the model was fed a sequence of static images stored in the dataset.
However, when deployed in the virtual environment in 4.deploy.ipynb, the images fed to the model for inference were captured by the robotic arm at 20Hz intervals.
This implies that during the inference process, the robotic arm will continue to move in real time. By the time the model outputs its movement strategy, the arm has already reached its next position.
This causes cumulative error, resulting in persistent jitter observed in the virtual environment.

Thank you for your reply. When I use the author's original model, the aforementioned issue does not occur—the robot arm does not jitter and completes the task normally. Additionally, it seems that the frequency (Hz) was indeed not set during training.

Are you referring to the original task involving moving the cup? Have you attempted replacing the cup with another object to construct a short-range task involving moving a different item?

We plan to validate the short-range task of moving a single non-cup object to determine whether the entire project can operate successfully (as we haven't actually achieved any meaningful results in this project so far). If so, we intend to explore extending the project to other long-range tasks.

I have not worked on any other tasks, as I am just starting to learn this content and am still not quite sure how to modify this object. You may refer to projects such as lerobot and mujoco_playground.

[victorming666]

Same issue as above. The model trained following '3.train.ipynb' keeps the arm not moving while the author's pre-trained model works normally. @JinLong-Qin, do you have any idea regarding to configing the HZ param during traing?

[JinLong-Qin]

Same issue as above. The model trained following '3.train.ipynb' keeps the arm not moving while the author's pre-trained model works normally. @JinLong-Qin, do you have any idea regarding to configing the HZ param during traing?

In fact, I skipped step 3 and step 4 altogether and proceeded directly to the subsequent steps with smolvla and pi0.

During my subsequent attempts, I discovered that PnPEnv.env.loop_every(HZ=20) appears to be problematic. This observation arose while performing data collection in step 5, when I employed additional step_env() methods to stabilise object placement, like this:

# in y_env2.py
def reset(self):
      # xxx
      for _ in range(50):
            self.step_env()

def step(self):
      xxx
      for _ in range(10):
            self.step_env()

I found that in the step 5, dataset.add_frame failed to execute correctly, yielding no collected data whatsoever during the collection process – the dataset remained empty.

I suspect this occurred because excessive use of step_env() caused the 20Hz interval to be skipped entirely, resulting in no data capture. My hypothesis is that PnPEnv.env.loop_every(HZ=20) is inherently problematic.

Then, I ceased using PnPEnv.env.loop_every(HZ=20) and instead employed timestamps from the simulation environment to control the data acquisition frequency. The approximate code is as follows:

# in 5.language_env copy.ipynb
xxx
TARGET_HZ = 20
FRAME_DT = 1.0 / TARGET_HZ # 0.05s
last_sim_time = 0.0 # record the simulation time at the last data save

xxx
# in data collection part
current_sim_time = PnPEnv.env.data.time 
if (current_sim_time - last_sim_time) < FRAME_DT:
     return # Skipping the current data collection cycle when it is not yet the scheduled collection time

# update timestamp
last_sim_time = current_sim_time

# original data collection part
agent_image, wrist_image = PnPEnv.grab_image()
xxx
# and I recommend replacing Image with cv2, as processing speeds may be faster

I'm afraid I cannot provide much more information, as I have only just completed step 5 of data collection and have not yet proceeded with the smolvla training process. Consequently, I am unsure whether the data collected in this manner will prove useful.

You might try using timestamps instead. If this works, please do let me know.

I hope you find this helpful.

[cls333429]

为啥我采集一组数据，生成的每一帧图片是一摸一样的，这个怎么解决呢