Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KITTI Dataset Full Frame Output Mismatch #18

Open
MichealRW opened this issue Dec 18, 2024 · 2 comments
Open

KITTI Dataset Full Frame Output Mismatch #18

MichealRW opened this issue Dec 18, 2024 · 2 comments

Comments

@MichealRW
Copy link

您好!在尝试复现您在 GitHub 上提供的 dufomap 算法时,我遇到了一些问题,希望能够得到您的帮助和指导。

具体来说,当我使用完整的 KITTI 数据集中的所有帧数作为输入时,算法的输出未能达到预期效果。尽管我在 GT 生成方法、被评价算法输出点云生成方法以及评估方法方面均参考了您在 DynamicMap_Benchmark 中提供的实现,但最终的结果并不理想,尤其是 DA 值极低。然而,令人困惑的是,当我只使用部分帧进行测试时,算法的输出较为符合论文中的结果。

希望能通过您的帮助了解可能存在的问题,并获取一些调试或改进的建议。

感谢您的时间与帮助,期待您的回复。
test-05-A

I hope this message finds you well. I am currently working on reproducing the dufomap algorithm that you have provided on GitHub, and I have encountered some issues that I would greatly appreciate your assistance with.

Specifically, when I use the full set of frames from the KITTI dataset as input, the algorithm's output does not meet the expected results. Despite using the same methods for generating the ground truth (GT), the point cloud generation method from the evaluated algorithm, and the evaluation methodology as described in the DynamicMap_Benchmark, the final results are not satisfactory, with particularly low DA values. However, when I test the algorithm with only a subset of the frames, the results are more in line with those presented in the paper.

I would greatly appreciate any insights you might have regarding potential issues or suggestions for debugging or improving the implementation.

Thank you for your time and assistance. I look forward to hearing from you.

@Kin-Zhang
Copy link
Member

Kin-Zhang commented Dec 21, 2024

所有帧数作为输入时,算法的输出未能达到预期效果

是指从可视化结果上,无法达到这个网页的效果吗?https://kth-rpl.github.io/dufomap/ 【第二个部分是Full 07 Sequence】

所有帧数作为输入时 但最终的结果并不理想,尤其是 DA 值极低

这个所有帧数作为输入,大概我没记错应该有4000frames 其中我一年前做的时候 统计过 dynamic/static gt的比例,前者1%不到,所以这就是我还是遵循之前的工作(他们也在报告里提及了这点)的frame 范围进行的评估;
不过我好奇DA值低,可以更具体一点吗?比如哪一个sequence?DA SA AA分别是多少?如果是partial的eval又是多少?
如果可行的话:https://github.com/KTH-RPL/DynamicMap_Benchmark 这个benchmark下还有很多方法,你可以都试试full seq

@MichealRW
Copy link
Author

使用的帧数为00序列的所有帧(4541),使用https://github.com/KTH-RPL/DynamicMap_Benchmark 中的extract_gtcloud生成真值,使用export_eval_pcd导出用于评估的点云文件(整个文件有点大,我这边都有点可视化不了了),最后的评估结果如下图所示。
test_all_00

我还试过使用141 frame(论文中提到的部分)、200 frame、500 frame 、 2000 frame的情况,大概是逐级递减的效果,其中500 frame仍然能保留较好的效果。我猜测可能是遇到了回环部分,导致了一些问题。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants