[Feedback] PR Review Bot

[lupyuen]

Please provide your feedback for the NuttX PR Review Bot. Thanks!

• How It Works: "LLM Bot that reviews Pull Requests for Apache NuttX RTOS"
• Earlier Discussion: Update GitHub Pull Request Template. #13494 (comment)
• Source Code for PR Bot: nuttx-pr-bot
• PR Tips: "Preparing a Pull Request for Apache NuttX RTOS"

[raiden00pl]

My comment on this message from bot:

Squash The Commits: This PR contains 2 Commits. Please Squash the Multiple Commits into a Single Commit.

A suggestion like this shouldn't be based solely on the number of commits in PR.
What should be squashed and what shouldn't require a greater understanding of PR content.
Having multiple commits in one PR isn't always bad. Sometimes it's even necessary.

[lupyuen]

A suggestion like this shouldn't be based solely on the number of commits in PR.
What should be squashed and what shouldn't require a greater understanding of PR content.

Sorry @cederom and @acassis: What are your thoughts about this? Shall I remove the "Squash Commit" message, since our bot can't analyse the PR Content? Thanks!

[raiden00pl]

maybe we can just change "Please squash commits" to "consider squashing commits if that makes sense for this PR" (or something similar)

[lupyuen]

OK I removed the check for Squash Commits, I don't we're ready to handle it now :-)

[acassis]

Thank you @lupyuen I think it is better not to include it to avoid inducing contributors to wrong direction.

It could be something that reviewers could do better

[nuttxpr]

Yes we need increase PR and git commit descriptions quality (both need to be self explanatory).. bot was created to help.. you can .. wait was there this nice copy-paste section where all sections were filled in already before

@cederom Yeah that's the problem with LLMs, we can't force it to say the same thing all the time :-) It seems to have a mind of its own / multiple personas?

[resyfer]

PR Text:

## Testing
...
Target(s): arch(sim), sim:mnemofs.

Bot reply:

Target Specificity: "sim:mnemofs" is good, but be even more specific if possible (e.g., which simulator, exact config used). More detail makes it easier for reviewers to reproduce your results.

This might be due to less fine tuning for the sim cases?

[resyfer]

A suggestion to add a way to trigger the bot again. For example...if it gave some feedback, and I improved it, I don't have any way of checking it again without creating another PR (which would mean wasted GH checks as PR text and build runs are often unrelated).