Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation to inform users about Rhat warning #419

Merged
merged 1 commit into from
Aug 20, 2024
Merged

Conversation

nociale
Copy link
Collaborator

@nociale nociale commented Aug 2, 2024

Closes #288

@nociale nociale requested review from gowerc and wolbersm August 2, 2024 07:57
@gowerc gowerc changed the title #288 Documentation to inform users about Rhat warning Aug 2, 2024
@gowerc
Copy link
Collaborator

gowerc commented Aug 2, 2024

@nociale - I was thinking about this a bit more and I was wondering if it would be better instead to resolve this by just having a higher sample amount and then thinning down to the requested number of samples ? e.g. if the user selects n_samples = 50 sampling 500 and then select 50 from the 500.

That would (hopefully) supress the warning but also provide better diagnostics and hopefully ensue that the samples are actually cover the posterior distribution space. With Rhats >> 1 I would be worried our samples aren't a good coverage of the distribution

@nociale
Copy link
Collaborator Author

nociale commented Aug 5, 2024

Hi @gowerc, I think your proposal could be implemented by setting the thin argument of stan() call to the default 1, doing burn_in + burn_between*n_samples with the thinning of 1, and then select only the samples every burn_between (after the burn_in). Was this your idea?

This would maybe get rid of the warning because the diagnostics are run in a larger amount of MCMC iterations, but it would be overriding the choice of stan to run the diagnostics on the n_samples only.

Despite I do think this would be an elegant solution to the problem, I would maybe still go with the note to the warning message in the documentation looks as it is an easy and reasonable solution. With a sufficiently high number of samples, the warning should disappear anyway.

I am tagging @wolbersm as well as we discussed this together earlier today.

@wolbersm
Copy link
Collaborator

wolbersm commented Aug 6, 2024

Thanks @nociale. As discussed, this reflects my opinion on the topic as well.

@gowerc gowerc merged commit 0aecd54 into main Aug 20, 2024
5 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Aug 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Warning message "The largest R-hat is 2.12, indicating chains have not mixed" often appears for low n_samples
3 participants