PredictionBook transitioning to read-only mode at the end of 2023

[michaelwebb76]

Over the years, we've been proud to support and witness the growth of this vibrant community. The insightful predictions, thoughtful conversations, and collaborative spirit of our users have made PredictionBook a unique platform in the realm of predictive analytics. Our collective exploration of the future, and our shared lessons from both accurate and off-mark predictions, have made this journey incredibly rewarding.

However, after a careful review and evaluation, we've found that the current level of active users does not justify the resources required for ongoing maintenance. It's a tough decision, but effective January 10, 2024, PredictionBook.com will transition to a read-only mode.

We understand the value of the work everyone has put into their predictions, so we encourage users to migrate to Fatebook.io, a similar platform where you can continue your predictive pursuits. We believe Fatebook.io will serve as a good home for the PredictionBook community.

Important note: You can also export your data from PredictionBook in CSV format at any time if you'd like to take it to another platform. You will find the command to do this on your profile page.

Please be aware that one year after becoming read-only, PredictionBook.com is scheduled for permanent shutdown.

We want to thank each and everyone of you for your invaluable contributions to PredictionBook. The spirit of our community has always been our biggest achievement, and although the site's active run is ending, the insights and knowledge shared will continue to inspire.

[niplav]

Having used PB for more than half a decade, thank you a lot for work on it. Is there any chance you will publish a static dump of the website somewhere?

[michaelwebb76]

@niplav For it to be truly static, it would only include the publicly accessible predictions made on the site... and I think something like the Wayback Machine would already cater for something like that? We don't plan to continue to maintain a server or database after Jan 2025, so authenticating would no longer be possible.

[brunoparga]

@michaelwebb76 I think a programmatically-accessible dump would be more valuable than the Wayback Machine. Do you think we could have that?

[michaelwebb76]

@brunoparga I don't know how that could work in a way that kept data private in the same way the application currently does, without us having to continue support the application beyond the cutoff. So no, I don't think so.

[brunoparga]

@michaelwebb76 if you don't mind me asking, is that mostly about user data like password hashes, and private predictions? Couldn't that just be deleted from the DB before taking the dump?

[michaelwebb76]

This comment doesn't quite make sense to me - perhaps I am just not understanding what you're actually wanting here.

The current site is a Rails application sitting on top of a Postgres database, all in an EC2 instance. We no longer have the time or skillset to maintain a Rails application or its dependencies in a way that will keep the application safe from the inevitable security vulnerabilities that will arise over time.

How are you imagining that its possible for us to provide a programmatically accessible dump of the site without having to maintain the Rails app? If you know of any services that can crawl the public site and produce a static snapshot that could be stored in S3, I'd be open to that, but it wouldn't have a non-HTML API any more.

[jacobgreenleaf]

Hi, are you willing to entertain credible purchase offers? I've been using PredictionBook for a while, I have experience running Rails applications as well as other scaled Internet infrastructure and services, and would be happy to take the maintenance and development burden from you.

[matthewfallshaw]

Hi @jacobgreenleaf
I'm Matt, one of the founders.

First I note that this code is open source, and if you want to set up your own instance of it (on your own domain) and invite users to move over, you’re very welcome to do that (and users can already export their own data).

If you want either the domain, or to provide continuity of service to existing users, I'll lay out some of our goals and concerns to see if that helps make the sort of proposals we'd consider clear:

1. We take the trust placed in us by our users very seriously, and I can't currently imagine accepting any proposal that involves transmitting their private information or predictions to a new maintainer of the site without each user individually consenting.
2. I'd like to be able to provide continuity of service to our users, but this priority is significantly lower than 1.
3. I do care about our name and reputation, so would like any transition plan to include a prominent notice displayed for, say, six months, indicating that the handover had occurred and we were no longer involved.
4. We're not looking to gain from this handover, but we don’t have a lot of time available to implement complicated handover schemes ourselves.

Point 1 above seems like the main challenge. I can imagine two basic approaches to handling this concern (and I think I can’t quite make the second work):

1. We give all users time to explicitly consent to their private information staying in the db after handover, and, before handover, we delete the private information (email address and private predictions) of all of those who did not consent. This seems relatively easy to implement but likely to lead to many users not consenting, their private predictions being deleted, and the new owner of the site becoming unable to email them unless and until they log in and re-supply their email address.
2. As above for email address, and someone proposes an encryption scheme that convinces us that the new site owner can't read users’ private predictions (at least until they do something that implies consent). I don't think that this is mathematically complicated, but I think that it is practically complicated since the private predictions are not currently encrypted under such a scheme. (For the avoidance of doubt: I would not consider handing over users’ securely encrypted private information to conflict with our first concern if only the user held the decryption key.)

If you were serious about taking over it would also seem reasonable to notify at least “active” users to invite other proposals to take over the site. We’re not looking to make money from this, but something like a Request For Proposals (RFP) process, judged by us on likely benefits to the active users (I don’t expect to actually receive any other submissions should we run such a process, but it seems like the right thing to do to at least invite them).

So, if you or anyone else is interested in taking over the domain, this public repo and the existing db, any proposals would have to include detailing how the above concerns would be addressed, then we run an RFP, then you (and any other strong RFP submitters) submit any required code to support the transition. We would review that code and host it during the handover period, then actually hand over the site once we were comfortable that we were not sharing private information that our users had not consented to being shared.

… and all of that is rather a lot. I am open to other suggestions that address the concerns listed above.

[jacobgreenleaf]

Hi Matt, thank you for outlining some of the privacy concerns among others that I hadn't considered. I think it may not be the best use of our time, at least for me personally as I am less interested in owning and operating the site than in easing the migration process so that it's a bit more turnkey.

It sounds like we could use some tooling that can be used on top of the existing export CSV to assist users in migrating to Fatebook or other platforms.

Is there anything you can think of that isn't covered by the existing export that users might want to preserve when migrating? I am noticing after a quick scan of both the exported data and the table schema that perhaps tags are not, perhaps group memberships are not as well. Would you accept changes to this repository that add additional export facilities if needed?

[michaelwebb76]

@jacobgreenleaf We're still absolutely open to contributions to the repository, especially if they aid in this transition.

[brunoparga]

@michaelwebb76 I think my idea is simpler than somehow keeping the site active, while addressing your very valid privacy concerns.

The process would be:

1 - on the date the site is scheduled to become read-only, make a copy of the final state of the database;
2 - delete from the copy all information you're not comfortable sharing (e.g. e-mail addresses and private predictions);
3 - use pg_dump to generate an SQL dump file of the remainder of the data (i.e. public predictions with their comments - I personally don't even mind if all users are anonymized, as long as distinct users are given distinct pseudonyms);
4 - share that SQL dump file.

That way, I and other people would be able to restore the data to analyze it directly using PostgreSQL or other tools. Keep in mind I am only talking about data that can already be accessed by anyone; this access is currently via the Web page, but that is going away so I'm proposing a different way to access it.

[matthewfallshaw]

@brunoparga, that seems like a very reasonable process and addresses the concerns that @michaelwebb76 and I have about private user data.
Mike might be an even lovelier fellow than I currently believe him to be (for the avoidance of doubt: a very lovely fellow 😁), or might have some trick to find the time to do what you describe (other than his normal trick of just burning his own life force to work longer hours)… but I know that he's busy, and what you describe sounds like something that both Mike and I would really appreciate receiving as a pull request (for, say, a migration? or a Rake task? or go hard and write it all in sed? 🤪 or anything else that produces the extract you describe). Any chance you or anyone else would help me preserve Mike's life force so that he can spend it on his family instead?