[sumac] feat: migrate from Ruby to forum v2 Python app #48

regisb · 2024-10-17T22:07:25Z

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

~~Note that this branch should actually be merged in the "sumac" branch, not nightly. This is pending on the creation of the "sumac" branch.~~ EDIT: PR has ben rebased on top of the sumac branch.

DawoudSheraz · 2024-10-18T13:24:10Z

tutorforum/plugin.py

+(./manage.py lms waffle_flag --list | grep forum_v2.enable_forum_v2) || ./manage.py lms waffle_flag --create --everyone forum_v2.enable_forum_v2
+(./manage.py lms waffle_flag --list | grep forum_v2.enable_mysql_backend) || ./manage.py lms waffle_flag --create --everyone forum_v2.enable_mysql_backend


where will the migration of content from mongo to MySQL take place?

Good question :) For now I've left it as a manual step. Basically, people would have to run:

./manage.py lms forum_migrate_course_from_mongodb_to_mysql --no-toggle all ./manage.py lms waffle_flag --create --everyone forum_v2.enable_forum_v2 ./manage.py lms forum_delete_course_from_mongodb all

Source: https://github.com/edly-io/forum/?tab=readme-ov-file#migration-from-mongodb-to-mysql

But ideally these steps should be performed automatically as part of tutor local upgrade --from=redwood. But the problem is that currently there exists no filter for that. We should probably create a UPGRADE_TASKS filter.

ghassanmas · 2024-10-23T12:12:47Z

Happy to see this change, which it shall fruther simplify the maintance and of the platform.
I have few question regarding this change, but most importantly can you confirm if this from Summac there will be in no way to run the forum other than v2? Because looking at this openedx/cs_comments_service/issues/437 I assume for Summac opreator shall have the choice between both?

Faraz32123 · 2024-10-24T07:10:34Z

Happy to see this change, which it shall fruther simplify the maintance and of the platform. I have few question regarding this change, but most importantly can you confirm if this from Summac there will be in no way to run the forum other than v2? Because looking at this openedx/cs_comments_service/issues/437 I assume for Summac opreator shall have the choice between both?

(Testing that operators can perform)Like we have 2 coursewaffle flags i.e. forum_v2.enable_forum_v2 and forum_v2.enable_mysql_backend. And both of these flags will be enabled by default. But yes user can just disable these flags(from lms django admin panel) to use forum_V1 and just play around with these flags and shift to forumV2 course wise.
- Enable forum_v2.enable_forum_v2 for one course and keep forum_v2.enable_mysql_backend disabled. Check forum_v2 working with mongodb(data that forum_V1 created/added in mongodb), everything's fine. Then enable forum_v2.enable_mysql_backend, run migration command for that course. Again check the behaviour. But these are just the testing steps i have written course wise.

I hope this answers most of the part.

ghassanmas · 2024-10-24T07:28:10Z

@Faraz32123 yes I got this part to my understanding once opreator choose to run forum_v2.enable_mysql_backend there is no way back?

Also my inquriy is whether I need to test cs_comments_service or not, and also I assume we can't merge this PR as it's, or is it planned to be merged later after the release of summac?

Faraz32123 · 2024-10-24T08:35:09Z

@Faraz32123 yes I got this part to my understanding once opreator choose to run forum_v2.enable_mysql_backend there is no way back?

yes, there's a way back as long as user don't run this command forum_delete_course_from_mongodb as it will delete the previous data in mongodb but still user can shift to V1 and use it with empty data. But after testing it with mysql, we expect that users with stick to mysql because Forum_v1 will be totally removed/deprecated in future.

Also my inquriy is whether I need to test cs_comments_service or not, and also I assume we can't merge this PR as it's, or is it planned to be merged later after the release of summac?

According to the description that regis added in this PR, we should merge this change in sumac branch.

create a sumac branch from nightly.
change the base branch in this PR to sumac, it will be merged in sumac(we have tested it at our end, you can also do some testing). So that it can be tested before actual sumac release(i.e. merging sumac branch into master).

ghassanmas · 2024-10-24T08:40:44Z

But then if we merge as it's to Summa opreator will not be able to to run forum_v1, (the ruby service), or am I missing something?

ghassanmas · 2024-10-24T09:17:09Z

And also consider this scenario:

An opreator choose to try check v2, they enable and did the mysql migration as well
The platform is running for a week, (learners made comments/ repond to threads..etc)
For somereason they decided to switch back v1 and mongodb

They didn't delete mongodb data, so they shall be able to switch back but data/changes in step 2 will not be reflected back in mongodb I suppose.

This is of course an edge case, but I think we need to mention all of these cases and scenarios in the release notes.

Faraz32123 · 2024-10-24T10:04:51Z

And also consider this scenario:

An opreator choose to try check v2, they enable and did the mysql migration as well

The platform is running for a week, (learners made comments/ repond to threads..etc)

For somereason they decided to switch back v1 and mongodb

They didn't delete mongodb data, so they shall be able to switch back but data/changes in step 2 will not be reflected back in mongodb I suppose.

This is of course an edge case, but I think we need to mention all of these cases and scenarios in the release notes.

Yeah, that's totally fine! We don't expect users to switch back to V1 again and there's no way to migrate data from mysql to mongo.

But then if we merge as it's to Summa opreator will not be able to to run forum_v1, (the ruby service), or am I missing something?

I think to answer this, let's hear it from @regisb, because I was answering it you according to our initial requirements(there might be a twist in it now) and local testing of forum with edx-platform. Although I think Forum_V1 settings should remain there for a while so that it can be tested by the community. Regis can tell better :)

ghassanmas · 2024-10-24T19:00:20Z

tutorforum/plugin.py

+RUN git remote add edly https://github.com/edly-io/edx-platform \
+    && git fetch edly edly/forumv2 \
+    && git merge edly/edly/forumv2""",
+        ),


Applying/merging that path caused an error on tutor sumac branch: 255.9 fatal: refusing to merge unrelated histories

May be edx/forumv2 brnach needs to be rebased on top of open-release/sumac.master?

Alright I think the above error is because tutor makes a shalllow clone, so I tried 1) git merge edly/edly/forumv2 --allow-unrelated-histories, but then there is a huge a merge conflcit. So then I noticed edly branch is quite update to with master/sumac, so I tried git checkout edly/edly/forumv2 but then there I got pypi dependency conflict

I'll let you know after updating(resolving dependency conflict) the edly/forum branch with upstream.

dependency Issues were resolved, forgot to add it here as well. You can test it...

Faraz32123 · 2024-10-25T06:41:16Z

But then if we merge as it's to Summa opreator will not be able to to run forum_v1, (the ruby service), or am I missing something?

@ghassanmas regarding your question, I had a chat with regis. So, right now we are not going to support forum_V1 in the tutor-forum plugin. And if users want to run forum V1, they may have to create a separate plugin and this tutor-forum plugin will remain as it is now :)

changelog.d/20241017_163150_regis_forumv2.md

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

Faraz32123 · 2024-11-01T13:22:34Z

tutorforum/plugin.py

+(./manage.py lms waffle_flag --list | grep forum_v2.enable_forum_v2) || ./manage.py lms waffle_flag --create --everyone forum_v2.enable_forum_v2
+(./manage.py lms waffle_flag --list | grep forum_v2.enable_mysql_backend) || ./manage.py lms waffle_flag --create --everyone forum_v2.enable_mysql_backend


can we add below to the init task of forum... although migrations will run during launch, just a suggestion.
(./manage.py lms showmigrations forum | grep '\[ \]') && ./manage.py lms migrate forum

no need for this condition ./manage.py lms showmigrations forum | grep '\[ \]', it'll throw an error in init jobs if there are no new migrations to apply. The idea was to add ./manage.py lms migrate forum for forum app.

But those migrations will be automatically taken care of during lms init, right? I don't see the need to migrate the forum app individually.

DawoudSheraz · 2024-11-04T03:47:27Z

I am going to merge this PR. Since ES was removed as part of Meilisearch migration (overhangio/tutor#1141), tutor-forum is failing on sandbox deploy (https://github.com/overhangio/openedx-release-demo/actions/runs/11657227578/job/32454473726). Any nits can be addressed via a followup PR.

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

DawoudSheraz requested a review from ghassanmas October 18, 2024 07:15

regisb force-pushed the regisb/forumv2 branch 2 times, most recently from cd4e4f7 to 11d1ef1 Compare October 18, 2024 10:48

DawoudSheraz reviewed Oct 18, 2024

View reviewed changes

pomegranited mentioned this pull request Oct 22, 2024

feat: replace Elasticsearch by Meilisearch overhangio/tutor#1141

Merged

ghassanmas reviewed Oct 24, 2024

View reviewed changes

regisb force-pushed the regisb/forumv2 branch from 11d1ef1 to 0a5913b Compare October 29, 2024 06:42

regisb changed the base branch from nightly to sumac October 29, 2024 06:42

DawoudSheraz reviewed Oct 30, 2024

View reviewed changes

changelog.d/20241017_163150_regis_forumv2.md Outdated Show resolved Hide resolved

DawoudSheraz approved these changes Oct 30, 2024

View reviewed changes

feat: migrate from Ruby to forum v2 Python app

48687c8

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

regisb force-pushed the regisb/forumv2 branch from 0a5913b to 48687c8 Compare October 31, 2024 09:47

DawoudSheraz requested review from Faraz32123 and ghassanmas November 1, 2024 07:00

Faraz32123 reviewed Nov 1, 2024

View reviewed changes

DawoudSheraz merged commit 132507d into sumac Nov 4, 2024

regisb deleted the regisb/forumv2 branch November 4, 2024 07:50

regisb added a commit that referenced this pull request Nov 14, 2024

feat: migrate from Ruby to forum v2 Python app (#48)

e46d770

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

regisb added a commit that referenced this pull request Nov 14, 2024

feat: migrate from Ruby to forum v2 Python app (#48)

c50b736

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

regisb added a commit that referenced this pull request Nov 15, 2024

feat: migrate from Ruby to forum v2 Python app (#48)

57e4eaf

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

DawoudSheraz pushed a commit that referenced this pull request Nov 29, 2024

feat: migrate from Ruby to forum v2 Python app (#48)

3036c4a

Obviously, this is a very important change. But it drastically simplifies this plugin, in the sense that we no longer have to run a separate Ruby container.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[sumac] feat: migrate from Ruby to forum v2 Python app #48

[sumac] feat: migrate from Ruby to forum v2 Python app #48

regisb commented Oct 17, 2024 •

edited

Loading

DawoudSheraz Oct 18, 2024

regisb Oct 18, 2024

ghassanmas commented Oct 23, 2024

Faraz32123 commented Oct 24, 2024

ghassanmas commented Oct 24, 2024

Faraz32123 commented Oct 24, 2024

ghassanmas commented Oct 24, 2024 •

edited

Loading

ghassanmas commented Oct 24, 2024

Faraz32123 commented Oct 24, 2024

ghassanmas Oct 24, 2024

ghassanmas Oct 24, 2024

Faraz32123 Oct 25, 2024

Faraz32123 Nov 1, 2024

Faraz32123 commented Oct 25, 2024

Faraz32123 Nov 1, 2024

Faraz32123 Nov 1, 2024

regisb Nov 4, 2024

DawoudSheraz commented Nov 4, 2024

		(./manage.py lms waffle_flag --list \| grep forum_v2.enable_forum_v2) \|\| ./manage.py lms waffle_flag --create --everyone forum_v2.enable_forum_v2
		(./manage.py lms waffle_flag --list \| grep forum_v2.enable_mysql_backend) \|\| ./manage.py lms waffle_flag --create --everyone forum_v2.enable_mysql_backend

[sumac] feat: migrate from Ruby to forum v2 Python app #48

[sumac] feat: migrate from Ruby to forum v2 Python app #48

Conversation

regisb commented Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghassanmas commented Oct 23, 2024

Faraz32123 commented Oct 24, 2024

ghassanmas commented Oct 24, 2024

Faraz32123 commented Oct 24, 2024

ghassanmas commented Oct 24, 2024 • edited Loading

ghassanmas commented Oct 24, 2024

Faraz32123 commented Oct 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Faraz32123 commented Oct 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DawoudSheraz commented Nov 4, 2024

regisb commented Oct 17, 2024 •

edited

Loading

ghassanmas commented Oct 24, 2024 •

edited

Loading