Jetpack Sync: Checksum performance optimizations for Meta Sync Module

[fgiannar]

During meta Fix Checksums, it's possible to call Automattic\Jetpack\Sync\Modules\Metas::get_objects_by_id with a config of max 1200 metas to fetch from the DB.
Up to know we used to fetch each one of them via a separate DB query.
This PR refactors get_objects_by_id so that we fetch those via a single query.
However, taking into account that the resulting DB query can be very long, we have incorporated a buffer logic that compares the resulting DB query length with a default max length to split the queries if needed.

Proposed changes:

• Automattic\Jetpack\Sync\Modules\Module: Introduce MAX_DB_QUERY_LENGTH class constant, which holds the maximum allowed DB query length.
• Automattic\Jetpack\Sync\Modules\Meta: Refactor get_objects_by_id to fetch all metas via a single DB query
• Automattic\Jetpack\Sync\Modules\Meta: Introduce get_prepared_meta_object private method, used both by get_object_by_id and get_objecst_by_id in order to prepare the meta results in the same expected format.

Other information:

• Have you written new tests for your changes, if applicable?
• Have you checked the E2E test CI results, and verified that your changes do not break them?
• Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

pf5801-1mi-p2

Does this pull request change what data or activity we track or use?

No

Testing instructions:

• We'll need to test with woocommerce_order_itemmeta so make sure you have WooCommerce plugin active and a few orders too
• Wipe your postmeta table on your WPCOM Cache site
• Trigger a Fix Checksum for postmeta
• Confirm the postmeta table on your WPCOM Cache site is populated. Note the total count and some sample metas
• Repeat the same but on trunk
• Confirm the postmeta table on your WPCOM Cache site matches the total count from before and the sample metas you noted hold the same values
• Repeat with commentmeta and woocommerce_order_itemmeta

[github-actions]

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

• To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the update/sync-meta-module branch.

  • For jetpack-mu-wpcom changes, also add define( 'JETPACK_MU_WPCOM_LOAD_VIA_BETA_PLUGIN', true ); to your wp-config.php file.

• To test on Simple, run the following command on your sandbox:

  bin/jetpack-downloader test jetpack update/sync-meta-module
  

  bin/jetpack-downloader test jetpack-mu-wpcom-plugin update/sync-meta-module
  

Interested in more tips and information?

• In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
• Read more about our development workflow here: PCYsg-eg0-p2
• Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Team Review, ...).
  
• ✅ Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

---

Follow this PR Review Process:

1. Ensure all required checks appearing at the bottom of this PR are passing.
2. Choose a review path based on your changes:
   • A. Team Review: add the "[Status] Needs Team Review" label
     • For most changes, including minor cross-team impacts.
     • Example: Updating a team-specific component or a small change to a shared library.
   • B. Crew Review: add the "[Status] Needs Review" label
     • For significant changes to core functionality.
     • Example: Major updates to a shared library or complex features.
   • C. Both: Start with Team, then request Crew
     • For complex changes or when you need extra confidence.
     • Example: Refactor affecting multiple systems.
3. Get at least one approval before merging.

Still unsure? Reach out in #jetpack-developers for guidance!

---

Jetpack plugin:

The Jetpack plugin has different release cadences depending on the platform:

• WordPress.com Simple releases happen semi-continuously (PCYsg-Jjm-p2).
• WoA releases happen weekly.
• Releases to self-hosted sites happen monthly. The next release is scheduled for February 4, 2025 (scheduled code freeze on February 4, 2025).

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

[github-actions]

Code Coverage Summary

Coverage changed in 1 file.

File	Coverage	Δ%	Δ Uncovered
projects/packages/sync/src/modules/class-meta.php	1/70 (1.43%)	-1.20%	32 💔

Full summary · PHP report · JS report

[darssen]

Can we extract the code inside the foreach to a method and call it inside the ifs of lines 80 and 87?

[fgiannar]

Nice suggestion! Done with 92e223e but I refactored a lot of things so we'll need to re-test everything now to be on the safe side.

[darssen]

Thanks for working on this to improve Sync's performance.

I really like the approach and appreciate the unit tests.

Added two comments to try and reduce the number of foreach loops for clarity. Let me know what you think :)

I tested too and it worked flawlessly, so the comments shouldn't be a blocker though.

[darssen]

Thanks for taking care of the comment.

I tested again and thinks still work as expected. Furthermore, it was already noticeable that fetching time was lower than in trunk with my testing site which doesn't have a huge amount of data. Great job!
Really excited to see the impact.