feat: add DATA WAREHOUSE to integrations object in shopify pixel source

[yashasvibajpai]

What are the changes introduced in this PR?

We will need to append this integrations object to every incoming event coming through the Shopify source. The path should point to the Shopify Details object inside the context object.

We send some events from Shopify to Warehouse destinations. Now, within the sent event’s context, there is a field called shopifyDetails which actually serves as a dump for the untransformed event received from Shopify by our source transformer. Now, if this shopifyDetails is sent as is, because of its large size and large number of fields within that object, it would result in large number of columns getting created within the warehouse.
so for this we want to leverage JSON paths.

integrations: {
                    SHOPIFY: true,
                    DATA_WAREHOUSE: {
                      options: {
                        jsonPaths: [`${event type}.context.shopifyDetails`],
                      },
                    },
                  },

For 4 providers as of now (rs, postgres, snowflake and bq), we wanted to support a new value within the integrations object. DATA_WAREHOUSE that will allow us to propagate shopifyDetails to be a jsonpath from the source event.

What is the related Linear task?

Resolves INT-3098 INT-3097

Please explain the objectives of your changes below

Put down any required details on the broader aspect of your changes. If there are any dependent changes, mandatorily mention them here

Any changes to existing capabilities/behaviour, mention the reason & what are the changes ?

N/A

Any new dependencies introduced with this change?

N/A

Any new generic utility introduced or modified. Please explain the changes.

N/A

Any technical or performance related pointers to consider with the change?

N/A

@coderabbitai review

---

Developer checklist

• My code follows the style guidelines of this project

• No breaking changes are being introduced.

• All related docs linked with the PR?

• All changes manually tested?

• Any documentation changes needed with this change?

• Is the PR limited to 10 file changes?

• Is the PR limited to one linear task?

• Are relevant unit and component test-cases added in new readability format?

Reviewer checklist

• Is the type of change in the PR title appropriate as per the changes?

• Verified that there are no credentials or confidential data exposed with the changes.

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/47/index.html

[codecov]

Codecov Report

Attention: Patch coverage is 82.60870% with 4 lines in your changes missing coverage. Please review.

Project coverage is 90.80%. Comparing base (ba8ad19) to head (6b96a44).
Report is 8 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/warehouse/util.js	75.00%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3980      +/-   ##
===========================================
+ Coverage    90.78%   90.80%   +0.02%     
===========================================
  Files          624      626       +2     
  Lines        32553    32676     +123     
  Branches      7727     7760      +33     
===========================================
+ Hits         29552    29672     +120     
+ Misses        2777     2748      -29     
- Partials       224      256      +32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/48/index.html

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/60/index.html

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/61/index.html

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/146/index.html

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/161/index.html

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/168/index.html

The base branch was changed.

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/172/index.html

[devops-github-rudderstack]

Allure Test reports for this run are available at:

• Allure Report: https://rudderlabs.github.io/rudder-transformer/178/index.html

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
83.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud