Skip to content

Commit

Permalink
Merge branch 'develop' into fix_improper_type_validations_snyc
Browse files Browse the repository at this point in the history
  • Loading branch information
sandeepdsvs authored Dec 18, 2023
2 parents 4fd3b34 + eea6511 commit 25c64c2
Showing 21 changed files with 576 additions and 22 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/create-hotfix-branch.yml
Original file line number Diff line number Diff line change
@@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest

# Only allow these users to create new hotfix branch from 'main'
if: github.ref == 'refs/heads/main' && (github.actor == 'ItsSudip' || github.actor == 'krishna2020' || github.actor == 'saikumarrs' || github.actor == 'sandeepdsvs' || github.actor == 'shrouti1507' || github.actor == 'anantjain45823' || github.actor == 'chandumlg' || github.actor == 'mihir-4116' || github.actor == 'ujjwal-ab') && (github.triggering_actor == 'ItsSudip' || github.triggering_actor == 'krishna2020' || github.triggering_actor == 'saikumarrs' || github.triggering_actor == 'sandeepdsvs' || github.triggering_actor == 'shrouti1507' || github.triggering_actor == 'anantjain45823' || github.triggering_actor == 'chandumlg' || github.triggering_actor == 'mihir-4116' || github.triggering_actor == 'ujjwal-ab')
if: github.ref == 'refs/heads/main' && (github.actor == 'ItsSudip' || github.actor == 'krishna2020' || github.actor == 'koladilip' || github.actor == 'saikumarrs' || github.actor == 'sandeepdsvs' || github.actor == 'shrouti1507' || github.actor == 'anantjain45823' || github.actor == 'chandumlg' || github.actor == 'mihir-4116' || github.actor == 'ujjwal-ab') && (github.triggering_actor == 'ItsSudip' || github.triggering_actor == 'krishna2020' || github.triggering_actor == 'saikumarrs' || github.triggering_actor == 'sandeepdsvs' || github.triggering_actor == 'koladilip' || github.triggering_actor == 'shrouti1507' || github.triggering_actor == 'anantjain45823' || github.triggering_actor == 'chandumlg' || github.triggering_actor == 'mihir-4116' || github.triggering_actor == 'ujjwal-ab')
steps:
- name: Create Branch
uses: peterjgrainger/action-create-branch@v2.4.0
2 changes: 1 addition & 1 deletion .github/workflows/housekeeping.yml
Original file line number Diff line number Diff line change
@@ -36,7 +36,7 @@ jobs:
uses: actions/checkout@v4.1.1

- name: Delete Old Branches
uses: beatlabs/delete-old-branches-action@v0.0.9
uses: beatlabs/delete-old-branches-action@v0.0.10
with:
repo_token: ${{ secrets.PAT }}
date: '3 months ago'
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -2,6 +2,32 @@

All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.

### [1.52.3](https://github.com/rudderlabs/rudder-transformer/compare/v1.52.2...v1.52.3) (2023-12-18)


### Bug Fixes

* email validation for braze ([#2929](https://github.com/rudderlabs/rudder-transformer/issues/2929)) ([28207d0](https://github.com/rudderlabs/rudder-transformer/commit/28207d02a1b39b25325fa30be12c4dccd05c844e))
* pinterest event value is restricted to string ([#2933](https://github.com/rudderlabs/rudder-transformer/issues/2933)) ([7f6d519](https://github.com/rudderlabs/rudder-transformer/commit/7f6d519b811a5d8f83f7a2103d9ba50efed8a923))
* remove log from dcm_floodlight ([#2934](https://github.com/rudderlabs/rudder-transformer/issues/2934)) ([c5d9a3c](https://github.com/rudderlabs/rudder-transformer/commit/c5d9a3cc7d0270238c102cec809edcccad5b270d))
* tiktok remove lowercasing for custom events ([#2930](https://github.com/rudderlabs/rudder-transformer/issues/2930)) ([1a90719](https://github.com/rudderlabs/rudder-transformer/commit/1a9071931e9768a3fd02b749b4b705e8c28d9763))

### [1.52.2](https://github.com/rudderlabs/rudder-transformer/compare/v1.52.1...v1.52.2) (2023-12-15)


### Bug Fixes

* braze dedup for adding non-billable attributes ([#2921](https://github.com/rudderlabs/rudder-transformer/issues/2921)) ([4196b1b](https://github.com/rudderlabs/rudder-transformer/commit/4196b1bfec2fc3f7c2c5d0b305853b39033e894c))
* inclusion of right status-code & error details in proxy response ([#2916](https://github.com/rudderlabs/rudder-transformer/issues/2916)) ([5d295c2](https://github.com/rudderlabs/rudder-transformer/commit/5d295c232ff91bfb8564d4d6fcce49dc1e07958b))
* supportTransformerProxyV1 set to false ([#2920](https://github.com/rudderlabs/rudder-transformer/issues/2920)) ([90ee7ad](https://github.com/rudderlabs/rudder-transformer/commit/90ee7ad13921830c920ee60361adb0e0c5dc1d43))

### [1.52.1](https://github.com/rudderlabs/rudder-transformer/compare/v1.52.0...v1.52.1) (2023-12-14)


### Bug Fixes

* update response handling for v1 proxy ([7d275ca](https://github.com/rudderlabs/rudder-transformer/commit/7d275ca1d58c38859fce95d7d0677e11342c662a))

## [1.52.0](https://github.com/rudderlabs/rudder-transformer/compare/v1.51.1...v1.52.0) (2023-12-12)


4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "rudder-transformer",
"version": "1.52.0",
"version": "1.52.3",
"description": "",
"homepage": "https://github.com/rudderlabs/rudder-transformer#readme",
"bugs": {
1 change: 0 additions & 1 deletion src/cdk/v2/destinations/dcm_floodlight/procWorkflow.yaml
Original file line number Diff line number Diff line change
@@ -84,7 +84,6 @@ steps:
$.transformCustomVariable(conversionEvent.customVariables || [], .message)
);
$.context.salesTag = conversionEvent.salesTag;
console.log("$.context.salesTag", $.context.salesTag);
- name: handleSalesTag
condition: $.context.salesTag
template: |
5 changes: 4 additions & 1 deletion src/cdk/v2/destinations/pinterest_tag/procWorkflow.yaml
Original file line number Diff line number Diff line change
@@ -9,6 +9,8 @@ bindings:
path: ../../../../v0/util
- name: removeUndefinedAndNullAndEmptyValues
path: ../../../../v0/util
- name: validateEventName
path: ../../../../v0/util
steps:
- name: checkIfProcessed
condition: .message.statusCode
@@ -175,6 +177,7 @@ steps:
name: eventNamesForOthers
template: |
let event = .message.event ?? .message.name;
$.validateEventName(event);
let eventInLowerCase = event.toLowerCase();
let eventNames = .destination.Config.eventsMapping.(){.from === event}.to[] ?? [];
eventNames = $.convertToSnakeCase(eventNames);
@@ -243,4 +246,4 @@ steps:
},
"params": $.outputs.checkSendTestEventConfig,
"files": {}
})[]
})[]
2 changes: 1 addition & 1 deletion src/controllers/delivery.ts
Original file line number Diff line number Diff line change
@@ -84,7 +84,7 @@ export class DeliveryController {
);
}
ctx.body = { output: deliveryResponse };
ControllerUtility.deliveryPostProcess(ctx);
ControllerUtility.deliveryPostProcess(ctx, deliveryResponse.status);

logger.debug('Native(Delivery):: Response from transformer::', JSON.stringify(ctx.body));
return ctx;
2 changes: 1 addition & 1 deletion src/features.json
Original file line number Diff line number Diff line change
@@ -66,5 +66,5 @@
"REDDIT": true
},
"supportSourceTransformV1": true,
"supportTransformerProxyV1": true
"supportTransformerProxyV1": false
}
3 changes: 3 additions & 0 deletions src/services/destination/nativeIntegration.ts
Original file line number Diff line number Diff line change
@@ -218,6 +218,9 @@ export class NativeIntegrationDestinationService implements DestinationService {
);
responseProxy = {
response: jobStates,
status: v0Response.status,
message: v0Response.message,
authErrorCategory: v0Response.authErrorCategory,
} as DeliveriesResponse;
}
return responseProxy;
3 changes: 3 additions & 0 deletions src/services/destination/postTransformation.ts
Original file line number Diff line number Diff line change
@@ -186,6 +186,9 @@ export class DestinationPostTransformationService {
const resp = {
response: responses,
statTags: errObj.statTags,
authErrorCategory: errObj.authErrorCategory,
message: errObj.message.toString(),
status: errObj.status,
} as DeliveriesResponse;

ErrorReportingService.reportError(error, metaTo.errorContext, resp);
53 changes: 53 additions & 0 deletions src/v0/destinations/braze/braze.util.test.js
Original file line number Diff line number Diff line change
@@ -660,9 +660,58 @@ describe('dedup utility tests', () => {
color: 'green',
age: 30,
gender: 'male',
country: 'US',
language: 'en',
email_subscribe: true,
push_subscribe: false,
subscription_groups: ['group1', 'group2'],
};
const storeData = {
external_id: '123',
country: 'US',
language: 'en',
email_subscribe: true,
push_subscribe: false,
subscription_groups: ['group1', 'group2'],
custom_attributes: {
color: 'blue',
age: 25,
},
};
store.set('123', storeData);
const result = BrazeDedupUtility.deduplicate(userData, store);
expect(store.size).toBe(1);
expect(result).toEqual({
external_id: '123',
color: 'green',
age: 30,
gender: 'male',
country: 'US',
language: 'en',
email_subscribe: true,
push_subscribe: false,
subscription_groups: ['group1', 'group2'],
});
});

test('deduplicates user data correctly 2', () => {
const userData = {
external_id: '123',
color: 'green',
age: 30,
gender: 'male',
language: 'en',
email_subscribe: true,
push_subscribe: false,
subscription_groups: ['group1', 'group2'],
};
const storeData = {
external_id: '123',
country: 'US',
language: 'en',
email_subscribe: true,
push_subscribe: false,
subscription_groups: ['group1', 'group2'],
custom_attributes: {
color: 'blue',
age: 25,
@@ -676,6 +725,10 @@ describe('dedup utility tests', () => {
color: 'green',
age: 30,
gender: 'male',
language: 'en',
email_subscribe: true,
push_subscribe: false,
subscription_groups: ['group1', 'group2'],
});
});

23 changes: 13 additions & 10 deletions src/v0/destinations/braze/transform.js
Original file line number Diff line number Diff line change
@@ -154,16 +154,19 @@ function getUserAttributesObject(message, mappingJson, destination) {
Object.keys(mappingJson).forEach((destKey) => {
let value = get(traits, mappingJson[destKey]);
if (value || (value === null && reservedKeys.includes(destKey))) {
// if email is not string remove it from attributes
if (destKey === 'email' && typeof value !== 'string') {
throw new InstrumentationError('Invalid email, email must be a valid string');
}

// handle gender special case
if (destKey === 'gender') {
value = formatGender(value);
} else if (destKey === 'email' && isDefinedAndNotNull(value)) {
value = value.toString().toLowerCase();
switch (destKey) {
case 'gender':
value = formatGender(value);
break;
case 'email':
if (typeof value === 'string') {
value = value.toLowerCase();
} else if (isDefinedAndNotNull(value)) {
throw new InstrumentationError('Invalid email, email must be a valid string');
}
break;
default:
break;
}
data[destKey] = value;
}
8 changes: 8 additions & 0 deletions src/v0/destinations/braze/util.js
Original file line number Diff line number Diff line change
@@ -301,6 +301,14 @@ const BrazeDedupUtility = {
};
const identifier = external_id || user_alias?.alias_name;
store.set(identifier, { ...storedUserData, ...deduplicatedUserData });

// add non billable attributes back to the deduplicated user object
BRAZE_NON_BILLABLE_ATTRIBUTES.forEach((key) => {
if (isDefinedAndNotNull(userData[key])) {
deduplicatedUserData[key] = userData[key];
}
});

return removeUndefinedValues(deduplicatedUserData);
},
};
2 changes: 2 additions & 0 deletions src/v0/destinations/pinterest_tag/utils.js
Original file line number Diff line number Diff line change
@@ -8,6 +8,7 @@ const {
isDefined,
getHashFromArrayWithDuplicate,
removeUndefinedAndNullValues,
validateEventName,
} = require('../../util');
const { COMMON_CONFIGS, CUSTOM_CONFIGS, API_VERSION } = require('./config');

@@ -170,6 +171,7 @@ const deduceTrackScreenEventName = (message, Config) => {
if (!trackEventOrScreenName) {
throw new InstrumentationError('event_name could not be mapped. Aborting');
}
validateEventName(trackEventOrScreenName);

/*
Step 1: If the event is not amongst the above list of ecommerce events, will look for
3 changes: 2 additions & 1 deletion src/v0/destinations/tiktok_ads/transform.js
Original file line number Diff line number Diff line change
@@ -155,7 +155,8 @@ const trackResponseBuilder = async (message, { Config }) => {
return responseList;
}
// Doc https://ads.tiktok.com/help/article/standard-events-parameters?lang=en
event = eventNameMapping[event] || event;
// For custom event we do not want to lower case the event or trim it we just want to send those as it is
event = eventNameMapping[event] || message.event;
// if there exists no event mapping we will build payload with custom event recieved
responseList.push(getTrackResponse(message, Config, event));

77 changes: 77 additions & 0 deletions test/__tests__/data/braze_input.json
Original file line number Diff line number Diff line change
@@ -1955,5 +1955,82 @@
"type": "identify",
"userId": ""
}
},
{
"destination": {
"Config": {
"restApiKey": "dummyApiKey",
"prefixProperties": true,
"useNativeSDK": false,
"sendPurchaseEventWithExtraProperties": true
},
"DestinationDefinition": {
"DisplayName": "Braze",
"ID": "1WhbSZ6uA3H5ChVifHpfL2H6sie",
"Name": "BRAZE"
},
"Enabled": true,
"ID": "1WhcOCGgj9asZu850HvugU2C3Aq",
"Name": "Braze",
"Transformations": []
},
"message": {
"anonymousId": "e6ab2c5e-2cda-44a9-a962-e2f67df78bca",
"channel": "web",
"context": {
"traits": {
"city": "Disney",
"country": "USA",
"email": null,
"firstname": "Mickey"
},
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
},
"event": "Order Completed",
"integrations": {
"All": true
},
"messageId": "aa5f5e44-8756-40ad-ad1e-b0d3b9fa710a",
"originalTimestamp": "2020-01-24T06:29:02.367Z",
"properties": {
"affiliation": "Google Store",
"checkout_id": "fksdjfsdjfisjf9sdfjsd9f",
"coupon": "hasbros",
"currency": "USD",
"discount": 2.5,
"order_id": "50314b8e9bcf000000000000",
"products": [
{
"category": "Games",
"image_url": "https:///www.example.com/product/path.jpg",
"name": "Monopoly: 3rd Edition",
"price": 0,
"product_id": "507f1f77bcf86cd799439023",
"quantity": 1,
"sku": "45790-32",
"url": "https://www.example.com/product/path"
},
{
"category": "Games",
"name": "Uno Card Game",
"price": 0,
"product_id": "505bd76785ebb509fc183724",
"quantity": 2,
"sku": "46493-32"
}
],
"revenue": 25,
"shipping": 3,
"subtotal": 22.5,
"tax": 2,
"total": 27.5
},
"receivedAt": "2020-01-24T11:59:02.403+05:30",
"request_ip": "[::1]:53712",
"sentAt": "2020-01-24T06:29:02.368Z",
"timestamp": "2020-01-24T11:59:02.402+05:30",
"type": "track",
"userId": ""
}
}
]
Loading

0 comments on commit 25c64c2

Please sign in to comment.