Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data flow: toBase64 adds break lines after 76th character in specific ADF location #661

Open
danyltsiv opened this issue Apr 25, 2024 · 1 comment

Comments

@danyltsiv
Copy link

danyltsiv commented Apr 25, 2024

Hi!
Recently we've got unexpected outputs from ADF pipelines with no prior changes to the source code.
Suddenly, the toBase64 function in data flow activities began to add break lines (\r\n) after the 76th character for output values.
During the issue investigation, we had an assumption that it could be some Virtual Machine or SDK's specific thing, based on this we tried to play with the locations of ADF, and it worked out:
If the location of ADF is West Europe then all good and break lines aren't added:
image

if we change it to UK West then suddenly break lines start to appear:
image

I am attaching the ARM template of ADF, containing the pipeline that uses the dataflow that converts input into base64 and the blob files used as input and output.
The payload is simple in JSON format and contains a single example property "ContactEmail".
You will also need a Storage Account with container "testbase64" and blobs: "files/test1" that would be as input and "output/test1output" that would be as output.

Steps to reproduce:

  1. Create the Storage Account
    1.1) Create container "testbase64"
    1.2) Create blobs "files/test1", "output/test1output" in that container
  2. Create 2 ADFs:
    2.1) Location = West Europe
    2.2) Location = UK West
  3. Import ARM template into created ADFs
  4. Put a long enough value into "test1" file so that it exceeds 76 chars in output, like:
    {"ContactEmail":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
  5. Run pipelines and compare results

As a solution, we are considering redeploying ADF into a different location.
If you have some other options on your mind, we'd appreciate it.

Thanks in advance!

Files:
ARM Template and Blobs.zip

@JamesGreenough
Copy link

Uk south affected too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants