add origin_referrer_url, origin_url and zone_identifier to the file attribute

[AsuNa-jp]

Changes

This PR adds the following attributes.

• file.origin_referrer_url
• file.origin_url
• file.zone_identifier

(Thanks @trisch-me for all the advice you gave me in creating this PR!)

Background: What are these fields for? (Updated 2024/Oct/21)

When downloading files from the internet (or network) using a web browser (such as Chrome or Edge) or a certain application, information about where the file came from is generally added to the file. This is a general behavior that can occur on all operating systems, and its primary use is to enhance security by providing context about the file’s source, allowing the system to assess potential risks and enforce appropriate security measures.

The details are explained below.

Windows

In Windows, it is known as the Mark of the Web(ref1, ref2), and is added to the file's NTFS alternate data stream.

For example, when you download an image file (image17.webp) from this webpage using a web browser, the download source URL is automatically added to the file's Alternate Data Stream (ADS) as following.

• Inside image17.webp:Zone.Identifier:$DATA

This PR adds a field to store the URL of the file's origin, which is saved in the NTFS alternate data stream (ADS).

• ZoneId is inteded to be stored in the zone_identifier field.
• ReferrerUrl is intended to be stored in the origin_referrer_url field
• HostUrl is inteded to be stored in the origin_url field.

Note - In the case of Windows, MotW can be used not only with NTFS but also with ReFS (8.1/2012 R2 or later)

Linux

In Linux, some applications may store the file origin metadata in extended attributes (xattr) or Gnome virtual filesystem(gvfs) to track the source of a file.

For example, when you download an image file (image17.webp) from this webpage using a web browser, the download source URL is automatically added to gvfs.

example of a file downloaded by using firefox

Additionally, by using Curl or Wget, the referer URL(user.xdg.referrer.url) and origin URL(user.xdg.origin.url) can be attached to the file's extended attributes. (Google Chrome used to add user.xdg.referrer.url and user.xdg.origin.url as well but it currently turned off this feature.)

• https://wiki.archlinux.org/title/Extended_attributes
  
  

example of a file downloaded by using curl

• user.xdg.referrer.url is intended to be stored in the origin_referrer_url field
• user.xdg.origin.url is inteded to be stored in the origin_url field.

Note - As written in this web page, all major Linux file systems including Ext4, Btrfs, ZFS, and XFS support extended attributes.

MacOS

(Since I don't have a Mac device, my investigation will be based on the internet.)

In MacOS, some applications may store the file origin metadata in extended attributes to track the source of a file as follows. It seems that both the referrer and origin URL are being saved.

The image source is as follows:
https://stackoverflow.com/questions/70444996/obtaining-metadata-where-from-of-a-file-on-mac

The same thing is mentioned on another website as well. (https://exiftool.org/forum/index.php?topic=14991.0)

Usually if we save a file from browser, the file will have 2 strings in the 'Where from' attribute:

Background: the use cases. (Updated 2024/Oct/21)

• (A). For example, in Elastic Security (Elastic Defend), a file open event may be generated when a file is opened. By including the file's origin information, such as the Origin URL and Referrer URL, the system can assess whether the file might be malware downloaded from a malicious website based on those URLs.

• (B). Another example would be adding file origin information (such as the Origin URL and Referrer URL) to the file creation event when a file is downloaded from the internet. This would make it possible to detect if the file was downloaded from a website on a blocklist and take actions such as deleting the file.

Merge requirement checklist

• CONTRIBUTING.md guidelines followed.
• Change log entry added, according to the guidelines in When to add a changelog entry.
  • If your PR does not need a change log, start the PR title with [chore]
• schema-next.yaml updated with changes to existing conventions.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: AsuNa-jp / name: Asuka Nakajima (4a2fed9, 088e959, 1f6663a, daa081e, ec834e6, f4e241c, 7d5331c, 264fd13, 968e8c2, 0793ef9, d82f261, 17107e4, a1a4867, 130bf61, f4ad750, 37c9710, c55cb2b, a2b4b35, 481bd1c, 160b7ee, b0016f6, 4b11936, 22d06ab, a3dfd75, b590595, c6b8d78, e8256e6, 358f21a, ed796cc, ba508e4)
• ✅ login: trisch-me / name: Alexandra Konrad (329df10)

[AsuNa-jp]

Hi @trisch-me
Thank you for the prompt feedback on this PR. All of your points are absolutely valid. I have updated the PR based on your suggestions. (160b7ee, 37c9710)
If there is anything else, please feel free to let me know!

[trisch-me]

LGTM

[lmolkova]

there could be alternative solutions such as:

• reusing existing url attributes on the same telemetry item that describes a file
• defining specific security events which list all applicable metadata as event fields (not attributes)

The current solution seems to be very specific to a certain use-case (I download a file and capture it's metadata which will not be accessible later). These attributes will not be available or applicable to any other case.
It won't make sense to reuse those attributes in other conventions.

So please consider alternative solutions and please provide a use-case for these attributes.

[AsuNa-jp]

Thank you all for your comments. Based on feedback from various sources, I have added file.zone_identifier to this PR.

However, since there also have been concerns raised about whether the fields we plan to add are even necessary, we are considering having @trisch-me (and @magermark ) lead a more in-depth discussion during the upcoming Otel Semantic Convention meeting.

[AsuNa-jp]

Hi @trisch-me @lmolkova @joaopgrassi
Based on last week’s discussion at the Symantec convention meeting, I have added additional explanations to this PR. If you need further explanations before approving this PR, please don't hesitate to let me know.

[trisch-me]

@AsuNa-jp could you please fix conflicts? thanks

[jsuereth]

My only remaining concern with this PR (and based on @lmolkova's comments) is whether the things you're defining should be event fields vs. attributes.

The two use cases you mention both involve an event. Are these attributes you're defining things we'd want to include in Spans and Metrics?

I think it'd be reasonable to define a file.open Event that has these fields within it, but I'm not positive how you'd possible have that "turn into a metric" or otherwise interact with spans.

[AsuNa-jp]

Hi @jsuereth
Thank you for taking the time to provide feedback on this PR!
I understand your concern that, based on the use cases I provided, origin_referrer_url, origin_url, and zone_identifier may not seem necessary to include in the Attribute.

However, I personally believe that this information should be included in the Attributes, as they may need to be referenced later. For example, these fields could be invaluable in determining whether a previously downloaded file was actually sourced from a newly identified malicious website.

This is just an example, but let’s say it was discovered today that https://outlook.office.com/ is a malicious website. These fields would be helpful to find out, 'How many files were actually downloaded from this site?

I hope this helps address your concerns. However, if you have any further concerns, please don’t hesitate to let me know.

[AsuNa-jp]

Hi @jsuereth!
I heard from @trisch-me that you were busy with KubeCon last week. I’m sorry to ask this so soon after your return, but could you kindly review the comments above and approve this PR if there are no issues?

[jsuereth]

had a discussion with @trask and @lmolkova about attributes and Events.

1. I'm comfortable now having these as attributes. It seems we're moving more in this direction with our guidance.
2. I'm still going to push back on a lack of "Event" definition in opentelemetry for these attributes.

Regarding (2) adding the attribute to the registry does NOT allow opentelemetry to use this attribute yet. There is still no defined signal (event, span or metric) which is defined to generate this data. You have a clear use case outlined in the PR. I suspect these attributes should be part of a file.open or file.access event.

I would like to see this event defined, but I won't block this PR if others feel it's ok as-is and approve these attributes without that detail.

[jsuereth]

Should this be file.windows_zone.id?

Given the generic nature of the word of 'zone' I think this should be clarified. Additionally we should stick to a similar convention across otel for id, and we don't use identifier anywhere else that I'm aware of.