add example code and docs to demonstrate matching data: and tel: URLs

[GoogleCodeExporter]

There is no existing documentation on how to allow data: and tel: URLs.  Add an 
example that demonstrates how HtmlPolicyBuilder.allowUrlProcotols can be used 
with data: and tel:.

Original issue reported on code.google.com by [email protected] on 21 Jan 2014 at 4:04

[jmanico]

Bump to revisit. Mike, perhaps just give me one complex example and I'll use it to flesh out the docs.

[pickle-weasle]

Can't this be achieved with just
this.policy = this.policy.allowUrlProtocols("tel","data");
In the readme there is the following:

PolicyFactory policy = new HtmlPolicyBuilder()
.allowElements("a")
.allowUrlProtocols("https")
.allowAttributes("href").onElements("a")
.requireRelNofollowOnLinks()
.toFactory();
String safeHTML = policy.sanitize(untrustedHTML);

It's hardly much of a stretch to figure how to change that for other protocols.
Could this be closed?

[DavesMan]

Can't this be achieved with just
this.policy = this.policy.allowUrlProtocols("tel","data");
In the readme there is the following:

No. I have allowUrlProtocols("mailto", "javascript", "tel", "https") and this allows https and mailto. But tel & javascript gets stripped.

[mikesamuel]

I put together #126 to try to fill this gap. It integrates a URL classifier engine that lets you express predicates over URLs like

  /** We define a classifier with a declarative syntax */
  static final UrlClassifier CLASSIFIER = UrlClassifiers.builder()
      // We want to allow HTTP and HTTPS for this example
      .scheme(BuiltinScheme.HTTP, BuiltinScheme.HTTPS)
      .authority(
          AuthorityClassifiers.builder()
          // We whitelist some subdomains of hosts we trust.
          .host("**.example.com", "**.example.net")
          .build())
      // We allow access to .html files
      .pathGlob("**.html")
      .build();

I haven't integrated it because I don't know if anyone needs that level of flexibility.

It does have better support for data and tel URLs.

https://github.com/OWASP/url-classifier/blob/8715ef5f8d5623b8023b6c59b931621431188f21/src/main/java/org/owasp/url/BuiltinScheme.java#L230-L241

If that looks useful, I can move that in.

[DavesMan]

Seems it could save a lot of "custom" work. No I also do some substring and regex checks of URLs using apply method of ElementPolicy.

[mikesamuel]

This issue is about docs and examples.
If you have a problem with mis-encoding of tel: URLs could you open a new issue with an example of a tel: URL which is mis-encoded.

[jmanico]

There is no hostility in this thread that I see. We mean well.

[mikesamuel]

@jmiserez

I'm trying to fit my pro-bono work, including this project, in between a lot of other stuff. Sorry if I was abrupt, but I have no hostility towards you.

https://greatergood.berkeley.edu/article/item/six_tips_for_reading_emotions_in_text_messages

Keep in mind that texts are a difficult medium for communicating emotion. We have no facial expressions, or tone of voice, or conversation to give us more information.

If the text doesn’t say, “I’m angry,” then don’t assume that the texter is angry.

I have not read the tel: RFC recently. It would help out if you could provide an example of what you want. Yes, scope creep in issues happens, and it's happened in this thread. It would help me out again though if, something with a concrete success case could happen in a thread that's narrowly tailored, or in the PR comments.

If issue #22 is where you want to discuss it that's great. Recency doesn't matter to me.

What I am asking is whether you had given any thought to protocol-dependent escaping of characters in the above PR.

That PR includes TEL as a BuiltinScheme:

https://github.com/OWASP/url-classifier/blob/8715ef5f8d5623b8023b6c59b931621431188f21/src/main/java/org/owasp/url/BuiltinScheme.java#L225-L228

Scheme's control how raw parts are composed into a string.

https://github.com/OWASP/url-classifier/blob/8715ef5f8d5623b8023b6c59b931621431188f21/src/main/java/org/owasp/url/Scheme.java#L381-L392

I don't know if I got that correct, because I've not seen a failing testcase. https://github.com/OWASP/url-classifier/blob/master/src/test/java/org/owasp/url/UrlValueTest.java has tests for encoding of percents so that, e.g. javascript:alert(9 % 2) doesn't have % reencoded to %22 but it doesn't have many tel: examples.

[jmiserez]

I have misinterpreted your response and I apologize. I must admit my initial comment concerning the PR wasn't too clear either.

Thank you for your detailed response, the part concerning javascript escaping is especially relevant and helpful in this context.

I'll try out the test cases and the classifier library in more detail and comment with a test case once I have something more tangible.