Server to Server Replication

[santanusinha]

Problem
Right now, if two ranger servers point to each other as upstream, they each receive all nodes from each other leading to a circular infinite loop.

Solution
We identify HTTP sources from which we are replicating and provide a flag in the /nodes and /services APIs to skip sending out data from hubs identified as replication sources.

Effect

• A client calling the apis without the skipReplicatedData (basically all normal ranger clients) will get data from everywhere as it does currently.
• If HttpClientConfig for a HTTP based client has the replicationSource set to true then the query skipReplicatedData parameter is set to true so that the upstream sends it only the data it has from it's non-replication upstreams (zk, drove, other lower level http servers etc).

Addresses #52

[koushikr]

This may not work if the other side is a semi composite. Imagine if there are two sites S1 and S2 with the hubs H1 and H2 respectively. The hub H1 uses H2, along with all the possible local backends in S1 and H2 uses H1, along with all the possible local backends in H2. In this case, we can't ignore the source H2 on H1 and vice-versa for H1 on H2 - since both of them are composite backends.

Should we instead move it to set? so the duplicates are arrested and composite backends can be supported?