Skip to content
Jack Wampler edited this page May 11, 2020 · 9 revisions

Dark Decoy Project

Welcome to dark decoys project wiki. Before beginning know that there exist issues documenting problems and speedbumps with previous station standups. If you encounter a new problem please document it in an issue, and link it here.

  • #43

Install

Install dependencies

sudo apt install libzmq3-dev redis-server libgmp3-dev pkg-config

Install PF_RING

  1. See the PF_RING wiki page

  2. Install PF_RING kernel module

  3. Install PF_RING Libpfring and Libpcap

Build the station

make

#future
# sudo make install

Setup

The setup that needs to be done on a station before it can begin operating currently involves setting up the iptables and ip route plumbing to route packets through the OS from detector to app and back. You can look at the Environment Setup page for full instructions, here we will run an abbreviated scripted version.

  1. Decide / determine the answer to the following questions

    • Do we need to run with Tapdance?
    • How many cores you will be running on? (How many cores is Tapdance running on?)
    • If tapdance is running, which cores is it running on?
    • What are the IP addresses (v4 and v6) of the interface that will be used for sending proxy traffic?
  2. Modify the on-reboot.sh script on a per machine basis based on the answers above:

IP4_ADDR="<YOUR_LOCAL_IPV4_ADDR>"
IP6_ADDR="[<YOUR_LOCAL_IPV6_ADDR>]"
CORE_COUNT=<NUMBER_OF_CORES_TO_RUN_ON>
OFFSET=<OFFSET_FOR_PFRING_QUEUE_ID>

If you are not running Tapdance on the same station you should set OFFSET_FOR_PFRING_QUEUE_ID to 0.

The CORE_COUNT and OFFSET here are used to set up the tun interfaces. During this set-up we align the tun numbers with the PF_RING queue IDs. This is not required, but it allows us to ensure that queues used by different stations (Tapdance or Conjure-Dev) cant collide on either tun or PF_RING queue IDs.

Running the Station

1/3 Start PF_RING ZC Load balancer

If tapdance is not running then we need to start the pf_ring zbalance_ipc ourselves.

cd PF_RING/userland/examples_zc
sudo ./zbalance_ipc -i zc:eth1,zc:eth0 -n 2 -c 10 -g 1 -m 1

If tapdance is running we need to add a second output queue to the /opt/tapdance/start_zbalance_ipc.sh script then restart zbalance.service. We add another queue of ${TD_CORECOUNT} to the command assuming we want to run the same number of cores for Tapdance that we do for Conjure.

$ git diff start_zbalance_ipc.sh 
diff --git a/start_zbalance_ipc.sh b/start_zbalance_ipc.sh
index 74253e1..d565757 100755
--- a/start_zbalance_ipc.sh
+++ b/start_zbalance_ipc.sh
@@ -28,4 +28,4 @@ do
        didfirst=1
     fi
 done
-sudo ./PF_RING/userland/examples_zc/zbalance_ipc -i $ifcarg -c ${TD_CLUSTER_ID} -n ${TD_CORECOUNT} -m ${ZBALANCE_HASH_MODE} -g ${ZBALANCE_CORE}
+sudo ./PF_RING/userland/examples_zc/zbalance_ipc -i $ifcarg -c ${TD_CLUSTER_ID} -n ${TD_CORECOUNT},${TD_CORECOUNT} -m ${ZBALANCE_HASH_MODE} -g ${ZBALANCE_CORE}

2/3 Start processing packets

# simple
sudo ./dark-decoy -c 98 -n 2 -K conf/station_key -l 5

# Avoiding Tapdance
sudo ./dark-decoy -c 98 -n 6 -K ./conf/statin_key -l 5 -o 9 -z 6 > log/detector.log 2>&1

# Options

#     REQUIRED
#     -c  <cluster_id> - The cluster id specified to PF_RING when starting `zbalance_ipc`

#     OPTIONAL
#     -n  <cpu_procs> - Number of cores to be used (default -1 = all cores)
#     -s <skip_core> - specify core_id to be skipped when allocating threads.
#     -K <keyfile_name> - Specify custom private key to be used by station
#     -a <zmq_address> - Custom Address of ZMQ server

#     DEBUG
#     -l <log_interval> - In seconds, interval between logging of bandwidth, tag checks/s, etc.
#     -o <core_affinity_offset> - Start processes on $core_affinity_offset+$cpu_procs.
#             This allows us to run debug/production pf_rings on different cores
#             entirely (which rust likes), and with different cluster_ids.
#     -z <queue_offset> PF_RING queue ID offset to avoid using the same queues and tunX interfaces
#             as another Tapdance or Conjure instance on the server.  

3/3 Start application

cd application
./application