diff --git a/Readme.md b/Readme.md index 6fd3b8cc..b6d84a74 100644 --- a/Readme.md +++ b/Readme.md @@ -7,7 +7,6 @@ Brought to you by [Active Countermeasures](https://www.activecountermeasures.com [![Build Status](https://travis-ci.org/activecm/rita.svg?branch=master)](https://travis-ci.org/activecm/rita) --- -### What is Here RITA is an open source framework for network traffic analysis. @@ -16,31 +15,42 @@ The framework ingests [Bro/Zeek Logs](https://www.zeek.org/) in TSV format, and - **DNS Tunneling Detection** Search for signs of DNS based covert channels - **Blacklist Checking**: Query blacklists to search for suspicious domains and hosts -### Automatic Installation -**The automatic installer is officially supported on Ubuntu 16.04 LTS, Security Onion\*, and CentOS 7** +## Install -* Download the latest `install.sh` file from the [release page](https://github.com/activecm/rita/releases/latest) -* Make the installer executable: `chmod +x ./install.sh` -* Run the installer: `sudo ./install.sh` +Please see our recommended [System Requirements](docs/System%20Requirements.md) document if you wish to use RITA in a production environment. + +### Automated Install + +RITA provides an install script that works on Ubuntu 18.04 LTS, Ubuntu 16.04 LTS, Security Onion\*, and CentOS 7. + +Download the latest `install.sh` file [here](https://github.com/activecm/rita/releases/latest) and make it executable: `chmod +x ./install.sh` + +Then choose one of the following install methods: + +* `sudo ./install.sh` will install RITA as well as supported versions of Bro/Zeek and MongoDB. This is suitable if you want to get started as quickly as possible or you don't already have Bro/Zeek or MongoDB. + +* `sudo ./install.sh --disable-bro --disable-mongo` will install RITA only, without Bro/Zeek or MongoDB. You may also use these flags individually. + * If you choose not to install Bro/Zeek you will need to [provide your own logs](#obtaining-data-generating-brozeek-logs). + * If you choose not to install MongoDB you will need to configure RITA to [use your existing MongoDB server](docs/Mongo%20Configuration.md). \* Please see the [Security Onion RITA wiki page](https://securityonion.net/docs/RITA) for further information pertaining to using RITA on Security Onion. +### Docker Install + +See [here](docs/Docker%20Usage.md). + ### Manual Installation -To install each component of RITA by hand, [check out the instructions in the docs](docs/Manual%20Installation.md). + +To install each component of RITA by manually see [here](docs/Manual%20Installation.md). ### Upgrading RITA + See [this guide](docs/Upgrading.md) for upgrade instructions. ### Getting Started -#### System Requirements -* Operating System - The preferred platform is 64-bit Ubuntu 16.04 LTS. The system should be patched and up to date using apt-get. -* Processor (when installed alongside Bro/Zeek) - Two cores plus an additional core for every 100 Mb of traffic being captured. (three cores minimum). This should be dedicated hardware, as resource congestion with other VMs can cause packets to be dropped or missed. -* Memory - 16GB minimum. 64GB if monitoring 100Mb or more of network traffic. 128GB if monitoring 1Gb or more of network traffic. -* Storage - 300GB minimum. 1TB or more is recommended to reduce log maintenance. -* Network - In order to capture traffic with Bro/Zeek, you will need at least 2 network interface cards (NICs). One will be for management of the system and the other will be the dedicated capture port. Intel NICs perform well and are recommended. - #### Configuration File + RITA's config file is located at `/etc/rita/config.yaml` though you can specify a custom path on individual commands with the `-c` command line flag. :exclamation: **IMPORTANT** :exclamation: @@ -51,7 +61,8 @@ You may also wish to change the defaults for the following option: Note that any value listed in the `Filtering` section should be in CIDR format. So a single IP of `192.168.1.1` would be written as `192.168.1.1/32`. -#### Obtaining Data (Generating Bro/Zeek Logs): +#### Obtaining Data (Generating Bro/Zeek Logs) + * **Option 1**: Generate PCAPs outside of Bro/Zeek * Generate PCAP files with a packet sniffer ([tcpdump](http://www.tcpdump.org/), [wireshark](https://www.wireshark.org/), etc.) * (Optional) Merge multiple PCAP files into one PCAP file @@ -65,6 +76,7 @@ Note that any value listed in the `Filtering` section should be in CIDR format. * Provide the `--disable-bro` flag when running the installer if you intend to compile Bro/Zeek from source #### Importing and Analyzing Data With RITA + After installing RITA, setting up the `InternalSubnets` section of the config file, and collecting some Bro/Zeek logs, you are ready to begin hunting. Filtering and whitelisting happens at import time. These optional settings can be found alongside `InternalSubnets` in the configuration file. @@ -102,6 +114,7 @@ rita import --rolling --numchunks 48 /opt/bro/logs/current 48-hour-dataset ``` #### Examining Data With RITA + * Use the **show-X** commands * `show-databases`: Print the datasets currently stored * `show-beacons`: Print hosts which show signs of C2 software @@ -120,11 +133,14 @@ rita import --rolling --numchunks 48 /opt/bro/logs/current 48-hour-dataset ### Getting help + Please create an issue on GitHub if you have any questions or concerns. ### Contributing to RITA + To contribute to RITA visit our [Contributing Guide](Contributing.md) ### License + GNU GPL V3 © Active Countermeasures ™ diff --git a/docker-compose.yml b/docker-compose.yml index 53b12780..ed2c0f5b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,29 +1,20 @@ version: '3' services: - # bro: - # image: blacktop/bro:2.5 - # network_mode: host - # volumes: - # - logs:/pcap - # cap_add: - # - NET_RAW - db: image: mongo:3.6 volumes: - db:/data/db/ - + rita: - image: quay.io/activecm/rita:${RITA_VERSION:-master} + image: quay.io/activecm/rita:${VERSION:-latest} build: . links: # give db an alias of "localhost" so that RITA's default config works unchanged - db:localhost volumes: - - ${BRO_LOGS:?You must provide a path to your BRO_LOGS}:/opt/bro/logs:ro - # - logs:/opt/bro/logs + - ${CONFIG:?You must provide a path to your CONFIG}:/etc/rita/config.yaml:ro + - ${LOGS:?You must provide a path to your LOGS}:/logs:ro volumes: db: - # logs: \ No newline at end of file diff --git a/docs/Docker Usage.md b/docs/Docker Usage.md index 02efb70f..28f7bfa9 100644 --- a/docs/Docker Usage.md +++ b/docs/Docker Usage.md @@ -1,69 +1,51 @@ # Docker Usage You can run RITA using Docker! You have several options depending on your specific needs. -* Running RITA with Docker Compose - This is the simplest option and requires the least setup. You will have to provide your own Bro logs. -* Running RITA with Docker Using External Mongo - This option is useful if you do not want to use Docker Compose or you have an external Mongo server you wish to use. -* Using Docker to Build RITA - You can use Docker to build a standalone RITA binary that runs on any Linux 64-bit CPU. This is useful if you want a portable binary but don't want to use Docker to actually run RITA. +* [Running RITA with Docker Compose](#running-rita-with-docker-compose) - This is the simplest option and requires the least setup. You will have to provide your own Bro logs. +* [Running RITA with Docker Using External Mongo](#running-rita-with-docker-using-external-mongo) - This option is useful if you do not want to use Docker Compose or you have an external Mongo server you wish to use. +* [Using Docker to Build RITA](#using-docker-to-build-rita) - You can use Docker to build a standalone RITA binary that runs on any Linux 64-bit CPU. This is useful if you want a portable binary but don't want to use Docker to actually run RITA. ## Obtaining the RITA Docker Image The easiest way is to pull down the pre-built image. ``` -docker pull quay.io/activecm/rita:master +docker pull quay.io/activecm/rita ``` You can also build the image from source. ``` -docker build -t quay.io/activecm/rita:master . +docker build -t quay.io/activecm/rita . ``` ## Running RITA with Docker Compose -At the very least, you will have to provide RITA with the path to your Bro log files using the `BRO_LOGS` environment variable. +You will need a config file where you have [put in your `InternalSubnets`](../Readme.md#configuration-file). +You will also need the path to your Bro/Zeek log files. ``` -export BRO_LOGS=/path/to/your/logs -docker-compose run --rm rita import -docker-compose run --rm rita analyze +export CONIFG=/path/to/your/rita/config.yaml +export LOGS=/path/to/your/zeek/logs +docker-compose run --rm rita import /logs your-dataset ``` -You can also call it this way if you wish. +Note: If you'd like to use a different version of RITA than the default `latest` you can do so using the `RITA_VERSION` variable. ``` -BRO_LOGS=/path/to/your/logs docker-compose run --rm rita import -BRO_LOGS=/path/to/your/logs docker-compose run --rm rita analyze -``` - -RITA will use the default `config.yaml` file which will work out of the box. If you wish to specify your own config file you can do so like this: - -``` -export BRO_LOGS=/path/to/your/logs -docker-compose run --rm -v /path/to/your/rita/config.yaml:/etc/rita/config.yaml rita show-databases -``` - -Note: If you'd like to use a different version of RITA than the default `master` you can do so using the `RITA_VERSION` variable. - -``` -export RITA_VERSION=dev -export BRO_LOGS=/path/to/your/logs -docker-compose run --rm -v /path/to/your/rita/config.yaml:/etc/rita/config.yaml rita show-databases +export VERSION=v3.0.6 +docker-compose run --rm rita --version ``` ## Running RITA with Docker Using External Mongo -If you don't need/want the convenience of Docker Compose running the Mongo server for you, you can also use RITA without it. You will need to modify RITA's config file to point to your external Mongo server. +If you don't need/want the convenience of Docker Compose running the Mongo server for you, you can also use RITA without it. You will need to modify RITA's config file to point to your external Mongo server and invoke RITA like this: ``` docker run -it --rm \ - -v /path/to/your/bro/logs:/opt/bro/logs/:ro \ - -v /path/to/your/rita/config.yaml:/etc/rita/config.yaml:ro \ - quay.io/activecm/rita:master import -docker run -it --rm \ - -v /path/to/your/bro/logs:/opt/bro/logs/:ro \ + -v /path/to/your/bro/logs:/logs:ro \ -v /path/to/your/rita/config.yaml:/etc/rita/config.yaml:ro \ - quay.io/activecm/rita:master analyze + quay.io/activecm/rita import /logs your-dataset ``` ## Using Docker to Build RITA @@ -71,7 +53,7 @@ docker run -it --rm \ You can use Docker to build a statically linked RITA binary for you. This binary should be portable between Linux 64-bit systems. Once you've obtained the RITA docker image (see the "Obtaining the RITA Docker Image" section above) you can run the following commands to copy the binary to your host system. ``` -docker create --name rita quay.io/activecm/rita:master +docker create --name rita quay.io/activecm/rita docker cp rita:/rita ./rita docker rm rita ``` diff --git a/docs/Manual Installation.md b/docs/Manual Installation.md index a2b036ee..01fa3fc2 100644 --- a/docs/Manual Installation.md +++ b/docs/Manual Installation.md @@ -27,7 +27,6 @@ RITA requires Mongo for storing and processing data. The current recommended ver You have a few options for installing RITA. 1. The main install script. You can disable Bro and Mongo from being installed with the `--disable-bro` and `--disable-mongo` flags. 1. A prebuilt binary is available for download on [RITA's release page](https://github.com/activecm/rita/releases). In this case you will need to download the config file from the same release and create some directories manually, as described below in the "Configuring the system" section. -1. [Use RITA with docker](Docker%20Usage.md) 1. Compile RITA manually from source. See below. ##### Installing Golang @@ -63,4 +62,4 @@ RITA requires a few directories to be created for it to function correctly. Copy the config file from your local RITA source code. * ```sudo cp $GOPATH/src/github.com/activecm/rita/etc/rita.yaml /etc/rita/config.yaml && sudo chmod 666 /etc/rita/config.yaml``` -At this point, you can modify the config file as needed and test using the ```rita test-config``` command. There will be empty quotes or 0's assigned to empty fields. [RITA's readme](../Readme.md) has more information on changing the configuration. +At this point, you can modify the config file as needed and test using the ```rita test-config``` command. There will be empty quotes or 0's assigned to empty fields. [RITA's readme](../Readme.md#configuration-file) has more information on changing the configuration. diff --git a/docs/System Requirements.md b/docs/System Requirements.md new file mode 100644 index 00000000..d68d8c29 --- /dev/null +++ b/docs/System Requirements.md @@ -0,0 +1,18 @@ +# System Requirements + +* Operating System - The preferred platform is 64-bit Ubuntu 16.04 LTS. The system should be patched and up to date using apt-get. + * The automated installer will also support Security Onion and CentOS 7. You can install on other operating systems using [docker](Docker%20Usage.md) or our [manual installation](Manual%20Installation.md). + +If RITA is used on a separate system from Bro/Zeek our recommended specs are: +* Processor - Two or more cores. RITA uses parallel processing and benefits from more CPU cores. +* Memory - 16GB. Larger datasets may require more memory. +* Storage - RITA's datasets are significantly smaller than the Bro/Zeek logs so storage requirements are minimal compared to retaining the Bro/Zeek log files. + + +## Bro/Zeek +The following requirements apply to the Bro/Zeek system. + +* Processor - Two cores plus an additional core for every 100 Mb of traffic being captured. (three cores minimum). This should be dedicated hardware, as resource congestion with other VMs can cause packets to be dropped or missed. +* Memory - 16GB minimum. 64GB if monitoring 100Mb or more of network traffic. 128GB if monitoring 1Gb or more of network traffic. +* Storage - 300GB minimum. 1TB or more is recommended to reduce log maintenance. +* Network - In order to capture traffic with Bro/Zeek, you will need at least 2 network interface cards (NICs). One will be for management of the system and the other will be the dedicated capture port. Intel NICs perform well and are recommended.