diff --git a/.github/workflows/python-tests.yml b/.github/workflows/python-tests.yml
index 5bd34ec2..375414b6 100644
--- a/.github/workflows/python-tests.yml
+++ b/.github/workflows/python-tests.yml
@@ -24,8 +24,10 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y libemail-outlook-message-perl
- curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.2.2-amd64.deb
- sudo dpkg -i --force-confnew elasticsearch-*.deb
+ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
+ sudo apt-get install apt-transport-https
+ echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
+ sudo apt-get update && sudo apt-get install elasticsearch
sudo systemctl restart elasticsearch
sudo systemctl --no-pager status elasticsearch
- name: Install Python dependencies
@@ -46,7 +48,6 @@ jobs:
- name: Test sample DMARC reports
continue-on-error: true
run: |
- python setup.py install
parsedmarc --debug -c ci.ini samples/aggregate/*
parsedmarc --debug -c ci.ini samples/forensic/*"
- name: Test building packages
diff --git a/docs/source/elasticsearch.md b/docs/source/elasticsearch.md
index 856cf093..52dd2711 100644
--- a/docs/source/elasticsearch.md
+++ b/docs/source/elasticsearch.md
@@ -1,15 +1,13 @@
# Elasticsearch and Kibana
-:::{note}
-Splunk is also supported starting with `parsedmarc` 4.3.0
-:::
-
To set up visual dashboards of DMARC data, install Elasticsearch and Kibana.
:::{note}
Elasticsearch and Kibana 6 or later are required
:::
+## Installation
+
On Debian/Ubuntu based systems, run:
```bash
@@ -126,7 +124,7 @@ server.ssl.certificate: /etc/kibana/kibana.crt
server.ssl.key: /etc/kibana/kibana.key
```
-::{note}
+:::{note}
For more security, you can configure Kibana to use a local network connexion
to elasticsearch :
```text
@@ -136,6 +134,7 @@ elasticsearch.hosts: ['https://SERVER_IP:9200']
```text
elasticsearch.hosts: ['https://127.0.0.1:9200']
```
+:::
```bash
sudo systemctl restart kibana
diff --git a/docs/source/installation.md b/docs/source/installation.md
index c34abfd7..aaea4587 100644
--- a/docs/source/installation.md
+++ b/docs/source/installation.md
@@ -1,15 +1,17 @@
# Installation
-## Testing multiple report analyzers
+## Prerequisites
+
+`parsedmarc` works with Python 3 only.
+
+### Testing multiple report analyzers
If you would like to test parsedmarc and another report processing
solution at the same time, you can have up to two `mailto` URIs in each of the rua and ruf
tags in your DMARC record, separated by commas.
-:::
-`parsedmarc` works with Python 3 only.
+### Using a web proxy
-:::{note}
If your system is behind a web proxy, you need to configure your system
to use that proxy. To do this, edit `/etc/environment` and add your
proxy details there, for example:
@@ -29,18 +31,17 @@ ftp_proxy=http://prox-server:3128
```
This will set the proxy up for use system-wide, including for `parsedmarc`.
-:::
-:::{warning}
+### Using Microsoft Exchange
+
If your mail server is Microsoft Exchange, ensure that it is patched to at
least:
-- Exchange Server 2010 Update Rollup 22 ([KB4295699](https://support.microsoft.com/KB/4295699))
-- Exchange Server 2013 Cumulative Update 21 ([KB4099855](https://support.microsoft.com/KB/4099855))
-- Exchange Server 2016 Cumulative Update 11 ([KB4134118](https://support.microsoft.com/kb/4134118))
-:::
+- Exchange Server 2010 Update Rollup 22 ([KB4295699])
+- Exchange Server 2013 Cumulative Update 21 ([KB4099855])
+- Exchange Server 2016 Cumulative Update 11 ([KB4134118])
-## geoipupdate setup
+### geoipupdate setup
:::{note}
Starting in `parsedmarc` 7.1.0, a static copy of the
@@ -84,9 +85,8 @@ The latest builds for Linux, macOS, and Windows can be downloaded
from the [geoipupdate releases page on GitHub].
On December 30th, 2019, MaxMind started requiring free accounts to
-access the free Geolite2 databases, in order [to
-comply with various privacy
-regulations][to comply with various privacy regulations].
+access the free Geolite2 databases, in order
+[to comply with various privacy regulations].
Start by [registering for a free GeoLite2 account], and signing in.
@@ -146,7 +146,7 @@ sudo dnf install -y python39 python3-virtualenv python3-setuptools python3-devel
```
Python 3 installers for Windows and macOS can be found at
-
+.
Create a system user
@@ -191,6 +191,9 @@ On Debian or Ubuntu systems, run:
sudo apt-get install libemail-outlook-message-perl
```
+[KB4295699]: https://support.microsoft.com/KB/4295699
+[KB4099855]: https://support.microsoft.com/KB/4099855
+[KB4134118]: https://support.microsoft.com/kb/4134118
[Component "contrib"]: https://wiki.debian.org/SourcesList#Component
[geoipupdate]: https://github.com/maxmind/geoipupdate
[geoipupdate releases page on github]: https://github.com/maxmind/geoipupdate/releases
diff --git a/docs/source/usage.md b/docs/source/usage.md
index 41c7dd92..4a0f1f1a 100644
--- a/docs/source/usage.md
+++ b/docs/source/usage.md
@@ -111,17 +111,17 @@ The full set of configuration options are:
- `forensic_json_filename` - str: filename for the forensic
JSON output file
- `ip_db_path` - str: An optional custom path to a MMDB file
- - from MaxMind or DBIP
+ from MaxMind or DBIP
- `offline` - bool: Do not use online queries for geolocation
or DNS
- - `nameservers` - str: A comma separated list of
- DNS resolvers (Default: [Cloudflare's public resolvers])
+ - `nameservers` - str: A comma separated list of
+ DNS resolvers (Default: `[Cloudflare's public resolvers]`)
- `dns_timeout` - float: DNS timeout period
- `debug` - bool: Print debugging messages
- - `silent` - bool: Only print errors (Default: True)
+ - `silent` - bool: Only print errors (Default: `True`)
- `log_file` - str: Write log messages to a file at this path
- `n_procs` - int: Number of process to run in parallel when
- parsing in CLI mode (Default: 1)
+ parsing in CLI mode (Default: `1`)
- `chunk_size` - int: Number of files to give to each process
when running in parallel.
@@ -134,7 +134,7 @@ The full set of configuration options are:
- `reports_folder` - str: The mailbox folder (or label for
Gmail) where the incoming reports can be found
(Default: `INBOX`)
- - `archive_folder` - str: The mailbox folder (or label for
+ - `archive_folder` - str: The mailbox folder (or label for
Gmail) to sort processed emails into (Default: `Archive`)
- `watch` - bool: Use the IMAP `IDLE` command to process
- messages as they arrive or poll MS Graph for new messages
@@ -165,7 +165,7 @@ The full set of configuration options are:
:::
- `ssl` - bool: Use an encrypted SSL/TLS connection
- (Default: True)
+ (Default: `True`)
- `skip_certificate_verification` - bool: Skip certificate
verification (not recommended)
- `user` - str: The IMAP user
@@ -188,7 +188,7 @@ The full set of configuration options are:
- `token_file` - str: Path to save the token file
(Default: `.token`)
- `allow_unencrypted_storage` - bool: Allows the Azure Identity
- module to fall back to unencrypted token cache (Default: False).
+ module to fall back to unencrypted token cache (Default: `False`).
Even if enabled, the cache will always try encrypted storage first.
:::{note}
@@ -245,14 +245,14 @@ The full set of configuration options are:
- `hosts` - str: A comma separated list of Kafka hosts
- `user` - str: The Kafka user
- `passsword` - str: The Kafka password
- - `ssl` - bool: Use an encrypted SSL/TLS connection (Default: True)
+ - `ssl` - bool: Use an encrypted SSL/TLS connection (Default: `True`)
- `skip_certificate_verification` - bool: Skip certificate
verification (not recommended)
- `aggregate_topic` - str: The Kafka topic for aggregate reports
- `forensic_topic` - str: The Kafka topic for forensic reports
- `smtp`
- `host` - str: The SMTP hostname
- - `port` - int: The SMTP port (Default: 25)
+ - `port` - int: The SMTP port (Default: `25`)
- `ssl` - bool: Require SSL/TLS instead of using STARTTLS
- `skip_certificate_verification` - bool: Skip certificate
verification (not recommended)
@@ -272,26 +272,26 @@ The full set of configuration options are:
:::
- `s3`
- `bucket` - str: The S3 bucket name
- - `path` - str: The path to upload reports to (Default: /)
+ - `path` - str: The path to upload reports to (Default: `/`)
- `region_name` - str: The region name (Optional)
- `endpoint_url` - str: The endpoint URL (Optional)
- `access_key_id` - str: The access key id (Optional)
- `secret_access_key` - str: The secret access key (Optional)
- `syslog`
- `server` - str: The Syslog server name or IP address
- - `port` - int: The UDP port to use (Default: 514)
+ - `port` - int: The UDP port to use (Default: `514`)
- `gmail_api`
- `credentials_file` - str: Path to file containing the
- credentials, None to disable (Default: None)
+ credentials, None to disable (Default: `None`)
- `token_file` - str: Path to save the token file
- (Default: .token)
+ (Default: `.token`)
- `include_spam_trash` - bool: Include messages in Spam and
- Trash when searching reports (Default: False)
+ Trash when searching reports (Default: `False`)
- `scopes` - str: Comma separated list of scopes to use when
acquiring credentials
(Default: `https://www.googleapis.com/auth/gmail.modify`)
- `oauth2_port` - int: The TCP port for the local server to
- listen on for the OAuth2 response (Default: 8080)
+ listen on for the OAuth2 response (Default: `8080`)
- `log_analytics`
- `client_id` - str: The app registration's client ID
- `client_secret` - str: The app registration's client secret
@@ -315,7 +315,7 @@ The `nameservers` option should only be used if your network
blocks DNS requests to outside resolvers.
:::
-:::{warning}
+:::{note}
`save_aggregate` and `save_forensic` are separate options
because you may not want to save forensic reports
(also known as failure reports) to your Elasticsearch instance,