Curator

We're currently working on integrating Curator into Security Onion with our move to the Elastic stack.

Description

From: https://www.elastic.co/guide/en/elasticsearch/client/curator/5.1/about.html#about

Elasticsearch Curator helps you curate, or manage, your Elasticsearch indices and snapshots by:

1. Obtaining the full list of indices (or snapshots) from the cluster, as the actionable list
2. Iterate through a list of user-defined filters to progressively remove indices (or snapshots) from this actionable list as needed.
3. Perform various actions on the items which remain in the actionable list.

Curator runs as a Docker container within Security Onion, and, by default, is configured to perform a dry run of closing and deleting of indices after two days.

Configuration

The default configuration for Curator can be modified by making changes to the "action" files located in /etc/curator/action.

When Curator completes an action, it logs such activity in a log file found at /var/log/curator/curator.log.

Curator runs every minute, and is controlled by cron jobs defined in /etc/cron.d/.

To allow Curator to run without the --dry-run option, simply remove it from the respective /etc/cron.d/curator-[action] file.

#/etc/cron.d/curator-close
#
#crontab entry curator 'close' action

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/localbin:/sbin:/bin?usr/sbin:/usr/bin
0 1 * * * root docker exec so-curator curator --config /etc/curator/config/curator.yml 
/etc/curator/action/close.yml > /dev/null 2>&1

To change the number of days for which Curator might close/delete indices modify the value for unit_count in the respective .yml file:

  unit: days
  unit_count: 7