Filter Anomaly

Anomaly filter

Description

It's a filter that apply algorithms on connection data, and find anomalies in this data.

Filter Code

0x414D4C59

Dependencies

• armadillo 9.400 or above, with:
  • lapack (preferably lapacke)
  • blas (preferably openblas)
• mlpack 3.0.1 or above

Darwin configuration

Example of darwin configuration for this filter :

{
    "anomaly_1": {
        "exec_path": "/root/darwin/build/darwin_anomaly",
        "config_file":"",
        "output": "LOG",
        "next_filter": "",
        "nb_thread": 1,
        "log_level": "DEBUG",
        "cache_size": 0
    }
}

Config file

• redis_socket_path (optional) : the redis in which the filter will store the data
• alert_redis_list_name (optional) : the key under which the filter will store the raised alerts in the redis
• alert_redis_channel_name (optional) : the redis channel in which the raised alerts will be publish
• log_file_path (optional) : the path in which the filter will write it's results

Example :

{
    "redis_socket_path": "/var/sockets/redis/redis.sock",
    "alert_redis_list_name": "darwin_alerts",
    "alert_redis_channel_name": "darwin.alerts",
    "log_file_path": "/var/log/darwin/alerts.log"
}

Body

[ 
   [ 
      ["<net_src_ip>",<nb_dst_port_udp>,<nb_host_udp>,<nb_dst_port_tcp>,<nb_host_tcp>,<nb_host_icmp>], 
      [..] 
   ], 
      [..] 
]

Each array provided need at least 10 lines of data.

Example

Here is an example of a body:

[ 
   [     
      ["20.245.247.107",16,29,100,99,98],
      ["219.218.162.234",90,27,40,96,82],
      ["186.128.98.80",84,11,20,87,59],
      ["168.84.159.133",44,67,84,78,35],
      ["187.254.171.124",57,57,51,8,94],
      ["12.114.194.153",15,24,7,24,74],
      ["91.204.62.178",77,51,48,12,31],
      ["82.92.146.207",101,93,55,94,53],
      ["158.87.120.73",20,34,44,95,74],
      ["245.196.164.104",6,35,62,41,89],
      ["106.175.196.38",29,31,96,18,87],
   ], 
   [
      ["140.217.116.146",74,61,26,2,79],
      ["39.67.10.76",93,34,47,86,97],
      ["53.101.255.67",36,36,4,45,60],
      ["110.214.19.254",22,75,62,26,75],
      ["186.9.165.125",34,36,29,60,46],
      ["57.220.242.136",23,88,15,99,95],
      ["223.118.198.40",86,22,30,39,30],
      ["59.11.64.89",7,16,38,37,73],
      ["17.159.187.225",70,69,57,50,81],
      ["192.82.162.41",61,47,76,51,31],
      ["73.62.247.250",64,70,37,84,7],
      ["56.42.45.251",28,75,14,88,81],
      ["110.75.226.138",75,52,22,39,87],
   ],
]

Results

Send back certitude 0 or 100. 100 for anomaly found in the data cluster, 0 else. For example we can have [0, 100] for the body precised upside.

If the LOG output is precised in the darwin configuration, the body send by the filter will be for example :

{ 
    "evt_id": "<header's_evt_id>",
    "time": "<ISO8601>",
    "anomaly": {
        "ip": "10.5.22.144",
        "udp_nb_host": 16.000000,
        "udp_nb_port": 2.000000,
        "tcp_nb_host": 520.000000,
        "tcp_nb_port": 4.000000,
        "icmp_nb_host": 0.000000,
        "distance": 624.351380
    }
}

Alert Details

"details": {
    "udp_nb_host": <float, number of unique host connected via udp>,
    "udp_nb_port": <float, number of unique port connected via udp>,
    "tcp_nb_host": <float, number of unique host connected via tcp>,
    "tcp_nb_port": <float, number of unique port connected via tcp>,
    "distance": <float, distance to the closest normal asset>
}