Compare commits
29 Commits
beta-2.0
...
a9289d8e47
| Author | SHA1 | Date | |
|---|---|---|---|
| a9289d8e47 | |||
| 67ec8af3ae | |||
| 691f45a1dc | |||
| 80197454a3 | |||
| 841cd09f2d | |||
| e7183864d0 | |||
| 12ac3dbf62 | |||
| e41ec93058 | |||
| 8a86e3142a | |||
| 9d003d12b4 | |||
| 63c638751b | |||
| a3e85b7207 | |||
| 4974be02b4 | |||
| 7fe2f9a5c1 | |||
| f4634d03bd | |||
| e1ca9fadcd | |||
| adb7700300 | |||
| ced0d4c2fc | |||
| f483c76638 | |||
| f65116aec8 | |||
| bdcb6de4b2 | |||
| af8e27d075 | |||
| accf926ff7 | |||
| acf387bd0e | |||
| ab7a91e020 | |||
| a1a0d6b757 | |||
| 2fb089805c | |||
| 6cf2a94431 | |||
| 162636e60f |
53
.github/ISSUE_TEMPLATE/bug_report.md
vendored
53
.github/ISSUE_TEMPLATE/bug_report.md
vendored
@ -1,42 +1,41 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Create a report to help us improve
|
||||
name: Rapport de bug
|
||||
about: Créez un rapport pour nous aider à nous améliorer
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Describe the bug**
|
||||
A clear and concise description of what the bug is.
|
||||
**Décrivez le bug**
|
||||
Une description claire et concise de ce qu'est le bug.
|
||||
|
||||
**Affected module**
|
||||
Which one is the module that is not working as expected, e.g. Nessus, Qualys WAS, Qualys VM, OpenVAS, ELK, Jira...).
|
||||
**Module affecté**
|
||||
Lequel des modules ne fonctionne pas comme prévu, par exemple, Nessus, Qualys WAS, Qualys VM, OpenVAS, ELK, Jira...
|
||||
|
||||
**VulnWhisperer debug trail**
|
||||
If applicable, paste the VulnWhisperer debug trail of the execution for further detail (execute with '-d' flag).
|
||||
**Trace de débogage de VulnWhisperer**
|
||||
Si possible, veuillez joindre la trace de débogage de l'exécution pour une enquête plus approfondie (exécuter avec l'option `-d`).
|
||||
|
||||
**To Reproduce**
|
||||
Steps to reproduce the behavior:
|
||||
1. Go to '...'
|
||||
2. Click on '....'
|
||||
3. Scroll down to '....'
|
||||
4. See error
|
||||
**Pour reproduire**
|
||||
Étapes pour reproduire le comportement :
|
||||
1. Allez à '...'
|
||||
2. Cliquez sur '....'
|
||||
3. Voir l'erreur
|
||||
|
||||
**Expected behavior**
|
||||
A clear and concise description of what you expected to happen.
|
||||
**Comportement attendu**
|
||||
Une description claire et concise de ce à quoi vous vous attendiez.
|
||||
|
||||
**Screenshots**
|
||||
If applicable, add screenshots to help explain your problem.
|
||||
**Captures d'écran**
|
||||
Si applicable, ajoutez des captures d'écran pour aider à expliquer votre problème.
|
||||
|
||||
**System in which VulnWhisperer runs (please complete the following information):**
|
||||
- OS: [e.g. Ubuntu Server]
|
||||
- Version: [e.g. 18.04.2 LTS]
|
||||
- VulnWhisperer Version: [e.g. 1.7.1]
|
||||
**Système sur lequel VulnWhisperer s'exécute (veuillez compléter les informations suivantes) :**
|
||||
- OS : [ex. Ubuntu Server]
|
||||
- Version : [ex. 18.04.2 LTS]
|
||||
- Version de VulnWhisperer : [ex. 1.7.1]
|
||||
|
||||
**Additional context**
|
||||
Add any other context about the problem here.
|
||||
**Contexte additionnel**
|
||||
Ajoutez tout autre contexte sur le problème ici.
|
||||
|
||||
## Important Note
|
||||
As VulnWhisperer relies on ELK for the data aggregation, it is expected that you already have an ELK instance or the knowledge to deploy one.
|
||||
In order to speed up deployment, we provide an updated and tested docker-compose file which deploys all the needed infrastructure and we will support its deployment, but we will not be giving support to ELK instances.
|
||||
**Note importante**
|
||||
Comme VulnWhisperer s'appuie sur ELK pour l'agrégation de données, il est attendu que vous ayez déjà une instance ELK ou les connaissances pour en déployer une.
|
||||
Pour accélérer le déploiement, nous fournissons un fichier docker-compose à jour et testé qui déploie toute l'infrastructure nécessaire et nous supporterons son déploiement, mais nous ne donnerons pas de support pour les instances ELK.
|
||||
1
.gitmodules
vendored
1
.gitmodules
vendored
@ -1,4 +1,3 @@
|
||||
[submodule "tests/data"]
|
||||
path = tests/data
|
||||
url = https://github.com/HASecuritySolutions/VulnWhisperer-tests.git
|
||||
branch = master
|
||||
|
||||
@ -22,13 +22,13 @@ before_install:
|
||||
install:
|
||||
- pip install -r requirements.txt
|
||||
- pip install flake8 # pytest # add another testing frameworks later
|
||||
- python setup.py install
|
||||
before_script:
|
||||
# stop the build if there are Python syntax errors or undefined names
|
||||
- flake8 . --count --exclude=deps/qualysapi --select=E901,E999,F821,F822,F823 --show-source --statistics
|
||||
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
|
||||
- flake8 . --count --exit-zero --exclude=deps/qualysapi --max-complexity=10 --max-line-length=127 --statistics
|
||||
script:
|
||||
- python setup.py install
|
||||
- bash tests/test-vuln_whisperer.sh
|
||||
- bash tests/test-docker.sh
|
||||
notifications:
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
FROM centos:latest
|
||||
FROM centos:7
|
||||
|
||||
MAINTAINER Justin Henderson justin@hasecuritysolutions.com
|
||||
|
||||
|
||||
188
README.md
188
README.md
@ -1,19 +1,15 @@
|
||||
<p align="center"><img src="https://github.com/austin-taylor/vulnwhisperer/blob/master/docs/source/vuln_whisperer_logo_s.png" width="400px"></p>
|
||||
<p align="center"> <i>Create <u><b>actionable data</b></u> from your vulnerability scans </i> </p>
|
||||
<p align="center"><img src="https://git.gudita.com/Cyberdefense/VulnWhisperer/raw/branch/master/docs/source/vuln_whisperer_logo_s.png" width="400px"></p>
|
||||
<p align="center"> <i>Créez des <u><b>données exploitables</b></u> à partir de vos scans de vulnérabilités</i> </p>
|
||||
|
||||
<p align="center" style="width:400px"><img src="https://github.com/austin-taylor/vulnwhisperer/blob/master/docs/source/vulnWhispererWebApplications.png" style="width:400px"></p>
|
||||
<p align="center" style="width:400px"><img src="https://git.gudita.com/Cyberdefense/VulnWhisperer/raw/branch/master/docs/source/vulnWhispererWebApplications.png" style="width:400px"></p>
|
||||
|
||||
|
||||
VulnWhisperer is a vulnerability management tool and report aggregator. VulnWhisperer will pull all the reports from the different Vulnerability scanners and create a file with a unique filename for each one, using that data later to sync with Jira and feed Logstash. Jira does a closed cycle full Sync with the data provided by the Scanners, while Logstash indexes and tags all of the information inside the report (see logstash files at /resources/elk6/pipeline/). Data is then shipped to ElasticSearch to be indexed, and ends up in a visual and searchable format in Kibana with already defined dashboards.
|
||||
VulnWhisperer est un outil de gestion des vulnérabilités et un agrégateur de rapports. VulnWhisperer récupère tous les rapports des différents scanners de vulnérabilités et crée un fichier avec un nom unique pour chacun, utilisant ensuite ces données pour se synchroniser avec Jira et alimenter Logstash. Jira effectue une synchronisation complète en cycle fermé avec les données fournies par les scanners, tandis que Logstash indexe et étiquette toutes les informations contenues dans le rapport (voir les fichiers logstash dans `/resources/elk6/pipeline/`). Les données sont ensuite envoyées à ElasticSearch pour être indexées, et finissent dans un format visuel et consultable dans Kibana avec des tableaux de bord déjà définis.
|
||||
|
||||
[](https://travis-ci.org/HASecuritySolutions/VulnWhisperer)
|
||||
[](http://choosealicense.com/licenses/mit/)
|
||||
[](https://twitter.com/VulnWhisperer)
|
||||
VulnWhisperer est un projet open-source financé par la communauté. VulnWhisperer est actuellement fonctionnel mais nécessite une refonte de la documentation et une revue de code. Si vous souhaitez de l'aide, si vous êtes intéressé par de nouvelles fonctionnalités, ou si vous recherchez un support payant, veuillez nous contacter à **info@sahelcyber.com**.
|
||||
|
||||
Currently Supports
|
||||
-----------------
|
||||
|
||||
### Vulnerability Frameworks
|
||||
### Scanners de Vulnérabilités Supportés
|
||||
|
||||
- [X] [Nessus (**v6**/**v7**/**v8**)](https://www.tenable.com/products/nessus/nessus-professional)
|
||||
- [X] [Qualys Web Applications](https://www.qualys.com/apps/web-app-scanning/)
|
||||
@ -26,143 +22,117 @@ Currently Supports
|
||||
- [ ] [NMAP](https://nmap.org/)
|
||||
- [ ] [Burp Suite](https://portswigger.net/burp)
|
||||
- [ ] [OWASP ZAP](https://www.zaproxy.org/)
|
||||
- [ ] More to come
|
||||
- [ ] Et d'autres à venir
|
||||
|
||||
### Reporting Frameworks
|
||||
### Plateformes de Reporting Supportées
|
||||
|
||||
- [X] [ELK](https://www.elastic.co/elk-stack)
|
||||
- [X] [Elastic Stack (**v6**/**v7**)](https://www.elastic.co/elk-stack)
|
||||
- [ ] [OpenSearch - Envisagé pour la prochaine mise à jour](https://opensearch.org/)
|
||||
- [X] [Jira](https://www.atlassian.com/software/jira)
|
||||
- [ ] [Splunk](https://www.splunk.com/)
|
||||
|
||||
Getting Started
|
||||
===============
|
||||
## Démarrage
|
||||
|
||||
1) Follow the [install requirements](#installreq)
|
||||
2) Fill out the section you want to process in <a href="https://github.com/HASecuritySolutions/VulnWhisperer/blob/master/configs/frameworks_example.ini">frameworks_example.ini file</a>
|
||||
3) [JIRA] If using Jira, fill Jira config in the config file mentioned above.
|
||||
3) [ELK] Modify the IP settings in the <a href="https://github.com/HASecuritySolutions/VulnWhisperer/tree/master/resources/elk6/pipeline">Logstash files to accommodate your environment</a> and import them to your logstash conf directory (default is /etc/logstash/conf.d/)
|
||||
4) [ELK] Import the <a href="https://github.com/HASecuritySolutions/VulnWhisperer/blob/master/resources/elk6/kibana.json">Kibana visualizations</a>
|
||||
5) [Run Vulnwhisperer](#run)
|
||||
1) Suivez les [prérequis d'installation](#installreq)
|
||||
2) Remplissez la section que vous souhaitez traiter dans le fichier <a href="https://git.gudita.com/Cyberdefense/VulnWhisperer/src/branch/master/configs/frameworks_example.ini">frameworks_example.ini</a>
|
||||
3) [JIRA] Si vous utilisez Jira, remplissez la configuration Jira dans le fichier de configuration mentionné ci-dessus.
|
||||
3) [ELK] Modifiez les paramètres IP dans les <a href="https://git.gudita.com/Cyberdefense/VulnWhisperer/src/branch/master/resources/elk6/pipeline">fichiers Logstash pour correspondre à votre environnement</a> et importez-les dans votre répertoire de configuration logstash (par défaut `/etc/logstash/conf.d/`)
|
||||
4) [ELK] Importez les <a href="https://git.gudita.com/Cyberdefense/VulnWhisperer/src/branch/master/resources/elk6/kibana.json">visualisations Kibana</a>
|
||||
5) [Exécutez Vulnwhisperer](#run)
|
||||
|
||||
Need assistance or just want to chat? Join our [slack channel](https://join.slack.com/t/vulnwhisperer/shared_invite/enQtNDQ5MzE4OTIyODU0LWQxZTcxYTY0MWUwYzA4MTlmMWZlYWY2Y2ZmM2EzNDFmNWVlOTM4MzNjYzI0YzdkMDA0YmQyYWRhZGI2NGUxNGI)
|
||||
> **Note importante concernant les liens du Wiki :** La migration de Gitea ne transfère pas toujours le Wiki d'un projet GitHub (qui est techniquement un dépôt séparé). Si les liens vers le Wiki (comme le guide de déploiement ELK) ne fonctionnent pas, vous devrez peut-être recréer ces pages manuellement dans l'onglet "Wiki" de votre dépôt sur Gitea.
|
||||
|
||||
Requirements
|
||||
-------------
|
||||
####
|
||||
* Python 2.7
|
||||
* Vulnerability Scanner
|
||||
* Reporting System: Jira / ElasticStack 6.6
|
||||
Besoin d'aide ou juste envie de discuter ? Rejoignez notre [canal Slack](https://join.slack.com/t/vulnwhisperer/shared_invite/enQtNDQ5MzE4OTIyODU0LWQxZTcxYTY0MWUwYzA4MTlmMWZlYWY2Y2ZmM2EzNDFmNWVlOTM4MzNjYzI0YzdkMDA0YmQyYWRhZGI2NGUxNGI)
|
||||
|
||||
<a id="installreq">Install Requirements-VulnWhisperer(may require sudo)</a>
|
||||
--------------------
|
||||
**Install OS packages requirement dependencies** (Debian-based distros, CentOS don't need it)
|
||||
## Prérequis
|
||||
* Python 2.7
|
||||
* Un Scanner de Vulnérabilités
|
||||
* Un Système de Reporting : Jira / ElasticStack 6.6
|
||||
|
||||
<a id="installreq"></a>
|
||||
## Prérequis d'Installation - VulnWhisperer (peut nécessiter sudo)
|
||||
**Installez les dépendances des paquets du système d'exploitation** (pour les distributions basées sur Debian, CentOS n'en a pas besoin)
|
||||
```shell
|
||||
sudo apt-get install zlib1g-dev libxml2-dev libxslt1-dev
|
||||
|
||||
sudo apt-get install zlib1g-dev libxml2-dev libxslt1-dev
|
||||
```
|
||||
(Optionnel) Utilisez un environnement virtuel python pour ne pas perturber les bibliothèques python de l'hôte
|
||||
|
||||
**(Optional) Use a python virtualenv to not mess with host python libraries**
|
||||
```shell
|
||||
virtualenv venv (will create the python 2.7 virtualenv)
|
||||
source venv/bin/activate (start the virtualenv, now pip will run there and should install libraries without sudo)
|
||||
virtualenv venv # créera l'environnement virtuel python 2.7
|
||||
|
||||
deactivate (for quitting the virtualenv once you are done)
|
||||
```
|
||||
source venv/bin/activate # démarre l'environnement, pip s'exécutera ici et devrait installer les bibliothèques sans sudo
|
||||
|
||||
**Install python libraries requirements**
|
||||
deactivate # pour quitter l'environnement virtuel une fois que vous avez terminé
|
||||
|
||||
```python
|
||||
pip install -r /path/to/VulnWhisperer/requirements.txt
|
||||
cd /path/to/VulnWhisperer
|
||||
Installez les dépendances des bibliothèques python
|
||||
|
||||
pip install -r /chemin/vers/VulnWhisperer/requirements.txt
|
||||
cd /chemin/vers/VulnWhisperer
|
||||
python setup.py install
|
||||
```
|
||||
|
||||
**(Optional) If using a proxy, add proxy URL as environment variable to PATH**
|
||||
```shell
|
||||
export HTTP_PROXY=http://example.com:8080
|
||||
export HTTPS_PROXY=http://example.com:8080
|
||||
```
|
||||
(Optionnel) Si vous utilisez un proxy, ajoutez l'URL du proxy comme variable d'environnement au PATH
|
||||
|
||||
Now you're ready to pull down scans. (see <a href="#run">run section</a>)
|
||||
export HTTP_PROXY=[http://exemple.com:8080](http://exemple.com:8080)
|
||||
export HTTPS_PROXY=[http://exemple.com:8080](http://exemple.com:8080)
|
||||
|
||||
Vous êtes maintenant prêt à télécharger les scans.
|
||||
|
||||
Configuration
|
||||
-----
|
||||
Il y a quelques étapes de configuration pour mettre en place VulnWhisperer :
|
||||
|
||||
There are a few configuration steps to setting up VulnWhisperer:
|
||||
* Configure Ini file
|
||||
* Setup Logstash File
|
||||
* Import ElasticSearch Templates
|
||||
* Import Kibana Dashboards
|
||||
Configurer le fichier Ini
|
||||
|
||||
<a href="https://github.com/austin-taylor/VulnWhisperer/blob/master/configs/frameworks_example.ini">frameworks_example.ini file</a>
|
||||
<p align="left" style="width:200px"><img src="https://github.com/austin-taylor/vulnwhisperer/blob/master/docs/source/config_example.png" style="width:200px"></p>
|
||||
Configurer le fichier Logstash
|
||||
|
||||
Importer les modèles ElasticSearch
|
||||
|
||||
<a id="run">Run</a>
|
||||
-----
|
||||
To run, fill out the configuration file with your vulnerability scanner settings. Then you can execute from the command line.
|
||||
```python
|
||||
(optional flag: -F -> provides "Fancy" log colouring, good for comprehension when manually executing VulnWhisperer)
|
||||
Importer les tableaux de bord Kibana
|
||||
|
||||
Exécution
|
||||
Pour exécuter, remplissez le fichier de configuration avec les paramètres de votre scanner de vulnérabilités. Ensuite, vous pouvez l'exécuter depuis la ligne de commande.
|
||||
|
||||
# (optionnel : -F -> fournit une coloration "Fantaisie" des logs, utile pour la compréhension lors de l'exécution manuelle de VulnWhisperer)
|
||||
vuln_whisperer -c configs/frameworks_example.ini -s nessus
|
||||
or
|
||||
# ou
|
||||
vuln_whisperer -c configs/frameworks_example.ini -s qualys
|
||||
|
||||
```
|
||||
If no section is specified (e.g. -s nessus), vulnwhisperer will check on the config file for the modules that have the property `enabled=true` and run them sequentially.
|
||||
|
||||
<p align="center" style="width:300px"><img src="https://github.com/austin-taylor/vulnwhisperer/blob/master/docs/source/running_vuln_whisperer.png" style="width:400px"></p>
|
||||
Next you'll need to import the visualizations into Kibana and setup your logstash config. You can either follow the sample setup instructions [here](https://github.com/HASecuritySolutions/VulnWhisperer/wiki/Sample-Guide-ELK-Deployment) or go for the `docker-compose` solution we offer.
|
||||
|
||||
Si aucune section n'est spécifiée (ex. -s nessus), vulnwhisperer vérifiera dans le fichier de configuration les modules ayant la propriété enabled=true et les exécutera séquentiellement.
|
||||
|
||||
Docker-compose
|
||||
-----
|
||||
ELK is a whole world by itself, and for newcomers to the platform, it requires basic Linux skills and usually a bit of troubleshooting until it is deployed and working as expected. As we are not able to provide support for each users ELK problems, we put together a docker-compose which includes:
|
||||
ELK est un monde en soi, et pour les nouveaux venus sur la plateforme, cela nécessite des compétences de base sous Linux et généralement un peu de dépannage jusqu'à ce qu'il soit déployé et fonctionne comme prévu. Comme nous ne sommes pas en mesure de fournir un support pour les problèmes ELK de chaque utilisateur, nous avons mis en place un docker-compose qui inclut :
|
||||
|
||||
- VulnWhisperer
|
||||
- Logstash 6.6
|
||||
- ElasticSearch 6.6
|
||||
- Kibana 6.6
|
||||
VulnWhisperer
|
||||
|
||||
The docker-compose just requires specifying the paths where the VulnWhisperer data will be saved, and where the config files reside. If ran directly after `git clone`, with just adding the Scanner config to the VulnWhisperer config file ([/resources/elk6/vulnwhisperer.ini](https://github.com/HASecuritySolutions/VulnWhisperer/blob/master/resources/elk6/vulnwhisperer.ini)), it will work out of the box.
|
||||
Logstash 6.6
|
||||
|
||||
It also takes care to load the Kibana Dashboards and Visualizations automatically through the API, which needs to be done manually otherwise at Kibana's startup.
|
||||
ElasticSearch 6.6
|
||||
|
||||
For more info about the docker-compose, check on the [docker-compose wiki](https://github.com/HASecuritySolutions/VulnWhisperer/wiki/docker-compose-Instructions) or the [FAQ](https://github.com/HASecuritySolutions/VulnWhisperer/wiki).
|
||||
Kibana 6.6
|
||||
|
||||
Getting Started
|
||||
===============
|
||||
Le docker-compose nécessite simplement de spécifier les chemins où les données de VulnWhisperer seront sauvegardées, et où se trouvent les fichiers de configuration. S'il est exécuté directement après un git clone, en ajoutant simplement la configuration du scanner au fichier de configuration de VulnWhisperer (/resources/elk6/vulnwhisperer.ini), il fonctionnera immédiatement.
|
||||
|
||||
Our current Roadmap is as follows:
|
||||
- [ ] Create a Vulnerability Standard
|
||||
- [ ] Map every scanner results to the standard
|
||||
- [ ] Create Scanner module guidelines for easy integration of new scanners (consistency will allow #14)
|
||||
- [ ] Refactor the code to reuse functions and enable full compatibility among modules
|
||||
- [ ] Change Nessus CSV to JSON (Consistency and Fix #82)
|
||||
- [ ] Adapt single Logstash to standard and Kibana Dashboards
|
||||
- [ ] Implement Detectify Scanner
|
||||
- [ ] Implement Splunk Reporting/Dashboards
|
||||
Il se charge également de charger automatiquement les tableaux de bord et les visualisations Kibana via l'API, ce qui doit être fait manuellement autrement au démarrage de Kibana.
|
||||
|
||||
On top of this, we try to focus on fixing bugs as soon as possible, which might delay the development. We also very welcome PR's, and once we have the new standard implemented, it will be very easy to add compatibility with new scanners.
|
||||
Pour plus d'informations sur le docker-compose, consultez le wiki docker-compose ou la FAQ.
|
||||
|
||||
The Vulnerability Standard will initially be a new simple one level JSON with all the information that matches from the different scanners having standardized variable names, while maintaining the rest of the variables as they are. In the future, once everything is implemented, we will evaluate moving to an existing standard like ECS or AWS Vulnerability Schema; we prioritize functionality over perfection.
|
||||
Feuille de route
|
||||
Notre feuille de route actuelle est la suivante :
|
||||
|
||||
Video Walkthrough -- Featured on ElasticWebinar
|
||||
----------------------------------------------
|
||||
<a href="http://www.youtube.com/watch?feature=player_embedded&v=zrEuTtRUfNw?start=30
|
||||
" target="_blank"><img src="https://github.com/austin-taylor/vulnwhisperer/blob/master/docs/source/elastic_webinar.png"
|
||||
alt="Elastic presentation on VulnWhisperer" border="10" /></a>
|
||||
[ ] Créer un standard de vulnérabilité
|
||||
|
||||
Authors
|
||||
------
|
||||
- [Austin Taylor (@HuntOperator)](https://github.com/austin-taylor)
|
||||
- [Justin Henderson (@smapper)](https://github.com/SMAPPER)
|
||||
[ ] Mapper les résultats de chaque scanner au standard
|
||||
|
||||
Contributors
|
||||
------------
|
||||
- [Quim Montal (@qmontal)](https://github.com/qmontal)
|
||||
- [@pemontto](https://github.com/pemontto)
|
||||
- [@cybergoof](https://github.com/cybergoof)
|
||||
[ ] Créer des directives de module de scanner pour une intégration facile de nouveaux scanners
|
||||
|
||||
AS SEEN ON TV
|
||||
-------------
|
||||
<p align="center" style="width:400px"><a href="https://twitter.com/MalwareJake/status/935654519471353856"><img src="https://github.com/austin-taylor/vulnwhisperer/blob/master/docs/source/as_seen_on_tv.png" style="width:400px"></a></p>
|
||||
[ ] Refactoriser le code pour réutiliser les fonctions et permettre une compatibilité totale entre les modules
|
||||
|
||||
[ ] Changer Nessus CSV en JSON
|
||||
|
||||
[ ] Adapter le Logstash unique au standard et aux tableaux de bord Kibana
|
||||
|
||||
[ ] Implémenter le scanner Detectify
|
||||
|
||||
[ ] Implémenter le reporting/tableaux de bord Splunk
|
||||
|
||||
En plus de cela, nous essayons de nous concentrer sur la correction des bugs dès que possible, ce qui peut retarder le développement. Nous accueillons également très volontiers les PR (Pull Requests), et une fois que nous aurons implémenté le nouveau standard, il sera très facile d'ajouter la compatibilité avec de nouveaux scanners.
|
||||
|
||||
Le standard de vulnérabilité sera initialement un nouveau JSON simple à un niveau avec toutes les informations correspondantes des différents scanners ayant des noms de variables standardisés, tout en conservant le reste des variables telles quelles.
|
||||
|
||||
@ -3,14 +3,13 @@
|
||||
__author__ = 'Austin Taylor'
|
||||
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
from vulnwhisp.vulnwhisp import vulnWhisperer
|
||||
from vulnwhisp.base.config import vwConfig
|
||||
from vulnwhisp.test.mock import mockAPI
|
||||
from vulnwhisp.vulnwhisp import vulnWhisperer
|
||||
import os
|
||||
import argparse
|
||||
import sys
|
||||
import logging
|
||||
|
||||
|
||||
def isFileValid(parser, arg):
|
||||
@ -28,18 +27,16 @@ def main():
|
||||
help='Path of config file', type=lambda x: isFileValid(parser, x.strip()))
|
||||
parser.add_argument('-s', '--section', dest='section', required=False,
|
||||
help='Section in config')
|
||||
parser.add_argument('-f', '--filter', dest='scan_filter', required=False,
|
||||
help='Retrieve scans matching this regex pattern')
|
||||
parser.add_argument('--days', dest='days', type=int, required=False,
|
||||
help='Retrieve scans from this many days ago to now')
|
||||
parser.add_argument('-l', '--list', dest='list_scans', required=False, action="store_true",
|
||||
help='List available scans')
|
||||
parser.add_argument('--source', dest='source', required=False,
|
||||
help='JIRA required only! Source scanner to report')
|
||||
parser.add_argument('-n', '--scanname', dest='scanname', required=False,
|
||||
help='JIRA required only! Scan name from scan to report')
|
||||
parser.add_argument('-v', '--verbose', dest='verbose', action='store_true',
|
||||
parser.add_argument('-v', '--verbose', dest='verbose', action='store_true', default=True,
|
||||
help='Prints status out to screen (defaults to True)')
|
||||
parser.add_argument('-u', '--username', dest='username', required=False, default=None,
|
||||
help='The NESSUS username', type=lambda x: x.strip())
|
||||
parser.add_argument('-p', '--password', dest='password', required=False, default=None,
|
||||
help='The NESSUS password', type=lambda x: x.strip())
|
||||
parser.add_argument('-F', '--fancy', action='store_true',
|
||||
help='Enable colourful logging output')
|
||||
parser.add_argument('-d', '--debug', action='store_true',
|
||||
@ -53,14 +50,14 @@ def main():
|
||||
# First setup logging
|
||||
logging.basicConfig(
|
||||
stream=sys.stdout,
|
||||
# format only applies when not using -F flag for colouring
|
||||
#format only applies when not using -F flag for colouring
|
||||
format='%(levelname)s:%(name)s:%(funcName)s:%(message)s',
|
||||
level=logging.DEBUG if args.debug else logging.INFO if args.verbose else logging.WARNING
|
||||
level=logging.DEBUG if args.debug else logging.INFO
|
||||
)
|
||||
logger = logging.getLogger()
|
||||
# we set up the logger to log as well to file
|
||||
fh = logging.FileHandler('vulnwhisperer.log')
|
||||
fh.setLevel(logging.DEBUG if args.debug else logging.INFO if args.verbose else logging.WARNING)
|
||||
fh.setLevel(logging.DEBUG if args.debug else logging.INFO)
|
||||
fh.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(name)s - %(funcName)s:%(message)s", "%Y-%m-%d %H:%M:%S"))
|
||||
logger.addHandler(fh)
|
||||
|
||||
@ -77,46 +74,45 @@ def main():
|
||||
try:
|
||||
if args.config and not args.section:
|
||||
# this remains a print since we are in the main binary
|
||||
print(
|
||||
"WARNING: No section was specified, vulnwhisperer will scrape enabled modules from config file. \
|
||||
\nPlease specify a section using -s. \
|
||||
\nExample vuln_whisperer -c config.ini -s nessus"
|
||||
)
|
||||
print('WARNING: {warning}'.format(warning='No section was specified, vulnwhisperer will scrape enabled modules from config file. \
|
||||
\nPlease specify a section using -s. \
|
||||
\nExample vuln_whisperer -c config.ini -s nessus'))
|
||||
logger.info('No section was specified, vulnwhisperer will scrape enabled modules from the config file.')
|
||||
|
||||
config = vwConfig(config_in=args.config)
|
||||
enabled_sections = config.get_sections_with_attribute('enabled')
|
||||
|
||||
for section in enabled_sections:
|
||||
vw = vulnWhisperer(config=args.config,
|
||||
profile=section,
|
||||
verbose=args.verbose,
|
||||
debug=args.debug,
|
||||
source=args.source,
|
||||
scan_filter=args.scan_filter,
|
||||
days=args.days,
|
||||
scanname=args.scanname,
|
||||
list_scans=args.list_scans)
|
||||
exit_code += vw.whisper_vulnerabilities()
|
||||
try:
|
||||
vw = vulnWhisperer(config=args.config,
|
||||
profile=section,
|
||||
verbose=args.verbose,
|
||||
username=args.username,
|
||||
password=args.password,
|
||||
source=args.source,
|
||||
scanname=args.scanname)
|
||||
exit_code += vw.whisper_vulnerabilities()
|
||||
except Exception as e:
|
||||
logger.error("VulnWhisperer was unable to perform the processing on '{}'".format(args.source))
|
||||
else:
|
||||
logger.info('Running vulnwhisperer for section {}'.format(args.section))
|
||||
vw = vulnWhisperer(config=args.config,
|
||||
profile=args.section,
|
||||
verbose=args.verbose,
|
||||
debug=args.debug,
|
||||
username=args.username,
|
||||
password=args.password,
|
||||
source=args.source,
|
||||
scan_filter=args.scan_filter,
|
||||
days=args.days,
|
||||
scanname=args.scanname,
|
||||
list_scans=args.list_scans)
|
||||
scanname=args.scanname)
|
||||
exit_code += vw.whisper_vulnerabilities()
|
||||
|
||||
close_logging_handlers(logger)
|
||||
sys.exit(exit_code)
|
||||
|
||||
except Exception as e:
|
||||
logger.error('{}'.format(str(e)))
|
||||
print('ERROR: {error}'.format(error=e))
|
||||
if args.verbose:
|
||||
# this will remain a print since we are in the main binary
|
||||
logger.error('{}'.format(str(e)))
|
||||
print('ERROR: {error}'.format(error=e))
|
||||
# TODO: fix this to NOT be exit 2 unless in error
|
||||
close_logging_handlers(logger)
|
||||
sys.exit(2)
|
||||
|
||||
@ -9,8 +9,7 @@ password=nessus_password
|
||||
write_path=/opt/VulnWhisperer/data/nessus/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
trash=false
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[tenable]
|
||||
enabled=true
|
||||
@ -23,8 +22,7 @@ password=tenable.io_password
|
||||
write_path=/opt/VulnWhisperer/data/tenable/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
trash=false
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[qualys_web]
|
||||
#Reference https://www.qualys.com/docs/qualys-was-api-user-guide.pdf to find your API
|
||||
@ -35,7 +33,6 @@ password = examplepass
|
||||
write_path=/opt/VulnWhisperer/data/qualys_web/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=true
|
||||
scan_filter=
|
||||
|
||||
# Set the maximum number of retries each connection should attempt.
|
||||
#Note, this applies only to failed connections and timeouts, never to requests where the server returns a response.
|
||||
@ -44,15 +41,14 @@ max_retries = 10
|
||||
template_id = 126024
|
||||
|
||||
[qualys_vuln]
|
||||
#Reference https://www.qualys.com/docs/qualys-was-api-user-guide.pdf to find your API
|
||||
#Reference https://www.qualys.com/docs/qualys-api-vmpc-user-guide.pdf to find your API
|
||||
enabled = true
|
||||
hostname = qualysapi.qg2.apps.qualys.com
|
||||
username = exampleuser
|
||||
password = examplepass
|
||||
write_path=/opt/VulnWhisperer/data/qualys_vuln/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[detectify]
|
||||
#Reference https://developer.detectify.com/
|
||||
@ -65,7 +61,6 @@ password = examplepass
|
||||
write_path =/opt/VulnWhisperer/data/detectify/
|
||||
db_path = /opt/VulnWhisperer/data/database
|
||||
verbose = true
|
||||
scan_filter=
|
||||
|
||||
[openvas]
|
||||
enabled = false
|
||||
@ -75,8 +70,7 @@ username = exampleuser
|
||||
password = examplepass
|
||||
write_path=/opt/VulnWhisperer/data/openvas/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[jira]
|
||||
enabled = false
|
||||
|
||||
@ -9,8 +9,7 @@ password=nessus_password
|
||||
write_path=/opt/VulnWhisperer/data/nessus/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
trash=false
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[tenable]
|
||||
enabled=true
|
||||
@ -23,78 +22,73 @@ password=tenable.io_password
|
||||
write_path=/opt/VulnWhisperer/data/tenable/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
trash=false
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[qualys_was]
|
||||
[qualys_web]
|
||||
#Reference https://www.qualys.com/docs/qualys-was-api-user-guide.pdf to find your API
|
||||
enabled=true
|
||||
hostname=qualys_was
|
||||
username=exampleuser
|
||||
password=examplepass
|
||||
write_path=/opt/VulnWhisperer/data/qualys_was/
|
||||
enabled = false
|
||||
hostname = qualys_web
|
||||
username = exampleuser
|
||||
password = examplepass
|
||||
write_path=/opt/VulnWhisperer/data/qualys_web/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
# Set the maximum number of retries each connection should attempt.
|
||||
#Note, this applies only to failed connections and timeouts, never to requests where the server returns a response.
|
||||
max_retries=10
|
||||
max_retries = 10
|
||||
# Template ID will need to be retrieved for each document. Please follow the reference guide above for instructions on how to get your template ID.
|
||||
template_id=289109
|
||||
template_id = 126024
|
||||
|
||||
[qualys_vm]
|
||||
[qualys_vuln]
|
||||
#Reference https://www.qualys.com/docs/qualys-was-api-user-guide.pdf to find your API
|
||||
enabled=true
|
||||
hostname=qualys_vm
|
||||
username=exampleuser
|
||||
password=examplepass
|
||||
write_path=/opt/VulnWhisperer/data/qualys_vm/
|
||||
enabled = true
|
||||
hostname = qualys_vuln
|
||||
username = exampleuser
|
||||
password = examplepass
|
||||
write_path=/opt/VulnWhisperer/data/qualys_vuln/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[detectify]
|
||||
#Reference https://developer.detectify.com/
|
||||
enabled=false
|
||||
hostname=detectify
|
||||
enabled = false
|
||||
hostname = detectify
|
||||
#username variable used as apiKey
|
||||
username=exampleuser
|
||||
username = exampleuser
|
||||
#password variable used as secretKey
|
||||
password=examplepass
|
||||
password = examplepass
|
||||
write_path =/opt/VulnWhisperer/data/detectify/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=false
|
||||
scan_filter=
|
||||
db_path = /opt/VulnWhisperer/data/database
|
||||
verbose = true
|
||||
|
||||
[openvas]
|
||||
enabled=true
|
||||
hostname=openvas
|
||||
port=4000
|
||||
username=exampleuser
|
||||
password=examplepass
|
||||
enabled = false
|
||||
hostname = openvas
|
||||
port = 4000
|
||||
username = exampleuser
|
||||
password = examplepass
|
||||
write_path=/opt/VulnWhisperer/data/openvas/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=false
|
||||
scan_filter=
|
||||
verbose=true
|
||||
|
||||
[jira]
|
||||
enabled=false
|
||||
hostname=jira-host
|
||||
username=username
|
||||
password=password
|
||||
write_path=/opt/VulnWhisperer/data/jira/
|
||||
db_path=/opt/VulnWhisperer/data/database
|
||||
verbose=false
|
||||
dns_resolv=False
|
||||
enabled = false
|
||||
hostname = jira-host
|
||||
username = username
|
||||
password = password
|
||||
write_path = /opt/VulnWhisperer/data/jira/
|
||||
db_path = /opt/VulnWhisperer/data/database
|
||||
verbose = true
|
||||
dns_resolv = False
|
||||
|
||||
#Sample jira report scan, will automatically be created for existent scans
|
||||
#[jira.qualys_vm.test_scan]
|
||||
#source=qualys_vm
|
||||
#scan_name=Test Scan
|
||||
#jira_project=PROJECT
|
||||
; if multiple components, separate by ","=None
|
||||
#[jira.qualys_vuln.test_scan]
|
||||
#source = qualys_vuln
|
||||
#scan_name = Test Scan
|
||||
#jira_project = PROJECT
|
||||
; if multiple components, separate by "," = None
|
||||
#components =
|
||||
; minimum criticality to report (low, medium, high or critical)=None
|
||||
#min_critical_to_report=high
|
||||
; minimum criticality to report (low, medium, high or critical) = None
|
||||
#min_critical_to_report = high
|
||||
|
||||
|
||||
@ -9,7 +9,6 @@ services:
|
||||
- "ES_JAVA_OPTS=-Xms1g -Xmx1g"
|
||||
- xpack.security.enabled=false
|
||||
- cluster.routing.allocation.disk.threshold_enabled=false
|
||||
- path.repo=/snapshots
|
||||
ulimits:
|
||||
memlock:
|
||||
soft: -1
|
||||
@ -81,7 +80,6 @@ services:
|
||||
entrypoint: [
|
||||
"vuln_whisperer",
|
||||
"-F",
|
||||
"-v",
|
||||
"-c",
|
||||
"/opt/VulnWhisperer/vulnwhisperer.ini",
|
||||
"--mock",
|
||||
|
||||
@ -8,7 +8,6 @@ services:
|
||||
- bootstrap.memory_lock=true
|
||||
- "ES_JAVA_OPTS=-Xms1g -Xmx1g"
|
||||
- xpack.security.enabled=false
|
||||
- path.repo=/snapshots
|
||||
ulimits:
|
||||
memlock:
|
||||
soft: -1
|
||||
@ -19,7 +18,6 @@ services:
|
||||
mem_limit: 8g
|
||||
volumes:
|
||||
- esdata1:/usr/share/elasticsearch/data
|
||||
- ./data/es_snapshots:/snapshots
|
||||
ports:
|
||||
- 9200:9200
|
||||
#restart: always
|
||||
|
||||
@ -1,12 +1,12 @@
|
||||
pandas==0.20.3
|
||||
setuptools==40.4.3
|
||||
pytz==2017.2
|
||||
Requests==2.18.3
|
||||
lxml==4.1.1
|
||||
Requests==2.20.0
|
||||
lxml==4.6.5
|
||||
future-fstrings
|
||||
bs4
|
||||
jira
|
||||
bottle
|
||||
coloredlogs
|
||||
qualysapi>=5.1.0
|
||||
qualysapi==6.0.0
|
||||
httpretty
|
||||
@ -0,0 +1,72 @@
|
||||
version: '2'
|
||||
services:
|
||||
vulnwhisp-es1:
|
||||
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.2
|
||||
container_name: vulnwhisp-es1
|
||||
environment:
|
||||
- cluster.name=vulnwhisperer
|
||||
- bootstrap.memory_lock=true
|
||||
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
|
||||
ulimits:
|
||||
memlock:
|
||||
soft: -1
|
||||
hard: -1
|
||||
nofile:
|
||||
soft: 65536
|
||||
hard: 65536
|
||||
mem_limit: 8g
|
||||
volumes:
|
||||
- esdata1:/usr/share/elasticsearch/data
|
||||
ports:
|
||||
- 9200:9200
|
||||
environment:
|
||||
- xpack.security.enabled=false
|
||||
#restart: always
|
||||
networks:
|
||||
esnet:
|
||||
aliases:
|
||||
- vulnwhisp-es1.local
|
||||
vulnwhisp-ks1:
|
||||
image: docker.elastic.co/kibana/kibana:5.6.2
|
||||
environment:
|
||||
SERVER_NAME: vulnwhisp-ks1
|
||||
ELASTICSEARCH_URL: http://vulnwhisp-es1:9200
|
||||
ports:
|
||||
- 5601:5601
|
||||
depends_on:
|
||||
- vulnwhisp-es1
|
||||
networks:
|
||||
esnet:
|
||||
aliases:
|
||||
- vulnwhisp-ks1.local
|
||||
vulnwhisp-ls1:
|
||||
image: docker.elastic.co/logstash/logstash:5.6.2
|
||||
container_name: vulnwhisp-ls1
|
||||
volumes:
|
||||
- ./docker/1000_nessus_process_file.conf:/usr/share/logstash/pipeline/1000_nessus_process_file.conf
|
||||
- ./docker/2000_qualys_web_scans.conf:/usr/share/logstash/pipeline/2000_qualys_web_scans.conf
|
||||
- ./docker/3000_openvas.conf:/usr/share/logstash/pipeline/3000_openvas.conf
|
||||
- ./docker/4000_jira.conf:/usr/share/logstash/pipeline/4000_jira.conf
|
||||
- ./docker/logstash.yml:/usr/share/logstash/config/logstash.yml
|
||||
- ./data/:/opt/VulnWhisperer/data
|
||||
environment:
|
||||
- xpack.monitoring.enabled=false
|
||||
depends_on:
|
||||
- vulnwhisp-es1
|
||||
networks:
|
||||
esnet:
|
||||
aliases:
|
||||
- vulnwhisp-ls1.local
|
||||
vulnwhisp-vulnwhisperer:
|
||||
image: hasecuritysolutions/vulnwhisperer:latest
|
||||
container_name: vulnwhisp-vulnwhisperer
|
||||
volumes:
|
||||
- ./data/:/opt/VulnWhisperer/data
|
||||
- ./configs/frameworks_example.ini:/opt/VulnWhisperer/frameworks_example.ini
|
||||
network_mode: host
|
||||
volumes:
|
||||
esdata1:
|
||||
driver: local
|
||||
|
||||
networks:
|
||||
esnet:
|
||||
@ -0,0 +1,220 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: email@austintaylor.io
|
||||
# Last Update: 12/20/2017
|
||||
# Version 0.3
|
||||
# Description: Take in nessus reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/nessus/**/*"
|
||||
start_position => "beginning"
|
||||
tags => "nessus"
|
||||
type => "nessus"
|
||||
}
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/tenable/*.csv"
|
||||
start_position => "beginning"
|
||||
tags => "tenable"
|
||||
type => "tenable"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "nessus" in [tags] or "tenable" in [tags] {
|
||||
# Drop the header column
|
||||
if [message] =~ "^Plugin ID" { drop {} }
|
||||
|
||||
csv {
|
||||
# columns => ["plugin_id", "cve", "cvss", "risk", "asset", "protocol", "port", "plugin_name", "synopsis", "description", "solution", "see_also", "plugin_output"]
|
||||
columns => ["plugin_id", "cve", "cvss", "risk", "asset", "protocol", "port", "plugin_name", "synopsis", "description", "solution", "see_also", "plugin_output", "asset_uuid", "vulnerability_state", "ip", "fqdn", "netbios", "operating_system", "mac_address", "plugin_family", "cvss_base", "cvss_temporal", "cvss_temporal_vector", "cvss_vector", "cvss3_base", "cvss3_temporal", "cvss3_temporal_vector", "cvss_vector", "system_type", "host_start", "host_end"]
|
||||
separator => ","
|
||||
source => "message"
|
||||
}
|
||||
|
||||
ruby {
|
||||
code => "if event.get('description')
|
||||
event.set('description', event.get('description').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('synopsis')
|
||||
event.set('synopsis', event.get('synopsis').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('solution')
|
||||
event.set('solution', event.get('solution').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('see_also')
|
||||
event.set('see_also', event.get('see_also').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('plugin_output')
|
||||
event.set('plugin_output', event.get('plugin_output').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end"
|
||||
}
|
||||
|
||||
#If using filebeats as your source, you will need to replace the "path" field to "source"
|
||||
grok {
|
||||
match => { "path" => "(?<scan_name>[a-zA-Z0-9_.\-]+)_%{INT:scan_id}_%{INT:history_id}_%{INT:last_updated}.csv$" }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => ["last_updated"]
|
||||
}
|
||||
|
||||
if [risk] == "None" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
}
|
||||
if [risk] == "Low" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
}
|
||||
if [risk] == "Medium" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
}
|
||||
if [risk] == "High" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
}
|
||||
if [risk] == "Critical" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
}
|
||||
|
||||
if ![cve] or [cve] == "nan" {
|
||||
mutate { remove_field => [ "cve" ] }
|
||||
}
|
||||
if ![cvss] or [cvss] == "nan" {
|
||||
mutate { remove_field => [ "cvss" ] }
|
||||
}
|
||||
if ![cvss_base] or [cvss_base] == "nan" {
|
||||
mutate { remove_field => [ "cvss_base" ] }
|
||||
}
|
||||
if ![cvss_temporal] or [cvss_temporal] == "nan" {
|
||||
mutate { remove_field => [ "cvss_temporal" ] }
|
||||
}
|
||||
if ![cvss_temporal_vector] or [cvss_temporal_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss_temporal_vector" ] }
|
||||
}
|
||||
if ![cvss_vector] or [cvss_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss_vector" ] }
|
||||
}
|
||||
if ![cvss3_base] or [cvss3_base] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_base" ] }
|
||||
}
|
||||
if ![cvss3_temporal] or [cvss3_temporal] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_temporal" ] }
|
||||
}
|
||||
if ![cvss3_temporal_vector] or [cvss3_temporal_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_temporal_vector" ] }
|
||||
}
|
||||
if ![description] or [description] == "nan" {
|
||||
mutate { remove_field => [ "description" ] }
|
||||
}
|
||||
if ![mac_address] or [mac_address] == "nan" {
|
||||
mutate { remove_field => [ "mac_address" ] }
|
||||
}
|
||||
if ![netbios] or [netbios] == "nan" {
|
||||
mutate { remove_field => [ "netbios" ] }
|
||||
}
|
||||
if ![operating_system] or [operating_system] == "nan" {
|
||||
mutate { remove_field => [ "operating_system" ] }
|
||||
}
|
||||
if ![plugin_output] or [plugin_output] == "nan" {
|
||||
mutate { remove_field => [ "plugin_output" ] }
|
||||
}
|
||||
if ![see_also] or [see_also] == "nan" {
|
||||
mutate { remove_field => [ "see_also" ] }
|
||||
}
|
||||
if ![synopsis] or [synopsis] == "nan" {
|
||||
mutate { remove_field => [ "synopsis" ] }
|
||||
}
|
||||
if ![system_type] or [system_type] == "nan" {
|
||||
mutate { remove_field => [ "system_type" ] }
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => [ "message" ]
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
mutate {
|
||||
convert => { "risk_score" => "float" }
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
|
||||
# Compensating controls - adjust risk_score
|
||||
# Adobe and Java are not allowed to run in browser unless whitelisted
|
||||
# Therefore, lower score by dividing by 3 (score is subjective to risk)
|
||||
|
||||
#Modify and uncomment when ready to use
|
||||
#if [risk_score] != 0 {
|
||||
# if [plugin_name] =~ "Adobe" and [risk_score] > 6 or [plugin_name] =~ "Java" and [risk_score] > 6 {
|
||||
# ruby {
|
||||
# code => "event.set('risk_score', event.get('risk_score') / 3)"
|
||||
# }
|
||||
# mutate {
|
||||
# add_field => { "compensating_control" => "Adobe and Flash removed from browsers unless whitelisted site." }
|
||||
# }
|
||||
# }
|
||||
#}
|
||||
|
||||
# Add tags for reporting based on assets or criticality
|
||||
|
||||
if [asset] == "dc01" or [asset] == "dc02" or [asset] == "pki01" or [asset] == "192.168.0.54" or [asset] =~ "^192\.168\.0\." or [asset] =~ "^42.42.42." {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
#if [asset] =~ "^192\.168\.[45][0-9][0-9]\.1$" or [asset] =~ "^192.168\.[50]\.[0-9]{1,2}\.1$"{
|
||||
# mutate {
|
||||
# add_tag => [ "has_hipaa_data" ]
|
||||
# }
|
||||
#}
|
||||
#if [asset] =~ "^192\.168\.[45][0-9][0-9]\." {
|
||||
# mutate {
|
||||
# add_tag => [ "hipaa_asset" ]
|
||||
# }
|
||||
#}
|
||||
if [asset] =~ "^hr" {
|
||||
mutate {
|
||||
add_tag => [ "pci_asset" ]
|
||||
}
|
||||
}
|
||||
#if [asset] =~ "^10\.0\.50\." {
|
||||
# mutate {
|
||||
# add_tag => [ "web_servers" ]
|
||||
# }
|
||||
#}
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
if "nessus" in [tags] or "tenable" in [tags] or [type] in [ "nessus", "tenable" ] {
|
||||
# stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "vulnwhisp-es1.local:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,153 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: austin@hasecuritysolutions.com
|
||||
# Last Update: 12/30/2017
|
||||
# Version 0.3
|
||||
# Description: Take in qualys web scan reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/qualys/*.json"
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "qualys" ]
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "qualys" in [tags] {
|
||||
grok {
|
||||
match => { "path" => [ "(?<tags>qualys_vuln)_scan_%{DATA}_%{INT:last_updated}.json$", "(?<tags>qualys_web)_%{INT:app_id}_%{INT:last_updated}.json$" ] }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
mutate {
|
||||
replace => [ "message", "%{message}" ]
|
||||
#gsub => [
|
||||
# "message", "\|\|\|", " ",
|
||||
# "message", "\t\t", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", "nan", " ",
|
||||
# "message",'\n',''
|
||||
#]
|
||||
}
|
||||
|
||||
if "qualys_web" in [tags] {
|
||||
mutate {
|
||||
add_field => { "asset" => "%{web_application_name}" }
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
} else if "qualys_vuln" in [tags] {
|
||||
mutate {
|
||||
add_field => { "asset" => "%{ip}" }
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
}
|
||||
|
||||
if [risk] == "1" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
mutate { replace => { "risk" => "info" }}
|
||||
}
|
||||
if [risk] == "2" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
mutate { replace => { "risk" => "low" }}
|
||||
}
|
||||
if [risk] == "3" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
mutate { replace => { "risk" => "medium" }}
|
||||
}
|
||||
if [risk] == "4" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
mutate { replace => { "risk" => "high" }}
|
||||
}
|
||||
if [risk] == "5" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
mutate { replace => { "risk" => "critical" }}
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => "message"
|
||||
}
|
||||
|
||||
if [first_time_detected] {
|
||||
date {
|
||||
match => [ "first_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_detected"
|
||||
}
|
||||
}
|
||||
if [first_time_tested] {
|
||||
date {
|
||||
match => [ "first_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_tested"
|
||||
}
|
||||
}
|
||||
if [last_time_detected] {
|
||||
date {
|
||||
match => [ "last_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_detected"
|
||||
}
|
||||
}
|
||||
if [last_time_tested] {
|
||||
date {
|
||||
match => [ "last_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_tested"
|
||||
}
|
||||
}
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => "last_updated"
|
||||
}
|
||||
mutate {
|
||||
convert => { "plugin_id" => "integer"}
|
||||
convert => { "id" => "integer"}
|
||||
convert => { "risk_number" => "integer"}
|
||||
convert => { "risk_score" => "float"}
|
||||
convert => { "total_times_detected" => "integer"}
|
||||
convert => { "cvss_temporal" => "float"}
|
||||
convert => { "cvss" => "float"}
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
|
||||
if [asset] =~ "\.yourdomain\.(com|net)$" {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
output {
|
||||
if "qualys" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "vulnwhisp-es1.local:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
146
resources/elk5-old_compatibility/docker/3000_openvas.conf
Normal file
146
resources/elk5-old_compatibility/docker/3000_openvas.conf
Normal file
@ -0,0 +1,146 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: austin@hasecuritysolutions.com
|
||||
# Last Update: 03/04/2018
|
||||
# Version 0.3
|
||||
# Description: Take in Openvas web scan reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/openvas/*.json"
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "openvas_scan", "openvas" ]
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "openvas_scan" in [tags] {
|
||||
mutate {
|
||||
replace => [ "message", "%{message}" ]
|
||||
gsub => [
|
||||
"message", "\|\|\|", " ",
|
||||
"message", "\t\t", " ",
|
||||
"message", " ", " ",
|
||||
"message", " ", " ",
|
||||
"message", " ", " ",
|
||||
"message", "nan", " ",
|
||||
"message",'\n',''
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
grok {
|
||||
match => { "path" => "openvas_scan_%{DATA:scan_id}_%{INT:last_updated}.json$" }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
mutate {
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
|
||||
if [risk] == "1" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
mutate { replace => { "risk" => "info" }}
|
||||
}
|
||||
if [risk] == "2" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
mutate { replace => { "risk" => "low" }}
|
||||
}
|
||||
if [risk] == "3" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
mutate { replace => { "risk" => "medium" }}
|
||||
}
|
||||
if [risk] == "4" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
mutate { replace => { "risk" => "high" }}
|
||||
}
|
||||
if [risk] == "5" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
mutate { replace => { "risk" => "critical" }}
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => "message"
|
||||
}
|
||||
|
||||
if [first_time_detected] {
|
||||
date {
|
||||
match => [ "first_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_detected"
|
||||
}
|
||||
}
|
||||
if [first_time_tested] {
|
||||
date {
|
||||
match => [ "first_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_tested"
|
||||
}
|
||||
}
|
||||
if [last_time_detected] {
|
||||
date {
|
||||
match => [ "last_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_detected"
|
||||
}
|
||||
}
|
||||
if [last_time_tested] {
|
||||
date {
|
||||
match => [ "last_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_tested"
|
||||
}
|
||||
}
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => "last_updated"
|
||||
}
|
||||
mutate {
|
||||
convert => { "plugin_id" => "integer"}
|
||||
convert => { "id" => "integer"}
|
||||
convert => { "risk_number" => "integer"}
|
||||
convert => { "risk_score" => "float"}
|
||||
convert => { "total_times_detected" => "integer"}
|
||||
convert => { "cvss_temporal" => "float"}
|
||||
convert => { "cvss" => "float"}
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
# Add your critical assets by subnet or by hostname. Comment this field out if you don't want to tag any, but the asset panel will break.
|
||||
if [asset] =~ "^10\.0\.100\." {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
output {
|
||||
if "openvas" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "vulnwhisp-es1.local:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
21
resources/elk5-old_compatibility/docker/4000_jira.conf
Executable file
21
resources/elk5-old_compatibility/docker/4000_jira.conf
Executable file
@ -0,0 +1,21 @@
|
||||
# Description: Take in jira tickets from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/Vulnwhisperer/jira/*.json"
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "jira" ]
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
if "jira" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "vulnwhisp-es1.local:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
5
resources/elk5-old_compatibility/docker/logstash.yml
Normal file
5
resources/elk5-old_compatibility/docker/logstash.yml
Normal file
@ -0,0 +1,5 @@
|
||||
path.config: /usr/share/logstash/pipeline/
|
||||
xpack.monitoring.elasticsearch.password: changeme
|
||||
xpack.monitoring.elasticsearch.url: vulnwhisp-es1.local:9200
|
||||
xpack.monitoring.elasticsearch.username: elastic
|
||||
xpack.monitoring.enabled: false
|
||||
@ -0,0 +1,122 @@
|
||||
{
|
||||
"order": 0,
|
||||
"template": "logstash-vulnwhisperer-*",
|
||||
"settings": {
|
||||
"index": {
|
||||
"routing": {
|
||||
"allocation": {
|
||||
"total_shards_per_node": "2"
|
||||
}
|
||||
},
|
||||
"mapping": {
|
||||
"total_fields": {
|
||||
"limit": "3000"
|
||||
}
|
||||
},
|
||||
"refresh_interval": "5s",
|
||||
"number_of_shards": "1",
|
||||
"number_of_replicas": "0"
|
||||
}
|
||||
},
|
||||
"mappings": {
|
||||
"_default_": {
|
||||
"_all": {
|
||||
"enabled": false
|
||||
},
|
||||
"dynamic_templates": [
|
||||
{
|
||||
"message_field": {
|
||||
"path_match": "message",
|
||||
"match_mapping_type": "string",
|
||||
"mapping": {
|
||||
"type": "text",
|
||||
"norms": false
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"string_fields": {
|
||||
"match": "*",
|
||||
"match_mapping_type": "string",
|
||||
"mapping": {
|
||||
"type": "text",
|
||||
"norms": false,
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"plugin_id": {
|
||||
"type": "float"
|
||||
},
|
||||
"last_updated": {
|
||||
"type": "date"
|
||||
},
|
||||
"geoip": {
|
||||
"dynamic": true,
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ip": {
|
||||
"type": "ip"
|
||||
},
|
||||
"latitude": {
|
||||
"type": "float"
|
||||
},
|
||||
"location": {
|
||||
"type": "geo_point"
|
||||
},
|
||||
"longitude": {
|
||||
"type": "float"
|
||||
}
|
||||
}
|
||||
},
|
||||
"risk_score": {
|
||||
"type": "float"
|
||||
},
|
||||
"source": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"synopsis": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"see_also": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"@timestamp": {
|
||||
"type": "date"
|
||||
},
|
||||
"cve": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"solution": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"port": {
|
||||
"type": "integer"
|
||||
},
|
||||
"host": {
|
||||
"type": "text"
|
||||
},
|
||||
"@version": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"risk": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"assign_ip": {
|
||||
"type": "ip"
|
||||
},
|
||||
"cvss": {
|
||||
"type": "float"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"aliases": {}
|
||||
}
|
||||
116
resources/elk5-old_compatibility/filebeat/filebeat.yml
Executable file
116
resources/elk5-old_compatibility/filebeat/filebeat.yml
Executable file
@ -0,0 +1,116 @@
|
||||
###################### Filebeat Configuration Example #########################
|
||||
|
||||
# This file is an example configuration file highlighting only the most common
|
||||
# options. The filebeat.full.yml file from the same directory contains all the
|
||||
# supported options with more comments. You can use it as a reference.
|
||||
#
|
||||
# You can find the full configuration reference here:
|
||||
# https://www.elastic.co/guide/en/beats/filebeat/index.html
|
||||
|
||||
#=========================== Filebeat prospectors =============================
|
||||
|
||||
filebeat.prospectors:
|
||||
|
||||
# Each - is a prospector. Most options can be set at the prospector level, so
|
||||
# you can use different prospectors for various configurations.
|
||||
# Below are the prospector specific configurations.
|
||||
|
||||
- input_type: log
|
||||
# Paths that should be crawled and fetched. Glob based paths.
|
||||
paths:
|
||||
# Linux Example
|
||||
#- /var/log/*.log
|
||||
|
||||
#Windows Example
|
||||
- c:\nessus\My Scans\*
|
||||
|
||||
# Exclude lines. A list of regular expressions to match. It drops the lines that are
|
||||
# matching any regular expression from the list.
|
||||
#exclude_lines: ["^DBG"]
|
||||
|
||||
# Include lines. A list of regular expressions to match. It exports the lines that are
|
||||
# matching any regular expression from the list.
|
||||
#include_lines: ["^ERR", "^WARN"]
|
||||
|
||||
# Exclude files. A list of regular expressions to match. Filebeat drops the files that
|
||||
# are matching any regular expression from the list. By default, no files are dropped.
|
||||
#exclude_files: [".gz$"]
|
||||
|
||||
# Optional additional fields. These field can be freely picked
|
||||
# to add additional information to the crawled log files for filtering
|
||||
#fields:
|
||||
# level: debug
|
||||
# review: 1
|
||||
|
||||
### Multiline options
|
||||
|
||||
# Mutiline can be used for log messages spanning multiple lines. This is common
|
||||
# for Java Stack Traces or C-Line Continuation
|
||||
|
||||
# The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
|
||||
#multiline.pattern: ^\[
|
||||
|
||||
# Defines if the pattern set under pattern should be negated or not. Default is false.
|
||||
#multiline.negate: false
|
||||
|
||||
# Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
|
||||
# that was (not) matched before or after or as long as a pattern is not matched based on negate.
|
||||
# Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
|
||||
#multiline.match: after
|
||||
|
||||
|
||||
#================================ General =====================================
|
||||
|
||||
# The name of the shipper that publishes the network data. It can be used to group
|
||||
# all the transactions sent by a single shipper in the web interface.
|
||||
#name:
|
||||
|
||||
# The tags of the shipper are included in their own field with each
|
||||
# transaction published.
|
||||
#tags: ["service-X", "web-tier"]
|
||||
|
||||
# Optional fields that you can specify to add additional information to the
|
||||
# output.
|
||||
#fields:
|
||||
# env: staging
|
||||
|
||||
#================================ Outputs =====================================
|
||||
|
||||
# Configure what outputs to use when sending the data collected by the beat.
|
||||
# Multiple outputs may be used.
|
||||
|
||||
#-------------------------- Elasticsearch output ------------------------------
|
||||
#output.elasticsearch:
|
||||
# Array of hosts to connect to.
|
||||
# hosts: ["logstash01:9200"]
|
||||
|
||||
# Optional protocol and basic auth credentials.
|
||||
#protocol: "https"
|
||||
#username: "elastic"
|
||||
#password: "changeme"
|
||||
|
||||
#----------------------------- Logstash output --------------------------------
|
||||
output.logstash:
|
||||
# The Logstash hosts
|
||||
hosts: ["logstashserver1:5044", "logstashserver2:5044", "logstashserver3:5044"]
|
||||
|
||||
# Optional SSL. By default is off.
|
||||
# List of root certificates for HTTPS server verifications
|
||||
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
|
||||
|
||||
# Certificate for SSL client authentication
|
||||
#ssl.certificate: "/etc/pki/client/cert.pem"
|
||||
|
||||
# Client Certificate Key
|
||||
#ssl.key: "/etc/pki/client/cert.key"
|
||||
|
||||
#================================ Logging =====================================
|
||||
|
||||
# Sets log level. The default log level is info.
|
||||
# Available log levels are: critical, error, warning, info, debug
|
||||
#logging.level: debug
|
||||
|
||||
# At debug level, you can selectively enable logging only for some components.
|
||||
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
|
||||
# "publish", "service".
|
||||
#logging.selectors: ["*"]
|
||||
@ -0,0 +1,450 @@
|
||||
[
|
||||
{
|
||||
"_id": "80158c90-57c1-11e7-b484-a970fc9d150a",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - HIPAA TL",
|
||||
"visState": "{\"type\":\"timelion\",\"title\":\"VulnWhisperer - HIPAA TL\",\"params\":{\"expression\":\".es(index=logstash-vulnwhisperer-*,q='risk_score:>9 AND tags:pci_asset').label(\\\"PCI Assets\\\"),.es(index=logstash-vulnwhisperer-*,q='risk_score:>9 AND tags:has_hipaa_data').label(\\\"Has HIPAA Data\\\"),.es(index=logstash-vulnwhisperer-*,q='risk_score:>9 AND tags:hipaa_asset').label(\\\"HIPAA Assets\\\")\",\"interval\":\"auto\"}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "479deab0-8a39-11e7-a58a-9bfcb3761a3d",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL - TaggedAssetsPluginNames",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL - TaggedAssetsPluginNames\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*', q='tags:critical_asset OR tags:hipaa_asset OR tags:pci_asset', split=\\\"plugin_name.keyword:10\\\").bars(width=4).label(regex=\\\".*:(.+)>.*\\\",label=\\\"$1\\\")\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "84f5c370-8a38-11e7-a58a-9bfcb3761a3d",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL - CriticalAssetsPluginNames",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL - CriticalAssetsPluginNames\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*', q='tags:critical_asset', split=\\\"plugin_name.keyword:10\\\").bars(width=4).label(regex=\\\".*:(.+)>.*\\\",label=\\\"$1\\\")\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "307cdae0-8a38-11e7-a58a-9bfcb3761a3d",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL - PluginNames",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL - PluginNames\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*', split=\\\"plugin_name.keyword:25\\\").bars(width=4).label(regex=\\\".*:(.+)>.*\\\",label=\\\"$1\\\")\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "5093c620-44e9-11e7-8014-ede06a7e69f8",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Mitigation Readme",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Mitigation Readme\",\"type\":\"markdown\",\"params\":{\"markdown\":\"** Legend **\\n\\n* [Common Vulnerability Scoring System (CVSS)](https://nvd.nist.gov/vuln-metrics/cvss) is the NIST vulnerability scoring system\\n* Risk Number is residual risk score calculated from CVSS, which is adjusted to be specific to the netowrk owner, which accounts for services not in use such as Java and Flash\\n* Vulnerabilities by Tag are systems tagged with HIPAA and PCI identification.\\n\\n\\n** Workflow **\\n* Select 10.0 under Risk Number to identify Critical Vulnerabilities. \\n* For more information about a CVE, scroll down and click the CVE link.\\n* To filter by tags, use one of the following filters:\\n** tags:has_hipaa_data, tags:pci_asset, tags:hipaa_asset, tags:critical_asset**\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "7e7fbc90-3df2-11e7-a44e-c79ca8efb780",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-PluginID",
|
||||
"visState": "{\"title\":\"VulnWhisperer-PluginID\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showMeticsAtAllLevels\":false,\"showPartialRows\":false,\"showTotal\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"plugin_id\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "5a3c0340-3eb3-11e7-a192-93f36fbd9d05",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-CVSSHeatmap",
|
||||
"visState": "{\"title\":\"VulnWhisperer-CVSSHeatmap\",\"type\":\"heatmap\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"enableHover\":false,\"legendPosition\":\"right\",\"times\":[],\"colorsNumber\":4,\"colorSchema\":\"Yellow to Red\",\"setColorRange\":false,\"colorsRange\":[],\"invertColors\":false,\"percentageMode\":false,\"valueAxes\":[{\"show\":false,\"id\":\"ValueAxis-1\",\"type\":\"value\",\"scale\":{\"type\":\"linear\",\"defaultYExtents\":false},\"labels\":{\"show\":false,\"rotate\":0,\"color\":\"#555\"}}]},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"segment\",\"params\":{\"field\":\"host.keyword\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\"}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"cvss.keyword\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"_term\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 3500\":\"rgb(255,255,204)\",\"3500 - 7000\":\"rgb(254,217,118)\",\"7000 - 10500\":\"rgb(253,141,60)\",\"10500 - 14000\":\"rgb(227,27,28)\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "1de9e550-3df1-11e7-a44e-c79ca8efb780",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-Description",
|
||||
"visState": "{\"title\":\"VulnWhisperer-Description\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"description.keyword\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Description\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "13c7d4e0-3df3-11e7-a44e-c79ca8efb780",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-Solution",
|
||||
"visState": "{\"title\":\"VulnWhisperer-Solution\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showMeticsAtAllLevels\":false,\"showPartialRows\":false,\"showTotal\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"solution.keyword\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Solution\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "297df800-3f7e-11e7-bd24-6903e3283192",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Plugin Name",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Plugin Name\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"plugin_name.keyword\",\"size\":10,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Plugin Name\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "de1a5f40-3f85-11e7-97f9-3777d794626d",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - ScanName",
|
||||
"visState": "{\"title\":\"VulnWhisperer - ScanName\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"scan_name.keyword\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Scan Name\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "ecbb99c0-3f84-11e7-97f9-3777d794626d",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Total",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Total\",\"type\":\"metric\",\"params\":{\"handleNoResults\":true,\"fontSize\":60},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Total\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "471a3580-3f6b-11e7-88e7-df1abe6547fb",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Vulnerabilities by Tag",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Vulnerabilities by Tag\",\"type\":\"table\",\"params\":{\"perPage\":3,\"showMeticsAtAllLevels\":false,\"showPartialRows\":false,\"showTotal\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"3\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"bucket\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"tags:has_hipaa_data\",\"analyze_wildcard\":true}}},\"label\":\"Systems with HIPAA data\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"tags:pci_asset\",\"analyze_wildcard\":true}}},\"label\":\"PCI Systems\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"tags:hipaa_asset\",\"analyze_wildcard\":true}}},\"label\":\"HIPAA Systems\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "35b6d320-3f7f-11e7-bd24-6903e3283192",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Residual Risk",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Residual Risk\",\"type\":\"table\",\"params\":{\"perPage\":15,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":0,\"direction\":\"desc\"},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"risk_score\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Risk Number\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"desc\"}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "a9225930-3df2-11e7-a44e-c79ca8efb780",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-Risk",
|
||||
"visState": "{\"title\":\"VulnWhisperer-Risk\",\"type\":\"table\",\"params\":{\"perPage\":4,\"showMeticsAtAllLevels\":false,\"showPartialRows\":false,\"showTotal\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"risk\",\"size\":10,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Risk Severity\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "2f979030-44b9-11e7-a818-f5f80dfc3590",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - ScanBarChart",
|
||||
"visState": "{\"aggs\":[{\"enabled\":true,\"id\":\"1\",\"params\":{},\"schema\":\"metric\",\"type\":\"count\"},{\"enabled\":true,\"id\":\"2\",\"params\":{\"customLabel\":\"Scan Name\",\"field\":\"plugin_name.keyword\",\"order\":\"desc\",\"orderBy\":\"1\",\"size\":10},\"schema\":\"segment\",\"type\":\"terms\"}],\"listeners\":{},\"params\":{\"addLegend\":true,\"addTimeMarker\":false,\"addTooltip\":true,\"defaultYExtents\":false,\"legendPosition\":\"right\",\"mode\":\"stacked\",\"scale\":\"linear\",\"setYExtents\":false,\"times\":[]},\"title\":\"VulnWhisperer - ScanBarChart\",\"type\":\"histogram\"}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "a6508640-897a-11e7-bbc0-33592ce0be1e",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Critical Assets Aggregated",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Critical Assets Aggregated\",\"type\":\"heatmap\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"enableHover\":true,\"legendPosition\":\"right\",\"times\":[],\"colorsNumber\":4,\"colorSchema\":\"Green to Red\",\"setColorRange\":true,\"colorsRange\":[{\"from\":0,\"to\":3},{\"from\":3,\"to\":7},{\"from\":7,\"to\":9},{\"from\":9,\"to\":11}],\"invertColors\":false,\"percentageMode\":false,\"valueAxes\":[{\"show\":false,\"id\":\"ValueAxis-1\",\"type\":\"value\",\"scale\":{\"type\":\"linear\",\"defaultYExtents\":false},\"labels\":{\"show\":true,\"rotate\":0,\"color\":\"white\"}}]},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"max\",\"schema\":\"metric\",\"params\":{\"field\":\"risk_score\",\"customLabel\":\"Residual Risk Score\"}},{\"id\":\"3\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{},\"customLabel\":\"Date\"}},{\"id\":\"4\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"host\",\"size\":10,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Critical Asset IP\"}},{\"id\":\"5\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"split\",\"params\":{\"field\":\"plugin_name.keyword\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"row\":true}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"colors\":{\"0 - 3\":\"#7EB26D\",\"3 - 7\":\"#EAB839\",\"7 - 9\":\"#EF843C\",\"8 - 10\":\"#BF1B00\",\"9 - 11\":\"#BF1B00\"},\"defaultColors\":{\"0 - 3\":\"rgb(0,104,55)\",\"3 - 7\":\"rgb(135,203,103)\",\"7 - 9\":\"rgb(255,255,190)\",\"9 - 11\":\"rgb(249,142,82)\"},\"legendOpen\":false}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[{\"$state\":{\"store\":\"appState\"},\"meta\":{\"alias\":\"Critical Asset\",\"disabled\":false,\"index\":\"logstash-vulnwhisperer-*\",\"key\":\"tags\",\"negate\":false,\"type\":\"phrase\",\"value\":\"critical_asset\"},\"query\":{\"match\":{\"tags\":{\"query\":\"critical_asset\",\"type\":\"phrase\"}}}}]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "099a3820-3f68-11e7-a6bd-e764d950e506",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "Timelion VulnWhisperer Example",
|
||||
"visState": "{\"type\":\"timelion\",\"title\":\"Timelion VulnWhisperer Example\",\"params\":{\"expression\":\".es(index=logstash-vulnwhisperer-*,q=risk:high).label(\\\"Current High Risk\\\"),.es(index=logstash-vulnwhisperer-*,q=risk:high,offset=-1y).label(\\\"Last 1 Year High Risk\\\"),.es(index=logstash-vulnwhisperer-*,q=risk:medium).label(\\\"Current Medium Risk\\\"),.es(index=logstash-vulnwhisperer-*,q=risk:medium,offset=-1y).label(\\\"Last 1 Year Medium Risk\\\")\",\"interval\":\"auto\"}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "67d432e0-44ec-11e7-a05f-d9719b331a27",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL-Critical Risk",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL-Critical Risk\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*',q='(risk_score:>=9 AND risk_score:<=10)').label(\\\"Original\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=9 AND risk_score:<=10)',offset=-1w).label(\\\"One week offset\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=9 AND risk_score:<=10)').subtract(.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=9 AND risk_score:<=10)',offset=-1w)).label(\\\"Difference\\\").lines(steps=3,fill=2,width=1)\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "a91b9fe0-44ec-11e7-a05f-d9719b331a27",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL-Medium Risk",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL-Medium Risk\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*',q='(risk_score:>=4 AND risk_score:<7)').label(\\\"Original\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=4 AND risk_score:<7)',offset=-1w).label(\\\"One week offset\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=4 AND risk_score:<7)').subtract(.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=4 AND risk_score:<7)',offset=-1w)).label(\\\"Difference\\\").lines(steps=3,fill=2,width=1)\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "8d9592d0-44ec-11e7-a05f-d9719b331a27",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL-High Risk",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL-High Risk\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*',q='(risk_score:>=7 AND risk_score:<9)').label(\\\"Original\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=7 AND risk_score:<9)',offset=-1w).label(\\\"One week offset\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=7 AND risk_score:<9)').subtract(.es(index='logstash-vulnwhisperer-*',q='(risk_score:>=7 AND risk_score:<9)',offset=-1w)).label(\\\"Difference\\\").lines(steps=3,fill=2,width=1)\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "a2d66660-44ec-11e7-a05f-d9719b331a27",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL-Low Risk",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL-Low Risk\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*',q='(risk_score:>0 AND risk_score:<4)').label(\\\"Original\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>0 AND risk_score:<4)',offset=-1w).label(\\\"One week offset\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk_score:>0 AND risk_score:<4)').subtract(.es(index='logstash-vulnwhisperer-*',q='(risk_score:>0 AND risk_score:<4)',offset=-1w)).label(\\\"Difference\\\").lines(steps=3,fill=2,width=1)\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "fb6eb020-49ab-11e7-8f8c-57ad64ec48a6",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Critical Risk Score for Tagged Assets",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Critical Risk Score for Tagged Assets\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index=logstash-vulnwhisperer-*,q='risk_score:>9 AND tags:hipaa_asset').label(\\\"HIPAA Assets\\\"),.es(index=logstash-vulnwhisperer-*,q='risk_score:>9 AND tags:pci_asset').label(\\\"PCI Systems\\\"),.es(index=logstash-vulnwhisperer-*,q='risk_score:>9 AND tags:has_hipaa_data').label(\\\"Has HIPAA Data\\\")\",\"interval\":\"auto\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "b2f2adb0-897f-11e7-a2d2-c57bca21b3aa",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: Total",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: Total\",\"type\":\"goal\",\"params\":{\"addLegend\":true,\"addTooltip\":true,\"gauge\":{\"autoExtend\":false,\"backStyle\":\"Full\",\"colorSchema\":\"Green to Red\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"gaugeColorMode\":\"Background\",\"gaugeStyle\":\"Full\",\"gaugeType\":\"Metric\",\"invertColors\":false,\"labels\":{\"color\":\"black\",\"show\":false},\"orientation\":\"vertical\",\"percentageMode\":false,\"scale\":{\"color\":\"#333\",\"labels\":false,\"show\":true,\"width\":2},\"style\":{\"bgColor\":true,\"bgFill\":\"white\",\"fontSize\":\"34\",\"labelColor\":false,\"subText\":\"Risk\"},\"type\":\"simple\",\"useRanges\":false,\"verticalSplit\":false},\"type\":\"gauge\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Total\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}}},\"label\":\"Critical\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"colors\":{\"0 - 10000\":\"#64B0C8\"},\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "465c5820-8977-11e7-857e-e1d56b17746d",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Critical Assets",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Critical Assets\",\"type\":\"heatmap\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"enableHover\":true,\"legendPosition\":\"right\",\"times\":[],\"colorsNumber\":4,\"colorSchema\":\"Green to Red\",\"setColorRange\":true,\"colorsRange\":[{\"from\":0,\"to\":3},{\"from\":3,\"to\":7},{\"from\":7,\"to\":9},{\"from\":9,\"to\":11}],\"invertColors\":false,\"percentageMode\":false,\"valueAxes\":[{\"show\":false,\"id\":\"ValueAxis-1\",\"type\":\"value\",\"scale\":{\"type\":\"linear\",\"defaultYExtents\":false},\"labels\":{\"show\":false,\"rotate\":0,\"color\":\"white\"}}],\"type\":\"heatmap\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"max\",\"schema\":\"metric\",\"params\":{\"field\":\"risk_score\",\"customLabel\":\"Residual Risk Score\"}},{\"id\":\"2\",\"enabled\":false,\"type\":\"terms\",\"schema\":\"split\",\"params\":{\"field\":\"risk_score\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"row\":true}},{\"id\":\"3\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{},\"customLabel\":\"Date\"}},{\"id\":\"4\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"asset.keyword\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Critical Asset\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 3\":\"rgb(0,104,55)\",\"3 - 7\":\"rgb(135,203,103)\",\"7 - 9\":\"rgb(255,255,190)\",\"9 - 11\":\"rgb(249,142,82)\"},\"colors\":{\"8 - 10\":\"#BF1B00\",\"9 - 11\":\"#BF1B00\",\"7 - 9\":\"#EF843C\",\"3 - 7\":\"#EAB839\",\"0 - 3\":\"#7EB26D\"},\"legendOpen\":false}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[{\"meta\":{\"index\":\"logstash-vulnwhisperer-*\",\"negate\":false,\"disabled\":false,\"alias\":\"Critical Asset\",\"type\":\"phrase\",\"key\":\"tags\",\"value\":\"critical_asset\"},\"query\":{\"match\":{\"tags\":{\"query\":\"critical_asset\",\"type\":\"phrase\"}}},\"$state\":{\"store\":\"appState\"}}]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "852816e0-3eb1-11e7-90cb-918f9cb01e3d",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-CVSS",
|
||||
"visState": "{\"title\":\"VulnWhisperer-CVSS\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showMeticsAtAllLevels\":false,\"showPartialRows\":false,\"showTotal\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"totalFunc\":\"sum\",\"type\":\"table\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"cvss.keyword\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"CVSS Score\"}},{\"id\":\"4\",\"enabled\":true,\"type\":\"cardinality\",\"schema\":\"metric\",\"params\":{\"field\":\"asset.keyword\",\"customLabel\":\"# of Assets\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"desc\"}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "d048c220-80b3-11e7-8790-73b60225f736",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: High",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: High\",\"type\":\"goal\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"type\":\"gauge\",\"gauge\":{\"verticalSplit\":false,\"autoExtend\":false,\"percentageMode\":false,\"gaugeType\":\"Metric\",\"gaugeStyle\":\"Full\",\"backStyle\":\"Full\",\"orientation\":\"vertical\",\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"gaugeColorMode\":\"Background\",\"colorsRange\":[{\"from\":0,\"to\":1000}],\"invertColors\":false,\"labels\":{\"show\":false,\"color\":\"black\"},\"scale\":{\"show\":true,\"labels\":false,\"color\":\"#333\",\"width\":2},\"type\":\"simple\",\"style\":{\"bgFill\":\"white\",\"bgColor\":true,\"labelColor\":false,\"subText\":\"\",\"fontSize\":\"34\"},\"extendRange\":true}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"High Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:high\"}}},\"label\":\"\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"},\"legendOpen\":true,\"colors\":{\"0 - 10000\":\"#EF843C\",\"0 - 1000\":\"#E0752D\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "db55bce0-80b3-11e7-8790-73b60225f736",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: Critical",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: Critical\",\"type\":\"goal\",\"params\":{\"addLegend\":true,\"addTooltip\":true,\"gauge\":{\"autoExtend\":false,\"backStyle\":\"Full\",\"colorSchema\":\"Green to Red\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"gaugeColorMode\":\"Background\",\"gaugeStyle\":\"Full\",\"gaugeType\":\"Metric\",\"invertColors\":false,\"labels\":{\"color\":\"black\",\"show\":false},\"orientation\":\"vertical\",\"percentageMode\":false,\"scale\":{\"color\":\"#333\",\"labels\":false,\"show\":true,\"width\":2},\"style\":{\"bgColor\":true,\"bgFill\":\"white\",\"fontSize\":\"34\",\"labelColor\":false,\"subText\":\"Risk\"},\"type\":\"simple\",\"useRanges\":false,\"verticalSplit\":false},\"type\":\"gauge\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Critical Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:critical\"}}},\"label\":\"Critical\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"colors\":{\"0 - 10000\":\"#BF1B00\"},\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "56f0f5f0-3ebe-11e7-a192-93f36fbd9d05",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-RiskOverTime",
|
||||
"visState": "{\"title\":\"VulnWhisperer-RiskOverTime\",\"type\":\"line\",\"params\":{\"addLegend\":true,\"addTimeMarker\":false,\"addTooltip\":true,\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"labels\":{\"show\":true,\"truncate\":100},\"position\":\"bottom\",\"scale\":{\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"@timestamp per 12 hours\"},\"type\":\"category\"}],\"defaultYExtents\":false,\"drawLinesBetweenPoints\":true,\"grid\":{\"categoryLines\":false,\"style\":{\"color\":\"#eee\"},\"valueAxis\":\"ValueAxis-1\"},\"interpolate\":\"linear\",\"legendPosition\":\"right\",\"orderBucketsBySum\":false,\"radiusRatio\":9,\"scale\":\"linear\",\"seriesParams\":[{\"data\":{\"id\":\"1\",\"label\":\"Count\"},\"drawLinesBetweenPoints\":true,\"interpolate\":\"linear\",\"mode\":\"normal\",\"show\":\"true\",\"showCircles\":true,\"type\":\"line\",\"valueAxis\":\"ValueAxis-1\"}],\"setYExtents\":false,\"showCircles\":true,\"times\":[],\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"labels\":{\"filter\":false,\"rotate\":0,\"show\":true,\"truncate\":100},\"name\":\"LeftAxis-1\",\"position\":\"left\",\"scale\":{\"mode\":\"normal\",\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"Count\"},\"type\":\"value\"}],\"type\":\"line\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{}}},{\"id\":\"3\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:info\"}}},\"label\":\"Info\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:low\"}}},\"label\":\"Low\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:medium\"}}},\"label\":\"Medium\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:high\"}}},\"label\":\"High\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:critical\"}}},\"label\":\"Critical\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"colors\":{\"Critical\":\"#962D82\",\"High\":\"#BF1B00\",\"Low\":\"#629E51\",\"Medium\":\"#EAB839\",\"Info\":\"#65C5DB\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "c1361da0-80b3-11e7-8790-73b60225f736",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: Medium",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: Medium\",\"type\":\"goal\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"type\":\"gauge\",\"gauge\":{\"verticalSplit\":false,\"autoExtend\":false,\"percentageMode\":false,\"gaugeType\":\"Metric\",\"gaugeStyle\":\"Full\",\"backStyle\":\"Full\",\"orientation\":\"vertical\",\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"gaugeColorMode\":\"Background\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"invertColors\":false,\"labels\":{\"show\":false,\"color\":\"black\"},\"scale\":{\"show\":true,\"labels\":false,\"color\":\"#333\",\"width\":2},\"type\":\"simple\",\"style\":{\"bgFill\":\"white\",\"bgColor\":true,\"labelColor\":false,\"subText\":\"\",\"fontSize\":\"34\"},\"extendRange\":false},\"isDisplayWarning\":false},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Medium Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:medium\"}}},\"label\":\"Medium Risk\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":true,\"colors\":{\"0 - 10000\":\"#EAB839\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "e46ff7f0-897d-11e7-934b-67cec0a7da65",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: Low",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: Low\",\"type\":\"goal\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"type\":\"gauge\",\"gauge\":{\"verticalSplit\":false,\"autoExtend\":false,\"percentageMode\":false,\"gaugeType\":\"Metric\",\"gaugeStyle\":\"Full\",\"backStyle\":\"Full\",\"orientation\":\"vertical\",\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"gaugeColorMode\":\"Background\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"invertColors\":false,\"labels\":{\"show\":false,\"color\":\"black\"},\"scale\":{\"show\":true,\"labels\":false,\"color\":\"#333\",\"width\":2},\"type\":\"simple\",\"style\":{\"bgFill\":\"white\",\"bgColor\":true,\"labelColor\":false,\"subText\":\"\",\"fontSize\":\"34\"},\"extendRange\":false}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Low Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk_score_name:low\"}}},\"label\":\"Low Risk\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":true,\"colors\":{\"0 - 10000\":\"#629E51\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "995e2280-3df3-11e7-a44e-c79ca8efb780",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-Asset",
|
||||
"visState": "{\"title\":\"VulnWhisperer-Asset\",\"type\":\"table\",\"params\":{\"perPage\":15,\"showMeticsAtAllLevels\":false,\"showPartialRows\":false,\"showTotal\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"totalFunc\":\"sum\",\"type\":\"table\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"asset.keyword\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Asset\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
@ -0,0 +1,43 @@
|
||||
[
|
||||
{
|
||||
"_id": "72051530-448e-11e7-a818-f5f80dfc3590",
|
||||
"_type": "dashboard",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Reporting",
|
||||
"hits": 0,
|
||||
"description": "",
|
||||
"panelsJSON": "[{\"col\":1,\"id\":\"2f979030-44b9-11e7-a818-f5f80dfc3590\",\"panelIndex\":5,\"row\":12,\"size_x\":6,\"size_y\":4,\"type\":\"visualization\"},{\"col\":1,\"id\":\"8d9592d0-44ec-11e7-a05f-d9719b331a27\",\"panelIndex\":12,\"row\":8,\"size_x\":6,\"size_y\":4,\"type\":\"visualization\"},{\"col\":7,\"id\":\"67d432e0-44ec-11e7-a05f-d9719b331a27\",\"panelIndex\":14,\"row\":4,\"size_x\":6,\"size_y\":4,\"type\":\"visualization\"},{\"col\":10,\"id\":\"297df800-3f7e-11e7-bd24-6903e3283192\",\"panelIndex\":15,\"row\":8,\"size_x\":3,\"size_y\":4,\"type\":\"visualization\"},{\"col\":7,\"id\":\"471a3580-3f6b-11e7-88e7-df1abe6547fb\",\"panelIndex\":20,\"row\":8,\"size_x\":3,\"size_y\":4,\"type\":\"visualization\"},{\"col\":11,\"id\":\"995e2280-3df3-11e7-a44e-c79ca8efb780\",\"panelIndex\":22,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":9,\"id\":\"b2f2adb0-897f-11e7-a2d2-c57bca21b3aa\",\"panelIndex\":23,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":7,\"id\":\"db55bce0-80b3-11e7-8790-73b60225f736\",\"panelIndex\":25,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":5,\"id\":\"d048c220-80b3-11e7-8790-73b60225f736\",\"panelIndex\":26,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":1,\"id\":\"e46ff7f0-897d-11e7-934b-67cec0a7da65\",\"panelIndex\":27,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":3,\"id\":\"c1361da0-80b3-11e7-8790-73b60225f736\",\"panelIndex\":28,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":1,\"id\":\"479deab0-8a39-11e7-a58a-9bfcb3761a3d\",\"panelIndex\":29,\"row\":4,\"size_x\":6,\"size_y\":4,\"type\":\"visualization\"}]",
|
||||
"optionsJSON": "{\"darkTheme\":false}",
|
||||
"uiStateJSON": "{\"P-15\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-20\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-21\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-22\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-23\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-24\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-25\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-26\":{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-27\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-28\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-5\":{\"vis\":{\"legendOpen\":false}}}",
|
||||
"version": 1,
|
||||
"timeRestore": true,
|
||||
"timeTo": "now",
|
||||
"timeFrom": "now-1y",
|
||||
"refreshInterval": {
|
||||
"display": "Off",
|
||||
"pause": false,
|
||||
"value": 0
|
||||
},
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"filter\":[{\"query\":{\"match_all\":{}}}],\"highlightAll\":true,\"version\":true}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCUqesWib22Ai8JwW3u",
|
||||
"_type": "dashboard",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk Mitigation",
|
||||
"hits": 0,
|
||||
"description": "",
|
||||
"panelsJSON": "[{\"col\":11,\"id\":\"995e2280-3df3-11e7-a44e-c79ca8efb780\",\"panelIndex\":20,\"row\":8,\"size_x\":2,\"size_y\":6,\"type\":\"visualization\"},{\"col\":1,\"id\":\"852816e0-3eb1-11e7-90cb-918f9cb01e3d\",\"panelIndex\":21,\"row\":10,\"size_x\":3,\"size_y\":5,\"type\":\"visualization\"},{\"col\":4,\"id\":\"297df800-3f7e-11e7-bd24-6903e3283192\",\"panelIndex\":27,\"row\":8,\"size_x\":3,\"size_y\":5,\"type\":\"visualization\"},{\"col\":9,\"id\":\"35b6d320-3f7f-11e7-bd24-6903e3283192\",\"panelIndex\":28,\"row\":8,\"size_x\":2,\"size_y\":6,\"type\":\"visualization\"},{\"col\":11,\"id\":\"471a3580-3f6b-11e7-88e7-df1abe6547fb\",\"panelIndex\":30,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":7,\"id\":\"de1a5f40-3f85-11e7-97f9-3777d794626d\",\"panelIndex\":31,\"row\":8,\"size_x\":2,\"size_y\":5,\"type\":\"visualization\"},{\"col\":10,\"id\":\"5093c620-44e9-11e7-8014-ede06a7e69f8\",\"panelIndex\":37,\"row\":4,\"size_x\":3,\"size_y\":4,\"type\":\"visualization\"},{\"col\":1,\"columns\":[\"host\",\"risk\",\"risk_score\",\"cve\",\"plugin_name\",\"solution\",\"plugin_output\"],\"id\":\"54648700-3f74-11e7-852e-69207a3d0726\",\"panelIndex\":38,\"row\":15,\"size_x\":12,\"size_y\":6,\"sort\":[\"@timestamp\",\"desc\"],\"type\":\"search\"},{\"col\":1,\"id\":\"fb6eb020-49ab-11e7-8f8c-57ad64ec48a6\",\"panelIndex\":39,\"row\":8,\"size_x\":3,\"size_y\":2,\"type\":\"visualization\"},{\"col\":5,\"id\":\"465c5820-8977-11e7-857e-e1d56b17746d\",\"panelIndex\":40,\"row\":4,\"size_x\":5,\"size_y\":4,\"type\":\"visualization\"},{\"col\":1,\"id\":\"56f0f5f0-3ebe-11e7-a192-93f36fbd9d05\",\"panelIndex\":46,\"row\":4,\"size_x\":4,\"size_y\":4,\"type\":\"visualization\"},{\"col\":1,\"id\":\"e46ff7f0-897d-11e7-934b-67cec0a7da65\",\"panelIndex\":47,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":3,\"id\":\"c1361da0-80b3-11e7-8790-73b60225f736\",\"panelIndex\":48,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":5,\"id\":\"d048c220-80b3-11e7-8790-73b60225f736\",\"panelIndex\":49,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":7,\"id\":\"db55bce0-80b3-11e7-8790-73b60225f736\",\"panelIndex\":50,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":9,\"id\":\"b2f2adb0-897f-11e7-a2d2-c57bca21b3aa\",\"panelIndex\":51,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"}]",
|
||||
"optionsJSON": "{\"darkTheme\":false}",
|
||||
"uiStateJSON": "{\"P-11\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-2\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-20\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-21\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"desc\"}}}},\"P-27\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-28\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"desc\"}}}},\"P-3\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"asc\"}}}},\"P-30\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-31\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-40\":{\"vis\":{\"defaultColors\":{\"0 - 3\":\"rgb(0,104,55)\",\"3 - 7\":\"rgb(135,203,103)\",\"7 - 9\":\"rgb(255,255,190)\",\"9 - 11\":\"rgb(249,142,82)\"}}},\"P-41\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-42\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-43\":{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"}}},\"P-44\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-45\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-46\":{\"vis\":{\"legendOpen\":true}},\"P-47\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-48\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-49\":{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-5\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-50\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-51\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-6\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-8\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}}",
|
||||
"version": 1,
|
||||
"timeRestore": false,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"filter\":[{\"query\":{\"match_all\":{}}}],\"highlightAll\":true,\"version\":true}"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
@ -0,0 +1,170 @@
|
||||
[
|
||||
{
|
||||
"_id": "AWCUo-jRib22Ai8JwW1N",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: High Qualys Scoring",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: High Qualys Scoring\",\"type\":\"goal\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"type\":\"gauge\",\"gauge\":{\"verticalSplit\":false,\"autoExtend\":false,\"percentageMode\":false,\"gaugeType\":\"Metric\",\"gaugeStyle\":\"Full\",\"backStyle\":\"Full\",\"orientation\":\"vertical\",\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"gaugeColorMode\":\"Background\",\"colorsRange\":[{\"from\":0,\"to\":1000}],\"invertColors\":false,\"labels\":{\"show\":false,\"color\":\"black\"},\"scale\":{\"show\":true,\"labels\":false,\"color\":\"#333\",\"width\":2},\"type\":\"simple\",\"style\":{\"bgFill\":\"white\",\"bgColor\":true,\"labelColor\":false,\"subText\":\"\",\"fontSize\":\"34\"},\"extendRange\":true}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"High Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:high\"}}},\"label\":\"\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"},\"legendOpen\":true,\"colors\":{\"0 - 10000\":\"#EF843C\",\"0 - 1000\":\"#E0752D\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCUozGBib22Ai8JwW1B",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: Medium Qualys Scoring",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: Medium Qualys Scoring\",\"type\":\"goal\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"type\":\"gauge\",\"gauge\":{\"verticalSplit\":false,\"autoExtend\":false,\"percentageMode\":false,\"gaugeType\":\"Metric\",\"gaugeStyle\":\"Full\",\"backStyle\":\"Full\",\"orientation\":\"vertical\",\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"gaugeColorMode\":\"Background\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"invertColors\":false,\"labels\":{\"show\":false,\"color\":\"black\"},\"scale\":{\"show\":true,\"labels\":false,\"color\":\"#333\",\"width\":2},\"type\":\"simple\",\"style\":{\"bgFill\":\"white\",\"bgColor\":true,\"labelColor\":false,\"subText\":\"\",\"fontSize\":\"34\"},\"extendRange\":false}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Medium Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:medium\"}}},\"label\":\"Medium Risk\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":true,\"colors\":{\"0 - 10000\":\"#EAB839\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCUpE3Kib22Ai8JwW1c",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: Critical Qualys Scoring",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: Critical Qualys Scoring\",\"type\":\"goal\",\"params\":{\"addLegend\":true,\"addTooltip\":true,\"gauge\":{\"autoExtend\":false,\"backStyle\":\"Full\",\"colorSchema\":\"Green to Red\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"gaugeColorMode\":\"Background\",\"gaugeStyle\":\"Full\",\"gaugeType\":\"Metric\",\"invertColors\":false,\"labels\":{\"color\":\"black\",\"show\":false},\"orientation\":\"vertical\",\"percentageMode\":false,\"scale\":{\"color\":\"#333\",\"labels\":false,\"show\":true,\"width\":2},\"style\":{\"bgColor\":true,\"bgFill\":\"white\",\"fontSize\":\"34\",\"labelColor\":false,\"subText\":\"Risk\"},\"type\":\"simple\",\"useRanges\":false,\"verticalSplit\":false},\"type\":\"gauge\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Critical Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:critical\"}}},\"label\":\"Critical\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"colors\":{\"0 - 10000\":\"#BF1B00\"},\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCUyeHGib22Ai8JwX62",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer-RiskOverTime Qualys Scoring",
|
||||
"visState": "{\"title\":\"VulnWhisperer-RiskOverTime Qualys Scoring\",\"type\":\"line\",\"params\":{\"addLegend\":true,\"addTimeMarker\":false,\"addTooltip\":true,\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"labels\":{\"show\":true,\"truncate\":100},\"position\":\"bottom\",\"scale\":{\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"@timestamp per 12 hours\"},\"type\":\"category\"}],\"defaultYExtents\":false,\"drawLinesBetweenPoints\":true,\"grid\":{\"categoryLines\":false,\"style\":{\"color\":\"#eee\"},\"valueAxis\":\"ValueAxis-1\"},\"interpolate\":\"linear\",\"legendPosition\":\"right\",\"orderBucketsBySum\":false,\"radiusRatio\":9,\"scale\":\"linear\",\"seriesParams\":[{\"data\":{\"id\":\"1\",\"label\":\"Count\"},\"drawLinesBetweenPoints\":true,\"interpolate\":\"linear\",\"mode\":\"normal\",\"show\":\"true\",\"showCircles\":true,\"type\":\"line\",\"valueAxis\":\"ValueAxis-1\"}],\"setYExtents\":false,\"showCircles\":true,\"times\":[],\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"labels\":{\"filter\":false,\"rotate\":0,\"show\":true,\"truncate\":100},\"name\":\"LeftAxis-1\",\"position\":\"left\",\"scale\":{\"mode\":\"normal\",\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"Count\"},\"type\":\"value\"}],\"type\":\"line\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{}}},{\"id\":\"3\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:info\"}}},\"label\":\"Info\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:low\"}}},\"label\":\"Low\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:medium\"}}},\"label\":\"Medium\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:high\"}}},\"label\":\"High\"},{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:critical\"}}},\"label\":\"Critical\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"colors\":{\"Critical\":\"#962D82\",\"High\":\"#BF1B00\",\"Low\":\"#629E51\",\"Medium\":\"#EAB839\",\"Info\":\"#65C5DB\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCUos-Fib22Ai8JwW0y",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk: Low Qualys Scoring",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Risk: Low Qualys Scoring\",\"type\":\"goal\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"type\":\"gauge\",\"gauge\":{\"verticalSplit\":false,\"autoExtend\":false,\"percentageMode\":false,\"gaugeType\":\"Metric\",\"gaugeStyle\":\"Full\",\"backStyle\":\"Full\",\"orientation\":\"vertical\",\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"gaugeColorMode\":\"Background\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"invertColors\":false,\"labels\":{\"show\":false,\"color\":\"black\"},\"scale\":{\"show\":true,\"labels\":false,\"color\":\"#333\",\"width\":2},\"type\":\"simple\",\"style\":{\"bgFill\":\"white\",\"bgColor\":true,\"labelColor\":false,\"subText\":\"\",\"fontSize\":\"34\"},\"extendRange\":false}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{\"customLabel\":\"Low Risk\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"filters\",\"schema\":\"group\",\"params\":{\"filters\":[{\"input\":{\"query\":{\"query_string\":{\"query\":\"risk:low\"}}},\"label\":\"Low Risk\"}]}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":true,\"colors\":{\"0 - 10000\":\"#629E51\"}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCg9Wsfib22Ai8Jww3v",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Qualys: Category Description",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Qualys: Category Description\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\",\"type\":\"table\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"category_description.keyword\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Category Description\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"match_all\":{}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCg88f1ib22Ai8Jww3C",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - QualysOS",
|
||||
"visState": "{\"title\":\"VulnWhisperer - QualysOS\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\",\"type\":\"table\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"operating_system.keyword\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"match_all\":{}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCg9JUAib22Ai8Jww3Y",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - QualysOwner",
|
||||
"visState": "{\"title\":\"VulnWhisperer - QualysOwner\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\",\"type\":\"table\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"owner.keyword\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"match_all\":{}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCg9tE6ib22Ai8Jww4R",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Qualys: Impact",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Qualys: Impact\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\",\"type\":\"table\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"impact.keyword\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Impact\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"match_all\":{}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCg9igvib22Ai8Jww36",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Qualys: Level",
|
||||
"visState": "{\"title\":\"VulnWhisperer - Qualys: Level\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\",\"type\":\"table\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"level.keyword\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Level\"}}],\"listeners\":{}}",
|
||||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"match_all\":{}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCUsp_3ib22Ai8JwW7R",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL-Critical Risk Qualys Scoring",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL-Critical Risk Qualys Scoring\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*',q='(risk:critical)').label(\\\"Original\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk:critical)',offset=-1w).label(\\\"One week offset\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk:critical)').subtract(.es(index='logstash-vulnwhisperer-*',q='(risk:critical)',offset=-1w)).label(\\\"Difference\\\").lines(steps=3,fill=2,width=1)\",\"interval\":\"auto\",\"type\":\"timelion\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "AWCUtHETib22Ai8JwW79",
|
||||
"_type": "visualization",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - TL-High Risk Qualys Scoring",
|
||||
"visState": "{\"title\":\"VulnWhisperer - TL-High Risk Qualys Scoring\",\"type\":\"timelion\",\"params\":{\"expression\":\".es(index='logstash-vulnwhisperer-*',q='(risk:high)').label(\\\"Original\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk:high)',offset=-1w).label(\\\"One week offset\\\"),.es(index='logstash-vulnwhisperer-*',q='(risk:high)').subtract(.es(index='logstash-vulnwhisperer-*',q='(risk:high)',offset=-1w)).label(\\\"Difference\\\").lines(steps=3,fill=2,width=1)\",\"interval\":\"auto\",\"type\":\"timelion\"},\"aggs\":[],\"listeners\":{}}",
|
||||
"uiStateJSON": "{}",
|
||||
"description": "",
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
@ -0,0 +1,50 @@
|
||||
[
|
||||
{
|
||||
"_id": "AWCUrIBqib22Ai8JwW43",
|
||||
"_type": "dashboard",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Reporting Qualys Scoring",
|
||||
"hits": 0,
|
||||
"description": "",
|
||||
"panelsJSON": "[{\"col\":1,\"id\":\"2f979030-44b9-11e7-a818-f5f80dfc3590\",\"panelIndex\":5,\"row\":11,\"size_x\":6,\"size_y\":4,\"type\":\"visualization\"},{\"col\":10,\"id\":\"297df800-3f7e-11e7-bd24-6903e3283192\",\"panelIndex\":15,\"row\":7,\"size_x\":3,\"size_y\":4,\"type\":\"visualization\"},{\"col\":7,\"id\":\"471a3580-3f6b-11e7-88e7-df1abe6547fb\",\"panelIndex\":20,\"row\":7,\"size_x\":3,\"size_y\":4,\"type\":\"visualization\"},{\"col\":11,\"id\":\"995e2280-3df3-11e7-a44e-c79ca8efb780\",\"panelIndex\":22,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":9,\"id\":\"b2f2adb0-897f-11e7-a2d2-c57bca21b3aa\",\"panelIndex\":23,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":1,\"id\":\"479deab0-8a39-11e7-a58a-9bfcb3761a3d\",\"panelIndex\":29,\"row\":4,\"size_x\":6,\"size_y\":4,\"type\":\"visualization\"},{\"size_x\":6,\"size_y\":3,\"panelIndex\":30,\"type\":\"visualization\",\"id\":\"AWCUtHETib22Ai8JwW79\",\"col\":1,\"row\":8},{\"size_x\":6,\"size_y\":3,\"panelIndex\":31,\"type\":\"visualization\",\"id\":\"AWCUsp_3ib22Ai8JwW7R\",\"col\":7,\"row\":4},{\"size_x\":2,\"size_y\":3,\"panelIndex\":33,\"type\":\"visualization\",\"id\":\"AWCUozGBib22Ai8JwW1B\",\"col\":3,\"row\":1},{\"size_x\":2,\"size_y\":3,\"panelIndex\":34,\"type\":\"visualization\",\"id\":\"AWCUo-jRib22Ai8JwW1N\",\"col\":5,\"row\":1},{\"size_x\":2,\"size_y\":3,\"panelIndex\":35,\"type\":\"visualization\",\"id\":\"AWCUpE3Kib22Ai8JwW1c\",\"col\":7,\"row\":1},{\"size_x\":2,\"size_y\":3,\"panelIndex\":36,\"type\":\"visualization\",\"id\":\"AWCUos-Fib22Ai8JwW0y\",\"col\":1,\"row\":1}]",
|
||||
"optionsJSON": "{\"darkTheme\":false}",
|
||||
"uiStateJSON": "{\"P-15\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-20\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-21\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-22\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-23\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-24\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-5\":{\"vis\":{\"legendOpen\":false}},\"P-33\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-34\":{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-35\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-27\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-28\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-26\":{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"}}},\"P-25\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-32\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-36\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}}}",
|
||||
"version": 1,
|
||||
"timeRestore": true,
|
||||
"timeTo": "now",
|
||||
"timeFrom": "now-30d",
|
||||
"refreshInterval": {
|
||||
"display": "Off",
|
||||
"pause": false,
|
||||
"value": 0
|
||||
},
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"filter\":[{\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"-vulnerability_category:\\\"INFORMATION_GATHERED\\\"\"}}}],\"highlightAll\":true,\"version\":true}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "5dba30c0-3df3-11e7-a44e-c79ca8efb780",
|
||||
"_type": "dashboard",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Risk Mitigation Qualys Web Scoring",
|
||||
"hits": 0,
|
||||
"description": "",
|
||||
"panelsJSON": "[{\"col\":11,\"id\":\"995e2280-3df3-11e7-a44e-c79ca8efb780\",\"panelIndex\":20,\"row\":8,\"size_x\":2,\"size_y\":7,\"type\":\"visualization\"},{\"col\":1,\"id\":\"852816e0-3eb1-11e7-90cb-918f9cb01e3d\",\"panelIndex\":21,\"row\":10,\"size_x\":3,\"size_y\":5,\"type\":\"visualization\"},{\"col\":4,\"id\":\"297df800-3f7e-11e7-bd24-6903e3283192\",\"panelIndex\":27,\"row\":8,\"size_x\":3,\"size_y\":4,\"type\":\"visualization\"},{\"col\":9,\"id\":\"35b6d320-3f7f-11e7-bd24-6903e3283192\",\"panelIndex\":28,\"row\":8,\"size_x\":2,\"size_y\":7,\"type\":\"visualization\"},{\"col\":11,\"id\":\"471a3580-3f6b-11e7-88e7-df1abe6547fb\",\"panelIndex\":30,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":7,\"id\":\"de1a5f40-3f85-11e7-97f9-3777d794626d\",\"panelIndex\":31,\"row\":8,\"size_x\":2,\"size_y\":4,\"type\":\"visualization\"},{\"col\":10,\"id\":\"5093c620-44e9-11e7-8014-ede06a7e69f8\",\"panelIndex\":37,\"row\":4,\"size_x\":3,\"size_y\":4,\"type\":\"visualization\"},{\"col\":1,\"columns\":[\"host\",\"risk\",\"risk_score\",\"cve\",\"plugin_name\",\"solution\",\"plugin_output\"],\"id\":\"54648700-3f74-11e7-852e-69207a3d0726\",\"panelIndex\":38,\"row\":15,\"size_x\":12,\"size_y\":6,\"sort\":[\"@timestamp\",\"desc\"],\"type\":\"search\"},{\"col\":1,\"id\":\"fb6eb020-49ab-11e7-8f8c-57ad64ec48a6\",\"panelIndex\":39,\"row\":8,\"size_x\":3,\"size_y\":2,\"type\":\"visualization\"},{\"col\":5,\"id\":\"465c5820-8977-11e7-857e-e1d56b17746d\",\"panelIndex\":40,\"row\":4,\"size_x\":5,\"size_y\":4,\"type\":\"visualization\"},{\"col\":9,\"id\":\"b2f2adb0-897f-11e7-a2d2-c57bca21b3aa\",\"panelIndex\":45,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":1,\"id\":\"AWCUos-Fib22Ai8JwW0y\",\"panelIndex\":47,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":3,\"id\":\"AWCUozGBib22Ai8JwW1B\",\"panelIndex\":48,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":5,\"id\":\"AWCUo-jRib22Ai8JwW1N\",\"panelIndex\":49,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":7,\"id\":\"AWCUpE3Kib22Ai8JwW1c\",\"panelIndex\":50,\"row\":1,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"},{\"col\":1,\"id\":\"AWCUyeHGib22Ai8JwX62\",\"panelIndex\":51,\"row\":4,\"size_x\":4,\"size_y\":4,\"type\":\"visualization\"},{\"col\":4,\"id\":\"AWCg88f1ib22Ai8Jww3C\",\"panelIndex\":52,\"row\":12,\"size_x\":3,\"size_y\":3,\"type\":\"visualization\"},{\"col\":7,\"id\":\"AWCg9JUAib22Ai8Jww3Y\",\"panelIndex\":53,\"row\":12,\"size_x\":2,\"size_y\":3,\"type\":\"visualization\"}]",
|
||||
"optionsJSON": "{\"darkTheme\":false}",
|
||||
"uiStateJSON": "{\"P-11\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-2\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-20\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-21\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"desc\"}}}},\"P-27\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-28\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"desc\"}}}},\"P-3\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":0,\"direction\":\"asc\"}}}},\"P-30\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-31\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-40\":{\"vis\":{\"defaultColors\":{\"0 - 3\":\"rgb(0,104,55)\",\"3 - 7\":\"rgb(135,203,103)\",\"7 - 9\":\"rgb(255,255,190)\",\"9 - 11\":\"rgb(249,142,82)\"}}},\"P-41\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-42\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-43\":{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"}}},\"P-44\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-45\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-47\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-48\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-49\":{\"vis\":{\"defaultColors\":{\"0 - 1000\":\"rgb(0,104,55)\"},\"legendOpen\":false}},\"P-5\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-50\":{\"vis\":{\"defaultColors\":{\"0 - 10000\":\"rgb(0,104,55)\"}}},\"P-6\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-8\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-52\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}},\"P-53\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}}",
|
||||
"version": 1,
|
||||
"timeRestore": true,
|
||||
"timeTo": "now",
|
||||
"timeFrom": "now-30d",
|
||||
"refreshInterval": {
|
||||
"display": "Off",
|
||||
"pause": false,
|
||||
"value": 0
|
||||
},
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"filter\":[{\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"-vulnerability_category:\\\"INFORMATION_GATHERED\\\"\"}}}],\"highlightAll\":true,\"version\":true}"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
@ -0,0 +1,28 @@
|
||||
[
|
||||
{
|
||||
"_id": "54648700-3f74-11e7-852e-69207a3d0726",
|
||||
"_type": "search",
|
||||
"_source": {
|
||||
"title": "VulnWhisperer - Saved Search",
|
||||
"description": "",
|
||||
"hits": 0,
|
||||
"columns": [
|
||||
"host",
|
||||
"risk",
|
||||
"risk_score",
|
||||
"cve",
|
||||
"plugin_name",
|
||||
"solution",
|
||||
"plugin_output"
|
||||
],
|
||||
"sort": [
|
||||
"@timestamp",
|
||||
"desc"
|
||||
],
|
||||
"version": 1,
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"logstash-vulnwhisperer-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[],\"highlight\":{\"pre_tags\":[\"@kibana-highlighted-field@\"],\"post_tags\":[\"@/kibana-highlighted-field@\"],\"fields\":{\"*\":{}},\"require_field_match\":false,\"fragment_size\":2147483647}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
14
resources/elk5-old_compatibility/logstash/0001_input_beats.conf
Executable file
14
resources/elk5-old_compatibility/logstash/0001_input_beats.conf
Executable file
@ -0,0 +1,14 @@
|
||||
input {
|
||||
beats {
|
||||
port => 5044
|
||||
tags => "beats"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if [beat][hostname] == "filebeathost" {
|
||||
mutate {
|
||||
add_tag => ["nessus"]
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,220 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: email@austintaylor.io
|
||||
# Last Update: 12/20/2017
|
||||
# Version 0.3
|
||||
# Description: Take in nessus reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/nessus/**/*"
|
||||
start_position => "beginning"
|
||||
tags => "nessus"
|
||||
type => "nessus"
|
||||
}
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/tenable/*.csv"
|
||||
start_position => "beginning"
|
||||
tags => "tenable"
|
||||
type => "tenable"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "nessus" in [tags] or "tenable" in [tags] {
|
||||
# Drop the header column
|
||||
if [message] =~ "^Plugin ID" { drop {} }
|
||||
|
||||
csv {
|
||||
# columns => ["plugin_id", "cve", "cvss", "risk", "asset", "protocol", "port", "plugin_name", "synopsis", "description", "solution", "see_also", "plugin_output"]
|
||||
columns => ["plugin_id", "cve", "cvss", "risk", "asset", "protocol", "port", "plugin_name", "synopsis", "description", "solution", "see_also", "plugin_output", "asset_uuid", "vulnerability_state", "ip", "fqdn", "netbios", "operating_system", "mac_address", "plugin_family", "cvss_base", "cvss_temporal", "cvss_temporal_vector", "cvss_vector", "cvss3_base", "cvss3_temporal", "cvss3_temporal_vector", "cvss3_vector", "system_type", "host_start", "host_end"]
|
||||
separator => ","
|
||||
source => "message"
|
||||
}
|
||||
|
||||
ruby {
|
||||
code => "if event.get('description')
|
||||
event.set('description', event.get('description').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('synopsis')
|
||||
event.set('synopsis', event.get('synopsis').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('solution')
|
||||
event.set('solution', event.get('solution').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('see_also')
|
||||
event.set('see_also', event.get('see_also').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('plugin_output')
|
||||
event.set('plugin_output', event.get('plugin_output').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end"
|
||||
}
|
||||
|
||||
#If using filebeats as your source, you will need to replace the "path" field to "source"
|
||||
grok {
|
||||
match => { "path" => "(?<scan_name>[a-zA-Z0-9_.\-]+)_%{INT:scan_id}_%{INT:history_id}_%{INT:last_updated}.csv$" }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => ["last_updated"]
|
||||
}
|
||||
|
||||
if [risk] == "None" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
}
|
||||
if [risk] == "Low" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
}
|
||||
if [risk] == "Medium" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
}
|
||||
if [risk] == "High" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
}
|
||||
if [risk] == "Critical" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
}
|
||||
|
||||
if ![cve] or [cve] == "nan" {
|
||||
mutate { remove_field => [ "cve" ] }
|
||||
}
|
||||
if ![cvss] or [cvss] == "nan" {
|
||||
mutate { remove_field => [ "cvss" ] }
|
||||
}
|
||||
if ![cvss_base] or [cvss_base] == "nan" {
|
||||
mutate { remove_field => [ "cvss_base" ] }
|
||||
}
|
||||
if ![cvss_temporal] or [cvss_temporal] == "nan" {
|
||||
mutate { remove_field => [ "cvss_temporal" ] }
|
||||
}
|
||||
if ![cvss_temporal_vector] or [cvss_temporal_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss_temporal_vector" ] }
|
||||
}
|
||||
if ![cvss_vector] or [cvss_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss_vector" ] }
|
||||
}
|
||||
if ![cvss3_base] or [cvss3_base] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_base" ] }
|
||||
}
|
||||
if ![cvss3_temporal] or [cvss3_temporal] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_temporal" ] }
|
||||
}
|
||||
if ![cvss3_temporal_vector] or [cvss3_temporal_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_temporal_vector" ] }
|
||||
}
|
||||
if ![description] or [description] == "nan" {
|
||||
mutate { remove_field => [ "description" ] }
|
||||
}
|
||||
if ![mac_address] or [mac_address] == "nan" {
|
||||
mutate { remove_field => [ "mac_address" ] }
|
||||
}
|
||||
if ![netbios] or [netbios] == "nan" {
|
||||
mutate { remove_field => [ "netbios" ] }
|
||||
}
|
||||
if ![operating_system] or [operating_system] == "nan" {
|
||||
mutate { remove_field => [ "operating_system" ] }
|
||||
}
|
||||
if ![plugin_output] or [plugin_output] == "nan" {
|
||||
mutate { remove_field => [ "plugin_output" ] }
|
||||
}
|
||||
if ![see_also] or [see_also] == "nan" {
|
||||
mutate { remove_field => [ "see_also" ] }
|
||||
}
|
||||
if ![synopsis] or [synopsis] == "nan" {
|
||||
mutate { remove_field => [ "synopsis" ] }
|
||||
}
|
||||
if ![system_type] or [system_type] == "nan" {
|
||||
mutate { remove_field => [ "system_type" ] }
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => [ "message" ]
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
mutate {
|
||||
convert => { "risk_score" => "float" }
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
|
||||
# Compensating controls - adjust risk_score
|
||||
# Adobe and Java are not allowed to run in browser unless whitelisted
|
||||
# Therefore, lower score by dividing by 3 (score is subjective to risk)
|
||||
|
||||
#Modify and uncomment when ready to use
|
||||
#if [risk_score] != 0 {
|
||||
# if [plugin_name] =~ "Adobe" and [risk_score] > 6 or [plugin_name] =~ "Java" and [risk_score] > 6 {
|
||||
# ruby {
|
||||
# code => "event.set('risk_score', event.get('risk_score') / 3)"
|
||||
# }
|
||||
# mutate {
|
||||
# add_field => { "compensating_control" => "Adobe and Flash removed from browsers unless whitelisted site." }
|
||||
# }
|
||||
# }
|
||||
#}
|
||||
|
||||
# Add tags for reporting based on assets or criticality
|
||||
|
||||
if [asset] == "dc01" or [asset] == "dc02" or [asset] == "pki01" or [asset] == "192.168.0.54" or [asset] =~ "^192\.168\.0\." or [asset] =~ "^42.42.42." {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
#if [asset] =~ "^192\.168\.[45][0-9][0-9]\.1$" or [asset] =~ "^192.168\.[50]\.[0-9]{1,2}\.1$"{
|
||||
# mutate {
|
||||
# add_tag => [ "has_hipaa_data" ]
|
||||
# }
|
||||
#}
|
||||
#if [asset] =~ "^192\.168\.[45][0-9][0-9]\." {
|
||||
# mutate {
|
||||
# add_tag => [ "hipaa_asset" ]
|
||||
# }
|
||||
#}
|
||||
if [asset] =~ "^hr" {
|
||||
mutate {
|
||||
add_tag => [ "pci_asset" ]
|
||||
}
|
||||
}
|
||||
#if [asset] =~ "^10\.0\.50\." {
|
||||
# mutate {
|
||||
# add_tag => [ "web_servers" ]
|
||||
# }
|
||||
#}
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
if "nessus" in [tags] or "tenable" in [tags] or [type] in [ "nessus", "tenable" ] {
|
||||
# stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "localhost:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,153 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: austin@hasecuritysolutions.com
|
||||
# Last Update: 12/30/2017
|
||||
# Version 0.3
|
||||
# Description: Take in qualys web scan reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => [ "/opt/VulnWhisperer/data/qualys/*.json" , "/opt/VulnWhisperer/data/qualys_web/*.json", "/opt/VulnWhisperer/data/qualys_vuln/*.json" ]
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "qualys" ]
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "qualys" in [tags] {
|
||||
grok {
|
||||
match => { "path" => [ "(?<tags>qualys_vuln)_scan_%{DATA}_%{INT:last_updated}.json$", "(?<tags>qualys_web)_%{INT:app_id}_%{INT:last_updated}.json$" ] }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
mutate {
|
||||
replace => [ "message", "%{message}" ]
|
||||
#gsub => [
|
||||
# "message", "\|\|\|", " ",
|
||||
# "message", "\t\t", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", "nan", " ",
|
||||
# "message",'\n',''
|
||||
#]
|
||||
}
|
||||
|
||||
if "qualys_web" in [tags] {
|
||||
mutate {
|
||||
add_field => { "asset" => "%{web_application_name}" }
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
} else if "qualys_vuln" in [tags] {
|
||||
mutate {
|
||||
add_field => { "asset" => "%{ip}" }
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
}
|
||||
|
||||
if [risk] == "1" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
mutate { replace => { "risk" => "info" }}
|
||||
}
|
||||
if [risk] == "2" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
mutate { replace => { "risk" => "low" }}
|
||||
}
|
||||
if [risk] == "3" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
mutate { replace => { "risk" => "medium" }}
|
||||
}
|
||||
if [risk] == "4" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
mutate { replace => { "risk" => "high" }}
|
||||
}
|
||||
if [risk] == "5" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
mutate { replace => { "risk" => "critical" }}
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => "message"
|
||||
}
|
||||
|
||||
if [first_time_detected] {
|
||||
date {
|
||||
match => [ "first_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_detected"
|
||||
}
|
||||
}
|
||||
if [first_time_tested] {
|
||||
date {
|
||||
match => [ "first_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_tested"
|
||||
}
|
||||
}
|
||||
if [last_time_detected] {
|
||||
date {
|
||||
match => [ "last_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_detected"
|
||||
}
|
||||
}
|
||||
if [last_time_tested] {
|
||||
date {
|
||||
match => [ "last_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_tested"
|
||||
}
|
||||
}
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => "last_updated"
|
||||
}
|
||||
mutate {
|
||||
convert => { "plugin_id" => "integer"}
|
||||
convert => { "id" => "integer"}
|
||||
convert => { "risk_number" => "integer"}
|
||||
convert => { "risk_score" => "float"}
|
||||
convert => { "total_times_detected" => "integer"}
|
||||
convert => { "cvss_temporal" => "float"}
|
||||
convert => { "cvss" => "float"}
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
|
||||
if [asset] =~ "\.yourdomain\.(com|net)$" {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
output {
|
||||
if "qualys" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "localhost:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
146
resources/elk5-old_compatibility/logstash/3000_openvas.conf
Normal file
146
resources/elk5-old_compatibility/logstash/3000_openvas.conf
Normal file
@ -0,0 +1,146 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: austin@hasecuritysolutions.com
|
||||
# Last Update: 03/04/2018
|
||||
# Version 0.3
|
||||
# Description: Take in qualys web scan reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/openvas/*.json"
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "openvas_scan", "openvas" ]
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "openvas_scan" in [tags] {
|
||||
mutate {
|
||||
replace => [ "message", "%{message}" ]
|
||||
gsub => [
|
||||
"message", "\|\|\|", " ",
|
||||
"message", "\t\t", " ",
|
||||
"message", " ", " ",
|
||||
"message", " ", " ",
|
||||
"message", " ", " ",
|
||||
"message", "nan", " ",
|
||||
"message",'\n',''
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
grok {
|
||||
match => { "path" => "openvas_scan_%{DATA:scan_id}_%{INT:last_updated}.json$" }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
mutate {
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
|
||||
if [risk] == "1" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
mutate { replace => { "risk" => "info" }}
|
||||
}
|
||||
if [risk] == "2" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
mutate { replace => { "risk" => "low" }}
|
||||
}
|
||||
if [risk] == "3" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
mutate { replace => { "risk" => "medium" }}
|
||||
}
|
||||
if [risk] == "4" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
mutate { replace => { "risk" => "high" }}
|
||||
}
|
||||
if [risk] == "5" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
mutate { replace => { "risk" => "critical" }}
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => "message"
|
||||
}
|
||||
|
||||
if [first_time_detected] {
|
||||
date {
|
||||
match => [ "first_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_detected"
|
||||
}
|
||||
}
|
||||
if [first_time_tested] {
|
||||
date {
|
||||
match => [ "first_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_tested"
|
||||
}
|
||||
}
|
||||
if [last_time_detected] {
|
||||
date {
|
||||
match => [ "last_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_detected"
|
||||
}
|
||||
}
|
||||
if [last_time_tested] {
|
||||
date {
|
||||
match => [ "last_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_tested"
|
||||
}
|
||||
}
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => "last_updated"
|
||||
}
|
||||
mutate {
|
||||
convert => { "plugin_id" => "integer"}
|
||||
convert => { "id" => "integer"}
|
||||
convert => { "risk_number" => "integer"}
|
||||
convert => { "risk_score" => "float"}
|
||||
convert => { "total_times_detected" => "integer"}
|
||||
convert => { "cvss_temporal" => "float"}
|
||||
convert => { "cvss" => "float"}
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
# Add your critical assets by subnet or by hostname. Comment this field out if you don't want to tag any, but the asset panel will break.
|
||||
if [asset] =~ "^10\.0\.100\." {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
output {
|
||||
if "openvas" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "localhost:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
21
resources/elk5-old_compatibility/logstash/4000_jira.conf
Normal file
21
resources/elk5-old_compatibility/logstash/4000_jira.conf
Normal file
@ -0,0 +1,21 @@
|
||||
# Description: Take in jira tickets from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/jira/*.json"
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "jira" ]
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
if "jira" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "localhost:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
13
resources/elk5-old_compatibility/logstash/9998_input_broker_rabbitmq.conf
Executable file
13
resources/elk5-old_compatibility/logstash/9998_input_broker_rabbitmq.conf
Executable file
@ -0,0 +1,13 @@
|
||||
input {
|
||||
rabbitmq {
|
||||
key => "nessus"
|
||||
queue => "nessus"
|
||||
durable => true
|
||||
exchange => "nessus"
|
||||
user => "logstash"
|
||||
password => "yourpassword"
|
||||
host => "buffer01"
|
||||
port => 5672
|
||||
tags => [ "queue_nessus", "rabbitmq" ]
|
||||
}
|
||||
}
|
||||
16
resources/elk5-old_compatibility/logstash/9998_output_broker_rabbitmq.conf
Executable file
16
resources/elk5-old_compatibility/logstash/9998_output_broker_rabbitmq.conf
Executable file
@ -0,0 +1,16 @@
|
||||
output {
|
||||
if "nessus" in [tags]{
|
||||
rabbitmq {
|
||||
key => "nessus"
|
||||
exchange => "nessus"
|
||||
exchange_type => "direct"
|
||||
user => "logstash"
|
||||
password => "yourbufferpassword"
|
||||
host => "buffer01"
|
||||
port => 5672
|
||||
durable => true
|
||||
persistent => true
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@ -1,15 +0,0 @@
|
||||
import json
|
||||
import requests
|
||||
|
||||
api_objects = []
|
||||
|
||||
for object_type in ['dashboard', 'visualization', 'search', 'index-pattern', 'timelion-sheet']:
|
||||
r = requests.get('http://localhost:5601/api/saved_objects/_find?per_page=500&type={}'.format(object_type)).json()
|
||||
api_objects += r['saved_objects']
|
||||
print object_type, len(r['saved_objects'])
|
||||
print len(api_objects)
|
||||
|
||||
for api_object in api_objects:
|
||||
api_object.pop('updated_at', None)
|
||||
|
||||
json.dump(sorted(api_objects, key=lambda x:x['id']), open('kibana_APIonly.json', 'w'), indent=2)
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@ -1,6 +1,5 @@
|
||||
{
|
||||
"index_patterns": "logstash-vulnwhisperer-*",
|
||||
"version": 2019041701,
|
||||
"mappings": {
|
||||
"doc": {
|
||||
"properties": {
|
||||
@ -23,34 +22,34 @@
|
||||
"asset_uuid": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"assign_ip": {
|
||||
"type": "ip"
|
||||
},
|
||||
"category": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cve": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss_base": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss_temporal_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss_temporal": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss_severity": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss2_base": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss2_severity": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss2_temporal": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss2_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss3_base": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss3_severity": {
|
||||
"cvss3_temporal_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss3_temporal": {
|
||||
@ -118,14 +117,24 @@
|
||||
"host_start": {
|
||||
"type": "date"
|
||||
},
|
||||
"impact": {
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"ignore_above": 256,
|
||||
"type": "keyword"
|
||||
}
|
||||
},
|
||||
"norms": false,
|
||||
"type": "text"
|
||||
},
|
||||
"ip_status": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"ip": {
|
||||
"type": "ip"
|
||||
},
|
||||
"mac_address": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"netbios": {
|
||||
"type": "keyword"
|
||||
"last_updated": {
|
||||
"type": "date"
|
||||
},
|
||||
"operating_system": {
|
||||
"type": "keyword"
|
||||
@ -139,10 +148,10 @@
|
||||
"plugin_family": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"signature_id": {
|
||||
"plugin_id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"signature": {
|
||||
"plugin_name": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"plugin_output": {
|
||||
@ -161,9 +170,18 @@
|
||||
"protocol": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"results": {
|
||||
"type": "text"
|
||||
},
|
||||
"risk_number": {
|
||||
"type": "integer"
|
||||
},
|
||||
"risk_score_name": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"risk_score": {
|
||||
"type": "float"
|
||||
},
|
||||
"risk": {
|
||||
"type": "keyword"
|
||||
},
|
||||
@ -171,41 +189,43 @@
|
||||
"type": "keyword"
|
||||
},
|
||||
"scan_name": {
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"ignore_above": 256,
|
||||
"type": "keyword"
|
||||
}
|
||||
},
|
||||
"norms": false,
|
||||
"type": "text"
|
||||
},
|
||||
"scan_source": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"severity": {
|
||||
"scan_reference": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"see_also": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"solution": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"ssl": {
|
||||
"source": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"state": {
|
||||
"ssl": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"synopsis": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"system_type": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"tags": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"threat": {
|
||||
"type": "text"
|
||||
},
|
||||
"type": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"vendor_reference": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"vulnerability_state": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
231
resources/elk6/logstash-vulnwhisperer-template_elk7.json
Executable file
231
resources/elk6/logstash-vulnwhisperer-template_elk7.json
Executable file
@ -0,0 +1,231 @@
|
||||
{
|
||||
"index_patterns": "logstash-vulnwhisperer-*",
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"@timestamp": {
|
||||
"type": "date"
|
||||
},
|
||||
"@version": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"asset": {
|
||||
"type": "text",
|
||||
"norms": false,
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword",
|
||||
"ignore_above": 256
|
||||
}
|
||||
}
|
||||
},
|
||||
"asset_uuid": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"assign_ip": {
|
||||
"type": "ip"
|
||||
},
|
||||
"category": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cve": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss_base": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss_temporal_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss_temporal": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss3_base": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss3_temporal_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss3_temporal": {
|
||||
"type": "float"
|
||||
},
|
||||
"cvss3_vector": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"cvss3": {
|
||||
"type": "float"
|
||||
},
|
||||
"description": {
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"ignore_above": 256,
|
||||
"type": "keyword"
|
||||
}
|
||||
},
|
||||
"norms": false,
|
||||
"type": "text"
|
||||
},
|
||||
"dns": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"exploitability": {
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"ignore_above": 256,
|
||||
"type": "keyword"
|
||||
}
|
||||
},
|
||||
"norms": false,
|
||||
"type": "text"
|
||||
},
|
||||
"fqdn": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"geoip": {
|
||||
"dynamic": true,
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ip": {
|
||||
"type": "ip"
|
||||
},
|
||||
"latitude": {
|
||||
"type": "float"
|
||||
},
|
||||
"location": {
|
||||
"type": "geo_point"
|
||||
},
|
||||
"longitude": {
|
||||
"type": "float"
|
||||
}
|
||||
}
|
||||
},
|
||||
"history_id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"host": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"host_end": {
|
||||
"type": "date"
|
||||
},
|
||||
"host_start": {
|
||||
"type": "date"
|
||||
},
|
||||
"impact": {
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"ignore_above": 256,
|
||||
"type": "keyword"
|
||||
}
|
||||
},
|
||||
"norms": false,
|
||||
"type": "text"
|
||||
},
|
||||
"ip_status": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"ip": {
|
||||
"type": "ip"
|
||||
},
|
||||
"last_updated": {
|
||||
"type": "date"
|
||||
},
|
||||
"operating_system": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"path": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"pci_vuln": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"plugin_family": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"plugin_id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"plugin_name": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"plugin_output": {
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"ignore_above": 256,
|
||||
"type": "keyword"
|
||||
}
|
||||
},
|
||||
"norms": false,
|
||||
"type": "text"
|
||||
},
|
||||
"port": {
|
||||
"type": "integer"
|
||||
},
|
||||
"protocol": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"results": {
|
||||
"type": "text"
|
||||
},
|
||||
"risk_number": {
|
||||
"type": "integer"
|
||||
},
|
||||
"risk_score_name": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"risk_score": {
|
||||
"type": "float"
|
||||
},
|
||||
"risk": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"scan_id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"scan_name": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"scan_reference": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"see_also": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"solution": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"source": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"ssl": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"synopsis": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"system_type": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"tags": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"threat": {
|
||||
"type": "text"
|
||||
},
|
||||
"type": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"vendor_reference": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"vulnerability_state": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,110 +0,0 @@
|
||||
input {
|
||||
file {
|
||||
codec => json
|
||||
mode => "read"
|
||||
path => ["/opt/VulnWhisperer/data/nessus/**/*.json", "/opt/VulnWhisperer/data/openvas/*.json", "/opt/VulnWhisperer/data/qualys_vm/*.json", "/opt/VulnWhisperer/data/qualys_was/*.json", "/opt/VulnWhisperer/data/tenable/*.json"]
|
||||
start_position => "beginning"
|
||||
file_completed_action => "delete"
|
||||
file_chunk_size => 262144
|
||||
}
|
||||
file {
|
||||
codec => json
|
||||
mode => "read"
|
||||
path => "/opt/VulnWhisperer/data/jira/*.json"
|
||||
tags => [ "jira" ]
|
||||
start_position => "beginning"
|
||||
file_completed_action => "delete"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if [scan_source] in ["nessus", "tenable", "qualys_vm", "qualys_was", "openvas"] {
|
||||
|
||||
# Parse the date/time from scan_time
|
||||
date {
|
||||
match => [ "scan_time", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => ["scan_time"]
|
||||
}
|
||||
|
||||
# Add scan_source to tags
|
||||
mutate {
|
||||
add_field => { "[tags]" => "%{scan_source}" }
|
||||
}
|
||||
|
||||
# Create a unique document_id if _unique field exists
|
||||
if [_unique] {
|
||||
# Set document ID from _unique
|
||||
mutate {
|
||||
rename => { "_unique" => "[@metadata][id]" }
|
||||
}
|
||||
}
|
||||
|
||||
# Do we need this?
|
||||
mutate {
|
||||
convert => { "cvss" => "float"}
|
||||
convert => { "cvss2" => "float"}
|
||||
convert => { "cvss2_base" => "float"}
|
||||
convert => { "cvss2_temporal" => "float"}
|
||||
convert => { "cvss3" => "float"}
|
||||
convert => { "cvss3_base" => "float"}
|
||||
convert => { "cvss3_temporal" => "float"}
|
||||
convert => { "risk_number" => "integer"}
|
||||
convert => { "total_times_detected" => "integer"}
|
||||
}
|
||||
}
|
||||
|
||||
if [scan_source] == "qualys_was" {
|
||||
if [first_time_detected] {
|
||||
date {
|
||||
match => [ "first_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_detected"
|
||||
}
|
||||
}
|
||||
if [first_time_tested] {
|
||||
date {
|
||||
match => [ "first_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_tested"
|
||||
}
|
||||
}
|
||||
if [last_time_detected] {
|
||||
date {
|
||||
match => [ "last_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_detected"
|
||||
}
|
||||
}
|
||||
if [last_time_tested] {
|
||||
date {
|
||||
match => [ "last_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_tested"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
if [scan_source] in ["nessus", "tenable", "qualys_vm", "qualys_was", "openvas"] {
|
||||
if [@metadata][id] {
|
||||
elasticsearch {
|
||||
hosts => [ "elasticsearch:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
document_id => "%{[@metadata][id]}"
|
||||
manage_template => false
|
||||
}
|
||||
} else {
|
||||
elasticsearch {
|
||||
hosts => [ "elasticsearch:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
manage_template => false
|
||||
}
|
||||
}
|
||||
}
|
||||
# Should these go to the same index?
|
||||
if "jira" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "elasticsearch:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
182
resources/elk6/pipeline/1000_nessus_process_file.conf
Normal file
182
resources/elk6/pipeline/1000_nessus_process_file.conf
Normal file
@ -0,0 +1,182 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: email@austintaylor.io
|
||||
# Last Update: 12/20/2017
|
||||
# Version 0.3
|
||||
# Description: Take in nessus reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/nessus/**/*"
|
||||
mode => "read"
|
||||
start_position => "beginning"
|
||||
file_completed_action => "delete"
|
||||
tags => "nessus"
|
||||
}
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/tenable/*.csv"
|
||||
mode => "read"
|
||||
start_position => "beginning"
|
||||
file_completed_action => "delete"
|
||||
tags => "tenable"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "nessus" in [tags] or "tenable" in [tags] {
|
||||
# Drop the header column
|
||||
if [message] =~ "^Plugin ID" { drop {} }
|
||||
|
||||
csv {
|
||||
# columns => ["plugin_id", "cve", "cvss", "risk", "asset", "protocol", "port", "plugin_name", "synopsis", "description", "solution", "see_also", "plugin_output"]
|
||||
columns => ["plugin_id", "cve", "cvss", "risk", "asset", "protocol", "port", "plugin_name", "synopsis", "description", "solution", "see_also", "plugin_output", "asset_uuid", "vulnerability_state", "ip", "fqdn", "netbios", "operating_system", "mac_address", "plugin_family", "cvss_base", "cvss_temporal", "cvss_temporal_vector", "cvss_vector", "cvss3_base", "cvss3_temporal", "cvss3_temporal_vector", "cvss3_vector", "system_type", "host_start", "host_end"]
|
||||
separator => ","
|
||||
source => "message"
|
||||
}
|
||||
|
||||
ruby {
|
||||
code => "if event.get('description')
|
||||
event.set('description', event.get('description').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('synopsis')
|
||||
event.set('synopsis', event.get('synopsis').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('solution')
|
||||
event.set('solution', event.get('solution').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('see_also')
|
||||
event.set('see_also', event.get('see_also').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end
|
||||
if event.get('plugin_output')
|
||||
event.set('plugin_output', event.get('plugin_output').gsub(92.chr + 'n', 10.chr).gsub(92.chr + 'r', 13.chr))
|
||||
end"
|
||||
}
|
||||
|
||||
#If using filebeats as your source, you will need to replace the "path" field to "source"
|
||||
# Remove when scan name is included in event (current method is error prone)
|
||||
grok {
|
||||
match => { "path" => "(?<scan_name>[a-zA-Z0-9_.\-]+)_%{INT:scan_id}_%{INT:history_id}_%{INT:last_updated}.csv$" }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
# TODO remove when @timestamp is included in event
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => ["last_updated"]
|
||||
}
|
||||
|
||||
if [risk] == "None" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
}
|
||||
if [risk] == "Low" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
}
|
||||
if [risk] == "Medium" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
}
|
||||
if [risk] == "High" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
}
|
||||
if [risk] == "Critical" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
}
|
||||
|
||||
if ![cve] or [cve] == "nan" {
|
||||
mutate { remove_field => [ "cve" ] }
|
||||
}
|
||||
if ![cvss] or [cvss] == "nan" {
|
||||
mutate { remove_field => [ "cvss" ] }
|
||||
}
|
||||
if ![cvss_base] or [cvss_base] == "nan" {
|
||||
mutate { remove_field => [ "cvss_base" ] }
|
||||
}
|
||||
if ![cvss_temporal] or [cvss_temporal] == "nan" {
|
||||
mutate { remove_field => [ "cvss_temporal" ] }
|
||||
}
|
||||
if ![cvss_temporal_vector] or [cvss_temporal_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss_temporal_vector" ] }
|
||||
}
|
||||
if ![cvss_vector] or [cvss_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss_vector" ] }
|
||||
}
|
||||
if ![cvss3_base] or [cvss3_base] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_base" ] }
|
||||
}
|
||||
if ![cvss3_temporal] or [cvss3_temporal] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_temporal" ] }
|
||||
}
|
||||
if ![cvss3_temporal_vector] or [cvss3_temporal_vector] == "nan" {
|
||||
mutate { remove_field => [ "cvss3_temporal_vector" ] }
|
||||
}
|
||||
if ![description] or [description] == "nan" {
|
||||
mutate { remove_field => [ "description" ] }
|
||||
}
|
||||
if ![mac_address] or [mac_address] == "nan" {
|
||||
mutate { remove_field => [ "mac_address" ] }
|
||||
}
|
||||
if ![netbios] or [netbios] == "nan" {
|
||||
mutate { remove_field => [ "netbios" ] }
|
||||
}
|
||||
if ![operating_system] or [operating_system] == "nan" {
|
||||
mutate { remove_field => [ "operating_system" ] }
|
||||
}
|
||||
if ![plugin_output] or [plugin_output] == "nan" {
|
||||
mutate { remove_field => [ "plugin_output" ] }
|
||||
}
|
||||
if ![see_also] or [see_also] == "nan" {
|
||||
mutate { remove_field => [ "see_also" ] }
|
||||
}
|
||||
if ![synopsis] or [synopsis] == "nan" {
|
||||
mutate { remove_field => [ "synopsis" ] }
|
||||
}
|
||||
if ![system_type] or [system_type] == "nan" {
|
||||
mutate { remove_field => [ "system_type" ] }
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => [ "message" ]
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
mutate {
|
||||
convert => { "risk_score" => "float" }
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
if "nessus" in [tags] or "tenable" in [tags]{
|
||||
stdout {
|
||||
codec => dots
|
||||
}
|
||||
elasticsearch {
|
||||
hosts => [ "elasticsearch:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
160
resources/elk6/pipeline/2000_qualys_web_scans.conf
Normal file
160
resources/elk6/pipeline/2000_qualys_web_scans.conf
Normal file
@ -0,0 +1,160 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: austin@hasecuritysolutions.com
|
||||
# Last Update: 12/30/2017
|
||||
# Version 0.3
|
||||
# Description: Take in qualys web scan reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => [ "/opt/VulnWhisperer/data/qualys/*.json" , "/opt/VulnWhisperer/data/qualys_web/*.json", "/opt/VulnWhisperer/data/qualys_vuln/*.json"]
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "qualys" ]
|
||||
mode => "read"
|
||||
start_position => "beginning"
|
||||
file_completed_action => "delete"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "qualys" in [tags] {
|
||||
grok {
|
||||
match => { "path" => [ "(?<tags>qualys_vuln)_scan_%{DATA}_%{INT:last_updated}.json$", "(?<tags>qualys_web)_%{INT:app_id}_%{INT:last_updated}.json$" ] }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
mutate {
|
||||
replace => [ "message", "%{message}" ]
|
||||
#gsub => [
|
||||
# "message", "\|\|\|", " ",
|
||||
# "message", "\t\t", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", " ", " ",
|
||||
# "message", "nan", " ",
|
||||
# "message",'\n',''
|
||||
#]
|
||||
}
|
||||
|
||||
if "qualys_web" in [tags] {
|
||||
mutate {
|
||||
add_field => { "asset" => "%{web_application_name}" }
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
} else if "qualys_vuln" in [tags] {
|
||||
mutate {
|
||||
add_field => { "asset" => "%{ip}" }
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
}
|
||||
|
||||
if [risk] == "1" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
mutate { replace => { "risk" => "info" }}
|
||||
}
|
||||
if [risk] == "2" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
mutate { replace => { "risk" => "low" }}
|
||||
}
|
||||
if [risk] == "3" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
mutate { replace => { "risk" => "medium" }}
|
||||
}
|
||||
if [risk] == "4" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
mutate { replace => { "risk" => "high" }}
|
||||
}
|
||||
if [risk] == "5" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
mutate { replace => { "risk" => "critical" }}
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => "message"
|
||||
}
|
||||
|
||||
if [first_time_detected] {
|
||||
date {
|
||||
match => [ "first_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_detected"
|
||||
}
|
||||
}
|
||||
if [first_time_tested] {
|
||||
date {
|
||||
match => [ "first_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_tested"
|
||||
}
|
||||
}
|
||||
if [last_time_detected] {
|
||||
date {
|
||||
match => [ "last_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_detected"
|
||||
}
|
||||
}
|
||||
if [last_time_tested] {
|
||||
date {
|
||||
match => [ "last_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_tested"
|
||||
}
|
||||
}
|
||||
|
||||
# TODO remove when @timestamp is included in event
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => "last_updated"
|
||||
}
|
||||
mutate {
|
||||
convert => { "plugin_id" => "integer"}
|
||||
convert => { "id" => "integer"}
|
||||
convert => { "risk_number" => "integer"}
|
||||
convert => { "risk_score" => "float"}
|
||||
convert => { "total_times_detected" => "integer"}
|
||||
convert => { "cvss_temporal" => "float"}
|
||||
convert => { "cvss" => "float"}
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
|
||||
if [asset] =~ "\.yourdomain\.(com|net)$" {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
output {
|
||||
if "qualys" in [tags] {
|
||||
stdout {
|
||||
codec => dots
|
||||
}
|
||||
elasticsearch {
|
||||
hosts => [ "elasticsearch:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
154
resources/elk6/pipeline/3000_openvas.conf
Normal file
154
resources/elk6/pipeline/3000_openvas.conf
Normal file
@ -0,0 +1,154 @@
|
||||
# Author: Austin Taylor and Justin Henderson
|
||||
# Email: austin@hasecuritysolutions.com
|
||||
# Last Update: 03/04/2018
|
||||
# Version 0.3
|
||||
# Description: Take in qualys web scan reports from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/openvas/*.json"
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
tags => [ "openvas_scan", "openvas" ]
|
||||
mode => "read"
|
||||
start_position => "beginning"
|
||||
file_completed_action => "delete"
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if "openvas_scan" in [tags] {
|
||||
mutate {
|
||||
replace => [ "message", "%{message}" ]
|
||||
gsub => [
|
||||
"message", "\|\|\|", " ",
|
||||
"message", "\t\t", " ",
|
||||
"message", " ", " ",
|
||||
"message", " ", " ",
|
||||
"message", " ", " ",
|
||||
"message", "nan", " ",
|
||||
"message",'\n',''
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
grok {
|
||||
match => { "path" => "openvas_scan_%{DATA:scan_id}_%{INT:last_updated}.json$" }
|
||||
tag_on_failure => []
|
||||
}
|
||||
|
||||
mutate {
|
||||
add_field => { "risk_score" => "%{cvss}" }
|
||||
}
|
||||
|
||||
if [risk] == "1" {
|
||||
mutate { add_field => { "risk_number" => 0 }}
|
||||
mutate { replace => { "risk" => "info" }}
|
||||
}
|
||||
if [risk] == "2" {
|
||||
mutate { add_field => { "risk_number" => 1 }}
|
||||
mutate { replace => { "risk" => "low" }}
|
||||
}
|
||||
if [risk] == "3" {
|
||||
mutate { add_field => { "risk_number" => 2 }}
|
||||
mutate { replace => { "risk" => "medium" }}
|
||||
}
|
||||
if [risk] == "4" {
|
||||
mutate { add_field => { "risk_number" => 3 }}
|
||||
mutate { replace => { "risk" => "high" }}
|
||||
}
|
||||
if [risk] == "5" {
|
||||
mutate { add_field => { "risk_number" => 4 }}
|
||||
mutate { replace => { "risk" => "critical" }}
|
||||
}
|
||||
|
||||
mutate {
|
||||
remove_field => "message"
|
||||
}
|
||||
|
||||
if [first_time_detected] {
|
||||
date {
|
||||
match => [ "first_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_detected"
|
||||
}
|
||||
}
|
||||
if [first_time_tested] {
|
||||
date {
|
||||
match => [ "first_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "first_time_tested"
|
||||
}
|
||||
}
|
||||
if [last_time_detected] {
|
||||
date {
|
||||
match => [ "last_time_detected", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_detected"
|
||||
}
|
||||
}
|
||||
if [last_time_tested] {
|
||||
date {
|
||||
match => [ "last_time_tested", "dd MMM yyyy HH:mma 'GMT'ZZ", "dd MMM yyyy HH:mma 'GMT'" ]
|
||||
target => "last_time_tested"
|
||||
}
|
||||
}
|
||||
|
||||
# TODO remove when @timestamp is included in event
|
||||
date {
|
||||
match => [ "last_updated", "UNIX" ]
|
||||
target => "@timestamp"
|
||||
remove_field => "last_updated"
|
||||
}
|
||||
mutate {
|
||||
convert => { "plugin_id" => "integer"}
|
||||
convert => { "id" => "integer"}
|
||||
convert => { "risk_number" => "integer"}
|
||||
convert => { "risk_score" => "float"}
|
||||
convert => { "total_times_detected" => "integer"}
|
||||
convert => { "cvss_temporal" => "float"}
|
||||
convert => { "cvss" => "float"}
|
||||
}
|
||||
if [risk_score] == 0 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "info" }
|
||||
}
|
||||
}
|
||||
if [risk_score] > 0 and [risk_score] < 3 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "low" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 3 and [risk_score] < 6 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "medium" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >=6 and [risk_score] < 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "high" }
|
||||
}
|
||||
}
|
||||
if [risk_score] >= 9 {
|
||||
mutate {
|
||||
add_field => { "risk_score_name" => "critical" }
|
||||
}
|
||||
}
|
||||
# Add your critical assets by subnet or by hostname. Comment this field out if you don't want to tag any, but the asset panel will break.
|
||||
if [asset] =~ "^10\.0\.100\." {
|
||||
mutate {
|
||||
add_tag => [ "critical_asset" ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
output {
|
||||
if "openvas" in [tags] {
|
||||
stdout {
|
||||
codec => dots
|
||||
}
|
||||
elasticsearch {
|
||||
hosts => [ "elasticsearch:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
25
resources/elk6/pipeline/4000_jira.conf
Normal file
25
resources/elk6/pipeline/4000_jira.conf
Normal file
@ -0,0 +1,25 @@
|
||||
# Description: Take in jira tickets from vulnWhisperer and pumps into logstash
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/opt/VulnWhisperer/data/jira/*.json"
|
||||
type => json
|
||||
codec => json
|
||||
start_position => "beginning"
|
||||
mode => "read"
|
||||
start_position => "beginning"
|
||||
file_completed_action => "delete"
|
||||
|
||||
tags => [ "jira" ]
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
if "jira" in [tags] {
|
||||
stdout { codec => rubydebug }
|
||||
elasticsearch {
|
||||
hosts => [ "elasticsearch:9200" ]
|
||||
index => "logstash-vulnwhisperer-%{+YYYY.MM}"
|
||||
}
|
||||
}
|
||||
}
|
||||
Submodule tests/data updated: 1d0e07075e...55dc6832f8
@ -29,49 +29,49 @@ done
|
||||
green "✅ Elasticsearch status is green..."
|
||||
|
||||
count=0
|
||||
until [[ $(curl -s "$logstash_url/_node/stats" | jq '.events.out') -ge 1617 ]]; do
|
||||
yellow "Waiting for Logstash load to finish... $(curl -s "$logstash_url/_node/stats" | jq '.events.out') of 1617 (attempt $count of 60)"
|
||||
until [[ $(curl -s "$logstash_url/_node/stats" | jq '.events.out') -ge 1236 ]]; do
|
||||
yellow "Waiting for Logstash load to finish... $(curl -s "$logstash_url/_node/stats" | jq '.events.out') of 1236 (attempt $count of 60)"
|
||||
((count++)) && ((count==60)) && break
|
||||
sleep 5
|
||||
done
|
||||
|
||||
if [[ count -le 60 && $(curl -s "$logstash_url/_node/stats" | jq '.events.out') -ge 1617 ]]; then
|
||||
green "✅ Logstash load finished $(curl -s "$logstash_url/_node/stats" | jq '.events.out') logs processed..."
|
||||
if [[ count -le 60 && $(curl -s "$logstash_url/_node/stats" | jq '.events.out') -ge 1236 ]]; then
|
||||
green "✅ Logstash load finished..."
|
||||
else
|
||||
red "❌ Logstash load didn't complete $(curl -s "$logstash_url/_node/stats" | jq '.events.out') logs processed... $(curl -s "$logstash_url/_node/stats" | jq '.events.out')"
|
||||
red "❌ Logstash load didn't complete... $(curl -s "$logstash_url/_node/stats" | jq '.events.out')"
|
||||
fi
|
||||
|
||||
|
||||
count=0
|
||||
until [[ $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_count" | jq '.count') -ge 1617 ]] ; do
|
||||
yellow "Waiting for Elasticsearch index to sync... $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_count" | jq '.count') of 1617 logs loaded (attempt $count of 150)"
|
||||
until [[ $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_count" | jq '.count') -ge 1232 ]] ; do
|
||||
yellow "Waiting for Elasticsearch index to sync... $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_count" | jq '.count') of 1232 logs loaded (attempt $count of 150)"
|
||||
((count++)) && ((count==150)) && break
|
||||
sleep 2
|
||||
done
|
||||
if [[ count -le 50 && $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_count" | jq '.count') -ge 1617 ]]; then
|
||||
green "✅ logstash-vulnwhisperer-* document count $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_count" | jq '.count') >= 1617"
|
||||
if [[ count -le 50 && $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_count" | jq '.count') -ge 1232 ]]; then
|
||||
green "✅ logstash-vulnwhisperer-2019.03 document count >= 1232"
|
||||
else
|
||||
red "❌ TIMED OUT waiting for logstash-vulnwhisperer-* document count: $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_count" | jq) != 1617"
|
||||
red "❌ TIMED OUT waiting for logstash-vulnwhisperer-2019.03 document count: $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_count" | jq) != 1232"
|
||||
fi
|
||||
|
||||
# if [[ $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_count" | jq '.count') == 1232 ]]; then
|
||||
# green "✅ Passed: logstash-vulnwhisperer-* document count == 1232"
|
||||
# if [[ $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_count" | jq '.count') == 1232 ]]; then
|
||||
# green "✅ Passed: logstash-vulnwhisperer-2019.03 document count == 1232"
|
||||
# else
|
||||
# red "❌ Failed: logstash-vulnwhisperer-* document count == 1232 was: $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_count") instead"
|
||||
# red "❌ Failed: logstash-vulnwhisperer-2019.03 document count == 1232 was: $(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_count") instead"
|
||||
# ((return_code = return_code + 1))
|
||||
# fi
|
||||
|
||||
# Test Nessus signature:Backported Security Patch Detection (FTP)
|
||||
nessus_doc=$(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_search?q=signature:%22Backported%20Security%20Patch%20Detection%20(FTP)%22%20AND%20asset:176.28.50.164%20AND%20tags:nessus" | jq '.hits.hits[]._source')
|
||||
if echo $nessus_doc | jq '.risk' | grep -q "none"; then
|
||||
green "✅ Passed: Nessus risk == none"
|
||||
# Test Nessus plugin_name:Backported Security Patch Detection (FTP)
|
||||
nessus_doc=$(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_search?q=plugin_name:%22Backported%20Security%20Patch%20Detection%20(FTP)%22%20AND%20asset:176.28.50.164%20AND%20tags:nessus" | jq '.hits.hits[]._source')
|
||||
if echo $nessus_doc | jq '.risk' | grep -q "None"; then
|
||||
green "✅ Passed: Nessus risk == None"
|
||||
else
|
||||
red "❌ Failed: Nessus risk == none was: $(echo $nessus_doc | jq '.risk') instead"
|
||||
red "❌ Failed: Nessus risk == None was: $(echo $nessus_doc | jq '.risk') instead"
|
||||
((return_code = return_code + 1))
|
||||
fi
|
||||
|
||||
# Test Tenable signature:Backported Security Patch Detection (FTP)
|
||||
tenable_doc=$(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_search?q=signature:%22Backported%20Security%20Patch%20Detection%20(FTP)%22%20AND%20asset:176.28.50.164%20AND%20tags:tenable" | jq '.hits.hits[]._source')
|
||||
# Test Tenable plugin_name:Backported Security Patch Detection (FTP)
|
||||
tenable_doc=$(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_search?q=plugin_name:%22Backported%20Security%20Patch%20Detection%20(FTP)%22%20AND%20asset:176.28.50.164%20AND%20tags:tenable" | jq '.hits.hits[]._source')
|
||||
# Test asset
|
||||
if echo $tenable_doc | jq .asset | grep -q '176.28.50.164'; then
|
||||
green "✅ Passed: Tenable asset == 176.28.50.164"
|
||||
@ -88,21 +88,21 @@ else
|
||||
((return_code = return_code + 1))
|
||||
fi
|
||||
|
||||
# Test Qualys signature:OpenSSL Multiple Remote Security Vulnerabilities
|
||||
qualys_vm_doc=$(curl -s "$elasticsearch_url/logstash-vulnwhisperer-*/_search?q=tags:qualys_vm%20AND%20ip:%22176.28.50.164%22%20AND%20signature:%22OpenSSL%20Multiple%20Remote%20Security%20Vulnerabilities%22%20AND%20port:465" | jq '.hits.hits[]._source')
|
||||
# Test Qualys plugin_name:OpenSSL Multiple Remote Security Vulnerabilities
|
||||
qualys_vuln_doc=$(curl -s "$elasticsearch_url/logstash-vulnwhisperer-2019.03/_search?q=tags:qualys_vuln%20AND%20ip:%22176.28.50.164%22%20AND%20plugin_name:%22OpenSSL%20Multiple%20Remote%20Security%20Vulnerabilities%22%20AND%20port:465" | jq '.hits.hits[]._source')
|
||||
# Test @timestamp
|
||||
if echo $qualys_vm_doc | jq '.["@timestamp"]' | grep -q '2019-03-30T10:17:41.000Z'; then
|
||||
if echo $qualys_vuln_doc | jq '.["@timestamp"]' | grep -q '2019-03-30T10:17:41.000Z'; then
|
||||
green "✅ Passed: Qualys VM @timestamp == 2019-03-30T10:17:41.000Z"
|
||||
else
|
||||
red "❌ Failed: Qualys VM @timestamp == 2019-03-30T10:17:41.000Z was: $(echo $qualys_vm_doc | jq '.["@timestamp"]') instead"
|
||||
red "❌ Failed: Qualys VM @timestamp == 2019-03-30T10:17:41.000Z was: $(echo $qualys_vuln_doc | jq '.["@timestamp"]') instead"
|
||||
((return_code = return_code + 1))
|
||||
fi
|
||||
|
||||
# Test @XXXX
|
||||
if echo $qualys_vm_doc | jq '.cvss' | grep -q '5.6'; then
|
||||
green "✅ Passed: Qualys VM cvss == 5.6"
|
||||
if echo $qualys_vuln_doc | jq '.cvss' | grep -q '6.8'; then
|
||||
green "✅ Passed: Qualys VM cvss == 6.8"
|
||||
else
|
||||
red "❌ Failed: Qualys VM cvss == 5.6 was: $(echo $qualys_vm_doc | jq '.cvss') instead"
|
||||
red "❌ Failed: Qualys VM cvss == 6.8 was: $(echo $qualys_vuln_doc | jq '.cvss') instead"
|
||||
((return_code = return_code + 1))
|
||||
fi
|
||||
|
||||
|
||||
@ -59,8 +59,8 @@ yellow "\n*********************************************"
|
||||
yellow "* Test two failed scans *"
|
||||
yellow "*********************************************"
|
||||
rm -rf /opt/VulnWhisperer/*
|
||||
yellow "Removing ${TEST_PATH}/qualys_vm/scan_1553941061.87241"
|
||||
mv "${TEST_PATH}/qualys_vm/scan_1553941061.87241"{,.bak}
|
||||
yellow "Removing ${TEST_PATH}/qualys_vuln/scan_1553941061.87241"
|
||||
mv "${TEST_PATH}/qualys_vuln/scan_1553941061.87241"{,.bak}
|
||||
if vuln_whisperer -F -c configs/test.ini --mock --mock_dir "${TEST_PATH}"; [[ $? -eq 2 ]]; then
|
||||
green "\n✅ Passed: Test two failed scans"
|
||||
else
|
||||
@ -79,11 +79,11 @@ else
|
||||
((return_code = return_code + 1))
|
||||
fi
|
||||
|
||||
yellow "\n*********************************************"
|
||||
yellow "*********************************************"
|
||||
yellow "* Test only Qualys VM with one failed scan *"
|
||||
yellow "*********************************************"
|
||||
rm -rf /opt/VulnWhisperer/*
|
||||
if vuln_whisperer -F -c configs/test.ini -s qualys_vm --mock --mock_dir "${TEST_PATH}"; [[ $? -eq 1 ]]; then
|
||||
if vuln_whisperer -F -c configs/test.ini -s qualys_vuln --mock --mock_dir "${TEST_PATH}"; [[ $? -eq 1 ]]; then
|
||||
green "\n✅ Passed: Test only Qualys VM with one failed scan"
|
||||
else
|
||||
red "\n❌ Failed: Test only Qualys VM with one failed scan"
|
||||
@ -91,7 +91,7 @@ else
|
||||
fi
|
||||
|
||||
# Restore the removed files
|
||||
mv "${TEST_PATH}/qualys_vm/scan_1553941061.87241.bak" "${TEST_PATH}/qualys_vm/scan_1553941061.87241"
|
||||
mv "${TEST_PATH}/qualys_vuln/scan_1553941061.87241.bak" "${TEST_PATH}/qualys_vuln/scan_1553941061.87241"
|
||||
mv "${TEST_PATH}/nessus/GET_scans_exports_164_download.bak" "${TEST_PATH}/nessus/GET_scans_exports_164_download"
|
||||
|
||||
exit $return_code
|
||||
|
||||
@ -2,7 +2,7 @@ import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import datetime
|
||||
|
||||
import pytz
|
||||
import requests
|
||||
@ -17,38 +17,21 @@ class NessusAPI(object):
|
||||
SCANS = '/scans'
|
||||
SCAN_ID = SCANS + '/{scan_id}'
|
||||
HOST_VULN = SCAN_ID + '/hosts/{host_id}'
|
||||
PLUGINS = HOST_VULN + '/plugins/{signature_id}'
|
||||
PLUGINS = HOST_VULN + '/plugins/{plugin_id}'
|
||||
EXPORT = SCAN_ID + '/export'
|
||||
EXPORT_TOKEN_DOWNLOAD = '/scans/exports/{token_id}/download'
|
||||
EXPORT_FILE_DOWNLOAD = EXPORT + '/{file_id}/download'
|
||||
EXPORT_STATUS = EXPORT + '/{file_id}/status'
|
||||
EXPORT_HISTORY = EXPORT + '?history_id={history_id}'
|
||||
# All column mappings should be lowercase
|
||||
COLUMN_MAPPING = {
|
||||
'cvss base score': 'cvss2_base',
|
||||
'cvss temporal score': 'cvss2_temporal',
|
||||
'cvss temporal vector': 'cvss2_temporal_vector',
|
||||
'cvss vector': 'cvss2_vector',
|
||||
'cvss3 base score': 'cvss3_base',
|
||||
'cvss3 temporal score': 'cvss3_temporal',
|
||||
'cvss3 temporal vector': 'cvss3_temporal_vector',
|
||||
'fqdn': 'dns',
|
||||
'host': 'asset',
|
||||
'ip address': 'ip',
|
||||
'name': 'signature',
|
||||
'os': 'operating_system',
|
||||
'plugin id': 'signature_id',
|
||||
'see also': 'exploitability',
|
||||
'system type': 'category',
|
||||
'vulnerability state': 'state'
|
||||
}
|
||||
|
||||
def __init__(self, hostname=None, port=None, username=None, password=None, verbose=True, profile=None, access_key=None, secret_key=None):
|
||||
self.logger = logging.getLogger('NessusAPI')
|
||||
self.logger.setLevel(logging.DEBUG if verbose else logging.INFO)
|
||||
if verbose:
|
||||
self.logger.setLevel(logging.DEBUG)
|
||||
if not all((username, password)) and not all((access_key, secret_key)):
|
||||
raise Exception('ERROR: Missing username, password or API keys.')
|
||||
|
||||
self.profile = profile
|
||||
self.user = username
|
||||
self.password = password
|
||||
self.api_keys = False
|
||||
@ -56,7 +39,6 @@ class NessusAPI(object):
|
||||
self.secret_key = secret_key
|
||||
self.base = 'https://{hostname}:{port}'.format(hostname=hostname, port=port)
|
||||
self.verbose = verbose
|
||||
self.profile = profile
|
||||
|
||||
self.session = requests.Session()
|
||||
self.session.verify = False
|
||||
@ -81,6 +63,9 @@ class NessusAPI(object):
|
||||
else:
|
||||
self.login()
|
||||
|
||||
self.scans = self.get_scans()
|
||||
self.scan_ids = self.get_scan_ids()
|
||||
|
||||
def login(self):
|
||||
auth = '{"username":"%s", "password":"%s"}' % (self.user, self.password)
|
||||
resp = self.request(self.SESSION, data=auth, json_output=False)
|
||||
@ -89,7 +74,7 @@ class NessusAPI(object):
|
||||
else:
|
||||
raise Exception('[FAIL] Could not login to Nessus')
|
||||
|
||||
def request(self, url, data=None, headers=None, method='POST', download=False, json_output=False, params=None):
|
||||
def request(self, url, data=None, headers=None, method='POST', download=False, json_output=False):
|
||||
timeout = 0
|
||||
success = False
|
||||
|
||||
@ -98,7 +83,7 @@ class NessusAPI(object):
|
||||
self.logger.debug('Requesting to url {}'.format(url))
|
||||
|
||||
while (timeout <= 10) and (not success):
|
||||
response = getattr(self.session, method)(url, data=data, params=params)
|
||||
response = getattr(self.session, method)(url, data=data)
|
||||
if response.status_code == 401:
|
||||
if url == self.base + self.SESSION:
|
||||
break
|
||||
@ -127,16 +112,12 @@ class NessusAPI(object):
|
||||
return response_data
|
||||
return response
|
||||
|
||||
def get_scans(self, days=None):
|
||||
parameters = {}
|
||||
if days != None:
|
||||
parameters = {
|
||||
"last_modification_date": (datetime.now() - timedelta(days=days)).strftime("%s")
|
||||
}
|
||||
scans = self.request(self.SCANS, method="GET", params=parameters, json_output=True)
|
||||
def get_scans(self):
|
||||
scans = self.request(self.SCANS, method='GET', json_output=True)
|
||||
return scans
|
||||
|
||||
def get_scan_ids(self, scans):
|
||||
def get_scan_ids(self):
|
||||
scans = self.scans
|
||||
scan_ids = [scan_id['id'] for scan_id in scans['scans']] if scans['scans'] else []
|
||||
self.logger.debug('Found {} scan_ids'.format(len(scan_ids)))
|
||||
return scan_ids
|
||||
@ -145,8 +126,10 @@ class NessusAPI(object):
|
||||
data = self.request(self.SCAN_ID.format(scan_id=scan_id), method='GET', json_output=True)
|
||||
return data['history']
|
||||
|
||||
def download_scan(self, scan_id=None, history=None, export_format=''):
|
||||
def download_scan(self, scan_id=None, history=None, export_format=""):
|
||||
running = True
|
||||
counter = 0
|
||||
|
||||
data = {'format': export_format}
|
||||
if not history:
|
||||
query = self.EXPORT.format(scan_id=scan_id)
|
||||
@ -160,12 +143,19 @@ class NessusAPI(object):
|
||||
token_id = req['token'] if 'token' in req else req['temp_token']
|
||||
except Exception as e:
|
||||
self.logger.error('{}'.format(str(e)))
|
||||
self.logger.info('Downloading file id {}'.format(str(file_id)))
|
||||
self.logger.info('Download for file id {}'.format(str(file_id)))
|
||||
while running:
|
||||
time.sleep(2)
|
||||
counter += 2
|
||||
report_status = self.request(self.EXPORT_STATUS.format(scan_id=scan_id, file_id=file_id), method='GET',
|
||||
json_output=True)
|
||||
running = report_status['status'] != 'ready'
|
||||
sys.stdout.write(".")
|
||||
sys.stdout.flush()
|
||||
# FIXME: why? can this be removed in favour of a counter?
|
||||
if counter % 60 == 0:
|
||||
self.logger.info("Completed: {}".format(counter))
|
||||
self.logger.info("Done: {}".format(counter))
|
||||
if self.profile == 'tenable' or self.api_keys:
|
||||
content = self.request(self.EXPORT_FILE_DOWNLOAD.format(scan_id=scan_id, file_id=file_id), method='GET', download=True)
|
||||
else:
|
||||
@ -175,7 +165,7 @@ class NessusAPI(object):
|
||||
def get_utc_from_local(self, date_time, local_tz=None, epoch=True):
|
||||
date_time = datetime.fromtimestamp(date_time)
|
||||
if local_tz is None:
|
||||
local_tz = pytz.timezone('US/Central')
|
||||
local_tz = pytz.timezone('UTC')
|
||||
else:
|
||||
local_tz = pytz.timezone(local_tz)
|
||||
local_time = local_tz.normalize(local_tz.localize(date_time))
|
||||
@ -192,43 +182,3 @@ class NessusAPI(object):
|
||||
'Pacific Standard Time': 'US/Pacific',
|
||||
'None': 'US/Central'}
|
||||
return time_map.get(tz, None)
|
||||
|
||||
def normalise(self, df):
|
||||
self.logger.debug('Normalising data')
|
||||
df = self.map_fields(df)
|
||||
df = self.transform_values(df)
|
||||
return df
|
||||
|
||||
def map_fields(self, df):
|
||||
self.logger.debug('Mapping fields')
|
||||
# Any specific mappings here
|
||||
if self.profile == 'tenable':
|
||||
# Prefer CVSS Base Score over CVSS for tenable
|
||||
self.logger.debug('Dropping redundant tenable fields')
|
||||
df.drop('CVSS', axis=1, inplace=True, errors='ignore')
|
||||
|
||||
# Lowercase and map fields from COLUMN_MAPPING
|
||||
df.columns = [x.lower() for x in df.columns]
|
||||
df.rename(columns=self.COLUMN_MAPPING, inplace=True)
|
||||
df.columns = [x.replace(' ', '_') for x in df.columns]
|
||||
|
||||
return df
|
||||
|
||||
def transform_values(self, df):
|
||||
self.logger.debug('Transforming values')
|
||||
|
||||
df.fillna('', inplace=True)
|
||||
|
||||
if self.profile == 'nessus':
|
||||
# Set IP from asset field
|
||||
df["ip"] = df.loc[df["asset"].str.match("^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$"), "asset"]
|
||||
|
||||
# upper/lowercase fields
|
||||
self.logger.debug('Changing case of fields')
|
||||
df['cve'] = df['cve'].str.upper()
|
||||
df['protocol'] = df['protocol'].str.lower()
|
||||
df['risk'] = df['risk'].str.lower()
|
||||
|
||||
df.fillna('', inplace=True)
|
||||
|
||||
return df
|
||||
|
||||
@ -13,20 +13,6 @@ from bs4 import BeautifulSoup
|
||||
|
||||
class OpenVAS_API(object):
|
||||
OMP = '/omp'
|
||||
COLUMN_MAPPING = {
|
||||
'affected software/os': 'affected_software',
|
||||
'cves': 'cve',
|
||||
'impact': 'description',
|
||||
'nvt name': 'signature',
|
||||
'nvt oid': 'signature_id',
|
||||
'other references': 'exploitability',
|
||||
'port protocol': 'protocol',
|
||||
'severity': 'risk',
|
||||
'solution type': 'category',
|
||||
'task name': 'scan_name',
|
||||
'specific result': 'plugin_output',
|
||||
'summary': 'synopsis',
|
||||
}
|
||||
|
||||
def __init__(self,
|
||||
hostname=None,
|
||||
@ -124,7 +110,6 @@ class OpenVAS_API(object):
|
||||
]
|
||||
token = requests.post(self.base + self.OMP, data=data, verify=False)
|
||||
return token
|
||||
|
||||
def get_report_formats(self):
|
||||
params = (
|
||||
('cmd', 'get_report_formats'),
|
||||
@ -205,25 +190,3 @@ class OpenVAS_API(object):
|
||||
self.processed_reports += 1
|
||||
merged_df = pd.merge(report_df, self.openvas_reports, on='report_ids').reset_index().drop('index', axis=1)
|
||||
return merged_df
|
||||
|
||||
def normalise(self, df):
|
||||
self.logger.debug('Normalising data')
|
||||
df = self.map_fields(df)
|
||||
df = self.transform_values(df)
|
||||
return df
|
||||
|
||||
def map_fields(self, df):
|
||||
self.logger.debug('Mapping fields')
|
||||
# Lowercase and map fields from COLUMN_MAPPING
|
||||
df.columns = [x.lower() for x in df.columns]
|
||||
df.rename(columns=self.COLUMN_MAPPING, inplace=True)
|
||||
df.columns = [x.replace(' ', '_') for x in df.columns]
|
||||
return df
|
||||
|
||||
def transform_values(self, df):
|
||||
self.logger.debug('Transforming values')
|
||||
df['port'].fillna(0).astype(int)
|
||||
df['risk'] = df['risk'].str.lower()
|
||||
df['asset'] = df['ip']
|
||||
df.fillna('', inplace=True)
|
||||
return df
|
||||
|
||||
@ -5,7 +5,6 @@ __author__ = 'Nathan Young'
|
||||
import logging
|
||||
import sys
|
||||
import xml.etree.ElementTree as ET
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import dateutil.parser as dp
|
||||
import pandas as pd
|
||||
@ -19,7 +18,7 @@ class qualysWhisperAPI(object):
|
||||
self.logger = logging.getLogger('qualysWhisperAPI')
|
||||
self.config = config
|
||||
try:
|
||||
self.qgc = qualysapi.connect(config, 'qualys_vm')
|
||||
self.qgc = qualysapi.connect(config, 'qualys_vuln')
|
||||
# Fail early if we can't make a request or auth is incorrect
|
||||
self.qgc.request('about.php')
|
||||
self.logger.info('Connected to Qualys at {}'.format(self.qgc.server))
|
||||
@ -29,9 +28,7 @@ class qualysWhisperAPI(object):
|
||||
|
||||
def scan_xml_parser(self, xml):
|
||||
all_records = []
|
||||
root = ET.XML(xml.encode('utf-8'))
|
||||
if len(root.find('.//SCAN_LIST')) == 0:
|
||||
return pd.DataFrame(columns=['id', 'status'])
|
||||
root = ET.XML(xml.encode("utf-8"))
|
||||
for child in root.find('.//SCAN_LIST'):
|
||||
all_records.append({
|
||||
'name': child.find('TITLE').text,
|
||||
@ -43,17 +40,12 @@ class qualysWhisperAPI(object):
|
||||
})
|
||||
return pd.DataFrame(all_records)
|
||||
|
||||
def get_all_scans(self, days=None):
|
||||
if days == None:
|
||||
self.launched_date = '0001-01-01'
|
||||
else:
|
||||
self.launched_date = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
|
||||
def get_all_scans(self):
|
||||
parameters = {
|
||||
'action': 'list',
|
||||
'echo_request': 0,
|
||||
'show_op': 0,
|
||||
'state': 'Finished',
|
||||
'launched_after_datetime': self.launched_date
|
||||
'launched_after_datetime': '0001-01-01'
|
||||
}
|
||||
scans_xml = self.qgc.request(self.SCANS, parameters)
|
||||
return self.scan_xml_parser(scans_xml)
|
||||
@ -86,17 +78,6 @@ class qualysUtils:
|
||||
|
||||
class qualysVulnScan:
|
||||
|
||||
COLUMN_MAPPING = {
|
||||
'cve_id': 'cve',
|
||||
'impact': 'synopsis',
|
||||
'ip_status': 'state',
|
||||
'os': 'operating_system',
|
||||
'qid': 'signature_id',
|
||||
'results': 'plugin_output',
|
||||
'threat': 'description',
|
||||
'title': 'signature'
|
||||
}
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
config=None,
|
||||
@ -141,51 +122,3 @@ class qualysVulnScan:
|
||||
return scan_report
|
||||
|
||||
return scan_report
|
||||
|
||||
def normalise(self, df):
|
||||
self.logger.debug('Normalising data')
|
||||
df = self.map_fields(df)
|
||||
df = self.transform_values(df)
|
||||
return df
|
||||
|
||||
def map_fields(self, df):
|
||||
self.logger.info('Mapping fields')
|
||||
|
||||
# Lowercase and map fields from COLUMN_MAPPING
|
||||
df.columns = [x.lower() for x in df.columns]
|
||||
df.rename(columns=self.COLUMN_MAPPING, inplace=True)
|
||||
df.columns = [x.replace(' ', '_') for x in df.columns]
|
||||
|
||||
return df
|
||||
|
||||
def transform_values(self, df):
|
||||
self.logger.info('Transforming values')
|
||||
|
||||
df.fillna('', inplace=True)
|
||||
|
||||
# upper/lowercase fields
|
||||
self.logger.info('Changing case of fields')
|
||||
df['cve'] = df['cve'].str.upper()
|
||||
df['protocol'] = df['protocol'].str.lower()
|
||||
|
||||
# Contruct the CVSS vector
|
||||
self.logger.info('Extracting CVSS components')
|
||||
df['cvss2_vector'] = df['cvss_base'].str.extract('\((.*)\)', expand=False)
|
||||
df['cvss2_base'] = df['cvss_base'].str.extract('^(\d+(?:\.\d+)?)', expand=False)
|
||||
df['cvss2_temporal_vector'] = df['cvss_temporal'].str.extract('\((.*)\)', expand=False)
|
||||
df['cvss2_temporal'] = df['cvss_temporal'].str.extract('^(\d+(?:\.\d+)?)', expand=False)
|
||||
df.drop('cvss_base', axis=1, inplace=True, errors='ignore')
|
||||
df.drop('cvss_temporal', axis=1, inplace=True, errors='ignore')
|
||||
|
||||
# Set asset to ip
|
||||
df['asset'] = df['ip']
|
||||
|
||||
# Set dns to fqdn if missing
|
||||
df.loc[df['dns'] == '', 'dns'] = df['fqdn']
|
||||
|
||||
# Convert Qualys severity to standardised risk number
|
||||
df['risk_number'] = df['severity'].astype(int)-1
|
||||
|
||||
df.fillna('', inplace=True)
|
||||
|
||||
return df
|
||||
@ -2,20 +2,18 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
__author__ = 'Austin Taylor'
|
||||
|
||||
import csv
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from lxml import objectify
|
||||
from lxml.builder import E
|
||||
import xml.etree.ElementTree as ET
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import dateutil.parser as dp
|
||||
import pandas as pd
|
||||
import qualysapi
|
||||
import qualysapi.config as qcconf
|
||||
import requests
|
||||
from lxml import objectify
|
||||
from lxml.builder import E
|
||||
import sys
|
||||
import os
|
||||
import csv
|
||||
import logging
|
||||
import dateutil.parser as dp
|
||||
|
||||
|
||||
class qualysWhisperAPI(object):
|
||||
@ -39,7 +37,7 @@ class qualysWhisperAPI(object):
|
||||
self.logger = logging.getLogger('qualysWhisperAPI')
|
||||
self.config = config
|
||||
try:
|
||||
self.qgc = qualysapi.connect(config, 'qualys_was')
|
||||
self.qgc = qualysapi.connect(config, 'qualys_web')
|
||||
self.logger.info('Connected to Qualys at {}'.format(self.qgc.server))
|
||||
except Exception as e:
|
||||
self.logger.error('Could not connect to Qualys: {}'.format(str(e)))
|
||||
@ -47,7 +45,7 @@ class qualysWhisperAPI(object):
|
||||
#"content-type": "text/xml"}
|
||||
"Accept" : "application/json",
|
||||
"Content-Type": "application/json"}
|
||||
self.config_parse = qcconf.QualysConnectConfig(config, 'qualys_was')
|
||||
self.config_parse = qcconf.QualysConnectConfig(config, 'qualys_web')
|
||||
try:
|
||||
self.template_id = self.config_parse.get_template_id()
|
||||
except:
|
||||
@ -61,12 +59,10 @@ class qualysWhisperAPI(object):
|
||||
"""
|
||||
Checks number of scans, used to control the api limits
|
||||
"""
|
||||
parameters = E.ServiceRequest(
|
||||
parameters = (
|
||||
E.ServiceRequest(
|
||||
E.filters(
|
||||
E.Criteria({"field": "status", "operator": "EQUALS"}, status),
|
||||
E.Criteria({"field": "launchedDate", "operator": "GREATER"}, self.launched_date)
|
||||
)
|
||||
)
|
||||
E.Criteria({'field': 'status', 'operator': 'EQUALS'}, status))))
|
||||
xml_output = self.qgc.request(self.COUNT_WASSCAN, parameters)
|
||||
root = objectify.fromstring(xml_output.encode('utf-8'))
|
||||
return root.count.text
|
||||
@ -74,8 +70,8 @@ class qualysWhisperAPI(object):
|
||||
def generate_scan_result_XML(self, limit=1000, offset=1, status='FINISHED'):
|
||||
report_xml = E.ServiceRequest(
|
||||
E.filters(
|
||||
E.Criteria({'field': 'status', 'operator': 'EQUALS'}, status),
|
||||
E.Criteria({"field": "launchedDate", "operator": "GREATER"}, self.launched_date)
|
||||
E.Criteria({'field': 'status', 'operator': 'EQUALS'}, status
|
||||
),
|
||||
),
|
||||
E.preferences(
|
||||
E.startFromOffset(str(offset)),
|
||||
@ -107,17 +103,13 @@ class qualysWhisperAPI(object):
|
||||
all_records.append(record)
|
||||
return pd.DataFrame(all_records)
|
||||
|
||||
|
||||
def get_all_scans(self, limit=1000, offset=1, status='FINISHED', days=None):
|
||||
if days == None:
|
||||
self.launched_date = '0001-01-01'
|
||||
else:
|
||||
self.launched_date = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
|
||||
def get_all_scans(self, limit=1000, offset=1, status='FINISHED'):
|
||||
qualys_api_limit = limit
|
||||
dataframes = []
|
||||
_records = []
|
||||
try:
|
||||
total = int(self.get_was_scan_count(status=status))
|
||||
self.logger.error('Already have WAS scan count')
|
||||
self.logger.info('Retrieving information for {} scans'.format(total))
|
||||
for i in range(0, total):
|
||||
if i % limit == 0:
|
||||
@ -128,8 +120,6 @@ class qualysWhisperAPI(object):
|
||||
_records.append(scan_info)
|
||||
self.logger.debug('Converting XML to DataFrame')
|
||||
dataframes = [self.xml_parser(xml) for xml in _records]
|
||||
if not dataframes:
|
||||
return pd.DataFrame(columns=['id'])
|
||||
except Exception as e:
|
||||
self.logger.error("Couldn't process all scans: {}".format(e))
|
||||
|
||||
@ -292,26 +282,13 @@ class qualysUtils:
|
||||
def iso_to_epoch(self, dt):
|
||||
return dp.parse(dt).strftime('%s')
|
||||
|
||||
def cleanser(self, _data):
|
||||
repls = (('\n', '|||'), ('\r', '|||'), (',', ';'), ('\t', '|||'))
|
||||
if _data:
|
||||
_data = reduce(lambda a, kv: a.replace(*kv), repls, str(_data))
|
||||
return _data
|
||||
|
||||
class qualysScanReport:
|
||||
|
||||
COLUMN_MAPPING = {
|
||||
'CVSS Base': 'cvss2_base',
|
||||
'CVSS Temporal': 'cvss2_temporal',
|
||||
'DescriptionCatSev': 'category_description',
|
||||
'DescriptionSeverity': 'synopsis',
|
||||
'Evidence #1': 'evidence',
|
||||
'Payload #1': 'payload',
|
||||
'QID': 'signature_id',
|
||||
'Request Headers #1': 'request_headers',
|
||||
'Request Method #1': 'request_method',
|
||||
'Request URL #1': 'request_url',
|
||||
'Response #1': 'plugin_output',
|
||||
'Title': 'signature',
|
||||
'Url': 'uri',
|
||||
'URL': 'url',
|
||||
'Vulnerability Category': 'type',
|
||||
}
|
||||
|
||||
# URL Vulnerability Information
|
||||
WEB_SCAN_VULN_BLOCK = list(qualysReportFields.VULN_BLOCK)
|
||||
WEB_SCAN_VULN_BLOCK.insert(WEB_SCAN_VULN_BLOCK.index('QID'), 'Detection ID')
|
||||
@ -320,16 +297,6 @@ class qualysScanReport:
|
||||
WEB_SCAN_VULN_HEADER[WEB_SCAN_VULN_BLOCK.index(qualysReportFields.CATEGORIES[0])] = \
|
||||
'Vulnerability Category'
|
||||
|
||||
# Add an alternative vulnerability header
|
||||
WEB_SCAN_VULN_BLOCK_ALT = WEB_SCAN_VULN_BLOCK[:]
|
||||
WEB_SCAN_VULN_BLOCK_ALT.insert(WEB_SCAN_VULN_BLOCK_ALT.index('First Time Detected'), 'Detection Date')
|
||||
remove_fields = ['Last Time Tested', 'Times Detected', 'First Time Detected', 'Last Time Detected']
|
||||
WEB_SCAN_VULN_BLOCK_ALT = [x for x in WEB_SCAN_VULN_BLOCK_ALT if x not in remove_fields]
|
||||
|
||||
WEB_SCAN_VULN_HEADER_ALT = WEB_SCAN_VULN_BLOCK_ALT[:]
|
||||
WEB_SCAN_VULN_HEADER_ALT[WEB_SCAN_VULN_BLOCK_ALT.index(qualysReportFields.CATEGORIES[0])] = \
|
||||
'Vulnerability Category'
|
||||
|
||||
WEB_SCAN_SENSITIVE_HEADER = list(WEB_SCAN_VULN_HEADER)
|
||||
WEB_SCAN_SENSITIVE_HEADER.insert(WEB_SCAN_SENSITIVE_HEADER.index('Url'
|
||||
), 'Content')
|
||||
@ -391,17 +358,6 @@ class qualysScanReport:
|
||||
self.WEB_SCAN_INFO_BLOCK],
|
||||
pop_last=True),
|
||||
columns=self.WEB_SCAN_VULN_HEADER)
|
||||
if len(dict_tracker['WEB_SCAN_VULN_BLOCK']) == 0:
|
||||
# Try alternative headers
|
||||
dict_tracker["WEB_SCAN_VULN_BLOCK"] = pd.DataFrame(
|
||||
self.utils.grab_section(
|
||||
report,
|
||||
self.WEB_SCAN_VULN_BLOCK_ALT,
|
||||
end=[self.WEB_SCAN_SENSITIVE_BLOCK, self.WEB_SCAN_INFO_BLOCK],
|
||||
pop_last=True,
|
||||
),
|
||||
columns=self.WEB_SCAN_VULN_HEADER_ALT,
|
||||
)
|
||||
dict_tracker['WEB_SCAN_SENSITIVE_BLOCK'] = pd.DataFrame(self.utils.grab_section(report,
|
||||
self.WEB_SCAN_SENSITIVE_BLOCK,
|
||||
end=[
|
||||
@ -467,6 +423,9 @@ class qualysScanReport:
|
||||
'Request Headers #1', 'Response #1', 'Evidence #1',
|
||||
'Description', 'Impact', 'Solution', 'Url', 'Content']
|
||||
|
||||
for col in columns_to_cleanse:
|
||||
merged_df[col] = merged_df[col].apply(self.utils.cleanser)
|
||||
|
||||
merged_df = merged_df.drop(['QID_y', 'QID_x'], axis=1)
|
||||
merged_df = merged_df.rename(columns={'Id': 'QID'})
|
||||
|
||||
@ -485,7 +444,7 @@ class qualysScanReport:
|
||||
return merged_df
|
||||
|
||||
def download_file(self, path='', file_id=None):
|
||||
report = self.qw.download_report(file_id).encode('utf-8')
|
||||
report = self.qw.download_report(file_id)
|
||||
filename = path + str(file_id) + '.csv'
|
||||
file_out = open(filename, 'w')
|
||||
for line in report.splitlines():
|
||||
@ -504,40 +463,3 @@ class qualysScanReport:
|
||||
merged_data.sort_index(axis=1, inplace=True)
|
||||
|
||||
return merged_data
|
||||
|
||||
def normalise(self, df):
|
||||
self.logger.debug('Normalising data')
|
||||
df = self.map_fields(df)
|
||||
df = self.transform_values(df)
|
||||
return df
|
||||
|
||||
def map_fields(self, df):
|
||||
self.logger.debug('Mapping fields')
|
||||
|
||||
df.rename(columns=self.COLUMN_MAPPING, inplace=True)
|
||||
|
||||
# Lowercase and map fields from COLUMN_MAPPING
|
||||
df.columns = [x.lower() for x in df.columns]
|
||||
df.columns = [x.replace(' ', '_') for x in df.columns]
|
||||
|
||||
return df
|
||||
|
||||
def transform_values(self, df):
|
||||
self.logger.debug('Transforming values')
|
||||
df.fillna('', inplace=True)
|
||||
|
||||
self.logger.info('Changing case of fields')
|
||||
df['cwe'] = df['cwe'].str.upper()
|
||||
|
||||
# Convert Qualys severity to standardised risk number
|
||||
df['risk_number'] = df['severity'].astype(int)-1
|
||||
|
||||
# Extract dns field from URL
|
||||
df['dns'] = df['url'].str.extract('https?://([^/]+)', expand=False)
|
||||
df['dns'] = df.loc[df['uri'] != '','uri'].str.extract('https?://([^/]+)', expand=False)
|
||||
|
||||
# Set asset to web_application_name
|
||||
df['asset'] = df['web_application_name']
|
||||
|
||||
df.fillna('', inplace=True)
|
||||
return df
|
||||
@ -68,17 +68,22 @@ class JiraAPI(object):
|
||||
self.logger.error("Error creating Ticket: component {} not found".format(component))
|
||||
return 0
|
||||
|
||||
new_issue = self.jira.create_issue(project=project,
|
||||
summary=title,
|
||||
description=desc,
|
||||
issuetype={'name': 'Bug'},
|
||||
labels=labels,
|
||||
components=components_ticket)
|
||||
try:
|
||||
new_issue = self.jira.create_issue(project=project,
|
||||
summary=title,
|
||||
description=desc,
|
||||
issuetype={'name': 'Bug'},
|
||||
labels=labels,
|
||||
components=components_ticket)
|
||||
|
||||
self.logger.info("Ticket {} created successfully".format(new_issue))
|
||||
self.logger.info("Ticket {} created successfully".format(new_issue))
|
||||
|
||||
if attachment_contents:
|
||||
self.add_content_as_attachment(new_issue, attachment_contents)
|
||||
if attachment_contents:
|
||||
self.add_content_as_attachment(new_issue, attachment_contents)
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error("Failed to create ticket on Jira Project '{}'. Error: {}".format(project, e))
|
||||
new_issue = False
|
||||
|
||||
return new_issue
|
||||
|
||||
@ -226,32 +231,44 @@ class JiraAPI(object):
|
||||
def ticket_get_unique_fields(self, ticket):
|
||||
title = ticket.raw.get('fields', {}).get('summary').encode("ascii").strip()
|
||||
ticketid = ticket.key.encode("ascii")
|
||||
assets = []
|
||||
try:
|
||||
affected_assets_section = ticket.raw.get('fields', {}).get('description').encode("ascii").split("{panel:title=Affected Assets}")[1].split("{panel}")[0]
|
||||
assets = list(set(re.findall(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b", affected_assets_section)))
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error("Ticket IPs regex failed. Ticket ID: {}. Reason: {}".format(ticketid, e))
|
||||
assets = []
|
||||
|
||||
try:
|
||||
if not assets:
|
||||
#check if attachment, if so, get assets from attachment
|
||||
affected_assets_section = self.check_ips_attachment(ticket)
|
||||
if affected_assets_section:
|
||||
assets = list(set(re.findall(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b", affected_assets_section)))
|
||||
except Exception as e:
|
||||
self.logger.error("Ticket IPs Attachment regex failed. Ticket ID: {}. Reason: {}".format(ticketid, e))
|
||||
assets = self.get_assets_from_description(ticket)
|
||||
if not assets:
|
||||
#check if attachment, if so, get assets from attachment
|
||||
assets = self.get_assets_from_attachment(ticket)
|
||||
|
||||
return ticketid, title, assets
|
||||
|
||||
def check_ips_attachment(self, ticket):
|
||||
affected_assets_section = []
|
||||
def get_assets_from_description(self, ticket, _raw = False):
|
||||
# Get the assets as a string "host - protocol/port - hostname" separated by "\n"
|
||||
# structure the text to have the same structure as the assets from the attachment
|
||||
affected_assets = ""
|
||||
try:
|
||||
affected_assets = ticket.raw.get('fields', {}).get('description').encode("ascii").split("{panel:title=Affected Assets}")[1].split("{panel}")[0].replace('\n','').replace(' * ','\n').replace('\n', '', 1)
|
||||
except Exception as e:
|
||||
self.logger.error("Unable to process the Ticket's 'Affected Assets'. Ticket ID: {}. Reason: {}".format(ticket, e))
|
||||
|
||||
if affected_assets:
|
||||
if _raw:
|
||||
# from line 406 check if the text in the panel corresponds to having added an attachment
|
||||
if "added as an attachment" in affected_assets:
|
||||
return False
|
||||
return affected_assets
|
||||
|
||||
try:
|
||||
# if _raw is not true, we return only the IPs of the affected assets
|
||||
return list(set(re.findall(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b", affected_assets)))
|
||||
except Exception as e:
|
||||
self.logger.error("Ticket IPs regex failed. Ticket ID: {}. Reason: {}".format(ticket, e))
|
||||
return False
|
||||
|
||||
def get_assets_from_attachment(self, ticket, _raw = False):
|
||||
# Get the assets as a string "host - protocol/port - hostname" separated by "\n"
|
||||
affected_assets = []
|
||||
try:
|
||||
fields = self.jira.issue(ticket.key).raw.get('fields', {})
|
||||
attachments = fields.get('attachment', {})
|
||||
affected_assets_section = ""
|
||||
affected_assets = ""
|
||||
#we will make sure we get the latest version of the file
|
||||
latest = ''
|
||||
attachment_id = ''
|
||||
@ -265,12 +282,44 @@ class JiraAPI(object):
|
||||
if latest < item.get('created'):
|
||||
latest = item.get('created')
|
||||
attachment_id = item.get('id')
|
||||
affected_assets_section = self.jira.attachment(attachment_id).get()
|
||||
affected_assets = self.jira.attachment(attachment_id).get()
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error("Failed to get assets from ticket attachment. Ticket ID: {}. Reason: {}".format(ticket, e))
|
||||
|
||||
return affected_assets_section
|
||||
if affected_assets:
|
||||
if _raw:
|
||||
return affected_assets
|
||||
|
||||
try:
|
||||
# if _raw is not true, we return only the IPs of the affected assets
|
||||
affected_assets = list(set(re.findall(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b", affected_assets)))
|
||||
return affected_assets
|
||||
except Exception as e:
|
||||
self.logger.error("Ticket IPs Attachment regex failed. Ticket ID: {}. Reason: {}".format(ticket, e))
|
||||
|
||||
return False
|
||||
|
||||
def parse_asset_to_json(self, asset):
|
||||
hostname, protocol, port = "", "", ""
|
||||
asset_info = asset.split(" - ")
|
||||
ip = asset_info[0]
|
||||
proto_port = asset_info[1]
|
||||
# in case there is some case where hostname is not reported at all
|
||||
if len(asset_info) == 3:
|
||||
hostname = asset_info[2]
|
||||
if proto_port != "N/A/N/A":
|
||||
protocol, port = proto_port.split("/")
|
||||
port = int(float(port))
|
||||
|
||||
asset_dict = {
|
||||
"host": ip,
|
||||
"protocol": protocol,
|
||||
"port": port,
|
||||
"hostname": hostname
|
||||
}
|
||||
|
||||
return asset_dict
|
||||
|
||||
def clean_old_attachments(self, ticket):
|
||||
fields = ticket.raw.get('fields')
|
||||
@ -441,7 +490,7 @@ class JiraAPI(object):
|
||||
if transition.get('name') == self.JIRA_REOPEN_ISSUE:
|
||||
self.logger.debug("Ticket is reopenable")
|
||||
return True
|
||||
self.logger.warn("Ticket can't be opened. Check Jira transitions.")
|
||||
self.logger.error("Ticket {} can't be opened. Check Jira transitions.".format(ticket_obj))
|
||||
return False
|
||||
|
||||
def is_ticket_closeable(self, ticket_obj):
|
||||
@ -449,7 +498,7 @@ class JiraAPI(object):
|
||||
for transition in transitions:
|
||||
if transition.get('name') == self.JIRA_CLOSE_ISSUE:
|
||||
return True
|
||||
self.logger.warn("Ticket can't closed. Check Jira transitions.")
|
||||
self.logger.error("Ticket {} can't closed. Check Jira transitions.".format(ticket_obj))
|
||||
return False
|
||||
|
||||
def is_ticket_resolved(self, ticket_obj):
|
||||
@ -522,7 +571,7 @@ class JiraAPI(object):
|
||||
def close_obsolete_tickets(self):
|
||||
# Close tickets older than 12 months, vulnerabilities not solved will get created a new ticket
|
||||
self.logger.info("Closing obsolete tickets older than {} months".format(self.max_time_tracking))
|
||||
jql = "labels=vulnerability_management AND created <startOfMonth(-{}) and resolution=Unresolved".format(self.max_time_tracking)
|
||||
jql = "labels=vulnerability_management AND NOT labels=advisory AND created <startOfMonth(-{}) and resolution=Unresolved".format(self.max_time_tracking)
|
||||
tickets_to_close = self.jira.search_issues(jql, maxResults=0)
|
||||
|
||||
comment = '''This ticket is being closed for hygiene, as it is more than {} months old.
|
||||
@ -553,9 +602,36 @@ class JiraAPI(object):
|
||||
return True
|
||||
try:
|
||||
self.logger.info("Saving locally tickets from the last {} months".format(self.max_time_tracking))
|
||||
jql = "labels=vulnerability_management AND created >=startOfMonth(-{})".format(self.max_time_tracking)
|
||||
jql = "labels=vulnerability_management AND NOT labels=advisory AND created >=startOfMonth(-{})".format(self.max_time_tracking)
|
||||
tickets_data = self.jira.search_issues(jql, maxResults=0)
|
||||
|
||||
#TODO process tickets, creating a new field called "_metadata" with all the affected assets well structured
|
||||
# for future processing in ELK/Splunk; this includes downloading attachments with assets and processing them
|
||||
|
||||
processed_tickets = []
|
||||
|
||||
for ticket in tickets_data:
|
||||
assets = self.get_assets_from_description(ticket, _raw=True)
|
||||
if not assets:
|
||||
# check if attachment, if so, get assets from attachment
|
||||
assets = self.get_assets_from_attachment(ticket, _raw=True)
|
||||
# process the affected assets to save them as json structure on a new field from the JSON
|
||||
_metadata = {"affected_hosts": []}
|
||||
if assets:
|
||||
if "\n" in assets:
|
||||
for asset in assets.split("\n"):
|
||||
assets_json = self.parse_asset_to_json(asset)
|
||||
_metadata["affected_hosts"].append(assets_json)
|
||||
else:
|
||||
assets_json = self.parse_asset_to_json(assets)
|
||||
_metadata["affected_hosts"].append(assets_json)
|
||||
|
||||
|
||||
temp_ticket = ticket.raw.get('fields')
|
||||
temp_ticket['_metadata'] = _metadata
|
||||
|
||||
processed_tickets.append(temp_ticket)
|
||||
|
||||
#end of line needed, as writelines() doesn't add it automatically, otherwise one big line
|
||||
to_save = [json.dumps(ticket.raw.get('fields'))+"\n" for ticket in tickets_data]
|
||||
with open(fname, 'w') as outfile:
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
import logging
|
||||
import os
|
||||
|
||||
import logging
|
||||
import httpretty
|
||||
|
||||
|
||||
@ -17,36 +16,26 @@ class mockAPI(object):
|
||||
self.logger.setLevel(logging.DEBUG)
|
||||
|
||||
self.logger.info('mockAPI initialised, API requests will be mocked')
|
||||
self.logger.info('Test path resolved as {}'.format(self.mock_dir))
|
||||
self.logger.debug('Test path resolved as {}'.format(self.mock_dir))
|
||||
|
||||
def get_directories(self, path):
|
||||
dir, subdirs, files = next(os.walk(path))
|
||||
return sorted(subdirs)
|
||||
return subdirs
|
||||
|
||||
def get_files(self, path):
|
||||
dir, subdirs, files = next(os.walk(path))
|
||||
return sorted(files)
|
||||
return files
|
||||
|
||||
def create_nessus_resource(self, framework):
|
||||
for filename in self.get_files('{}/{}'.format(self.mock_dir, framework)):
|
||||
method, resource = filename.split('_', 1)
|
||||
resource = resource.replace('_', '/')
|
||||
self.logger.info('Adding mocked {} endpoint {} {}'.format(framework, method, resource))
|
||||
httpretty.register_uri(
|
||||
getattr(httpretty, method), 'https://{}:443/{}'.format(framework, resource),
|
||||
body=open('{}/{}/{}'.format(self.mock_dir, framework, filename)).read()
|
||||
)
|
||||
|
||||
def qualys_vm_callback(self, request, uri, response_headers):
|
||||
self.logger.info('Simulating response for {} ({})'.format(uri, request.body))
|
||||
def qualys_vuln_callback(self, request, uri, response_headers):
|
||||
self.logger.debug('Simulating response for {} ({})'.format(uri, request.body))
|
||||
if 'list' in request.parsed_body['action']:
|
||||
return [200,
|
||||
response_headers,
|
||||
open(self.qualys_vm_path + '/scans').read()]
|
||||
open('{}/{}'.format(self.qualys_vuln_path, 'scans')).read()]
|
||||
elif 'fetch' in request.parsed_body['action']:
|
||||
try:
|
||||
response_body = open('{}/{}'.format(
|
||||
self.qualys_vm_path,
|
||||
self.qualys_vuln_path,
|
||||
request.parsed_body['scan_ref'][0].replace('/', '_'))
|
||||
).read()
|
||||
except:
|
||||
@ -54,75 +43,34 @@ class mockAPI(object):
|
||||
response_body = ''
|
||||
return [200, response_headers, response_body]
|
||||
|
||||
def create_qualys_vm_resource(self, framework):
|
||||
def create_nessus_resource(self, framework):
|
||||
for filename in self.get_files('{}/{}'.format(self.mock_dir, framework)):
|
||||
method, resource = filename.split('_', 1)
|
||||
resource = resource.replace('_', '/')
|
||||
self.logger.debug('Adding mocked {} endpoint {} {}'.format(framework, method, resource))
|
||||
httpretty.register_uri(
|
||||
getattr(httpretty, method), 'https://{}:443/{}'.format(framework, resource),
|
||||
body=open('{}/{}/{}'.format(self.mock_dir, framework, filename)).read()
|
||||
)
|
||||
|
||||
def create_qualys_vuln_resource(self, framework):
|
||||
# Create health check endpoint
|
||||
self.logger.info('Adding mocked {} endpoint GET msp/about.php'.format(framework))
|
||||
self.logger.debug('Adding mocked {} endpoint {} {}'.format(framework, 'GET', 'msp/about.php'))
|
||||
httpretty.register_uri(
|
||||
httpretty.GET,
|
||||
'https://{}:443/msp/about.php'.format(framework),
|
||||
'https://{}:443/{}'.format(framework, 'msp/about.php'),
|
||||
body='')
|
||||
|
||||
self.logger.info('Adding mocked {} endpoint {} {}'.format(framework, 'POST', 'api/2.0/fo/scan'))
|
||||
self.logger.debug('Adding mocked {} endpoint {} {}'.format(framework, 'POST', 'api/2.0/fo/scan'))
|
||||
httpretty.register_uri(
|
||||
httpretty.POST, 'https://{}:443/api/2.0/fo/scan/'.format(framework),
|
||||
body=self.qualys_vm_callback)
|
||||
|
||||
def qualys_was_callback(self, request, uri, response_headers):
|
||||
self.logger.info('Simulating response for {} ({})'.format(uri, request.body))
|
||||
report_id = request.parsed_body.split('<WasScan><id>')[1].split('<')[0]
|
||||
response_body = open('{}/create_{}'.format(self.qualys_was_path, report_id)).read()
|
||||
return [200, response_headers, response_body]
|
||||
|
||||
def create_qualys_was_resource(self, framework):
|
||||
for filename in self.get_files('{}/{}'.format(self.mock_dir, framework)):
|
||||
if filename.startswith('POST') or filename.startswith('GET'):
|
||||
method, resource = filename.split('_', 1)
|
||||
resource = resource.replace('_', '/')
|
||||
self.logger.info('Adding mocked {} endpoint {} {}'.format(framework, method, resource))
|
||||
httpretty.register_uri(
|
||||
getattr(httpretty, method), 'https://{}:443/{}'.format(framework, resource),
|
||||
body=open('{}/{}/{}'.format(self.mock_dir, framework, filename)).read()
|
||||
)
|
||||
|
||||
self.logger.info('Adding mocked {} endpoint {} {}'.format(framework, 'POST', 'qps/rest/3.0/create/was/report'))
|
||||
httpretty.register_uri(
|
||||
httpretty.POST, 'https://{}:443/qps/rest/3.0/create/was/report'.format(framework),
|
||||
body=self.qualys_was_callback)
|
||||
|
||||
def openvas_callback(self, request, uri, response_headers):
|
||||
self.logger.info('Simulating response for {} ({})'.format(uri, request.body))
|
||||
if request.querystring['cmd'][0] in ['get_reports', 'get_report_formats']:
|
||||
response_body = open('{}/{}'.format(self.openvas_path, request.querystring['cmd'][0])).read()
|
||||
|
||||
if request.querystring['cmd'][0] == 'get_report':
|
||||
response_body = open('{}/report_{}'.format(self.openvas_path, request.querystring['report_id'][0])).read()
|
||||
|
||||
return [200, response_headers, response_body]
|
||||
|
||||
def create_openvas_resource(self, framework):
|
||||
# Create login endpoint
|
||||
httpretty.register_uri(
|
||||
httpretty.POST, 'https://{}:4000/omp'.format(framework),
|
||||
body=open('{}/{}/{}'.format(self.mock_dir, framework, 'login')).read()
|
||||
)
|
||||
|
||||
# Create GET requests endpoint
|
||||
httpretty.register_uri(
|
||||
httpretty.GET, 'https://{}:4000/omp'.format(framework),
|
||||
body=self.openvas_callback
|
||||
)
|
||||
httpretty.POST, 'https://{}:443/{}'.format(framework, 'api/2.0/fo/scan/'),
|
||||
body=self.qualys_vuln_callback)
|
||||
|
||||
def mock_endpoints(self):
|
||||
for framework in self.get_directories(self.mock_dir):
|
||||
if framework in ['nessus', 'tenable']:
|
||||
self.create_nessus_resource(framework)
|
||||
elif framework == 'qualys_vm':
|
||||
self.qualys_vm_path = self.mock_dir + '/' + framework
|
||||
self.create_qualys_vm_resource(framework)
|
||||
elif framework == 'qualys_was':
|
||||
self.qualys_was_path = self.mock_dir + '/' + framework
|
||||
self.create_qualys_was_resource(framework)
|
||||
elif framework == 'openvas':
|
||||
self.openvas_path = self.mock_dir + '/' + framework
|
||||
self.create_openvas_resource(framework)
|
||||
elif framework == 'qualys_vuln':
|
||||
self.qualys_vuln_path = self.mock_dir + '/' + framework
|
||||
self.create_qualys_vuln_resource(framework)
|
||||
httpretty.enable()
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user