354 Commits

Author SHA1 Message Date
03f7a4cedb fixed line 2019-04-04 11:05:39 +02:00
a30a22ab98 fix wrong parenthesis on qualys was 2019-04-03 15:15:31 +02:00
f33644b814 fix reported tracking for jira 2019-04-02 11:58:44 +02:00
fa0b3c867b added tracking of scans processed by jira, will only process if new scans now (backwards compatibility 2019-04-01 15:55:02 +02:00
e32c9bf55d Fix *some* unicode issues for nessus and qualys (#160)
* Fix *some* unicode issues for nessus and qualys

* More unicode fixes
2019-04-01 10:06:16 +02:00
9619a47d7a Fix Tenable and Nessus scan listing (#162)
* Prevent multiple requests to nessus scans endpoint

* Remove unnecessary call
2019-04-01 10:04:12 +02:00
383e7f5478 Fix closing logging handlers (#159) 2019-04-01 09:07:29 +02:00
3601ace5e1 improved file logging format 2019-03-22 10:42:30 +01:00
97e4f073bf added logging to file 2019-03-22 10:38:55 +01:00
a4b1b9cdd4 fixed issue where, asset after a removed one, was ignored due to python listing 2019-03-21 15:52:18 +01:00
843aac6a83 fixing issue with new vulns of already risk accepted issues not being reported anymore; now, new ticket is raised, excluding all the assets that have been previously considered risk accepted in another ticket 2019-03-20 16:37:50 +01:00
47df1ee538 typo 2019-03-20 10:55:54 +01:00
a4420b7df8 reverse unintended change on frameworks_example.ini 2019-03-20 09:11:18 +01:00
9d52596be9 fix xml encoding issue #156 2019-03-20 08:49:36 +01:00
5cdb2552f0 Merge branch 'beta-1.8' of https://github.com/HASecuritySolutions/VulnWhisperer into beta-1.8 2019-03-20 08:35:32 +01:00
70e1d7703f fix missing section specification on qualys was connector #156 2019-03-20 08:35:03 +01:00
2d3a140042 fix bug 2019-03-19 15:19:27 +01:00
936c4a3e1b added automatic jira server_decommission label removal after x time 2019-03-19 12:58:38 +01:00
e7bd4d2a55 deleting dependency and pulling qualysapi official library, vulnwhisperer compatible 2019-03-15 12:03:02 +01:00
401dfec2c8 fix #143, added a temporary container to upload through kibana API 2019-03-04 15:10:51 +01:00
86e792f5aa workaround regarding ignoring ticket updates after risk accepted 2019-03-01 15:18:49 +01:00
a288f416f7 added label *false positive* for reporting on jira 2019-02-27 18:06:16 +01:00
623c881928 fix jira issue index when comparing created tickets 2019-02-27 11:27:44 +01:00
4e94bef245 fix bug not detecting existent label due to string format 2019-02-26 15:26:14 +01:00
a3da41e487 added to readme openvas supported versions 2019-02-26 09:59:50 +01:00
46ddee391b confirm openvas 9 works 2019-02-25 22:09:29 +01:00
b36e31566e fix #142 2019-02-25 22:02:20 +01:00
05420ddfd0 readding docker-compose credentials template 2019-02-25 12:32:32 +01:00
bdbe31d425 resources reorg 2 2019-02-25 12:29:00 +01:00
f170dcb05f reorg resources files 2019-02-25 12:27:30 +01:00
5dd6503d38 Merge branch 'beta-1.8' of https://github.com/HASecuritySolutions/VulnWhisperer into beta-1.8 2019-02-25 12:09:46 +01:00
2c7965d2d9 fix #151 2019-02-25 12:08:04 +01:00
521184d079 Update bug_report.md
added debug trail request
2019-02-21 22:20:19 +01:00
c2d80c7fce made host resolution optional from the config file with dns_resolv var 2019-02-15 16:24:52 +01:00
587546a726 fix typo 2019-02-14 14:16:31 +01:00
177c2548ba allow jira sync module to run after the rest 2019-02-12 18:18:24 +01:00
bc3367e310 exception of empty scans 2019-02-12 18:01:46 +01:00
8c53987270 tracking of processing was in debug instead of info logging 2019-02-12 16:56:00 +01:00
ccf2e4b1d1 fix #147 2019-02-12 16:51:26 +01:00
b0caccdc89 fixed issues plus jira comment formatting 2019-02-12 16:25:28 +01:00
4ea384c9cc fix issue #110 (one line) 2019-02-08 10:56:32 +01:00
699fc75446 Update README.md
Nessus v8 also supported
2019-02-08 09:10:04 +01:00
53dc65e492 fix qualysapi library dependencies 2019-02-08 09:08:21 +01:00
0ea144bf87 Qualysapi fix (#146)
* moved qualysapi to branch master-update

* fixing bug of qualys scan without vulnerabilities: vulnWhispererQualysVuln[1361] ERROR Could not process scan/1549159480.84792: 'severity'

* change to fixed qualysapi branch

* fix bug and changed to qualysapi fork master branch

* updated submodule to master branch
2019-02-06 17:00:43 +01:00
14b71a25b8 Created the version 6 for ELK. Fixed #135 (#145)
* Created the version 6 for ELK. Fixed #135

* Needed to make sure all the data volumes were set up properly.  Some paths had VulnWhisperer, vulnwhisperer, vulnwhisp/data.

* Delete 9998_output_broker_rabbitmq.conf

* Delete 9998_input_broker_rabbitmq.conf

* Delete 0001_input_beats.conf

* add to gitignore creds files + correct elk5 docker-compose

* elk changed to 6.6.0 from 6.5.2, output path from logstash to elasticsearch host
2019-02-05 17:30:51 +01:00
3cd13229a3 Update issue templates (#144)
* Update issue templates

Add an issue template for bug reports

* Update bug_report.md

Changing the "Desktop" label to "System in which VulnWhisperer runs"
2019-02-01 11:01:49 +01:00
177d384353 Fixed #134 (#139) 2019-01-15 23:57:09 -05:00
b1404cf0be change ./dep/qualysapi origin to https due to Github complains 2018-12-14 15:47:11 +01:00
48b17c5cbe Add a Dockerfile (#132)
* updating my base to match original vulnwhisperer (#1)

* Create docker-compose.yml

* Update 9000_output_nessus.conf

* Added an argument for username and password, which takes precendece over nessus.  Fixed #5

* Update README.md

* Silence NoneType object

* Put in a check to make sure that the config file exists.  FIXES austin-taylor/VulnWhisperer#4

* remove leading and trailing spaces around all input switches. Fixes austi-taylor/VulnWhisperer#6

* Update README.md

* Allow for any directories to be monitored

* Addition of Qualys WebApp Processing

* Addition of Qualys WebApp Processing

* Fixed multiple bugs, cleaned up formatting, produces solid csv output for Qualys Web App scans

* Adding custom version of QualysAPI

* Field Cleanup

* Addition of submodules, update to connectors, base class start

* Addition of submodules, update to connectors, base class start

* Addition of submodules, update to connectors, base class start

* Refactored classes to be more modular, update to ini file and submodules

* Refactored classes to be more modular, update to ini file and submodules

* Removing commented code

* Addition of category class and special class for Qualys Scanning Reports. Also added additional enrichments to reports

* Column update for scans and N/A cleanup

* Fix for str casting

* Update README.md

* Update to README

* Update to README

* Update to README

* Update to requirements.txt

* Support for json output

* Database tracking for processed Qualys scans

* Database tracking for processed Qualys scans

* Bug fix for counter in Nessus and format fix for qualys

* Check for new records

* Update to count tracker

* Update to write path logic

* Better database handling

* Addition of VulnWhisperer-Qualys logstash files

* Addition of VulnWhisperer-Qualys logstash files

* Update to logstash template

* Updated dashboard

* Update to README

* Update to README

* Logo update

* Readme Update

* Readme Update

* Readme Update

* Adding name of scan and scan reference

* Plugin name converted to scan name

* Update to README

* Documentation update

* README Update

* README Update

* Update README.md

* Add free automated flake8 testing of pull requests

[Flake8](http://flake8.pycqa.org) tests can help to find Python syntax errors, undefined names, and other code quality issues.  [Travis CI](https://travis-ci.org) is a framework for running flake8 and other tests which is free for open source projects like this one.  The owner of the this repo would need to go to https://travis-ci.org/profile and flip the repository switch __on__ to enable free automated flake8 testing of each pull request.

* Testing build with no submodules

* flake8 --ignore=deps/qualysapi

* flake8 . --exclude=/deps/qualysapi

* Remove leading slash

* Add build status to README

* Travis Config update

* README Update

* README Update

* Create CNAME

* Set theme jekyll-theme-leap-day

* README Update

* Getting started steps

* Getting started steps

* Remind user to select section if using a config

* Update to readme

* Update to readme

* Update to readme

* Update to readme

* Update to README

* Update to README

* Update to example logstash config

* Update to qualys logstash conf to reflect example config

* Update to README

* Update to README

* Readme update

* Rename logstash-nessus-template.json to logstash-vulnwhisperer-template.json

* Update 1000_nessus_process_file.conf

* Delete LICENSE

* Create LICENSE

* Update to make nessus visualizations consistent with qualys

* Update to README

* Update to README

* Badge addition

* Badge addition

* Addition of OpenVAS Connector

* Addition of OpenVAS

* Update 9000_output_nessus.conf

* Delete 9000_output_nessus.conf

* Update 1000_nessus_process_file.conf

* Automatically create filepath and directory if it does not exist

* Addition of OpenVas -- ready for alpha

* Addition of OpenVas -- ready for alpha

* Allow template defined config form IDs

* Completion of OpenVAS module

* Completion of OpenVAS module

* Remove template format

* Addition of openvas logstash config

* Update setup.py

* Update README.md

* ELK Sample Install (#37)

Updated Readme.md to include a Sample ELK Install guide addressing multiple issues around ELK Cluster/Node Configuration.

* Update vulnwhisp.py

* VulnFramework Links (#39)

Quick update regarding issue #33

* Updating config to be consistent with conf files

*  Preserving newlines & carriage returns  (#48)

* Preserve newlines & carriage returns

* Convert '\n' & '\r' to newlines & carriage returns

* Removed no longer supported InsecureRequestWarning workaround. (#55)

* Removed no longer supported InsecureRequestWarning workaround.

* Add dependencies to README.md

* Update vulnwhisp.py

* Fix to apt-get install

* Nessus bugfixes (#68)

* Handle cases where no scans are present

* Prevent infinite login loop with incorrect creds

* Print actual config file path

* Don't overwrite Nessus Synopsis with Description

* Tenable.io support (#70)

* Basic tenable.io support

* Add tenable config section

* Use existing variable

* Fix indent

* Fix paren

* Use ternary syntax

* Update Logstash config for tenable.io

* Update README.md

* Update template to version 5.x (#73)

* Update template to Elasticsearch 5.x

* Update template to Elasticsearch 5.x

I think _all field is no longer needed from ES 5.x because of the search all field execution if _all is disabled

* Qualys Vulnerability Management integration (#74)

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Fix error: "Cannot convert non-finite values (NA or inf) to integer"

When trying to download the results of Qualys Vulnerability Management scans, the following error pops up:

[FAIL] - Could not process scan/xxxxxxxxxx.xxxxx - Cannot convert non-finite values (NA or inf) to integer

This error is due to pandas operating with the scan results json file, as the last element from the json doesn't fir with the rest of the response's scheme: that element is "target_distribution_across_scanner_appliances", which contains the scanners used and the IP ranges that each scanner went through.

Taking out the last line solves the issue.

Also adding the qualys_vuln scheme to the frameworks_example.ini

* Update README.md

* example.ini is frameworks_example.ini (#77)

* No need to specify section to run (#88)

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Fix error: "Cannot convert non-finite values (NA or inf) to integer"

When trying to download the results of Qualys Vulnerability Management scans, the following error pops up:

[FAIL] - Could not process scan/xxxxxxxxxx.xxxxx - Cannot convert non-finite values (NA or inf) to integer

This error is due to pandas operating with the scan results json file, as the last element from the json doesn't fir with the rest of the response's scheme: that element is "target_distribution_across_scanner_appliances", which contains the scanners used and the IP ranges that each scanner went through.

Taking out the last line solves the issue.

Also adding the qualys_vuln scheme to the frameworks_example.ini

* No need to specify section to run

Until now it vulnwhisperer was not running if a section was not specified,
but there is the variable "enabled" on each module config, so now it will
check which modules are enabled and run them sequentialy.

Made mainly in order to be able to automate with docker-compose instance,
as the docker with vulnwhisperer (https://github.com/HASecuritySolutions/docker_vulnwhisperer)
has that command run at the end.

* added to readme + detectify

* Silence requests warnings

* Docker-compose fully working with vulnwhisperer integrated (#90)

* ignore nessus requests warnings

* docker-compose fully working with vulnwhisperer integrated

* remove comments docker-compose

* documenting docker-compose

* Readme corrections

* fix after recheck everything works out of the box

* fix exits that break the no specified section execution mode

* fix docker qualysapi issue, updated README

* revert change on deps/qualysapi/qualysapi/util.py (no effect)

* temporarily changed Dockerfile link to the working one

* Update README.md

* Update README.md

* Fix docker-compose logstash config (#92)

* ignore nessus requests warnings

* docker-compose fully working with vulnwhisperer integrated

* remove comments docker-compose

* documenting docker-compose

* Readme corrections

* fix after recheck everything works out of the box

* fix exits that break the no specified section execution mode

* fix docker qualysapi issue, updated README

* revert change on deps/qualysapi/qualysapi/util.py (no effect)

* temporarily changed Dockerfile link to the working one

* fix docker-compose logstash config

* permissions needed for logstash container to work

* changing default path qualys, there are no folders

* Update 1000_vulnWhispererBaseVisuals.json

Update field to include keyword to prevent error: TypeError: "field" is a required parameter

* Update docker-compose.yml (#93)

increase file descriptors to allow elasticsearch to start.

* Update Slack link on README.md

* Update README.md

Added to README.md @pemontto as contributor

* Jira module fully working (#104)

* clean OS X .DS_Store files

* fix nessus end of line carriage, added JIRA args

* JIRA module fully working

* jira module working with nessus

* added check on already existing jira config, update README

* qualys_vm<->jira working, qualys_vm database entries with qualys_vm, improved checks

* JIRA module updates ticket's assets and comments update

* added JIRA auto-close function for resolved vulnerabitilies

* fix if components variable empty issue

* fix creation of new ticket after updating existing one

* final fixes, added extra line in template

* added vulnerability criticality as label in order to be able to filter

* Added jira section to config file and fail check for config variable (#105)

* clean OS X .DS_Store files

* fix nessus end of line carriage, added JIRA args

* JIRA module fully working

* jira module working with nessus

* added check on already existing jira config, update README

* qualys_vm<->jira working, qualys_vm database entries with qualys_vm, improved checks

* JIRA module updates ticket's assets and comments update

* added JIRA auto-close function for resolved vulnerabitilies

* fix if components variable empty issue

* fix creation of new ticket after updating existing one

* final fixes, added extra line in template

* added vulnerability criticality as label in order to be able to filter

* jira module gets now minimum criticality from config file

* added jira config to frameworks_example.ini

* fail check for config variable in case it is left empty

* fix issue jira-qualys criticality comparison

* update qualysapi to latest + PR and refactored vulnwhisperer qualys module to qualys-web (#108)

* update qualysapi to latest + PR and refactored vulnwhisperer qualys module to qualys-web

* changing config template paths for qualys

* Update frameworks_example.ini

Will leave for now qualys local folder as "qualys" instead of changing to one for each module, as like this it will still be compatible with the current logstash and we will be able to update master to drop the qualysapi fork once the new version is uploaded to PyPI repository.
PR from qualysapi repo has already been merged, so the only missing is the upload to PyPI.

* Rework logging using the stdlib machinery (#116)

* Rework logging using the stdlib machinery
Use the verbose or debug flag to enable/disable logging.DEBUG
Remove the vprint function from all classes
Remove bcolors from all code
Cleanup [INFO], [ERROR], {success} and similar

* fix some errors my local linter missed but travis catched

* add coloredlogs and --fancy command line flag

* qualysapi dependency removal

* Qualysapi update (#118)

* update qualysapi to latest + PR and refactored vulnwhisperer qualys module to qualys-web

* changing config template paths for qualys

* Update frameworks_example.ini

Will leave for now qualys local folder as "qualys" instead of changing to one for each module, as like this it will still be compatible with the current logstash and we will be able to update master to drop the qualysapi fork once the new version is uploaded to PyPI repository.
PR from qualysapi repo has already been merged, so the only missing is the upload to PyPI.

* delete qualysapi fork and added to requirements

* merge with testing

* Jira extras (#120)

* changing config template paths for qualys

* Update frameworks_example.ini

Will leave for now qualys local folder as "qualys" instead of changing to one for each module, as like this it will still be compatible with the current logstash and we will be able to update master to drop the qualysapi fork once the new version is uploaded to PyPI repository.
PR from qualysapi repo has already been merged, so the only missing is the upload to PyPI.

* initialize variable fullpath to avoid break

* fix get latest scan entry from db and ignore 'potential' not verified vulns

* added host resolv + cache to speed already resolved, jira logging

* make sure that vulnerability criticality appears as a label on ticket + automatic actions

* jira bulk report of scans, fix on nessus logging, jira time resolution and list all ticket reported assets

* added jira ticket data download + change default time window from 6 to 12 months

* small fixes

* jira logstash files

* fix variable confusion (thx Travis :)

* update readme (#121)

* Add ansible provisioning (#122)

* first ansible skeleton

* first commit of ansible installation of vulnwhisperer outside docker

* first ansible skeleton

* first commit of ansible installation of vulnwhisperer outside docker

* refactor the ansible role a bit

* update readme, add fail validation step to provision.yml and fix
typo when calling a logging funciton

* removing ansible from vulnwhisperer, creating a new repo for ansible deployment

* closed ticket metrics only get last 12 months tickets

* Update README.md

Fixing travis link

* Restoring custom qualys wrapper

* Restoring custom qualys wrapper

* Update README.md

* Created the dockerfile

* Updating dockerfile

* in a production system, it is not advisable to have git pulling repos from inside a docker image when there is a pypi repo.

* builds the vulnwhisperer image without any of the ELK configs.  It can also be used in the same directory as the main project

* reverted the qualys call
2018-12-14 15:23:54 +01:00
a5972cfacd V6 Dashboard (#131)
* updating my base to match original vulnwhisperer (#1)

* Create docker-compose.yml

* Update 9000_output_nessus.conf

* Added an argument for username and password, which takes precendece over nessus.  Fixed #5

* Update README.md

* Silence NoneType object

* Put in a check to make sure that the config file exists.  FIXES austin-taylor/VulnWhisperer#4

* remove leading and trailing spaces around all input switches. Fixes austi-taylor/VulnWhisperer#6

* Update README.md

* Allow for any directories to be monitored

* Addition of Qualys WebApp Processing

* Addition of Qualys WebApp Processing

* Fixed multiple bugs, cleaned up formatting, produces solid csv output for Qualys Web App scans

* Adding custom version of QualysAPI

* Field Cleanup

* Addition of submodules, update to connectors, base class start

* Addition of submodules, update to connectors, base class start

* Addition of submodules, update to connectors, base class start

* Refactored classes to be more modular, update to ini file and submodules

* Refactored classes to be more modular, update to ini file and submodules

* Removing commented code

* Addition of category class and special class for Qualys Scanning Reports. Also added additional enrichments to reports

* Column update for scans and N/A cleanup

* Fix for str casting

* Update README.md

* Update to README

* Update to README

* Update to README

* Update to requirements.txt

* Support for json output

* Database tracking for processed Qualys scans

* Database tracking for processed Qualys scans

* Bug fix for counter in Nessus and format fix for qualys

* Check for new records

* Update to count tracker

* Update to write path logic

* Better database handling

* Addition of VulnWhisperer-Qualys logstash files

* Addition of VulnWhisperer-Qualys logstash files

* Update to logstash template

* Updated dashboard

* Update to README

* Update to README

* Logo update

* Readme Update

* Readme Update

* Readme Update

* Adding name of scan and scan reference

* Plugin name converted to scan name

* Update to README

* Documentation update

* README Update

* README Update

* Update README.md

* Add free automated flake8 testing of pull requests

[Flake8](http://flake8.pycqa.org) tests can help to find Python syntax errors, undefined names, and other code quality issues.  [Travis CI](https://travis-ci.org) is a framework for running flake8 and other tests which is free for open source projects like this one.  The owner of the this repo would need to go to https://travis-ci.org/profile and flip the repository switch __on__ to enable free automated flake8 testing of each pull request.

* Testing build with no submodules

* flake8 --ignore=deps/qualysapi

* flake8 . --exclude=/deps/qualysapi

* Remove leading slash

* Add build status to README

* Travis Config update

* README Update

* README Update

* Create CNAME

* Set theme jekyll-theme-leap-day

* README Update

* Getting started steps

* Getting started steps

* Remind user to select section if using a config

* Update to readme

* Update to readme

* Update to readme

* Update to readme

* Update to README

* Update to README

* Update to example logstash config

* Update to qualys logstash conf to reflect example config

* Update to README

* Update to README

* Readme update

* Rename logstash-nessus-template.json to logstash-vulnwhisperer-template.json

* Update 1000_nessus_process_file.conf

* Delete LICENSE

* Create LICENSE

* Update to make nessus visualizations consistent with qualys

* Update to README

* Update to README

* Badge addition

* Badge addition

* Addition of OpenVAS Connector

* Addition of OpenVAS

* Update 9000_output_nessus.conf

* Delete 9000_output_nessus.conf

* Update 1000_nessus_process_file.conf

* Automatically create filepath and directory if it does not exist

* Addition of OpenVas -- ready for alpha

* Addition of OpenVas -- ready for alpha

* Allow template defined config form IDs

* Completion of OpenVAS module

* Completion of OpenVAS module

* Remove template format

* Addition of openvas logstash config

* Update setup.py

* Update README.md

* ELK Sample Install (#37)

Updated Readme.md to include a Sample ELK Install guide addressing multiple issues around ELK Cluster/Node Configuration.

* Update vulnwhisp.py

* VulnFramework Links (#39)

Quick update regarding issue #33

* Updating config to be consistent with conf files

*  Preserving newlines & carriage returns  (#48)

* Preserve newlines & carriage returns

* Convert '\n' & '\r' to newlines & carriage returns

* Removed no longer supported InsecureRequestWarning workaround. (#55)

* Removed no longer supported InsecureRequestWarning workaround.

* Add dependencies to README.md

* Update vulnwhisp.py

* Fix to apt-get install

* Nessus bugfixes (#68)

* Handle cases where no scans are present

* Prevent infinite login loop with incorrect creds

* Print actual config file path

* Don't overwrite Nessus Synopsis with Description

* Tenable.io support (#70)

* Basic tenable.io support

* Add tenable config section

* Use existing variable

* Fix indent

* Fix paren

* Use ternary syntax

* Update Logstash config for tenable.io

* Update README.md

* Update template to version 5.x (#73)

* Update template to Elasticsearch 5.x

* Update template to Elasticsearch 5.x

I think _all field is no longer needed from ES 5.x because of the search all field execution if _all is disabled

* Qualys Vulnerability Management integration (#74)

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Fix error: "Cannot convert non-finite values (NA or inf) to integer"

When trying to download the results of Qualys Vulnerability Management scans, the following error pops up:

[FAIL] - Could not process scan/xxxxxxxxxx.xxxxx - Cannot convert non-finite values (NA or inf) to integer

This error is due to pandas operating with the scan results json file, as the last element from the json doesn't fir with the rest of the response's scheme: that element is "target_distribution_across_scanner_appliances", which contains the scanners used and the IP ranges that each scanner went through.

Taking out the last line solves the issue.

Also adding the qualys_vuln scheme to the frameworks_example.ini

* Update README.md

* example.ini is frameworks_example.ini (#77)

* No need to specify section to run (#88)

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Add Qualys vulnerability scans

* Use non-zero exit codes for failures

* Convert to strings for Logstash

* Update logstash config for vulnerability scans

* Update README

* Grab all scans statuses

* Fix error: "Cannot convert non-finite values (NA or inf) to integer"

When trying to download the results of Qualys Vulnerability Management scans, the following error pops up:

[FAIL] - Could not process scan/xxxxxxxxxx.xxxxx - Cannot convert non-finite values (NA or inf) to integer

This error is due to pandas operating with the scan results json file, as the last element from the json doesn't fir with the rest of the response's scheme: that element is "target_distribution_across_scanner_appliances", which contains the scanners used and the IP ranges that each scanner went through.

Taking out the last line solves the issue.

Also adding the qualys_vuln scheme to the frameworks_example.ini

* No need to specify section to run

Until now it vulnwhisperer was not running if a section was not specified,
but there is the variable "enabled" on each module config, so now it will
check which modules are enabled and run them sequentialy.

Made mainly in order to be able to automate with docker-compose instance,
as the docker with vulnwhisperer (https://github.com/HASecuritySolutions/docker_vulnwhisperer)
has that command run at the end.

* added to readme + detectify

* Silence requests warnings

* Docker-compose fully working with vulnwhisperer integrated (#90)

* ignore nessus requests warnings

* docker-compose fully working with vulnwhisperer integrated

* remove comments docker-compose

* documenting docker-compose

* Readme corrections

* fix after recheck everything works out of the box

* fix exits that break the no specified section execution mode

* fix docker qualysapi issue, updated README

* revert change on deps/qualysapi/qualysapi/util.py (no effect)

* temporarily changed Dockerfile link to the working one

* Update README.md

* Update README.md

* Fix docker-compose logstash config (#92)

* ignore nessus requests warnings

* docker-compose fully working with vulnwhisperer integrated

* remove comments docker-compose

* documenting docker-compose

* Readme corrections

* fix after recheck everything works out of the box

* fix exits that break the no specified section execution mode

* fix docker qualysapi issue, updated README

* revert change on deps/qualysapi/qualysapi/util.py (no effect)

* temporarily changed Dockerfile link to the working one

* fix docker-compose logstash config

* permissions needed for logstash container to work

* changing default path qualys, there are no folders

* Update 1000_vulnWhispererBaseVisuals.json

Update field to include keyword to prevent error: TypeError: "field" is a required parameter

* Update docker-compose.yml (#93)

increase file descriptors to allow elasticsearch to start.

* Update Slack link on README.md

* Update README.md

Added to README.md @pemontto as contributor

* Jira module fully working (#104)

* clean OS X .DS_Store files

* fix nessus end of line carriage, added JIRA args

* JIRA module fully working

* jira module working with nessus

* added check on already existing jira config, update README

* qualys_vm<->jira working, qualys_vm database entries with qualys_vm, improved checks

* JIRA module updates ticket's assets and comments update

* added JIRA auto-close function for resolved vulnerabitilies

* fix if components variable empty issue

* fix creation of new ticket after updating existing one

* final fixes, added extra line in template

* added vulnerability criticality as label in order to be able to filter

* Added jira section to config file and fail check for config variable (#105)

* clean OS X .DS_Store files

* fix nessus end of line carriage, added JIRA args

* JIRA module fully working

* jira module working with nessus

* added check on already existing jira config, update README

* qualys_vm<->jira working, qualys_vm database entries with qualys_vm, improved checks

* JIRA module updates ticket's assets and comments update

* added JIRA auto-close function for resolved vulnerabitilies

* fix if components variable empty issue

* fix creation of new ticket after updating existing one

* final fixes, added extra line in template

* added vulnerability criticality as label in order to be able to filter

* jira module gets now minimum criticality from config file

* added jira config to frameworks_example.ini

* fail check for config variable in case it is left empty

* fix issue jira-qualys criticality comparison

* update qualysapi to latest + PR and refactored vulnwhisperer qualys module to qualys-web (#108)

* update qualysapi to latest + PR and refactored vulnwhisperer qualys module to qualys-web

* changing config template paths for qualys

* Update frameworks_example.ini

Will leave for now qualys local folder as "qualys" instead of changing to one for each module, as like this it will still be compatible with the current logstash and we will be able to update master to drop the qualysapi fork once the new version is uploaded to PyPI repository.
PR from qualysapi repo has already been merged, so the only missing is the upload to PyPI.

* Rework logging using the stdlib machinery (#116)

* Rework logging using the stdlib machinery
Use the verbose or debug flag to enable/disable logging.DEBUG
Remove the vprint function from all classes
Remove bcolors from all code
Cleanup [INFO], [ERROR], {success} and similar

* fix some errors my local linter missed but travis catched

* add coloredlogs and --fancy command line flag

* qualysapi dependency removal

* Qualysapi update (#118)

* update qualysapi to latest + PR and refactored vulnwhisperer qualys module to qualys-web

* changing config template paths for qualys

* Update frameworks_example.ini

Will leave for now qualys local folder as "qualys" instead of changing to one for each module, as like this it will still be compatible with the current logstash and we will be able to update master to drop the qualysapi fork once the new version is uploaded to PyPI repository.
PR from qualysapi repo has already been merged, so the only missing is the upload to PyPI.

* delete qualysapi fork and added to requirements

* merge with testing

* Jira extras (#120)

* changing config template paths for qualys

* Update frameworks_example.ini

Will leave for now qualys local folder as "qualys" instead of changing to one for each module, as like this it will still be compatible with the current logstash and we will be able to update master to drop the qualysapi fork once the new version is uploaded to PyPI repository.
PR from qualysapi repo has already been merged, so the only missing is the upload to PyPI.

* initialize variable fullpath to avoid break

* fix get latest scan entry from db and ignore 'potential' not verified vulns

* added host resolv + cache to speed already resolved, jira logging

* make sure that vulnerability criticality appears as a label on ticket + automatic actions

* jira bulk report of scans, fix on nessus logging, jira time resolution and list all ticket reported assets

* added jira ticket data download + change default time window from 6 to 12 months

* small fixes

* jira logstash files

* fix variable confusion (thx Travis :)

* update readme (#121)

* Add ansible provisioning (#122)

* first ansible skeleton

* first commit of ansible installation of vulnwhisperer outside docker

* first ansible skeleton

* first commit of ansible installation of vulnwhisperer outside docker

* refactor the ansible role a bit

* update readme, add fail validation step to provision.yml and fix
typo when calling a logging funciton

* removing ansible from vulnwhisperer, creating a new repo for ansible deployment

* closed ticket metrics only get last 12 months tickets

* Update README.md

Fixing travis link

* Restoring custom qualys wrapper

* Restoring custom qualys wrapper

* Update README.md

* Updated the visualizations to support the 6.x ELK stack

* making the text message more generic

* removed visualizations that were not part of a dashboard

* Built a single file, since Kibana allows for that.  Created a new scripted value in the logstash-vulnwhisperer that will allow uniqu fingerprinting. Updated all visualizations to support the unqiue count of the scan_fingerprint. Fixes #130 Fixes #126 Fixes #111
2018-12-14 15:22:27 +01:00