Re-discover your company network with Ivre
Introduction
In medium to large company networks, it is easy to loose track of the different servers and assets deployed if a strict asset management database is not updated and checked regularly. One option to check the quality of the database and discover assets that would have gone under the radar is to perform active network scanning on the network.
Note that besides the discovery of assets, this method can be combined with vulnerability management as we will discuss later on. Some commercial solutions can do it for you like Nessus or Tripwire. Today I am going to talk about an open source solution called Ivre (or Drunk in English).
Tool Selection
There are multiple ways to perform network scanning. Those of you who already used Nmap knows how many configuration options you can set. The tool I am presenting today relies on Nmap (or Masscan) and is called Ivre (french word for Drunk). It is an open source tool that you can clone easily from github.
Among other things, it can execute Nmap and store the results in a MongoDB database which is then used by the front-end WebGUI to refine searches and explore in details the results of Nmap scans and the plugins output.
Although it is primarily focused on external scanning (scanning the Internet) by integrating for instance automatic geo-tagging of IPs, I found it very useful for internal scanning as well. You can see it as a "Shodan" like tool but open source and with more options that you can host on premises.
Installation
Installation can be as simple as pip install ivre if you want to install it on a machine with Internet access.
To have an efficient setup, I recommend to install MongoDB on a powerful machine so that your queries return quickly even with millions of records. It is possible to install it on a separate machine, you just have to specify it in the configuration file /etc/ivre.conf. The installation of MongoDB will rely on your existing setup and is outside of the scope of this article, you can check the official guide on MongoDB website.
If you want to have a secure setup between your database host and the Ivre host, you can patch the code on line 105 in the file <python install>\sites-packages\ivre\db\mongo.py to add your client certificate and your CA to connect to MongoDB using client certificates. Hopefully at some point, I'll find the time to submit a pull request.
Note that if you don't want to bother too much, there are also docker images ready for duty at docker hub. This is probably the fastest way to try it out for an evaluation phase. Let me know if you think it would be helpful to have a step by step installation guide but I believe the official documentation is complete enough.
Agent Setup
One feature that is not really well advertised but that I find extremely useful in the context of an internal deployment is the notion of "scanning agents". This becomes a must if your Ivre master installation and database is to be stored on an isolated network. Find here the official documentation. I thought that the "runscansagentdb" approach was the best one as it leverages the database to store the scanning targets and can run in "daemon" mode where it feeds new targets when the agents have finished their jobs.
Imagine for instance that a company has segregated its network to enhance security and all security tools are in a dedicated network. It would be extremely inefficient to perform network scanning from this network across the firewall. Instead this company could deploy a lightweight agent on each of the network and only allow communications between these agents and the Ivre master. The agent can be as simple as a virtual machine with Nmap installed on it. This would give the following deployment:
How to communicate between the agents and the master ?
There is only little words about the communication possibilities between the agents and the master in the doc so I thought it was worth expanding a bit more. By default, if you try to use the syntax
$ ivre runscansagentdb --source MySource --add-agent agenthost1:/path/to/agent/dir
Ivre will try to rsync to agenthost1. With the setup described above, it will most likely fail. So what you can do instead is to point to a local directory such as:
$ ivre runscansagentdb --source MySource --add-agent /var/agent1
And then take care of the folders synchronisation using and ad-hoc mechanism such as rsync on top of SSH with private key authentication. This could be performed by a cron job for instance.
Consuming the results
Once your setup is ready and you start getting scan results in your database. It's time to actually explore them. The Web GUI is extremely well done and gives you plenty of opportunities to spot vulnerable hosts. Of course the results will mostly depend on the Nmap scripts that you decided to load. All the commands that you can type in the query part have auto-completion enabled which let you learn the different commands as you go.
The Web GUI is really nice to perform hunting on specific vulnerabilities or hosts. If you want to process your results with a script, the easier is then to use the ivre scancli method which can export the results in JSON format.
Automating further
Now that you have a nice little setup with your different agents and results getting into your database, you'll notice that it can become annoying to have to log in to the Ivre box and define your scan manually with the ivre runscansagentdb command.
There is unfortunately no out of the box solution to have recurrent scanning (in the case of internal monitoring of the network, it quickly becomes needed). However, the beauty of this open source solution is that you can easily interact with the Python API and implement your own orchestration logic on top of Ivre. I might do a second article on this if there is interest.
Conclusion
I have presented to you a nice open source solution to perform network scanning in a professional way and discussed potential deployment options into a company network in the scenario of internal asset tracking / vulnerability management. Thanks a lot to Pierre for the development of this amazing tool and hit me up on Twitter to let me know your thoughts and comments.