Recon is the main part of a Bug Bounty process where everything begins. It basically consists in, given a scope, let’s say a domain/group of domains like *.domain.com, search as much subdomains as possible from public sources. Finally, over that list, we will look for vulnerabilities.

At first, manual recon way is good way for learning but, after hundred of recons, it becomes something boring. So, why not to automate recon?. In this post we will see some VPS to store our scripts and we will create a recon script with notifications to our Telegram bot.

WHERE DO I START?

Okay, let’s focus. As said above, we need at least the following two elements:

• VPS: Virtual Private Server in order to execute recon scripts automatically or any other tasks.
• Recon script: recon script that will collect all subdomains from a scope.

Let’s go into detail with each of that elements.

VPS

There is a lot of VPS services, each one with diferent specifications and pricing. You don’t need the best requirements but I had a script on a cheap VPS with minimum specifications and sometimes when i used xargs with –threads 200 launched, the VPS got stucked.

If you don’t want to spend a lot of money, you have to find a VPS with a good quality/price ratio so that you spend as little as possible and the script can run without problems. If you don’t mind to spend money, you have a great variety of options to choose from.

Next, there is a list about some VPS services.

OVH

URL: https://www.ovhcloud.com/es-es/vps/

The minimum specification: 1 vCore, 2 GB RAM, 40 GB SSD (5,57 €)

DIGITAL OCEAN

URL: https://www.digitalocean.com/pricing/

The minimum specification: 1 CPU, 1 GB RAM, 25 GB SSD (5,00 $)

LINODE

There is a video where todaynews said he use this VPS service.

URL: https://www.linode.com/es/pricing/

The minimum specification: 1 CPU, 1 GB RAM, 25 GB SSD (5,00 $).

TIME4VPS

URL: https://www.time4vps.com/linux-vps/

The minimum specification: 1 CPU, 2 GB RAM, 20 GB SSD (3,99 €).

And a lot of more VPS services. Just look for them !!!

HARDENING SSH

Once we finally purchased the VPS, we will connect and will configure SSH using an old post from this blog, hardening SSH (sorry it’s in Spanish. I will translate it soon), where following steps are reviewed:

• Creation of public/private SSH keys.
• Configuring SSH service.
• Modify SSH server parameters in sshd_config.

With this, we will connect securely via SSH using our generated private key.

INSTALLING COMPONENTS

In order to be able to execute any kind of scripts we will need to install some components: go, python, ruby, etc …

PYTHON & RUBY

Although Python2 is obsolete, we will install it anyway because of some scripts could be in Python2:

apt install python2 python3 ruby-dev ruby

To verify that python was correctly installed, execute:

GO

To install go, we will download the .tar.gz file from his oficial page https://golang.org/dl/ and will descompress it into /usr/local/ directory:

wget https://golang.org/dl/go1.16.6.linux-amd64.tar.gz

rm -rf /usr/local/go && tar -C /usr/local -xzf go1.16.6.linux-amd64.tar.gz

It’s important, once Go is installed, to set correctly the GOROOT and GOPATH variables and append GOROOT to PATH variable in /etc/profile (all login shells will read from this file) or in /home/<user>/.bashrc (all interactive shells will read from this file after /etc/profile):

export GOROOT=/usr/local/go/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOROOT

To verify that Go is correctly installed, we will execute go -version:

SUBWALKER

To install Subwalker we will clone the repo from Github and execute the install.sh file:

git clone https://github.com/m8r0wn/subwalker
cd subwalker; chmod +x install.sh subwalker.sh
sudo ./install.sh

To verify Subwalker is correctly installed, execute ./subwalker.sh from directory:

FINDOMAIN

To install Findomain we will donwload the binary from Github and we will move it to /usr/local/bin:

wget https://github.com/findomain/findomain/releases/latest/download/findomain-linux
chmod +x findomain-linux

To verify Findomain is correctly installed, execute findomain-linux:

SUBFINDER

To install the script:

GO111MODULE=on go get -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder

To verify Subfinder was correctly installed, execute subfinder:

ANEW

This tool from tomnomnom will be used with each script to compare, as example, subdomains found from a script with subdomains saved in a file, appending only new ones to the file and showing them on screen:

To install the script:

go get -u github.com/tomnomnom/anew

To verify anew was correctly installed, execute anew -h:

HTTPX

This tool will be used, once we have a list of subdomains, to show only subdomains with a HTTP/HTTPS service running (alive subdomains):

go get -v github.com/projectdiscovery/httpx/cmd/httpx

CREATING THE SCRIPT

This script will have a common objective: create a centralized list of subdomains so we can begin looking for vulnerabilities or use with another scripts. At first, we will use three known scripts but you can append all scripts you want.

These three scripts are the following:

• Subwalker: includes assetfinder, subl1st3r and subscraper (https://github.com/m8r0wn/SubWalker)
• Findomain: APIs that Findomain is using at the moment: Certspotter, Crt.sh, Virustotal, Sublist3r, Facebook, Spyse (CertDB), Bufferover, Threatcrowd, Virustotal with apikey, AnubisDB, Urlscan.io, SecurityTrails, Threatminer, C99, Archive.org, CTSearch (https://github.com/Findomain/Findomain)
• Subfinder: APIs that Findomain is using at the moment: Binaryedge, Certspotter, Censys, Chaos, DnsDB, Github, Intelx, Passivetotal, Recon.dev, Robtex, SecurityTrails, Shodan, Spyse, Threatbook, Virustotal, Zoomeye (https://github.com/projectdiscovery/subfinder)

We will centralize all the outputs to a common file, as example, all_subdomains.txt. Next, we will use httpx to get only alive subdomains and save them into engineer file, as example, alive_subdomains.txt. The below image shows the flow:

1. Each script will be launched, using anew to compare all subdomains found with the file all_subdomains.txt where all domains will be saved and appending new ones. It will also create the temporal file new_subdomains.txt whose content will be only new subdomains discovered.
2. When all scripts were executed, httpx will be used on temporal file new_subdomains.txt to discover only alive subdomains in specific ports. Concatenated with anew, alive subdomains will be compared with domains from the file alive_subdomains.txt, appending only new ones and redirecting them to the temporal file alive_new_subdomains.txt.
3. Using the Telegram API, we will send temporary file all_new_subdomains.txt, receiving a chat notification.

PUTTING IT ALL TOGETHER

Now, let’s put these above three scripts in a bash file. The code for the script is the following:

#!/bin/bash

domain=$1

echo ">>> SUBWALKER"
~/Hacking/Tools/Recon/subwalker/subwalker.sh $domain

# Subwalker will save domains in subwalker.txt in subwalker's directory
cat ~/Hacking/Tools/Recon/subwalker/subwalker.txt | anew all_subdomains.txt > new_subdomains.txt

echo ">>> SUBFINDER"
subfinder -t 200 -d "$domain" -nW -silent | anew all_subdomains.txt >> new_subdomains.txt

echo ">>> FINDOMAIN"
findomain-linux --threads 200 -t "$domain" -w ~/Hacking/Wordlists/my_wordlist.txt --resolved | anew all_subdomains.txt >> new_subdomains.txt
echo ">>> DISCOVERING ALIVE SUBDOMAINS"
cat new_subdomains.txt | httpx -ports 80,443,8000,8443,8080 | anew alive_subdomains.txt > alive_new_subdomains.txt

TELEGRAM API

As a final step, we will implement the Telegram API to our script in order to get notifications about new subdomains in telegram bot. First of all, let’s install Telegram-notify:

apt-get install curl
wget -O /etc/telegram-notify.conf https://raw.githubusercontent.com/NicolasBernaerts/debian-scripts/master/telegram/telegram-notify.conf
wget -O /usr/local/sbin/telegram-notify https://raw.githubusercontent.com/NicolasBernaerts/debian-scripts/master/telegram/telegram-notify
chmod +x /usr/local/sbin/telegram-notify

To verify the correctly installation we will execute telegram-notify:

For receiving messages, we need to create a Bot (https://core.telegram.org/bots). Bots are considered like an application account associated to a real person’s account (doesn’t need to be associated to a phone number) and is able to send or receive messages.

To create a Bot we will do it through @BotFather:

Through /newbot command, we will create a bot, choosing a name for it:

When bot is created, BotFather will assign us an API-KEY token to access the bot via HTTP API:

Apart from the API-KEY token, we also need the USER-ID to invoke the API. We will make a request to https://api.telegram.org/botXXXXXXXXX:YYYYYYY-YYYYYYYYYYYYYYYYY_YY/getUpdates (replace XXXXXXXXX:YYYYYYY-YYYYYYYYYYYYYYYYY_YY with your token) and will obtain a response with JSON format with the USER-ID among any other data. (More info in http://www.bernaerts-nicolas.fr/linux/75-debian/351-debian-send-telegram-notification).

Next, we will put API-KEY token and USER-ID in telegram-notify.conf file:

Ok. Everything is ready. Let’s do a test with the telegram-notify API:

As final step, append the following line at bottom in the script:

telegram-notify --title "[NEW SUBDOMAINS]" --text alive_new_subdomains.txt

Everything should be ready.

RUNNING THE SCRIPT

Ok, we implemented the three scripts and Telegram API. Let’s see what happen once script is launched:

As we can see, the script will try to get all subdomains from the main domain using public sources and, after that, it will send new alive subdomains to Telegram:

CRONTAB

To execute the script everyday each four hours as example, we will add a new line into CRONTAB file:

Sometimes, go cannot find some packages and the script will raise errors. To resolve this, add the PATH and GOPATH variables to the crontab file: 

I hope this article has helped you to create your first recon automation script and that you have learned something new. See you soon.

“Don’t give up. Great things take time.“

(Image from intographics in Pixabay)