Ever wondered whether you ever missed a hidden subdomain that would have unlocked a critical vulnerability and a large bounty reward?
In this article, you will discover how cutting-edge subdomain enumeration techniques, both passive and active, can reveal the unseen corners of your target’s infrastructure.
We’ll explain the advantages of various techniques, recommend enumeration tools for executing them, and show you how to implement these techniques, supported by illustrative examples performed on a real public bug bounty program (that of Swedish betting brand ATG).
When Bug Bounty hunting or conducting a pentest, subdomain enumeration is a crucial first reconnaissance step before you actually start hacking – maximising the exposed attack surface and therefore your chances of finding vulnerabilities.
Outline
- What is subdomain enumeration?
- Passive subdomain enumeration
- Enumerate subdomains using public databases
- Using Google dorking to enumerate subdomains
- Active subdomain enumeration
- Best practices for avoiding WAF bans
- DNS brute-forcing
- Virtual host fuzzing
- Reverse DNS lookups
- Crawling for subdomains
- Performing HTTP probes on collected domains
- Conclusion
- References
What is subdomain enumeration?
Subdomain enumeration is a technique for collecting subdomains from a domain owned by the program you are testing. The goal is to enumerate as many assets from the program as possible, to gain an understanding of total attack surface and how the infrastructure works. Once you’ve collected all these subdomains, it's easier to scale up your attacks and achieve better results.
Passive subdomain enumeration
Passive subdomain enumeration is an efficient way to collect information about a target without directly interacting with it. The passive enumeration process analyses publicly available data or third-party resources – such as DNS records, SSL/TLS certificate logs or web archives like Wayback Machine – to find subdomains. This will allow us to gather historic data without generating suspicious traffic to our target.
Enumerate subdomains using public databases
The most common, and one of the most effective, techniques for enumerating subdomains is by using third-party subdomain enumeration scanners that rely on passive enumeration of third-party databases. Tools such as Censys, Shodan and SecurityTrails can quickly search vast repositories of DNS information.
This technique works so well is because you are aggregating subdomains collected from multiple databases over a long period of time. By broadening your asset discovery, you can get an extensive view of the target’s digital footprint from the outset – encompassing assets that traditional forms of active scanning might miss.
Censys offers the ability to search within its database for subdomains and other juicy information about the target infrastructure. As you can see below, we have performed a simple query that contains a couple of domains of our target. Censys then provides us with subdomains and other useful information.
You can perform a similar technique with Shodan to dig even deeper into your target’s infrastructure. Since Shodan details exposed services associated with indexed subdomains – such as open ports, misconfigured servers or publicly accessible APIs – it can help you focus on attack vectors with a higher risk profile. Here's our Shodan query and the results:
hostname:atg.se
You can find subdomains even more rapidly by using subdomain enumeration tools that query a much wider range of public databases.
For instance, Subfinder, a DNS enumeration tool built by ProjectDiscovery, offers versatile output formatting capabilities that streamline the analysis workflow. As a result, researchers can easily integrate results into other security tools or create detailed reports.
Subfinder uses the following databases to search for the subdomains of a domain:
Here is a Subfinder command that scans atg.se as the hostname and outputs the result to a JSON file, followed by the results generated:
subfinder -d atg.se -oJ domains-atg.se.json
Another great tool for performing subdomain enumeration is Amass. Developed by OWASP, Amass can perform external asset discovery by combining open source intelligence (OSINT) with active reconnaissance techniques. This utility's strength lies in correlating data from diverse sources, which is crucial for building an accurate picture of the target’s attack surface.
Amass uses the following databases to search for the subdomains of a given domain:
This Amass command (which is followed by the results generated) uses the enum subcommand to enumerate atg.se, outputs the result to a file for later use, and sets a timeout of 12 seconds.
amass enum -d atg.se -o domains-amass.atg.se.txt -timeout 12 -v
We then output the file in a JSON format because it preserves information about where the subdomains were discovered, and allows us to seamlessly insert this data into a database for future use. This structured approach is essential for analysis and for linking reconnaissance data with further testing steps.
Using Google dorking to enumerate subdomains
Google dorking (aka Google hacking) is an advanced way to perform queries on Google’s search engine. The power of this method lies in how it taps into Google's incomparably vast index of domains. Although Google's captcha protections make this technique trickier to automate, Google dorking remains a potent tool since it can surface active subdomains missed by automated tools.
Google's advanced search operators enable precise subdomain discovery through targeted queries such as 'site:*.example.com'. This query returns all subdomains of example.com collected by Google's crawler.
The google dork below searches for all known subdomains of *.atg.se, other than www.atg.se, within Google’s database. The subsequent screenshot shows how it helped us discover several subdomains of our target domain:
site:*.atg.se -site:www.atg.se
Active subdomain enumeration
In contrast to passive enumeration, active subdomain enumeration involves interacting with the target itself in order to find all subdomains of the domain. This direct approach is necessary for uncovering any subdomains that are not publicly indexed, yet in active use. Methods such as brute-force attacks, virtual host (vhost) fuzzing and crawlers play a big role in this process.
Best practices for avoiding WAF bans
While active enumeration might be essential for reconning comprehensively, it also carries the risk of getting the attacker or hunter blocked or generating false positive or false negative responses from web application firewalls (WAFs), bot protection mechanisms or other proxy services.
Getting banned is especially likely if you perform a large number of requests in a short space of time, particularly when a bot protection mechanism is active across multiple domains that you are scanning.
Being banned by a particular bot protection mechanism will most likely hamper your efforts to scan any other domains under its protection.
You can reduce the risk of being blocked by implementing rate limiting, using rotating proxies and adhering to robots.txt guidelines. If you're hunting on Bug Bounty scopes, be sure to follow the program's testing guidelines too.
DNS brute-forcing
DNS brute-forcing, which is among the oldest and most popular techniques for subdomain enumeration, involves creating a wordlist of potential subdomain names and testing them against the target domain. This technique is invaluable because it systematically uncovers subdomains that may not be registered in public databases, thereby revealing hidden parts of the infrastructure.
When a subdomain responds, it is likely a valid, newly discovered asset. If the domain has a wildcard configuration, additional filtering may be required for accurate results.
DNS brute-forcing is only as effective as the wordlist used. As well as a target domain, you need a wordlist covering an extensive range of permutations to successfully execute a brute-force attack. We recommend sourcing wordlists from SecLists.
As for performing brute-force attacks, Gobuster is a popular and highly effective tool.
In this Gobuster command, we use the dns technique to brute-force the hostname atg.se with our chosen wordlist file:
gobuster dns -d atg.se -w wordlist.txt
As we can see above, our Gobuster scan uncovered a couple of related subdomains. To adapt enumeration to your target, you should create a custom wordlist by analysing patterns in subdomains discovered from third-party databases. These patterns typically reflect an organisation's naming conventions.
If you then use this wordlist with mksub by Trickest, you can further refine these patterns and generate additional subdomain variations. We can then perform a more optimised brute-force attack against our target domain.
Virtual host fuzzing
Once you have a list of domains, we recommend fuzzing for virtual hosts (vhosts), which are domains that share the same IP as another domain on a web server. Vhosts can uncover hidden services sharing the same server, potentially exposing vulnerabilities in less secure subdomains.
If a subdomain resolves to the same IP address as its parent domain, it may be part of a vhost setup, where multiple domains or subdomains share the same server but serve different content based on the Host header in HTTP requests.
Recommended tools with vhost-fuzzing capabilities include gobuster, ffuf and wfuzz. In the example below we’ve used Joohoi’s ffu to fuzz the hostname atg.se for vhosts using a wordlist file with an easy-to-read CLI output:
ffuf -c -r -u 'https://www.atg.se/' -H 'Host: FUZZ.atg.se' -w dns-wordlist.txt
In our example, we have performed fuzzing against the target domain (www.atg.se) by sending requests with different subdomain names in the Host header, using a wordlist of possible subdomain names. If we receive an unusual response, we have most likely discovered a new vhost for our target.
Reverse DNS lookups
A reverse DNS (rDNS) lookup resolves an IP address to its associated domain name. This technique is essential because it can reveal domains that are not listed in public records. rDNS is particularly useful when investigating a range of IP targets. It exposes hidden domains associated with an IP address that might not appear in forward DNS lookups, thus broadening your domain reconnaissance.
DnsX is a versatile DNS toolkit that can perform reverse DNS lookups and execute multiple probe types against both IP addresses and hostnames. Below we've executed a reverse DNS lookup on DnsX by querying PTR records against a list of IP addresses:
cat ips.txt | dnsx -ptr -resp-only
Crawling for subdomains
Web crawling is a powerful reconnaissance technique for discovering additional subdomains after initial enumeration. By analysing HTTP responses and the Document Object Model (DOM), crawlers can discover hidden API endpoints or internal subdomains.
Burp Suite is a great tool for implementing advanced crawling techniques, such as passively collecting subdomains by analysing HTTP responses or from the DOM by running a headless browser.
To demonstrate the impressive power of this crawler, we simply navigate to the domain of our target – www.atg.se – and see how many subdomains we can collect:
As you can see above, we collected a bunch of subdomains from a single domain visit. It’s typical to get such as positive result, because modern websites tend to use multiple API endpoints and microservices, each with their own dedicated subdomain.
Performing HTTP probes on collected domains
HTTP probing filters out false positives from among your collected domains by determining which ones are actually running as web servers. This is necessary because it’s highly likely that at least some of your collected domains will not be active.
We won't delve too deeply into this topic here, as it deserves its own dedicated article, which we plan to write in the future. However, we will cover the basics of performing HTTP probing on your collected domains.
To perform a HTTP probe, we first gather all collected domains into a single file. We then input this file into a HTTP probing tool, such as httpx or httprobe, to generate a list of domains that are running as web servers.
We can run a simple HTTP probe with httpx using the following command:
cat domains.txt | httpx -o domains-webserver.txt
Or we can run httprobe with this command:
cat domains.txt | httprobe >> domains-webserver.txtConclusion
We’ve explained the benefits of various popular techniques, both passive and active, for performing subdomain enumeration. We’ve also recommended some subdomain discovery tools for implementing these techniques, along with practical examples of how to execute them on a real target.
Used in combination, these subdomain enumeration methods expose much larger attack surfaces than any techniques can discover alone. Establishing a strong baseline for your target in this way is a crucial early step in any pentest or bug bounty engagement.
From capturing subdomains from third-party databases to leveraging Google dorking and executing DNS brute-force attacks, these techniques can together optimise your reconnaissance and maximise your success rate when Bug Bounty hunting.
References
DOJO CTF training ground
Having improved your recon knowledge, why not try exploiting some bugs in a fun, risk-free environment? Dojo features free trainings on common vulnerabilities, hands-on labs and monthly CTF challenges where the three best writeups earn hunters YesWeHack swag and points that influence who gets invited to hack on private programs.