Download Wordlist Dirbuster [BETTER]
Gertrud Inabinet
Perimeter discovery is an important step during a web pentest and can, in some cases, lead to a website compromise. In order to carry out this recognition, several tools are available, including web content wordlists for web fuzzing:
Covered wordlists are located into Discovery/Web-Content/. We can notice that there is a lot of available wordlists (121 in the main folder). Some of them are specific for a given technology (CGIs.txt, coldfusion.txt, oracle.txt ...), others are specific for a given language (common-and-french.txt, common-and-dutch.txt ...). The main wordlist family present in SecList is the "RAFT Word Lists".
dicc.txt is a wordlist provided with dirsearch tool. The wordlist has the particularity to provide the variable extension %EXT%. Therefore, the wordlist must be used with tools that support %EXT% format (see post about web discovery tools).The wordlist has a total of 9021 lines distributed as follows :
Wfuzz tool is provided with a lot of wordlists. Some of them in "general" directory are dedicated for directories and files enumeration. That's the case of megabeast.txt, big.txt, medium.txt and common.txt. None of those wordlist have words containing extensions. They are distributed as follows :
In some case, an auditor may look for a specific wordlist. Wordlistctl is a tool design to fetch, install, update and search for a given wordlists. This python script offers more than 6400 wordlists and is maintained by BlackArch Linux distribution.
An alternative would be to run the common wordlists you are used to fuzz withand build only an additional wordlist including only files like security.txtor other files that may be missing from most wordlists so you don't have toupdate the generic part on your own.
Without further ado, here is a comparative table of the different wordlists discussed in this post. Colored cases represent a high correlation between wordlists. To understand the matrix you should read: "N% of the wordlist at line Y is contained in wordlist at column X".
Adversaries may iteratively probe infrastructure using brute-forcing and crawling techniques. While this technique employs similar methods to Brute Force, its goal is the identification of content and infrastructure rather than the discovery of valid credentials. Wordlists used in these scans may contain generic, commonly used names and file extensions or terms specific to a particular software. Adversaries may also create custom, target-specific wordlists using data gathered from other Reconnaissance techniques (ex: Gather Victim Org Information, or Search Victim-Owned Websites).
As cloud storage solutions typically use globally unique names, adversaries may also use target-specific wordlists and tools such as s3recon and GCPBucketBrute to enumerate public and private buckets on cloud infrastructure.[2][3] Once storage objects are discovered, adversaries may leverage Data from Cloud Storage to access valuable information that can be exfiltrated or used to escalate privileges and move laterally.
As mentioned previously, DirBuster needs a list of words to start a brute force scan. But don't worry, you don't need to make your own list or necessarily search for a list in Internet as DirBuster has already a couple of important and useful lists that can be used for your attack. Just click on the Browser button and selected the wordlist file (they're normally located at /usr/share/dirbuster/wordlists) that you want to use for the brute force scan:
Dir mode is used to enumerate URLs for directories and files. This mode will be used to find content within a known target domain or subdomain. Gobuster will uncover hidden directories or files. -u is the URL that will define our target domain. -w is the wordlist we use to help identify the names of possible common directories or file names. -x are the string extensions we are expecting to find.
VHOST mode or Virtual host brute-forcing mode will find virtual hosts within the domain. Virtual Hosting occurs when a domain is hosting other domain names on a single server or multiple other servers. This allows companies to share resources on a single server. This works by having Gobuster visit a URL and check the associated IP address. -v is for verbose mode. -u defines the target URL. -w is the wordlist that can help enumerate common virtual host site names. -o will output the results to an assigned file.
S3 mode will enumerate publicly available Amazon Web Service (AWS) S3 buckets. While Gobuster can help determine the names of the potential S3 buckets, it does not indicate that the buckets are able to be accessed or modified. You may still be able to use the information to access files available such as web support, logs, videos, or images. -w is the wordlist to define the names we will look to enumerate.
So I've been attempting to use dirbuster to fuzz a few vulnerable machines. I haven't been satisfied with the outputs so I started trying some manual fuzzing and then referencing the default dirbuster wordlist as well as others to make sure it wasn't a singular issue.
For example, when fuzzing using the default dirbuster medium size wordlist, 5 results appear. I know I can manually get 200 & 403 responses from pages like /config, /admin, or /mail, but they are not appearing in my dirbuster results even though they exist in the wordlist I'm using. I get an output like /error, /icons, /mailman, /pipermail, /cgi-bin, and nothing else, even though I verified the other pages exist in the wordlist and manually test correctly. Does anyone out ther have an idea on what mistake I'm making that is getting such a weird output?
I love the OWASP project and community (I'm actually part of it), but I don't like DirBuster at all, which is an OWASP project. It's very slow compared to other similar tools, and it easily crashes if you try to load a big wordlist.
If you don't have one already, I recommend you get a Kali Linux VM so you can have lots of tools (including gobuster, dirb, and DirBuster) to play around with. Kali also comes with many good wordlists such as the one referenced in the command above.
Recently I was trying to install the wordlists (similar to the ones present in Kali and BlackArch operating systems). However, in my case, I got an error while installing the wordlists package from the AUR using pamac.
However there is one fundamental problem with these wordlists. Whilst the purpose of these wordlists is finding hidden and not linked resources, ironically they are generated only from known and linked resources. To address this problem we came up with the idea of generating wordlists from open source code repositories. This way it would be possible to see all file/directory names and create much more useful wordlists.
We have extracted the directory structure and file names of many projects from Google Code and SourceForge to prepare a good wordlist for discovering hidden files/folders on a targeted web application.
We are going to see some advantages Gobuster has over other similar tools like dirbuster and we are also going to explore its features and see examples of it. Finally a cheat-sheet is included in the end to have the most used cases in one place.
Here we simply run gobuster against erev0s.com using a very small wordlist. We can see some details for the attack and below we can see which paths were found along with the status. It only found one path blog, which is actually the only correct path that was included in the wordlist.
In the example we are trying to find subdomains of erev0s.com using the awesome_wordlist.txt, which is a custom wordlist we made with only five lines. We can see that it found the www.erev0s.com, as it indeed exists and it redirects to non-www version of the domain.
As you can see Gobuster reports as found all the entries we have in the dummy wordlist we created. This is due to the fact that Cloudflare uses a different http code to respond to such a request that it does not exist. As you can see the rest of the subdomains except www.erev0s.com report that the status it was returned is Status: 530. As you can see here, this error is the origin DNS error from Cloudflare and since Gobuster does not recognize it, it considers it as found.
Gobuster needs wordlists. One of the essential flags for gobuster is -w . Wordlists can be obtained from various places. Depending on the individual setup, wordlists may be preinstalled or found within other packages, including wordlists from Dirb or Dirbuster. The ultimate source and "Pentesters friend" is SecLists - which is a compilation of numerous lists held in one location.
Not too many results and was quite heavy on the system processess. Results depend on the wordlist selected. It is worth working out which one is best for the job. The length of time depends on how large the wordlist is. It can also be worth creating a wordlist specific to the job at hand using a variety of resources.
As shown above the Global flags are the same as for the all modes. Again, the 2 essential flags are the -u URL and -w wordlist. Not essential but useful -o output file and -t threads, -q for quiet mode to show the results only.
Gobuster is a useful tool for recon and increasing the knowledge of the attack surface. Start with a smaller size wordlist and move to the larger ones as results will depend on the wordlist chosen. Keep enumerating. Don't stop at one search, it is surprising what is just sitting there waiting to be discovered.
If you are new to wordlists, a wordlist is a list of commonly used terms. This can be a password wordlist, username wordlist, subdomain wordlist, and so on. You can find a lot of useful wordlists here.
I would recommend downloading Seclists. Seclists is a collection of multiple types of lists used during security assessments. This includes usernames, passwords, URLs, etc. If you are using Kali Linux, you can find seclists under /usr/share/wordlists.
For example, if we have a company named Acme, we can use a wordlist with acme-admin, acme-user, acme-images, and so on. This wordlist can then be fed into Gobuster to find if there are public buckets matching the bucket names in the wordlist.
df19127ead