Like ‘machine learning’ and ‘quantum computing’, ‘dark web’ is a term that has achieved buzzword status in recent times. But what are the dark web, deep web, and dark net – besides a vague place where the bad guys live?
Strictly speaking, the inscrutable Internet begins with the deep web, a catchall term for anything that can’t be found by search engines. This doesn’t include sites that have nicely asked Google and other spiders not to index them – it’s only for content that is inaccessible or “unlisted” and thus can’t be easily found. For example, a password-protected section of a website, or one that requires you to interact with a search bar to access content, are both part of the deep web. If you can’t access content simply by following links, it’s in the deep web.
The dark web is a subset of the deep web, representing content that is specifically designed not to be easily accessible. A private Internet Relay Chat (IRC) server could be part of the dark web. In practice, though, people talking about the dark web are usually referring to content that is only accessible via a dark net. This is a network that sits on top of the existing Internet infrastructure (the ‘clearnet’) and facilitates the dark web by allowing secure, anonymous, private communication – i.e. the TOR network.
A metropolitan transit system is a good metaphor for these confusing terms. The dark web is the entire London Underground. The TOR network is, say, the Central Line. In that case, an individual hidden service within TOR might be, for example, Oxford Circus.
From a threat intelligence perspective, the dark web is a rich – though unreliable – source of open source intelligence (OSINT). Beyond its strictest definition as “content found on dark nets”, the definition of the dark web as “all content that is designed to hinder access to the uninitiated” communicates the incentive of collecting OSINT from the dark web, along with the challenges it poses.
To understand the mindset of the dark web community, and the conditions of anonymity and deep mistrust under which it operates, it is necessary to address the circumstances that have created the dark web as it is today. The TOR project is a powerful enabler of privacy, anonymity and security for its users. Activists in oppressive regimes use it daily to communicate, organise and protect their fundamental rights. Unfortunately, a tool like TOR is fundamentally ambivalent. It is equally useful to the oppressed - who must hide their identity to avoid persecution – as it is to those with more nefarious motives.
For cybercriminals, dark nets such as the TOR network represent a technological revolution in facilitating crime. The balance between the convenience the dark net offers and the risk it can represent to criminals is essential to understand. The condition of anonymity under which these communities operate is their greatest asset, but it also means that no one can be fully trusted.
The more sophisticated an actor, the less easily accessible they are likely to be. It is relatively easy to infiltrate the “outer circles” – public IRC chatrooms, hacking forums, and open marketplaces such as Alphabay and TRDeal. These are the “low-hanging fruit” of the dark net.
Although cybercrime is a booming, lucrative industry, and has been for some time, the total size of the community is still relatively small. Spread across multiple dark nets, the user base would be too fragmented to make large communities particularly viable.
Dark nets such as I2P, ZeroNet and others have their uses, and are all certainly utilised by cybercriminals to some extent. While these networks are worth monitoring for intelligence collection purposes, the de facto standard for marketplaces, forums, IRC servers and so on is the TOR network. This is where famous names such as Hell Forum, the Silk Road, and AlphaBay hail from.
Low-hanging fruit isn’t all bad – after all, the reason the dark web is so fragmented in the first place is the abiding atmosphere of paranoia. Anything which forces criminals into insular communities and makes it more difficult to openly collaborate and recruit is a positive force for the security community. The occasional high-profile arrest of criminals or the closure of a dark net community also helps to dispel the illusion that TOR and similar dark nets provide bulletproof anonymity. While the many options available for passively monitoring dark net communities from the outside could be the subject of an entire blog, there is nonetheless a limit to what can be achieved from the outside looking in.
Beyond the public wikis and forums with little participation, and the popular open marketplaces where anything from data to drugs to weapons can be purchased, there is an opaquer breed of community. Many of these communities, particularly the forums and marketplaces, require a special invite code or referral by an existing member. Examples include the old HeLL forum (the new one requires 0.1-0.5 BTC as an entry fee) and Valhalla forum/marketplace.
Some of them are entirely private, with .onion addresses that are not shared publicly and authentication to ensure that members can communicate privately and securely. In other cases, public forums make it easy for anonymous individuals to get in touch, but all the interesting stuff goes on behind closed doors.
One way of unlocking these doors is by brute force – such as the FBI hacking campaign that targeted ‘Playpen’, a dark web community dedicated to distributing child pornography. In this case, they seized the server being used to host the site and continued to run it, attempting to distribute a “Network Investigative Technique” (read: malware) to as many of Playpen’s 215,000 users as possible to uncover their identities. The ethics of the FBI running a child pornography site aside, at least 1,300 “real” IP addresses were uncovered by the FBI, which will lead to an unspecified number of arrests in the future.
In more recent history, there is a story breaking as of the time of writing that details a vulnerability in Alphabay, a very popular marketplace. This vulnerability, which exposed more than 218,000 private messages between Alphabay users in the past 30 days, was reported to Alphabay by “Cipher2007”, but it is entirely possible that this flaw was known and being used by others to snoop on Alphabay users’ unencrypted conversations.
“Hacking the hackers” may seem like an excessively blunt approach, but infiltrating underground communities is even more difficult and labour intensive. A certain level of monitoring can be accomplished, especially on invite-only or pay-to-enter forums with large memberships. It’s well within the realm of feasibility for a researcher to get, for example, an invite code to the Valhalla Forum, or to pay for entry to HeLL forum. This is a very manual process, though; there are no easy ways to broadly monitor these forums. Actively monitoring these kinds of communities is a full-time job in of itself.
There are good reasons why the dark web represents a technological revolution for cybercriminals. It allows communication anonymously and securely. Individual users may make mistakes that allow them to be tracked down – for a great example of this, see Brian Kreb’s exposé on Anna-Senpai, author of the Mirai botnet. In rare instances, zero-day exploits in critical software such as the TOR Browser Bundle may allow for this anonymity to be compromised – or at least for a layer of anonymity to be stripped away. Somewhat more commonly, seizure or exploitation of the central servers employed by dark web communities may allow unsuspecting users to be tricked into incriminating themselves.
Nonetheless, the general rule on the dark web seems to be this: the only information available is the information users choose to make public. Anonymity is the watchword, and intelligence-gathering on the dark web is usually more on the order of gathering pre-emptive intelligence about an attack or learning about a breach before it becomes public knowledge. Even this is restricted; we usually only learn about breaches once they’re being actively sold online, for example. It’s easy for groups to set up secret, secure and private channels of communication, and without an exploit or “someone on the inside”, these channels are closed to us.
Despite this, the dark web is such a rich trove of intelligence that it would be remiss to ignore it. IRC channels can be logged; forums can be monitored; marketplaces watched; paste sites scraped and analysed; and so on. There are no easy solutions to the problem of anonymity, and it is not one that is likely to just go away. This is the price that is paid in exchange for the guarantee of privacy and anonymity. The ability to exercise free speech without fear of repercussions will always carry with it the risk of criminals using these same tools for their own ends.
