“The Deep Web is the fastest growing category of new information on the internet … The value of Deep Web content is immeasurable … Internet searches are searching only 0,03% … of the [total web] pages available.” (Micheal K. Bergman).
The dark web is a parallel internet that includes all the websites not indexed by regular “openweb” search engines. Although traditionally (and notoriously) the preserve of paedophiles, terrorists, drug dealers, gangsters and other people and material that most right-headed internet users would want nothing to do with, much of the dark web is simply comprised of private websites (some of which have taken active measures to avoid being listed by search engines), IIRC chat forums, Usenet groups, and other perfectly legitimate web use.
Increasing awareness of pervasive government surveillance (thank you Mr Snowden) and ever more draconian copyright enforcement measures are now fuelling a surge of public interest in an internet that is ‘off-grid’. Indeed, highly respectable websites such as Facebook and privacy search engine DuckDuckGo now host dark web websites that are only accessible through the most well-known and established of the various dark webs – Tor hidden services.
One of the biggest weaknesses of the Tor anonymity network is that there are only a limited number of exit nodes, as these are run by volunteers at some cost and risk to themselves (they can be held liable for what other users do when accessing the internet through their IP address.) This means that Tor exit nodes can be easily blocked, monitored, or, because anyone can run one, operated by a malicious entity.
Tor hidden services solves this problem by removing the need for an exit node, so that websites exist entirely ‘within’ the Tor network. These websites have a .onion suffix (for example https://facebookcorewwwi.onion/), and can only be accessed by visitors who are connected to the Tor network using the Tor Browser.
On 9 September, the legitimacy of the Tor hidden services dark web was given an important boost. The .onion domain name has now been officially designated by the Internet Assigned Numbers Authority (IANA) and Internet Engineering Task Force (IETF) as a Special-Use Domain Name.
The changes were proposed by respected security researcher and privacy campaigner Jacob Appelbaum and Facebook security engineer Alec Muffett. IANA is a department of ICANN (the non-profit private American corporation that oversees global IP address allocation), which makes this official recognition significant.
In addition to the increased respectability this move gives dark web sites, it opens the way for .onion sites to be issued SSL certificates that have been authenticated by a recognized Certificate Authority (CA). This will give visitors the same protection they receive when visiting SSL secured websites on the regular openweb. As Richard Barnes, security expert for Firefox at Mozilla, told Motherboard,
“This enables the Tor .onion ecosystem to benefit from the same level of security you can get in the rest of the web. It adds a layer of security on top.”
Facebook also expressed delight over this development,
“Jointly, these actions enable “.onion” as special-use, top-level domain name for which SSL certificates may be issued in accordance with the Certificate-Authority & Browser Forum “Ballot 144” – which was passed in February this year. Together, this assures the validity and future availability of SSL certificates in order to assert and protect the ownership of Onion sites throughout the whole of the Tor network – including https://www.facebookcorewwwi.onion/ and https://m.facebookcorewwwi.onion/.”