1st, please recall my Computer Forensics demonstration as the Deep Web is not only not indexed and therefore accessible by the average Internet user but there are many mechanisms to hide information within information.
Recall the Web today is characterized by a n-tier architecture. Typically this is a 3-tier architecture comprised of a database core, middle tier business logic and a Web server. When a user requests a webpage, the middle tier business logic accesses information from the database core and constructs a webpage which is sent out by the Web server. For one of the CISS 100 Final Projects, you will have an option to implement this architecture using the LAMP (Linux, Apache Web server, MySQL database and PHP programming for the middle tier business logic).
A result of this architecture that constructs pages dynamically as necessary is that massive amounts of information is not indexed by search engines and thus remains hidden. This can be done purposely or simply as an artifact. For more information on the Deep Web but here is a quick start you can also search the Emergent Topics.
So to put this in context some 2014 stats from bot.net:
Public information on the deep Web is currently 400 to 550 times larger than the commonly defined indexed/accessible World Wide Web.
The deep Web contains 7,500 terabytes of information compared to 19 terabytes of information in the surface Web.
The deep Web contains nearly 550 billion individual documents compared to the 1 billion of the surface Web.
More than 200,000 deep Web sites presently exist.
Sixty of the largest deep-Web sites collectively contain about 750 terabytes of information — sufficient by themselves to exceed the size of the surface Web forty times.
The deep Web is the largest growing category of new information on the Internet.
Deep Web sites tend to be narrower, with deeper content, than conventional surface sites.
More than half of the deep Web content resides in topic-specific databases.
The currency on the Dark Web is often the bitcoin.
If you are accessing the Deep Web, you probably want to hide your footprints by using Tor – https://www.torproject.org/
Here is a great intro by Jamie Bartlett and as you watch please keep in mind the Business-IT-Society triangle.
First note that we are obligated to remain abreast of technology per the ACM Code of Ethics and Professional Conduct. Also note the Dark Net is not constrained to TOR sites so some questions:
- Were you aware of the Deep/Dark Net.
- Were you aware of its extent?
- Do you see collective or crowd sourced innovation?
- Could businesses learn from the Deep/Dark Net especially w/respect to CRM?
- Note some of these consensus principles can even be applied to distributed computing applications.
- Could consumers learn from the Deep/Dark Net especially w/respect to safeguarding their privacy?
The Dark Net from an Internet Historical Perspective w/nice TOR description/presentation
A more detailed and comprehensive look at the Deep Web, its technologies (e.g. anonymized encryption, Tor & BitCoin), and its negative and positive uses by Joseph Cox
Lastly, could the Dark Net be the key to our privacy?
For further reading see:
http://www.ciss100.com/2013/08/16/deep-web-a-primer/