What is Hyphanet?
I worry about my child and the Internet all the time, even though she's too young to have logged on yet. Here's what I worry about. I worry that 10 or 15 years from now, she will come to me and say 'Daddy, where were you when they took freedom of the press away from the Internet?
--Mike Godwin, Electronic Frontier Foundation
Hyphanet is free software which lets you anonymously share files, browse and publish "freesites" (web sites accessible only through Hyphanet) and chat on forums, without fear of censorship. Hyphanet is decentralised to make it less vulnerable to attack, and if used in "darknet" mode, where users only connect to their friends, is very difficult to detect.
Communications by Hyphanet nodes are encrypted and are routed through other nodes to make it extremely difficult to determine who is requesting the information and what its content is.
Users contribute to the network by giving bandwidth and a portion of their hard drive (called the "data store") for storing files. Files are automatically kept or deleted depending on how popular they are, with the least popular being discarded to make way for newer or more popular content. Files are encrypted, so generally the user cannot easily discover what is in his datastore, and hopefully can't be held accountable for it. Chat forums, websites, and search functionality, are all built on top of this distributed data store.
Hyphanet has been downloaded over 2 million times since the project started, and used for the distribution of censored information all over the world including countries such as China and in the Middle East. Ideas and concepts pioneered in Hyphanet have had a significant impact in the academic world. Our 2000 paper "Freenet: A Distributed Anonymous Information Storage and Retrieval System" was the most cited computer science paper of 2000 according to Citeseer, and Hyphanet has also inspired papers in the worlds of law and philosophy. Ian Clarke, Hyphanet's creator was selected as one of the top 100 innovators of 2003 by MIT's Technology Review magazine.
An important recent development, which very few other networks have, is the "darknet": By only connecting to people they trust, users can greatly reduce their vulnerability, and yet still connect to a global network through their friends' friends' friends and so on. This enables people to use Hyphanet even in places where Hyphanet may be illegal, makes it very difficult for governments to block it, and does not rely on tunneling to the "free world".
Written by Ian Clarke
- A Disclaimer
- Suggested prior reading
- The importance of the Free flow of information
- Censorship and freedom
- The solution
- Isn't censorship sometimes necessary?
- But why is anonymity necessary?
- And what of copyright?
- But how will artists be rewarded for their work without copyright?
- Alternatives to Copyright
The Philosophy Behind Hyphanet
There are many reasons why people get involved in the Hyphanet Project. Some share the views outlined in this document; some share variations of these views, which are also served by what we are trying to achieve; and some just enjoy the technical challenge. These are the ideas which motivated me to architect the system in the first place, but not necessarily the views that everyone involved in the Hyphanet Project holds.
Suggested prior reading
For this document to make sense, you should probably know what Hyphanet is. You can get a good overview from What is Hyphanet?.
The importance of the Free flow of information
Freedom of speech, in most western cultures, is generally considered to be one of the most important rights any individual might have. Why is the freedom to share ideas and opinions so important? There are several ways to answer this question.
Communication is what makes us human
One of the most obvious differences between mankind and the rest of the animal kingdom is our ability to communicate sophisticated and abstract concepts. While we constantly discover that animal's communication ability is more sophisticated than previously assumed, it is unlikely that any other animal approaches our own level of ability in this area.
Knowledge is good
Most people, given the option of knowing something and not knowing something, will choose to have more information rather than less. Wars have been won and lost over who was better-informed. This is because being better-informed allows us to make better decisions, and generally improve our ability to survive and be successful.
Democracy assumes a well informed population
Many people today live under democratic governments, and those who don't, probably want to. Democracy is an answer to the question of how to create leaders, while preventing them from abusing that power. It achieves this by giving the population the power to regulate their government through voting, yet the ability to vote does not necessarily mean that you live in a democratic country. For a population to regulate their government effectively it must know what their government is doing, they must be well informed. It is a feedback loop, but this loop can be broken if the government has the power to control the information the population has access to.
Censorship and freedom
Everyone values their freedom, in fact, many consider it so important that they will die for it. People like to think that they are free to form and hold whatever opinions they like, particularly in western countries. Consider now that someone had the ability to control the information you have access to. This would give them the ability to manipulate your opinions by hiding some facts from you, by presenting you with lies and censoring anything that contradicted those lies. This is not some Orwellian fiction, it is standard practice for most western governments to lie to their populations, so much so, that people now take it for granted, despite the fact that this undermines the very democratic principles which justify the government's existence in the first place.
The only way to ensure that a democracy will remain effective is to ensure that the government cannot control its population's ability to share information, to communicate. So long as everything we see and hear is filtered, we are not truly free. Hyphanet's aim is to allow two or more people who wish to share information, to do so.
Isn't censorship sometimes necessary?
Of course no issue is black and white, and there are many who feel that censorship is a good thing in some circumstances. For example, in some European countries propagating information deemed to be racist is illegal. Governments seek to prevent people from advocating ideas which are deemed damaging to society. There are two answers to this however. The first is that you can't allow those in power to impose "good" censorship, without also enabling them to impose "bad" censorship. To impose any form of censorship a government must have the ability to monitor and thus restrict communication. There are already criticisms that the anti-racism censorship in many European countries is hampering legitimate historical analysis of events such as the second world war.
The second argument is that this "good" censorship is counter-productive even when it does not leak into other areas. For example, it is generally more effective when trying to persuade someone of something to present them with the arguments against it, and then answer those arguments. Unfortunately, preventing people from being aware of the often sophisticated arguments used by racists, makes them vulnerable to those arguments when they do eventually encounter them.
Of course the first argument is the stronger one, and would still hold-true even if you didn't accept the second. Basically, you either have censorship, or you don't. There is no middle-ground.
But why is anonymity necessary?
You cannot have freedom of speech without the option to remain anonymous. Most censorship is retrospective, it is generally much easier to curtail free speech by punishing those who exercise it afterward, rather than preventing them from doing it in the first place. The only way to prevent this is to remain anonymous. It is a common misconception that you cannot trust anonymous information. This is not necessarily true, using digital signatures people can create a secure anonymous pseudonym which, in time, people can learn to trust. Hyphanet incorporates a mechanism called "subspaces" to facilitate this.
And what of copyright?
Of course much of Hyphanet's publicity has centered around the issue of copyright, and thus I will speak to it briefly. The core problem with copyright is that enforcement of it requires monitoring of communications, and you cannot be guaranteed free speech if someone is monitoring everything you say. This is important, most people fail to see or address this point when debating the issue of copyright, so let me make it clear:
You cannot guarantee freedom of speech and enforce copyright law
It is for this reason that Hyphanet, a system designed to protect Freedom of Speech, must prevent enforcement of copyright.
But how will artists be rewarded for their work without copyright?
Firstly, even if copyright were the only way that artists could be rewarded for their work, then I would contend that freedom is more important than having professional artists (those who claim that we would have no art do not understand creativity: people will always create, it is a compulsion, the only question is whether they can do it for a living).
Secondly, it could be questioned whether copyright is effective even now. The music industry is one of the most vocally opposed to enhancements in communication technology, yet according to many of the artists who should be rewarded by copyright, it is failing to do so. Rather it has allowed middle-men to gain control over the mechanisms of distribution, to the detriment of both artists and the public.
Alternatives to Copyright
Fortunately it won't come to this. There are many alternative ways to reward artists. The simplest is voluntary payment. This is an extension of the patronage system which was frequently used to reward artists prior to copyright, where a wealthy person would fund an artist to allow them to create full-time. The Internet permits an interesting extension of this idea, where rather than having just one wealthy patron, you could have hundreds of thousands, contributing small amounts of money over the Internet.
We actually practice what we preach in this regard too, on the 15th of March 2001 the Hyphanet Project started taking donations, and within a week we had collected over $1000.
- Ian Clarke
Hyphanet is based on Ian's paper "A Distributed Decentralised Information Storage and Retrieval System". Ian started the Hyphanet Project around July of 1999, and continues to coordinate the project. Learn more about Ian on his personal website.
- Florent Daignière
Since 2003, Florent has improved various aspects of the software and performed the project's system administration. In his day job, he is the Technical Director of Matta Consulting, a boutique security consultancy firm and currently works on safepass.me, an Active Directory password filter.
- Steve Dougherty
The current release manager. He joined in GSoC 2013 and has been a driving force behind tackling long standing issues in Hyphanet.
The Developer of the Web of Trust and Freetalk. He worked on the Web of Trust in part-time for one year and is now working as volunteer again.
- David (Bombe) Roden
The developer of the site insertion tool jSite and of Sone, the Social Network over Hyphanet.
A long term contributor who, among other things, helps keep the contact between the core developers and users in active subgroups.
A Gentoo packager who untangled all the dependencies of Hyphanet and packaged it cleanly in Gentoo.
- Arne Babenhauserheide
The current release manager of Hyphanet and maintainer of pyHyphanet and infocalypse. He also writes articles and tutorials for Hyphanet.
A dilligent team of people from various backgrounds who make it possible to ship Hyphanet and this website in many different languages.
Many more great hackers
This list is missing many freesite authors, plugin writers, and a host of other people who contributed in various ways.
Works on Hyphanet core and communicates via FMS.
The developer of the Hyphanet-based Forum system FMS, of HyphanetHG and of FLIP, chat over Hyphanet.
- The folks from Frost
A group of users and programmers who use an old spammable Hyphanet-based forum system which has been abandoned by most of the core developers. They are active, however, and though it takes time for their contributions to reach to core development, they take part in Hyphanet development.
- Matthew Toseland
Matthew has been working on Hyphanet since before the 0.5 release. His work and that of others has resulted in dramatic improvements to the performance and stability of the network.
- Oskar Sandberg
Oskar was also one of the earliest contributors to the Hyphanet Project, and has made some important theoretical breakthroughs that lead to the beginning of Hyphanet 0.7, see the papers page.
- Michael Rogers
Michael has mostly contributed detailed simulations as part of the Google Summer of Code. He has been helpful in designing the new transport layer.
- Ximin Luo
A debian developer who currently worked on packaging Hyphanet.
- Bert Massop
Works on the Hyphanet core and wherever there is need.
- Thomas Markus
A dutch developer and statistic-enthusiast. He now works at Topicus.Education.
- Scott Miller
Scott is responsible for the implementation of much of the cryptography elements within Hyphanet.
- Steven Starr
Steven helps with administration of Hyphanet Project Inc, and is an advisor to the project on business and publicity matters.
- Dave Baker
Dave's main contribution has been Freemail, his Summer of Code project to build a working email-over-Hyphanet system, as well as some debugging and core work in various places.
- Robert Hailey
Robert has helped improve the speed and security of Hyphanet by finding two major bugs, and has recently contributed some code.
- David Sowder
David (Zothar) has helped the Hyphanet Project as time permits and interest directs, including configuration, statistics and peer management via FCP, the FProxy stats page and Node 2 Node Messages (N2NM/N2NTMs).
And hundreds of others, who either haven't asked to be added here, who prefer to remain nameless, or who we just haven't got around to thanking. Not to mention thousands of users, testers, and donors!
Measuring Freenet in the Wild: Censorship-resilience under Observation (PDF) Observations and measurements on the live Hyphanet network. Includes suggestions for improvement. This was submitted to PETS 2014.
The Dark Freenet (PDF) Detailed paper about the Hyphanet 0.7.5 network, as opposed to its routing algorithm, which is detailed in the below papers. Includes some new simulations. This has been submitted to PET 2010.
Video of Small World talk, Berlin, December 2005
This is a video of a talk given by Ian Clarke and Oskar Sandberg at the Chaos Computer Congress in Berlin, December 2005, describing the (then) new architecture for Hyphanet 0.7. You can also download the slideshow, and the source for the Java demo (requires Java 1.5).
Searching in a Small World (PDF) Oskar Sandberg's licentiate thesis describing a simple decentralized mechanism for constructing small world networks that is inspired by Hyphanet's original design. Part II of the thesis describes the basis for the new Darknet architecture.
Distributed routing in Small World Networks (PDF) A paper by Oskar Sandberg describing the theoretical basis for the new "Darknet" routing mechanism employed by Hyphanet 0.7.
Chaos Computer Congress Talk (slideshow)
This is a slideshow for a talk given at the Chaos Computer Congress on 30th Dec 2005 in Berlin, Germany by Ian Clarke and Oskar Sandberg. It described the new "darknet" approach to be employed in Hyphanet 0.7. A Java demonstration to accompany the talk is also available.
Switching for a small world (PDF) A thesis by Vilhelm Verendel exploring ways to optimise the swapping algorithm.
Protecting Freedom of Information Online with Freenet (PDF) An IEEE Internet Computing article describing the Hyphanet architecture circa 2002 - probably the best introduction to the theory behind Hyphanet.
FreeNet White Paper (PDF) Original white paper by Ian Clarke, Division of Informatics, University of Edinburgh 1999.
Attack Resistant Network Embeddings for Darknets (PDF)
A proposal for changing the darknet swapping algorithm which we are still considering (we have some doubts about long-term performance).
A Contribution to Analyzing and Enhancing Darknet Routing (PDF)
A proposal for changing the routing algorithm which we are still considering (the worst case performance i.e. when a block has been lost may be unacceptable).
Presentation: Towards "Dark" Social Networking Services (Strufe et al.) (PDF) An interesting presentation by the group responsible for the two above papers.
Pisces: Anonymous Communication Using Social Networks
An algorithm for setting up onion-like tunnels on darknets.
Routing in the Dark: Pitch Black (citeseer) (PDF)
A paper describing some attacks on Hyphanet 0.7's location swapping algorithm. We have solutions for this but they are still being tested.
The most up to date reference is of course the source code, but there is also some useful documentation on the wiki (you may have to search a bit), and most implemented ideas have been discussed in detail on the mailing lists at some point, more recently often in-Hyphanet forums such as FMS, or the bug tracker hosted by MantisHub.