Commit e5c19657 authored by Bernd Paysan's avatar Bernd Paysan

Avalanche tree added to distributed data

parent 838ec57b
This diff is collapsed.
# Distributed Data #
_This is still quite incomplete_
Following the “everything is a file” philosophy of Unix, every data object is
a file. It's uniquely referenced by its hash. Further metadata are called
“tags”, and organized in a distributed hash table. There are also “subjects”
(persons, computers), which are referenced by their public keys; necessary
metadata for those subjects is also found in the DHT.
As the DHT contains all the metadata, objects which are not shared public
should also not be visible in a public hash table; therefore, there are private
or group-related hash tables, as well.
## Efficient distribution of data to large numbers of peers ##
For distributing data to many peers, these peers are arranged in a colored
tree. Data (e.g. video streems) are divided into different chunks, and sent
down different colored branches of the tree. The leaf nodes of each colored
branch then distribute the data to the other branches. It can be shown that
each node receives as much data as it sends (when the tree is balanced). The
latency of the tree is O(log n); the actual base is a tradeoff of bandwidth
and sending latency. The rule of thumb is to use the hop-to-hop latency time
to send out packets, so higher latency per hop means higher fanout of the
tree.
Trees are formed ad hoc, and since nodes can come and go as they like,
there needs to be self-healing capabilities. Nodes know 2n neighbors for
a tree base n. These trees are used for file-sharing, for group message
delivery, and to keep the DHT in sync.
![Avalance](../doc/avalanche.svg)
<h1>Distributed Data</h1>
<div><i>This is still quite incomplete</i></div>
<div>Following the "everything is a file" philosophy of Unix, every data object
is a file. &nbsp;It's uniquely referenced by its hash. &nbsp;Further metadata
are called "tags", and organized in a distributed prefix hash tree. &nbsp;There
are also "subjects" (persons, computers), which are referenced by their public
keys; necessary metadata for those subjects is also found in the DPHT.</div>
<div>As the DPHT contains all the metadata, objects which are not shared public
should also not be visible in a public hash tree; therefore, there are private
or group-related hash trees, as well.</div>
<h2>Efficient distribution of data to large numbers of peers</h2>
<div>For distributing data to many peers, these peers are arranged in a colored
tree. &nbsp;Data (e.g. video streams) are divided into different chunks, and
sent down different colored branches of the tree. &nbsp;The leaf nodes of each
colored branch then distribute the data to the other branches. &nbsp;It can be
shown that each node receives as much data as it sends (when the tree is
balanced). &nbsp;The latency of the tree is O(log n); the actual base is a
tradeoff of bandwidth and sending latency. &nbsp;The rule of thumb is to use
the hop-to-hop latency time to send out packets, so higher latency means higher
fanout of the tree.</div>
<div>Trees are formed ad hoc, and since nodes can come and go as they like,
there needs to be self-healing capabilities. &nbsp;Nodes know 2n neighbors for
a tree base n. &nbsp;These trees are used for file-sharing, for group message
delivery, and to keep the DPHT in sync.</div>
\ No newline at end of file
......@@ -128,6 +128,7 @@ point:
* [$quid CryptoCurrency](squid.md)
* [Guidelines of Conduct](guidelines.md)
* [Social network](social.md)
* [Search](search.md)
[de](/net2o/wiki?name=net2o.de)
[中文](net2o.zh.md)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment