Commit 50003a3d authored by Guillaume Bury's avatar Guillaume Bury

Merge branch 'master' of https://git.erp5.org/repos/vifibnet

Conflicts:
	vifibnet.py
parents 356e1468 3368dc9b
...@@ -2,8 +2,8 @@ Vifibnet is a daemon setting up a resilient virtual private network over the ...@@ -2,8 +2,8 @@ Vifibnet is a daemon setting up a resilient virtual private network over the
internet internet
HOW TO: HOW TO:
Vifibnet ( sic ) has three separate components : a setup, a server and Vifibnet ( sic ) has three separate components : a setup (setup.py), a
a client. server (registry.py) and a client (vifibnet.py.
Lambda users only have to launch the setup and then their client. Lambda users only have to launch the setup and then their client.
The server is meant to be started once on a node which also will be running The server is meant to be started once on a node which also will be running
a client instance. a client instance.
......
G : We should focus on clearing the TODO List, then go to testing phase,
since the end is nearing... ( I finish on august, 3 )
To be done : To be done :
Catch a more precise exception thant Exception at line 108 in vifibnet.py
( UPnP forwarding )
Upgrade the logging function in order to be able to log message like Upgrade the logging function in order to be able to log message like
"Refreshing peers DB ... done", or add log messages to specify that an "Refreshing peers DB ... done", or add log messages to specify that an
action advertised by a previous log message has been completed action advertised by a previous log message has been completed
...@@ -14,6 +8,8 @@ To be done : ...@@ -14,6 +8,8 @@ To be done :
Use an algorithm to choose which connections to keep and/or establish Use an algorithm to choose which connections to keep and/or establish
instead of pure randomness instead of pure randomness
|-> number of routes / tunnel : doesn't work
|-> favorise most used roads ?
Replace comments at the beginning of functions with docstrings & give all Replace comments at the beginning of functions with docstrings & give all
fn docstrings fn docstrings
...@@ -24,27 +20,12 @@ To be done : ...@@ -24,27 +20,12 @@ To be done :
Use a timeout for the server peersDB so we can flag unreachable peers and Use a timeout for the server peersDB so we can flag unreachable peers and
remove the peers whose certificate is no longer valid remove the peers whose certificate is no longer valid
Specify a lease duration in ForwardViaUPnP Specify a lease duration in ForwardViaUPnP and also forward a tcp port
Handle LAN internally in order not to have catastrophic results .... Handle LAN internally in order not to have catastrophic results ....
( avahi could be used ) ( avahi could be used )
To be discussed: To be discussed:
Ideas for the name :
CON ( cloud overlay network ), cloudnet. From J.P -> already taken
G, J : To get traffic stats ( bytes in/out ), you can use
/sys/class/net/interface/statistics/rx_bytes, etc...
or /proc/net/dev/snmp6/interface ( all in one file ). This can be enough
if used as follows: get traffic diff from last time we checked in order
to choose which connection is significantly unused compared to others,
and close it. Of course, too recent connections (i.e. those for which we
have no previous stat) would be always kept.
This should be combined with routing table (i.e. how many nodes are
served by each tunnel), which is possibly redundant.
ip6tables should be avoided if possible.
U : Great !!!
U : Babel seems to be very long to establish the routes : maybe we should U : Babel seems to be very long to establish the routes : maybe we should
tell him thant we are not on a wired network but on a mobile network ? tell him thant we are not on a wired network but on a mobile network ?
G : babel establish routes quickly enough i'd say. There are two new G : babel establish routes quickly enough i'd say. There are two new
...@@ -69,28 +50,13 @@ To be discussed: ...@@ -69,28 +50,13 @@ To be discussed:
on nexedi's server downtime ? it could be useful for the internship on nexedi's server downtime ? it could be useful for the internship
rapport rapport
U : The peer DB size should depend on the number of connection and the
refresh time
G : ?! I don't agree, the db size should be proportional ( with a factor
like 0.01 or less ) to the total number of peers in the entire network,
with maybe a max size.
U : what we need to do is to keep the randomness. For this, we need a big
enought DB to ensure we can still choose a peer as if it was choosen
directly from the server. The requiered db size can be calculated from
the number of connections and the refresh time.
G : ok, you can erase this talk
U : Why are --ip and internal-port mutually exclusive ?
Currently upnp only forward via UDP. Should he also forward via TCP ?
Why dont we only use UDP ?
No error should be raised when no upnp is detected : we should allow
machines having public IP to do an automatic configuration using the
discovery by an other peer
G : Actually, i was wrong, --ip and internal-port are no longer exclusive
Julien said udp might not be used by some people because of
restrictions imposed by the ISP ( FAI in french ), so we should
allow both, and act according to the options specifying which servers
to start (upd, tcp-server)
G : I think the number of route going through an interface should be a G : I think the number of route going through an interface should be a
Connection attribute, not a dict in tunnelManager Connection attribute, not a dict in tunnelManager
U : Yes, it was planned, just wait for me to finish implementing it
U : '--up', 'ovpn-server %s/%u' % (server_ip, len(network)) in plib.py
if you use len(network), this means that all our network is on the
same LAN and that the interface of the server is connected to it
wich means that any packet should be routed to this interface
an interface should only advertise the /64 (or less) which has been
attributed to it
...@@ -22,7 +22,7 @@ def server(server_ip, ip_length, max_clients, dh_path, pipe_fd, port, proto, hel ...@@ -22,7 +22,7 @@ def server(server_ip, ip_length, max_clients, dh_path, pipe_fd, port, proto, hel
return openvpn(hello_interval, return openvpn(hello_interval,
'--tls-server', '--tls-server',
'--mode', 'server', '--mode', 'server',
'--up', 'ovpn-server %s/%u' % (server_ip, ip_length), '--up', 'ovpn-server %s/%u' % (server_ip, 64),
'--client-connect', 'ovpn-server ' + str(pipe_fd), '--client-connect', 'ovpn-server ' + str(pipe_fd),
'--client-disconnect', 'ovpn-server ' + str(pipe_fd), '--client-disconnect', 'ovpn-server ' + str(pipe_fd),
'--dh', dh_path, '--dh', dh_path,
......
...@@ -2,31 +2,64 @@ ...@@ -2,31 +2,64 @@
#include <cmath> #include <cmath>
#include <map> #include <map>
#include <queue> #include <queue>
#include <stack>
void erase(vector<int>& v, int value)
{
for(int i=0;;i++)
if(v[i] == value)
{
v[i] = v.back();
v.pop_back();
break;
}
}
Graph::Graph(int size, int k, int maxPeers, mt19937& rng) : Graph::Graph(int size, int k, int maxPeers, mt19937& rng) :
distrib(uniform_int_distribution<int>(0, size-1)), distrib(uniform_int_distribution<int>(0, size-1)),
size(size), generator(rng) size(size), generator(rng), k(k), maxPeers(maxPeers)
{ {
adjacency = new vector<int>[size]; adjacency = new vector<int>[size];
generated = new vector<int>[size];
for(int i=0; i<size; i++) for(int i=0; i<size; i++)
{ {
int otherNode;
unordered_set<int> alreadyConnected; unordered_set<int> alreadyConnected;
alreadyConnected.insert(i); alreadyConnected.insert(i);
for(int j=0; j<k; j++) for(int j=0; j<k; j++)
{ {
int otherNode; while(alreadyConnected.count(otherNode = distrib(generator)) >= 1
|| otherNode > i && adjacency[otherNode].size() > maxPeers-k
while(alreadyConnected.count(otherNode = distrib(rng)) == 1
|| otherNode > i && adjacency[otherNode].size() > maxPeers-10
|| adjacency[otherNode].size() > maxPeers) || adjacency[otherNode].size() > maxPeers)
{ } { }
adjacency[i].push_back(otherNode); adjacency[i].push_back(otherNode);
generated[i].push_back(otherNode);
adjacency[otherNode].push_back(i); adjacency[otherNode].push_back(i);
} }
} }
} }
int Graph::AddEdge(int from, unordered_set<int>& alreadyConnected)
{
int otherNode;
while(alreadyConnected.count(otherNode = distrib(generator)) >= 1
|| (adjacency[otherNode].size() > maxPeers ))
{ }
adjacency[from].push_back(otherNode);
generated[from].push_back(otherNode);
adjacency[otherNode].push_back(from);
return otherNode;
}
int Graph::RemoveEdge(int from, int to)
{
erase(generated[from], to);
erase(adjacency[from], to);
erase(adjacency[to], from);
}
void Graph::GetDistancesFrom(int node, int* distance) void Graph::GetDistancesFrom(int node, int* distance)
{ {
for(int j=0; j<size; j++) for(int j=0; j<size; j++)
...@@ -50,6 +83,54 @@ void Graph::GetDistancesFrom(int node, int* distance) ...@@ -50,6 +83,54 @@ void Graph::GetDistancesFrom(int node, int* distance)
} }
} }
void Graph::GetRoutesFrom(int node, int* distance, float* routesCount)
{
unordered_set<int> prevs[size];
for(int i=0; i<size; i++)
{
distance[i] = -1;
routesCount[i] = 1;
}
distance[node] = 0;
queue<int> remainingNodes;
remainingNodes.push(node);
stack<int> order;
// Get the order
while(!remainingNodes.empty())
{
int node = remainingNodes.front();
int d = distance[node];
remainingNodes.pop();
order.push(node);
for(int neighbor : adjacency[node])
if(distance[neighbor] == -1)
{
distance[neighbor] = d+1;
prevs[neighbor].insert(node);
remainingNodes.push(neighbor);
}
else if(distance[neighbor] == d+1)
prevs[neighbor].insert(node);
}
// get the BC
while(!order.empty())
{
int node = order.top();
order.pop();
float w = routesCount[node];
for(int i : prevs[node])
routesCount[i] += w;
}
}
int Graph::CountUnreachableFrom(int node) int Graph::CountUnreachableFrom(int node)
{ {
bool accessibility[size]; bool accessibility[size];
......
...@@ -14,8 +14,8 @@ Results Simulate(int seed, int n, int k, int maxPeer, int maxDistanceFrom, floa ...@@ -14,8 +14,8 @@ Results Simulate(int seed, int n, int k, int maxPeer, int maxDistanceFrom, floa
for(int r=0; r<runs; r++) for(int r=0; r<runs; r++)
{ {
Graph graph(n, k, maxPeer, rng); Graph graph(n, k, maxPeer, rng);
graph.KillMachines(alivePercent); //graph.KillMachines(alivePercent);
results.AddAccessibilitySample(((double)graph.CountUnreachableFrom(0))/((double)n)); //results.AddAccessibilitySample(((double)graph.CountUnreachableFrom(0))/((double)n));
//int minCut = graph.GetMinCut(); //int minCut = graph.GetMinCut();
//if(results.minKConnexity == -1 || results.minKConnexity > minCut) //if(results.minKConnexity == -1 || results.minKConnexity > minCut)
//results.minKConnexity = minCut; //results.minKConnexity = minCut;
...@@ -28,6 +28,56 @@ Results Simulate(int seed, int n, int k, int maxPeer, int maxDistanceFrom, floa ...@@ -28,6 +28,56 @@ Results Simulate(int seed, int n, int k, int maxPeer, int maxDistanceFrom, floa
graph.GetDistancesFrom(i, distance); graph.GetDistancesFrom(i, distance);
results.UpdateDistance(distance, graph.size); results.UpdateDistance(distance, graph.size);
}*/ }*/
int distance[graph.size];
float routesCount[graph.size];
int nRefresh = 1;
graph.GetDistancesFrom(0, distance);
double moy = 0;
for(int i=0; i<graph.size; i++)
moy += distance[i];
moy /= graph.size;
cout << "Avg distance : " << moy << endl; cout.flush();
for(int i = 0; i<100; i++)
{
for(int j=0; j<graph.size; j++)
{
graph.GetRoutesFrom(j, distance, routesCount);
unordered_set<int> alreadyConnected;
// erase some edge
for(int k=0; k<nRefresh; k++)
{
int minNode = -1;
int minimum = -1;
for(int index = 0; index < graph.generated[j].size(); index++)
if(minNode == -1 || routesCount[graph.generated[j][index]] < minimum)
{
minNode = graph.generated[j][index];
minimum = routesCount[minNode];
}
graph.RemoveEdge(j, minNode);
}
// Add new edges
alreadyConnected.insert(j);
for(int k : graph.adjacency[j])
alreadyConnected.insert(k);
for(int k=0; k<nRefresh; k++)
alreadyConnected.insert(graph.AddEdge(j, alreadyConnected));
}
graph.GetDistancesFrom(0, distance);
moy = 0;
for(int i=0; i<graph.size; i++)
moy += distance[i];
moy /= graph.size;
cout << "Avg distance : " << moy << endl;
}
} }
results.Finalise(); results.Finalise();
...@@ -46,12 +96,12 @@ int main(int argc, char** argv) ...@@ -46,12 +96,12 @@ int main(int argc, char** argv)
vector<future<string>> outputStrings; vector<future<string>> outputStrings;
for(int n=2000; n<=2000; n*=2) for(int n=2000; n<=2000; n*=2)
for(int k=10; k<=10; k+=5) for(int k=10; k<=10; k+=5)
for(float a=0.01; a<=1; a+=0.05) for(float a=1; a<=1; a+=0.05)
{ {
int seed = rng(); int seed = rng();
outputStrings.push_back(async(launch::async, [seed, n, k, a]() outputStrings.push_back(async(launch::async, [seed, n, k, a]()
{ {
Results results = Simulate(seed, n, k, 3*k, 10000, a, 1); Results results = Simulate(seed, n, k, 2.5*k, 10000, a, 1);
ostringstream out; ostringstream out;
out << n << "," << k << "," << a << "," << 3*k << "," out << n << "," << k << "," << a << "," << 3*k << ","
<< results.avgDistance << "," << results.avgDistance << ","
......
...@@ -35,22 +35,28 @@ class Graph ...@@ -35,22 +35,28 @@ class Graph
{ {
public: public:
Graph(int size, int k, int maxPeers, mt19937& rng); Graph(int size, int k, int maxPeers, mt19937& rng);
~Graph() { delete[] adjacency; }; ~Graph() { delete[] adjacency; delete[] generated; };
void GetDistancesFrom(int node, int* distance); void GetDistancesFrom(int node, int* distance);
int AddEdge(int from, unordered_set<int>& alreadyConnected);
int RemoveEdge(int from, int to);
int GetMinCut(); int GetMinCut();
int CountUnreachableFrom(int node); int CountUnreachableFrom(int node);
void GetRoutesFrom(int node, int* distance, float* routesCount);
void KillMachines(float proportion); void KillMachines(float proportion);
//void SplitAS(float proportionAS1, float proportionAS2); //void SplitAS(float proportionAS1, float proportionAS2);
vector<int>* adjacency; vector<int>* adjacency;
vector<int>* generated;
int size; int size;
private: private:
int GetMinCut(MinCutGraph& graph); int GetMinCut(MinCutGraph& graph);
uniform_int_distribution<int> distrib; uniform_int_distribution<int> distrib;
mt19937& generator; mt19937& generator;
int maxPeers;
int k;
}; };
class Results class Results
......
...@@ -14,6 +14,7 @@ class Connection: ...@@ -14,6 +14,7 @@ class Connection:
os.O_WRONLY|os.O_CREAT|os.O_TRUNC)) os.O_WRONLY|os.O_CREAT|os.O_TRUNC))
self.iface = iface self.iface = iface
self.routes = 0
self._prefix = prefix self._prefix = prefix
self._creation_date = time.time() self._creation_date = time.time()
self._bandwidth = None self._bandwidth = None
...@@ -64,7 +65,7 @@ class TunnelManager: ...@@ -64,7 +65,7 @@ class TunnelManager:
self._write_pipe = write_pipe self._write_pipe = write_pipe
self._peer_db = peer_db self._peer_db = peer_db
self._connection_dict = {} self._connection_dict = {}
self._route_count = {} self._iface_to_prefix = {}
self._ovpn_args = openvpn_args self._ovpn_args = openvpn_args
self._hello = hello_interval self._hello = hello_interval
self._refresh_time = refresh self._refresh_time = refresh
...@@ -87,6 +88,7 @@ class TunnelManager: ...@@ -87,6 +88,7 @@ class TunnelManager:
def _cleanDeads(self): def _cleanDeads(self):
for prefix in self._connection_dict.keys(): for prefix in self._connection_dict.keys():
if not self._connection_dict[prefix].refresh(): if not self._connection_dict[prefix].refresh():
self._kill(prefix) self._kill(prefix)
...@@ -106,7 +108,7 @@ class TunnelManager: ...@@ -106,7 +108,7 @@ class TunnelManager:
pass pass
self.free_interface_set.add(connection.iface) self.free_interface_set.add(connection.iface)
self._peer_db.unusePeer(prefix) self._peer_db.unusePeer(prefix)
del self._route_count[connection.iface] del self._iface_to_prefix[connection.iface]
def _makeNewTunnels(self): def _makeNewTunnels(self):
utils.log('Trying to make %i new tunnels' % utils.log('Trying to make %i new tunnels' %
...@@ -119,7 +121,7 @@ class TunnelManager: ...@@ -119,7 +121,7 @@ class TunnelManager:
self._connection_dict[prefix] = Connection(address, self._connection_dict[prefix] = Connection(address,
self._write_pipe, self._hello, iface, self._write_pipe, self._hello, iface,
prefix, self._ovpn_args) prefix, self._ovpn_args)
self._route_count[iface] = 0 self._iface_to_prefix[iface] = prefix
self._peer_db.usePeer(prefix) self._peer_db.usePeer(prefix)
except KeyError: except KeyError:
utils.log("""Can't establish connection with %s utils.log("""Can't establish connection with %s
...@@ -129,16 +131,18 @@ class TunnelManager: ...@@ -129,16 +131,18 @@ class TunnelManager:
def _countRoutes(self): def _countRoutes(self):
utils.log('Starting to count the routes on each interface', 3) utils.log('Starting to count the routes on each interface', 3)
for iface in self._route_count.keys(): for iface in self._iface_to_prefix.keys():
self._route_count[iface] = 0 self._connection_dict[self._iface_to_prefix[iface]].routes = 0
f = open('/proc/net/ipv6_route', 'r') f = open('/proc/net/ipv6_route', 'r')
for line in f: for line in f:
ip, subnet_size, iface = struct.unpack("""32s x 2s x 32x x 2x x ip, subnet_size, iface = struct.unpack("""32s x 2s 106x
32x x 8x x 8x x 8x x 8x x %ss x""" % (len(line)-142), line) %ss x""" % (len(line)-142), line)
iface = iface.replace(' ', '') iface = iface.replace(' ', '')
if iface in self._route_count.keys(): if iface in self._iface_to_prefix.keys():
self._route_count[iface] += 1 self._connection_dict[self._iface_to_prefix[iface]].routes += 1
for iface in self._route_count.keys(): for p in self._connection_dict.keys():
utils.log('Routes on iface %s : %s' % (iface,self._route_count[iface] ), 5) utils.log('Routes on iface %s : %s' % (
self._connection_dict[p].iface,
self._connection_dict[p].routes ), 5)
...@@ -2,17 +2,32 @@ import miniupnpc ...@@ -2,17 +2,32 @@ import miniupnpc
import socket import socket
# return (address, port) # return (address, port)
def ForwardViaUPnP(local_port): def ForwardViaUPnP(local_port, protos):
u = miniupnpc.UPnP() u = miniupnpc.UPnP()
u.discoverdelay = 200 u.discoverdelay = 200
u.discover() u.discover()
u.selectigd() u.selectigd()
external_port = 1194 external_port = 1000
while True: while True:
while u.getspecificportmapping(external_port, 'UDP') != None: if 'udp' in protos:
external_port = max(externalPort + 1, 49152) while u.getspecificportmapping(external_port, 'UDP') != None :
external_port += 1
if external_port == 65536:
raise Exception
if 'tcp-server' in protos:
while u.getspecificportmapping(external_port, 'TCP') != None :
external_port += 1
if external_port == 65536: if external_port == 65536:
raise Exception raise Exception
if u.addportmapping(external_port, 'UDP', u.lanaddr, local_port, 'Vifib openvpn server', ''):
if 'udp' in protos:
u.addportmapping(external_port, 'UDP', u.lanaddr, local_port,
'Vifib openvpn server', '')
if 'tcp-server' in protos:
u.addportmapping(external_port, 'TCP', u.lanaddr, local_port,
'Vifib openvpn server', '')
print (u.externalipaddress(), external_port)
return (u.externalipaddress(), external_port) return (u.externalipaddress(), external_port)
...@@ -101,8 +101,8 @@ def main(): ...@@ -101,8 +101,8 @@ def main():
else: else:
utils.log('Attempting automatic configuration via UPnP', 4) utils.log('Attempting automatic configuration via UPnP', 4)
try: try:
ext_ip, ext_port = upnpigd.ForwardViaUPnP(config.internal_port) ext_ip, ext_port = upnpigd.ForwardViaUPnP(config.internal_port, config.proto)
config.address = list([ext_ip, ext_port, proto] config.address = list([ext_ip, str(ext_port), proto]
for proto in config.proto) for proto in config.proto)
except Exception: except Exception:
utils.log('An atempt to forward a port via UPnP failed', 4) utils.log('An atempt to forward a port via UPnP failed', 4)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment