In simulation : we can now see the effects of the tunnel manager

in upnpigd : now open a port in tcp too

README

@@ -2,8 +2,8 @@ Vifibnet is a daemon setting up a resilient virtual private network over the
 internet
 
 HOW TO:
-    Vifibnet ( sic ) has three separate components : a setup, a server and
-    a client.
+    Vifibnet ( sic ) has three separate components : a setup (setup.py), a 
+    server (registry.py) and a client (vifibnet.py.
     Lambda users only have to launch the setup and then their client.
     The server is meant to be started once on a node which also will be running
     a client instance.

TODO

-G : We should focus on clearing the TODO List, then go to testing phase,
-    since the end is nearing... ( I finish on august, 3 )
-
 To be done :
-    Catch a more precise exception thant Exception at line 108 in vifibnet.py
-    ( UPnP forwarding )
-
     Upgrade the logging function in order to be able to log message like
     "Refreshing peers DB ... done", or add log messages to specify that an
     action advertised by a previous log message has been completed
@@ -14,6 +8,8 @@ To be done :
 
     Use an algorithm to choose which connections to keep and/or establish
     instead of pure randomness
+        |-> number of routes / tunnel : doesn't work
+        |-> favorise most used roads ?
 
     Find a name for the project : the projet is ( or should be ) independant
     from vifib, openvpn, babel, and so should the name; btw we aim to build a
@@ -28,14 +24,11 @@ To be done :
     Use a timeout for the server peersDB so we can flag unreachable peers and
     remove the peers whose certificate is no longer valid
 
-    Specify a lease duration in ForwardViaUPnP
+    Specify a lease duration in ForwardViaUPnP and also forward a tcp port
 
     Handle LAN internally in order not to have catastrophic results ....
     ( avahi could be used )
 
-    When we count the number of routes througt an interface, we should filter on
-    the prefix size and the subnet
-
 To be discussed:
     U : Babel seems to be very long to establish the routes : maybe we should
         tell him thant we are not on a wired network but on a mobile network ?
@@ -61,18 +54,6 @@ To be discussed:
         on nexedi's server downtime ? it could be useful for the internship
         rapport
 
-    U : Why are --ip and internal-port mutually exclusive ?
-        Currently upnp only forward via UDP. Should he also forward via TCP ?
-        Why dont we only use UDP ?
-        No error should be raised when no upnp is detected : we should allow
-        machines having public IP to do an automatic configuration using the
-        discovery by an other peer
-    G : Actually, i was wrong, --ip and internal-port are no longer exclusive
-        Julien said udp might not be used by some people because of
-        restrictions imposed by the ISP ( FAI in french ), so we should
-        allow both, and act according to the options specifying which servers
-        to start (upd, tcp-server)
-
     G : I think the number of route going through an interface should be a
         Connection attribute, not a dict in tunnelManager
     U : Yes, it was planned, just wait for me to finish implementing it

plib.py

@@ -22,8 +22,7 @@ def server(server_ip, network, max_clients, dh_path, pipe_fd, port, proto, hello
     return openvpn(hello_interval,
         '--tls-server',
         '--mode', 'server',
-        #'--up', 'ovpn-server %s/%u' % (server_ip, len(network)),
-        '--up', 'ovpn-server %s/%u' % (server_ip, 64), # Isn't this better ?
+        '--up', 'ovpn-server %s/%u' % (server_ip, 64), 
         '--client-connect', 'ovpn-server ' + str(pipe_fd),
         '--client-disconnect', 'ovpn-server ' + str(pipe_fd),
         '--dh', dh_path,

simulation/graph.cpp

@@ -2,31 +2,64 @@
 #include <cmath>
 #include <map>
 #include <queue>
+#include <stack>
+
+void erase(vector<int>& v, int value)
+{
+    for(int i=0;;i++)
+        if(v[i] == value)
+        {
+            v[i] = v.back();
+            v.pop_back();
+            break;
+        }
+}
 
 Graph::Graph(int size, int k, int maxPeers, mt19937& rng) :
     distrib(uniform_int_distribution<int>(0, size-1)),
-    size(size), generator(rng)
+    size(size), generator(rng), k(k), maxPeers(maxPeers)
 {
     adjacency = new vector<int>[size];
+    generated = new vector<int>[size];
+
     for(int i=0; i<size; i++)
     {
+        int otherNode;
         unordered_set<int> alreadyConnected;
         alreadyConnected.insert(i);
 
         for(int j=0; j<k; j++)
         {
-            int otherNode;
-
-            while(alreadyConnected.count(otherNode = distrib(rng)) == 1
-                || otherNode > i && adjacency[otherNode].size() > maxPeers-10
+            while(alreadyConnected.count(otherNode = distrib(generator)) >= 1
+                || otherNode > i && adjacency[otherNode].size() > maxPeers-k
                 || adjacency[otherNode].size() > maxPeers)
             { }
             adjacency[i].push_back(otherNode);
+            generated[i].push_back(otherNode);
             adjacency[otherNode].push_back(i);
         }
     }
 }
 
+int Graph::AddEdge(int from, unordered_set<int>& alreadyConnected)
+{
+    int otherNode;
+    while(alreadyConnected.count(otherNode = distrib(generator)) >= 1
+        || (adjacency[otherNode].size() > maxPeers ))
+    { }
+    adjacency[from].push_back(otherNode);
+    generated[from].push_back(otherNode);
+    adjacency[otherNode].push_back(from);
+    return otherNode;
+}
+
+int Graph::RemoveEdge(int from, int to)
+{
+    erase(generated[from], to);
+    erase(adjacency[from], to);
+    erase(adjacency[to], from);
+}
+
 void Graph::GetDistancesFrom(int node, int* distance)
 {
     for(int j=0; j<size; j++)
@@ -50,6 +83,54 @@ void Graph::GetDistancesFrom(int node, int* distance)
     }
 }
 
+void Graph::GetRoutesFrom(int node, int* distance, float* routesCount)
+{
+    
+    unordered_set<int> prevs[size];
+
+    for(int i=0; i<size; i++)
+    {
+        distance[i] = -1;
+        routesCount[i] = 1;
+    }
+    distance[node] = 0;
+
+    queue<int> remainingNodes;
+    remainingNodes.push(node);
+
+    stack<int> order;
+
+    // Get the order
+    while(!remainingNodes.empty())
+    {
+        int node = remainingNodes.front();
+        int d = distance[node];
+        remainingNodes.pop();
+        order.push(node);
+
+        for(int neighbor : adjacency[node])
+            if(distance[neighbor] == -1)
+            {
+                distance[neighbor] = d+1;
+                prevs[neighbor].insert(node);
+                remainingNodes.push(neighbor);
+            }
+            else if(distance[neighbor] == d+1)
+                prevs[neighbor].insert(node);
+    }
+
+    // get the BC
+    while(!order.empty())
+    {
+        int node = order.top();
+        order.pop();
+        float w = routesCount[node];
+        for(int i : prevs[node])
+            routesCount[i] += w;
+    }
+
+}
+
 int Graph::CountUnreachableFrom(int node)
 {
     bool accessibility[size];

simulation/main.cpp

@@ -14,8 +14,8 @@ Results Simulate(int seed,  int n, int k, int maxPeer, int maxDistanceFrom, floa
     for(int r=0; r<runs; r++)
     {
         Graph graph(n, k, maxPeer, rng);
-        graph.KillMachines(alivePercent);
-        results.AddAccessibilitySample(((double)graph.CountUnreachableFrom(0))/((double)n));
+        //graph.KillMachines(alivePercent);
+        //results.AddAccessibilitySample(((double)graph.CountUnreachableFrom(0))/((double)n));
         //int minCut = graph.GetMinCut();
         //if(results.minKConnexity == -1 || results.minKConnexity > minCut)
         //results.minKConnexity = minCut;
@@ -28,6 +28,56 @@ Results Simulate(int seed,  int n, int k, int maxPeer, int maxDistanceFrom, floa
             graph.GetDistancesFrom(i, distance);
             results.UpdateDistance(distance, graph.size);
         }*/
+
+        int distance[graph.size];
+        float routesCount[graph.size];
+        int nRefresh = 1;
+
+        graph.GetDistancesFrom(0, distance);
+        double moy = 0;
+        for(int i=0; i<graph.size; i++)
+            moy += distance[i];
+        moy /= graph.size;
+        cout << "Avg distance : " << moy << endl; cout.flush();
+
+        for(int i = 0; i<100; i++)
+        {
+            for(int j=0; j<graph.size; j++)
+            {
+                graph.GetRoutesFrom(j, distance, routesCount);
+                unordered_set<int> alreadyConnected;
+
+                // erase some edge
+                for(int k=0; k<nRefresh; k++)
+                {
+                    int minNode = -1;
+                    int minimum = -1;
+                    for(int index = 0; index < graph.generated[j].size(); index++)
+                        if(minNode == -1 || routesCount[graph.generated[j][index]] < minimum)
+                        {
+                            minNode = graph.generated[j][index];
+                            minimum = routesCount[minNode];
+                        }
+
+                    graph.RemoveEdge(j, minNode);
+                }
+
+                // Add new edges
+                alreadyConnected.insert(j);
+                for(int k : graph.adjacency[j])
+                    alreadyConnected.insert(k);
+
+                for(int k=0; k<nRefresh; k++)
+                    alreadyConnected.insert(graph.AddEdge(j, alreadyConnected));
+            }
+
+            graph.GetDistancesFrom(0, distance);
+            moy = 0;
+            for(int i=0; i<graph.size; i++)
+                moy += distance[i];
+            moy /= graph.size;
+            cout << "Avg distance : " << moy << endl;
+        }
     }
 
     results.Finalise();
@@ -46,12 +96,12 @@ int main(int argc, char** argv)
     vector<future<string>> outputStrings;
     for(int n=2000; n<=2000; n*=2)
         for(int k=10; k<=10; k+=5)
-            for(float a=0.01; a<=1; a+=0.05)
+            for(float a=1; a<=1; a+=0.05)
             {
                 int seed = rng();
                 outputStrings.push_back(async(launch::async, [seed, n, k, a]()
                     {
-                        Results results = Simulate(seed, n, k, 3*k, 10000, a, 1);
+                        Results results = Simulate(seed, n, k, 2.5*k, 10000, a, 1);
                         ostringstream out;
                         out << n << "," << k << "," << a << "," << 3*k << ","
                             << results.avgDistance << ","

simulation/main.h

@@ -35,22 +35,28 @@ class Graph
 {
 public:
     Graph(int size, int k, int maxPeers, mt19937& rng);
-    ~Graph() { delete[] adjacency; };
+    ~Graph() { delete[] adjacency; delete[] generated; };
 
     void GetDistancesFrom(int node, int* distance);
+    int AddEdge(int from, unordered_set<int>& alreadyConnected);
+    int RemoveEdge(int from, int to);
     int GetMinCut();
     int CountUnreachableFrom(int node);
+    void GetRoutesFrom(int node, int* distance, float* routesCount);
 
     void KillMachines(float proportion);
     //void SplitAS(float proportionAS1, float proportionAS2);
 
     vector<int>* adjacency;
+    vector<int>* generated;
     int size;
 private:
     int GetMinCut(MinCutGraph& graph);
 
     uniform_int_distribution<int> distrib;
     mt19937& generator;
+    int maxPeers;
+    int k;
 };
 
 class Results

tunnel.py

@@ -88,6 +88,7 @@ class TunnelManager:
 
     def _cleanDeads(self):
         for prefix in self._connection_dict.keys():
+
             if not self._connection_dict[prefix].refresh():
                 self._kill(prefix)
 

upnpigd.py

@@ -2,17 +2,32 @@ import miniupnpc
 import socket
 
 # return (address, port)
-def ForwardViaUPnP(local_port):
+def ForwardViaUPnP(local_port, protos):
     u = miniupnpc.UPnP()
     u.discoverdelay = 200
     u.discover()
     u.selectigd()
-    external_port = 1194
+    external_port = 1000
+
     while True:
-        while u.getspecificportmapping(external_port, 'UDP') != None:
-            external_port = max(externalPort + 1, 49152)
+        if 'udp' in protos:
+            while u.getspecificportmapping(external_port, 'UDP') != None :
+                external_port += 1
+                if external_port == 65536:
+                    raise Exception
+        if 'tcp-server' in protos:
+            while u.getspecificportmapping(external_port, 'TCP') != None :
+                external_port += 1
                 if external_port == 65536:
                     raise Exception
-        if u.addportmapping(external_port, 'UDP', u.lanaddr, local_port, 'Vifib openvpn server', ''):
+
+        if 'udp' in protos:
+            u.addportmapping(external_port, 'UDP', u.lanaddr, local_port,
+                'Vifib openvpn server', '')
+        if 'tcp-server' in protos:
+            u.addportmapping(external_port, 'TCP', u.lanaddr, local_port, 
+                'Vifib openvpn server', '')
+
+        print (u.externalipaddress(), external_port)
         return (u.externalipaddress(), external_port)
 

vifibnet.py

@@ -101,8 +101,8 @@ def main():
     else:
         utils.log('Attempting automatic configuration via UPnP', 4)
         try:
-            ext_ip, ext_port = upnpigd.ForwardViaUPnP(config.internal_port)
-            config.address = list([ext_ip, ext_port, proto]
+            ext_ip, ext_port = upnpigd.ForwardViaUPnP(config.internal_port, config.proto)
+            config.address = list([ext_ip, str(ext_port), proto]
                                   for proto in config.proto)
         except Exception:
             utils.log('An atempt to forward a port via UPnP failed', 4)