[P2P] Setting up IPOP

[Javier Ruiz Cosignani]

Hello everybody. First of all thank you for letting me be part of this
group.

Well I am writing this new post because of several doubts and
difficulties I have ran into trying to run IPOP. Most probably this
difficulties are due to me not knowing some important things of the
setup process. So I hope that the experts could give me a hand with
this.

Right now I have managed to run P2PNode.exe contained in the binary
release of IPOP (http://www.grid-appliance.org/files/ipop/ipop.zip). I
used, as the guide recommended the Current-Free-to-Join-Brunet-
Overlays (http://www.grid-appliance.org/wiki/index.php/
CurrentOverlays), and changed the content of the tag <BrunetNamespace>
to "ZyndariusTest" (I don't know if this is right but I read it was
suggested to avoid collisions). The port for the XmlRpcManager is
10000.
So my first step was running the following command on a Debian Squeeze
system:

mono P2PNode.exe --IpopConfig=ipop.config --
NodeConfig=node.config.icebear.P2PNode --Count=12 (about the number of
virtual nodes I am not sure if the option was necessary, since I am
trying to run the system using only to physical nodes I decided to run
the instance of P2PNode.exe with virtual nodes)

the output of the program was the following:

I am connected to ZyndariusTest as
brunet:node:P2C2PSMDRGGYXCOU27W2GBXIKWEKUHVM. Current time is
4/27/2010 11:44:55 AM.

I thing this meant that my first step was successful. Nonetheless I
have doubts if everything was alright. It is supposed that I created a
new namespace called "ZyndariusTest" in the overlay? was this the
right thing to do?

As second step I used the script bput.py to deploy the DhtDHCPServer
using the example configuration shown in the guide (http://www.grid-
appliance.org/documentation/IpopNodeReadme.html) and changing the
content of the tag <Namespace> to "ZynIPOP1" for example. I deleted
all the whitespaces (leaving just one big line within the file) and
used it as the script's input.
The command I ran was:

python bput.py --ttl=3600 --input=dhcpconfig.xml
dhcp:ipop_namespace:ZynIPOP1 (I procured that this namespace match the
IpopNamespace contained within the ipop.config file used to run
P2PNode.exe)

Upon using the script sometimes it did not work but some others did
work saying as output "True". When it did not work it threw a very
ugly error that i beg you excuse me for posting it but one never knows
where important details could be:

Traceback (most recent call last):
File "bput.py", line 41, in <module>
print rpc.localproxy("DhtClient.Put", xmlrpclib.Binary(args[0]),
xmlrpclib.Binary(value), ttl)
File "/usr/lib/python2.5/xmlrpclib.py", line 1147, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.5/xmlrpclib.py", line 1437, in __request
verbose=self.__verbose
File "/usr/lib/python2.5/xmlrpclib.py", line 1201, in request
return self._parse_response(h.getfile(), sock)
File "/usr/lib/python2.5/xmlrpclib.py", line 1340, in
_parse_response
return u.close()
File "/usr/lib/python2.5/xmlrpclib.py", line 787, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: 'Server returned a fault exception: [0]
Brunet.AdrException: Operation failed Total/Positive/Negative:
8/0/4\n at Brunet.RpcResult.AssertNotException () [0x00000] \n at
Brunet.RpcResult.get_Result () [0x00000] \n at
Brunet.Rpc.XmlRpcManager.Proxy (ISender sender, Int32
maxResultsToWait, System.String method, System.Object[] args)
[0x00000] \nError Code: 0\nthrown by: Brunet.StructuredNode'>

As I was reading about running IPOP some FAQ said something about
running the script every 30 seconds until it returned true. So it lead
me to think that this was part of the "normal" behavior. But if
anybody can enlighten me about it I will be very thankful.

When the script returned true I went to my 3rd step which was running
DhtIpopNode.exe as root user for this I typed the following command:

mono DhtIpopNode.exe --NodeConfig=node.config.1 --
IpopConfig=ipop.config &> logs &

The only difference between node.config.1, used here, with node.config
used to start P2PNode.exe is the port of the XmlRpcManager that I had
to change to 10001 becuase if not the DhtIpopNode.exe won't start and
the logs contained an error saying the following: (I beg for excuses
again :s)

Starting IPOP: 4/27/2010 12:19:38 PM

Unhandled Exception: System.Net.Sockets.SocketException: Address
already in use
at System.Net.Sockets.Socket.Bind (System.Net.EndPoint local_end)
[0x00000]
at System.Runtime.Remoting.Channels.Http.RemotingHttpListener..ctor
(System.Net.IPAddress addr, Int32 port,
System.Runtime.Remoting.Channels.Http.HttpServerTransportSink sink)
[0x00000]
at
System.Runtime.Remoting.Channels.Http.HttpServerChannel.StartListening
(System.Object data) [0x00000]
[3]+ Exit 1 mono DhtIpopNode.exe --
NodeConfig=node.config.icebear.P2PNode --IpopConfig=ipop.config &>logs

Having the port changed in node.config.1 the logs just reported the
"Starting IPOP: 4/27/2010 12:19:38 PM" and IPOP started without
problem for the tap device turned alive (I procured to have the tun
module previously added to the kernel as instructed in the guide).

The main problem is that when reaching this stage the tap device does
not get IP Address when doing dhclient tapipop and in Wireshark, apart
from other packets sniffed, the device keeps sending DHCP Discover
messages but there is no response. What could be happening?, I have
revised the dhcpconfig.xml file used to run the bput.py script and
everything seems in order. I have also read about using the bget.py
script but I don't know what to use as valid arguments. Could anybody
guide me an give me a generic example of how to use it to see if the
DhtDHCPServer is working.

Thanks for your patience and time. =)

--
You received this message because you are subscribed to the Google Groups "acis.p2p" group.
To post to this group, send email to aci...@googlegroups.com.
To unsubscribe from this group, send email to acisp2p+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/acisp2p?hl=en.

[David Isaac Wolinsky]

On 04/27/2010 08:32 AM, Javier Ruiz Cosignani wrote:
> Hello everybody. First of all thank you for letting me be part of this
> group.
>
> Well I am writing this new post because of several doubts and
> difficulties I have ran into trying to run IPOP. Most probably this
> difficulties are due to me not knowing some important things of the
> setup process. So I hope that the experts could give me a hand with
> this.
>
> Right now I have managed to run P2PNode.exe contained in the binary
> release of IPOP (http://www.grid-appliance.org/files/ipop/ipop.zip). I
> used, as the guide recommended the Current-Free-to-Join-Brunet-
> Overlays (http://www.grid-appliance.org/wiki/index.php/
> CurrentOverlays), and changed the content of the tag<BrunetNamespace>
> to "ZyndariusTest" (I don't know if this is right but I read it was
> suggested to avoid collisions). The port for the XmlRpcManager is
> 10000.
>

You should use a unique Ipop Namespace, not Brunet, if you want to
leverage an existing bootstrap. Otherwise, you won't be leveraging the
bootstrap. (The BrunetNamespace prevents two peers in different P2P
systems from connecting with each other). I think you just wanted to
ensure someone else using IPOP, wouldn't collide with your ipop system.
(Check IpopConfig)

> So my first step was running the following command on a Debian Squeeze
> system:
>
> mono P2PNode.exe --IpopConfig=ipop.config --
> NodeConfig=node.config.icebear.P2PNode --Count=12 (about the number of
> virtual nodes I am not sure if the option was necessary, since I am
> trying to run the system using only to physical nodes I decided to run
> the instance of P2PNode.exe with virtual nodes)
>

If you're using a shared overlay, you probably don't need to use any
P2PNodes. If you're using a private overlay, you'll appreciate having
some virtual nodes, as they'll keep the underlying p2p system content.

> the output of the program was the following:
>
> I am connected to ZyndariusTest as
> brunet:node:P2C2PSMDRGGYXCOU27W2GBXIKWEKUHVM. Current time is
> 4/27/2010 11:44:55 AM.
>
> I thing this meant that my first step was successful. Nonetheless I
> have doubts if everything was alright. It is supposed that I created a
> new namespace called "ZyndariusTest" in the overlay? was this the
> right thing to do?
>

No, this creates a new overlay called ZyndariusTest.

> As second step I used the script bput.py to deploy the DhtDHCPServer
> using the example configuration shown in the guide (http://www.grid-
> appliance.org/documentation/IpopNodeReadme.html) and changing the
> content of the tag<Namespace> to "ZynIPOP1" for example. I deleted
> all the whitespaces (leaving just one big line within the file) and
> used it as the script's input.
> The command I ran was:
>
> python bput.py --ttl=3600 --input=dhcpconfig.xml
> dhcp:ipop_namespace:ZynIPOP1 (I procured that this namespace match the
> IpopNamespace contained within the ipop.config file used to run
> P2PNode.exe)
>

This has been superseded by specifying the dhcp config to DhtIpopNode
directly... it resolved some of the awkward components (such as
attempting to republish every 30 seconds until success).

This isn't pretty, its a result of an XML parsing bug. The XML configs,
on some platforms, seems to have such behavior.

> As I was reading about running IPOP some FAQ said something about
> running the script every 30 seconds until it returned true. So it lead
> me to think that this was part of the "normal" behavior. But if
> anybody can enlighten me about it I will be very thankful.
>

See, above, (pass the dhcp config in directly to dhtipopnode, see mono
DhtIpopNode.exe --help)

> When the script returned true I went to my 3rd step which was running
> DhtIpopNode.exe as root user for this I typed the following command:
>
> mono DhtIpopNode.exe --NodeConfig=node.config.1 --
> IpopConfig=ipop.config&> logs&
>
> The only difference between node.config.1, used here, with node.config
> used to start P2PNode.exe is the port of the XmlRpcManager that I had
> to change to 10001 becuase if not the DhtIpopNode.exe won't start and
> the logs contained an error saying the following: (I beg for excuses
> again :s)
>
> Starting IPOP: 4/27/2010 12:19:38 PM
>
> Unhandled Exception: System.Net.Sockets.SocketException: Address
> already in use
> at System.Net.Sockets.Socket.Bind (System.Net.EndPoint local_end)
> [0x00000]
> at System.Runtime.Remoting.Channels.Http.RemotingHttpListener..ctor
> (System.Net.IPAddress addr, Int32 port,
> System.Runtime.Remoting.Channels.Http.HttpServerTransportSink sink)
> [0x00000]
> at
> System.Runtime.Remoting.Channels.Http.HttpServerChannel.StartListening
> (System.Object data) [0x00000]
>

Makes sense, you can't (easily) bind twice to the same TCP port.

> [3]+ Exit 1 mono DhtIpopNode.exe --

> NodeConfig=node.config.icebear.P2PNode --IpopConfig=ipop.config&>logs

>
> Having the port changed in node.config.1 the logs just reported the
> "Starting IPOP: 4/27/2010 12:19:38 PM" and IPOP started without
> problem for the tap device turned alive (I procured to have the tun
> module previously added to the kernel as instructed in the guide).
>
> The main problem is that when reaching this stage the tap device does
> not get IP Address when doing dhclient tapipop and in Wireshark, apart
> from other packets sniffed, the device keeps sending DHCP Discover
> messages but there is no response. What could be happening?, I have
> revised the dhcpconfig.xml file used to run the bput.py script and
> everything seems in order. I have also read about using the bget.py
> script but I don't know what to use as valid arguments. Could anybody
> guide me an give me a generic example of how to use it to see if the
> DhtDHCPServer is working.
>

You should enable logging and see what's going on (copy log.config to
DhtIpopNode.exe.config and enable pertinent options). It appears that
the Ipop nodes may not be connecting to the P2P nodes. Also check the
system health with the crawler, described at
http://www.grid-appliance.org/wiki/index.php/BasicNodeReadme

I'll work on writing a new DhtIpopNode in the near future. Also, if you
want to leverage our shared overlay, you should consider using the
GroupVPN, it makes using ipop significantly easier.
http://www.grid-appliance.org/wiki/index.php/GroupVPN

> Thanks for your patience and time. =)
>
>

Glad you're continuing on. I'll run through the documentation to make
it more clear.

Cheers,
David

[Javier Ruiz Cosignani]

First of all, thanks for your reply.

Now, to see if I am getting this.
The first thing I have to correct is the BrunetNamespace. Since I am
planning to use, at first instance, your free to join overlay the name
had to be
<BrunetNamespace>ufl_test0_15</BrunetNamespace> right?
Because as you said, you are right. My intention was to run an IPOP
system using your free to join overlay and there "play" a little.

About the virtual nodes. In the case of using your overlay there are
not necessary right? Then if I try to setup an independent overlay
(i.e. using a different BrunetNamespace) I will need them to make the
system "healthier". Correct me if I am mistaken, please.

One more question about the DhtDHCPServer. If I pass the "dhcpconfig"
file directly to DhtIpopNode.exe I don't have to use the script
bput.py to put the DHCPServer into the DHT right?

Making the corrections I think were necessary. Now when I run:
mono P2PNode.exe --IpopConfig=ipop.config --NodeConfig=node.config
it returns:
I am connected to ufl_test0_15 as
brunet:node:NAV3V3AW4GRZM5UASIG3K35XRUFHZUFM. Current time is
4/28/2010 10:32:03 AM.

As I am understanding now, I joined the ufl_test0_15 overlay as a P2P
node, instead of creating another overlay, right?

After that, when trying to pass the dhcpconfig file directly to
DhtIpopNode.exe I get an error and IPOP doesn't start:

mono DhtIpopNode.exe --NodeConfig=node.config --IpopConfig=ipop.config
--DhcpConfig=dhcpconfig.xml

and the output error is:

Invalid DhcpConfig file:
XmlSchema error: Element declaration for DHCPServerConfig is missing.
XML URI: file:///home/zyndarius/Dropbox/05_3rd_term_2010/ipop_sources_and_binary/ipop/bin/dhcpconfig.xml
. Line 1, Position 2.

I don't know what is happening because this dhcpconfig file is exactly
the same that I was using with the bput.py script. Am I missing
something here, some modifications that I have to do to the file?

Again, thanks in advance for the help. I appreciate it a lot =)

> system health with the crawler, described athttp://www.grid-appliance.org/wiki/index.php/BasicNodeReadme

[David Isaac Wolinsky]

Sorry Javier,

I was in the middle of traveling yesterday and didn't have time to write
back. Responses inlined.

On 04/28/2010 08:05 AM, Javier Ruiz Cosignani wrote:
> First of all, thanks for your reply.
>
> Now, to see if I am getting this.
> The first thing I have to correct is the BrunetNamespace. Since I am
> planning to use, at first instance, your free to join overlay the name
> had to be
> <BrunetNamespace>ufl_test0_15</BrunetNamespace> right?
> Because as you said, you are right. My intention was to run an IPOP
> system using your free to join overlay and there "play" a little.
>

Correct.

> About the virtual nodes. In the case of using your overlay there are
> not necessary right? Then if I try to setup an independent overlay
> (i.e. using a different BrunetNamespace) I will need them to make the
> system "healthier". Correct me if I am mistaken, please.
>

I would say more along the lines of calmer. Most P2P systems
aggressively try to maintain X amount of connections, when it has less
than X, it can be difficult to switch it from that state, especially
when there are a lot of moving pieces. By running many virtual nodes,
you ensure that a single machine in the overlay does not stay in this
volatile state. The state isn't all that bad, it probably just has a
negative effect on latencies in the system.

> One more question about the DhtDHCPServer. If I pass the "dhcpconfig"
> file directly to DhtIpopNode.exe I don't have to use the script
> bput.py to put the DHCPServer into the DHT right?
>

Correct

> Making the corrections I think were necessary. Now when I run:
> mono P2PNode.exe --IpopConfig=ipop.config --NodeConfig=node.config
> it returns:
> I am connected to ufl_test0_15 as
> brunet:node:NAV3V3AW4GRZM5UASIG3K35XRUFHZUFM. Current time is
> 4/28/2010 10:32:03 AM.
>
> As I am understanding now, I joined the ufl_test0_15 overlay as a P2P
> node, instead of creating another overlay, right?
>

Correct.

> After that, when trying to pass the dhcpconfig file directly to
> DhtIpopNode.exe I get an error and IPOP doesn't start:
>
> mono DhtIpopNode.exe --NodeConfig=node.config --IpopConfig=ipop.config
> --DhcpConfig=dhcpconfig.xml
>
> and the output error is:
>
> Invalid DhcpConfig file:
> XmlSchema error: Element declaration for DHCPServerConfig is missing.
> XML URI: file:///home/zyndarius/Dropbox/05_3rd_term_2010/ipop_sources_and_binary/ipop/bin/dhcpconfig.xml
> . Line 1, Position 2.
>

I see, documentation is out of whack... look at
ipop/config/dhcp.config. Also I just updated the wiki to reflect (at a
minimum) the proper DHCPConfig.
http://www.grid-appliance.org/wiki/index.php/IpopNodeReadme

Sorry about the fragmentation, when we switched to the GroupVPN
methodology, we attempted to simplify some of the configuration files
and other components of the system. In the process, we further confused
individuals like you that want to work with the base system.

> I don't know what is happening because this dhcpconfig file is exactly
> the same that I was using with the bput.py script. Am I missing
> something here, some modifications that I have to do to the file?
>

bput just inserts it into the DHT (distributed light-weight database).
There is no real checking.

[Javier Ruiz Cosignani]

Thanks David for your valuable help.

The dhcpconfigfile indeed had another syntax and I paid no attention
to the example file contained in /ipop/config/

Using the syntax of that file everything went alright and the system
started without problems.

At first instance it took a little to get a valid and correct IP
address from the DHCP server but it made it after some tries of
dhclient tapipop.

In windows XP I noticed something weird. At first instance the tap
device acquired an IP address from somewhere else, but after that, it
acquired a correct IP address.

Well, thanks again for your disposal and time. I will continue
familiarizing myself with the system.

Regards,
Javier

> ...
>
> read more »

[David Isaac Wolinsky]

That's great, Javier, some comments inline.

On 04/30/2010 06:25 AM, Javier Ruiz Cosignani wrote:
> Thanks David for your valuable help.
>
> The dhcpconfigfile indeed had another syntax and I paid no attention
> to the example file contained in /ipop/config/
>
> Using the syntax of that file everything went alright and the system
> started without problems.
>
> At first instance it took a little to get a valid and correct IP
> address from the DHCP server but it made it after some tries of
> dhclient tapipop.
>

It could be due to the network your computers are behind. If it is
NATed in such a way that only outbound connections are viable, it can
sometimes take longer than desirable. Its a todo item to get this fixed.

> In windows XP I noticed something weird. At first instance the tap
> device acquired an IP address from somewhere else, but after that, it
> acquired a correct IP address.
>

That's related to the above problem. Windows applies an APIPA IP if a
device doesn't get a DHCP address in a reasonable amount of time.

> Well, thanks again for your disposal and time. I will continue
> familiarizing myself with the system.
>

You're welcome. Sounds good. Out of curiosity, how are you planning on
using it? And let us know how things turn out.

Cheers,
David

[Javier Ruiz Cosignani]

On May 5, 2:16 am, David Isaac Wolinsky <davi...@ufl.edu> wrote:
> That's great, Javier, some comments inline.
>
> On 04/30/2010 06:25 AM, Javier Ruiz Cosignani wrote:> Thanks David for your valuable help.
>
> > The dhcpconfigfile indeed had another syntax and I paid no attention
> > to the example file contained in /ipop/config/
>
> > Using the syntax of that file everything went alright and the system
> > started without problems.
>
> > At first instance it took a little to get a valid and correct IP
> > address from the DHCP server but it made it after some tries of
> > dhclient tapipop.
>
> It could be due to the network your computers are behind.  If it is
> NATed in such a way that only outbound connections are viable, it can
> sometimes take longer than desirable.  Its a todo item to get this fixed.> In windows XP I noticed something weird. At first instance the tap
> > device acquired an IP address from somewhere else, but after that, it
> > acquired a correct IP address.
>

> That's related to the above problem.  Windows applies an APIPA IP if a
> device doesn't get a DHCP address in a reasonable amount of time.> Well, thanks again for your disposal and time. I will continue
> > familiarizing myself with the system.
>

Ahaa, now I understand the why of the other provisional IP. I wasn't
aware of the APIPA mechanism on windows. Seems everyday one learns
something new =)

> You're welcome.  Sounds good.  Out of curiosity, how are you planning on
> using it?  And let us know how things turn out.
>

Well, I am interested in virtualization that IPOP provides and the
vision that several researchers have about a "future internet". So I
am trying IPOP becuase is a good platform to provide the kind of
separation mini Internets would have in the future. Broadcast and
multicast features are the ones I am studying right now, related to
the exact replication of a physical IP network. In case of ending with
any possible contribution to the system I would gladly let you know
but to be honest I still have pretty much to learn to come up with
something like that hehehe.

Regards,
Javier