Unable to create experiments for multiple nodes

44 views
Skip to first unread message

eric

unread,
Jan 9, 2019, 4:26:30 AM1/9/19
to emulab-admins
Hello everyone, I can successfully create a single node experiment on testbed, but I can't create experiments for multiple nodes. Below are the ns files and log files for creating multiple node experiments. I hope to get some suggestions, thank you.


#generated by Netbuild 1.03
set ns [new Simulator]
source tb_compat.tcl

set node0 [$ns node]
set node1 [$ns node]

set link0 [$ns duplex-link $node0 $node1 100Mb 0ms DropTail]

$ns rtproto Static
$ns run
#netbuild-generated ns file ends.


Running 'tbprerun  -e 60 twoNode1.ns'
Beginning pre run for testbed/twoNode1. 08:21:49:220821
Running parser ... 08:21:49:485984
Parser done! 08:21:52:288181
Precomputing visualization ...
Image rendering proceeding in background mode ...
Setting up static routes (if requested) ... 
Generating topomap ...
Doing a pre-assign: '/usr/testbed/bin/vtopgen -p testbed twoNode1' ...
Checking for feature LibVtopTest.
Minimum nodes   = 3
Maximum nodes   = 3
Writing environment strings ...
Setting up additional program agent support ...
Setting up additional network agent support ...
Writing program agent info ...
Pre run finished. 08:21:54:333225
Running 'tbswap in  testbed twoNode1'
Beginning swap-in for testbed/twoNode1 (60). 01/09/2019 08:21:54
TIMESTAMP: 08:21:54:809806 tbswap in started
Checking for feature SyncVlans.
Checking with Admission Control ...
Checking for feature ExternalNodeManagement.
Allocating public addresses...
Mapping to physical reality ...
TIMESTAMP: 08:21:55:182867 mapper wrapper started
Checking for feature OldMapper.
Checking for feature NewAssign.
Checking for feature ControlNetVlans.
Checking for feature LibVtopTest.
Starting the new and improved mapper wrapper.
Minimum nodes   = 3
Maximum nodes   = 3
Clearing physical state before updating.
Mapper loop 1
ptopargs: '-p testbed -e twoNode1 '
assign command: 'assign -P -f text testbed-twoNode1-1484.ptop testbed-twoNode1-1484.vtop'
Reading assign results.
pc10
pc4
pc6
TIMESTAMP: 08:21:58:943364 mapper wrapper finished
Mapped to physical reality!
Fetching tarballs and RPMs (if any) ...
TIMESTAMP: 08:21:58:948015 tarfiles_setup started
TIMESTAMP: 08:21:59:383122 tarfiles_setup finished
Creating dynamic blobs.
Locking down arp entries
TIMESTAMP: 08:21:59:454951 arplockdown started
TIMESTAMP: 08:21:59:665534 arplockdown finished
Setting up mountpoints.
TIMESTAMP: 08:21:59:667150 mountpoints started
TIMESTAMP: 08:22:02:163085 mountpoints finished
TIMESTAMP: 08:22:02:164224 named started
Setting up named maps.
TIMESTAMP: 08:22:02:594548 named finished
TIMESTAMP: 08:22:02:596457 gentopofile started
Generating ltmap (again) ...
TIMESTAMP: 08:22:02:945930 gentopofile finished
Resetting OS and rebooting.
TIMESTAMP: 08:22:02:948256 launching os_setup
Checking for feature OldOsSetup.
Setting up VLANs.
TIMESTAMP: 08:22:02:951331 snmpit started
TIMESTAMP: 08:22:03:362946 os_setup started
Adding pc6 to type object pc
pc6 - [OS 10000:0 emulab-ops,UBUNTU14-64-STD]
Adding pc4 to type object pc
pc4 - [OS 10000:0 emulab-ops,UBUNTU14-64-STD]
Adding pc10 to type object pc
pc10 - [OS 10003:0 emulab-ops,FBSD102-64-STD]
Asking [pc] for volunteers
[pc] returns volunteers: [Node: pc6] [Node: pc10] [Node: pc4]
Lighting up nodes in parallel ...
getTrunksForVlan: 5035: huawei2
getTrunksForVlan: 5034: huawei2
mapVlansToSwitches: huawei2
Lighting up nodes: [Node: pc6] [Node: pc10] [Node: pc4]
pc6 will be rebooted
pc10 will be reloaded with [Image 10003:0 emulab-ops,FBSD102-64-STD]
pc4 will be rebooted
osload: Using [Image 10003:0 emulab-ops,FBSD102-64-STD] for pc10
osload (pc10): Changing default OS to [OS 10003:0 emulab-ops,FBSD102-64-STD]
Setting up reload for pc10 (mode: Frisbee)
TIMESTAMP: 08:22:04:602704 osload: database setup done
osload: Issuing reboot for pc10 and then waiting ...
TIMESTAMP: 08:22:04:605110 osload: event handler enabled
reboot: no privs; invoking real nodereboot script!
reboot: Running in pipemode (26).
BATCH: rebooting pc10
reboot (pc10): Attempting to reboot ...
pc10: in PXEWAIT, sending PXEWAKEUP
reboot (pc10): Successful!
reboot: Done. There were 0 failures.
reboot (pc10): child returned 0 status.
Waiting for pc10 to finish reloading
Wed Jan  9 08:22:04 CST 2019
TIMESTAMP: 08:22:04:936344 osload: starting reload-done wait
reboot: no privs; invoking real nodereboot script!
reboot: Running in pipemode (23).
BATCH: rebooting pc4 pc6
reboot (pc4): Attempting to reboot ...
pc4: in PXEWAIT, sending PXEWAKEUP
reboot (pc6): Attempting to reboot ...
pc6: in PXEWAIT, sending PXEWAKEUP
reboot (pc4): Successful!
reboot (pc6): Successful!
reboot: Done. There were 0 failures.
reboot (pc6): child returned 0 status.
reboot (pc4): child returned 0 status.
huawei2::get1(sysDescr.0) ... will try again
osload: eventhandler: pc10 => RELOADSETUP @ 1546993355
osload: eventhandler: pc10 => RELOADING @ 1546993355
got 2 events
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
osload (pc10): still waiting; it has been 1 minute(s)
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
osload (pc10): left reloading mode at Wed Jan  9 08:23:50 CST 2019
osload: Done! There were 0 failures.
reload (pc10): child returned 0 status.
Waiting for nodes ...
pc6 has reported state ISUP
pc6 is alive and well
pc4 has reported state ISUP
pc4 is alive and well
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
*** os_setup: Still waiting for pc10 (BOOTING) - it's been 1 minute(s).
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
pc10 has reported state ISUP
pc10 is alive and well
huawei2::get1(sysDescr.0) ... will try again
Asking [pc] for volunteers
No virt info for node pc10 (tbdelay0)
TIMESTAMP: 08:25:20:483835 os_setup finished
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) ... will try again
huawei2::get1(sysDescr.0) .. giving up
Use of uninitialized value $snmpit_lib::snmpitErrorString in concatenation (.) or string at /usr/testbed/lib/snmpit_test/snmpit_huawei.pm line 1076, <DATA> line 6010.
huawei2::get1(sysDescr.0) failed - 
WARNING: Unable to retrieve via SNMP from huawei2
Device huawei2 could not be instantiated, 
*** ERROR: snmpit_test: huawei2 device setup failed
*** ERROR: tbswap: Failed to set up VLANs.
Cleaning up after errors.
Waiting for os_setup to finish
Stopping the event system
Checking for feature NewEventScheduler.
Closing TCP proxy ports...
Releasing public address pools...
TIMESTAMP: 08:26:46:104039 snmpit started
Removing VLANs.
snmpit: testbed/twoNode1 has VLANs, but none have been created on switches
  ... skipping
TIMESTAMP: 08:26:46:792483 snmpit finished
Removing dynamic blobs.
Clearing shared port vlans.
Tearing down virtual nodes.
TIMESTAMP: 08:26:46:831727 vnode_setup -k started
vnode_setup running at parallelization: 10 wait_time: 3000
Vnode teardown finished.
TIMESTAMP: 08:26:48:125027 vnode_setup finished
Removing logical wires.
Freeing nodes.
TIMESTAMP: 08:26:48:127975 nfree started
Releasing all nodes from experiment [Experiment: testbed/twoNode1].
Moving [Node: pc6] to [Experiment: emulab-ops/reloadpending]
Moving [Node: pc4] to [Experiment: emulab-ops/reloadpending]
Moving [Node: pc10] to [Experiment: emulab-ops/reloadpending]
TIMESTAMP: 08:26:48:721988 nfree finished
Resetting DB.
Failingly finished swap-in for testbed/twoNode1. 08:26:48:738780
TIMESTAMP: 08:26:48:739340 tbswap in finished (failed)
*** ERROR: batchexp: tbswap in failed!
Cleaning up and exiting with status 1 ... 
Running 'tbend -f -e 60'
Beginning cleanup for testbed/twoNode1. 08:26:48:956303
Clearing out virtual state.
Removing visualization data...
Cleanup finished! 08:26:49:194587
**** Experimental information, please ignore ****
Session ID = 9182
Likely Cause of the Problem:
  Failed to set up VLANs.
Cause: unknown
Confidence: 0.7
Script: tbswap
**** End experimental information ****
SENDMAIL: Could not finish sendmail: 
Removing experiment directories ... 

Leigh Stoller

unread,
Jan 9, 2019, 10:13:09 AM1/9/19
to emulab...@googlegroups.com, eric
> Hello everyone, I can successfully create a single node experiment on
> testbed, but I can't create experiments for multiple nodes. Below are
> the ns files and log files for creating multiple node experiments. I
> hope to get some suggestions, thank you.

Hi. Your problem appears to be here:

> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) .. giving up
> Use of uninitialized value $snmpit_lib::snmpitErrorString in
> concatenation (.) or string at
> /usr/testbed/lib/snmpit_test/snmpit_huawei.pm line 1076, <DATA> line 6010.
> huawei2::get1(sysDescr.0) failed -
> WARNING: Unable to retrieve via SNMP from huawei2
> Device huawei2 could not be instantiated,
> *** ERROR: snmpit_test: huawei2 device setup failed
> *** ERROR: tbswap: Failed to set up VLANs.


We do not know anything about this switch or switch module (we did not
write it). You will need to provide more details. :-)

Leigh

Keith Sklower

unread,
Jan 10, 2019, 2:03:26 AM1/10/19
to emulab...@googlegroups.com
I did not write snmpit_huawei.pm .... however my impression is that the newer line of HP switches (the one running the comware software) are actually huawei swtiches OEMed by HP.  Consequently the driver is likely to be very similar to snmpit_hp, and the reason that the driver failed here is that the switch was not allowed to have snmpv2 access from the boss, so I think it is likely a configuration error.  I think it would be good for whoever did write snmpit_huawei.pm to contribute it back to emulab, along with the necessary set of MIBs.

Regards,

Keith Sklower

--
You received this message because you are subscribed to the Google Groups "emulab-admins" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emulab-admin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

eric

unread,
Jan 10, 2019, 8:24:00 AM1/10/19
to emulab-admins
Thank you for your reply. The detailed information you mentioned refers to the hardware information of the switch and snmpit_huawei.pm? The specific model of the switch, I need to check it out.
Message has been deleted

eric

unread,
Jan 10, 2019, 8:31:11 AM1/10/19
to emulab-admins
Thank you very much for your suggestion, I will try it.

eric

unread,
Jan 16, 2019, 3:20:33 AM1/16/19
to emulab-admins
I tried to configure snmpv2, the problem still exists. I found another problem, I can't ping the switch(192.168.1.2/192.168.1.3)  from boss (10.1.32.70) but I can ping 192.168.1.1.

root@boss:~  #ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2): 56data bytes
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
Reply all
Reply to author
Forward
0 new messages