Unable to create experiments for multiple nodes

eric

unread,

Jan 9, 2019, 4:26:30 AM1/9/19

to emulab-admins

Hello everyone, I can successfully create a single node experiment on testbed, but I can't create experiments for multiple nodes. Below are the ns files and log files for creating multiple node experiments. I hope to get some suggestions, thank you.

#generated by Netbuild 1.03

set ns [new Simulator]

source tb_compat.tcl

set node0 [$ns node]

set node1 [$ns node]

set link0 [$ns duplex-link $node0 $node1 100Mb 0ms DropTail]

$ns rtproto Static

$ns run

#netbuild-generated ns file ends.

Running 'tbprerun -e 60 twoNode1.ns'

Beginning pre run for testbed/twoNode1. 08:21:49:220821

Running parser ... 08:21:49:485984

Parser done! 08:21:52:288181

Precomputing visualization ...

Image rendering proceeding in background mode ...

Setting up static routes (if requested) ...

Generating topomap ...

Doing a pre-assign: '/usr/testbed/bin/vtopgen -p testbed twoNode1' ...

Checking for feature LibVtopTest.

Minimum nodes = 3

Maximum nodes = 3

Writing environment strings ...

Setting up additional program agent support ...

Setting up additional network agent support ...

Writing program agent info ...

Pre run finished. 08:21:54:333225

Running 'tbswap in testbed twoNode1'

Beginning swap-in for testbed/twoNode1 (60). 01/09/2019 08:21:54

TIMESTAMP: 08:21:54:809806 tbswap in started

Checking for feature SyncVlans.

Checking with Admission Control ...

Checking for feature ExternalNodeManagement.

Allocating public addresses...

Mapping to physical reality ...

TIMESTAMP: 08:21:55:182867 mapper wrapper started

Checking for feature OldMapper.

Checking for feature NewAssign.

Checking for feature ControlNetVlans.

Checking for feature LibVtopTest.

Starting the new and improved mapper wrapper.

Minimum nodes = 3

Maximum nodes = 3

Clearing physical state before updating.

Mapper loop 1

ptopargs: '-p testbed -e twoNode1 '

assign command: 'assign -P -f text testbed-twoNode1-1484.ptop testbed-twoNode1-1484.vtop'

Reading assign results.

pc10

pc4

pc6

TIMESTAMP: 08:21:58:943364 mapper wrapper finished

Mapped to physical reality!

Fetching tarballs and RPMs (if any) ...

TIMESTAMP: 08:21:58:948015 tarfiles_setup started

TIMESTAMP: 08:21:59:383122 tarfiles_setup finished

Creating dynamic blobs.

Locking down arp entries

TIMESTAMP: 08:21:59:454951 arplockdown started

TIMESTAMP: 08:21:59:665534 arplockdown finished

Setting up mountpoints.

TIMESTAMP: 08:21:59:667150 mountpoints started

TIMESTAMP: 08:22:02:163085 mountpoints finished

TIMESTAMP: 08:22:02:164224 named started

Setting up named maps.

TIMESTAMP: 08:22:02:594548 named finished

TIMESTAMP: 08:22:02:596457 gentopofile started

Generating ltmap (again) ...

TIMESTAMP: 08:22:02:945930 gentopofile finished

Resetting OS and rebooting.

TIMESTAMP: 08:22:02:948256 launching os_setup

Checking for feature OldOsSetup.

Setting up VLANs.

TIMESTAMP: 08:22:02:951331 snmpit started

TIMESTAMP: 08:22:03:362946 os_setup started

Adding pc6 to type object pc

pc6 - [OS 10000:0 emulab-ops,UBUNTU14-64-STD]

Adding pc4 to type object pc

pc4 - [OS 10000:0 emulab-ops,UBUNTU14-64-STD]

Adding pc10 to type object pc

pc10 - [OS 10003:0 emulab-ops,FBSD102-64-STD]

Asking [pc] for volunteers

[pc] returns volunteers: [Node: pc6] [Node: pc10] [Node: pc4]

Lighting up nodes in parallel ...

getTrunksForVlan: 5035: huawei2

getTrunksForVlan: 5034: huawei2

mapVlansToSwitches: huawei2

Lighting up nodes: [Node: pc6] [Node: pc10] [Node: pc4]

pc6 will be rebooted

pc10 will be reloaded with [Image 10003:0 emulab-ops,FBSD102-64-STD]

pc4 will be rebooted

osload: Using [Image 10003:0 emulab-ops,FBSD102-64-STD] for pc10

osload (pc10): Changing default OS to [OS 10003:0 emulab-ops,FBSD102-64-STD]

Setting up reload for pc10 (mode: Frisbee)

TIMESTAMP: 08:22:04:602704 osload: database setup done

osload: Issuing reboot for pc10 and then waiting ...

TIMESTAMP: 08:22:04:605110 osload: event handler enabled

reboot: no privs; invoking real nodereboot script!

reboot: Running in pipemode (26).

BATCH: rebooting pc10

reboot (pc10): Attempting to reboot ...

pc10: in PXEWAIT, sending PXEWAKEUP

reboot (pc10): Successful!

reboot: Done. There were 0 failures.

reboot (pc10): child returned 0 status.

Waiting for pc10 to finish reloading

Wed Jan 9 08:22:04 CST 2019

TIMESTAMP: 08:22:04:936344 osload: starting reload-done wait

reboot: no privs; invoking real nodereboot script!

reboot: Running in pipemode (23).

BATCH: rebooting pc4 pc6

reboot (pc4): Attempting to reboot ...

pc4: in PXEWAIT, sending PXEWAKEUP

reboot (pc6): Attempting to reboot ...

pc6: in PXEWAIT, sending PXEWAKEUP

reboot (pc4): Successful!

reboot (pc6): Successful!

reboot: Done. There were 0 failures.

reboot (pc6): child returned 0 status.

reboot (pc4): child returned 0 status.

huawei2::get1(sysDescr.0) ... will try again

osload: eventhandler: pc10 => RELOADSETUP @ 1546993355

osload: eventhandler: pc10 => RELOADING @ 1546993355

got 2 events

huawei2::get1(sysDescr.0) ... will try again

osload (pc10): still waiting; it has been 1 minute(s)

huawei2::get1(sysDescr.0) ... will try again

osload (pc10): left reloading mode at Wed Jan 9 08:23:50 CST 2019

osload: Done! There were 0 failures.

reload (pc10): child returned 0 status.

Waiting for nodes ...

pc6 has reported state ISUP

pc6 is alive and well

pc4 has reported state ISUP

pc4 is alive and well

huawei2::get1(sysDescr.0) ... will try again

*** os_setup: Still waiting for pc10 (BOOTING) - it's been 1 minute(s).

huawei2::get1(sysDescr.0) ... will try again

pc10 has reported state ISUP

pc10 is alive and well

huawei2::get1(sysDescr.0) ... will try again

Asking [pc] for volunteers

No virt info for node pc10 (tbdelay0)

TIMESTAMP: 08:25:20:483835 os_setup finished

huawei2::get1(sysDescr.0) ... will try again

huawei2::get1(sysDescr.0) .. giving up

Use of uninitialized value $snmpit_lib::snmpitErrorString in concatenation (.) or string at /usr/testbed/lib/snmpit_test/snmpit_huawei.pm line 1076, <DATA> line 6010.

huawei2::get1(sysDescr.0) failed -

WARNING: Unable to retrieve via SNMP from huawei2

Device huawei2 could not be instantiated,

*** ERROR: snmpit_test: huawei2 device setup failed

*** ERROR: tbswap: Failed to set up VLANs.

Cleaning up after errors.

Waiting for os_setup to finish

Stopping the event system

Checking for feature NewEventScheduler.

Closing TCP proxy ports...

Releasing public address pools...

TIMESTAMP: 08:26:46:104039 snmpit started

Removing VLANs.

snmpit: testbed/twoNode1 has VLANs, but none have been created on switches

... skipping

TIMESTAMP: 08:26:46:792483 snmpit finished

Removing dynamic blobs.

Clearing shared port vlans.

Tearing down virtual nodes.

TIMESTAMP: 08:26:46:831727 vnode_setup -k started

vnode_setup running at parallelization: 10 wait_time: 3000

Vnode teardown finished.

TIMESTAMP: 08:26:48:125027 vnode_setup finished

Removing logical wires.

Freeing nodes.

TIMESTAMP: 08:26:48:127975 nfree started

Releasing all nodes from experiment [Experiment: testbed/twoNode1].

Moving [Node: pc6] to [Experiment: emulab-ops/reloadpending]

Moving [Node: pc4] to [Experiment: emulab-ops/reloadpending]

Moving [Node: pc10] to [Experiment: emulab-ops/reloadpending]

TIMESTAMP: 08:26:48:721988 nfree finished

Resetting DB.

Failingly finished swap-in for testbed/twoNode1. 08:26:48:738780

TIMESTAMP: 08:26:48:739340 tbswap in finished (failed)

*** ERROR: batchexp: tbswap in failed!

Cleaning up and exiting with status 1 ...

Running 'tbend -f -e 60'

Beginning cleanup for testbed/twoNode1. 08:26:48:956303

Clearing out virtual state.

Removing visualization data...

Cleanup finished! 08:26:49:194587

**** Experimental information, please ignore ****

Session ID = 9182

Likely Cause of the Problem:

Failed to set up VLANs.

Cause: unknown

Confidence: 0.7

Script: tbswap

**** End experimental information ****

SENDMAIL: Could not finish sendmail:

Removing experiment directories ...

Leigh Stoller

unread,

Jan 9, 2019, 10:13:09 AM1/9/19

to emulab...@googlegroups.com, eric

> Hello everyone, I can successfully create a single node experiment on
> testbed, but I can't create experiments for multiple nodes. Below are
> the ns files and log files for creating multiple node experiments. I
> hope to get some suggestions, thank you.

Hi. Your problem appears to be here:

> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) ... will try again
> huawei2::get1(sysDescr.0) .. giving up
> Use of uninitialized value $snmpit_lib::snmpitErrorString in
> concatenation (.) or string at
> /usr/testbed/lib/snmpit_test/snmpit_huawei.pm line 1076, <DATA> line 6010.
> huawei2::get1(sysDescr.0) failed -
> WARNING: Unable to retrieve via SNMP from huawei2
> Device huawei2 could not be instantiated,
> *** ERROR: snmpit_test: huawei2 device setup failed
> *** ERROR: tbswap: Failed to set up VLANs.

We do not know anything about this switch or switch module (we did not
write it). You will need to provide more details. :-)

Leigh

Keith Sklower

unread,

Jan 10, 2019, 2:03:26 AM1/10/19

to emulab...@googlegroups.com

I did not write snmpit_huawei.pm .... however my impression is that the newer line of HP switches (the one running the comware software) are actually huawei swtiches OEMed by HP. Consequently the driver is likely to be very similar to snmpit_hp, and the reason that the driver failed here is that the switch was not allowed to have snmpv2 access from the boss, so I think it is likely a configuration error. I think it would be good for whoever did write snmpit_huawei.pm to contribute it back to emulab, along with the necessary set of MIBs.

Regards,

Keith Sklower

--
You received this message because you are subscribed to the Google Groups "emulab-admins" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emulab-admin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

eric

unread,

Jan 10, 2019, 8:24:00 AM1/10/19

to emulab-admins

Thank you for your reply. The detailed information you mentioned refers to the hardware information of the switch and snmpit_huawei.pm? The specific model of the switch, I need to check it out.

Message has been deleted

eric

unread,

Jan 10, 2019, 8:31:11 AM1/10/19

to emulab-admins

Thank you very much for your suggestion, I will try it.

eric

unread,

Jan 16, 2019, 3:20:33 AM1/16/19

to emulab-admins

I tried to configure snmpv2, the problem still exists. I found another problem, I can't ping the switch(192.168.1.2/192.168.1.3) from boss (10.1.32.70) but I can ping 192.168.1.1.

root@boss：~ #ping 192.168.1.2

PING 192.168.1.2 (192.168.1.2): 56data bytes

ping: sendto: Host is down

Reply all

Reply to author

Forward