Running custom AFI using firesim

109 views

Skip to first unread message

Varun Gandhi

unread,

Dec 2, 2020, 8:56:05 PM12/2/20

to chip...@googlegroups.com

Hi,

I was able to build a medium-boom-quad-core config successfully, i.e., it comes up in built-hwdb-entries dir; but when I try to launch an F1 instance with it, I get the following error:

-------------

2020-12-03 01:49:39,304 [main        ] [INFO ]  FireSim Manager. Docs: http://docs.fires.im

Running: launchrunfarm

2020-12-03 01:49:39,304 [__init__    ] [DEBUG]  {'hwconf_dict': {'firesim-boom-singlecore-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f4dc0360b48>,

                 'firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f4dc0360b00>,

                 'firesim-rocket-quadcore-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f4dc0360bd8>,

                 'firesim-rocket-quadcore-no-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f4dc0360c68>,

                 'firesim-rocket-quadcore-no-nic-l2-llc4mb-ddr3-half-freq-uncore': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f4dc0360c20>,

                 'firesim-supernode-rocket-singlecore-nic-l2-lbp': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f4dc0360b90>}}

2020-12-03 01:49:39,312 [aws_resource] [DEBUG]  i-0c7b16b059cd11e69

2020-12-03 01:49:39,573 [aws_resource] [DEBUG]  {'Name': 'uvrs'}

2020-12-03 01:49:39,574 [__init__    ] [DEBUG]  {'autocounter_readrate': 0,

 'defaulthwconfig': 'firesim-boom-medium-quadcore-nic-l2-llc4mb-ddr3',

 'f1_16xlarges_requested': 0,

 'f1_2xlarges_requested': 0,

 'f1_4xlarges_requested': 1,

 'linklatency': 6405,

 'm4_16xlarges_requested': 0,

 'netbandwidth': 200,

 'no_net_num_nodes': 0,

 'print_cycle_prefix': True,

 'print_end': '-1',

 'print_start': '0',

 'profileinterval': -1,

 'run_instance_market': 'ondemand',

 'runfarmtag': 'mainrunfarm',

 'spot_interruption_behavior': 'terminate',

 'spot_max_price': 'ondemand',

 'suffixtag': '',

 'switchinglatency': 10,

 'terminateoncompletion': False,

 'topology': 'example_2config',

 'trace_enable': False,

 'trace_end': '-1',

 'trace_output_format': '0',

 'trace_select': '1',

 'trace_start': '0',

 'workload_name': 'fedora-uniform.json',

 'zerooutdram': False}

2020-12-03 01:49:39,580 [<module>    ] [ERROR]  Fatal error.

Traceback (most recent call last):

  File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 334, in <module>

    main(args)

  File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 280, in main

    simconf = RuntimeConfig(args)

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/runtime_config.py", line 359, in __init__

    self.innerconf.print_cycle_prefix)

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/firesim_topology_with_passes.py", line 66, in __init__

    self.phase_one_passes()

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/firesim_topology_with_passes.py", line 357, in phase_one_passes

    self.pass_apply_default_hwconfig()

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/firesim_topology_with_passes.py", line 279, in pass_apply_default_hwconfig

    defaulthwconfig_obj = self.hwdb.get_runtimehwconfig_from_name(self.defaulthwconfig)

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/runtime_config.py", line 222, in get_runtimehwconfig_from_name

    return self.hwconf_dict[name]

KeyError: 'firesim-boom-medium-quadcore-nic-l2-llc4mb-ddr3'

2020-12-03 01:49:39,580 [<module>    ] [INFO ]  The full log of this run is:

/home/centos/chipyard/sims/firesim/deploy/logs/2020-12-03--01-49-39-launchrunfarm-FUMH51YTHBCT57RT.log

David Biancolin

unread,

Dec 2, 2020, 9:34:06 PM12/2/20

to chip...@googlegroups.com

You also need to manually put the contents of that new hwdb entry in config_hwdb.ini. You can simply cat the entries in built-hwdb-entries onto config_hwdb.ini to do this.

- David

--
You received this message because you are subscribed to the Google Groups "Chipyard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chipyard+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/0913686B-A8D6-403E-BBE6-51D919D93471%40g.harvard.edu.

Varun Gandhi

unread,

Dec 2, 2020, 10:04:38 PM12/2/20

to chip...@googlegroups.com

Thanks, David! I’m now stuck at the infraseptup stage (is there a way to better understand these logs?):

2020-12-03 02:48:37,714 [main ] [INFO ] FireSim Manager. Docs: http://docs.fires.im

Running: infrasetup

2020-12-03 02:48:37,714 [__init__ ] [DEBUG] {'hwconf_dict': {'firesim-boom-medium-quadcore-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f2d5f088ef0>,

'firesim-rocket-quadcore-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7f2d5f088ea8>}}

2020-12-03 02:48:37,722 [aws_resource] [DEBUG] i-0c7b16b059cd11e69

2020-12-03 02:48:37,942 [aws_resource] [DEBUG] {'Name': 'uvrs'}

2020-12-03 02:48:37,943 [__init__ ] [DEBUG] {'autocounter_readrate': 0,

2020-12-03 02:48:37,952 [get_deploytr] [DEBUG] Setting deploytriplet by querying the AGFI's description.

2020-12-03 02:48:37,952 [get_afi_for_] [DEBUG] agfi-0079a8e312aeb2de3

2020-12-03 02:48:37,952 [get_afi_for_] [DEBUG] None

2020-12-03 02:48:41,509 [get_afi_for_] [DEBUG] {u'FpgaImages': [{u'UpdateTime': datetime.datetime(2020, 11, 28, 5, 12, 35, tzinfo=tzlocal()), u'Name': 'firesim-boom-medium-quadcore-nic-l2-llc4mb-ddr3', u'Tags': [], u'PciId': {u'SubsystemVendorId': '0xfedd', u'VendorId': '0x1d0f', u'DeviceId': '0xf000', u'SubsystemId': '0x1d51'}, u'FpgaImageGlobalId': 'agfi-0079a8e312aeb2de3', u'Public': False, u'State': {u'Code': 'available'}, u'ShellVersion': '0x04261818', u'OwnerId': '734394535448', u'FpgaImageId': 'afi-0c4bb1acf9858b015', u'CreateTime': datetime.datetime(2020, 11, 28, 4, 1, 16, tzinfo=tzlocal()), u'Description': 'firesim-buildtriplet:FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config,firesim-deploytriplet:FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config,firesim-commit:c2d8e3a46e59222e115a1fdaa7267592e1d3c503-dirty'}], 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '191fa9f8-c3a8-4073-9166-c5eccd606529', 'HTTPHeaders': {'x-amzn-requestid': '191fa9f8-c3a8-4073-9166-c5eccd606529', 'transfer-encoding': 'chunked', 'vary': 'accept-encoding', 'server': 'AmazonEC2', 'date': 'Thu, 03 Dec 2020 02:48:40 GMT', 'content-type': 'text/xml;charset=UTF-8'}}}

2020-12-03 02:48:42,181 [bind_real_in] [DEBUG] Using f1.16xlarge instances with IPs:

[]

2020-12-03 02:48:42,182 [bind_real_in] [DEBUG] Using f1.4xlarge instances with IPs:

['192.168.4.58']

2020-12-03 02:48:42,182 [bind_real_in] [DEBUG] Using f1.2xlarge instances with IPs:

[]

2020-12-03 02:48:42,182 [bind_real_in] [DEBUG] Using m4.16xlarge instances with IPs:

[]

2020-12-03 02:48:42,182 [build_fpga_d] [INFO ] Building FPGA software driver for FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config

2020-12-03 02:48:42,182 [flush ] [DEBUG] [localhost] local: make DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig PLATFORM_CONFIG=F90MHz_BaseF1Config f1

2020-12-03 02:54:59,630 [flush ] [DEBUG] Warning: local() encountered an error (return code 2) while executing 'make DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig PLATFORM_CONFIG=F90MHz_BaseF1Config f1'

2020-12-03 02:54:59,631 [build_fpga_d] [DEBUG] [localhost] success: firesim.pem available in ssh-agent

mkdir -p /home/centos/chipyard/sims/firesim/sim/generated-src/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config

cd /home/centos/chipyard && java -Xmx8G -Xss8M -XX:MaxPermSize=256M -jar /home/centos/chipyard/generators/rocket-chip/sbt-launch.jar "project firechip" "runMain chipyard.Generator \

--target-dir /home/centos/chipyard/sims/firesim/sim/generated-src/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config \

--name firesim.firesim.FireSim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig \

--top-module firesim.firesim.FireSim \

--legacy-configs firesim.firesim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig"

cd /home/centos/chipyard && java -Xmx8G -Xss8M -XX:MaxPermSize=256M -jar /home/centos/chipyard/generators/rocket-chip/sbt-launch.jar "project firechip" "runMain chipyard.Generator \

--target-dir /home/centos/chipyard/sims/firesim/sim/generated-src/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config \

--name firesim.firesim.FireSim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig \

--top-module firesim.firesim.FireSim \

--legacy-configs firesim.firesim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig"

[info] Loading settings for project chipyard-build from plugins.sbt ...

[info] Loading project definition from /home/centos/chipyard/project

[info] Loading settings for project chipyardRoot from build.sbt ...

[info] Loading settings for project barstoolsMacros from build.sbt ...

[info] Loading settings for project mdf from build.sbt ...

[info] Loading settings for project gemmini from build.sbt ...

[info] Loading settings for project ariane from build.sbt ...

[info] Loading settings for project boom from build.sbt ...

[info] Loading settings for project hwacha from build.sbt ...

[info] Loading settings for project icenet from build.sbt ...

[info] Loading settings for project testchipip from build.sbt ...

[info] Loading settings for project rocketConfig from build.sbt ...

[info] Loading settings for project hardfloat from build.sbt ...

[info] Loading settings for project chisel_testers from build.sbt ...

[info] Loading settings for project treadle from build.sbt ...

[info] Loading settings for project firrtl_interpreter from build.sbt ...

[info] Loading settings for project chisel from build.sbt ...

[info] Loading settings for project sim-build from plugins.sbt ...

[info] Loading project definition from /home/centos/chipyard/sims/firesim/sim/project

[info] Loading settings for project firesim from build.sbt ...

[info] Loading settings for project midas from build.sbt ...

[info] Loading settings for project targetutils from build.sbt ...

[info] Resolving key references (32007 settings) ...

[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.

[info] Loading settings for project firesim from build.sbt ...

[info] Loading settings for project midas from build.sbt ...

[info] Loading settings for project targetutils from build.sbt ...

[info] Resolving key references (32007 settings) ...

[info] Set current project to chipyardRoot (in build file:/home/centos/chipyard/)

[info] Set current project to firechip (in build file:/home/centos/chipyard/)

[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list

[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.

[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list

[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.

[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list

[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.

[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list

[error] java.lang.RuntimeException: Could not copy '/home/centos/chipyard/generators/ariane/target/scala-2.12/ariane_2.12-1.0.jar' to '/tmp/sbt_b5ac58b9/target/e1aaf1f7/e2bff954/ariane_2.12-1.0.jar' (12294944/12470751 bytes copied)

[error] at scala.sys.package$.error(package.scala:30)

[error] at sbt.io.IO$.$anonfun$copyFile$4(IO.scala:878)

[error] at sbt.io.IO$.$anonfun$copyFile$4$adapted(IO.scala:866)

[error] at sbt.io.Using.apply(Using.scala:27)

[error] at sbt.io.IO$.$anonfun$copyFile$3(IO.scala:866)

[error] at sbt.io.IO$.$anonfun$copyFile$3$adapted(IO.scala:865)

[error] at sbt.io.Using.apply(Using.scala:27)

[error] at sbt.io.IO$.copyFile(IO.scala:865)

[error] at sbt.io.IO$.copyFile(IO.scala:850)

[error] at sbt.io.IO$.copyFile(IO.scala:842)

[error] at sbt.internal.AbstractBackgroundJobService.syncTo$1(DefaultBackgroundJobService.scala:234)

[error] at sbt.internal.AbstractBackgroundJobService.$anonfun$copyClasspath$4(DefaultBackgroundJobService.scala:239)

[error] at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)

[error] at scala.collection.immutable.List.foreach(List.scala:392)

[error] (Compile / bgRunMain) Could not copy '/home/centos/chipyard/generators/ariane/target/scala-2.12/ariane_2.12-1.0.jar' to '/tmp/sbt_b5ac58b9/target/e1aaf1f7/e2bff954/ariane_2.12-1.0.jar' (12294944/12470751 bytes copied)

[error] Total time: 174 s (02:54), completed Dec 3, 2020 2:52:18 AM

[info] running chipyard.Generator --target-dir /home/centos/chipyard/sims/firesim/sim/generated-src/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config --name firesim.firesim.FireSim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig --top-module firesim.firesim.FireSim --legacy-configs firesim.firesim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig

[ [35minfo [0m] [0.001] Elaborating design...

L2 InclusiveCache Client Map:

0 <= blkdev-tracker0

1 <= serial

2 <= stream-reader

3 <= stream-writer

4 <= Core 0 DCache

5 <= Core 0 ICache

6 <= Core 1 DCache

7 <= Core 1 ICache

8 <= Core 2 DCache

9 <= Core 2 ICache

10 <= Core 3 DCache

11 <= Core 3 ICache

Interrupt map (8 harts 4 interrupts):

[1, 1] => frontend

[2, 2] => uart_0

[3, 4] => control

[C0] ======BOOM Tile 0 Params======

[C0] ====Overall Frontend Params====

[C0] ==L1-ICache==

[C0] Fetch bytes : 8

[C0] Block bytes : 64

[C0] Row bytes : 8

[C0] Word bits : 64

[C0] Sets : 64

[C0] Ways : 4

[C0] Refill cycles : 8

[C0] RAMs : (64 x 512) using 1 banks

[C0] Single-banked

[C0] I-TLB entries : 32

[C0] ==Branch Predictor Memory Sizes==

[C0] bank0 tage_l2: 128 x 44 = 704

[C0] bank0 tage_l4: 128 x 44 = 704

[C0] bank0 tage_l8: 256 x 48 = 1536

[C0] bank0 tage_l16: 256 x 48 = 1536

[C0] bank0 tage_l32: 128 x 52 = 832

[C0] bank0 tage_l64: 128 x 52 = 832

[C0] bank0 btb_meta_way0: 128 x 124 = 1984

[C0] bank0 btb_data_way0: 128 x 56 = 896

[C0] bank0 btb_meta_way1: 128 x 124 = 1984

[C0] bank0 btb_data_way1: 128 x 56 = 896

[C0] bank0 ebtb: 128 x 40 = 640

[C0] bank0 bim: 2048 x 8 = 2048

[C0] Total bpd size: 14 KB

[C0] ====Overall Core Params====

[C0] ===ExecutionUnits===

[C0] ==2-wide Machine==

[C0] ==4 Issue==

[C0] ==ExeUnit==

[C0] - Mem

[C0] ==ExeUnit==

[C0] - ALU

[C0] - Mul

[C0] - IFPU

[C0] ==ExeUnit==

[C0] - ALU

[C0] - Div

[C0] ===FP Pipeline===

[C0] ==Floating Point Regfile==

[C0] Num RF Read Ports : 3

[C0] Num RF Write Ports : 2

[C0] RF Cost (R+W)*(R+2W) : 35

[C0] Bypassable Units : List(false, false)

[C0] Num Wakeup Ports : 2

[C0] Num Bypass Ports : 0

[C0] ==ROB==

[C0] Machine Width : 2

[C0] Rob Entries : 64

[C0] Rob Rows : 32

[C0] Rob Row size : 5

[C0] log2Ceil(coreWidth): 1

[C0] FPU FFlag Ports : 2

[C0] ===Other Core Params===

[C0] Fetch Width : 4

[C0] Decode Width : 2

[C0] Issue Width : 4

[C0] ROB Size : 64

[C0] Issue Window Size : List(12, 20, 16) (Age-based Priority)

[C0] Load/Store Unit Size : 16/16

[C0] Num Int Phys Registers: 80

[C0] Num FP Phys Registers: 64

[C0] Max Branch Count : 12

[C0] ==Integer Regfile==

[C0] Num RF Read Ports : 6

[C0] Num RF Write Ports : 3

[C0] RF Cost (R+W)*(R+2W) : 108

[C0] Bypassable Units : List(true, true, true)

[C0] Num Slow Wakeup Ports : 3

[C0] Num Fast Wakeup Ports : 2

[C0] Num Bypass Ports : 4

[C0] DCache Ways : 4

[C0] DCache Sets : 64

[C0] DCache nMSHRs : 2

[C0] ICache Ways : 4

[C0] ICache Sets : 64

[C0] D-TLB Entries : 8

[C0] I-TLB Entries : 32

[C0] Paddr Bits : 35

[C0] Vaddr Bits : 39

[C0] Using FPU Unit? : true

[C0] Using FDivSqrt? : true

[C0] Using VM? : true

[C1] ======BOOM Tile 1 Params======

[C1] ====Overall Frontend Params====

[C1] ==L1-ICache==

[C1] Fetch bytes : 8

[C1] Block bytes : 64

[C1] Row bytes : 8

[C1] Word bits : 64

[C1] Sets : 64

[C1] Ways : 4

[C1] Refill cycles : 8

[C1] RAMs : (64 x 512) using 1 banks

[C1] Single-banked

[C1] I-TLB entries : 32

[C1] ==Branch Predictor Memory Sizes==

[C1] bank0 tage_l2: 128 x 44 = 704

[C1] bank0 tage_l4: 128 x 44 = 704

[C1] bank0 tage_l8: 256 x 48 = 1536

[C1] bank0 tage_l16: 256 x 48 = 1536

[C1] bank0 tage_l32: 128 x 52 = 832

[C1] bank0 tage_l64: 128 x 52 = 832

[C1] bank0 btb_meta_way0: 128 x 124 = 1984

[C1] bank0 btb_data_way0: 128 x 56 = 896

[C1] bank0 btb_meta_way1: 128 x 124 = 1984

[C1] bank0 btb_data_way1: 128 x 56 = 896

[C1] bank0 ebtb: 128 x 40 = 640

[C1] bank0 bim: 2048 x 8 = 2048

[C1] Total bpd size: 14 KB

[C1] ====Overall Core Params====

[C1] ===ExecutionUnits===

[C1] ==2-wide Machine==

[C1] ==4 Issue==

[C1] ==ExeUnit==

[C1] - Mem

[C1] ==ExeUnit==

[C1] - ALU

[C1] - Mul

[C1] - IFPU

[C1] ==ExeUnit==

[C1] - ALU

[C1] - Div

[C1] ===FP Pipeline===

[C1] ==Floating Point Regfile==

[C1] Num RF Read Ports : 3

[C1] Num RF Write Ports : 2

[C1] RF Cost (R+W)*(R+2W) : 35

[C1] Bypassable Units : List(false, false)

[C1] Num Wakeup Ports : 2

[C1] Num Bypass Ports : 0

[C1] ==ROB==

[C1] Machine Width : 2

[C1] Rob Entries : 64

[C1] Rob Rows : 32

[C1] Rob Row size : 5

[C1] log2Ceil(coreWidth): 1

[C1] FPU FFlag Ports : 2

[C1] ===Other Core Params===

[C1] Fetch Width : 4

[C1] Decode Width : 2

[C1] Issue Width : 4

[C1] ROB Size : 64

[C1] Issue Window Size : List(12, 20, 16) (Age-based Priority)

[C1] Load/Store Unit Size : 16/16

[C1] Num Int Phys Registers: 80

[C1] Num FP Phys Registers: 64

[C1] Max Branch Count : 12

[C1] ==Integer Regfile==

[C1] Num RF Read Ports : 6

[C1] Num RF Write Ports : 3

[C1] RF Cost (R+W)*(R+2W) : 108

[C1] Bypassable Units : List(true, true, true)

[C1] Num Slow Wakeup Ports : 3

[C1] Num Fast Wakeup Ports : 2

[C1] Num Bypass Ports : 4

[C1] DCache Ways : 4

[C1] DCache Sets : 64

[C1] DCache nMSHRs : 2

[C1] ICache Ways : 4

[C1] ICache Sets : 64

[C1] D-TLB Entries : 8

[C1] I-TLB Entries : 32

[C1] Paddr Bits : 35

[C1] Vaddr Bits : 39

[C1] Using FPU Unit? : true

[C1] Using FDivSqrt? : true

[C1] Using VM? : true

[C2] ======BOOM Tile 2 Params======

[C2] ====Overall Frontend Params====

[C2] ==L1-ICache==

[C2] Fetch bytes : 8

[C2] Block bytes : 64

[C2] Row bytes : 8

[C2] Word bits : 64

[C2] Sets : 64

[C2] Ways : 4

[C2] Refill cycles : 8

[C2] RAMs : (64 x 512) using 1 banks

[C2] Single-banked

[C2] I-TLB entries : 32

[C2] ==Branch Predictor Memory Sizes==

[C2] bank0 tage_l2: 128 x 44 = 704

[C2] bank0 tage_l4: 128 x 44 = 704

[C2] bank0 tage_l8: 256 x 48 = 1536

[C2] bank0 tage_l16: 256 x 48 = 1536

[C2] bank0 tage_l32: 128 x 52 = 832

[C2] bank0 tage_l64: 128 x 52 = 832

[C2] bank0 btb_meta_way0: 128 x 124 = 1984

[C2] bank0 btb_data_way0: 128 x 56 = 896

[C2] bank0 btb_meta_way1: 128 x 124 = 1984

[C2] bank0 btb_data_way1: 128 x 56 = 896

[C2] bank0 ebtb: 128 x 40 = 640

[C2] bank0 bim: 2048 x 8 = 2048

[C2] Total bpd size: 14 KB

[C2] ====Overall Core Params====

[C2] ===ExecutionUnits===

[C2] ==2-wide Machine==

[C2] ==4 Issue==

[C2] ==ExeUnit==

[C2] - Mem

[C2] ==ExeUnit==

[C2] - ALU

[C2] - Mul

[C2] - IFPU

[C2] ==ExeUnit==

[C2] - ALU

[C2] - Div

[C2] ===FP Pipeline===

[C2] ==Floating Point Regfile==

[C2] Num RF Read Ports : 3

[C2] Num RF Write Ports : 2

[C2] RF Cost (R+W)*(R+2W) : 35

[C2] Bypassable Units : List(false, false)

[C2] Num Wakeup Ports : 2

[C2] Num Bypass Ports : 0

[C2] ==ROB==

[C2] Machine Width : 2

[C2] Rob Entries : 64

[C2] Rob Rows : 32

[C2] Rob Row size : 5

[C2] log2Ceil(coreWidth): 1

[C2] FPU FFlag Ports : 2

[C2] ===Other Core Params===

[C2] Fetch Width : 4

[C2] Decode Width : 2

[C2] Issue Width : 4

[C2] ROB Size : 64

[C2] Issue Window Size : List(12, 20, 16) (Age-based Priority)

[C2] Load/Store Unit Size : 16/16

[C2] Num Int Phys Registers: 80

[C2] Num FP Phys Registers: 64

[C2] Max Branch Count : 12

[C2] ==Integer Regfile==

[C2] Num RF Read Ports : 6

[C2] Num RF Write Ports : 3

[C2] RF Cost (R+W)*(R+2W) : 108

[C2] Bypassable Units : List(true, true, true)

[C2] Num Slow Wakeup Ports : 3

[C2] Num Fast Wakeup Ports : 2

[C2] Num Bypass Ports : 4

[C2] DCache Ways : 4

[C2] DCache Sets : 64

[C2] DCache nMSHRs : 2

[C2] ICache Ways : 4

[C2] ICache Sets : 64

[C2] D-TLB Entries : 8

[C2] I-TLB Entries : 32

[C2] Paddr Bits : 35

[C2] Vaddr Bits : 39

[C2] Using FPU Unit? : true

[C2] Using FDivSqrt? : true

[C2] Using VM? : true

[C3] ======BOOM Tile 3 Params======

[C3] ====Overall Frontend Params====

[C3] ==L1-ICache==

[C3] Fetch bytes : 8

[C3] Block bytes : 64

[C3] Row bytes : 8

[C3] Word bits : 64

[C3] Sets : 64

[C3] Ways : 4

[C3] Refill cycles : 8

[C3] RAMs : (64 x 512) using 1 banks

[C3] Single-banked

[C3] I-TLB entries : 32

[C3] ==Branch Predictor Memory Sizes==

[C3] bank0 tage_l2: 128 x 44 = 704

[C3] bank0 tage_l4: 128 x 44 = 704

[C3] bank0 tage_l8: 256 x 48 = 1536

[C3] bank0 tage_l16: 256 x 48 = 1536

[C3] bank0 tage_l32: 128 x 52 = 832

[C3] bank0 tage_l64: 128 x 52 = 832

[C3] bank0 btb_meta_way0: 128 x 124 = 1984

[C3] bank0 btb_data_way0: 128 x 56 = 896

[C3] bank0 btb_meta_way1: 128 x 124 = 1984

[C3] bank0 btb_data_way1: 128 x 56 = 896

[C3] bank0 ebtb: 128 x 40 = 640

[C3] bank0 bim: 2048 x 8 = 2048

[C3] Total bpd size: 14 KB

[C3] ====Overall Core Params====

[C3] ===ExecutionUnits===

[C3] ==2-wide Machine==

[C3] ==4 Issue==

[C3] ==ExeUnit==

[C3] - Mem

[C3] ==ExeUnit==

[C3] - ALU

[C3] - Mul

[C3] - IFPU

[C3] ==ExeUnit==

[C3] - ALU

[C3] - Div

[C3] ===FP Pipeline===

[C3] ==Floating Point Regfile==

[C3] Num RF Read Ports : 3

[C3] Num RF Write Ports : 2

[C3] RF Cost (R+W)*(R+2W) : 35

[C3] Bypassable Units : List(false, false)

[C3] Num Wakeup Ports : 2

[C3] Num Bypass Ports : 0

[C3] ==ROB==

[C3] Machine Width : 2

[C3] Rob Entries : 64

[C3] Rob Rows : 32

[C3] Rob Row size : 5

[C3] log2Ceil(coreWidth): 1

[C3] FPU FFlag Ports : 2

[C3] ===Other Core Params===

[C3] Fetch Width : 4

[C3] Decode Width : 2

[C3] Issue Width : 4

[C3] ROB Size : 64

[C3] Issue Window Size : List(12, 20, 16) (Age-based Priority)

[C3] Load/Store Unit Size : 16/16

[C3] Num Int Phys Registers: 80

[C3] Num FP Phys Registers: 64

[C3] Max Branch Count : 12

[C3] ==Integer Regfile==

[C3] Num RF Read Ports : 6

[C3] Num RF Write Ports : 3

[C3] RF Cost (R+W)*(R+2W) : 108

[C3] Bypassable Units : List(true, true, true)

[C3] Num Slow Wakeup Ports : 3

[C3] Num Fast Wakeup Ports : 2

[C3] Num Bypass Ports : 4

[C3] DCache Ways : 4

[C3] DCache Sets : 64

[C3] DCache nMSHRs : 2

[C3] ICache Ways : 4

[C3] ICache Sets : 64

[C3] D-TLB Entries : 8

[C3] I-TLB Entries : 32

[C3] Paddr Bits : 35

[C3] Vaddr Bits : 39

[C3] Using FPU Unit? : true

[C3] Using FDivSqrt? : true

[C3] Using VM? : true

/dts-v1/;

/ {

#address-cells = <2>;

#size-cells = <2>;

compatible = "freechips,rocketchip-unknown-dev";

model = "freechips,rocketchip-unknown";

L25: aliases {

serial0 = &L20;

};

L24: cpus {

#address-cells = <1>;

#size-cells = <0>;

L8: cpu@0 {

clock-frequency = <0>;

compatible = "ucb-bar,boom0", "riscv";

d-cache-block-size = <64>;

d-cache-sets = <64>;

d-cache-size = <16384>;

d-tlb-sets = <1>;

d-tlb-size = <8>;

device_type = "cpu";

hardware-exec-breakpoint-count = <0>;

i-cache-block-size = <64>;

i-cache-sets = <64>;

i-cache-size = <16384>;

i-tlb-sets = <1>;

i-tlb-size = <32>;

mmu-type = "riscv,sv39";

next-level-cache = <&L2>;

reg = <0x0>;

riscv,isa = "rv64imafdc";

riscv,pmpregions = <8>;

status = "okay";

timebase-frequency = <1000000>;

tlb-split;

L7: interrupt-controller {

#interrupt-cells = <1>;

compatible = "riscv,cpu-intc";

interrupt-controller;

};

L10: cpu@1 {

clock-frequency = <0>;

compatible = "ucb-bar,boom0", "riscv";

d-cache-block-size = <64>;

d-cache-sets = <64>;

d-cache-size = <16384>;

d-tlb-sets = <1>;

d-tlb-size = <8>;

device_type = "cpu";

hardware-exec-breakpoint-count = <0>;

i-cache-block-size = <64>;

i-cache-sets = <64>;

i-cache-size = <16384>;

i-tlb-sets = <1>;

i-tlb-size = <32>;

mmu-type = "riscv,sv39";

next-level-cache = <&L2>;

reg = <0x1>;

riscv,isa = "rv64imafdc";

riscv,pmpregions = <8>;

status = "okay";

timebase-frequency = <1000000>;

tlb-split;

L9: interrupt-controller {

#interrupt-cells = <1>;

compatible = "riscv,cpu-intc";

interrupt-controller;

};

L12: cpu@2 {

clock-frequency = <0>;

compatible = "ucb-bar,boom0", "riscv";

d-cache-block-size = <64>;

d-cache-sets = <64>;

d-cache-size = <16384>;

d-tlb-sets = <1>;

d-tlb-size = <8>;

device_type = "cpu";

hardware-exec-breakpoint-count = <0>;

i-cache-block-size = <64>;

i-cache-sets = <64>;

i-cache-size = <16384>;

i-tlb-sets = <1>;

i-tlb-size = <32>;

mmu-type = "riscv,sv39";

next-level-cache = <&L2>;

reg = <0x2>;

riscv,isa = "rv64imafdc";

riscv,pmpregions = <8>;

status = "okay";

timebase-frequency = <1000000>;

tlb-split;

L11: interrupt-controller {

#interrupt-cells = <1>;

compatible = "riscv,cpu-intc";

interrupt-controller;

};

L14: cpu@3 {

clock-frequency = <0>;

compatible = "ucb-bar,boom0", "riscv";

d-cache-block-size = <64>;

d-cache-sets = <64>;

d-cache-size = <16384>;

d-tlb-sets = <1>;

d-tlb-size = <8>;

device_type = "cpu";

hardware-exec-breakpoint-count = <0>;

i-cache-block-size = <64>;

i-cache-sets = <64>;

i-cache-size = <16384>;

i-tlb-sets = <1>;

i-tlb-size = <32>;

mmu-type = "riscv,sv39";

next-level-cache = <&L2>;

reg = <0x3>;

riscv,isa = "rv64imafdc";

riscv,pmpregions = <8>;

status = "okay";

timebase-frequency = <1000000>;

tlb-split;

L13: interrupt-controller {

#interrupt-cells = <1>;

compatible = "riscv,cpu-intc";

interrupt-controller;

};

L16: memory@80000000 {

device_type = "memory";

reg = <0x0 0x80000000 0x4 0x0>;

};

L23: soc {

#address-cells = <2>;

#size-cells = <2>;

compatible = "freechips,rocketchip-unknown-soc", "simple-bus";

ranges;

L19: blkdev-controller@10015000 {

compatible = "ucbbar,blkdev";

interrupt-parent = <&L3>;

interrupts = <1>;

reg = <0x0 0x10015000 0x0 0x1000>;

reg-names = "control";

};

L2: cache-controller@2010000 {

cache-block-size = <64>;

cache-level = <2>;

cache-sets = <1024>;

cache-size = <524288>;

cache-unified;

compatible = "sifive,inclusivecache0", "cache";

next-level-cache = <&L16>;

reg = <0x0 0x2010000 0x0 0x1000>;

reg-names = "control";

sifive,mshr-count = <7>;

};

L4: clint@2000000 {

compatible = "riscv,clint0";

interrupts-extended = <&L7 3 &L7 7 &L9 3 &L9 7 &L11 3 &L11 7 &L13 3 &L13 7>;

reg = <0x0 0x2000000 0x0 0x10000>;

reg-names = "control";

};

L5: debug-controller@0 {

compatible = "sifive,debug-013", "riscv,debug-013";

debug-attach = "dmi";

interrupts-extended = <&L7 65535 &L9 65535 &L11 65535 &L13 65535>;

reg = <0x0 0x0 0x0 0x1000>;

reg-names = "control";

};

L1: error-device@3000 {

compatible = "sifive,error0";

reg = <0x0 0x3000 0x0 0x1000>;

};

L21: ice-nic@10016000 {

compatible = "ucbbar,ice-nic";

interrupt-parent = <&L3>;

interrupts = <3 4>;

reg = <0x0 0x10016000 0x0 0x1000>;

reg-names = "control";

};

L3: interrupt-controller@c000000 {

#interrupt-cells = <1>;

compatible = "riscv,plic0";

interrupt-controller;

interrupts-extended = <&L7 11 &L7 9 &L9 11 &L9 9 &L11 11 &L11 9 &L13 11 &L13 9>;

reg = <0x0 0xc000000 0x0 0x4000000>;

reg-names = "control";

riscv,max-priority = <7>;

riscv,ndev = <4>;

};

L18: rom@10000 {

compatible = "sifive,rom0";

reg = <0x0 0x10000 0x0 0x10000>;

reg-names = "mem";

};

L20: serial@54000000 {

clocks = <&L0>;

compatible = "sifive,uart0";

interrupt-parent = <&L3>;

interrupts = <2>;

reg = <0x0 0x54000000 0x0 0x1000>;

reg-names = "control";

};

L0: subsystem_pbus_clock {

#clock-cells = <0>;

clock-frequency = <3200000000>;

clock-output-names = "subsystem_pbus_clock";

compatible = "fixed-clock";

};

Generated Address Map

0 - 1000 ARWX debug-controller@0

3000 - 4000 ARWX error-device@3000

10000 - 20000 R X rom@10000

2000000 - 2010000 ARW clint@2000000

2010000 - 2011000 ARW cache-controller@2010000

c000000 - 10000000 ARW interrupt-controller@c000000

10015000 - 10016000 ARW blkdev-controller@10015000

10016000 - 10017000 ARW ice-nic@10016000

54000000 - 54001000 ARW serial@54000000

80000000 - 480000000 ARWXC memory@80000000

[ [34mdeprecated [0m] class boom.lsu.BoomDCacheBundle (4 calls): Unable to automatically infer cloneType on class boom.lsu.BoomDCacheBundle: constructor has parameters (edge) that are not both immutable and accessible. Either make all parameters immutable and accessible (vals) so cloneType can be inferred, or define a custom cloneType method.

[ [34mdeprecated [0m] Bits.scala:373 (4 calls): do_toBool is deprecated: "Use asBool instead"

[ [34mdeprecated [0m] class boom.lsu.LSUIO (4 calls): Unable to automatically infer cloneType on class boom.lsu.LSUIO: constructor has parameters (edge) that are not both immutable and accessible. Either make all parameters immutable and accessible (vals) so cloneType can be inferred, or define a custom cloneType method.

[ [34mdeprecated [0m] class testchipip.BlockDeviceFrontend$$anonfun$1$$anon$1 (1 calls): Unable to automatically infer cloneType on class testchipip.BlockDeviceFrontend$$anonfun$1$$anon$1: constructor has parameters ($outer, x$13$1) that are not both immutable and accessible. Either make all parameters immutable and accessible (vals) so cloneType can be inferred, or define a custom cloneType method.

[ [34mdeprecated [0m] class icenet.IceNicController$$anonfun$2$$anon$1 (1 calls): Unable to automatically infer cloneType on class icenet.IceNicController$$anonfun$2$$anon$1: constructor has parameters ($outer, x$1$1) that are not both immutable and accessible. Either make all parameters immutable and accessible (vals) so cloneType can be inferred, or define a custom cloneType method.

[ [34mdeprecated [0m] class firesim.bridges.TracerVTargetIO (4 calls): Unable to automatically infer cloneType on class firesim.bridges.TracerVTargetIO: constructor has parameters (insnWidths, numInsns) that are not both immutable and accessible. Either make all parameters immutable and accessible (vals) so cloneType can be inferred, or define a custom cloneType method.

[ [33mwarn [0m] [33mThere were 6 deprecated function(s) used. These may stop compiling in a future release - you are encouraged to fix these issues. [0m

[ [33mwarn [0m] Line numbers for deprecations reported by Chisel may be inaccurate; enable scalac compiler deprecation warnings via either of the following methods:

[ [33mwarn [0m] In the sbt interactive console, enter:

[ [33mwarn [0m] set scalacOptions in ThisBuild ++= Seq("-unchecked", "-deprecation")

[ [33mwarn [0m] or, in your build.sbt, add the line:

[ [33mwarn [0m] scalacOptions := Seq("-unchecked", "-deprecation")

[ [35minfo [0m] [138.343] Done elaborating.

2020-12-03 02:54:59,631 [build_fpga_d] [DEBUG] [localhost] /home/centos/chipyard/sims/firesim/sim/src/main/makefrag/firesim/Makefrag:71: warning: overriding recipe for target `/home/centos/chipyard/tools/dromajo/dromajo-src/src/libdromajo_cosim.a'

/home/centos/chipyard/tools/dromajo/dromajo.mk:23: warning: ignoring old recipe for target `/home/centos/chipyard/tools/dromajo/dromajo-src/src/libdromajo_cosim.a'

OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0

Waiting for lock on /home/centos/.sbt/boot/sbt.boot.lock to be available...

Waiting for lock on /home/centos/.ivy2/.sbt.ivy.lock to be available...

make: *** [/home/centos/chipyard/sims/firesim/sim/generated-src/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config/firesim.firesim.FireSim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig.fir] Error 1

make: *** Waiting for unfinished jobs....

/bin/bash: line 4: 4657 Killed java -Xmx8G -Xss8M -XX:MaxPermSize=256M -jar /home/centos/chipyard/generators/rocket-chip/sbt-launch.jar "project firechip" "runMain chipyard.Generator --target-dir /home/centos/chipyard/sims/firesim/sim/generated-src/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig-F90MHz_BaseF1Config --name firesim.firesim.FireSim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig --top-module firesim.firesim.FireSim --legacy-configs firesim.firesim.WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig"

2020-12-03 02:54:59,631 [build_fpga_d] [INFO ] FPGA software driver build failed. Exiting. See log for details.

2020-12-03 02:54:59,631 [build_fpga_d] [INFO ] You can also re-run 'make DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadMediumBoomNicConfig PLATFORM_CONFIG=F90MHz_BaseF1Config f1' in the 'firesim/sim' directory to debug this error.

2020-12-03 02:54:59,631 [<module> ] [ERROR] Fatal error.

Traceback (most recent call last):

File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 334, in <module>

main(args)

File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 282, in main

globals()[args.task](simconf)

File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 95, in infrasetup

runtime_conf.infrasetup()

File "/home/centos/chipyard/sims/firesim/deploy/runtools/runtime_config.py", line 376, in infrasetup

self.firesim_topology_with_passes.infrasetup_passes(use_mock_instances_for_testing)

File "/home/centos/chipyard/sims/firesim/deploy/runtools/firesim_topology_with_passes.py", line 387, in infrasetup_passes

self.pass_build_required_drivers()

File "/home/centos/chipyard/sims/firesim/deploy/runtools/firesim_topology_with_passes.py", line 370, in pass_build_required_drivers

server.get_server_hardware_config().build_fpga_driver()

File "/home/centos/chipyard/sims/firesim/deploy/runtools/runtime_config.py", line 201, in build_fpga_driver

exit(1)

File "/usr/lib64/python2.7/site.py", line 364, in __call__

raise SystemExit(code)

SystemExit: 1

2020-12-03 02:54:59,640 [<module> ] [INFO ] The full log of this run is:

/home/centos/chipyard/sims/firesim/deploy/logs/2020-12-03--02-48-37-infrasetup-OLN9EO3A7CQ0ZNV1.log

Varun Gandhi

unread,

Dec 3, 2020, 12:32:17 AM12/3/20

to chip...@googlegroups.com

Interestingly, this error is consistent with any kind of boom config, even the default ones.

Jerry Zhao

unread,

Dec 3, 2020, 12:39:07 AM12/3/20

to chip...@googlegroups.com

This error is due to OOM when building the BOOM RTL on the firesim manager instance.
In chipyard/variables.mk, change JAVA_HEAP_SIZE ?= 8G to JAVA_HEAP_SIZE ?= 32G

--
You received this message because you are subscribed to the Google Groups "Chipyard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chipyard+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/E985E0D9-49A8-4BEE-990D-7BE0AE172655%40g.harvard.edu.

Varun Gandhi

unread,

Dec 3, 2020, 12:59:42 AM12/3/20

to chip...@googlegroups.com

Hi Jerry,

Thanks! What’s the upper limit for the java_heap_size param? I’m trying to build a quad-core-medium-boom config and still getting the OOM error at 32G. Based on the firesim docs, each FPGA board has 16GB of RAM. Do I need to upgrade from a 4x large to a 16x large instance?

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/CAC%2BpDSGdJMJcbnscqt%2BdPGifNyrFQ7p%2BV3U5xNKDV%2BQpwQ5j%3Dg%40mail.gmail.com.

Jerry Zhao

unread,

Dec 3, 2020, 1:12:05 AM12/3/20

to chip...@googlegroups.com

The JAVA_HEAP_SIZE param controls the heap space allocated to the Java VM when running our Chisel/Firrtl compile steps. It has no relationship with the memory on the FPGA boards, or with the F1 instances.
I'm a little surprised that you are still hitting OOM at 32G. I'll try building a quad-medium boom config myself.

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/2A110139-AA98-4F5C-B12D-80C5C4E1807A%40g.harvard.edu.

Varun Gandhi

unread,

Dec 3, 2020, 1:25:36 AM12/3/20

to chip...@googlegroups.com

I'll try building a quad-medium boom config myself.

Thanks, Jerry! Because the 32G heap size works fine for the single core large boom config.

-Varun

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/CAC%2BpDSE5KRS6DKNz6VE3FVcEb41gQ9%3DRUpTY8Uk63F8N-U9i2Q%40mail.gmail.com.

Varun Gandhi

unread,

Dec 3, 2020, 1:53:36 AM12/3/20

to chip...@googlegroups.com

When I try to run the fedora-uniform workload, the simulated nodes fail to boot fedora successfully. And, when I ssh into either of the two simulated nodes, all I see is the message “commencing simulation”

Here’s a screenshot

Jerry Zhao

unread,

Dec 3, 2020, 1:43:23 PM12/3/20

to chip...@googlegroups.com

Hmmm. The fact that the simulation hangs before any printouts is unusual.
Are you able to run linux-uniform on the same setup?

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/7A55C5FE-F065-45E2-B31C-EC0CFEEAF90E%40g.harvard.edu.

Varun Gandhi

unread,

Dec 3, 2020, 1:44:34 PM12/3/20

to chip...@googlegroups.com

Yes, linux-uniform runs without any issues.

On Dec 3, 2020, at 13:42, Jerry Zhao <J...@berkeley.edu> wrote:

Hmmm. The fact that the simulation hangs before any printouts is unusual.
Are you able to run linux-uniform on the same setup?

On Wed, Dec 2, 2020 at 10:53 PM Varun Gandhi <vga...@g.harvard.edu> wrote:

When I try to run the fedora-uniform workload, the simulated nodes fail to boot fedora successfully. And, when I ssh into either of the two simulated nodes, all I see is the message “commencing simulation”

Here’s a screenshot

<fedora-workload-2Node-config.png>

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/CAC%2BpDSEhgUQCXFCjt-ZH-DENivExmRbxaE4JZKJ7trP5J10ynQ%40mail.gmail.com.

Varun Gandhi

unread,

Dec 3, 2020, 5:42:54 PM12/3/20

to chip...@googlegroups.com

Hi Jerry,

I tried running strace on the manager instance while it ran infrasetup and all that showed up was futex. Also, the 32G heap size works fine for a single-core large boom config. It seems like some other memory constraint/condition is not being met when trying to build a multi-core config.

I assume that the build process attempts to parallelize as much as possible — could it be possible that a memory constraint, other than the VM heap size (and presumably synchronization related), is not being met? The reason for this intuition is that I tried running infrasetup with heap sizes 64G & 128G on 4xlarge and 256G & 512G on 16xlarge and still got the same error.

Best,

Varun

On Dec 3, 2020, at 13:42, Jerry Zhao <J...@berkeley.edu> wrote:

Hmmm. The fact that the simulation hangs before any printouts is unusual.
Are you able to run linux-uniform on the same setup?

On Wed, Dec 2, 2020 at 10:53 PM Varun Gandhi <vga...@g.harvard.edu> wrote:

When I try to run the fedora-uniform workload, the simulated nodes fail to boot fedora successfully. And, when I ssh into either of the two simulated nodes, all I see is the message “commencing simulation”

Here’s a screenshot

<fedora-workload-2Node-config.png>

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/CAC%2BpDSEhgUQCXFCjt-ZH-DENivExmRbxaE4JZKJ7trP5J10ynQ%40mail.gmail.com.

Varun Gandhi

unread,

Dec 11, 2020, 8:12:03 PM12/11/20

to chip...@googlegroups.com

Hi Jerry,

I was wondering if you managed to deploy a quad-core medium boom config? Also, here’s an strace log of infrasetup on the quad-core config. All of the mmap and munmap syscalls were successful. I’m curious -- what makes you think this is an OOM error?

Best,

Varun

strace_infrasetup_output_7.log

Jerry Zhao

unread,

Dec 11, 2020, 8:43:16 PM12/11/20

to chip...@googlegroups.com

Yes, I have managed to build a QuadMediumBoomConfig.
Looking back at your messages, I noticed that you are seeing the issue only in infrasetup, and that your infrasetup step seems to be rebuilding all the RTL. This seems strange to me, since the buildafi step should build the RTL, and the infrasetup step should use cached results from the buildafi step. In my workflow infrasetup just compiles the driver copies the disk images to the FPGA instance. Are you modifying source RTL in between buildafi and infrasetup?

I initially suspected OOM because I have hit OOM issues in the past with large Boom designs, resulting in a `Killed` error message. However, looking back at your log, it seems suspicious that there is a long sequence of "Waiting for lock on /home/centos/.ivy2/.sbt.ivy.lock to be available..." .

I am not intimately familiar with sbt, but this issue seems to appear when there are multiple sbt instances running simultaneously. https://stackoverflow.com/questions/39903128/waiting-for-lock-on-ivy2/42778594

--
You received this message because you are subscribed to the Google Groups "Chipyard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chipyard+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/F47F8AA2-DDC3-427D-B0A8-E9E96B0D322D%40g.harvard.edu.

Varun Gandhi

unread,

Jan 4, 2021, 1:39:34 PM1/4/21

to chip...@googlegroups.com

Hi Jerry,

I created a new manager instance to try and resolve this issue, but now I’m getting a different error on running infrasetup [1]:

I also tried running "make DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig PLATFORM_CONFIG=F90MHz_BaseF1Config f1” in firesim/sim

But got the following error: make: *** No rule to make target `/lib/libfesvr.a', needed by `/home/centos/chipyard/sims/firesim/sim/output/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig-F90MHz_BaseF1Config/FireSim-f1'. Stop

Are you modifying source RTL in between buildafi and infrasetup?

I was doing that earlier because I am modifying some of the features of the NIC, and need to test the variants. I was under the impression that I can incrementally change the NIC implementation and build corresponding AFIs for every change for testing purposes. Is this not possible?

———————————————————————————————

[1] 2021-01-04--18-04-32-infrasetup-14VHN0PTR4KSRAM2.log

Running: infrasetup

2021-01-04 18:04:32,100 [__init__    ] [DEBUG]  {'hwconf_dict': {'firesim-boom-singlecore-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7feeddb56ea8>,
                 'firesim-boom-singlecore-no-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7feeddb56e60>,
                 'firesim-rocket-quadcore-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7feeddb56f38>,
                 'firesim-rocket-quadcore-no-nic-l2-llc4mb-ddr3': <runtools.runtime_config.RuntimeHWConfig instance at 0x7feeddb56fc8>,
                 'firesim-rocket-quadcore-no-nic-l2-llc4mb-ddr3-half-freq-uncore': <runtools.runtime_config.RuntimeHWConfig instance at 0x7feeddb56f80>,
                 'firesim-supernode-rocket-singlecore-nic-l2-lbp': <runtools.runtime_config.RuntimeHWConfig instance at 0x7feeddb56ef0>}}
2021-01-04 18:04:32,107 [aws_resource] [DEBUG]  i-060579c6bb49c8f70
2021-01-04 18:04:32,318 [aws_resource] [DEBUG]  {'Name': 'nfshield'}
2021-01-04 18:04:32,319 [__init__    ] [DEBUG]  {'autocounter_readrate': 0,

'defaulthwconfig': 'firesim-rocket-quadcore-nic-l2-llc4mb-ddr3',

 'f1_16xlarges_requested': 0,

 'f1_2xlarges_requested': 0,

 'f1_4xlarges_requested': 1,

 'linklatency': 6405,

 'm4_16xlarges_requested': 0,

 'netbandwidth': 200,

 'no_net_num_nodes': 2,

 'print_cycle_prefix': True,

 'print_end': '-1',

 'print_start': '0',

 'profileinterval': -1,

 'run_instance_market': 'ondemand',

 'runfarmtag': 'mainrunfarm',

 'spot_interruption_behavior': 'terminate',

 'spot_max_price': 'ondemand',

 'suffixtag': '',

 'switchinglatency': 10,

 'terminateoncompletion': False,

 'topology': 'example_2config',

 'trace_enable': False,

 'trace_end': '-1',

 'trace_output_format': '0',

 'trace_select': '1',

 'trace_start': '0',

 'workload_name': 'linux-uniform.json',

 'zerooutdram': False}

2021-01-04 18:04:32,323 [get_deploytr] [DEBUG]  Setting deploytriplet by querying the AGFI's description.

2021-01-04 18:04:32,324 [get_afi_for_] [DEBUG]  agfi-09dbf5cafb4ff4649

2021-01-04 18:04:32,324 [get_afi_for_] [DEBUG]  None

2021-01-04 18:04:35,514 [get_afi_for_] [DEBUG]  {u'FpgaImages': [{u'UpdateTime': datetime.datetime(2020, 5, 29, 13, 15, 44, tzinfo=tzlocal()), u'Name': 'firesim-rocket-quadcore-nic-l2-llc4mb-ddr3', u'Tags': [], u'PciId': {u'SubsystemVendorId': '0xfedd', u'VendorId': '0x1d0f', u'DeviceId': '0xf000', u'SubsystemId': '0x1d51'}, u'FpgaImageGlobalId': 'agfi-09dbf5cafb4ff4649', u'Public': True, u'State': {u'Code': 'available'}, u'ShellVersion': '0x04261818', u'OwnerId': '552479941143', u'FpgaImageId': 'afi-06de1054463e77c6d', u'CreateTime': datetime.datetime(2020, 5, 29, 12, 35, 43, tzinfo=tzlocal()), u'Description': 'firesim-buildtriplet:FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig-F90MHz_BaseF1Config,firesim-deploytriplet:FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig-F90MHz_BaseF1Config,firesim-commit:efe623bbdb17b84fdcfa527502a742219658c939-dirty'}], 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'a185e23e-3030-4465-9ff4-d48b23e3e0eb', 'HTTPHeaders': {'x-amzn-requestid': 'a185e23e-3030-4465-9ff4-d48b23e3e0eb', 'transfer-encoding': 'chunked', 'strict-transport-security': 'max-age=31536000; includeSubDomains', 'vary': 'accept-encoding', 'server': 'AmazonEC2', 'cache-control': 'no-cache, no-store', 'date': 'Mon, 04 Jan 2021 18:04:34 GMT', 'content-type': 'text/xml;charset=UTF-8'}}}

2021-01-04 18:04:36,151 [bind_real_in] [DEBUG]  Using f1.16xlarge instances with IPs:

[]

2021-01-04 18:04:36,152 [bind_real_in] [DEBUG]  Using f1.4xlarge instances with IPs:

['192.168.4.90']

2021-01-04 18:04:36,152 [bind_real_in] [DEBUG]  Using f1.2xlarge instances with IPs:

[]

2021-01-04 18:04:36,152 [bind_real_in] [DEBUG]  Using m4.16xlarge instances with IPs:

[]

2021-01-04 18:04:36,152 [build_fpga_d] [INFO ]  Building FPGA software driver for FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig-F90MHz_BaseF1Config

2021-01-04 18:04:36,152 [flush       ] [DEBUG]  [localhost] local: make DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig PLATFORM_CONFIG=F90MHz_BaseF1Config f1

2021-01-04 18:04:40,506 [flush       ] [DEBUG]  Warning: local() encountered an error (return code 2) while executing 'make DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig PLATFORM_CONFIG=F90MHz_BaseF1Config f1'

2021-01-04 18:04:40,506 [build_fpga_d] [DEBUG]  [localhost] success: firesim.pem available in ssh-agent

2021-01-04 18:04:40,506 [build_fpga_d] [DEBUG]  [localhost] /home/centos/chipyard/sims/firesim/sim/src/main/makefrag/firesim/Makefrag:71: warning: overriding recipe for target `/home/centos/chipyard/tools/dromajo/dromajo-src/src/libdromajo_cosim.a'

/home/centos/chipyard/tools/dromajo/dromajo.mk:23: warning: ignoring old recipe for target `/home/centos/chipyard/tools/dromajo/dromajo-src/src/libdromajo_cosim.a'

make: *** No rule to make target `/lib/libfesvr.a', needed by `/home/centos/chipyard/sims/firesim/sim/output/f1/FireSim-WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig-F90MHz_BaseF1Config/FireSim-f1'.  Stop.

2021-01-04 18:04:40,507 [build_fpga_d] [INFO ]  FPGA software driver build failed. Exiting. See log for details.

2021-01-04 18:04:40,507 [build_fpga_d] [INFO ]  You can also re-run 'make DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimQuadRocketConfig PLATFORM_CONFIG=F90MHz_BaseF1Config f1' in the 'firesim/sim' directory to debug this error.

2021-01-04 18:04:40,507 [<module>    ] [ERROR]  Fatal error.

Traceback (most recent call last):

  File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 334, in <module>

    main(args)

  File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 282, in main

    globals()[args.task](simconf)

  File "/home/centos/chipyard/sims/firesim/deploy/firesim", line 95, in infrasetup

    runtime_conf.infrasetup()

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/runtime_config.py", line 376, in infrasetup

    self.firesim_topology_with_passes.infrasetup_passes(use_mock_instances_for_testing)

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/firesim_topology_with_passes.py", line 387, in infrasetup_passes

    self.pass_build_required_drivers()

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/firesim_topology_with_passes.py", line 370, in pass_build_required_drivers

    server.get_server_hardware_config().build_fpga_driver()

  File "/home/centos/chipyard/sims/firesim/deploy/runtools/runtime_config.py", line 201, in build_fpga_driver

    exit(1)

  File "/usr/lib64/python2.7/site.py", line 364, in __call__

    raise SystemExit(code)

SystemExit: 1

2021-01-04 18:04:40,512 [<module>    ] [INFO ]  The full log of this run is:

/home/centos/chipyard/sims/firesim/deploy/logs/2021-01-04--18-04-32-infrasetup-14VHN0PTR4KSRAM2.log

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/CAC%2BpDSHwD0mJfRUqaDNvbJC0GRrUEjYhhX-y%2Biw9Qn5qxQE4og%40mail.gmail.com.

Reply all

Reply to author

Forward

0 new messages