Invalid AFI ID
245 views
Skip to first unread message
Farzad Farshchi
Jul 26, 2019, 1:23:26 PM7/26/19
to FireSim
Hi,
I have created a new AFI and when I run `infrasetup`, it fails at flashing the FPGA with the following error printed in the log:
Flashing FPGA Slot: 0 with agfi: agfi-0f6deb3593bf57481.
run: sudo fpga-load-local-image -S 0 -I agfi-0f6deb3593bf57481 -A
out: Error: (5) invalid-afi-id
out: The agfi id passed is invalid or you do not have permission to load
out: the AFI.
run: sudo fpga-load-local-image -S 0 -I agfi-0f6deb3593bf57481 -A
out: Error: (5) invalid-afi-id
out: The agfi id passed is invalid or you do not have permission to load
out: the AFI.
I compared the AFI ID with the build log and they match. I also tried running `fpga-load-local-image` in the F1 instance and it gave the same error.
Any thoughts on what is happing here? Is it because the build process accidentally didn't set the right permissions for the AFI?
I think this has happened for me once before for another build.
Thanks,
Farzad
David Biancolin
Jul 26, 2019, 2:33:18 PM7/26/19
to FireSim
Hi Farzad,
You should be able to query the AGFI's state with `describe-fpga-images`
You should be able to query the AGFI's state with `describe-fpga-images`
David Biancolin
Jul 26, 2019, 2:34:50 PM7/26/19
to FireSim
[sent prematurely]
and describe-fpga-image-attribute. Those might give you a better sense of what's going on. Are you building and running agfis in different regions?
Farzad Farshchi
Jul 26, 2019, 5:17:20 PM7/26/19
to FireSim
Hi David,
No, I am building and running in the same region. I copied the output of describe-fpga-images below. The "State" says it is failed.
So I looked back into the build log and saw that at the final step when firesim is polling the state of AGFI creation it eventually fails, but the firesim just ignores that and assumes the AGFI is created successfully. Please find the build log at the bottom.
$ aws ec2 describe-fpga-images --filter Name=fpga-image-global-id,Values=agfi-0f6deb3593bf57481
{
"FpgaImages": [
{
"UpdateTime": "2019-07-26T08:49:23.000Z",
"Name": "fireboom-quadcore-ddr3-1rank-llc4mb-16kl1",
"Tags": [],
"PciId": {
"SubsystemVendorId": "0xfedd",
"VendorId": "0x1d0f",
"DeviceId": "0xf000",
"SubsystemId": "0x1d51"
},
"FpgaImageGlobalId": "agfi-0f6deb3593bf57481",
"Public": false,
"State": {
"Message": "UNKNOWN_BITSTREAM_GENERATE_ERROR: An error occurred generating the FPGA image bitstream. If an S3 LogsStorageLocation was provided in the CreateFpgaImage request, review the captured bitstream generation logs saved to S3 under the FpgaImageId for this AFI.",
"Code": "failed"
},
"ShellVersion": "0x04261818",
"OwnerId": "xxx",
"FpgaImageId": "afi-02177f519cb27c173",
"CreateTime": "2019-07-26T08:19:29.000Z",
"Description": "firesim-buildtriplet:FireBoomNoNIC-With2GHzTarget_With6PerfCounters_With20Trackers_With5IdBits_With16kL1Booms_FireSimBoomQuadCoreConfig-With1RankDRAM_With20MSHRs_FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-deploytriplet:FireBoomNoNIC-With2GHzTarget_With6PerfCounters_With20Trackers_With5IdBits_With16kL1Booms_FireSimBoomQuadCoreConfig-With1RankDRAM_With20MSHRs_FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-commit:afc21c534dcac8e2b7f3dada0cd4d8576d785396-dirty"
}
]
}
{
"FpgaImages": [
{
"UpdateTime": "2019-07-26T08:49:23.000Z",
"Name": "fireboom-quadcore-ddr3-1rank-llc4mb-16kl1",
"Tags": [],
"PciId": {
"SubsystemVendorId": "0xfedd",
"VendorId": "0x1d0f",
"DeviceId": "0xf000",
"SubsystemId": "0x1d51"
},
"FpgaImageGlobalId": "agfi-0f6deb3593bf57481",
"Public": false,
"State": {
"Message": "UNKNOWN_BITSTREAM_GENERATE_ERROR: An error occurred generating the FPGA image bitstream. If an S3 LogsStorageLocation was provided in the CreateFpgaImage request, review the captured bitstream generation logs saved to S3 under the FpgaImageId for this AFI.",
"Code": "failed"
},
"ShellVersion": "0x04261818",
"OwnerId": "xxx",
"FpgaImageId": "afi-02177f519cb27c173",
"CreateTime": "2019-07-26T08:19:29.000Z",
"Description": "firesim-buildtriplet:FireBoomNoNIC-With2GHzTarget_With6PerfCounters_With20Trackers_With5IdBits_With16kL1Booms_FireSimBoomQuadCoreConfig-With1RankDRAM_With20MSHRs_FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-deploytriplet:FireBoomNoNIC-With2GHzTarget_With6PerfCounters_With20Trackers_With5IdBits_With16kL1Booms_FireSimBoomQuadCoreConfig-With1RankDRAM_With20MSHRs_FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-commit:afc21c534dcac8e2b7f3dada0cd4d8576d785396-dirty"
}
]
}
Build log:
2019-07-26 08:49:18,034 [flush ] [DEBUG] [localhost] local: aws ec2 describe-fpga-images --fpga-image-id afi-02177f519cb27c173 | tee AGFI_INFO
2019-07-26 08:49:18,472 [aws_build ] [INFO ] Current state: pending
2019-07-26 08:49:28,482 [flush ] [DEBUG] [localhost] local: aws ec2 describe-fpga-images --fpga-image-id afi-02177f519cb27c173 | tee AGFI_INFO
2019-07-26 08:49:28,914 [aws_build ] [INFO ] Current state: failed
2019-07-26 08:49:38,943 [copy_afi_to_] [INFO ] Copy starting region is: us-east-1
2019-07-26 08:49:38,943 [copy_afi_to_] [INFO ] Regions to copy to: ['us-west-2', 'eu-west-1']
2019-07-26 08:49:38,943 [copy_afi_to_] [INFO ] Copying AFI: afi-02177f519cb27c173
2019-07-26 08:49:39,412 [copy_afi_to_] [DEBUG] {u'FpgaImageId': 'afi-0f2e9a66ba0ebd2fc', 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'c2ed0ba4-3a9e-4d2f-a2bd-36fa946ac009', 'HTTPHeaders': {'transfer-encoding': 'chunked', 'vary': 'accept-encoding', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Fri, 26 Jul 2019 08:49:39 GMT'}}}
2019-07-26 08:49:39,413 [copy_afi_to_] [INFO ] Copy result: afi-0f2e9a66ba0ebd2fc
2019-07-26 08:49:39,890 [copy_afi_to_] [DEBUG] {u'FpgaImageId': 'afi-06620405f1fe73464', 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '69f29c70-6c43-45e1-ae66-b331171c6569', 'HTTPHeaders': {'transfer-encoding': 'chunked', 'vary': 'accept-encoding', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Fri, 26 Jul 2019 08:49:38 GMT'}}}
2019-07-26 08:49:39,890 [copy_afi_to_] [INFO ] Copy result: afi-06620405f1fe73464
2019-07-26 08:49:40,005 [aws_build ] [INFO ] FireSim FPGA Build Completed
2019-07-26 08:49:40,006 [aws_build ] [INFO ] Your AGFI has been created!
Add
[fireboom-quadcore-ddr3-1rank-llc4mb-16kl1]
agfi=agfi-0f6deb3593bf57481
deploytripletoverride=None
customruntimeconfig=None
2019-07-26 08:49:18,472 [aws_build ] [INFO ] Current state: pending
2019-07-26 08:49:28,482 [flush ] [DEBUG] [localhost] local: aws ec2 describe-fpga-images --fpga-image-id afi-02177f519cb27c173 | tee AGFI_INFO
2019-07-26 08:49:28,914 [aws_build ] [INFO ] Current state: failed
2019-07-26 08:49:38,943 [copy_afi_to_] [INFO ] Copy starting region is: us-east-1
2019-07-26 08:49:38,943 [copy_afi_to_] [INFO ] Regions to copy to: ['us-west-2', 'eu-west-1']
2019-07-26 08:49:38,943 [copy_afi_to_] [INFO ] Copying AFI: afi-02177f519cb27c173
2019-07-26 08:49:39,412 [copy_afi_to_] [DEBUG] {u'FpgaImageId': 'afi-0f2e9a66ba0ebd2fc', 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'c2ed0ba4-3a9e-4d2f-a2bd-36fa946ac009', 'HTTPHeaders': {'transfer-encoding': 'chunked', 'vary': 'accept-encoding', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Fri, 26 Jul 2019 08:49:39 GMT'}}}
2019-07-26 08:49:39,413 [copy_afi_to_] [INFO ] Copy result: afi-0f2e9a66ba0ebd2fc
2019-07-26 08:49:39,890 [copy_afi_to_] [DEBUG] {u'FpgaImageId': 'afi-06620405f1fe73464', 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '69f29c70-6c43-45e1-ae66-b331171c6569', 'HTTPHeaders': {'transfer-encoding': 'chunked', 'vary': 'accept-encoding', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Fri, 26 Jul 2019 08:49:38 GMT'}}}
2019-07-26 08:49:39,890 [copy_afi_to_] [INFO ] Copy result: afi-06620405f1fe73464
2019-07-26 08:49:40,005 [aws_build ] [INFO ] FireSim FPGA Build Completed
2019-07-26 08:49:40,006 [aws_build ] [INFO ] Your AGFI has been created!
Add
[fireboom-quadcore-ddr3-1rank-llc4mb-16kl1]
agfi=agfi-0f6deb3593bf57481
deploytripletoverride=None
customruntimeconfig=None
--
You received this message because you are subscribed to the Google Groups "FireSim" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/029b719a-4ef7-45d9-8db2-666e85879264%40googlegroups.com.
David Biancolin
Jul 26, 2019, 6:33:05 PM7/26/19
to FireSim
Ok that's a manager bug. Can you open an issue on firesim for it? Did you manage to find the log in the S3 bucket the status is referring to?
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+unsubscribe@googlegroups.com.
Armia Mrassy
Jul 26, 2019, 6:51:16 PM7/26/19
to fir...@googlegroups.com
I faced an error like that before. In my case, I was synthesizing a customized large design, and the router finished with a warning about a net that exists but it is not connected to anywhere. This net shall be pruned early in the synth as it was not fed to anywhere. My solution was to totally remove this signal in Chisel.
So, I would recommend to check the logs for any weired warning or error happened during the routing phase.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/029b719a-4ef7-45d9-8db2-666e85879264%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "FireSim" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/2d2f588a-56a0-49af-beb8-be62adbf5eb7%40googlegroups.com.
Farzad Farshchi
Jul 26, 2019, 7:50:47 PM7/26/19
to FireSim
I just found the S3 bucket log. The design has an unrouted net. :/ Sorry, I should have looked into the build log more carefully.
Thank you, Armia for your proposed solution. I'm not sure the net that is failing for me can be removed. It seems that it's part of the AR token channel:
WRAPPER_INST/CL/firesim_top/top/sim/ReadyValidChannel_mem_axi4_0_ar/tokens/bankHasReadyEntries_029_out
This is a bit strange. My design is congested but I have seen more congested designs to be successfully routed.
David, have you faced something similar? I attach part of the log here.
I open an issue on the firesim repo.
To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/CALmMyi%2Bq67kauU2mrp7t%3D%3DeRo03tYEotFOfCggfh1ru5cTFrFw%40mail.gmail.com.
David Biancolin
Jul 30, 2019, 1:26:43 PM7/30/19
to FireSim
Ok, so the root cause of the failure here is an unrouted net in part of the memory model when the memory model spans two SLRs. Since Vivado infers a GCLKCE to implement the clock-gating, all of the timing model needs to be in the same SLR for this work properly. However, the memory channels span multiple SLRs so without a good placement constraint this problem can arise. We're looking into it.
Thanks for you help guys.
Thanks for you help guys.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/029b719a-4ef7-45d9-8db2-666e85879264%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "FireSim" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/2d2f588a-56a0-49af-beb8-be62adbf5eb7%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "FireSim" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages