[mbedmicro/mbed] Path to BLE files for target NRF51822 is too long (#2236)

[Brian Daniels]

When building for any NRF51822 based target on Windows, you run into the Windows path limit of 260 characters very quickly. This causes the compile step to fail during copying.

@pan- Could the necessary files be flattened somewhat? I have to build in a directory that's fairly close to the disk root to avoid this problem currently.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[Brian Daniels]

Also FYI @sg-

[Martin Kojtal]

When building for any NRF51822 based target on Windows, you run into the Windows path limit of 260 characters very quickly

what toolchain ? didn't we fix that using intermediate files? Or is this for linking phase? please specify the command to reproduce.

[Brian Daniels]

@0xc0170 This is for any toolchain, and this occurred during the copy file step. This was the exact command that was ran in CI at the working directory C:/Jenkins/workspace/build_matrix_wrapper/node_name=build_node_2/135:

mbed test --compile -m NRF51_DK -t ARM --build-report-junit=C:/Jenkins/workspace/build_matrix_wrapper/node_name=build_node_2/135/.axes/target/NRF51_DK/toolchain/ARM/build_report_NRF51_DK_ARM.xml

And here's the full error:

22:02:31 + mbed test --compile -m NRF51_DK -t ARM --build-report-junit=C:/Jenkins/workspace/build_matrix_wrapper/node_name=build_node_2/135/.axes/target/NRF51_DK/toolchain/ARM/build_report_NRF51_DK_ARM.xml
22:02:32 Building library mbed-build (NRF51_DK, ARM)
22:02:32 Scan: 135
22:02:32 Scan: FEATURE_BLE
22:02:32 Scan: FEATURE_UVISOR
22:02:32 Scan: FEATURE_IPV4
22:02:32 Copy: test_env.h
22:02:32 Copy: unity.h
22:02:32 Copy: unity_config.h
22:02:32 Copy: unity_internals.h
22:02:32 Copy: unity_handler.h
22:02:32 Copy: utest.h
22:02:32 Copy: utest_case.h
22:02:32 Copy: utest_default_handlers.h
22:02:32 Copy: utest_harness.h
22:02:32 Copy: utest_scheduler.h
22:02:32 Copy: utest_serial.h
22:02:32 Copy: utest_shim.h
22:02:32 Copy: utest_specification.h
22:02:32 Copy: utest_stack_trace.h
22:02:32 Copy: utest_types.h
22:02:32 Copy: aes.h
22:02:32 Copy: aesni.h
22:02:32 Copy: arc4.h
22:02:32 Copy: asn1.h
22:02:32 Copy: asn1write.h
22:02:32 Copy: base64.h
22:02:32 Copy: bignum.h
22:02:32 Copy: blowfish.h
22:02:32 Copy: bn_mul.h
22:02:32 Copy: camellia.h
22:02:32 Copy: ccm.h
22:02:32 Copy: certs.h
22:02:32 Copy: check_config.h
22:02:32 Copy: cipher.h
22:02:32 Copy: cipher_internal.h
22:02:32 Copy: compat-1.3.h
22:02:32 Copy: config.h
22:02:32 Copy: ctr_drbg.h
22:02:32 Copy: debug.h
22:02:32 Copy: des.h
22:02:32 Copy: dhm.h
22:02:32 Copy: ecdh.h
22:02:32 Copy: ecdsa.h
22:02:32 Copy: ecjpake.h
22:02:32 Copy: ecp.h
22:02:32 Copy: entropy.h
22:02:32 Copy: entropy_poll.h
22:02:32 Copy: error.h
22:02:32 Copy: gcm.h
22:02:32 Copy: havege.h
22:02:32 Copy: hmac_drbg.h
22:02:32 Copy: md.h
22:02:32 Copy: md2.h
22:02:32 Copy: md4.h
22:02:32 Copy: md5.h
22:02:32 Copy: md_internal.h
22:02:32 Copy: memory_buffer_alloc.h
22:02:32 Copy: net.h
22:02:32 Copy: oid.h
22:02:32 Copy: padlock.h
22:02:32 Copy: pem.h
22:02:32 Copy: pk.h
22:02:32 Copy: pkcs11.h
22:02:32 Copy: pkcs12.h
22:02:32 Copy: pkcs5.h
22:02:32 Copy: pk_internal.h
22:02:32 Copy: platform.h
22:02:32 Copy: ripemd160.h
22:02:32 Copy: rsa.h
22:02:32 Copy: sha1.h
22:02:32 Copy: sha256.h
22:02:32 Copy: sha512.h
22:02:32 Copy: ssl.h
22:02:32 Copy: ssl_cache.h
22:02:32 Copy: ssl_ciphersuites.h
22:02:32 Copy: ssl_cookie.h
22:02:32 Copy: ssl_internal.h
22:02:32 Copy: ssl_ticket.h
22:02:32 Copy: threading.h
22:02:32 Copy: timing.h
22:02:32 Copy: version.h
22:02:32 Copy: x509.h
22:02:32 Copy: x509_crl.h
22:02:32 Copy: x509_crt.h
22:02:32 Copy: x509_csr.h
22:02:32 Copy: xtea.h
22:02:32 Copy: CellularInterface.h
22:02:32 Copy: EthInterface.h
22:02:32 Copy: MeshInterface.h
22:02:32 Copy: NetworkInterface.h
22:02:32 Copy: NetworkStack.h
22:02:32 Copy: nsapi.h
22:02:32 Copy: nsapi_types.h
22:02:32 Copy: Socket.h
22:02:32 Copy: SocketAddress.h
22:02:32 Copy: TCPServer.h
22:02:32 Copy: TCPSocket.h
22:02:32 Copy: UDPSocket.h
22:02:32 Copy: WiFiInterface.h
22:02:32 Copy: DnsQuery.h
22:02:32 Copy: AnalogIn.h
22:02:32 Copy: AnalogOut.h
22:02:32 Copy: BusIn.h
22:02:32 Copy: BusInOut.h
22:02:32 Copy: BusOut.h
22:02:32 Copy: Callback.h
22:02:32 Copy: CallChain.h
22:02:32 Copy: CAN.h
22:02:32 Copy: can_helper.h
22:02:32 Copy: CircularBuffer.h
22:02:32 Copy: critical.h
22:02:32 Copy: CThunk.h
22:02:32 Copy: DigitalIn.h
22:02:32 Copy: DigitalInOut.h
22:02:32 Copy: DigitalOut.h
22:02:32 Copy: DirHandle.h
22:02:32 Copy: Ethernet.h
22:02:32 Copy: FileBase.h
22:02:32 Copy: FileHandle.h
22:02:32 Copy: FileLike.h
22:02:32 Copy: FilePath.h
22:02:32 Copy: FileSystemLike.h
22:02:32 Copy: FunctionPointer.h
22:02:32 Copy: I2C.h
22:02:32 Copy: I2CSlave.h
22:02:32 Copy: InterruptIn.h
22:02:32 Copy: InterruptManager.h
22:02:32 Copy: LocalFileSystem.h
22:02:32 Copy: LowPowerTicker.h
22:02:32 Copy: LowPowerTimeout.h
22:02:32 Copy: LowPowerTimer.h
22:02:32 Copy: mbed.h
22:02:32 Copy: mbed_assert.h
22:02:32 Copy: mbed_debug.h
22:02:32 Copy: mbed_error.h
22:02:32 Copy: mbed_interface.h
22:02:32 Copy: platform.h
22:02:32 Copy: PortIn.h
22:02:32 Copy: PortInOut.h
22:02:32 Copy: PortOut.h
22:02:32 Copy: PwmOut.h
22:02:32 Copy: RawSerial.h
22:02:32 Copy: rtc_time.h
22:02:32 Copy: semihost_api.h
22:02:32 Copy: Serial.h
22:02:32 Copy: SerialBase.h
22:02:32 Copy: SingletonPtr.h
22:02:32 Copy: SPI.h
22:02:32 Copy: SPISlave.h
22:02:32 Copy: Stream.h
22:02:32 Copy: Ticker.h
22:02:32 Copy: Timeout.h
22:02:32 Copy: Timer.h
22:02:32 Copy: TimerEvent.h
22:02:32 Copy: toolchain.h
22:02:32 Copy: Transaction.h
22:02:32 Copy: wait_api.h
22:02:32 Copy: analogin_api.h
22:02:32 Copy: analogout_api.h
22:02:32 Copy: buffer.h
22:02:32 Copy: can_api.h
22:02:32 Copy: dma_api.h
22:02:32 Copy: ethernet_api.h
22:02:32 Copy: gpio_api.h
22:02:32 Copy: gpio_irq_api.h
22:02:32 Copy: i2c_api.h
22:02:32 Copy: lp_ticker_api.h
22:02:32 Copy: pinmap.h
22:02:32 Copy: port_api.h
22:02:32 Copy: pwmout_api.h
22:02:32 Copy: rtc_api.h
22:02:32 Copy: serial_api.h
22:02:32 Copy: sleep_api.h
22:02:32 Copy: spi_api.h
22:02:32 Copy: ticker_api.h
22:02:32 Copy: us_ticker_api.h
22:02:32 Copy: Driver_Common.h
22:02:32 Copy: Driver_Storage.h
22:02:32 Copy: arm_common_tables.h
22:02:32 Copy: arm_const_structs.h
22:02:32 Copy: arm_math.h
22:02:32 Copy: core_ca9.h
22:02:32 Copy: core_caFunc.h
22:02:32 Copy: core_caInstr.h
22:02:32 Copy: core_ca_mmu.h
22:02:32 Copy: core_cm0.h
22:02:32 Copy: core_cm0plus.h
22:02:32 Copy: core_cm3.h
22:02:32 Copy: core_cm4.h
22:02:32 Copy: core_cm4_simd.h
22:02:32 Copy: core_cm7.h
22:02:32 Copy: core_cmFunc.h
22:02:32 Copy: core_cmInstr.h
22:02:32 Copy: core_cmSecureAccess.h
22:02:32 Copy: core_cmSimd.h
22:02:32 Copy: core_sc000.h
22:02:32 Copy: core_sc300.h
22:02:32 Copy: cmsis.h
22:02:32 Copy: cmsis_nvic.h
22:02:32 Copy: compiler_abstraction.h
22:02:32 Copy: nrf.h
22:02:32 Copy: nrf51.h
22:02:32 Copy: nrf51_bitfields.h
22:02:32 Copy: nrf51_deprecated.h
22:02:32 Copy: nrf_delay.h
22:02:32 Copy: system_nrf51.h
22:02:32 Copy: gpio_object.h
22:02:32 Copy: objects.h
22:02:32 Copy: PeripheralNames.h
22:02:32 Copy: PortNames.h
22:02:32 Copy: twi_config.h
22:02:32 Copy: twi_master.h
22:02:32 Copy: crc16.h
22:02:32 Copy: app_scheduler.h
22:02:32 Copy: app_error.h
22:02:32 Copy: app_util.h
22:02:32 Copy: device.h
22:02:32 Copy: PinNames.h
22:02:32 Copy: Mail.h
22:02:32 Copy: MemoryPool.h
22:02:32 Copy: Mutex.h
22:02:32 Copy: Queue.h
22:02:32 Copy: rtos.h
22:02:32 Copy: RtosTimer.h
22:02:32 Copy: rtos_idle.h
22:02:32 Copy: Semaphore.h
22:02:32 Copy: Thread.h
22:02:32 Copy: cmsis_os.h
22:02:32 Copy: RTX_CM_lib.h
22:02:32 Copy: RTX_Config.h
22:02:32 Copy: rt_Event.h
22:02:32 Copy: rt_HAL_CM.h
22:02:32 Copy: rt_List.h
22:02:32 Copy: rt_Mailbox.h
22:02:32 Copy: rt_MemBox.h
22:02:32 Copy: rt_Memory.h
22:02:32 Copy: rt_Mutex.h
22:02:32 Copy: rt_OsEventObserver.h
22:02:32 Copy: rt_Robin.h
22:02:32 Copy: rt_Semaphore.h
22:02:32 Copy: rt_System.h
22:02:32 Copy: rt_Task.h
22:02:32 Copy: rt_Time.h
22:02:32 Copy: rt_Timer.h
22:02:32 Copy: rt_TypeDef.h
22:02:32 Copy: BLE.h
22:02:32 Copy: blecommon.h
22:02:32 Copy: BLEInstanceBase.h
22:02:32 Copy: BLEProtocol.h
22:02:32 Copy: CallChainOfFunctionPointersWithContext.h
22:02:32 Copy: CharacteristicDescriptorDiscovery.h
22:02:32 Copy: deprecate.h
22:02:32 Copy: DiscoveredCharacteristic.h
22:02:32 Copy: DiscoveredCharacteristicDescriptor.h
22:02:32 Copy: DiscoveredService.h
22:02:32 Copy: FunctionPointerWithContext.h
22:02:32 Copy: Gap.h
22:02:32 Copy: GapAdvertisingData.h
22:02:32 Copy: GapAdvertisingParams.h
22:02:32 Copy: GapEvents.h
22:02:32 Copy: GapScanningParams.h
22:02:32 Copy: GattAttribute.h
22:02:32 Copy: GattCallbackParamTypes.h
22:02:32 Copy: GattCharacteristic.h
22:02:32 Copy: GattClient.h
22:02:32 Copy: GattServer.h
22:02:32 Copy: GattServerEvents.h
22:02:32 Copy: GattService.h
22:02:32 Copy: SafeBool.h
22:02:32 Copy: SecurityManager.h
22:02:32 Copy: ServiceDiscovery.h
22:02:32 Copy: UUID.h
22:02:32 Copy: BatteryService.h
22:02:32 Copy: DeviceInformationService.h
22:02:32 Copy: DFUService.h
22:02:32 Copy: EddystoneConfigService.h
22:02:32 Copy: EddystoneService.h
22:02:32 Copy: EnvironmentalService.h
22:02:32 Copy: HealthThermometerService.h
22:02:32 Copy: HeartRateService.h
22:02:32 Copy: iBeacon.h
22:02:32 Copy: LinkLossService.h
22:02:32 Copy: UARTService.h
22:02:32 Copy: URIBeaconConfigService.h
22:02:32 Copy: ble_radio_notification.h
22:02:32 Copy: ble_dfu.h
22:02:32 Copy: ble_advdata.h
22:02:32 Copy: ble_conn_params.h
22:02:32 Copy: ble_conn_state.h
22:02:32 Copy: ble_date_time.h
22:02:32 Copy: ble_gatt_db.h
22:02:32 Copy: ble_sensor_location.h
22:02:32 Copy: ble_srv_common.h
22:02:32 Copy: device_manager.h
22:02:32 Copy: device_manager_cnfg.h
22:02:32 Copy: id_manager.h
22:02:32 Copy: peer_data.h
22:02:32 Copy: peer_database.h
22:02:32 Copy: peer_data_storage.h
22:02:32 Copy: peer_id.h
22:02:32 Copy: peer_manager_types.h
22:02:32 Copy: pm_buffer.h
22:02:32 Copy: pm_mutex.h
22:02:32 Copy: compiler_abstraction.h
22:02:32 Copy: nrf.h
22:02:32 Copy: nrf51.h
22:02:32 Copy: nrf51_bitfields.h
22:02:32 Copy: nrf51_deprecated.h
22:02:32 Copy: ble_flash.h
22:02:32 Copy: nrf_delay.h
22:02:32 Copy: nrf_ecb.h
22:02:32 Copy: nrf_gpio.h
22:02:32 Copy: nrf_gpiote.h
22:02:32 Copy: nrf_nvmc.h
22:02:32 Copy: nrf_temp.h
22:02:32 Copy: nrf_wdt.h
22:02:32 Copy: pstorage.h
22:02:32 Copy: pstorage_platform.h
22:02:32 Copy: bootloader.h
22:02:32 Copy: bootloader_types.h
22:02:32 Copy: bootloader_util.h
22:02:32 Copy: dfu.h
22:02:32 Copy: dfu_app_handler.h
22:02:32 Copy: dfu_bank_internal.h
22:02:32 Copy: dfu_ble_svc.h
22:02:32 Copy: dfu_ble_svc_internal.h
22:02:32 Copy: dfu_init.h
22:02:32 Copy: dfu_transport.h
22:02:32 Copy: dfu_types.h
22:02:32 Copy: hci_mem_pool_internal.h
22:02:32 Copy: crc16.h
22:02:32 Copy: section_vars.h
22:02:32 Copy: fds.h
22:02:32 Copy: fds_config.h
22:02:32 Copy: fds_types_internal.h
22:02:32 Copy: fstorage.h
22:02:32 Copy: fstorage_config.h
22:02:32 Copy: hci_mem_pool.h
22:02:32 Copy: app_scheduler.h
22:02:32 Copy: app_timer.h
22:02:32 Copy: app_error.h
22:02:32 Copy: app_util.h
22:02:32 Copy: app_util_platform.h
22:02:32 Copy: common.h
22:02:32 Copy: nordic_common.h
22:02:32 Copy: nrf_assert.h
22:02:32 Copy: sdk_common.h
22:02:32 Copy: sdk_errors.h
22:02:32 Copy: sdk_mapped_flags.h
22:02:32 Copy: sdk_os.h
22:02:32 Copy: ant_stack_handler_types.h
22:02:32 Copy: ble_stack_handler_types.h
22:02:32 Copy: softdevice_handler.h
22:02:32 Copy: softdevice_handler_appsh.h
22:02:32 [Errno 2] No such file or directory: 'C:\\Jenkins\\workspace\\build_matrix_wrapper\\node_name=build_node_2\\135\\.build/tests\\NRF51_DK\\ARM\\features\\FEATURE_BLE\\targets\\TARGET_NORDIC\\TARGET_MCU_NRF51822\\nrf51-sdk\\source\\nordic_sdk\\components\\softdevice\\common\\softdevice_handler\\softdevice_handler_appsh.h'
22:02:32 Failed to build library
22:02:32 Traceback (most recent call last):
22:02:32   File "C:\Jenkins\workspace\build_matrix_wrapper\node_name=build_node_2\135\tools\test.py", line 221, in <module>
22:02:32     report_exporter.report_to_file(build_report, options.build_report_junit, test_suite_properties=build_properties)
22:02:32   File "C:\Jenkins\workspace\build_matrix_wrapper\node_name=build_node_2\135\tools\test_exporters.py", line 100, in report_to_file
22:02:32     report = self.report(test_summary_ext, test_suite_properties=test_suite_properties)
22:02:32   File "C:\Jenkins\workspace\build_matrix_wrapper\node_name=build_node_2\135\tools\test_exporters.py", line 86, in report
22:02:32     return self.exporter_junit(test_summary_ext, test_suite_properties)
22:02:32   File "C:\Jenkins\workspace\build_matrix_wrapper\node_name=build_node_2\135\tools\test_exporters.py", line 291, in exporter_junit
22:02:32     message = test_result['result']
22:02:32 KeyError: 'result'
22:02:32 [ERROR] 'result'
22:02:32 [mbed] ERROR: "python" returned error code 1.

[Toyomasa Watarai]

@bridadan Just FYI, I used subst command to workaround this issue during OOB test.

C:\Users\toywat01\Documents\GitHub\mbed-os-example-ble>subst x: .

[Brian Daniels]

@TomoYamanaka I never knew about that command, good to know! Thanks a lot :)

[TomoYamanaka]

@bridadan

This is a message to Watarai-san, isn't it ? :)

[Bogdan Marinescu]

@bridadan: did subst actually help? That doesn't make sense. The copy files step doesn't have anything do to with the command line, since it calls copyfile in the shutil Python module, and copyfile shouldn't suffer from this limitation. Are you sure this isn't different problem?

[Vincent Coubard]

@bridadan I would like to avoid modifications in Nordic SDK layout, not because it is difficult or I'm lazy but for maintenance purposes.

This is one of this case when we ask to fix the code instead of fixing the tools.

Our build system is too limited to deal correctly with third parties library.

At the moment if you want to do the perfect import of a third party library in mbed_os, you should do the following steps:

• prefix all your file name including compilation unit and private headers.
• flatten your directories (I can't imagine boost imported in mbed).
• update your code so it take account of the modifications you done.

If a library A has been imported into mbed and a user need to import a library B then you could add another step in the process:

• update content in library B so it take into account changes in library A.

This is madness from a maintenance perspective.

[Brian Daniels]

Ah @TomoYamanaka sorry about that! There are too many people that show up when I type @to and I didn't have enough coffee that morning :) That was indeed for Watarai-san!

@bogdanm I haven't actually tried the subst command yet, but I agree with @pan- that it isn't a valid solution. I was also surprised that this cropped up during the "copy files" step because its all done within Python, not the command line. I will have have to look into this more.

@pan- I see your point about maintenance issues if you have to do lots of file manipulations. Though the prefixing of files I believe is necessary not just for our build system, but for other IDEs as well (though I admit I'm not sure which IDEs currently require this).

The path length issue has come up multiple times, I think it may be time soon to really look at this issue and try to solve it once and for all. We could use Unicode paths on Windows throughout the tools to prevent these kinds of errors.