UFS and NVMe device drivers
HanBin Yoon
Hi Dongjin,
As you already know, http://fxr/703158 (not merged) is unfinished code for a UFS driver. Please feel free to adopt it as a starting point for development if you find it useful.
I gave the lock-free NVMe driver implementation another thought. When http://fxr/774667 lands, only one lock will remain in the NVMe driver's fast path. It does appear that some duplicate locking may be happening between the NVMe driver and the core block driver (which lives in src/devices/block/drivers/core) in fifo transaction mode. So perhaps a variation on the BlockImpl Queue() interface that explicitly prohibits concurrent invocations will help reduce the locking overhead in this situation. But the performance gain of pursuing the above is unknown, and some early prototyping and/or performance profiling may be useful to scope that out first. There may be more locking or other inefficiencies within the core block driver or between it and the filesystem that could prove more significant.
Do you primarily use the emulator to test out your code during development? Or do you use actual hardware that you install Fuchsia on?
Thanks.
Han
Dongjin Kim
Hi,
First, regarding the UFS driver, we would like to start based on the code in the https://fuchsia-review.googlesource.com/c/fuchsia/+/703158 (same link as you sent). The code seems to be written for UFS version 3.1, so we are planning to target version 3.1 based operation in the first half of next year and expand it to version 4.0 in the second half.
In order to test it, we would like to add UFS-related features to the FEMU. One of the QEMU code on the public github (https://github.com/cqu611/qemu-ufs) is written based on version 2.1. It is the same GPL as FEMU, so it seems that we can refer it as base of UFS on FEMU. After the work is completed in the FEMU, it may be reflected in the public QEMU. Please let me know if the above external code can be referred for Fuchsia or if there is any problem with the license.
For NVMe driver, we are constantly tracking your efforts, and it seems that many improvements are being made. We will also continue to improve the function of the NVMe driver, like the recent FUA command-related work. On our side, tests for NVMe are being done on Fuchsia-installed NUC devices, and I think we can expand it based on emulator or dummy device in the future. In addition, the multiqueue utilization plan, which can target both NVMe and UFS, will be carried out in the second half of next year after completion of the UFS driver/femu version 3.1.
Finally, I have some questions for the future work.
- In your UFS driver code, "ZMS" and "MZS" keywords appear. Do you have any development plans for the Zoned devices?
- Do you have any plans for scsi? The usb mass storage already in Fuchsia uses the scsi interface like UFS, but neither of them seems to refer to the "src/device/block/lib/scsi" used in virtio. It seems that a separate scsi interface layer can be configured.
Thanks,
Dongjin Kim
--------- Original Message ---------
Sender : HanBin Yoon <hanbi...@google.com>
Date : 2022-12-14 04:14 (GMT+9)
Title : UFS and NVMe device drivers
HanBin Yoon
Thank you for sharing your plans.
My recommendation for qemu-ufs is to port it to the upstream (public) QEMU to avoid duplicate work. Then, that code will become available in refreshed prebuilt binaries in Fuchsia.
We don’t have any particular plans for adding zoned device support like ZNS in NVMe at this time. I suspect your interests lie in F2FS interoperability however, and we welcome contributions to that end.
Our current SCSI support is limited to some block and primary commands for use with virtio or over USB. The duplicate SCSI definitions in usb-mass-storage is technical debt, and I’ll look to address that shortly. The UFS driver should share the common SCSI definitions.
Thanks.
Han
Dongjin Kim
Hi.
We are developing UFS driver based on the code in the link below.
https://fuchsia-review.googlesource.com/c/fuchsia/+/703158
We've been writing first version of the UFS driver and QEMU with basic UFS operations, and we are going to proceed C++ style refactoring and some tests.
Unit tests will be written based on "mock DDK device", but it will take more time. Plus, we are looking for an integration test method that can be operated by attaching QEMU or real UFS device. We'd like to mkfs some filesystems on the UFS driver and do some I/Os.
Here are some questions about the integration test.
1. Do you have any plan for the integration test of NVMe driver? If so, which test framework do you consider?
2. Will the real UFS device or QEMU-based integration test be necessary for management of UFS driver?
3. Regarding the test framework, "driver test realm" seems to be used for integration tests of device drivers. Is it possible to access to the real device or QEMU device from "driver test realm" (or "storage driver test realm")?
4-1. If so, is it possible by using either "driver test realm" or "storage driver test realm"? What's the difference between them? Is it possible to operate in gerrit CI?
4-2. If not, we need to find another way. There are various candidates such as gtest and writing separate applications or scripts. Do you have any suggestion? What capability configuration is needed for component manifest to access to the device?
(In our environment, the path of the device is shown as /dev/sys/platform/pt/PCI0/bus/00:07.0/pci-00:07.0/ufs/lun-0/block)
Additionally, we'd like to reflect the QEMU changes on both mainline QEMU and FEMU. It may take a long time for the mainline QEMU to be reflected in the FEMU, so we are going to put the changes on the FEMU also and synchronize it later. The FEMU exists as a prebuilt in the fuchsia source tree cloned by JIRI, and the source code seems to be managed in the repo below.
https://fuchsia.googlesource.com/third_party/qemu
Then, if we upstream patches to the repo, will it be automatically reflected in the prebuilt?
Thanks.
Dongjin Kim and Jaeyoon Choi
Date : 2023-01-07 07:27 (GMT+9)
Title : Re: UFS and NVMe device drivers
HanBin Yoon
It’s great to hear about the progress you’re making!
> 1. Do you have any plan for the integration test of NVMe driver? If so, which test framework do you consider?
> 2. Will the real UFS device or QEMU-based integration test be necessary for management of UFS driver?
> 3. Regarding the test framework, "driver test realm" seems to be used for integration tests of device drivers. Is it possible to access to the real device or QEMU device from "driver test realm" (or "storage driver test realm")?
> 4-1. If so, is it possible by using either "driver test realm" or "storage driver test realm"? What's the difference between them? Is it possible to operate in gerrit CI?
https://cs.opensource.google/fuchsia/fuchsia/+/main:src/storage/testing/driver_test_realm/storage_driver_test_realm.cc
It comes packaged with the following storage (non-device) drivers:
https://cs.opensource.google/fuchsia/fuchsia/+/main:src/storage/testing/BUILD.gn?q=f:src%2Fstorage%2Ftesting%2FBUILD%20storage_drivers&ss=fuchsia%2Ffuchsia
You should be able to use either "driver test realm" or "storage driver test realm" as long as you include all of the drivers that you need in your integration test (run during CI/CQ).
My inclination here though is that an installation test in CI/CQ provides high coverage and is perhaps a more thorough test compared to a contrived integration test (unless there’s a particular scenario that you’re looking to test).
> Then, if we upstream patches to the repo, will it be automatically reflected in the prebuilt?
Thanks.
Han
HanBin Yoon
Could you please share with us any model/part numbers of UFS devices or UFS-capable platforms/boards you use for testing? This need not necessarily be for Fuchsia testing. I'm wondering what kind of devices/boards are available on your end that I might use similar or different ones when it comes to testing the UFS driver on a real device. Thanks.
Han
최재윤
Hi Hanbin,
We also had a hard time finding UFS-capable platforms/boards. We couldn't find a board that supports UFS and allows Fuchsia installation, but we could find a laptop that supports PCI-based UFSHCI with an Intel CPU.
The model name is Samsung Galaxy Book S (NT767XCL or NT767XCM) with an Intel Lakefield CPU (i5-L16G7). The Galaxy Book S has 512GB of eUFS 3.0 storage(KLUFG8RHDA-B2D1).
We installed windows and Linux on the laptops and verified that the PCI-based UFS driver supports UFS devices.
However, we have not yet tried installing Fuchsia OS, which we plan to do next month.
Therefore, we are currently only testing on QEMU and a mock device.
Thanks.
Jaeyoon
--------- Original Message ---------
Sender : 'HanBin Yoon' via storage-dev <stora...@fuchsia.dev>
Date : 2023-04-20 09:07 (GMT+9)
--
All posts must follow the Fuchsia Code of Conduct https://fuchsia.dev/fuchsia-src/CODE_OF_CONDUCT or may be removed.
---
You received this message because you are subscribed to the Google Groups "storage-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storage-dev...@fuchsia.dev.
To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/storage-dev/CACs%2BNxRdwY2WUfaFoS%2BsLzaC-2YJcSpgXrkAMk22MkuTnX%3DvrQ%40mail.gmail.com.
