Re: [LINUX RFC v2 0/4] spi: add dual parallel mode support in Zynq MPSoC GQSPI controller

[Marek Vasut]

On Wednesday, August 26, 2015 at 08:26:03 AM, Ranjit Waghmode wrote:
> This series adds dual parallel mode support for Zynq Ultrascale+
> MPSoC GQSPI controller driver.
>
> What is dual parallel mode?
> ---------------------------
> ZynqMP GQSPI controller supports Dual Parallel mode with following
> functionalities: 1) Supporting two SPI flash memories operating in
> parallel. 8 I/O lines. 2) Chip selects and clock are shared to both the
> flash devices
> 3) This mode is targeted for faster read/write speed and also doubles the
> size 4) Commands/data can be transmitted/received from both the
> devices(mirror), or only upper or only lower flash memory devices.
> 5) Data arrangement:
> With stripe enabled,
> Even bytes i.e. 0, 2, 4,... are transmitted on Lower Data Bus
> Odd bytes i.e. 1, 3, 5,.. are transmitted on Upper Data Bus.

This might be a dumb question, but why don't you just treat this as an
SPI NOR flash with 8-bit bus ?

Best regards,
Marek Vasut
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

[Jagan Teki]

On 26 August 2015 at 11:56, Ranjit Waghmode <ranjit....@xilinx.com> wrote:
> This series adds dual parallel mode support for Zynq Ultrascale+
> MPSoC GQSPI controller driver.
>
> What is dual parallel mode?
> ---------------------------
> ZynqMP GQSPI controller supports Dual Parallel mode with following functionalities:
> 1) Supporting two SPI flash memories operating in parallel. 8 I/O lines.
> 2) Chip selects and clock are shared to both the flash devices
> 3) This mode is targeted for faster read/write speed and also doubles the size
> 4) Commands/data can be transmitted/received from both the devices(mirror),
> or only upper or only lower flash memory devices.
> 5) Data arrangement:
> With stripe enabled,
> Even bytes i.e. 0, 2, 4,... are transmitted on Lower Data Bus
> Odd bytes i.e. 1, 3, 5,.. are transmitted on Upper Data Bus.
>

> This series also updated MTD layer files for adding parallel mode support.
>
> 1) Added Support for two flashes
> 2) Support to enable/disable data stripe as and when required.
> 3) Added required parameters to spi_nor structure. Initialized all
> added parameters in spi_nor_scan()
> 4) Added support for dual parallel in spi_nor_read/write/erase functions by:
> a) Increasing page_size, sector_size, erase_size and toatal flash size
> as and when required.
> b) Dividing address by 2
> c) Updating spi->master->flags for qspi driver to change CS
> 5) Updated read_sr() to get status of both flashes
> 6) Also updated read_fsr() to get status of both flashes
>
> These all are very high level changes and expected to make an idea clear.
> Comments and suggestions are always welcomed
>
> ---
> V2 Changes:
> a) Splitted patches based on logical changes
> b) Added error handling for newly added APIs in SPI core
> ---
>
> Ranjit Waghmode (4):
> spi: add support of two chip selects & data stripe
> mtd: add spi_device instance to spi_nor struct
> spi-nor: add dual parallel mode support
> spi: zynqmp: gqspi: add support for dual parallel mode configuration

I don't find any previous discussion about way to inform flash
dual-ness into spi-nor
from spi drivers.

Here is my idea, probably others may think same.
Informing dual_flash from drivers/spi through flags or any other mode
bits is not a better approach as dual flash feature is specific to
spi-nor flash controller (controller specially designed for spi-nor
flash not the generic spi controller). So if the driver sits on
drivers/mtd/spi-nor/ (ex: fsl-quadspi.c), may be we can inform flash
specific things to spi-nor as it's not touching generic spi stack in
Linux. But there is a defined-drawback if the driver is moved to
drivers/mtd/spi-nor ie it can't use spi core API's at-all.

thanks!
--
Jagan | openedev.

[punnaiah choudary kalluri]

Xilinx GQSPI is a generic quad spi controller. The primary goal is to support
Generic/Future command sequences and Future NOR/NAND flash devices.
This core can also be used for legacy SPI devices. Due to the generic nature
of the core, software can generate any command sequence. It also has additional
features like parallel and stacked configurations to double the data
rate and size.
Accessing spi-nor flash device is one particular use case and like
that there will be
many. So, we decided to keep this driver in generic spi framework and
that is the ideal
thing to do for the GQSPI controller.

Regards,
Punnaiah

[Jagan Teki]

On 26 August 2015 at 21:02, punnaiah choudary kalluri

Yes, I understand the generic nature of the GQSPI and it's good to
have all-in-one like generic spi, spi-nor and spi-nand and more
together, but Linux stacks were implemented in such a way to support
the each type of controller with connected slaves separably for better
handling.

Currently GQSPI driver is added in drivers/spi as it supports generic
spi nature and now it enhanced more through flags for supporting
spi-nor, what if we add spi-nand support for the same controller? do
we add one more driver in spi-nand framework (drivers/mtd/spi-nand -
an on going implementation)? My only observation here is even if the
controller is more generic to support more number of device classes,
and adding same driver and populate the slave stuff through flags or
different kind of mechanism to different driver stack, this is not a
better approach I thought.

Based on the above comments, there is an approach to handle this
support and I'm not 100% sure whether this fits or not but we
implemented the same - this is "probing child devices from parent"
(there was a discussion with Arnd earlier wrt this, but I'm unable to
get the mailing thread)

Added Arnd (probably will give more inputs or corrections)

Let me explain how we implemented on our design.
We have PCIe controller that support basic root complex handling, dma
and controller hotplug (not in-build pcie hp) and ideally we need to
write driver for handling root complex on drivers/pci/host and one
hotplug driver in drivers/pci and one more driver in drivers/dma for
handling pcie dma stuff. And some pcie calls need to navigate from
root complex driver to dma and hotplug driver that means there is call
transition from driver/pci to driver/dma which is absolutely not a
good approach (spi to spi-nor and spi-nand transition - in GQSPI case)

So the implementation we follow is like there is a pcie root complex
driver(probably generic spi driver in drivers/spi/*) and inside probe
we have register platform_device for hotplug (spi-nor) and dma
(spi-nand) and the dma driver in drivers/dma and hotplug driver in
driver/pci/ are platform drivers which is of legacy binding (not with
dts) so there should be a common dts for root complex driver
(drivers/spi/*) and individual child driver need to take those while
registering platform_device.

example pseudo:

drivers/dma/dma-child2.c

Legacy platform_driver binding and handling dma future as normal dma
driver, spi-nand in your case

drivers/pci/hotplug/hp-child1.c

Legacy platform_driver binding and handling hotplug future as normal
hotplug driver, spi-nor in your case.

drivers/pci/host/rc-parent-pci.c

static int rc_parent_pcie_probe_bridge(struct platform_device *pdev)
{
// Generic rc handling (genric spi stuff)

// Hotplug handling (spi-nor)
- platform_device_alloc
- assign need resources
- register pdev using platform_device_add

// DMA handling (spi-nand)
- same as above
}

static const struct of_device_id rc_parent_pcie_match_table[] = {
{.compatible = "abc,rc-parent",},
{},
};

static struct platform_driver rc_parent_pcie_driver = {
.driver = {
.name = "rc-parent",
.of_match_table = of_match_ptr(rc_parent_pcie_match_table),
},
.probe = rc_parent_pcie_probe_bridge,
};
module_platform_driver(rc_parent_pcie_driver);

I couldn't find any driver mainlined wrt this design, think more on
GQSPI front, whether this design fits well or not.

[punnaiah choudary kalluri]

On Thu, Aug 27, 2015 at 11:53 AM, Jagan Teki <jt...@openedev.com> wrote:
> On 26 August 2015 at 21:02, punnaiah choudary kalluri
> <pun...@xilinx.com> wrote:
>> On Wed, Aug 26, 2015 at 5:49 PM, Jagan Teki <jt...@openedev.com> wrote:
>>> On 26 August 2015 at 11:56, Ranjit Waghmode <ranjit....@xilinx.com> wrote:
>>>> This series adds dual parallel mode support for Zynq Ultrascale+
>>>> MPSoC GQSPI controller driver.
>>>>
>>>> What is dual parallel mode?
>>>> ---------------------------
>>>> ZynqMP GQSPI controller supports Dual Parallel mode with following functionalities:
>>>> 1) Supporting two SPI flash memories operating in parallel. 8 I/O lines.
>>>> 2) Chip selects and clock are shared to both the flash devices
>>>> 3) This mode is targeted for faster read/write speed and also doubles the size
>>>> 4) Commands/data can be transmitted/received from both the devices(mirror),
>>>> or only upper or only lower flash memory devices.
>>>> 5) Data arrangement:
>>>> With stripe enabled,
>>>> Even bytes i.e. 0, 2, 4,... are transmitted on Lower Data Bus
>>>> Odd bytes i.e. 1, 3, 5,.. are transmitted on Upper Data Bus.

<snip>

True and this is the reason we have controller drivers and protocol drivers.
GQSPI is the controller driver and spi-nor and spi-nand are the
protocol drivers.

>
> Currently GQSPI driver is added in drivers/spi as it supports generic
> spi nature and now it enhanced more through flags for supporting
> spi-nor, what if we add spi-nand support for the same controller? do
> we add one more driver in spi-nand framework (drivers/mtd/spi-nand -
> an on going implementation)? My only observation here is even if the
> controller is more generic to support more number of device classes,
> and adding same driver and populate the slave stuff through flags or
> different kind of mechanism to different driver stack, this is not a
> better approach I thought.

Just to clear, dual parallel( 2 CS and 8 IO lines) is not only specific
to flash parts, one can use for any other custom streaming protocols
I would say exporting dual parallel connection to protocol drivers is
something like depicting the spi bus topology to the protocol layer.

AFAIK, spi-nor and spi-nand are protocol drivers for accessing the
nor and nand flash devices sitting on the spi bus using the spi
controller driver.

Regards,
Punnaiah

> To unsubscribe from this list: send the line "unsubscribe linux-spi" in

[Jagan Teki]

On 27 August 2015 at 14:18, punnaiah choudary kalluri

So dual parallel may not used for spi-nor flash it can also used other
spi slaves that's what your saying is it?

>
> AFAIK, spi-nor and spi-nand are protocol drivers for accessing the
> nor and nand flash devices sitting on the spi bus using the spi
> controller driver.

Yes, I do agree with your point, but though driver stacks are
different with same kind of bus here, I'm trying to spit the GQSPI
into 3 different controller drivers as Linux understand it and fit on
to Linux stack with out disturbing the generic-ness.

Assumption is GQSPI shall split to various platform_drivers (if each
platform driver treated as a controller) thought it made up of spi
bus.

[punnaiah choudary kalluri]

Yes. As i said above, the main intention of this feature is to improve
the data rate with an overhead of few IO lines.

>
>>
>> AFAIK, spi-nor and spi-nand are protocol drivers for accessing the
>> nor and nand flash devices sitting on the spi bus using the spi
>> controller driver.
>
> Yes, I do agree with your point, but though driver stacks are
> different with same kind of bus here, I'm trying to spit the GQSPI
> into 3 different controller drivers as Linux understand it and fit on
> to Linux stack with out disturbing the generic-ness.

I feel this is not a nice idea. if there are 'n' functionalities and having
'n' controller drivers doesn't seem good in any direction.

Protocol driver can query the spi core about the bus topology and it is the
responsibility of the spi core and controller driver providing this information
to the upper layers.


Regards,
Punnaiah

[Jagan Teki]

On 27 August 2015 at 17:19, punnaiah choudary kalluri

Sorry, to be clear It doesn't depend on n-theory instead it divergent
based on the how many Linux stacks that the GQSPI handle. And also I
commented earlier on thread that it may not be a better solutions but
it could be one of the good approach to fit into Linux-where-it's-not
touching core stacks.

Yes, we can do by adding spi bus driver and adding the
generic-ness,but I'm feel it ended up talking to many stacks which is
advisably not a good idea.

>
> Protocol driver can query the spi core about the bus topology and it is the
> responsibility of the spi core and controller driver providing this information
> to the upper layers.

I agreed the protocol driver definition here,as per the spi-nor
framework the drivers/mtd/spi-nor driver not only a protocol or slave
or flash drivers but there are some controller driver as well ex:
fsl-qspi-spi-nor.c

OK, we both are in different directions - lets wait for any more
comments from others.

[Marek Vasut]

On Wednesday, September 02, 2015 at 07:12:14 PM, Ranjit Abhimanyu Waghmode
wrote:
> Hi Marek,
>
> > -----Original Message-----
> > From: Marek Vasut [mailto:ma...@denx.de]
> > Sent: Wednesday, August 26, 2015 12:26 PM
> > To: Ranjit Abhimanyu Waghmode
> > Cc: dw...@infradead.org; computer...@gmail.com;
> > bro...@kernel.org; Michal Simek; Soren Brinkmann; zaj...@gmail.com;
> > b...@decadent.org.uk; b32...@freescale.com; knut.w...@de.bosch.com;
> > juh...@openwrt.org; bea...@micron.com; linu...@lists.infradead.org;
> > linux-...@vger.kernel.org; linu...@vger.kernel.org; linux-arm-
> > ker...@lists.infradead.org; Harini Katakam; Punnaiah Choudary Kalluri
> > Subject: Re: [LINUX RFC v2 0/4] spi: add dual parallel mode support in
> > Zynq MPSoC GQSPI controller

> >
> > On Wednesday, August 26, 2015 at 08:26:03 AM, Ranjit Waghmode wrote:
> > > This series adds dual parallel mode support for Zynq Ultrascale+ MPSoC
> > > GQSPI controller driver.
> > >
> > > What is dual parallel mode?
> > > ---------------------------
> > > ZynqMP GQSPI controller supports Dual Parallel mode with following
> > > functionalities: 1) Supporting two SPI flash memories operating in
> > > parallel. 8 I/O lines. 2) Chip selects and clock are shared to both
> > > the flash devices
> > > 3) This mode is targeted for faster read/write speed and also doubles
> > > the size 4) Commands/data can be transmitted/received from both the
> > > devices(mirror), or only upper or only lower flash memory devices.
> > >
> > > 5) Data arrangement:
> > > With stripe enabled,
> > > Even bytes i.e. 0, 2, 4,... are transmitted on Lower Data Bus
> > > Odd bytes i.e. 1, 3, 5,.. are transmitted on Upper Data Bus.
> >
> > This might be a dumb question, but why don't you just treat this as an
> > SPI NOR flash with 8-bit bus ?
>

> In case of dual parallel configuration of this controller there are
> different modes like single, dual and quad mode. Whatever you are
> suggesting would fit only in the case of Quad mode operation as both buses
> would have 4 lines each. In case of single mode of parallel configuration,
> there would be two buses; but the line on each bus would one. So
> altogether there will be two lines. And in case of dual mode of parallel
> configuration each bus will be having two lines. So altogether 4 lines
> will be there. So keeping 8 lines would not support above two modes of
> parallel configuration correctly.
>
> Logically it is a single flash with 8 IO lines but physically it's a two
> flash devices and each has 4 IO lines. So, in this case, read and write
> addresses should be always even and minimum data that can be accessed is 2
> bytes.

Oh, I see what the issue is now. It has to do with configuring the flash
into correct bus-width mode, right ?

[Mark Brown]

On Wed, Aug 26, 2015 at 11:56:04AM +0530, Ranjit Waghmode wrote:

> To support dual parallel mode operation of ZynqMP GQSPI controller
> following API's are added inside the core:

As covered in SubmittingPatches please try to make each patch a single
change rather than having multiple separate changes in one commit.

> + /* Controller may support more than one chip.
> + * This flag will enable that feature.
> + */
> +#define SPI_MASTER_BOTH_CS BIT(8) /* enable both chips */

This isn't saying that the controller supports more than one chip, it's
saying that the controller supports asserting more than one chip select
at once which isn't the same thing. I'm also not entirely sure that
this makes sense as a separate feature to the data striping one - I'm
struggling to think of a way to use this sensibly separately to that.

[Marek Vasut]

On Thursday, September 03, 2015 at 03:25:00 PM, Ranjit Abhimanyu Waghmode wrote:
> Hi,

> Yes.

Thanks!

[Martin Sperl]

Well - there is one use-case that I can think of:
fbtft has the requirement for some devices to control a GPIO to
differentiate between command and data getting transferred
- sort of 9 bit.

Right now it is done outside of spi in the fbtft driver itself wrapping
spi_sync().

Similarly a “hold” line on an eeprom or similar could get (de)asserted
without requiring holding a spi-bus-lock.

But then the current patch would not allow this kind of “generic”
use-case.

Martin

[Mark Brown]

On Fri, Sep 04, 2015 at 02:35:52PM +0200, Martin Sperl wrote:
> > On 03.09.2015, at 14:12, Mark Brown <bro...@kernel.org> wrote:

> > This isn't saying that the controller supports more than one chip, it's
> > saying that the controller supports asserting more than one chip select
> > at once which isn't the same thing. I'm also not entirely sure that
> > this makes sense as a separate feature to the data striping one - I'm
> > struggling to think of a way to use this sensibly separately to that.

> Well - there is one use-case that I can think of:
> fbtft has the requirement for some devices to control a GPIO to
> differentiate between command and data getting transferred
> - sort of 9 bit.

That's another thing again, isn't it? It's one device switching between
two different control interfaces at runtime rather than two devices
controlled in lockstep.

[Martin Sperl]

I agree, but there may be a solution that can handle both, so I wanted
to mention it.

[Mark Brown]

On Fri, Sep 04, 2015 at 12:02:21PM +0000, Ranjit Abhimanyu Waghmode wrote:

Please fix your mail client to word wrap within paragraphs and to quote
text without reflowing it - your messages are very hard to read.

> > > + /* Controller may support more than one chip.
> > > + * This flag will enable that feature.
> > > + */
> > > +#define SPI_MASTER_BOTH_CS BIT(8) /* enable both
> > chips */

> > This isn't saying that the controller supports more than one chip, it's saying that
> > the controller supports asserting more than one chip select at once which isn't
> > the same thing. I'm also not entirely sure that this makes sense as a separate
> > feature to the data striping one - I'm struggling to think of a way to use this
> > sensibly separately to that.

> If the SPI controller is having more than one chip select and the data lines are distributed equally.
> And also there is requirement to activate all the chip selects in one go.

I'm not sure I understand the above, sorry. At least not in so far as
how it relates to my concerns, especially the fact that the comment says
this enables support for more than one chip which is obviously a basic
SPI feature.

> Now we can consider following use cases:

> Suppose we need to send the same data to multiple slaves of same kind:
> Here the application need not to do individual slave access for writing, instead it can send data to all the devices in one go.

That's a *very* specific application which will only work for write only
devices - I'd be surprised if such systems actually had distinct chip
select lines at the CPU level.

> Let's take another case where application is trying to send data in such a way that first nibble of the byte will got to the one slave and the second nibble of the byte will go to the other slave:
> Here data in slave devices can be organized by taking advantage of above topology along with the support in hardware.

But do such devices actually exist? I can imagine systems that might be
able to do that but I'd be very surprised to see anyone practically
designing them, they're going to be quite hard to use.

[Harini Katakam]

Hi Mark,

On Fri, Sep 11, 2015 at 6:06 PM, Mark Brown <bro...@kernel.org> wrote:
> On Fri, Sep 04, 2015 at 12:02:21PM +0000, Ranjit Abhimanyu Waghmode wrote:
>
> Please fix your mail client to word wrap within paragraphs and to quote
> text without reflowing it - your messages are very hard to read.
>
>> > > + /* Controller may support more than one chip.
>> > > + * This flag will enable that feature.
>> > > + */
>> > > +#define SPI_MASTER_BOTH_CS BIT(8) /* enable both
>> > chips */
>

>> Now we can consider following use cases:
>
>> Suppose we need to send the same data to multiple slaves of same kind:
>> Here the application need not to do individual slave access for writing, instead it can send data to all the devices in one go.
>
> That's a *very* specific application which will only work for write only
> devices - I'd be surprised if such systems actually had distinct chip
> select lines at the CPU level.
>

Agreed that it is very specific but here are a few ways it is used
when communicating with two flash devices in parallel configuration:
- Write enable is sent to both devices using a single operation.
- Writing to any configuration registers in the flash is done in one go
- Some application that want to mirror important data to both devices.
Even with reading, the assertion of multiple cs combined with stripe
will mean:
- Two status bytes, one form each will be obtained in one operation
- Similarly data that was written using stripe is read back and combined.

Such systems could still maintain separate chip selects to perform
individual operations such as reading flash ID, debugging failures or
locking specific sectors.

Regards,
Harini