PATA IDE is slower in newer versions of kernel
Joshua Hintze
I've been digging into this for a couple weeks now. I have an embedded board
that runs Linux Kernel 2.6.10 that is connected to a SSD over a PATA
connection. Data is transferred using UDMA4 speeds and I get a nice
sustained write speed of about 20 MBps which is adequate for us.
I have recently taken the plunge to update to a newer version of the kernel
starting at 2.6.32 (also tried 2.6.33) and what I am seeing writes speeds
drop to about 18 MBps peak with large jumps going from 8 MBps->16 MBs for
sustained throughput.
I decided to read up on the ATA specification and I connected a logic
analyzer to the PATA bus and here is what is happening...after a large chunk
of data is written to the device, instead of the device pausing the transfer
by asserting DDMARDY it actual initiates a device data-out termination by
pulling DMARQ low.
The old kernel 2.6.10 responds to this by strobing the STOP line 8 times
within 800uS and the device releases DMARQ shortly afterwards. On the newer
kernels the strobing of the STOP line takes near 8mS of time. Since the hard
drive only release DMARQ after this 8 pulse strobe I believe this is the
cause of the slower write speeds on the newer kernels. This whole process
happens thousands of times when writing megs of data. So those extra 7mS
begin to add up fast.
My problem is I've been digging through the ide.c/ide-dma.c and other code
but I'm not exactly sure where the code would jump to upon a device
initiated termination on a data-out dma transfer.
My guess is ide_dma_intr(...). Is this correct?
Any other areas I could look at?
Thanks in advance.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Alejandro Riveira Fernández
Joshua Hintze
http://i56.tinypic.com/oqhrug.png
Notice in the above link how there is a lot of activating and then large gaps of no activity. After about 9 mS the STOP line is strobed a few times and then DMARQ finally returns to normal operating. The link below shows the picture of an old 2.6.10 kernel where write speeds are much faster.
http://i51.tinypic.com/a9wiur.png
In the above image notice how the wait time is considerable less 100's of microseconds before the stop lines get strobed.
I'm dedicated to continue working on the problem, I just need to advice from the linux community. I've tried different I/O schedulers such as noop and anticipatory and I'm studying the ide-dma code but still lost to where this is happening.
Thanks!
[ Just CC linux-ide ]
To unsubscribe from this list: send the line "unsubscribe linux-ide" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.851 / Virus Database: 271.1.1/3132 - Release Date: 09/16/10 12:34:00
Maxim Levitsky
> Some updated information. I created a screen capture of what I am seeing. First is the logic analyzer when the newer linux kernel is writing slow to the PATA port.
>
> http://i56.tinypic.com/oqhrug.png
>
> Notice in the above link how there is a lot of activating and then large gaps of no activity. After about 9 mS the STOP line is strobed a few times and then DMARQ finally returns to normal operating. The link below shows the picture of an old 2.6.10 kernel where write speeds are much faster.
>
> http://i51.tinypic.com/a9wiur.png
>
> In the above image notice how the wait time is considerable less 100's of microseconds before the stop lines get strobed.
>
> I'm dedicated to continue working on the problem, I just need to advice from the linux community. I've tried different I/O schedulers such as noop and anticipatory and I'm studying the ide-dma code but still lost to where this is happening.
>
> Thanks!
>
Just a general advice, I don't know ide subsystem at all.
Best regards,
Maxim Levitsky
Joshua Hintze
I'm not sure what you mean by bisect, other than just the definition of the word. I am breaking this problem down adding printks etc. I think I'm finally beginng to understand how this works :)
Josh
-----Original Message-----
From: linux-i...@vger.kernel.org [mailto:linux-i...@vger.kernel.org] On Behalf Of Maxim Levitsky
Sent: Saturday, September 18, 2010 1:23 PM
To: Joshua Hintze
Cc: linux-...@vger.kernel.org; linu...@vger.kernel.org
Subject: RE: PATA IDE is slower in newer versions of kernel
On Sat, 2010-09-18 at 11:12 -0600, Joshua Hintze wrote:
> Some updated information. I created a screen capture of what I am seeing. First is the logic analyzer when the newer linux kernel is writing slow to the PATA port.
>
> http://i56.tinypic.com/oqhrug.png
>
> Notice in the above link how there is a lot of activating and then large gaps of no activity. After about 9 mS the STOP line is strobed a few times and then DMARQ finally returns to normal operating. The link below shows the picture of an old 2.6.10 kernel where write speeds are much faster.
>
> http://i51.tinypic.com/a9wiur.png
>
> In the above image notice how the wait time is considerable less 100's of microseconds before the stop lines get strobed.
>
> I'm dedicated to continue working on the problem, I just need to advice from the linux community. I've tried different I/O schedulers such as noop and anticipatory and I'm studying the ide-dma code but still lost to where this is happening.
>
> Thanks!
>
Maybe you can just bisect this?
Just a general advice, I don't know ide subsystem at all.
Best regards,
Maxim Levitsky
--
To unsubscribe from this list: send the line "unsubscribe linux-ide" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.851 / Virus Database: 271.1.1/3132 - Release Date: 09/18/10 00:34:00
Randy Dunlap
> Thanks,
>
> I'm not sure what you mean by bisect, other than just the definition of the word. I am breaking this problem down adding printks etc. I think I'm finally beginng to understand how this works :)
>
http://www.kernel.org/pub/software/scm/git/docs/git-bisect.html
http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#using-bisect
>
>
> -----Original Message-----
> From: linux-i...@vger.kernel.org [mailto:linux-i...@vger.kernel.org] On Behalf Of Maxim Levitsky
> Sent: Saturday, September 18, 2010 1:23 PM
> To: Joshua Hintze
> Cc: linux-...@vger.kernel.org; linu...@vger.kernel.org
> Subject: RE: PATA IDE is slower in newer versions of kernel
>
> On Sat, 2010-09-18 at 11:12 -0600, Joshua Hintze wrote:
> > Some updated information. I created a screen capture of what I am seeing. First is the logic analyzer when the newer linux kernel is writing slow to the PATA port.
> >
> > http://i56.tinypic.com/oqhrug.png
> >
> > Notice in the above link how there is a lot of activating and then large gaps of no activity. After about 9 mS the STOP line is strobed a few times and then DMARQ finally returns to normal operating. The link below shows the picture of an old 2.6.10 kernel where write speeds are much faster.
> >
> > http://i51.tinypic.com/a9wiur.png
> >
> > In the above image notice how the wait time is considerable less 100's of microseconds before the stop lines get strobed.
> >
> > I'm dedicated to continue working on the problem, I just need to advice from the linux community. I've tried different I/O schedulers such as noop and anticipatory and I'm studying the ide-dma code but still lost to where this is happening.
> >
> > Thanks!
> >
> Maybe you can just bisect this?
> Just a general advice, I don't know ide subsystem at all.
>
> Best regards,
> Maxim Levitsky
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
Arnd Bergmann
> On Sat, 2010-09-18 at 11:12 -0600, Joshua Hintze wrote:
> > I'm dedicated to continue working on the problem, I just need to advice from
> > the linux community. I've tried different I/O schedulers such as noop and
> > anticipatory and I'm studying the ide-dma code but still lost to where this
> > is happening.
> >
> > Thanks!
> >
> Maybe you can just bisect this?
That would probably help to pinpoint the problem. A regression somewhere
between 2.6.10 and 2.6.32 is not that interesting when people are working
on stabilizing 2.6.36 and adding features to 2.6.37.
One important question would be if this happens with the libata drivers
in 2.6.36-rc4 or linux-next as well. Note that the IDE subsystem has been
replaced with libata a few years ago, and that is what all development
happens on. If libata is good enough, there is probably no point in
chasing down ide bugs.
Arnd
Joshua Hintze
Thanks.
No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.851 / Virus Database: 271.1.1/3132 - Release Date: 09/18/10
00:34:00
--
Alan Cox
drivers/ide is pretty much obsolete - so what you are seeing is doubly
odd. Most stuff now is using the newer drivers/ata code which can support
SATA, NCQ etc unlike the old codebase.
One thing to check would be whether using the drivers/ata code shows the
same problem.
Alan
Joshua Hintze
because I'm using a TI Davinci ARM processor in an embedded system. They
wrote a PalmChip driver for the older IDE interface and I don't think they
have anything yet for the drivers/ata (out of curiousity is the newer
drivers/ata the reason why PATA devices show up as /dev/sda in newer
kernels?)
After spending most of the day today adding printks and learning about the
old ide driver I'm pretty much convinced that the IDE driver is working
fine. The long delays are actually between block driver request queues. In
the 2.6.10 kernel the time between block writes is less than 1 ms, where as
the 2.6.32 that TI has blessed takes 8+ ms between blocks that are pushed to
the IDE interface.
Like I mentioned earlier I tried changing the I/O scheduler to Noop and as
for the elevator. Anything else I should look to so I get blocks sent out
faster?
Thanks
-----Original Message-----
From: linux-i...@vger.kernel.org
[mailto:linux-i...@vger.kernel.org] On Behalf Of Alan Cox
Sent: Saturday, September 18, 2010 6:59 PM
To: Joshua Hintze
Cc: linux-...@vger.kernel.org; linu...@vger.kernel.org
Subject: Re: PATA IDE is slower in newer versions of kernel
> I'm dedicated to continue working on the problem, I just need to advice
from the linux community. I've tried different I/O schedulers such as noop
and anticipatory and I'm studying the ide-dma code but still lost to where
this is happening.
drivers/ide is pretty much obsolete - so what you are seeing is doubly
odd. Most stuff now is using the newer drivers/ata code which can support
SATA, NCQ etc unlike the old codebase.
One thing to check would be whether using the drivers/ata code shows the
same problem.
Alan
--
To unsubscribe from this list: send the line "unsubscribe linux-ide" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.851 / Virus Database: 271.1.1/3143 - Release Date: 09/18/10
00:34:00
--