Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
copy offload for SMB?
218 views
Skip to first unread message
Richard Sharpe
Feb 21, 2013, 9:08:27 PM2/21/13
to
Hi folks,
Does Microsoft have support for doing copies locally on the server
when the source and destination are on the same server?
Eg: cp E:\some-path-1\some-file E:\some-path-2\
--
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)
Does Microsoft have support for doing copies locally on the server
when the source and destination are on the same server?
Eg: cp E:\some-path-1\some-file E:\some-path-2\
--
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)
Christopher R. Hertel
Feb 21, 2013, 9:39:22 PM2/21/13
to
On 02/21/2013 08:08 PM, Richard Sharpe wrote:
> Hi folks,
>
> Does Microsoft have support for doing copies locally on the server
> when the source and destination are on the same server?
>
> Eg: cp E:\some-path-1\some-file E:\some-path-2\
Yes. See [MS-SMB2] section 2.2.31.1, covering SRV_COPYCHUNK_COPY
> Hi folks,
>
> Does Microsoft have support for doing copies locally on the server
> when the source and destination are on the same server?
>
> Eg: cp E:\some-path-1\some-file E:\some-path-2\
"The SRV_COPYCHUNK_COPY packet is sent in an SMB2 IOCTL Request by
the client to initiate a server-side copy of data."
Chris -)-----
--
"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/ -)----- c...@ubiqx.mn.org
OnLineBook -- http://ubiqx.org/cifs/ -)----- c...@ubiqx.org
Liujun (A)
Feb 21, 2013, 9:40:09 PM2/21/13
to
I have wrote a test to test offload feature, the is result is that Windows server 2012 does not support offload read (FSCTL_OFFLOAD_READ).
-----邮件原件-----
发件人: samba-techn...@lists.samba.org [mailto:samba-techn...@lists.samba.org] 代表 Richard Sharpe
发送时间: 2013年2月22日 10:08
收件人: samba-technical
主题: copy offload for SMB?
Hi folks,
Does Microsoft have support for doing copies locally on the server
when the source and destination are on the same server?
Eg: cp E:\some-path-1\some-file E:\some-path-2\
-----邮件原件-----
发件人: samba-techn...@lists.samba.org [mailto:samba-techn...@lists.samba.org] 代表 Richard Sharpe
发送时间: 2013年2月22日 10:08
收件人: samba-technical
主题: copy offload for SMB?
Hi folks,
Does Microsoft have support for doing copies locally on the server
when the source and destination are on the same server?
Eg: cp E:\some-path-1\some-file E:\some-path-2\
Christopher R. Hertel
Feb 21, 2013, 9:57:24 PM2/21/13
to
The Windows Offload read/write operation is not the same as copychunk.
See:
http://msdn.microsoft.com/en-us/library/windows/desktop/hh848056%28v=vs.85%29.aspx
for an overview of the offload read/write feature.
Chris -)-----
See:
http://msdn.microsoft.com/en-us/library/windows/desktop/hh848056%28v=vs.85%29.aspx
for an overview of the offload read/write feature.
Chris -)-----
Richard Sharpe
Feb 22, 2013, 12:49:35 AM2/22/13
to
On Thu, Feb 21, 2013 at 6:39 PM, Christopher R. Hertel <c...@samba.org> wrote:
> On 02/21/2013 08:08 PM, Richard Sharpe wrote:
> On 02/21/2013 08:08 PM, Richard Sharpe wrote:
>> Hi folks,
>>
>> Does Microsoft have support for doing copies locally on the server
>> when the source and destination are on the same server?
>>
>> Eg: cp E:\some-path-1\some-file E:\some-path-2\
>
>>
>> Does Microsoft have support for doing copies locally on the server
>> when the source and destination are on the same server?
>>
>> Eg: cp E:\some-path-1\some-file E:\some-path-2\
>
> Yes. See [MS-SMB2] section 2.2.31.1, covering SRV_COPYCHUNK_COPY
>
> "The SRV_COPYCHUNK_COPY packet is sent in an SMB2 IOCTL Request by
> the client to initiate a server-side copy of data."
Thanks for that. It looks like the steps are:
>
> "The SRV_COPYCHUNK_COPY packet is sent in an SMB2 IOCTL Request by
> the client to initiate a server-side copy of data."
1. Open the source file and send an FSCTL asking for a resume key.
2. Create the destination file.
3. Issue an FSCTL on the newly created destination file specifying the
resume key and the blocks to copy.
4. Close files.
Now to check if W2K08 sends such an FSCTL to see if it can get a resume key.
David Disseldorp
Feb 22, 2013, 5:30:21 AM2/22/13
to
On Thu, 21 Feb 2013 21:49:35 -0800
Richard Sharpe <realrich...@gmail.com> wrote:
> Thanks for that. It looks like the steps are:
>
> 1. Open the source file and send an FSCTL asking for a resume key.
>
> 2. Create the destination file.
>
> 3. Issue an FSCTL on the newly created destination file specifying the
> resume key and the blocks to copy.
>
> 4. Close files.
Yes, see the smb2.ioctl.copy_chunk_* torture tests.
Richard Sharpe <realrich...@gmail.com> wrote:
> Thanks for that. It looks like the steps are:
>
> 1. Open the source file and send an FSCTL asking for a resume key.
>
> 2. Create the destination file.
>
> 3. Issue an FSCTL on the newly created destination file specifying the
> resume key and the blocks to copy.
>
> 4. Close files.
> Now to check if W2K08 sends such an FSCTL to see if it can get a resume key.
See source3/smbd/smb2_ioctl_network_fs.c for the Samba implementation.
Cheers, David
Richard Sharpe
Feb 22, 2013, 9:45:36 AM2/22/13
to
something like that?
David Disseldorp
Feb 22, 2013, 10:08:50 AM2/22/13
to
Hi Richard,
On Fri, 22 Feb 2013 06:45:36 -0800
pre-2k12 that uses it on the client side, but the ioctl itself has been
around for donkey's years (SMB1), so there's likely something obscure
using it.
Cheers, David
1. [cifs-protocol] FSCTL_SRV_COPYCHUNK locking discrepancies
https://lists.samba.org/archive/cifs-protocol/2012-December/002407.html
On Fri, 22 Feb 2013 06:45:36 -0800
Richard Sharpe <realrich...@gmail.com> wrote:
> On Fri, Feb 22, 2013 at 2:30 AM, David Disseldorp <dd...@suse.de> wrote:
> > On Thu, 21 Feb 2013 21:49:35 -0800
> > Richard Sharpe <realrich...@gmail.com> wrote:
> >> Now to check if W2K08 sends such an FSCTL to see if it can get a resume key.
> >
> > 2k8 doesn't use it for copies initiated via Explorer, 2k12 does though.
> > See source3/smbd/smb2_ioctl_network_fs.c for the Samba implementation.
>
> Thanks. Do you know if it is exposed on W2K08 via CopyFileEx or
> something like that?
Server support is there, with a few quirks[1]. I'm not aware of anything
> On Fri, Feb 22, 2013 at 2:30 AM, David Disseldorp <dd...@suse.de> wrote:
> > On Thu, 21 Feb 2013 21:49:35 -0800
> > Richard Sharpe <realrich...@gmail.com> wrote:
> >> Now to check if W2K08 sends such an FSCTL to see if it can get a resume key.
> >
> > 2k8 doesn't use it for copies initiated via Explorer, 2k12 does though.
> > See source3/smbd/smb2_ioctl_network_fs.c for the Samba implementation.
>
> Thanks. Do you know if it is exposed on W2K08 via CopyFileEx or
> something like that?
pre-2k12 that uses it on the client side, but the ioctl itself has been
around for donkey's years (SMB1), so there's likely something obscure
using it.
Cheers, David
1. [cifs-protocol] FSCTL_SRV_COPYCHUNK locking discrepancies
https://lists.samba.org/archive/cifs-protocol/2012-December/002407.html
Richard Sharpe
Feb 22, 2013, 10:21:05 AM2/22/13
to
I am interested in how to access this functionality from a client.
David Disseldorp
Feb 22, 2013, 10:36:26 AM2/22/13
to
On Fri, 22 Feb 2013 02:40:09 +0000
"Liujun (A)" <liuj...@huawei.com> wrote:
> I have wrote a test to test offload feature, the is result is that Windows server 2012 does not support offload read (FSCTL_OFFLOAD_READ).
For the record, I think Windows Server 2k12 support for the new
"Liujun (A)" <liuj...@huawei.com> wrote:
> I have wrote a test to test offload feature, the is result is that Windows server 2012 does not support offload read (FSCTL_OFFLOAD_READ).
FSCTL_OFFLOAD_READ/WRITE SMBs is dependent on the underlying storage
subsystem.
It requires an advanced (expensive) storage array that handles
POPULATE_TOKEN and WRITE_USING_TOKEN SCSI commands. See
www.snia.org/sites/default/files2/SDC2011/presentations/monday/NealChristiansen_Win8FileSystemPerformance.pdf
Cheers, David
Christopher R. Hertel
Feb 22, 2013, 11:22:50 AM2/22/13
to
On 02/22/2013 09:08 AM, David Disseldorp wrote:
:
IOCTL handling in SMB1 is flaky at best. I believe that the
NT_TRANSACT_IOCTL call works, but SMB_COM_IOCTL is hamstrung and
TRANS2_IOCTL2 was never implemented. Do you have pointers to docs showing
when this IOCTL was introduced to Windows and/or how it was used?
The OS/2 LAN Manager SMB dialect (and all subsequent SMB/CIFS dialects)
supported the SMB_COM_COPY and SMB_COM_MOVE SMB commands. These commands
were supposed to use server-side operations to (respectively) copy and
rename files. The implementations of those two commands in Windows NT and
anything beyond Windows NT is quite broken. The commands are there, they
just don't work.
Chris -)-----
:
> Server support is there, with a few quirks[1]. I'm not aware of anything
> pre-2k12 that uses it on the client side, but the ioctl itself has been
> around for donkey's years (SMB1), so there's likely something obscure
> using it.
David,
> pre-2k12 that uses it on the client side, but the ioctl itself has been
> around for donkey's years (SMB1), so there's likely something obscure
> using it.
IOCTL handling in SMB1 is flaky at best. I believe that the
NT_TRANSACT_IOCTL call works, but SMB_COM_IOCTL is hamstrung and
TRANS2_IOCTL2 was never implemented. Do you have pointers to docs showing
when this IOCTL was introduced to Windows and/or how it was used?
The OS/2 LAN Manager SMB dialect (and all subsequent SMB/CIFS dialects)
supported the SMB_COM_COPY and SMB_COM_MOVE SMB commands. These commands
were supposed to use server-side operations to (respectively) copy and
rename files. The implementations of those two commands in Windows NT and
anything beyond Windows NT is quite broken. The commands are there, they
just don't work.
Chris -)-----
Christopher R. Hertel
Feb 22, 2013, 11:21:31 AM2/22/13
to
On 02/22/2013 09:36 AM, David Disseldorp wrote:
> On Fri, 22 Feb 2013 02:40:09 +0000
> "Liujun (A)" <liuj...@huawei.com> wrote:
>
>> I have wrote a test to test offload feature, the is result is that Windows server 2012 does not support offload read (FSCTL_OFFLOAD_READ).
>
> For the record, I think Windows Server 2k12 support for the new
> FSCTL_OFFLOAD_READ/WRITE SMBs is dependent on the underlying storage
> subsystem.
Yes. It is akin in some ways to NDMP.
> On Fri, 22 Feb 2013 02:40:09 +0000
> "Liujun (A)" <liuj...@huawei.com> wrote:
>
>> I have wrote a test to test offload feature, the is result is that Windows server 2012 does not support offload read (FSCTL_OFFLOAD_READ).
>
> For the record, I think Windows Server 2k12 support for the new
> FSCTL_OFFLOAD_READ/WRITE SMBs is dependent on the underlying storage
> subsystem.
> It requires an advanced (expensive) storage array that handles
> POPULATE_TOKEN and WRITE_USING_TOKEN SCSI commands. See
> www.snia.org/sites/default/files2/SDC2011/presentations/monday/NealChristiansen_Win8FileSystemPerformance.pdf
should be possible to support this at the lower end.
David Disseldorp
Feb 22, 2013, 11:42:53 AM2/22/13
to
Hi Christopher,
On Fri, 22 Feb 2013 10:22:50 -0600
Cheers, David
On Fri, 22 Feb 2013 10:22:50 -0600
> On 02/22/2013 09:08 AM, David Disseldorp wrote:
> :
> > Server support is there, with a few quirks[1]. I'm not aware of anything
> > pre-2k12 that uses it on the client side, but the ioctl itself has been
> > around for donkey's years (SMB1), so there's likely something obscure
> > using it.
>
> David,
>
> IOCTL handling in SMB1 is flaky at best. I believe that the
> NT_TRANSACT_IOCTL call works, but SMB_COM_IOCTL is hamstrung and
> TRANS2_IOCTL2 was never implemented. Do you have pointers to docs showing
> when this IOCTL was introduced to Windows and/or how it was used?
No, nothing aside from [MS-SMB].pdf.
> :
> > Server support is there, with a few quirks[1]. I'm not aware of anything
> > pre-2k12 that uses it on the client side, but the ioctl itself has been
> > around for donkey's years (SMB1), so there's likely something obscure
> > using it.
>
> David,
>
> IOCTL handling in SMB1 is flaky at best. I believe that the
> NT_TRANSACT_IOCTL call works, but SMB_COM_IOCTL is hamstrung and
> TRANS2_IOCTL2 was never implemented. Do you have pointers to docs showing
> when this IOCTL was introduced to Windows and/or how it was used?
Cheers, David
Christopher R. Hertel
Feb 22, 2013, 11:49:06 AM2/22/13
to
Ah... You're right. There it is. The notes show that CopyChunk was added
to W2K3, so it wasn't there in W2K or NT. That's not quite donkey's years
but it is well more than a few Wildebeest's weeks. :)
Thanks!
Chris -)-----
to W2K3, so it wasn't there in W2K or NT. That's not quite donkey's years
but it is well more than a few Wildebeest's weeks. :)
Thanks!
Chris -)-----
Richard Sharpe
Feb 22, 2013, 12:20:21 PM2/22/13
to
On Fri, Feb 22, 2013 at 8:49 AM, Christopher R. Hertel <c...@samba.org> wrote:
> Ah... You're right. There it is. The notes show that CopyChunk was added
> to W2K3, so it wasn't there in W2K or NT. That's not quite donkey's years
> but it is well more than a few Wildebeest's weeks. :)
Hmmm, according to this:
> Ah... You're right. There it is. The notes show that CopyChunk was added
> to W2K3, so it wasn't there in W2K or NT. That's not quite donkey's years
> but it is well more than a few Wildebeest's weeks. :)
http://blogs.technet.com/b/markrussinovich/archive/2008/02/04/2815674.aspx?PageIndex=71
CopyFileEx on W2K08 uses IOCTL_COPYCHUNK.
Liujun (A)
Feb 24, 2013, 8:35:43 PM2/24/13
to
I think the POPULATE_TOKEN and WRITE_USING_TOKEN is not only bind to SCSI. For example , app asks token and the request is send to filesystem, and then the filesystem is send to the lower block storage. So if the lower filesystem can generate the token by themselves, there is no need send to the block storage.
-----邮件原件-----
发件人: David Disseldorp [mailto:dd...@suse.de]
发送时间: 2013年2月22日 23:36
收件人: Liujun (A)
抄送: Richard Sharpe; samba-technical
主题: Re: copy offload for SMB?
On Fri, 22 Feb 2013 02:40:09 +0000
"Liujun (A)" <liuj...@huawei.com> wrote:
> I have wrote a test to test offload feature, the is result is that Windows server 2012 does not support offload read (FSCTL_OFFLOAD_READ).
For the record, I think Windows Server 2k12 support for the new
FSCTL_OFFLOAD_READ/WRITE SMBs is dependent on the underlying storage
subsystem.
-----邮件原件-----
发件人: David Disseldorp [mailto:dd...@suse.de]
发送时间: 2013年2月22日 23:36
收件人: Liujun (A)
抄送: Richard Sharpe; samba-technical
主题: Re: copy offload for SMB?
On Fri, 22 Feb 2013 02:40:09 +0000
"Liujun (A)" <liuj...@huawei.com> wrote:
> I have wrote a test to test offload feature, the is result is that Windows server 2012 does not support offload read (FSCTL_OFFLOAD_READ).
For the record, I think Windows Server 2k12 support for the new
FSCTL_OFFLOAD_READ/WRITE SMBs is dependent on the underlying storage
subsystem.
It requires an advanced (expensive) storage array that handles
POPULATE_TOKEN and WRITE_USING_TOKEN SCSI commands. See
www.snia.org/sites/default/files2/SDC2011/presentations/monday/NealChristiansen_Win8FileSystemPerformance.pdf
Cheers, David
POPULATE_TOKEN and WRITE_USING_TOKEN SCSI commands. See
www.snia.org/sites/default/files2/SDC2011/presentations/monday/NealChristiansen_Win8FileSystemPerformance.pdf
0 new messages