5GB files can not be FTPed
KTL
Has anyone worked with this size transfers?
Looks like there are all intergers and not int64???????????
Remy Lebeau (TeamB)
"KTL" <k...@rs-net.com> wrote in message
news:3f132180$1...@newsgroups.borland.com...
> It seems like we can not FTP very large files (4GB+) with INDY.
Indy uses TStream, which does not support 4GB+ files, It only supports up to
2GB.
> Looks like there are all intergers and not int64???????????
Correct. Int64 wasn't used for TStream under Delphi/BCB v6. However, Indy
still supports Delphi/BCB v5 at least, which don't support Int64 for
TStream.
Gambit
Ben Hochstrasser
>> It seems like we can not FTP very large files (4GB+) with INDY.
>
> Indy uses TStream, which does not support 4GB+ files, It only supports
> up to 2GB.
Which inevitably rises the question: Why so large files? (or why FTP with
such files?). If I had an inhouse connection, I'd probably look into a
SMB/CIFS protocol, else (aka WAN link) 4 GB is *huge*. One connection drop
and you're dead meat, so I'd split those biggies into at least CD-sized
chunks.
--
Ben
Kyle Cordes
news:Xns93B9B8...@207.105.83.65...
> > Indy uses TStream, which does not support 4GB+ files, It only supports
> > up to 2GB.
>
> Which inevitably rises the question: Why so large files? (or why FTP with
> such files?). If I had an inhouse connection, I'd probably look into a
> SMB/CIFS protocol, else (aka WAN link) 4 GB is *huge*. One connection drop
Most FTP servers and clients can restart a transfer partway through.
I can't think of any particular reason that a large file size would result
in some other protocol being better suited than FTP. HTTP would work just
as well also (it likewise supports restarts).
5 GB is not that large any more. A client of mine generates multi-gigabyte
files daily; they aren't particularly hard to deal with; they are stored on
multi-hundred-gigabyte drive arrays (which aren't all that expensive any
more) and backed up on writable DVDs.
--
[ Kyle Cordes * ky...@kylecordes.com * http://kylecordes.com ]
[ Consulting and Software development tips and techniques ]
[ for Java, EJB, Delphi, the BDE Alternatives Guide, & more ]
Maynard Philbrook
have it read up tp amout and then partition it ..
ect..
MpJin
MpJin
@+
"Kyle Cordes" <ky...@kylecordes.com> a écrit dans le message de
news:3f1353da$1...@newsgroups.borland.com...
KTL
We have 20TB online.
Moving to 1000BaseT now (on 100BaseT)
So basically I am out of luck with Indy and 4GB+ files being FTPed?
I have to GZIP and FTP now?
This is from windows to UNIX, by the way.
Is there a Delphi 6+ version of INDY that DOES support int64?
"MpJin" <mp@jin> wrote in message news:3f143192$1...@newsgroups.borland.com...
Dan F
you though, when doing an Int64 seek use soBeginning or soEnd etc as the
second parameter; using soFromBeginning or soFromEnd forces the 32bit
version to be used (and forces your Int64 to be casted to a 32 bit integer),
as can be seen from the TStream (or was it filestream) source, but which is
not apparent from the help files.
Dan
"KTL" <k...@rs-net.com> wrote in message
news:3f144153$1...@newsgroups.borland.com...
Remy Lebeau (TeamB)
> So basically I am out of luck with Indy and 4GB+ files being FTPed?
Basically, yes. As far as I know, you'll just have to implement the FTP
protocol yourself if you want to support large files. I haven't seen any
third-party FTP components yet that don't also use 'int' instead of 'Int64'
(that's not to say that they don't exist, it is always a possibility).
> Is there a Delphi 6+ version of INDY that DOES support int64?
No, and for good reason - Indy is portable to non-VCLv6 IDEs, whereas the
VCLv6's Int64-capable TStream is not. Perhaps Indy could write their own
custom Int64-capable stream class, but then it would still be limited to
TStream's existing limitations, unless they don't derive from TStream in the
first place but then compatibility with other code gets lost as a result.
Gambit
Yahia El-Qasem
As far as you have a FTP server which implements APPE command correctly and
it is running on a UNIX system with large file support ( take care - not all
filesystems on UNIX are capable to deal with files > 4 GB ! ) you could get
it to work as follows :
Just upload the file in slices of 1-2 GB and APPE the next Slices till you
are finished....
This way you should be fine...
Yahia
"Remy Lebeau (TeamB)" <gamb...@yahoo.com> schrieb im Newsbeitrag
news:3f145ec6$1...@newsgroups.borland.com...
Dan F
Delphi
if LAppend then begin
LStream.Position := LStream.Size;
Dont quote me on it though.
Dan
"Yahia El-Qasem" <Yahia.E...@mgh.metro-ag.de> wrote in message
news:3f147ea4$1...@newsgroups.borland.com...
ckd
>
>So basically I am out of luck with Indy and 4GB+ files being FTPed?
>I have to GZIP and FTP now?
>This is from windows to UNIX, by the way.
>
I think I saw support for scp via indy somewhere... maybe that ssh
package that's been mentioned a lot lately. That might be a workable
alternative.
Remy Lebeau (TeamB)
"Dan F" <Da...@RemoveME.Bigfoot.com> wrote in message
news:3f148a11$1...@newsgroups.borland.com...
> Might just work...since Size will return an Int64 in
> a later version of Delphi
That doesn't mean Indy will actually use the Int64, though. In fact, it
already doesn't even though it is available for VCL versions that do support
Int64 streams.
In order to transmit a file > 2GB using the existing VCL, you'll probably
have to derive your own TStream class that can work with larger files.
Expose an extra property or two that control the range of bytes to work with
each time, and then alter the Position and Size properties to be relative to
that range rather than the entire file as a whole as it usually is. That
way, you can set the range for the first 2GB, transmit the stream, reset the
range to the next 2GB, resend the same stream, etc. until the file is
uploaded. It would actually be a simple class to write under this scenerio.
If you wanted to, you could then put in a couple of precompiler conditionals
into the stream class which control how the Position, Size, and Range work
under different VCL versions, to take advantage of the Int64 support under
newer versions, and then you would be able to write a single block of code
for the FTP client that can run the same regardless of VCL version without
being changed :-)
Feel free to ask if you would like help in writing such a class. I already
have a good idea in my mind of how it would need to be implemented.
> if LAppend then begin
> LStream.Position := LStream.Size;
That puts the position at the very end of the stream, thus won't send any
actual data since Position and Size are always the same data type (int or
Int64) as each other regardless of the version you're using.
Gambit
Dan F
"Remy Lebeau (TeamB)" <gamb...@yahoo.com> wrote in message
news:3f148eec$1...@newsgroups.borland.com...
>
>
> > if LAppend then begin
> > LStream.Position := LStream.Size;
>
> That puts the position at the very end of the stream, thus won't send any
> actual data since Position and Size are always the same data type (int or
> Int64) as each other regardless of the version you're using.
>
>
> Gambit
>
I was just thinking that repeatedly appending 2Gb chunks, it would always
write to the end of the current file...but I forgot Indys
readstream/writestream arent Int64 compatible so maybe not.
Dan
Loren Pechtel
<gamb...@yahoo.com> wrote:
>> Is there a Delphi 6+ version of INDY that DOES support int64?
>
>No, and for good reason - Indy is portable to non-VCLv6 IDEs, whereas the
>VCLv6's Int64-capable TStream is not. Perhaps Indy could write their own
>custom Int64-capable stream class, but then it would still be limited to
>TStream's existing limitations, unless they don't derive from TStream in the
>first place but then compatibility with other code gets lost as a result.
I don't follow. Why not support int64's? If the underlying
tStream doesn't handle it then it doesn't work but that doesn't mean
it wouldn't work on those compiler versions with a 64bit tStream.
Remy Lebeau (TeamB)
"Loren Pechtel" <lorenp...@hotmail.com> wrote in message
news:97rdhvkueqkidnuf3...@4ax.com...
> I don't follow. Why not support int64's?
Because they can't. TStream doesn't support Int64 until Delphi/BCB 6, but
Indy makes use of TStream in many places. Indy is limited to what TStream
is limited to.
> If the underlying tStream doesn't handle it then it doesn't work
> but that doesn't mean it wouldn't work on those compiler versions
> with a 64bit tStream.
The Indy code would have to start being littered with precompiler conditions
to do compiler testing so the code can use Int64 for Int64-aware versions
and Integer otherwise. That will get pretty ugly after awhile, and in fact
is something that is generally prohibited by Indy's development rules.
Compiler-specific coding has to be isolated into as few units as possible.
Which then leads back to my earlier statements - either 1) Indy writes their
own Int64 stream that works regardless of version, 2) they don't use TStream
to begin with, or 3) they simply live with the limitation.
Gambit
Chad Z. Hower aka Kudzu
news:3f170f9b$1...@newsgroups.borland.com:
> conditions to do compiler testing so the code can use Int64 for
> Int64-aware versions and Integer otherwise. That will get pretty ugly
Actually we could define a new type thats based on a conditional, but thats
still a fairly big change.
--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programming is an art form that fights back"
Qualified help FAST with Indy Experts Support
from the experts themselves:
http://www.atozedsoftware.com/indy/experts/support.html
ELKNews - Get your free copy at http://www.atozedsoftware.com
Loren Pechtel
<gamb...@yahoo.com> wrote:
>> If the underlying tStream doesn't handle it then it doesn't work
>> but that doesn't mean it wouldn't work on those compiler versions
>> with a 64bit tStream.
>
>The Indy code would have to start being littered with precompiler conditions
>to do compiler testing so the code can use Int64 for Int64-aware versions
>and Integer otherwise. That will get pretty ugly after awhile, and in fact
>is something that is generally prohibited by Indy's development rules.
>Compiler-specific coding has to be isolated into as few units as possible.
>Which then leads back to my earlier statements - either 1) Indy writes their
>own Int64 stream that works regardless of version, 2) they don't use TStream
>to begin with, or 3) they simply live with the limitation.
Oh, I see--Indy itself wouldn't compile on the earlier
compilers!
Note, though, that it wouldn't need a whole bunch of
conditionals. One would do: BigInt = Int32 or Int64 depending on the
compiler.
Chad Z. Hower aka Kudzu
> conditionals. One would do: BigInt = Int32 or Int64 depending on the
> compiler.
Yes, this is what I suggested in my reply.
--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programming is an art form that fights back"
Need extra help with an Indy problem?
Remy Lebeau (TeamB)
> Note, though, that it wouldn't need a whole bunch
> of conditionals. One would do: BigInt = Int32 or Int64
> depending on the compiler.
The only catch with doing that would be to also make sure that you are never
testing for any values that are larger than the compiler can actually
support for that version. In other words, if BigInt is assigned to Int32
than never test the value for anything greater than MaxInt. I suppose in
this situation, a MaxBigInt const could be declared as well and then any
code that tests BigInt values would make sure that the test don't exceed
MaxBigInt..
Gambit