Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Program to open a file in binary, skip X bytes and write the rest of the file to a new file

2 views
Skip to first unread message

scad

unread,
May 27, 2009, 2:46:31 PM5/27/09
to
How would I approach this? I cannot store the entire file in memory
(they are 30GB+) but I need to skip the first 292 bytes and write the
rest to a new file.

Thanks,

Scott

Victor Bazarov

unread,
May 27, 2009, 3:03:42 PM5/27/09
to

1. Open the file to read (file_1)
1.a If file_1 in bad state, report and exit.

2. Counter = number of bytes to skip
3. While counter > 0 and not the end of file_1
read file_1

3.a. If file_1 in bad state, report and exit.

4. Open the file to write (file_2)
4.a If file_2 in bad state, report and exit.

5. While not the end of file_1
read file_1 into 'buffer'
if successful, write 'buffer' to file_2
if file_2 in bad state, report and exit.

V
--
Please remove capital 'A's when replying by e-mail
I do not respond to top-posted replies, please don't ask

mzdude

unread,
May 27, 2009, 4:47:19 PM5/27/09
to
On May 27, 2:03 pm, Victor Bazarov <v.Abaza...@comAcast.net> wrote:
> scad wrote:
> > How would I approach this?  I cannot store the entire file in memory
> > (they are 30GB+) but I need to skip the first 292 bytes and write the
> > rest to a new file.
>
>     1. Open the file to read (file_1)
>     1.a   If file_1 in bad state, report and exit.
>
>     2. Counter = number of bytes to skip
>     3. While counter > 0 and not the end of file_1
>          read file_1
optional step
2. Look up the function seek()

>
>     3.a.  If file_1 in bad state, report and exit.
>
>     4. Open the file to write (file_2)
>     4.a   If file_2 in bad state, report and exit.
>
>     5. While not the end of file_1
>          read file_1 into 'buffer'
>          if successful, write 'buffer' to file_2
>          if file_2 in bad state, report and exit.

If I know I'm dealing with large files, I might even check available
disk space before starting the copy. Hate to get 29G into the copy
and fail due to lack of disk space.

Victor Bazarov

unread,
May 27, 2009, 5:52:55 PM5/27/09
to

I don't think C++ has the means to do that.

James Kanze

unread,
May 28, 2009, 4:47:00 AM5/28/09
to
On May 27, 9:03 pm, Victor Bazarov <v.Abaza...@comAcast.net> wrote:
> scad wrote:
> > How would I approach this? I cannot store the entire file
> > in memory (they are 30GB+) but I need to skip the first 292
> > bytes and write the rest to a new file.

> 1. Open the file to read (file_1)
> 1.a If file_1 in bad state, report and exit.

> 2. Counter = number of bytes to skip
> 3. While counter > 0 and not the end of file_1
> read file_1

You could also just use istream::ignore. (Otherwise, of course,
you'll also want to decrement counter by the number of bytes
you've read each time:-).)

> 3.a. If file_1 in bad state, report and exit.

> 4. Open the file to write (file_2)
> 4.a If file_2 in bad state, report and exit.

> 5. While not the end of file_1
> read file_1 into 'buffer'
> if successful, write 'buffer' to file_2
> if file_2 in bad state, report and exit.

If there's an error in writing the file, I generally like to
delete it before exiting, so that the corrupt version doesn't
accidentally get used.

--
James Kanze (GABI Software) email:james...@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

0 new messages