Generate a 10,000 line file really fast?
Outsider
--
<!-Outsider//->
MS-DOS 6.22, Windows for Workgroups 3.11, Netscape Communicator 4.08
Line Counting --a programmer's disease.
Joe Geluso
<nonvali...@yahoo.com> wrote:
>Can someone show how to generate a 10,000 line ASCII file really fast?
Not elegant, but runs quickly and can be done in a batch file:
echo a > 1.txt
copy 1.txt+1.txt 2.txt
copy 2.txt+2.txt 4.txt
copy 4.txt+4.txt 8.txt
copy 8.txt+8.txt 16.txt
copy 16.txt+16.txt 32.txt
copy 32.txt+32.txt 64.txt
copy 64.txt+64.txt 128.txt
copy 128.txt+128.txt 256.txt
copy 256.txt+256.txt 512.txt
copy 512.txt+512.txt 1024.txt
copy 1024.txt+1024.txt 2048.txt
copy 2048.txt+2048.txt 4096.txt
copy 4096.txt+4096.txt 8192.txt
rem If you need _exactly_ 10,000 lines:
copy 8192.txt+1024.txt+512.txt+256.txt+16.txt 10000.txt
rem If you need _at least_ 10,000 lines:
copy 8192.txt+8192.txt 16384.txt
HTH
Joe Geluso
Outsider
>
> On Fri, 28 Dec 2001 18:09:32 +0100, Outsider
> <nonvali...@yahoo.com> wrote:
>
> >Can someone show how to generate a 10,000 line ASCII file really fast?
>
> Not elegant, but runs quickly and can be done in a batch file:
>
> echo a > 1.txt
> copy 1.txt+1.txt 2.txt
> copy 2.txt+2.txt 4.txt
> copy 4.txt+4.txt 8.txt
...snipped
That's terrific :). I was thinking about a geometric progression too.
Beautifully put together - I like simplicity. It's much faster
than I, for some reason(?), had imagined.
I have got it to this basic outline so I can quickly adapt it to
whatever need arises.
:: 10000lin.bat
@ECHO off
echo 0 > %temp%.\1.tmp
copy/y %temp%.\1.tmp+%temp%.\1.tmp %temp%.\2.tmp>nul
copy/y %temp%.\2.tmp+%temp%.\2.tmp %temp%.\4.tmp>nul
copy/y %temp%.\4.tmp+%temp%.\4.tmp %temp%.\8.tmp>nul
copy/y %temp%.\8.tmp+%temp%.\8.tmp %temp%.\16.tmp>nul
copy/y %temp%.\16.tmp+%temp%.\16.tmp %temp%.\32.tmp>nul
copy/y %temp%.\32.tmp+%temp%.\32.tmp %temp%.\64.tmp>nul
copy/y %temp%.\64.tmp+%temp%.\64.tmp %temp%.\128.tmp>nul
copy/y %temp%.\128.tmp+%temp%.\128.tmp %temp%.\256.tmp>nul
copy/y %temp%.\256.tmp+%temp%.\256.tmp %temp%.\512.tmp>nul
copy/y %temp%.\512.tmp+%temp%.\512.tmp %temp%.\1024.tmp>nul
copy/y %temp%.\1024.tmp+%temp%.\1024.tmp %temp%.\2048.tmp>nul
copy/y %temp%.\2048.tmp+%temp%.\2048.tmp %temp%.\4096.tmp>nul
copy/y %temp%.\4096.tmp+%temp%.\4096.tmp %temp%.\8192.tmp>nul
rem If you need _exactly_ 10,000 lines:
copy/y %temp%.\8192.tmp+%temp%.\1024.tmp+%temp%.\512.tmp+%temp%.\256.tmp+%temp%.\16.tmp %temp%.\10000.tmp>nul
:: If you need _at least_ 10,000 lines:
:: copy/y %temp%.\8192.tmp+%temp%.\8192.tmp %temp%.\16384.tmp>nul
FOR %%f in (%temp%.\*.tmp) do if not %%f==%temp%.\10000.TMP DEL %%f
FIND/C "0" <%temp%.\10000.TMP>nul
IF errorlevel 1 ECHO. not enough space
DEL %temp%.\10000.TMP %optional%
::
I may split that one very long command line, but so far there were
no problems.
Dr John Stockton
news:alt.msdos.batch, Outsider <nonvali...@yahoo.com> wrote at Fri,
28 Dec 2001 18:09:32 :-
>Can someone show how to generate a 10,000 line ASCII file really fast?
Really quickly; fast is an adjective.
echo. | COLS !9999 > file
If you want non-empty lines, that's another matter; another pass of COLS
would do it.
A simple program in QBASIC or other language will do it, too. For the
greatest speed, compose the entire file in RAM and then write it,
preferably with something like Pascal's BlockWrite; Pascal is free,
links via below. There'll be a BlockWrite example in Timo's Pascal FAQ.
HK would compose the file in RAM and write it with ASM.
You do not say, though, whether the speed refers to the time between
initiation and completion, or does not start until writing actually
begins.
--
© John Stockton, Surrey, UK. j...@merlyn.demon.co.uk Turnpike v4.00 MIME. ©
Web <URL: http://www.merlyn.demon.co.uk/> - FAQqish topics, acronyms & links.
PAS EXE TXT ZIP via <URL: http://www.merlyn.demon.co.uk/programs/00index.htm>
My DOS <URL: http://www.merlyn.demon.co.uk/batfiles.htm> - also batprogs.htm.
Benny Pedersen
"Outsider" <nonvali...@yahoo.com> wrote news:3C2CE899...@yahoo.com...
Hi,
Here's my suggestion
@ECHO OFF
ECHO.>%temp%.\1
FOR %%f in (%temp%.\1 %temp%.\10 %temp%.\100 %temp%.\1000) do COPY/b/y %%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f %%f0>NUL
:: ----If you want line numbers
FIND ""/n/v<%temp%.\10000>%temp%.\Line_num.tmp
:: ----If you want to show the contents
COPY %temp%.\Line_num.tmp CON>NUL
FOR %%f in (1 10 100 1000 10000) do DEL %temp%.\%%f
DEL/p %temp%.\Line_num.tmp
Benny Pedersen,
http://users.cybercity.dk/~bse26236/batutil/help/INDEX.HTM#y
Outsider
>
...snipped
> Here's my suggestion
>
> @ECHO OFF
> ECHO.>%temp%.\1
> FOR %%f in (%temp%.\1 %temp%.\10 %temp%.\100 %temp%.\1000) do COPY/b/y %%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f %%f0>NUL
>
> :: ----If you want line numbers
> FIND ""/n/v<%temp%.\10000>%temp%.\Line_num.tmp
>
> :: ----If you want to show the contents
> COPY %temp%.\Line_num.tmp CON>NUL
>
> FOR %%f in (1 10 100 1000 10000) do DEL %temp%.\%%f
> DEL/p %temp%.\Line_num.tmp
>
That is _very_ interesting too, Benny.
Ok, I had to make the %temp% current to do it in the %temp%
directory because my system balked at the long command line.
:: 10000lin2.bat
@ECHO OFF
ECHO @PROMPT $N:$_CD $P> %temp%.\setback1.bat
%COMSPEC%/E:2048/C%temp%.\setback1.bat> %temp%.\goback.bat
DEL %temp%.\setback1.bat
FOR %%c in (%temp%.\ CD) do %%c %temp%.
ECHO.>1
FOR %%f in (1 10 100 1000) do COPY/b/y %%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f %%f0>NUL
:: ----If you want line numbers
:::"FIND ""/n/v<10000>Line_num.tmp:::"
:: ----If you want to show the contents
:::"COPY Line_num.tmp CON>NUL:::"
FOR %%f in (1 10 100 1000) do DEL %%f
::DEL/p Line_num.tmp
FOR %%c in (CALL DEL) do %%c %temp%.\goback.bat
::
Here is the technique by Joe Geluso which I adjusted to also output only
a CR/LF for comparison purposes. I commented out unneeded lines in both
batches so they would not affect the runtime results.
:: 10000lin.bat
@ECHO off
echo.>%temp%.\1.tmp
copy/y/b %temp%.\1.tmp+%temp%.\1.tmp %temp%.\2.tmp>nul
copy/y/b %temp%.\2.tmp+%temp%.\2.tmp %temp%.\4.tmp>nul
copy/y/b %temp%.\4.tmp+%temp%.\4.tmp %temp%.\8.tmp>nul
copy/y/b %temp%.\8.tmp+%temp%.\8.tmp %temp%.\16.tmp>nul
copy/y/b %temp%.\16.tmp+%temp%.\16.tmp %temp%.\32.tmp>nul
copy/y/b %temp%.\32.tmp+%temp%.\32.tmp %temp%.\64.tmp>nul
copy/y/b %temp%.\64.tmp+%temp%.\64.tmp %temp%.\128.tmp>nul
copy/y/b %temp%.\128.tmp+%temp%.\128.tmp %temp%.\256.tmp>nul
copy/y/b %temp%.\256.tmp+%temp%.\256.tmp %temp%.\512.tmp>nul
copy/y/b %temp%.\512.tmp+%temp%.\512.tmp %temp%.\1024.tmp>nul
copy/y/b %temp%.\1024.tmp+%temp%.\1024.tmp %temp%.\2048.tmp>nul
copy/y/b %temp%.\2048.tmp+%temp%.\2048.tmp %temp%.\4096.tmp>nul
copy/y/b %temp%.\4096.tmp+%temp%.\4096.tmp %temp%.\8192.tmp>nul
rem If you need _exactly_ 10,000 lines:
copy/y/b %temp%.\8192.tmp+%temp%.\1024.tmp+%temp%.\512.tmp+%temp%.\256.tmp+%temp%.\16.tmp %temp%.\10000.tmp>nul
:: If you need _at least_ 10,000 lines:
:::"copy/y/b %temp%.\8192.tmp+%temp%.\8192.tmp %temp%.\16384.tmp>nul:::"
:::"FIND/C/V "" <%temp%.\10000.TMP >%temp%.\find.tmp:::"
:::"FIND "10000" <%temp%.\find.tmp> nul:::"
::IF not errorlevel 1 ECHO. %temp%.\10000lin.tmp created
::IF errorlevel 1 ECHO. not enough space
FOR %%f in (%temp%.\*.tmp) do if not %%f==%temp%.\10000.TMP DEL %%f
::DEL %temp%.\10000.TMP
10000 20.000 29-12-01 12.35 % 10000li2.bat %
10000 TMP 20.000 29-12-01 12.35 % 10000lin.bat%
Runtime comparison.
10000li2.bat
0.046801
0.101811
0.129071
0.129239
0.046837
0.046697
0.101375
0.101749
0.101292
0.073956
10000lin.bat
0.183023
0.180435
0.153354
0.180179
0.152729
0.180075
0.180187
0.125749
0.180704
0.180362
10000li2.bat is roughly about twice as fast as 10000lin.bat. This
difference, however, would be _completely_ negligible in at least
99.97% of all cases (according to my best guestimate).
Great stuff!
Thanks Benny and Joe :))
Outsider
>
> JRS: In article <3C2CA74C...@yahoo.com>, seen in
> news:alt.msdos.batch, Outsider <nonvali...@yahoo.com> wrote at Fri,
> 28 Dec 2001 18:09:32 :-
> >Can someone show how to generate a 10,000 line ASCII file really fast?
>
> Really quickly; fast is an adjective.
Thanks for the most welcome correction. I'm afraid I never learned
many rules of grammar.
> echo. | COLS !9999 > file
>
> If you want non-empty lines, that's another matter; another pass of COLS
> would do it.
Yes. I forgot to mention that I'd rather prefer to do it with native DOS
and batch language.
> A simple program in QBASIC or other language will do it, too. For the
> greatest speed, compose the entire file in RAM and then write it,
Yes, but...
1) I don't know qbasic.
2) I'd rather use internal commands whenever possible.
3) Hate that screen flash <g>.
> preferably with something like Pascal's BlockWrite; Pascal is free,
> links via below. There'll be a BlockWrite example in Timo's Pascal FAQ.
I don't know Pascal either, and as I say, I'd rather prefer to do it with
native DOS and batch language. After all, I am a DOS batch file programmer.
It wouldn't make much sense for me to learn another language to solve
something in a batch, and if I should choose to learn a language it would
probably be a DOS based AWK. IMHO, It makes more sense to me to use a
scripting language with batch than write dedicated executables, although
they may be a reasonable approach for non batch file programmers or
computer newbies.
> HK would compose the file in RAM and write it with ASM.
It would be cool in ASCII assembler. Debug is also ok.
My perspective is:
1) Internal commands first
2) Native external commands or ASCII assembler or any combination of 1 & 2.
3) Almost any sort of work-around or "kludge" to avoid number 4
4) Non-native extra utilities
5) Learn another programming language
> You do not say, though, whether the speed refers to the time between
> initiation and completion, or does not start until writing actually
> begins.
Well, the entire time from initiation to completion is what is ultimately
important. Your suggestions are appreciated, but nonetheless, I think that
for _my_ purposes I am satisfied by the suggestions given by Benny and Joe.
tom
> Can someone show how to generate a 10,000 line ASCII file really fast?
Yeah. There are TWO ways: Send a post informing that you made some
improper/off-topic post.
And, of course, post something which should be in NT.
Benny Pedersen
"Outsider" <nonvali...@yahoo.com> wrote news:3C2DBE1A...@yahoo.com...
> Benny Pedersen wrote:
> >
>
> ...snipped
>
> > Here's my suggestion
> >
> > @ECHO OFF
> > ECHO.>%temp%.\1
> > FOR %%f in (%temp%.\1 %temp%.\10 %temp%.\100 %temp%.\1000) do COPY/b/y %%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f %%f0>NUL
> >
> > :: ----If you want line numbers
> > FIND ""/n/v<%temp%.\10000>%temp%.\Line_num.tmp
> >
> > :: ----If you want to show the contents
> > COPY %temp%.\Line_num.tmp CON>NUL
> >
> > FOR %%f in (1 10 100 1000 10000) do DEL %temp%.\%%f
> > DEL/p %temp%.\Line_num.tmp
> >
>
>
> That is _very_ interesting too, Benny.
>
> Ok, I had to make the %temp% current to do it in the %temp%
> directory because my system balked at the long command line.
>
> :: 10000lin2.bat
> @ECHO OFF
> ECHO @PROMPT $N:$_CD $P> %temp%.\setback1.bat
> %COMSPEC%/E:2048/C%temp%.\setback1.bat> %temp%.\goback.bat
> DEL %temp%.\setback1.bat
> FOR %%c in (%temp%.\ CD) do %%c %temp%.
> ECHO.>1
> FOR %%f in (1 10 100 1000) do COPY/b/y %%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f+%%f %%f0>NUL
About the goback.bat:
I tried to find a better way, so, I tried an
"EXIT COMMAND /k PROMPT" technique but it's using
a pipe, so, I tried with a "%COMSPEC% /c" but then
I just got another pipe instead:
@ECHO OFF
%COMSPEC%/cFOR %%c in (1 2) do PROMPT $N:$_CD,$P$_|find " "/v>%temp%.\goback.bat
FOR %%c in (CAL DE) do %%cL %temp%.\goback.bat
Hmmm, what about this new method, (no pipes:)
ECHO.:EXIT>%temp%.\goback.bat
%COMSPEC%<%temp%.\goback.bat/kPROMPT $N:$_CD $P$_>>%temp%.\goback.bat
FOR %%c in (CAL DE) do %%cL %temp%.\goback.bat
I found it interesting that the above EXIT command works with a
prefixed colon.
Benny,
PS. The old one:
ECHO EXIT|%COMSPEC%/k ... $_:>>%temp%.\goback.bat
is one line less than the new method but it's
using a pipe.
I havn't time measured.
Outsider
>
...snipped
> About the goback.bat:
>
> I tried to find a better way, so, I tried an
> "EXIT COMMAND /k PROMPT" technique but it's using
> a pipe, so, I tried with a "%COMSPEC% /c" but then
> I just got another pipe instead:
>
> @ECHO OFF
> %COMSPEC%/cFOR %%c in (1 2) do PROMPT $N:$_CD,$P$_|find " "/v>%temp%.\goback.bat
> FOR %%c in (CAL DE) do %%cL %temp%.\goback.bat
>
> Hmmm, what about this new method, (no pipes:)
>
> ECHO.:EXIT>%temp%.\goback.bat
> %COMSPEC%<%temp%.\goback.bat/kPROMPT $N:$_CD $P$_>>%temp%.\goback.bat
> FOR %%c in (CAL DE) do %%cL %temp%.\goback.bat
That reminded me of the documentation provided with send.com, the great
557 byte utility from 1985. I am inserting the relevant portion:
If you use SEND for a while, you will wish you could send its
output to the command line rather than just to the screen. That is
typing "send ren myfile file$M$D" on December 15, will echo "ren
myfile fill1215" on the screen but will not issue that command to
DOS. One can use DOS's pipe facility to issue such commands to DOS
but some care is needed. If not done exactly right, your system
will hang although the only effect will be the need to reboot.
One can make a new "shell" of DOS by issuing the command
"command" if the DOS file COMMAND.COM is in the current directory
or in your DOS search path. This new shell remains resident until
the command "exit" is issued.
To explain the care needed consider some examples. Unless
you like rebooting, don't actually try these examples out! If you
type "send cls|command" at the DOS command line, the letters "cls"
will appear on your screen after the copyright for the appropriate
version of DOS. However, when you try to push the <Enter> key
nothing will happen. You told the new command.com to take its
input from the output of SEND and it stubbornly won't take any from
you; the only keystroke to which it will respond is the one
requiring three fingers! So you might try, "send cls$_|command".
This will work in the sense that your screen will clear but after
that, the system will hang. The correct thing to do is "send
cls$_exit$_|command". This will clear the screen and unload the
new shell of command.com returning control to you. So the moral is
whenever piping send's output through COMMAND.COM you should be
sure that the end of the string in "send 'string'|command" is
"$_exit$_"
Of course, you'd never use this redirection to clear the
screen. Here are two possible uses. You might make a batch file
called return.bat with the single line:
send %1 %2$_$n:$_cd $p$_exit$_|command
Thus, if you have a batch file lotus.bat which switches to the
directory with 123 and runs it, typing "return lotus" will run
lotus.bat and then return you to the drive and directory that you
started on. You needn't worry that having told COMMAND.COM to only
accept input from the output of SEND, your keyboard will be locked
up when you try to communicate with 123. COMMAND.COM will not
accept input from the keyboard, but it has loaded 123 which is
configured to take input from the keyboard. However, if you run a
program with return.bat that tries to issue commands to DOS, it may
not work. Mind you, there will be no problem with the program
using DOS services as it might in saving a file.
Here is another use. Suppose that your communication program
captures input into a file named "temp" by default. You want to
make a batch file which collects your electronic mail and stores it
in a file whose name is stamped with today's date. Include the
line
send ren temp mail$M$D.$Y $_exit$_|command
in the file and things will work exactly as you wish.
> I found it interesting that the above EXIT command works with a
> prefixed colon.
Yes.
> Benny,
> PS. The old one:
> ECHO EXIT|%COMSPEC%/k ... $_:>>%temp%.\goback.bat
> is one line less than the new method but it's
> using a pipe.
>
> I havn't time measured.
It is _very_ difficult to measure time with such fast batches.
Near as I can tell, these two versions have virtually the same
run time.
1)
@ECHO off
ECHO @PROMPT $N:$_CD $P> %temp%.\setback1.bat
%COMSPEC%/E:2048/C%temp%.\setback1.bat> %temp%.\goback.bat
DEL %temp%.\setback1.bat
FOR %%c in (CALL DEL) do %temp%.\goback.bat
2)
@ECHO off
ECHO.:EXIT>%temp%.\goback.bat
%COMSPEC%<%temp%.\goback.bat/kPROMPT $N:$_CD $P$_>>%temp%.\goback.bat
FOR %%c in (CAL DE) do %%cL %temp%.\goback.bat
Line counting (or byte counting) serves no practical purpose at all.
Version 2 is 26 bytes less than version 1 (big deal), but has the
serious drawback of adding an EXIT to the command history! Windows
only users don't know what this means, but for command line users like
myself, this IS a serious drawback; undesirable and indeed unwarranted.
The only situation in modern computing I can imagine where 26 bytes make
a difference is on a fully loaded floppy boot disk where 26 bytes will
increase a hypothetical file's size over 512 bytes (or otherwise run over
to the next cluster) so it won't fit on the floppy. It is a very very
rare situation. The above files' sizes are, 189 bytes and 163 bytes.
Todd Vargo
"tom" <ze...@yahoo.com> wrote in message
news:9c4f0f2c.01122...@posting.google.com...
> Outsider <nonvali...@yahoo.com> wrote in message
news:<3C2CA74C...@yahoo.com>...
> > Can someone show how to generate a 10,000 line ASCII file really fast?
>
> Yeah. There are TWO ways: Send a post informing that you made some
> improper/off-topic post.
Huh? I see an intellectual BATCH discussion.
Apparently it's over your head. :(
> And, of course, post something which should be in NT.
Sorry, wrong group for that!
Please stop this nonsense already. :(
--
Todd Vargo (body of message must contain my name to reply by email)
tom
> "tom" <ze...@yahoo.com> wrote in message
> news:9c4f0f2c.01122...@posting.google.com...
> > Outsider <nonvali...@yahoo.com> wrote in message
> news:<3C2CA74C...@yahoo.com>...
> > > Can someone show how to generate a 10,000 line ASCII file really fast?
> >
> > Yeah. There are TWO ways: Send a post informing that you made some
> > improper/off-topic post.
>
> Huh? I see an intellectual BATCH discussion.
> Apparently it's over your head. :(
Did I write _anything_ which hints at the possibility that
Outsider's question was less than a valid question evoking
intellectual discussion?? SHAME ON YOU!!! I won't attempt
to speak for others but I sit around all day and ponder how
to generate 10,000 line ASCII files, really fast. I thought
my two solutions were valid answers. I see that I will have to
prove my point at the next instance the Chief Czar Poombah
posts OT.
You know who that is, don't you? He's the one who published
results obtained from Ixquick to promote his self-importance.
Yeah, Ixquick. The outfit which claims that it "was created
by a 12 year old Himalayan mountain goat named Daphne."
>
> > And, of course, post something which should be in NT.
>
> Sorry, wrong group for that!
Sorry, but I think you are wrong, and paranoid. It seems
that the mere mention of NT sends you Poombahs into a dither.
> Please stop this nonsense already. :(
YOU first, please.
Charles Dye
>I won't attempt
>to speak for others but I sit around all day and ponder how
>to generate 10,000 line ASCII files, really fast. I thought
>my two solutions were valid answers.
And as Todd has thoughtfully provided supporting evidence
that your methods would in fact work, perhaps we can take
your point as proved -- without actually generating all
ten thousand?
--
Charles Dye ras...@highfiber.com
tom
OK. But I'm keeping it ready; Czar Poombah has already
threatened to post his FAQ 2-3 times per day, or even 2-3
times per hour if _he_ deems it necessary, so...
Todd Vargo
> Did I write _anything_ which hints at the possibility that
> Outsider's question was less than a valid question evoking
> intellectual discussion??
Duh!
> SHAME ON YOU!!! I won't attempt
> to speak for others but I sit around all day and ponder how
> to generate 10,000 line ASCII files, really fast.
What practical purpose it would serve is irrelevant. As long as the
discussion remained on topic (which the other posters were) _is_. But
actually, you arr the one that claimed it off topic, not me.
So, "SHAME ON YOU!!!"
> I thought
> my two solutions were valid answers. I see that I will have to
> prove my point at the next instance the Chief Czar Poombah
> posts OT.
And once again, your replies only serve to generate more off topic posts
which you claim to be against. Hypocrite!
>
> You know who that is, don't you? He's the one who published
> results obtained from Ixquick to promote his self-importance.
> Yeah, Ixquick. The outfit which claims that it "was created
> by a 12 year old Himalayan mountain goat named Daphne."
OFF-TOPIC! That crosspost (which this group was removed from) has nothing to
do with _this_ discussion in _this_ group. Get over it!
> >
> > > And, of course, post something which should be in NT.
> >
> > Sorry, wrong group for that!
>
> Sorry, but I think you are wrong, and paranoid. It seems
> that the mere mention of NT sends you Poombahs into a dither.
No, not paranoid. Just sick of this nonsense.
>
> > Please stop this nonsense already. :(
>
> YOU first, please.
No problem. You stop playing keystone cop, and I wont comment about it.
laura fairhead
>Can someone show how to generate a 10,000 line ASCII file really fast?
>
@ECHO OFF>%TEMP%.\@@
FOR %%_ IN (F100L4E20''D''A RCX 4E20 NFILE W Q) DO ECHO %%_>>%TEMP%.\@@
DEBUG <%TEMP%.\@@ >NUL
DEL %TEMP%.\@@
byefrom
--
: ${L:-aura} # http://lf.1accesshost.com
: # http://lf.8k.com:80
Outsider
>
> On Fri, 28 Dec 2001 18:09:32 +0100, Outsider <nonvali...@yahoo.com> wrote:
>
> >Can someone show how to generate a 10,000 line ASCII file really fast?
> >
>
> @ECHO OFF>%TEMP%.\@@
> FOR %%_ IN (F100L4E20''D''A RCX 4E20 NFILE W Q) DO ECHO %%_>>%TEMP%.\@@
> DEBUG <%TEMP%.\@@ >NUL
> DEL %TEMP%.\@@
>
Thanks Laura, that's quite fast, in fact it appears to be the
fastest so far.
3599.586603
0.071866
0.044401
0.016946
0.071938
0.044437
0.099338
3599.586561
0.016885
3599.586613
3599.586551
0.044392
0.071833
0.044404
I don't have any idea what causes those strange readings in-between,
but I have seen the same sort of readings with at least two other
timer programs.
tom
> "tom" <ze...@yahoo.com> wrote in message
> news:9c4f0f2c.01123...@posting.google.com...
>
> > Did I write _anything_ which hints at the possibility that
> > Outsider's question was less than a valid question evoking
> > intellectual discussion??
>
> Duh!
>
> > SHAME ON YOU!!! I won't attempt
> > to speak for others but I sit around all day and ponder how
> > to generate 10,000 line ASCII files, really fast.
>
> What practical purpose it would serve is irrelevant. As long as the
> discussion remained on topic (which the other posters were) _is_. But
> actually, you arr the one that claimed it off topic, not me.
You can't read at 5th grade level yet, can you?
> So, "SHAME ON YOU!!!"
No, SHAME ON YOU!
>
> > I thought
> > my two solutions were valid answers. I see that I will have to
> > prove my point at the next instance the Chief Czar Poombah
> > posts OT.
>
> And once again, your replies only serve to generate more off topic posts
> which you claim to be against. Hypocrite!
>
Geeez. You get dumber by the friggin' minute! What practical purpose
my
posts may serve are irrelevant, and since you say that's good enough
for
you, it is good enough for me.
You also claim in the above that "As long as the discussion remained
on topic... _is_." Well, you got me on that one, I think.
> > You know who that is, don't you? He's the one who published
> > results obtained from Ixquick to promote his self-importance.
> > Yeah, Ixquick. The outfit which claims that it "was created
> > by a 12 year old Himalayan mountain goat named Daphne."
>
> OFF-TOPIC! That crosspost (which this group was removed from) has nothing to
> do with _this_ discussion in _this_ group. Get over it!
>
You mean ALL fifteen (15) posts initiated by OUTSIDER concerning the
subject were OFF-TOPIC?? Have you written HIM about HIS
transgressions?
Or are you claiming High and Mighty Poombah privileges again?
> > > > And, of course, post something which should be in NT.
> > >
> > > Sorry, wrong group for that!
> >
> > Sorry, but I think you are wrong, and paranoid. It seems
> > that the mere mention of NT sends you Poombahs into a dither.
>
> No, not paranoid. Just sick of this nonsense.
>
I'm glad.
> > > Please stop this nonsense already. :(
> > YOU first, please.
>
> No problem. You stop playing keystone cop, and I wont comment about it.
Well, you know where you can stick that, don't you?
Happy New Year!
Outsider
>
> JRS: In article <3C2CA74C...@yahoo.com>, seen in
> news:alt.msdos.batch, Outsider <nonvali...@yahoo.com> wrote at Fri,
> 28 Dec 2001 18:09:32 :-
> >Can someone show how to generate a 10,000 line ASCII file really fast?
>
> Really quickly; fast is an adjective.
For the record,
I should never have accepted your correction without self verification.
Your correction is indeed erroneous, and you should never have made it.
Of course my statement was grammatically correct.
Rik D'haveloose
> laura fairhead wrote:
> > @ECHO OFF>%TEMP%.\@@
> > FOR %%_ IN (F100L4E20''D''A RCX 4E20 NFILE W Q) DO ECHO
%%_>>%TEMP%.\@@
> 3599.586603
==8<
> 3599.586561
==8<
> 3599.586613
> 3599.586551
==8<
> I don't have any idea what causes those strange readings in-between,
> but I have seen the same sort of readings with at least two other
> timer programs.
? ? From here that seems to suggest something wrong with the timer used.
I can hardly believe that you actually did run this really (as it takes
in total some 4 hours ~ 3600 secs), and some other odd behaviour of your
system seems unlikely to me (debug taking such time ? to load ??)
mcahny
--
TUF Greetings from Rumbeke
Belgium
Outsider
to be a relatively common occurance when trying to time a very fast
program, though.
Dr John Stockton
news:alt.msdos.batch, Outsider <nonvali...@yahoo.com> wrote at Tue,
1 Jan 2002 21:45:40 :-
>3599.586603
>0.071866
>0.044401
>0.016946
>0.071938
>0.044437
>0.099338
>3599.586561
>0.016885
>3599.586613
>3599.586551
>0.044392
>0.071833
>0.044404
>
>I don't have any idea what causes those strange readings in-between,
>but I have seen the same sort of readings with at least two other
>timer programs.
The error is about 3599.55 seconds, or one hour minus 0.45 seconds.
Multiply by 24, giving 10.8 seconds., which would be 196 ticks of the
DOS clock. The DOS clock does $1800B0 ticks per day, which is 24*65536
+ ($B0 = 176) ticks. 196-176 is, within uncertainty, 24 or one per hour.
Perhaps the DOS clock is being read non-atomically, and occasionally the
reading of the lower and upper words are separated by an updating
interrupt, at a stage where part of the number is resident only in the
carry (DOS being written for a 16-bit machine), or vice versa.
Perhaps the lower word only is being read, and the absolute value of the
difference is being used.
It's odd, though, if the large values are seen frequently.
You really should read The Timing FAQ, and my page pas-time.htm. Such
coding errors are sufficiently easy to avoid. They are analogous to
misreading Date+Time as midnight passes.
--
© John Stockton, Surrey, UK. j...@merlyn.demon.co.uk Turnpike v4.00 MIME. ©
Web <URL: http://www.merlyn.demon.co.uk/> - w. FAQish topics, links, acronyms
PAS, EXE in <URL: http://www.merlyn.demon.co.uk/programs/> - see 00index.txt.
Dates - miscdate.htm moredate.htm js-dates.htm pas-time.htm critdate.htm &c.
Todd Vargo
> Rik D'haveloose wrote:
> > ? ? From here that seems to suggest something wrong with the timer used.
> > I can hardly believe that you actually did run this really (as it takes
> > in total some 4 hours ~ 3600 secs), and some other odd behaviour of your
> > system seems unlikely to me (debug taking such time ? to load ??)
>
> <LOL> No, that was certainly not the correct runtime. It does seem
> to be a relatively common occurance when trying to time a very fast
> program, though.
Hence the reason I suggested using a slow down method for fast batch
comparisons. You could try using a different timer method. How about qbasic?
Too flashy even for these occasional time comparisons? (Sorry, couldn't
resist.<g>)
:: BATTIMER.BAT
:: Requires qbasic in PATH.
@echo off
if '==%1' echo Usage: %0 Batchname [args]
if '==%1' goto end
echo start = timer>%temp%.\$.bas
echo shell "%1 %2 %3 %4 %5 %6 %7 %8 %9">>%temp%.\$.bas
echo print using "###.######"; timer - start; " seconds">>%temp%.\$.bas
echo system>>%temp%.\$.bas
qbasic /run %temp%.\$.bas
del %temp%.\$.bas
:end
Outsider
>
...snipped
> comparisons.
Yeah, too bad I don't have a "turbo" button on my current box :(.
Your're right, the relative times will be just as relative which
is fine for comparison purposes as long as the timer is precise
enough.
> You could try using a different timer method. How about qbasic?
> Too flashy even for these occasional time comparisons? (Sorry, couldn't
> resist.<g>)
>
> :: BATTIMER.BAT
> :: Requires qbasic in PATH.
> @echo off
> if '==%1' echo Usage: %0 Batchname [args]
> if '==%1' goto end
> echo start = timer>%temp%.\$.bas
> echo shell "%1 %2 %3 %4 %5 %6 %7 %8 %9">>%temp%.\$.bas
> echo print using "###.######"; timer - start; " seconds">>%temp%.\$.bas
> echo system>>%temp%.\$.bas
> qbasic /run %temp%.\$.bas
> del %temp%.\$.bas
> :end
I'll try anything as far as timers go (I just put on a pair of sunglasses).
X:\TEMP>batimer c:\bat\parstime.BAT
Out of environment space
Bad command or file name
0.111328
X:\TEMP>env
Environment size is 2048 bytes
1562 bytes free in environment
$.bas contains:
start = TIMER
SHELL "c:\bat\parstime.BAT "
PRINT USING "###.######"; TIMER - start; " seconds"
SYSTEM
The error message is:
+-----------------------+
¦ ¦
¦ Type mismatch ¦
¦ ¦
+-----------------------¦
¦ < OK > < Help > ¦
+-----------------------+
I suspect there is a problem somewhere.
Outsider
>
...snipped
> The error is about 3599.55 seconds, or one hour minus 0.45 seconds.
> Multiply by 24, giving 10.8 seconds., which would be 196 ticks of the
> DOS clock. The DOS clock does $1800B0 ticks per day, which is 24*65536
> + ($B0 = 176) ticks. 196-176 is, within uncertainty, 24 or one per hour.
>
> Perhaps the DOS clock is being read non-atomically, and occasionally the
> reading of the lower and upper words are separated by an updating
> interrupt, at a stage where part of the number is resident only in the
> carry (DOS being written for a 16-bit machine), or vice versa.
>
> Perhaps the lower word only is being read, and the absolute value of the
> difference is being used.
>
> It's odd, though, if the large values are seen frequently.
>
> You really should read The Timing FAQ, and my page pas-time.htm. Such
> coding errors are sufficiently easy to avoid. They are analogous to
> misreading Date+Time as midnight passes.
>
Looks interesting alright, but I currently under tremendous time
pressure and unable to take on such a major project (like learning
Pascal) in order to try and follow the coding examples.
Todd Vargo
Oops<grin>. I forgot to mention... _Untested_ as posted. My bad.
The "Out of environment space" error could be related to qbasic shelling
back to DOS to run your batch. I assume the same rules that apply to
"command/c" without using /e:xxxx likely apply here. This can be fixed by
adding command/e:2048/c to my "shell" command.
The "Bad command or file name" error could possibly be caused by the
trailing spaces in the shell command under 6.22, but I don't know this for
sure. If it is, this is easily fixed by removing the spaces.
The "Type mismatch" error was a syntax error on my part. I forced two
"print" commands on to a single line for vertical compactness before posting
(I know, dumb move). Normally this would be ok except that I used BASIC's
"using" statement to format the output of the number. It prevents it from
possibly displaying numbers in scientific notation, but mainly I used it do
display the exact number of digits that I wanted returned. The "Type
mismatch" is actually caused by the trailing text on the same line.
Try it again with the following "patched" batch. Of course, this has the
same overhead of qbasic running in the background (190K), but that obstacle
can also easily be programmatically over come in the batch if need be (the
idea here was to make a simple batch timer).
:: BATTIMER.BAT
:: Requires qbasic in PATH.
@echo off
if '==%1' echo Usage: %0 Batchname [args]
if '==%1' goto end
echo start = timer>%temp%.\$.bas
echo cmd$ = rtrim$("%1 %2 %3 %4 %5 %6 %7 %8 %9")>>%temp%.\$.bas
echo shell "command/e:2048/c" + cmd$ >>%temp%.\$.bas
echo print using "###.######"; timer - start;>>%temp%.\$.bas
echo print " seconds">>%temp%.\$.bas
echo system>>%temp%.\$.bas
qbasic /run %temp%.\$.bas
del %temp%.\$.bas
:end
--
Todd Vargo (body of message must contain my name to reply by email)
PS. This was not an in-depth qbasic discussion. It was diagnostic
explanation for why the previous batch failed.
Todd Vargo
> Dr John Stockton wrote:
> > You really should read The Timing FAQ, and my page pas-time.htm. Such
> > coding errors are sufficiently easy to avoid. They are analogous to
> > misreading Date+Time as midnight passes.
Outsider,
I forgot to mention this in my other post. No effort was taken to compensate
for the midnight issue. Hopfully, you don't need to test a batch that runs
into the next day.
Outsider
>
> "Outsider" <nonvali...@yahoo.com> wrote in message
> news:3C394563...@yahoo.com...
> > Dr John Stockton wrote:
> > > You really should read The Timing FAQ, and my page pas-time.htm. Such
> > > coding errors are sufficiently easy to avoid. They are analogous to
> > > misreading Date+Time as midnight passes.
>
> Outsider,
>
> I forgot to mention this in my other post. No effort was taken to compensate
> for the midnight issue. Hopfully, you don't need to test a batch that runs
> into the next day.
I'm usually not up at that time, but thanks for the reminder.
Anyway, it is very good.
0.220703 seconds
0.220635
The first measurement is your battimer (on an arbitrary batch file).
The second measurement is a TSR timer. The small difference is
probably the qbasic.exe overhead. Nice batch timer :).
Dr John Stockton
news:alt.msdos.batch, Todd Vargo <t...@birdlover.com> wrote at Mon, 7 Jan
2002 20:50:21 :-
>
>"Outsider" <nonvali...@yahoo.com> wrote in message
>news:3C394563...@yahoo.com...
>> Dr John Stockton wrote:
>> > You really should read The Timing FAQ, and my page pas-time.htm. Such
>> > coding errors are sufficiently easy to avoid. They are analogous to
>> > misreading Date+Time as midnight passes.
>
>Outsider,
>
>I forgot to mention this in my other post. No effort was taken to compensate
>for the midnight issue. Hopfully, you don't need to test a batch that runs
>into the next day.
"analogous". If $40:$6C is read as two words rather than as one dword,
the point of danger occurs 25 times a day; every 65536 ticks, and at
midnight.
Todd Vargo
"Dr John Stockton" <sp...@merlyn.demon.co.uk> wrote
> "analogous". If $40:$6C is read as two words rather than as one dword,
> the point of danger occurs 25 times a day; every 65536 ticks, and at
> midnight.
Interesting quirk. Qbasic only has the midnight rollover (back to zero) to
deal with.
Todd Vargo
"Outsider" <nonvali...@yahoo.com> wrote
>
> 0.220703 seconds
> 0.220635
>
> The first measurement is your battimer (on an arbitrary batch file).
> The second measurement is a TSR timer. The small difference is
> probably the qbasic.exe overhead. Nice batch timer :).
The time difference is; the time it takes DOS to load the child command.com
instance compared to a TSR already in memory. Considering what I know of
your ramdrive usage, the time difference likely represents the amount of
time it takes to copy command.com from ramdrive into the working area of
conventional memory.
BTW, my meaning of "overhead" was about qbasic's foot print in memory. It's
mere presence in memory has (should have) no effect on the timing of the
batch.
Dr John Stockton
news:alt.msdos.batch, Todd Vargo <t...@birdlover.com> wrote at Tue, 8 Jan
2002 21:03:21 :-
>
>"Dr John Stockton" <sp...@merlyn.demon.co.uk> wrote
>
>> "analogous". If $40:$6C is read as two words rather than as one dword,
>> the point of danger occurs 25 times a day; every 65536 ticks, and at
>> midnight.
>
>Interesting quirk. Qbasic only has the midnight rollover (back to zero) to
>deal with.
Are you sure? If QBasic is written to run on sufficiently old machines,
it may not be able to perform 32-bit access in a single instruction.
How does one access that location in QBasic?
I can only see PEEK, which gives access to a byte; using that means that
errors of 256/18.2 = 14 seconds should be available at 14 second
intervals, and of one hour hourly.
Todd Vargo
"Dr John Stockton" <sp...@merlyn.demon.co.uk> wrote in message
news:NLeP+ZBv...@merlyn.demon.co.uk...
> JRS: In article <a1g8k5$p9k7f$1...@ID-25025.news.dfncis.de>, seen in
> news:alt.msdos.batch, Todd Vargo <t...@birdlover.com> wrote at Tue, 8 Jan
> 2002 21:03:21 :-
> >
> >"Dr John Stockton" <sp...@merlyn.demon.co.uk> wrote
> >
> >> "analogous". If $40:$6C is read as two words rather than as one dword,
> >> the point of danger occurs 25 times a day; every 65536 ticks, and at
> >> midnight.
> >
> >Interesting quirk. Qbasic only has the midnight rollover (back to zero)
to
> >deal with.
>
> Are you sure? If QBasic is written to run on sufficiently old machines,
> it may not be able to perform 32-bit access in a single instruction.
> How does one access that location in QBasic?
>
> I can only see PEEK, which gives access to a byte; using that means that
> errors of 256/18.2 = 14 seconds should be available at 14 second
> intervals, and of one hour hourly.
I guess I'm sure.(?) Qbasic's TIMER function (which is what I was referring
to) returns a value between 0 and 86400. Qbasic handles the low level
details for you. I guess one could PEEK the time from memory manually, but I
don't see why since the BASIC language provides the TIMER function to handle
this.
I think we've strayed a bit too far from batch though.
Rik D'haveloose
> Dr John Stockton wrote
> > >Interesting quirk. Qbasic only has the midnight rollover (back to
zero)
>
referring
> to) returns a value between 0 and 86400. Qbasic handles the low level
i fail to see how a per-second timer (24 x 60 x 60=86400) can 'time'
things in hundreds of seconds ?
(results which Outsider published here...??) (unless the process is
running at least 100 times)
> I think we've strayed a bit too far from batch though.
batch is universal :-)) (or ;-(( ??
Todd Vargo
"Rik D'haveloose" <brol.d...@xs4all.be> wrote in message
news:a1s233$qpp$1...@news1.xs4all.nl...
> Todd Vargo wrote
> > Dr John Stockton wrote
> > > >Interesting quirk. Qbasic only has the midnight rollover (back to
> zero)
> >
> > I guess I'm sure.(?) Qbasic's TIMER function (which is what I was
> referring
> > to) returns a value between 0 and 86400. Qbasic handles the low level
>
> i fail to see how a per-second timer (24 x 60 x 60=86400) can 'time'
> things in hundreds of seconds ?
Where is it docummented that qbasic's TIMER function returns it's value as
an integer. If you run PRINT TIMER, you'll notice that it displays a 2 place
decimal, thus indicating the value is not an integer or "per-second" value
as you may have believed. Wether it's a single or double precision value is
unknown to me. Knowing that undefined variables in qbasic default to single
precision, I would assume TIMER returns as single.
> (results which Outsider published here...??) (unless the process is
> running at least 100 times)
Hmm, I'm not sure what you mean here. It's not important to me so don't
sweat it.
>
> > I think we've strayed a bit too far from batch though.
>
> batch is universal :-)) (or ;-(( ??
Yes, but this has become an in-depth _qbasic_ discussion.
(something that I feared when posting the batch)