APPEND FROM adds only 1 record. . .

[Guillermo Varona Silupú]

Hi:
I have the following code:
---agenret2.prg---------------------------------------
PROCEDURE main
LOCAL aStru := { { "RUC" , "C", 11, 0 }, ;
{ "Nom_RzSo", "C", 100, 0 }, ;
{ "Desde_El", "C", 10, 0 }, ;
{ "Resoluci", "C", 050, 0 } }
LOCAL cFileTXT := "AgenRet_TXT.txt"
LOCAL cFileDBF := "AgenRet_.dbf"
CLS
dbCreate( cFileDBF, aStru )
USE ( cFileDBF )
APPEND FROM ( cFileTXT ) DELIMITED WITH PIPE
APPEND FROM ( cFileTXT ) DELIMITED WITH ( { "|","|" } )
list
wait
RETURN
------------------------------------------

. . . and the following TXT file:

--- AgenRet_TXT.txt ---------------------------------------
20512081372|'NEGOCIACION KIO' SOCIEDAD ANONIMA CERRADA|01/06/2012|RS
R.S.096-2012|
20505108672|1818 S.A.C|01/02/2006|RS R.S.254-2005|
20315738823|3A S.A.|01/11/2012|RS R.S.228-2012|
20100119227|3M PERU S A|01/06/2002|RS R.S.037-2002|
20498608036|5M DISTRIBUCIONES S.R.L.|01/11/2012|RS R.S.228-2012|
. . .
------------------------------------------

The result is a DBF with only two registers, when they should have 10 (5
for each APPEND FROM)
What is my mistake?

TIA

BestRegards
GVS

[Klas Engwall]

Hi Guillermo,

> APPEND FROM ( cFileTXT ) DELIMITED WITH PIPE
> APPEND FROM ( cFileTXT ) DELIMITED WITH ( { "|","|" } )

...

> --- AgenRet_TXT.txt ---------------------------------------
> 20512081372|'NEGOCIACION KIO' SOCIEDAD ANONIMA CERRADA|01/06/2012|RS
> R.S.096-2012|
> 20505108672|1818 S.A.C|01/02/2006|RS R.S.254-2005|
> 20315738823|3A S.A.|01/11/2012|RS R.S.228-2012|
> 20100119227|3M PERU S A|01/06/2002|RS R.S.037-2002|
> 20498608036|5M DISTRIBUCIONES S.R.L.|01/11/2012|RS R.S.228-2012|
> . . .
> ------------------------------------------
>
> The result is a DBF with only two registers, when they should have 10 (5
> for each APPEND FROM)
> What is my mistake?

You use the pipe character as a separator (but the separator should be a
comma), not a delimiter. The Clipper 5.3 Norton Guide explains it like this:

"DELIMITED WITH <xcDelimiter> identifies a delimited ASCII text file
where character fields are enclosed using the specified delimiter. You
can specify <xcDelimiter> as a literal character or as a character
expression enclosed in parentheses."

... (note the word "enclosed") ... and also ...

"This table shows the format specifications for DELIMITED and DELIMITED
WITH <xcDelimiter> ASCII text files:

DELIMITED Text File Format Specifications
---------------------------------------------------------------------
File Element Format
---------------------------------------------------------------------
Character fields May be delimited, with trailing blanks truncated
Date fields yyyymmdd
Logical fields T or F
Memo fields Ignored
Numeric fields Leading zeros may be truncated
Field separator Comma <==== here
Record separator Carriage return/linefeed
End of file marker 1A hex or CHR(26)
----------------------------------------------------------------------"

Regards,
Klas

[Guillermo Varona Silupú]

Hi Klas:

Anyway, TXT file (AgenRet_TXT.txt), Internet periodically download and
use it as it comes, (with PIPE as field separators), to place it in a
DBF file.
Is it possible to add the TXT file as it comes from Internet or
compulsorily have to edit it and change it?
How would the source code to make all records are added without having
to modify or edit the TXT file?
Why is the only current source adds a record?

TIA

BestRegards
GVS

[SPA]
El caso es que el archivo TXT, (AgenRet_TXT.txt), periódicamente se
descargar de internet y lo utilizo tal como viene, (con PIPE como
separadores de campo), para colocarlo en un archivo DBF.
¿Es posible agregar el archivo TXT tal como viene de internet u
obligatoriamente hay que editarlo y modificarlo?
¿Cómo quedaría el código fuente para hacer que se agreguen todos los
registros sin necesidad de modificar o editar el archivio TXT?
¿Porqué el código fuente actual solamente agrega un registro?

[Klas Engwall]

Hi Guillermo,

> Anyway, TXT file (AgenRet_TXT.txt), Internet periodically download and
> use it as it comes, (with PIPE as field separators), to place it in a
> DBF file.
> Is it possible to add the TXT file as it comes from Internet or
> compulsorily have to edit it and change it?
> How would the source code to make all records are added without having
> to modify or edit the TXT file?

The Clipper docs say that the fields must be separated with commas when
appending "DELIMITED" or "DELIMITED WITH <xcDelimiter>". Your text file
is separated (again: separated, not delimited) with pipe characters, so
I cannot see how you could append it as is. Maybe the easiest solution
is to read one line at a time, split it into tokens with hb_atokens()
and save the tokens "manually" in the dbf.

> Why is the only current source adds a record?

I suppose that APPEND FROM does its very best to import the file but
realizes after the first line that it does not understand the file
format. Ideally, it would have stopped before saving anything at all in
the target file.

Regards,
Klas

[Przemyslaw Czerpak]

On Mon, 12 Nov 2012, Klas Engwall wrote:

Hi Klass.

> The Clipper docs say that the fields must be separated with commas
> when appending "DELIMITED" or "DELIMITED WITH <xcDelimiter>". Your
> text file is separated (again: separated, not delimited) with pipe
> characters, so I cannot see how you could append it as is. Maybe the
> easiest solution is to read one line at a time, split it into tokens
> with hb_atokens() and save the tokens "manually" in the dbf.

Few years ago I implemented user settable string delimiters and field
separators in Harbour.
See ChangeLog entry below.

best regards,
Przemek



2006-08-10 14:00 UTC+0200 Przemyslaw Czerpak (druzus/at/priv.onet.pl)
* harbour/include/hbrdddel.h
* harbour/source/rdd/delim1.c
* harbour/source/rdd/dbcmd.c
+ extended COPY TO ... DELIMITED ... / APPEND FROM ... DELIMITED ...
to support some other common formats.
COPY TO ... DELIMITED WITH TAB
APPEND FROM ... DELIMITED WITH TAB
uses chr(9) as field separator and without string delimiters
COPY TO ... DELIMITED WITH PIPE
APPEND FROM ... DELIMITED WITH PIPE
uses '|' as field separator and without string delimiters
COPY TO ... DELIMITED WITH ( { cStrDelim, cFldSeparator } )
APPEND FROM ... DELIMITED WITH ( { cStrDelim, cFldSeparator } )
allows users to set and any character field delimiters and fields
separators

[Klas Engwall]

Hi Przemek,

> Few years ago I implemented user settable string delimiters and field
> separators in Harbour.
> See ChangeLog entry below.
>
> best regards,
> Przemek
>
> 2006-08-10 14:00 UTC+0200 Przemyslaw Czerpak (druzus/at/priv.onet.pl)
> * harbour/include/hbrdddel.h
> * harbour/source/rdd/delim1.c
> * harbour/source/rdd/dbcmd.c
> + extended COPY TO ... DELIMITED ... / APPEND FROM ... DELIMITED ...
> to support some other common formats.
> COPY TO ... DELIMITED WITH TAB
> APPEND FROM ... DELIMITED WITH TAB
> uses chr(9) as field separator and without string delimiters
> COPY TO ... DELIMITED WITH PIPE
> APPEND FROM ... DELIMITED WITH PIPE
> uses '|' as field separator and without string delimiters
> COPY TO ... DELIMITED WITH ( { cStrDelim, cFldSeparator } )
> APPEND FROM ... DELIMITED WITH ( { cStrDelim, cFldSeparator } )
> allows users to set and any character field delimiters and fields
> separators

Oops,

I have to admit I took my explanation from the docs. Bad strategy :-)

Guillermo, now I have actually tested your code and your text file. And
it worked (after unwrapping the first line, no other changes). I could
see all 2x5 records.

Regards,
Klas

[Guillermo Varona Silupú]

El 12/11/2012 05:57 p.m., Klas Engwall escribió:
> Hi Przemek,
>
>> Few years ago I implemented user settable string delimiters and field
>> separators in Harbour.
>> See ChangeLog entry below.

>> . . .

> Oops,
> I have to admit I took my explanation from the docs. Bad strategy :-)
> Guillermo, now I have actually tested your code and your text file.
> And it worked (after unwrapping the first line, no other changes). I
> could see all 2x5 records.

How could you?
Please could attach the files you used in the tests (TXT and PRG).

TIA

BestRegards
GVS

[Klas Engwall]

Hi Guillermo,

>> Guillermo, now I have actually tested your code and your text file.
>> And it worked (after unwrapping the first line, no other changes). I
>> could see all 2x5 records.
>
> How could you?
> Please could attach the files you used in the tests (TXT and PRG).

There is nothing new to attach. I only did what I said above: I copied
your five records to a text file with the same name you used (unwrapping
the first line), I copied your prg code to test.prg (straight with no
changes), then I ran "hbmk2 test". I did it with both 3.0.0. and 3.2,
and it worked in both cases.

No magic involved, except maybe what Przemek did six years ago :-)

Regards,
Klas

[Guillermo Varona Silupú]

Hi Klas:

Strange, with the attached TXT me not working.
Please could you try the attached TXT.

TIA

BestRegards
GVS

[Guillermo Varona Silupú]

I see why it works.
Seeing with TortoiseMerge and NotePad++, the line breaks are different (CR and CR LF).
How I can do to tell the APPEND FROM the record separator is CR?

TIA

BestRegards
GVS

[DaNiEl MaXiMiLiAnO]

CR an LF are control characters, respectively coded 0x0D (13 decimal) and 0x0A (10 decimal).

They are used to mark a line break in text file. A you indicated, Windows uses two characters the CR LF sequence; Unix only uses LF and MacIntosh CR.

An apocryphal historical perspective
As indicated by Peter, CR = Carriage Return and LF = Line Feed, two expressions which have their roots in the old typewriters / TTY. LF moved the paper up (but kept the horizontal position identical) and CR brought back the "carriage" so that the next character typed would be at the leftmost position on the paper (but on the same line). CR+LF was doing both, i.e. preparing to type a new line. As time went by the physical semantics of the codes were not applicable, and as memory and floppy disk space was at a premium, some OS designers decided to only use one of the characters, they just didn't communicate very well with one another ;-)

Most modern text editors and text-oriented application offer options/settings etc. that allow the automatic detection of the file's end-of-line convention and to display it accordingly.

 

Saludos

                DaNIEl MaXiMiLiAnO

---

De: Guillermo Varona Silupú <gvar...@hotmail.com>
Para: harbou...@googlegroups.com
Enviado: martes, 13 de noviembre de 2012 19:40
Asunto: Re: [harbour-users] APPEND FROM adds only 1 record. . .

--
You received this message because you are subscribed to the Google
Groups "Harbour Users" group.
Unsubscribe: harbour-user...@googlegroups.com
Web: http://groups.google.com/group/harbour-users

[Daniele Campagna]

Il 13/11/2012 23:40, Guillermo Varona Silupú ha scritto:

I see why it works.
Seeing with TortoiseMerge and NotePad++, the line breaks are different (CR and CR LF).
How I can do to tell the APPEND FROM the record separator is CR?

TIA

BestRegards
GVS

you can convert your file before appending from it.
fopen the file and fcreate a new file.
read line by line the first file , add LF and fwrite the line to the destination file.
(optionally after fclosing both files rename them so that the old becomes oldstuff.txt and the newly created <orgname.txt>)
To read lines you can use my function readline() or similar setting eol=chr(13)
HTH

[Guillermo Varona Silupú]

Hi Dan:

El 14/11/2012 01:25 p.m., Daniele Campagna escribió:

Il 13/11/2012 23:40, Guillermo Varona Silupú ha scritto:

I see why it works.
Seeing with TortoiseMerge and NotePad++, the line breaks are different (CR and CR LF).
How I can do to tell the APPEND FROM the record separator is CR?

you can convert your file before appending from it.

Indeed, I do not want is to make an edit / conversion intermediate internet download the file and add it to DBF

. . .

To read lines you can use my function readline() or similar setting eol=chr(13)

I tried like this:

APPEND FROM ( cFileTXT ) DELIMITED WITH PIPE
APPEND FROM ( cFileTXT ) DELIMITED WITH ( { "|",CHR(13) } )

and not performing, in both cases only add a record.

BestRegards
GVS

[Przemyslaw Czerpak]

On Wed, 14 Nov 2012, Guillermo Varona Silupú wrote:

Hi,

> >To read lines you can use my function readline() or similar
> >setting eol=chr(13)
>
> I tried like this:
>
> APPEND FROM ( cFileTXT ) DELIMITED WITH PIPE
> APPEND FROM ( cFileTXT ) DELIMITED WITH ( { "|",CHR(13) } )
>
> and not performing, in both cases only add a record.

I know that my English is fatal - for sure much more worse then
virtual languages I'm using creating algorithms and final programs :(
Sorry but I've never suggested that you can set record delimiter in
command:

COPY TO ... DELIMITED WITH ( { <cStrDelim>, <cFldSeparator> } )

<cStrDelim> is used for string fields as trailing and leading character.
<cFldSeparator> is used as field (column) delimiter.
If you want to set system line delimiters then you can use:

cDefEol := SET( _SET_EOL, chr( 13 ) )

COPY TO ... DELIMITED WITH ( { <cStrDelim>, <cFldSeparator> } )

SET( _SET_EOL, cDefEol )

best regards,
Przemek

[Guillermo Varona Silupú]

Hi Przemek:

El 14/11/2012 05:27 p.m., Przemyslaw Czerpak escribió:
> On Wed, 14 Nov 2012, Guillermo Varona Silupú wrote:
>
> Hi,
>
>>> To read lines you can use my function readline() or similar
>>> setting eol=chr(13)
>> I tried like this:
>>
>> APPEND FROM ( cFileTXT ) DELIMITED WITH PIPE
>> APPEND FROM ( cFileTXT ) DELIMITED WITH ( { "|",CHR(13) } )
>>
>> and not performing, in both cases only add a record.
> I know that my English is fatal - for sure much more worse then
> virtual languages I'm using creating algorithms and final programs :(

I also I have problems with my English. Even more so when you have to
translate technical terms.

> Sorry but I've never suggested that you can set record delimiter in
> command:
>
> COPY TO ... DELIMITED WITH ( { <cStrDelim>, <cFldSeparator> } )
>
> <cStrDelim> is used for string fields as trailing and leading character.
> <cFldSeparator> is used as field (column) delimiter.

Ok. Although I'm not sure I correctly interpreted the meaning of
<cStrDelim>, the following code worked great.

USE ( cFileDBF )

cDefEol := SET( _SET_EOL, chr( 13 ) )

APPEND FROM ( cFileTXT ) DELIMITED WITH PIPE

APPEND FROM ( cFileTXT ) DELIMITED WITH ( { "|","|" } )
SET( _SET_EOL, cDefEol )

Thank you very much.

BestRegards.
GVS