Use of UT_CSV_HANDLER with CSV file as input

[Gachoud Philippe]

Hi everybody!

Trying to parse csv files, I found the following lib (gobo) into $ISE_EIFFEL UT_CSV_HANDLER

and am trying to parse like that and the result is an infinite loop into

l_csv_handler.read_file (l_is, agent impl_action_for_each_csv_line (?))

any comment about my code and advices are also mostly welcome

    import_from_csv (a_rest_request: REST_REQUEST): detachable like items
            -- data: { "action": { "feature": "import_from_csv", "arg1": "/tmp/anyFile.csv", "arg2": "1"} }
            --
            -- argument 1 is file_name
            -- argument 2 is charge_template_id
            -- Works with item
        require
            a_rest_request.has_request_feature
            valid_arguments: attached a_rest_request.request_feature as l_req_f and then
                l_req_f.has_arguments and then
                l_req_f.arguments.count = 2
            first_argument_is_file_name: attached l_req_f.arguments.at(1) as l_arg1 and then
                not l_arg1.is_empty
            second_argument_is_integer: attached l_req_f.arguments.at(2) as l_arg2 and then
                l_arg2.is_integer and then
                l_arg2.to_integer > 0
            not header_has_been_passed
        local
            l_csv_handler: UT_CSV_HANDLER
            l_is: KL_STRING_INPUT_STREAM
            l_file: PLAIN_TEXT_FILE
        do
            if attached a_rest_request.request_feature as l_req_f and then
                attached l_req_f.arguments.at(1) as l_fn and then
                attached l_req_f.arguments.at(2) as l_charge_template_id_s and then
                l_charge_template_id_s.is_integer_64
            then
                create l_file.make_open_read (l_fn)
                l_file.read_stream (l_file.count)
                create l_csv_handler.make_with_separator (';')
                if attached l_file.last_string as l_file_s then
                    -- set item
                    item := item_prototype.twin
                    if attached item as l_item then
                        l_item.set_primary_key (l_charge_template_id_s.to_integer_64)
                    end
                    -- read file
                    create l_is.make (l_file_s)
                    l_csv_handler.read_file (l_is, agent impl_action_for_each_csv_line (?))
                else
                    check
                        file_empty: False
                    end
                end
                l_file.close
                if attached item as l_item then
                    if l_item.has_charge_units then
                        create_or_update_item
                    else
                        check
                            False
                        end
                    end
                else
                    check
                        item_not_void: False
                    end
                end
            else
                check
                    something_s_wrong_with_parameters_of_qry: False
                end
            end
        end

[Eric Bezault]

On 21/02/2019 12:53, Gachoud Philippe wrote:
> Hi everybody!
>
> Trying to parse csv files, I found the following lib (gobo) into
> $ISE_EIFFEL UT_CSV_HANDLER
>

> and am trying to parse like that and the result is an *infinite loop* into
>
> |
> l_csv_handler.read_file (l_is,agent impl_action_for_each_csv_line (?))
> |
>
> any *comment about my code and advices* are also mostly welcome

I see nothing wrong, apart from the fact that I would have used
a KL_TEXT_INPUT_FILE instead of the combination of a PLAIN_TEXT_FILE
and a KL_STRING_INPUT_STREAM.

Is it possible to see the file which is causing this infinite loop?

--
Eric Bezault
mailto:er...@gobosoft.com
http://www.gobosoft.com

[Gachoud Philippe]

Thx Eric, thx for your answer.

The file contains the following data:

;;;1545344;1545344
;;;519321;519321
Fecha;Hora;Kw;kWhD;kVarhD
01-01-2016;0:15;805;201;28
01-01-2016;0:30;804;201;29
01-01-2016;0:45;806;201;29
01-01-2016;1:00;808;202;29
....

70179 lines of that

I'm figuring out that with 10 lines it works... will push further my investigations and tryout with KL_TEXT_INPUT_FILE (why this instead of PLAIN_TEXT_FILE?) as you mentioned, to see if there is a difference.

The reason I used PLAIN_TEXT_FILE is I didn't find the type of file to use with descendents of

[Eric Bezault]

On 21/02/2019 14:16, Gachoud Philippe wrote:
> Thx Eric, thx for your answer.
>
> The file contains the following data:
>
> ;;;1545344;1545344
> ;;;519321;519321
> Fecha;Hora;Kw;kWhD;kVarhD
> 01-01-2016;0:15;805;201;28
> 01-01-2016;0:30;804;201;29
> 01-01-2016;0:45;806;201;29
> 01-01-2016;1:00;808;202;29
> ....
>
>
> 70179 lines of that
>
> I'm figuring out that with 10 lines it works... will push further my
> investigations and tryout with KL_TEXT_INPUT_FILE (why this instead of
> PLAIN_TEXT_FILE?) as you mentioned, to see if there is a difference.
> The reason I used PLAIN_TEXT_FILE is I didn't find the type of file to
> use with descendents of

You didn't find it in the list of descendants because it
was not compiled in your project yet.

In order to browse the code (and not miss what is not compiled,
i.e. not reachable from the root class), I usually have a second
project with <root all_classes="true"/> in the ECF file.

[Gachoud Philippe]

So, after further investigations,

• the reading seems quicker with KL_TEXT_INPUT_FILE
• I could have read the file in multiple times instead of reading the entire @ once as done by
  read_string(l_file.count)
• finalizing the system (with assertions) it goes now 1'12'' to read its content
  
  

So the final word is: just wait! don't be impatient and if you want to gain some time, finalize your system

I learned and hope some too