Refactoring DBD::File in preparation for ReadOnly and streams ...

Jens Rehsack

unread,

Sep 19, 2012, 11:21:32 AM9/19/12

to H.Merijn Brand, DBI Developers Mailing List

Hi Merijn,

while hacking around in DBD::File and DBI::DBD::SqlEngine I stumbled
over a major design fault made in the past:

sub DBD::File::Table::get_table_meta () ... evaluates
$dbh->{f_meta}{$table}{initialized} and does something magic else. This
magic is fully DBD::File addicted (heavily relies on file2table) and it
should be broken into separate pieces to differ between initialisation
done for DBI::DBD::SqlEngine and DBD::File and DBD::DBM ...

I'd like to discuss it tomorrow in IRC (but I read my e-Mail if you have
comments at the evening).

If anyone else has ideas - please feel free to speak (but primary
restriction is backward compatibility to avoid breakage of dependent DBD's).

Best regards,
Jens

H.Merijn Brand

unread,

Sep 19, 2012, 11:50:02 AM9/19/12

to dbi...@perl.org

On Wed, 19 Sep 2012 17:21:32 +0200, Jens Rehsack <reh...@cpan.org>
wrote:

> Hi Merijn,
>
> while hacking around in DBD::File and DBI::DBD::SqlEngine I stumbled
> over a major design fault made in the past:
>
> sub DBD::File::Table::get_table_meta () ... evaluates
> $dbh->{f_meta}{$table}{initialized} and does something magic else. This
> magic is fully DBD::File addicted (heavily relies on file2table) and it
> should be broken into separate pieces to differ between initialisation
> done for DBI::DBD::SqlEngine and DBD::File and DBD::DBM ...

IIRC this was what you already advertised on your first round of
DBD::File internals redesign.

As long as "users" of DBD::File are not harmed, go ahead.

I do not want a truckload of code like
http://repo.or.cz/w/DBD-CSV.git/blob/8d7f4284:/lib/DBD/CSV.pm#l90

which has now been removed as the prereq's are higher:

PREREQ_PM => {
"DBI" => 1.614,
"DBD::File" => 0.40,
"Text::CSV_XS" => 0.91,
"SQL::Statement" => 1.33,
"Test::More" => 0.90,
"Encode" => 0,
"charnames" => 0,
},

But in future upgrades/updates, code like that is not unlikely to
re-appear

> I'd like to discuss it tomorrow in IRC (but I read my e-Mail if you have
> comments at the evening).
>
> If anyone else has ideas - please feel free to speak (but primary
> restriction is backward compatibility to avoid breakage of dependent DBD's).
>
> Best regards,
> Jens

--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using perl5.00307 .. 5.14 porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

Jens Rehsack

unread,

Sep 20, 2012, 4:37:46 AM9/20/12

to dbi...@perl.org

On 19.09.2012 17:50, H.Merijn Brand wrote:
> On Wed, 19 Sep 2012 17:21:32 +0200, Jens Rehsack <reh...@cpan.org>
> wrote:
>
>> Hi Merijn,
>>
>> while hacking around in DBD::File and DBI::DBD::SqlEngine I stumbled
>> over a major design fault made in the past:
>>
>> sub DBD::File::Table::get_table_meta () ... evaluates
>> $dbh->{f_meta}{$table}{initialized} and does something magic else. This
>> magic is fully DBD::File addicted (heavily relies on file2table) and it
>> should be broken into separate pieces to differ between initialisation
>> done for DBI::DBD::SqlEngine and DBD::File and DBD::DBM ...
>
> IIRC this was what you already advertised on your first round of
> DBD::File internals redesign.

Same place, different construction site ;)

> As long as "users" of DBD::File are not harmed, go ahead.

I have an idea which ensures that. I introduce table local getters and
use the "local" keyword as seen in App::Cmd ;)

> I do not want a truckload of code like
> http://repo.or.cz/w/DBD-CSV.git/blob/8d7f4284:/lib/DBD/CSV.pm#l90

No one wants that - and I think meanwhile you can increase the
prerequisite of DBI version to remove some of that truckload ...

> which has now been removed as the prereq's are higher:
>
> PREREQ_PM => {
> "DBI" => 1.614,
> "DBD::File" => 0.40,
> "Text::CSV_XS" => 0.91,
> "SQL::Statement" => 1.33,
> "Test::More" => 0.90,
> "Encode" => 0,
> "charnames" => 0,
> },
>
> But in future upgrades/updates, code like that is not unlikely to
> re-appear

Well, business as usual ;)

/Jens

H.Merijn Brand

unread,

Sep 22, 2012, 6:23:08 AM9/22/12

to dbi...@perl.org

On Wed, 19 Sep 2012 17:21:32 +0200, Jens Rehsack <reh...@cpan.org>
wrote:

> Hi Merijn,
>
> while hacking around in DBD::File and DBI::DBD::SqlEngine I stumbled
> over a major design fault made in the past:

Some - backward compatible - thoughts:

Replace all dir-related parts in DBD::File with callbacks

Make streaming interfaces able to override dir-related parts

Backward compatible AND extendable

> sub DBD::File::Table::get_table_meta () ... evaluates
> $dbh->{f_meta}{$table}{initialized} and does something magic else. This
> magic is fully DBD::File addicted (heavily relies on file2table) and it
> should be broken into separate pieces to differ between initialisation
> done for DBI::DBD::SqlEngine and DBD::File and DBD::DBM ...
>
> I'd like to discuss it tomorrow in IRC (but I read my e-Mail if you have
> comments at the evening).
>
> If anyone else has ideas - please feel free to speak (but primary
> restriction is backward compatibility to avoid breakage of dependent DBD's).
>
> Best regards,
> Jens

Jens Rehsack

unread,

Sep 24, 2012, 10:48:56 AM9/24/12

to H.Merijn Brand, dbi...@perl.org

On 22.09.2012 12:23, H.Merijn Brand wrote:
> On Wed, 19 Sep 2012 17:21:32 +0200, Jens Rehsack <reh...@cpan.org>
> wrote:
>
>> Hi Merijn,
>>
>> while hacking around in DBD::File and DBI::DBD::SqlEngine I stumbled
>> over a major design fault made in the past:
>
> Some - backward compatible - thoughts:
>
> Replace all dir-related parts in DBD::File with callbacks
>
> Make streaming interfaces able to override dir-related parts
>
> Backward compatible AND extendable

First shot attached as committed (svn revert for the win ^^).
Needs some additional tests for streams as well as pod updates.

>> sub DBD::File::Table::get_table_meta () ... evaluates
>> $dbh->{f_meta}{$table}{initialized} and does something magic else. This
>> magic is fully DBD::File addicted (heavily relies on file2table) and it
>> should be broken into separate pieces to differ between initialisation
>> done for DBI::DBD::SqlEngine and DBD::File and DBD::DBM ...
>>
>> I'd like to discuss it tomorrow in IRC (but I read my e-Mail if you have
>> comments at the evening).
>>
>> If anyone else has ideas - please feel free to speak (but primary
>> restriction is backward compatibility to avoid breakage of dependent DBD's).
>>
>> Best regards,
>> Jens

Best regards,
Jens

patch-data_source