Working with services when multiple providers are used

[Alex Peshkoff]

Hi all!

It's quite clear that utilities that directly access database file (like
nbackup) can't work with non-native ODS. This does cause problems when
using that utilities as services - see
https://github.com/FirebirdSQL/firebird/issues/7579. Original
multi-providers architecture was not designed to be used with services -
as far as I can see, services came to life in a period when
multi-providers mode was almost unsupported.

Therefore some extension of access to multiple providers is needed. I
can suggest 2 possible solutions.

1. Add to Provider interface new entry performing invocation of service
thread in context of that provider. It will be invoked when native
thread failed to start (probably with limited set of errors returned).
That's straightforward approach and can be implemented relatively easy,
but has one disadvantage - need to add to public API call absolutely
unneeded to users, moreover it will go to the central part ob DB access
API.

2. Use already existing isc_spb_expected_db parameter when attaching to
services manager. Additional functionality will be check for attach to
that database (and detach at once) when connecting to services manager.
This will help to use correct services manager at once. But in a case
when isc_spb_expected_db is not specified by user (and that's normal
practice) we will need to chain the call to attach to services in a case
of utility startup error. In that chained call we can specify correct
isc_spb_expected_db parameter and chain all calls to services manager to
resulting interface until next service start call. This approach does
not require to expose something to users, but definitely more complex
and can probably have unexpected side effects.

What to be chosen? May be better ideas?

Alex.

[Dimitry Sibiryakov]

'Alex Peshkoff' via firebird-devel wrote 17.05.2023 17:03:
> It's quite clear that utilities that directly access database file (like nbackup) can't work with non-native ODS.

A silly question: why nbackup need to know ODS if all it does with the file
is a file-system copy?

> 1. Add to Provider interface new entry performing invocation of service thread in context of that provider. It will be invoked when native thread failed to start (probably with limited set of errors returned). That's straightforward approach and can be implemented relatively easy, but has one disadvantage - need to add to public API call absolutely unneeded to users, moreover it will go to the central part ob DB access API.

Using the same API for user application and providers was indeed a mistake,
may be it is time to divorce them. And services providers as well.

> 2. Use already existing isc_spb_expected_db parameter when attaching to services manager. Additional functionality will be check for attach to that database (and detach at once) when connecting to services manager. This will help to use correct services manager at once. But in a case when isc_spb_expected_db is not specified by user (and that's normal practice) we will need to chain the call to attach to services in a case of utility startup error. In that chained call we can specify correct isc_spb_expected_db parameter and chain all calls to services manager to resulting interface until next service start call. This approach does not require to expose something to users, but definitely more complex and can probably have unexpected side effects.

It is better if IProvider::attachServiceManager() and IService::start()
return something like isc_unavailable so Y-valve can proceed to the next provider.

> May be better ideas?

Perhaps it should be decided on per-service basic. For example gbak has some
backward compatibility so backup service from engine13 can perform ODS12 backup
nicely if it uses Y-valve for connection to database.
Another old idea that I had is to allow usage of provider name in URL-style
connection string so service with desired engine can be activated from
application using something like "inet://remote.host.net/engine12://service_mgr"
string.

--
WBR, SD.

[Alex Peshkoff]

On 5/17/23 18:24, 'Dimitry Sibiryakov' via firebird-devel wrote:
> 'Alex Peshkoff' via firebird-devel wrote 17.05.2023 17:03:
>> It's quite clear that utilities that directly access database file
>> (like nbackup) can't work with non-native ODS.
>
>   A silly question: why nbackup need to know ODS if all it does with
> the file is a file-system copy?
>

Since FB3 nbackup also works with SCN's inventory pages. That does
require ODS knowledge.

Yes, seems to be so.

>
>> May be better ideas?
>
>   Perhaps it should be decided on per-service basic. For example gbak
> has some backward compatibility so backup service from engine13 can
> perform ODS12 backup nicely if it uses Y-valve for connection to
> database.

It already works that way, is not it?

> Another old idea that I had is to allow usage of provider name in
> URL-style connection string so service with desired engine can be
> activated from application using something like
> "inet://remote.host.net/engine12://service_mgr" string.
>

Not bad (specially when one nneds to create DB with non-first provider)
but automatic provider search is also desired.

[Денис Симонов]

> Another old idea that I had is to allow usage of provider name in URL-style
> connection string so service with desired engine can be activated from
> application using something like "inet://remote.host.net/engine12://service_mgr"
> string.

+1

Prefixes would be useful not only for the service manager, but also for regular database connection strings. This would allow you to immediately address the desired provider, and not sort through them along the chain. Chaining can hide the true reason for a failed connection.

понедельник, 22 мая 2023 г. в 11:49:05 UTC+3, Alex Peshkoff:

[Mark Rotteveel]

On 17-05-2023 17:24, 'Dimitry Sibiryakov' via firebird-devel wrote:
>   Another old idea that I had is to allow usage of provider name in
> URL-style connection string so service with desired engine can be
> activated from application using something like
> "inet://remote.host.net/engine12://service_mgr" string.

Given specifying service_mgr is not necessary these days, I'd suggest to
use `inet://remote.host.net/engine12` and reuse the old method of
specifying the appropriate service to instead specify the desired provider.

Alternatively, inspired by the RFC 3986 User Information in the URL format:

inet://engi...@remote.host.net/

Mark
--
Mark Rotteveel

[Dimitry Sibiryakov]

Mark Rotteveel wrote 22.05.2023 16:07:
> Given specifying service_mgr is not necessary these days, I'd suggest to use
> `inet://remote.host.net/engine12` and reuse the old method of specifying the
> appropriate service to instead specify the desired provider.

Even if service_mgr is not necessary, it is better to replace it with
"expected database", IMHO.

> Alternatively, inspired by the RFC 3986 User Information in the URL format:

For me it would look confusing.

--
WBR, SD.

[Mark Rotteveel]

I find it less confusing than the suggested
inet://remote.host.net/engine12://service_mgr

Then I'd sooner prefer something like

inet://remote.host.net/service_mgr?provider=engine12

or

inet://remote.host.net/service_mgr;provider=engine12

but such syntax would then suggest the possibility to also specify SPB
or DPB items in that way (and also raises questions about
requiring/supporting URL encoding, etc).

Mark
--
Mark Rotteveel

[Dmitry Yemanov]

17.05.2023 18:24, 'Dimitry Sibiryakov' via firebird-devel wrote:
>
> Another old idea that I had is to allow usage of provider name in
> URL-style connection string so service with desired engine can be
> activated from application using something like
> "inet://remote.host.net/engine12://service_mgr" string.

The full syntax (with port number included) looks quite cryptic to me:

inet://remote.host.net:3051/engine12://service_mgr

I like Mark's suggestion better (actually, both of them, but I agree the
?param=value& really suggests something more than just the provider name
to be usable).


Dmitry

[Adriano dos Santos Fernandes]

On 22/05/2023 11:07, Mark Rotteveel wrote:
>
> Alternatively, inspired by the RFC 3986 User Information in the URL format:
>
> inet://engi...@remote.host.net/
>

Engine name is not user information so I don't think it should be used
for that.


Adriano

[Mark Rotteveel]

Generally I agree, but RFC 3986 says:

"""
The userinfo subcomponent may consist of a user name and, optionally,
scheme-specific information about how to gain authorization to access
the resource.
"""

and in a way, this could be considered "scheme-specific information
about how to gain authorization to access the resource" (though I admit
that requires a bit of twisting of the meaning of authorization).

Mark
--
Mark Rotteveel