gbak restore on older ODS

[Adriano dos Santos Fernandes]

Hi!

It's possible to restore databases with a giving gbak for Firebird
version X using Firebird versions < X with any older ODS.

This requires many variations of queries due to fields not present
before, and I doubt everything even works correctly, as it's not being
really tested.

I propose to get rid of this and officially support downgrade only for
the direct previous major ODS.


Adriano

[Alex Peshkoff]

Moreover - we always recommend the following way of ODS downgrade:
1. Create backup of DB to be downgraded using gbak of target FB version
(to which DB to be downgraded). To make it work reliable DB should be
opened by FB server of native for it version (from which downgrade is
done). Gbak attaches to it remotely (certainly, use of localhost: is OK).
2. Next resulting .gbak file is restored in usual way.

During that process ability to restore to ODS with version < current for
gbak in use is never needed. Therefore (if we are going to cleanup
sonething) I wonder - what for do we need an ability to support
downgrade for the direct previous major ODS? I.e. looks like even more
global cleanup is possible - always restore only to cirrent ODS.

What am I missing?


PS. If we have decision here suppose that's worth asking on generic FB
forum before doing cleanup - to let more people see this suggestion.

[Adriano dos Santos Fernandes]

On 22/08/2024 07:52, Alex Peshkoff wrote:
> On 8/22/24 13:36, Adriano dos Santos Fernandes wrote:
>> Hi!
>>
>> It's possible to restore databases with a giving gbak for Firebird
>> version X using Firebird versions < X with any older ODS.
>>
>> This requires many variations of queries due to fields not present
>> before, and I doubt everything even works correctly, as it's not being
>> really tested.
>>
>> I propose to get rid of this and officially support downgrade only for
>> the direct previous major ODS.
>
> Moreover - we always recommend the following way of ODS downgrade:
> 1. Create backup of DB to be downgraded using gbak of target FB version
> (to which DB to be downgraded). To make it work reliable DB should be
> opened by FB server of native for it version (from which downgrade is
> done). Gbak attaches to it remotely (certainly, use of localhost: is OK).
> 2. Next resulting .gbak file is restored in usual way.
>

Seems ok for me, except that it makes impossible to cleanup/refactor
system objects, unless older gbak is also changed.

> During that process ability to restore to ODS with version < current for
> gbak in use is never needed. Therefore (if we are going to cleanup
> sonething) I wonder - what for do we need an ability to support
> downgrade for the direct previous major ODS? I.e. looks like even more
> global cleanup is possible - always restore only to cirrent ODS.
>

What about upgrades? Should it be:

- backup with same gbak version from the source ODS
- restore with gbak version same as source or target ODS?


Adriano

[Alex Peshkoff]

On 8/29/24 03:44, Adriano dos Santos Fernandes wrote:
> On 22/08/2024 07:52, Alex Peshkoff wrote:
>> On 8/22/24 13:36, Adriano dos Santos Fernandes wrote:
>>> Hi!
>>>
>>> It's possible to restore databases with a giving gbak for Firebird
>>> version X using Firebird versions < X with any older ODS.
>>>
>>> This requires many variations of queries due to fields not present
>>> before, and I doubt everything even works correctly, as it's not being
>>> really tested.
>>>
>>> I propose to get rid of this and officially support downgrade only for
>>> the direct previous major ODS.
>> Moreover - we always recommend the following way of ODS downgrade:
>> 1. Create backup of DB to be downgraded using gbak of target FB version
>> (to which DB to be downgraded). To make it work reliable DB should be
>> opened by FB server of native for it version (from which downgrade is
>> done). Gbak attaches to it remotely (certainly, use of localhost: is OK).
>> 2. Next resulting .gbak file is restored in usual way.
>>
> Seems ok for me, except that it makes impossible to cleanup/refactor
> system objects, unless older gbak is also changed.

Can you provide an example of such cleanup/refactor?

>> During that process ability to restore to ODS with version < current for
>> gbak in use is never needed. Therefore (if we are going to cleanup
>> sonething) I wonder - what for do we need an ability to support
>> downgrade for the direct previous major ODS? I.e. looks like even more
>> global cleanup is possible - always restore only to cirrent ODS.
>>
> What about upgrades? Should it be:
>
> - backup with same gbak version from the source ODS
> - restore with gbak version same as source or target ODS?
>

The only feature needed to upgrade is ensure that gbak understands
backups with old format. And it always used to be so, first of all due
to the nature of backup format: tag/value.

[Adriano dos Santos Fernandes]

On 29/08/2024 05:20, Alex Peshkoff wrote:
> On 8/29/24 03:44, Adriano dos Santos Fernandes wrote:
>> On 22/08/2024 07:52, Alex Peshkoff wrote:
>>> On 8/22/24 13:36, Adriano dos Santos Fernandes wrote:
>>>> Hi!
>>>>
>>>> It's possible to restore databases with a giving gbak for Firebird
>>>> version X using Firebird versions < X with any older ODS.
>>>>
>>>> This requires many variations of queries due to fields not present
>>>> before, and I doubt everything even works correctly, as it's not being
>>>> really tested.
>>>>
>>>> I propose to get rid of this and officially support downgrade only for
>>>> the direct previous major ODS.
>>> Moreover - we always recommend the following way of ODS downgrade:
>>> 1. Create backup of DB to be downgraded using gbak of target FB version
>>> (to which DB to be downgraded). To make it work reliable DB should be
>>> opened by FB server of native for it version (from which downgrade is
>>> done). Gbak attaches to it remotely (certainly, use of localhost: is
>>> OK).
>>> 2. Next resulting .gbak file is restored in usual way.
>>>
>> Seems ok for me, except that it makes impossible to cleanup/refactor
>> system objects, unless older gbak is also changed.
>

For example, remove an unused for long time field.

But I found possible problems with downgrade using older gbak with schemas.

For example, if there is more than one schema (without repeated object
names), I support that downgrade should be possible. But if user objects
are separated from system objects, there will be permissions for user's
schemas and older gbak will read and write permissions for objects that
the older engine does not understand.

I'm not saying that new gbak will always work for downgrade, but tries
for this case:
---
if (tdgbl->runtimeODS <= DB_VERSION_DDL8)
{
// Discard roles for IB4.
if (user_type == obj_sql_role || object_type == obj_sql_role)
exists = false;
}
---

There is also this weird thing:
---
if ( (tdgbl->RESTORE_format < 11) && (object_type > 19) ) // FB 4 has a
shift :(
object_type++;
---


Adriano

[Alex Peshkoff]

On 9/1/24 04:50, Adriano dos Santos Fernandes wrote:
> On 29/08/2024 05:20, Alex Peshkoff wrote:
>> On 8/29/24 03:44, Adriano dos Santos Fernandes wrote:
>>> On 22/08/2024 07:52, Alex Peshkoff wrote:
>>>> On 8/22/24 13:36, Adriano dos Santos Fernandes wrote:
>>>>> Hi!
>>>>>
>>>>> It's possible to restore databases with a giving gbak for Firebird
>>>>> version X using Firebird versions < X with any older ODS.
>>>>>
>>>>> This requires many variations of queries due to fields not present
>>>>> before, and I doubt everything even works correctly, as it's not being
>>>>> really tested.
>>>>>
>>>>> I propose to get rid of this and officially support downgrade only for
>>>>> the direct previous major ODS.
>>>> Moreover - we always recommend the following way of ODS downgrade:
>>>> 1. Create backup of DB to be downgraded using gbak of target FB version
>>>> (to which DB to be downgraded). To make it work reliable DB should be
>>>> opened by FB server of native for it version (from which downgrade is
>>>> done). Gbak attaches to it remotely (certainly, use of localhost: is
>>>> OK).
>>>> 2. Next resulting .gbak file is restored in usual way.
>>>>
>>> Seems ok for me, except that it makes impossible to cleanup/refactor
>>> system objects, unless older gbak is also changed.
> For example, remove an unused for long time field.

If it's about a field in system table I doubt that can ever happen.
Existing system metadata is a kind of 'sacred cow' for us, it can be
only extended.

> But I found possible problems with downgrade using older gbak with schemas.
>
> For example, if there is more than one schema (without repeated object
> names), I support that downgrade should be possible.

We never promised to our users that downgrade of any DB, using new
features, will be possible. Certainly I do not say that it's something
bad :)
And yes, to enable such downgrade, one shold use new gbak able to
restore to old ODS, this single feature is definitely enough.

[Adriano dos Santos Fernandes]

On 8/22/24 07:52, Alex Peshkoff wrote:
> On 8/22/24 13:36, Adriano dos Santos Fernandes wrote:
>> Hi!
>>
>> It's possible to restore databases with a giving gbak for Firebird
>> version X using Firebird versions < X with any older ODS.
>>
>> This requires many variations of queries due to fields not present
>> before, and I doubt everything even works correctly, as it's not being
>> really tested.
>>
>> I propose to get rid of this and officially support downgrade only for
>> the direct previous major ODS.
>
> Moreover - we always recommend the following way of ODS downgrade:
> 1. Create backup of DB to be downgraded using gbak of target FB version
> (to which DB to be downgraded). To make it work reliable DB should be
> opened by FB server of native for it version (from which downgrade is
> done). Gbak attaches to it remotely (certainly, use of localhost: is OK).
> 2. Next resulting .gbak file is restored in usual way.
>

I'm backporting things to v5 so downgrade of v6 databases (with schemas)
works and found a problem with this approach.

gbak has select queries, for example, to discover generator values.

This obviously don't work as that objects are not in the schema search path.

While some changes in gbak (backup) can also be backported, I'm not
understanding why that is the recommended path.

For me the newer gbak has much more context about the older engine (to
restore on it) and can do things different than expect the older gbak is
going to understand every detail of the new engine (to backup from it).


Adriano