[Info-Ingres] Editing a configuration file

[Martin Bowes]

Hi All,

 

As part of a Disaster Recovery Plan I'm in the process of trialling a Perl based configuration file editor.

 

All I'm changing is the database extent directory. I'm not adding new locations, I'm simply amending the path strings to reflect the new host and installation.

 

Other than the path name, I'm changing only the integer field which holds the path length.

 

The amended file is then written to disk. It has the same length as the original. My editor has no problem re-reading the amended file and  displaying its contents….but Ingres sure doesn't like it!

 

An infodb bombs out with E_DM923A_CONFIG_OPEN_ERROR

 

And in the errlog we see:

E_DM923C_CONFIG_FORMAT_ERROR  Error in configuration file format.

E_DM923A_CONFIG_OPEN_ERROR    Error occurred opening configuration file (aaaaaaaa.cnf).

 

 

Anyone got any idea what else I need to change?

 

Martin Bowes

[Ingres Forums]

Only what I can tell to this is:
It incarnates the need of "finddbs" tool


--
bilgihan
------------------------------------------------------------------------
bilgihan's Profile: http://community.actian.com/forum/member.php?userid=8658
View this thread: http://community.actian.com/forum/showthread.php?t=14077

[Ian Kirkham]

Hi Marty,

You usually get this if another session is in CBF or the like & hence has the config.dat locked.

Regards,

Ian

[Martin Bowes]

Yep. I checked the open source and that was definitly the gist of what I read in dm0c.c etc.

 

This has me puzzelled! I've opened and closed the files to edit them. Nothing else should be holding a lock on them so I'm stumped. Is a lock status written into the config file itself? Something that’s says 'Hi I've been opened by application X.'?

 

Marty

[Armand Pirvu (gmail)]

I am not sure that config.dat is related to this.

I believe that the cnf has a special format.

Whenever I needed to do this I did it with a hex editor (If needed to do manually)

There was also in support an editor for cnf written in C IIRC so if still around maybe you can get it 

_______________________________________________
Info-Ingres mailing list
Info-...@kettleriverconsulting.com
http://ext-cando.kettleriverconsulting.com/mailman/listinfo/info-ingres

[Martin Bowes]

FYI. The config file structure is in:

$ING_SRC/back/dmf/hdr/dm0c.h

$ING_SRC/back/dmf/dmp/dm0c.c.

 

But best of luck reading the code!

 

Marty

[Karl Schendel]

On Nov 25, 2011, at 6:32 AM, Martin Bowes wrote:

> Hi All,
>
> As part of a Disaster Recovery Plan I'm in the process of trialling a Perl based configuration file editor.
>
> All I'm changing is the database extent directory. I'm not adding new locations, I'm simply amending the path strings to reflect the new host and installation.
>
> Other than the path name, I'm changing only the integer field which holds the path length.
>
> The amended file is then written to disk. It has the same length as the original. My editor has no problem re-reading the amended file and displaying its contents….but Ingres sure doesn't like it!
>
> An infodb bombs out with E_DM923A_CONFIG_OPEN_ERROR
>
> And in the errlog we see:
> E_DM923C_CONFIG_FORMAT_ERROR Error in configuration file format.
> E_DM923A_CONFIG_OPEN_ERROR Error occurred opening configuration file (aaaaaaaa.cnf).
>

DM923C looks like an extent count mismatch. There's only one of them in dm0c.c.
There are a few conditions that could cause the mismatch. If I were doing it,
I'd probably whip up a server with dm0c compiled debug, and step through that
loop to see which bit of the cnf file is buggered up.

Karl

[Martin Bowes]

Ah I see the problem....

There are seven integers at the end of the path string. I'd set them all to zeroes rather than preserve the original contents. Whoops.

So now my edit works and infodb likes the result but my recovery now fails with:
E_DM130C_RFP_DELETE_ALL Error deleting files from directory before restoring from checkpoint.

What I'd forgotten was that my edited cnf becomes the rfc and then the rollforwarddb extracts an un-edited cnf from the checkpoint tar file. Their disagreement on directory paths must be creating a conniption fit.

OK...I think I'll have to tackle this by making a special template file (cktmpl.def) that will monster the unedited cnf file before the delete phase starts via the BRDD: ...

Sometimes the best we can hope for is that the lesser of two evils is also the funniest.

Marty

-----Original Message-----
From: Karl Schendel [mailto:sche...@kbcomputer.com]
Sent: 25 November 2011 23:59
To: Ingres and related product discussion forum
Subject: Re: [Info-Ingres] Editing a configuration file

On Nov 25, 2011, at 6:32 AM, Martin Bowes wrote:

> Hi All,
>
> As part of a Disaster Recovery Plan I'm in the process of trialling a Perl based configuration file editor.
>
> All I'm changing is the database extent directory. I'm not adding new locations, I'm simply amending the path strings to reflect the new host and installation.
>
> Other than the path name, I'm changing only the integer field which holds the path length.
>

> The amended file is then written to disk. It has the same length as the original. My editor has no problem re-reading the amended file and displaying its contents....but Ingres sure doesn't like it!

>
> An infodb bombs out with E_DM923A_CONFIG_OPEN_ERROR
>
> And in the errlog we see:
> E_DM923C_CONFIG_FORMAT_ERROR Error in configuration file format.
> E_DM923A_CONFIG_OPEN_ERROR Error occurred opening configuration file (aaaaaaaa.cnf).
>

DM923C looks like an extent count mismatch. There's only one of them in dm0c.c.
There are a few conditions that could cause the mismatch. If I were doing it,
I'd probably whip up a server with dm0c compiled debug, and step through that
loop to see which bit of the cnf file is buggered up.

Karl

[lin.a.b...@uk.bnpparibas.com]

Marty,

We've been editing CNF files for many years as part of batch processes and we've never had to do that as part of the cktmpl file.

If you make the changes to the cnf file in the data area and the dmp area  AND also the c*.dmp file, it will do the delete perfectly.

Cheers,
Lin

Internet   martin...@ctsu.ox.ac.uk Sent by: info-ingr...@kettleriverconsulting.com 28/11/2011 11:36 Please respond to info-...@kettleriverconsulting.com

To info-...@kettleriverconsulting.com cc karl.s...@ingres.com

___________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/information/legal_information.asp?Code=ECAS-845C5H  for additional disclosures.

[Martin Bowes]

Hi Lin,

 

Thanks for that. I'll try monkeying with the c*.dmp first and see if that shakes the problem loose.

 

Marty

[Ingres Forums]

Hello Lin and Hello Marty and whoever monitors this discussion,

from support perspective when I observe this discussion, I hope all
goes well, that these things don't raise more problem later than it
solves currently.

I'm convinced what you try to achieve is for your business needs.
This is fact: you need it.

I was thinking rather incarnation project for finddbs, but:
I've been told in last Code Sprint there were some discussion around
this topic and Henrik would have some scripts doing the things better,
which were used for Tru64 platform.
Unfortunately I was not there neither I have any other detail on hand.

Best thing would be to start with the scripts and see whatelse needs to
be done.
Once done, we can consider putting them to into ingres/sig/...
directory so every one could take advantage.
This would allow us (customers and Actian engineers) operating on "safe
water".

Thanks - Bilgihan
PS: I'll send the same info to Henrik shortly

--
bilgihan
------------------------------------------------------------------------
bilgihan's Profile: http://community.actian.com/forum/member.php?userid=8658

View this thread: http://community.actian.com/forum/showthread.php?t=14087

[Martin Bowes]

Hi Bilgihan,

The more I think about this the more I think the best thing would be to remove all the hard-coded paths completely from the database config files and from the journals and dumps. I have a gut feeling we could rely on the location names to do what we need. This would make recovery in a DR box with different paths real easy, we would just need to ensure the same location names exist. It would remove any need to do editing on these critical files.

Marty

-----Original Message-----
From: Ingres Forums [mailto:info-...@kettleriverconsulting.com]
Sent: 01 December 2011 13:33
To: info-...@kettleriverconsulting.com
Subject: Re: [Info-Ingres] Editing a configuration file

[Roy Hann]

Martin Bowes wrote:

> The more I think about this the more I think the best thing would be
> to remove all the hard-coded paths completely from the database config
> files and from the journals and dumps. I have a gut feeling we could
> rely on the location names to do what we need. This would make
> recovery in a DR box with different paths real easy, we would just
> need to ensure the same location names exist. It would remove any need
> to do editing on these critical files.

Hear hear.

--
Roy

UK Ingres User Association Conference 2012 will be on Tuesday June 19 2012.
The latest information is available from www.uk-iua.org.uk.

[Sørensen.Henrik Georg HGS]

At the Codesprint in June 2011 as Bilgihan mentions I was working on porting
Our Company's "job" to do this work - then renamed to "relocateckp" into the Sig-directory.

The job is based upon some Shell-scripts and one C-program (which makes the changes in the CNF-file),
And makes it possible for us to load a database on another machine, with another installationcode
And the database under a different name.

One essential job for making database-copies on other machines for Slave-backups for different purposes
Also in the migration process where you need to have the time and possibility to unload a database to
Reload on another machine.type (although it should be possible to use "relocateckp" as well for most platforms,
As long as the Character-sets is identical).

By the way just remember to take partions off while creating the Unload/Reload-scripts with Unloaddb to avoid
Using the Logfile during Reload.

At the time of the Codesprint we were in the early face of our migration from TRU64 (AXP.OSF) towards
Linux, so didn't get it all to work during the Codesprint, and been quite busy since at work.

Al though our C-program was re-niced and our extract of the Ingres-include files replaced with the correct
Ingres-include files, and building set up the Ingres-way, lots of translation, replacing and removing internal stuff.

The work at that time has been parked in a Tar-archieve (on several hands) but not secretly hidden inside the Source-code :-).

Our present Linux-version of the Shell-scripts has now been given back to Ingres Support (with lots of Danish and Internal setups inside :-).

Kind regards
Henrik Georg Sørensen
Seniorconsultant, Ingres Database Administrator for Decades
Organisator - En del af KMD
-----Oprindelig meddelelse-----
Fra: info-ingr...@kettleriverconsulting.com [mailto:info-ingr...@kettleriverconsulting.com] På vegne af Martin Bowes
Sendt: 2. december 2011 09:45
Til: Ingres and related product discussion forum
Emne: Re: [Info-Ingres] Editing a configuration file

Hi Bilgihan,

The more I think about this the more I think the best thing would be to remove all the hard-coded paths completely from the database config files and from the journals and dumps. I have a gut feeling we could rely on the location names to do what we need. This would make recovery in a DR box with different paths real easy, we would just need to ensure the same location names exist. It would remove any need to do editing on these critical files.

Marty

-----Original Message-----
From: Ingres Forums [mailto:info-...@kettleriverconsulting.com]
Sent: 01 December 2011 13:33
To: info-...@kettleriverconsulting.com

Subject: Re: [Info-Ingres] Editing a configuration file

[lin.a.b...@uk.bnpparibas.com]

Sounds like something we could have done with a long time ago.

We have always used this method for taking copies of production into test or dev, not DR so we never needed to rollforward the journals .
We use it for changing paths, database names  - and even the dbid. However, the limitations are that we can only use as many characters as are in use already .
So all the paths etc have to be the same length afterwards as they were before.

More flexibility is the key.

Regards,
Lin

Internet   martin...@ctsu.ox.ac.uk Sent by: info-ingr...@kettleriverconsulting.com 02/12/2011 08:27 Please respond to info-...@kettleriverconsulting.com

To info-...@kettleriverconsulting.com cc

[Karl Schendel]

On Dec 2, 2011, at 3:27 AM, Martin Bowes wrote:

> Hi Bilgihan,
>
> The more I think about this the more I think the best thing would be to remove all the hard-coded paths completely from the database config files and from the journals and dumps.

Indeed.

The only reason (as far as I know) that it hasn't been done already is that
recovery can get tricky. Offline crash recovery sort-of-kind-of tries to use
as little context as possible, specifically including location knowledge and
location availability. Digging through recovery to see what can and
should be done takes the project from "maybe a week or so" up to
"maybe three weeks, a month, who knows", which is why it hasn't
been done already in stealth mode.

I completely agree, though. Storing an absolute path anywhere outside of
the location definition itself is wrong, broken, and evil.

Karl

[Martin Bowes]

So here is the latest position....

Having fixed a few bugs in my configuration file editor...

Realising that I just can't edit the hard-coded paths stored in the journal by a sysmod I decided to play with a recovery where a sysmod had not been performed on the test database.

So I created a journaled table in a database, inserted some initial data, did a backup, and then updated the table and exited the database. Using auditdb I confirmed the update was in the journals.

I transferred the journals etc to the DRH.

I edit the paths in the config and saved config files and do a recovery.

The recovery completes successfully...or at least claiming success.

The table with the initial data has been recovered, but the journalled updates have not been recovered.

Examining the recovery log I find that the recovery stopped at a transaction with LSN 4CE13251,0B20E2BE. Examining the audit trail on the source host we see that the updates start with a transaction LSN 4CE13251,0B20E2C3.

The audit trail on the DRH also refuses to display the updates. Yet the journal files on source and DRH are identical.

Ideas please.

FYI. The clock on the DRH is about 4minutes ahead of the clock on the source host.

Marty


-----Original Message-----
From: Karl Schendel [mailto:sche...@kbcomputer.com]
Sent: 02 December 2011 14:35
To: Ingres and related product discussion forum
Subject: Re: [Info-Ingres] Editing a configuration file