Generate config file from template using Python search and replace.

[Mr Zaug]

I need to generate a config file based on an existing "template" file. I need to replace a set of strings with other strings globally in the generated file.

Here is a snippet of the template file, where CONTENT_PATH and DAMPATH are two "placeholders" or variables. There are several other such placeholders.

$include "_dispatcher_publish_filters.any"
/1000 { /type "allow" /glob "* /CONTENT_PATH/*.html*" }
/1001 { /type "allow" /glob "POST /DAMPATH/www/*.html *" }

The script's user will be asked to type in unique values when prompted for DAMPATH or CONTENT_PATH.

Since I know the variables themselves are not going to change (because the contents of the template file don't spontaneously change) should I be using regex to search for them or is there a better way? I was planning on using re.sub but I don't know whether that's the best way. Here's what my script looks like today.

from sys import argv
import re
from os.path import exists

script, template_file = argv
print "Opening the template file..."

in_file = open(template_file)
lines = in_file.readlines()

print "What is the serial number of the site?",
_NNN = raw_input()

print "What is the brand, or product name?",
_BRAND = raw_input()

print "What is the content path?",
_CONTENT_PATH = raw_input()

out_file = open(_nnn + _brand + "_farm.any", 'w')

for line in lines:
re.sub('NNN', _NNN, line)
re.sub('BRAND, _BRAND', line)
re.sub('CONTENT_PATH', _CONTENT_PATH, line)

out_file.close()

[Mr Zaug]

I should mention the template file is small, just 98 lines long and the working config file will be the same size.

[Peter Otten]

There are many templating systems out there. Pick the one that suits you
best. A very basic one is str.format():

>>> "{foo} {bar}".format(foo=1, bar=2)
'1 2'

Here's what your script might become if you choose that one:

$ cat interactive_template.txt
$include "_dispatcher_publish_filters.any"
/1000 {{ /type "allow" /glob "* /{CONTENT_PATH}/*.html*" }}
/1001 {{ /type "allow" /glob "POST /{DAMPATH}/www/*.html *" }}
$ cat interactive_template.py

try:
format_map = str.format_map
except AttributeError: # python 2 compatibility
import string

def format_map(text, lookup):
return string.Formatter().vformat(text, (), lookup)
input = raw_input


class Lookup(dict):
def __missing__(self, key):
self[key] = value = input("{}: ".format(key))
return value


def main():
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("template_file")
parser.add_argument("generated_file", nargs="?")
args = parser.parse_args()

with open(args.template_file) as instream:
template_text = instream.read()

lookup = Lookup()
generated_text = format_map(template_text, lookup)
generated_file = args.generated_file
if generated_file is None:
generated_file = format_map("{nnn}{brand}_farm.any", lookup)
print("Writing {}".format(generated_file))
with open(generated_file, "w") as outstream:
outstream.write(generated_text)


if __name__ == "__main__":
main()
$ python interactive_template.py interactive_template.txt
CONTENT_PATH: foo
DAMPATH: bar
nnn: baz
brand: ham
Writing bazham_farm.any
$ cat bazham_farm.any
$include "_dispatcher_publish_filters.any"
/1000 { /type "allow" /glob "* /foo/*.html*" }
/1001 { /type "allow" /glob "POST /bar/www/*.html *" }

[Mr Zaug]

When I run this script on OS X El Capitan, I see,

# permission sensitive cache
$include "_dispatcher_shared_auth-checker:

Was I supposed to incorporate it into the script I posted?

[Peter Otten]

Are you referring to my post? I'm sorry, I can't make sense of your
question.

[Mr Zaug]

Yes. The snippet you posted went way over my head. When I ran it, it printed

[Mr Zaug]

On Sunday, November 29, 2015 at 5:50:51 PM UTC-5, Peter Otten wrote:

I seem to be heading in this direction.

#!/usr/bin/env python

import re
from os.path import exists

script, template_file = argv
print "Opening the template file..."

with open (template_file, "r") as a_string:
data=a_string.read().replace('BRAND', 'Fluxotine')
print(data)

So now the challenge is to use the read().replace magic for multiple values.

[Rob Hills]

A program I am writing at present does exactly this and I simply do
multiple calls to string.replace (see below)

On 30/11/15 10:31, Mr Zaug wrote:
> I seem to be heading in this direction.
>
> #!/usr/bin/env python
> import re
> from os.path import exists
>
> script, template_file = argv
> print "Opening the template file..."
>
> with open (template_file, "r") as a_string:
> data=a_string.read().replace('BRAND', 'Fluxotine')

data=data.replace('STRING_2', 'New String 2')
data=data.replace('STRING_3', 'New String 3')

> print(data)
>
> So now the challenge is to use the read().replace magic for multiple values.

It's crude, but it works well for me!

--
Rob Hills
Waikiki, Western Australia

[MRAB]

On 2015-11-30 02:40, Rob Hills wrote:
> A program I am writing at present does exactly this and I simply do
> multiple calls to string.replace (see below)
>
> On 30/11/15 10:31, Mr Zaug wrote:

>> I seem to be heading in this direction.
>>
>> #!/usr/bin/env python
>> import re
>> from os.path import exists
>>
>> script, template_file = argv
>> print "Opening the template file..."
>>
>> with open (template_file, "r") as a_string:
>> data=a_string.read().replace('BRAND', 'Fluxotine')
>

> data=data.replace('STRING_2', 'New String 2')

> data=data.replace('STRING_3', 'New String 3')

>
>> print(data)
>>
>> So now the challenge is to use the read().replace magic for multiple values.
>

> It's crude, but it works well for me!
>

You do need to watch out for matches that are part of something else.

For example, if you had this:

template = "REPLACE_THIS but DONT_REPLACE_THIS"

and you did:

result = template.replace("REPLACE_THIS", "Python")

you would get:

"Python but DONT_Python"

[Peter Otten]

It's hard to tell from that problem description what might have gone wrong.
However:

In your template file you have to enclose all words you want to replace in
braces ("you have foo options" becomes "you have {foo} options"), to replace
all literal { with {{ and all literal } with }}.

Did you do that?

[Mr Zaug]

Yup, I sure did. So here is my current script. The only remaining task now is figuring out how to write the contents to a new file.

#!/usr/bin/env python
import os
import sys

script, template_file = sys.argv

print "Opening the template file..."

print "What is the serial number of the site?",

nnn = raw_input()

print "What is the brand, or product name?",

brand = raw_input()

print "What is the (fqdn) ServerName?",
server_name = raw_input()

print "What is the content path?",

content_path = raw_input()

print "What is the DAM path?",
dampath = raw_input()

print "Which environment is this for?",
env = raw_input()

print "What is the cache document root?",
cache_docroot = raw_input()

with open (template_file, "r") as a_string:

data=a_string.read().replace('{SERVER_NAME}', server_name).replace('{BRAND}', brand).replace('{CONTENT_PATH}', content_path).replace('{DAMPATH}', dampath).replace('{ENV}', env).replace('{CACHE_DOCROOT}', cache_docroot)

[Peter Otten]

[...]

> with open (template_file, "r") as a_string:
> data=a_string.read().replace('{SERVER_NAME}',
> server_name).replace('{BRAND}', brand).replace('{CONTENT_PATH}',
> content_path).replace('{DAMPATH}', dampath).replace('{ENV}',
> env).replace('{CACHE_DOCROOT}', cache_docroot)

If all other braces are properly escaped you can use format instead of
replace:

data = a_string.read().format(
SERVER_NAME=server_name,
BRAND=brand,
CONTENT_PATH=content_path
DAMPATH=dampath,
ENV=env,
CACHE_DOCROOT=cache_doc_root)

[Mr Zaug]

Oh, that's much easier to read. Thanks!

[Mr Zaug]

Actually, I don't understand what you mean by "all other braces." What braces are you talking about? The placeholders in the template file (the file being read in) have braces around them but they are not escaped.

Also, do I really need curly braces to tell Python what my placeholders are?

[Peter Otten]

Mr Zaug wrote:

> Actually, I don't understand what you mean by "all other braces." What
> braces are you talking about? The placeholders in the template file (the
> file being read in) have braces around them but they are not escaped.

The example you provide was

$include "_dispatcher_publish_filters.any"
/1000 { /type "allow" /glob "* /CONTENT_PATH/*.html*" }
/1001 { /type "allow" /glob "POST /DAMPATH/www/*.html *" }

If you want to use string formatting to replace CONTENT_PATH and DAMPATH in
the above excerpt you have to change the template to

$include "_dispatcher_publish_filters.any"
/1000 {{ /type "allow" /glob "* /{CONTENT_PATH}/*.html*" }}
/1001 {{ /type "allow" /glob "POST /{DAMPATH}/www/*.html *" }}

> Also, do I really need curly braces to tell Python what my placeholders
> are?

Let's take a slightly modified example to answer that one:

$include "_dispatcher_publish_filters.any"
/1000 { /type "allow" /glob "* /CONTENT_PATH/*.html*" }
/1001 { /type "allow" /glob "POST /DAMPATH/www/*.html *" }

/1002 { /type "allow" /glob "* /ALTERNATIVE_CONTENT_PATH/*.html*" }


If you try to fill in the CONTENT_PATH with

data = data.replace("CONTENT_PATH", "foo")

the result will be

$include "_dispatcher_publish_filters.any"
/1000 { /type "allow" /glob "* /foo/*.html*" }

/1001 { /type "allow" /glob "POST /DAMPATH/www/*.html *" }
/1002 { /type "allow" /glob "* /ALTERNATIVE_foo/*.html*" }

Look at the butchered line starting with /1002, a problem that MRAB already
pointed out before. The braces are not necessary if the string you are
replacing doesn't occur elsewhere in the template, but they make the
replacement process both flexible and unambiguous.

[Mr Zaug]

That makes sense.

So I still can't see how to write the string object to a file whist naming the file with whatever values I provided for the NNN and BRAND variables.

Printing the contents of the string object is working with all the expected substitutions. Do I need to convert the string object into a file before I write it? Or do I need to open a new file and somehow stuff the string object into it?

[Peter Otten]

Yes, open a new file and write the string using the write() method. You can
use string formatting to build the filename, too:

text = ... # the string you want to write

nnn = raw_input("What is the serial number of the site? ")
brand = raw_input("What is the brand, or product name? ")

filename = "{NNN}{BRAND}_farm.any".format(BRAND=brand, NNN=nnn)
with open(filename, "w") as outstream:
outstream.write(text)

[Mr Zaug]

Ye, this does work. Many thanks!

filename = "{NNN}_{BRAND}_farm.any".format(BRAND=brand, NNN=nnn)

with open(filename, "w") as outstream:

outstream.write(data)