Trying to process IP addresses in a template using ipaddress. Can't seem to reference them.

[Daniel Flick]

I am VERY NEW to Python (or any programming) and I may be missing a basic concept here.  I have asked this question on Stack Overflow and have read the Mako documentation and searched on Google but I am really stuck.

I am building router configurations from a spreadsheet and I would like to be able to process the IP address so I don't have to add columns for IP, subnet, subnet mask, etc to the input file.  I have a working template now but the CSV is really big and I would like to make it smaller to reduce errors and let Python do some of the grunt work.

For example, my CSV contains variables BGP_AS and LAN_IP with values of '65530" and '192.168.1.1/25' respectively. so that the following variables are set:

BGP_AS = 65530

LAN_IP = 192.168.1.1/25

My template has a format like:

router bgp ${BGP_AS}
network ${get_subnet(LAN_IP)} mask ${get_netmask(LAN_IP)}

And I would expect the output from the template to be:

router bgp 65530

network 192.168.1.0 mask 255.255.255.128

I can successfully validate this in a template:

<%def name="get_netmask(ip_string)"><%
    import ipaddress
    return ipaddress.IPv4Interface(ip_string).netmask
%></%def>
<%def name="get_address(ip_string)"><%
    import ipaddress
    return ipaddress.IPv4Interface(ip_string).ip
%></%def>
<%def name="get_network(ip_string)"><%
    import ipaddress
    return ipaddress.IPv4Interface(ip_string).network
%></%def>

But when I try any type of reference, I get an error.  

${get_address(LAN_IP)}
${get_address(${LAN_IP})}

I am sure I am missing something simple or maybe doing this another way would make more sense.  I searched the posts for "def" and found the nettime post which seems to work but that has no input function so my guess is that I am doing something wrong when either passing variables or referencing them.  

[Jonathan Vanasco]

a few things here -- but without sharing your error it's a crapshoot knowing what is wrong.

1. use a compile-time import on your template.  put this block at the top of your file>

<%!

    import ipaddress

%>

that will import the ipaddress module into your template's namespace at compile time, instead of each render or function call.

the function you want do define is:

<%def name="get_address(ip_string)">

    <%
        return ipaddress.IPv4Interface(ip_string).ip
    %>

</%def>

and it should be called with:

    ${get_address(ip_string)}

there's a gotcha though - the ip_string needs to be a unicode string. this is automatic in python 3, but not in python 2.  if you're in python 2, you want to call:

    ${get_address(unicode(ip_string))}

or 

    return ipaddress.IPv4Interface(unicode(ip_string)).ip

[Daniel Flick]

<%!
    ## python module-level code
    import ipaddress
%>


<%def name="get_address(LAN_IP)">
    <%
    return ipaddress.IPv4Interface(LAN_IP).ip
    %>
</%def>


${get_address(LAN_IP)}

I get this error:

Invalid template, please correct the following error: Template Attribute error: Undefined

If I remove the call, it works:

<%!
    ## python module-level code
    import ipaddress
%>


<%def name="get_address(LAN_IP)">
    <%
    return ipaddress.IPv4Interface(LAN_IP).ip
    %>
</%def>

That works.

[Jonathan Vanasco]

can you share the full stack trace?

[Jonathan Vanasco]

adding...

in this version,

    ${get_address(LAN_IP)}

is sending the variable LAN_IP in the main template scope to get_address

and this line..

    <%def name="get_address(LAN_IP)">

    <%
    return ipaddress.IPv4Interface(LAN_IP).ip
    %>
    </%def>

is creating a locally scoped variable called `LAN_IP`

[Daniel Flick]

I am not sure how to get that.  I am plugging that config into this front end tool:

https://github.com/hoelsner/network-config-generator

That is the tool providing the error.  It is my understanding that the tool takes the template and variables and runs it through the render function.  

I may have gotten a bit farther though.  I tried this for a template:

<%!
    ## python module-level code
    import ipaddress
%>

<%def name="get_address(ip_string)">
    <%
    return ipaddress.IPv4Interface(ip_string).ip
    %>
</%def>
! Variable Input: ${LAN_IP}
${get_address(LAN_IP)}

and now I get Invalid template, please correct the following error: Template Attribute error: Expected 4 octets in 'test'

Since 'test' is no where in my template, I think this is a bug in the front end code.

[Daniel Flick]

I am hoping that the developer gets back to me but thanks so much for the help!

[Daniel Flick]

The problem was with the validation code.  Within the python section of the template, the class IPv4Interface will throw an exception due to the invalid value during the validation process. Therefore, the server rejects the form data and the template is not created.

Solution: It would work if you add an error handling to the python section of the template.

    <%!

        ## python module-level code

        import ipaddress

    %>

    <%def name="get_address(ip_string)">

        <%

        try:

            return ipaddress.IPv4Interface(ip_string).ip

        except Exception:

            return "FAIL_OR_EMPTY"

        %>

    </%def>

    ! Variable Input: ${LAN_IP}

    ${get_address(LAN_IP)}

Explanation: During the server-side validation of the HTML form, the configuration template is rendered with a dummy parameter set to verify the syntax (see file app/forms.py, class ConfigTemplateForm). Your config template is validated with the following parameter set during the form validation:

    {

        "hostname": "test",

        "LAN_IP": "test"

[Daniel Flick]

One more issue but it may be something obvious I am missing.  The template is working but the referenced opjects show up on a new line.  I tried using a "\" and a "\\" to strip the new line but that did not work (I got a \ or \\ in the output).  Here is the template:

<%!
    ## python module-level code
    import ipaddress
%>


<%def name="get_address(ip_string)">
    <%

    try:
        return ipaddress.IPv4Interface(ip_string).ip
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>

<%def name="get_mask(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).netmask
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>


<%def name="get_hostmask(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).hostmask
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>


<%def name="get_net(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).network.network_address
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>


<%def name="get_network(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).network
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>
#### DEFINATIONS END ####


ip address ${get_address(LAN_IP)} ${get_mask(LAN_IP)}
ip address ${get_address(WAN_IP)} ${get_mask(WAN_IP)}

and here is the output:

ip address 
    10.117.137.171 
    255.255.255.0
ip address 
    10.200.2.67 
    255.255.255.192

The big bit of whitespace before the output is a clue that something is amiss.  I would like the "ip address"  IP and subnet to be on the same line.  Any ideas?

[Jonathan Vanasco]

The whitespace in your output  is because of the whitespace in the defs.  

 

<%def name="get_network(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).network
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>

Option 1:

Since you have VERY simple defs here, it may be better to write them as python functions and inject them into the templates.  But you're using a framework which may not allow that, so...

Option2:

I think this should work::

[Jonathan Vanasco]

I'm sorry, I didn't have coffee this morning and just realized I forgot Option3, which is probably the easiest in your situation -- 

since your code is just Python code - and not about printing HTML - you can just define the functions in a python block.  They are part of the modules namespace and won't run into whitespace issues.

so it would look like this:

<%!
    ## python module-level code
    import ipaddress
    

    def get_address(ip_string):
       try:
          return ipaddress.IPv4Interface(ip_string).ip

[Daniel Flick]

Option 3 seems to work.  Thanks again!

I will bundle this snippet with the rest and have another go at it.  I owe you one!

[Daniel Flick]

Worked like a champ!  I am back in business.  There is still so much to learn!

Thanks again,

Daniel

Daniel Flick

Monday, October 30, 2017 6:24 PM

Option 3 seems to work.  Thanks again!

I will bundle this snippet with the rest and have another go at it.  I owe you one!

On Monday, October 30, 2017 at 1:39:38 PM UTC-5, Jonathan Vanasco wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Mako Templates for Python" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mako-discuss/usUBe2g3ykY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mako-discuss...@googlegroups.com.
To post to this group, send email to mako-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/mako-discuss.
For more options, visit https://groups.google.com/d/optout.

Jonathan Vanasco

Monday, October 30, 2017 1:39 PM

I'm sorry, I didn't have coffee this morning and just realized I forgot Option3, which is probably the easiest in your situation -- 

since your code is just Python code - and not about printing HTML - you can just define the functions in a python block.  They are part of the modules namespace and won't run into whitespace issues.

so it would look like this:

<%!
    ## python module-level code
    import ipaddress
    
    def get_address(ip_string):
       try:
          return ipaddress.IPv4Interface(ip_string).ip
       except Exception:
          return "FAIL_OR_EMPTY"
%>

--
You received this message because you are subscribed to a topic in the Google Groups "Mako Templates for Python" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mako-discuss/usUBe2g3ykY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mako-discuss...@googlegroups.com.
To post to this group, send email to mako-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/mako-discuss.
For more options, visit https://groups.google.com/d/optout.

Jonathan Vanasco

Monday, October 30, 2017 11:42 AM

The whitespace in your output  is because of the whitespace in the defs.  

 

Option 1:

Since you have VERY simple defs here, it may be better to write them as python functions and inject them into the templates.  But you're using a framework which may not allow that, so...

Option2:

I think this should work::

<%def name="get_network(ip_string)"><%
    try:
        return ipaddress.IPv4Interface(ip_string).

network
    except Exception:
        return "FAIL_OR_EMPTY"
%></%def>

 

--
You received this message because you are subscribed to a topic in the Google Groups "Mako Templates for Python" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mako-discuss/usUBe2g3ykY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mako-discuss...@googlegroups.com.
To post to this group, send email to mako-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/mako-discuss.
For more options, visit https://groups.google.com/d/optout.

Daniel Flick

Sunday, October 29, 2017 9:41 PM

One more issue but it may be something obvious I am missing.  The template is working but the referenced opjects show up on a new line.  I tried using a "\" and a "\\" to strip the new line but that did not work (I got a \ or \\ in the output).  Here is the template:

<%!
    ## python module-level code
    import

ipaddress
%>


<%def name="get_address(ip_string)">
    <%

    try:
        return ipaddress.IPv4Interface(ip_string).ip
    except Exception:
        return "FAIL_OR_EMPTY"

    %>
</%def>

<%def name="get_mask(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).

netmask
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>

<%def name="get_hostmask(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).

hostmask
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>

<%def name="get_net(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).network.

network_address
    except Exception:
        return "FAIL_OR_EMPTY"
    %>
</%def>

<%def name="get_network(ip_string)">
    <%
    try:
        return ipaddress.IPv4Interface(ip_string).

network
    except Exception:
        return "FAIL_OR_EMPTY"
    %>

</%def>
#### DEFINATIONS END ####


ip address ${get_address(LAN_IP)} ${get_mask(LAN_IP)}
ip address ${get_address(WAN_IP)} ${get_mask(WAN_IP)}

and here is the output:

ip address 
    10.117.137.171 
    255.255.255.0
ip address 
    10.200.2.67 
    255.255.255.192

The big bit of whitespace before the output is a clue that something is amiss.  I would like the "ip address"  IP and subnet to be on the same line.  Any ideas?

On Thursday, October 19, 2017 at 11:48:03 AM UTC-5, Daniel Flick wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Mako Templates for Python" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mako-discuss/usUBe2g3ykY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mako-discuss...@googlegroups.com.
To post to this group, send email to mako-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/mako-discuss.
For more options, visit https://groups.google.com/d/optout.

Daniel Flick

Thursday, October 19, 2017 11:48 AM

The problem was with the validation code.  Within the python section of the template, the class IPv4Interface will throw an exception due to the invalid value during the validation process. Therefore, the server rejects the form data and the template is not created.

Solution: It would work if you add an error handling to the python section of the template.

    <%!

        ## python module-level code

        import ipaddress

    %>

    <%def name="get_address(ip_string)">

        <%

        try:

            return ipaddress.IPv4Interface(ip_string).ip

        except Exception:

            return "FAIL_OR_EMPTY"

        %>

    </%def>

    ! Variable Input: ${LAN_IP}

    ${get_address(LAN_IP)}

Explanation: During the server-side validation of the HTML form, the configuration template is rendered with a dummy parameter set to verify the syntax (see file app/forms.py, class ConfigTemplateForm). Your config template is validated with the following parameter set during the form validation:

    {

        "hostname": "test",

        "LAN_IP": "test"

    }

On Tuesday, October 17, 2017 at 8:35:45 PM UTC-5, Daniel Flick wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Mako Templates for Python" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mako-discuss/usUBe2g3ykY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mako-discuss...@googlegroups.com.
To post to this group, send email to mako-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/mako-discuss.
For more options, visit https://groups.google.com/d/optout.