jinja2 export templates

[David Scanlan]

Hi there

I'm running netbox 2.6.1 on CentOS 7.

I'm currently looking at using export templates to report back on a few things, but I can't really find any documentation regarding them.

It seems that Jinja2 is the best way of doing this for various reason, but I have 2 specific questions:-

1) Does the Jinja2 process in Netbox support base templates? If so, where should they be located?

2) If there a way of introducing conditional logic working with tags at all? More specifically, how would I check if a tag "SaveConfig" was associated with a device?

I'm sorry to be asking what are probably very basic questions, but until I installed Netbox a month ago, I'd never used Python and Django, and I've still never used Jinja...

Many thanks

[David Socha]

David,

Export templates are a bit tricky to work with.  I have 2 built, 1 for devices and one for virtual machines.  They are both in the Django language.

For devices, the template code is 

"Site","Tenant","Rack","Device Name","Manufacturer","Device Type","Serial Number","Asset Tag","Role"

{% for device in queryset %}{% if device.device_role.name != "" and device.device_role.name != "Access Switch" and device.device_role.name != "Cable Management" and device.device_role.name != "Console Server" and device.device_role.name != "Network Card" and device.device_role.name != "Patch Panel" and device.device_role.name != "PBX" and device.device_role.name != "PBX Expansion Card" and device.device_role.name != "PDU" and device.device_role.name != "Shelving Unit" %}

"{{ device.site }}","{{ device.tenant }}","{{ device.rack }}","{{ device.name }}","{{ device.device_type.manufacturer }}","{{ device.device_type }}","{{ device.serial }}","{{ device.asset_tag }}","{{device.device_role}}"{% endif %}{% endfor %}

for VMs, the code is

"Site","Tenant","Cluster","Cluster Type","Virtual Machine Name","Role"

{% for virtualmachine in queryset %}"{{ virtualmachine.cluster.site }}","{{ virtualmachine.tenant }}","{{ virtualmachine.cluster }}","{{ virtualmachine.cluster.type }}","{{ virtualmachine.name }}","{{ virtualmachine.role}}"

{% endfor %}

I have attached the model documentation for devices and virtualization.  I pulled the models out of the code on github.  You will probably have to dig through it a bit to find the right model that contains tags.

You can see a bit how I have used some conditional logic on my device template.  Unfortunately, I haven't found a way to reduce the query with conditions, I have to iterate through each item, and perform the conditional logic on each one.

Hopefully this was somewhat helpful.

David

[David Scanlan]

Hi David

Many thanks for your help.

I think you can reduce the size of the source data by using manual filters for any view (ie for devices, select only devices for a particular site, then run the export).

However, I want to do this for management. I've always found that Management are always lazy when it comes to things, unless they are telling somebody else to do it! And this is wanted by senior management, who are the laziest of the lot!!! :D :D :D 

A part of the reason I want to use Jinja is because it's supposed to be more efficient than Django Template language, and some of the exports could get quite big once we've started populating it in earnest...

And my understanding of Python is still too noobish to be able to decipher how to use Tags for conditional data selection, from the original source code... Still, I will have a dig in what you've highlighted and maybe I will get a lightbulb moment!

Cheers

[David Scanlan]

So, I've now discovered the "Templates" directory, which contains lots of html templates!!! They are generally of the Django Template language/Jinja structure.

To do a bit of testing, I've tried a single line template:-

{% extends '_base.html' %}

I've also tried

{% extends '/opt/netbox/netbox/templates/_base.html' %}

I've tried defining this test export as Jinja2 and Django.

I've also deliberately misspelled the filename.

When trying to use Jinja I get the error message:-

There was an error rendering the selected export template (test): no loader for this environment specified

When defining it as Django, I get:-

There was an error rendering the selected export template (test): argument of type 'NoneType' is not iterable

Now, I really did expect to see a different error message if it couldn't find the misspelled file, so I'm sort of assuming that I have no access to the template files in the first place. Is there any way to be able to use these please? Or at least tell me which folder my base template might be stored in so I can populate them in?

Cheers

[Brian Candler]

I don't think export templates can use template inheritance.  In any case, I don't see how it would be that helpful - it's really intended for HTML layouts where you have headers and footers that are the same on every page.

Regarding defining filters in the export template, you might be able to use django ORM filters inline: I haven't tried it, but you could try e.g.

{% for device in queryset.filter(device_role__name='AccessSwitch') %} ...

{% for device in queryset.exclude(device_role__name__in=['Access Switch', 'Cable Management', 'Console Server']) %} ...

Finally, for tags the question was asked here before:

https://groups.google.com/forum/#!topic/netbox-discuss/VZuYfRMxTFo

Try:

{% if 'test' in obj.tags.names() %}...{% endif %}

[David Scanlan]

Hi Brian

Thank you for the reply.

The reason I was looking for template inheritance is that our managers are likely to almost use the export process as a quick online reporting tool for very specific things relating to things that aren't visible on main views. I wanted to have both a "csv" option for people wanting to do real stuff, and an HTML variant for the management to use if they wanted to check something quickly. When doing this, I want them to see a 'standard' look and feel. With inheritance, it means that I have to cut-and-paste everything all the time. It's just more cumbersome.

I hadn't seen the "tags" query, but that works great.

[Brian Candler]

I see: so you want to use export templates to generate pretty HTML pages.

I had a quick look into this.  The problem is that Netbox uses jinja2.Environment().from_string(...) to parse templates stored in the model.  It says here:

Using a template loader rather than passing strings to Template or Environment.from_string() has multiple advantages. Besides being a lot easier to use it also enables template inheritance.

There is a 'loader' attribute on Environment; maybe setting it to an instance of FileSystemLoader would work.  You could try something like this (completely untested):

--- netbox/extras/models.py.orig 2019-09-04 20:37:21.221751307 +0000

+++ netbox/extras/models.py 2019-09-17 16:19:49.180264520 +0000

@@ -12,7 +12,7 @@

 from django.template import Template, Context

 from django.urls import reverse

 import graphviz

-from jinja2 import Environment

+from jinja2 import Environment, FileSystemLoader

 from taggit.models import TagBase, GenericTaggedItemBase

 from dcim.constants import CONNECTION_STATUS_CONNECTED

@@ -467,7 +467,9 @@

             output = template.render(Context(context))

         elif self.template_language == TEMPLATE_LANGUAGE_JINJA2:

-            template = Environment().from_string(source=self.template_code)

+            template = Environment(

+                loader=FileSystemLoader("/opt/netbox/netbox/templates/export")

+            ).from_string(source=self.template_code)

             output = template.render(**context)

         else:

If it works, you could raise it as a FR.  (To do it properly, it would need to be modified to use TEMPLATES_DIR from netbox/settings.py).  But it's not a great solution IMO, as it means parts of your export templates would be in the database, and parts in the filesystem.

So what would probably be better is to write a custom Loader (subclass of BaseLoader), which is able to load other templates by name from the export templates table in the database.

[David Scanlan]

Thank you for this Brian.

I will have a look at testing next week hopefully, and see how it works out.

Cheers