Accessing attributes from external data sources

10 views
Skip to first unread message

S. Qiouyi Lu

unread,
Dec 19, 2024, 10:30:37 PM12/19/24
to nanoc
Hi,

I'm trying to create a directory of characters using the external data sources documentation here: https://nanoc.app/doc/guides/using-external-sources/

It's working fine for the main directory page, but this is where I'm running into an issue:

> Finally you have to stop Nanoc from writing out pages for every employee item provided by the data source.

I actually want this behavior from Nanoc so that each character has a profile page with their name, description, and works that they appear in.

My Rules for laying out the pages is creating outputs in the right location:

# Lay out database pages
compile '/character/*' do
filter :erb
layout '/default.*'
write @item.identifier.without_ext + '/index.html'
end

However, when I try to add an if statement to the layout page to get the title to display @item[:full_name] instead of my default @item[:title], the attribute full_name does not seem to exist.

Here is characters_db.rb:

require 'sequel'

class CharactersDataSource < ::Nanoc::DataSource
identifier :character

def up
@db = Sequel.sqlite('lib/data/characters.db')
end

def down
@db.disconnect
end

def items
@db[:characters].map do |character|
new_item(
'',
character,
"/character/#{character[:slug]}"
)
end
end
end

Here is the sorted characters helper:

def sorted_characters
characters = @items.find_all('/character/*')
characters.sort_by do |c|
[ c[:full_name] ]
end
end

And here is the layout code that is not working:

<h1 id="page-title" tabindex="-1">
<% if @item[:full_name] %>
<a href="<%= item.path %>" title="[permalink]"><%= @item[:full_name] %>
<% if item[:alt_lang] && item[:alt_name] %>
<span class="alt-name" lang="<%= item[:alt_lang] %>"><%= item[:alt_name] %></span>
<% end %>
</a>
<% else %>
<%= link_to(@item[:title], @item, title: '[permalink]') %>
<% end %>
</h1>

This is made more complicated by some characters having alternate names in non-Latin scripts (Chinese, Japanese, Greek) where I want the name surrounded in a proper lang tag.

I could theoretically copy name data into a new title column in the database, but that seems redundant when it seems like I'm either accessing the attributes array incorrectly, or when I could copy the attributes array into @item.attributes.

All the Ruby I know is from messing around with Nanoc, so I'm not sure if it's Ruby logic or Nanoc logic that I'm missing here. Or SQL logic, which I also don't know much about.

Any help would be greatly appreciated. If I can get this to work, I can pretty much ditch WordPress, as the main benefit there was the increased complexity provided by a database structure, and I'd like to reuse this structure for larger data sources.

Best,
S.

S. Qiouyi Lu

unread,
Dec 19, 2024, 10:51:00 PM12/19/24
to nanoc
I seem to have made things work by doing the mapping directly in the new item line:

require 'sequel'

class CharactersDataSource < ::Nanoc::DataSource
identifier :character

def up
@db = Sequel.sqlite('lib/data/characters.db')
end

def down
@db.disconnect
end

def items
@db[:characters].map do |character|
new_item(
'',
{ slug: character[:slug], full_name: character[:full_name], alt_lang: character[:alt_lang], alt_name: character[:alt_name], description: character[:description] },
"/character/#{character[:slug]}"
)
end
end
end

This still seems a bit inelegant and would have to be manually updated if I were to add or change columns, though.

I am also getting the following flags in my terminal:

Caution: Data source :character does not implement #item_changes; live compilation will not pick up changes in this data source.
Caution: Data source :character does not implement #layout_changes; live compilation will not pick up changes in this data source.

Any guidance would be appreciated. Thanks!

S.

Denis Defreyne

unread,
Dec 20, 2024, 1:29:59 AM12/20/24
to na...@googlegroups.com
Hey S,

Glad you got it to work!

I seem to have made things work by doing the mapping directly in the new item line:

I’m wondering whether that is because `character` is an instance of a database row (a class of the Sequel gem), and not a hash. Could you try and see whether this works?

new_item(
  '',
  character.to_h,
  "/character/#{character[:slug]}"
)

The call to #to_h would be what does the trick, hopefully.

I am also getting the following flags in my terminal:

Caution: Data source :character does not implement #item_changes; live compilation will not pick up changes in this data source.
Caution: Data source :character does not implement #layout_changes; live compilation will not pick up changes in this data source.

I realized just now that I forgot to document these two methods. I created a ticket for it: https://github.com/nanoc/nanoc.app/issues/305.

In short, these methods return a ChangesStream which triggers Nanoc whenever a change is detected. This is how it’s implemented for the filesystem data source: https://github.com/nanoc/nanoc/blob/2f566091247beab1a8ca3f3b2040b89fc9b80d40/nanoc/lib/nanoc/data_sources/filesystem.rb#L83-L110 You could adapt it for your own needs, though listening to database changes in realtime might not be the easiest. (If you’re using SQLite you could use the `listen` gem to detect file updates to the database file.)

You don’t need to implement these methods, but without them, updates to the database won’t be detected by `nanoc live` automatically.

Hope this helps,

--
You received this message because you are subscribed to the Google Groups "nanoc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nanoc+un...@googlegroups.com.

Message has been deleted

S. Qiouyi Lu

unread,
Dec 20, 2024, 3:41:35 PM12/20/24
to nanoc
Thanks, that seems to have worked! At least, no changes happened when I recompiled, so I assume it created the same output.

As for the changes flag, basically all I would have to do is exit and re-launch nanoc live if I were to make database changes? I anticipate doing those in bulk and not live anyway, so that's not a big deal for me.

Related question: How would I access separate tables within the same database? For example, currently I just have characters.db. But I might want to change that to fiction.db, with one table for works and one table for characters. Each work has multiple characters in them; each character could be in multiple works. Currently I'm manually listing characters in the YAML header of the work and using a render page to match works that have the selected character in the list, based on slugs. But is there a way to store these values as related in SQL and generate based on that? It would probably be like the tags helper, but with specific categories of tags.

S.

Denis Defreyne

unread,
Dec 21, 2024, 4:38:05 AM12/21/24
to na...@googlegroups.com
Hey S,

As for the changes flag, basically all I would have to do is exit and re-launch nanoc live if I were to make database changes? I anticipate doing those in bulk and not live anyway, so that's not a big deal for me.

That’s correct!

How would I access separate tables within the same database? For example, currently I just have characters.db. But I might want to change that to fiction.db, with one table for works and one table for characters. Each work has multiple characters in them; each character could be in multiple works. Currently I'm manually listing characters in the YAML header of the work and using a render page to match works that have the selected character in the list, based on slugs. But is there a way to store these values as related in SQL and generate based on that? It would probably be like the tags helper, but with specific categories of tags.

I think I’d convert the data source in a generic data source that can load from multiple tables. Here’s an example that creates pages for each ice cream flavour (it’s the only thing that came to mind right now):

    class FictionDataSource < ::Nanoc::DataSource
      identifier :fiction
    
      def up
        @db = Sequel.sqlite('lib/data/fiction.db')
      end
    
      def down
        @db.disconnect
      end
    
      def items
        characters = @db[:characters].map do |row|
          …
        end
    
        ice_cream_flavours = @db[:ice_cream_flavours].map do |row|
          …
        end
    
        characters + ice_cream_flavours
      end
    end

If you have a join table between characters and ice cream flavours, you could read that information too, and append a list of ice cream flavours to the attributes of each character. (I wouldn’t create Nanoc items from the join table, as I think that’s a little hard to work with.)

In pseudocode (I haven’t used Sequel in a while so this is probably not quite correct):

    character.to_h.merge(
      favourite_ice_cream_flavours:
        @db[:characters_ice_cream_flavours]
          .join(:ice_cream_flavours, id: :ice_cream_flavour_id)
          .where(Sequel[:characters][:id] => character[:id])
          .map { |row| row[:name] }
    )

Alternatively, even better would be to have not the names, but references to the items. Something like this (again, probably not quite correct with my limited Sequel skills):

    character.to_h.merge(
      favourite_ice_cream_flavours:
        @db[:characters_ice_cream_flavours]
          .where(Sequel[:characters][:id] => character[:id])
          .map { |row| "/flavours/#{row[:ice_cream_flavour_id]}.txt" }
    )

This way, it’s easier to create links between pages or even just look up related information programmatically.

Hope this helps!
Reply all
Reply to author
Forward
0 new messages