What types of resources are appropriate to store in a NoSQL database?

[Benjamin Polen]

Hi,

What types of resources are appropriate to store in a NoSQL database?

I'm considering using traversal for my next web project.  If I have a NoSQL

database behind my traversal interface, what web resources should I store there.

For example, should html files, jinja2 templates, javascript files

images, css be stored in my resource tree?

Thanks,

Ben

[Steve Piercy]

That's a very general question, not specific to Pylons Project
projects. You might get more information from a search engine.

https://duckduckgo.com/?q=what+should+i+store+in+a+nosql+database

In very general terms, only data should be stored in a database,
not your application's code, which includes templates and static assets.

--steve


On 7/26/18 at 10:08 AM, bpol...@gmail.com (Benjamin Polen) pronounced:

------------------------
Steve Piercy, Eugene, OR

[Mike Orr]

These don't actually answer the question; they describe the
differences between SQL and NoSQL databases.

The question of what to store in a database is somewhat similar to
when to use traversal. Why are you using traversal? Because it's cool,
or because your data structure fits well with its paradigm? While you
can use traversal anytime, its main strength comes when it's
unpredictable beforehand which URLs will be which resource types. In
other words, when users can create arbitrary URLs and folder with
multiple document types, and you can't predict beforehand which
structure they'll choose. In that case you might have resource types
for JPG, PNG, HTMLFile, Folder, etc, and you calculate the context
class based on the filename extension. You don't know where the JPGs
are or which ones are JPGs, but traversal can figure it out.

So, what do you put in a database? User-created data. And
preinitialized but user-editable data. Maybe a huge number of canned
records that you want to search and display on demand. (I have a
couple applications with this, where the application is essentially a
viewer for a large chemical database.)

Don't put templates into the database unless they meet this criteria.
E.g., if they're user-edited, or somehow semantically belong as
"data". If they'll be stable for a long time and only application
developers will ever edit them, put them in a regular templates
directory. That also makes it easier to edit them, since vim can
natively edit and syntax-highlight a file but needs an opener plugin
for a database object. Which you then have to write to a temporary
file, and it has a useless filename, not convenient for debugging,
etc.

The only other reason to put templates in a datbase is if you're
making a framework like Zope, where "application developers" will
create templates that you can't predict. In that case, everything
about the site is data, everything is user-edited. (From your
perspective they're users. From their perspective they're application
admins, and they have users you don't know about, even if they use an
"authentication/authorization framework" that you provide.) So this is
a straightforward case of user-edited data.

But don't try to recreate Zope if you don't have to! That will
inevitably lead to overengineering that will be more trouble than it's
worth. K.I.S.S. (Keep It Simple Stupid.) Put everything in the code
except things that vary per deployment (which go in the deployment
config) or is user-editable, semantically "data", or temporary caching
(which go in a database).

By the way, I love Reids for its automatic-expiration feature: I can
set an expire time and forget about the object, and it will
automatically be deleted when it expires. No need to run a cleanup
routine or have items remain forever overstaying their welcome.

> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discus...@googlegroups.com.
> To post to this group, send email to pylons-...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/r480Ps-10126i-B34B8613FB90482C9F312FBC6DFE9FE9%40cns316-6061fsn.univ.fairfield.local.
>
> For more options, visit https://groups.google.com/d/optout.



--
Mike Orr <slugg...@gmail.com>

[Jonathan Vanasco]

On Thursday, July 26, 2018 at 2:30:31 PM UTC-4, Benjamin Polen wrote:

what web resources should I store there.

None. 

On Thursday, July 26, 2018 at 2:30:31 PM UTC-4, Benjamin Polen wrote:

For example, should html files, jinja2 templates, javascript files images, css be stored in my resource tree?

No.

The only time to store those types things in nosql is when you're storing "user overrides" that are implemented via a customization or whitelabel system and the data HAS to be queried. In those scenarios you can't avoid the overhead that nosql lookups and IO cost.

In every other situation, you want to leverage things like:
* built-in caching of template libraries
* filesystem IO optimizations of web servers and operating systems

[Benjamin Polen]

These last two answers are really helpful.  Thanks alot.

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/212939ff-b1f3-45e6-8804-a40b8717ce95%40googlegroups.com.