Project Structure - Lots of scattered custom views

[chad petzoldt]

Right now my project only has 2 apps that use a database (*real* apps). The rest of the website is composed of many custom views, scattered all over the place. I am new to Django, and I havn't quite figured out how to structure my project directory just yet.

I think what I should really be looking at is the beginnings of a Content Management System. But at this point, I think that is too far over my head.The other *issue* is related to my design/develop workflow. Working completely through an internet browser just isnt possible for 80% of the project cycle. We have many programs which want to work directly on the filesystem (on an offline machine), mostly coming from the Adobe Creative Suite.

 There are all kinds of media documents and markups forming "Articles". These Articles are passed around as bundles during design, and then when they get to me I have to tie them together very neatly and push them onto the website. The one thing I do have control over is the Url layout, and the navigation of these articles.

Should I be splitting each of these "articles" into their own apps (a directory with little more than 'templates/' and 'static/'). This makes synchronization easy with the server, there is no database interaction required; the only task left after the sync is for me to update my navigation widgets (which I currently just do by hand).

Am I going the right direction here? Is there some App that helps me accomplish this task a little cleaner. I think what I really want, is an App Application (hotplugging apps or something like that).

[Mario Gudelj]

There's no reason why you can't split these into separate apps, then create a templates directory and urls.py in each app folder, if this will help the site organisation. You'll have to make sure you add each app to settings.py, create __init__.py in every app folder. Just go for it. Get everyone working on those articles to use a repo and teach them how to pull and push changes. Then you simply deploy the changes when you want to... That's just my 2 cents.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/django-users/-/yJsqstRt2JIJ.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

[Lachlan Musicman]

On Tue, Jan 8, 2013 at 9:52 AM, chad petzoldt <cpet...@gmail.com> wrote:
> Right now my project only has 2 apps that use a database (*real* apps). The
> rest of the website is composed of many custom views, scattered all over the
> place. I am new to Django, and I havn't quite figured out how to structure
> my project directory just yet.

Every project needs to be rewritten once! (or so they used to say)

> I think what I should really be looking at is the beginnings of a Content
> Management System. But at this point, I think that is too far over my
> head.The other *issue* is related to my design/develop workflow. Working
> completely through an internet browser just isnt possible for 80% of the
> project cycle. We have many programs which want to work directly on the
> filesystem (on an offline machine), mostly coming from the Adobe Creative
> Suite.
>
> There are all kinds of media documents and markups forming "Articles".
> These Articles are passed around as bundles during design, and then when
> they get to me I have to tie them together very neatly and push them onto
> the website. The one thing I do have control over is the Url layout, and the
> navigation of these articles.

You have actually just spelled it out for yourself. Make an app that
looks after Articles. Each Article can have Many to Many/One links to
files (the "bundle") as models.FileFields
(django-docs-1.4-en/ref/models/fields.html#filefield ) NB: I've never
used these and haven't read the docs thoroughly.

The phrase tie them together neatly isn't expressive enough. What are
you putting together? Are you taking InDesign files and creating
css/html/js? Are they pdfs?

> Should I be splitting each of these "articles" into their own apps (a
> directory with little more than 'templates/' and 'static/'). This makes
> synchronization easy with the server, there is no database interaction
> required; the only task left after the sync is for me to update my
> navigation widgets (which I currently just do by hand).

I think that splitting each article into it's own app sounds like a
disaster waiting to happen. A massive clusterfuck of disaster. The
idea of apps is to abstract out any commonalities to make workflow and
presentation more simple. It is possible, but you need to do the
mental abstracting. Personally I use trial and error - implement
obvious solution, make changes as more complexity is required. My
sites are simple. For more complex apps, you might want to sit down
with a text/UML editor and flesh it out first. Get other eyeballs on
the abstraction too!

Cheers
L.

> Am I going the right direction here? Is there some App that helps me
> accomplish this task a little cleaner. I think what I really want, is an App
> Application (hotplugging apps or something like that).
>

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-users/-/yJsqstRt2JIJ.
> To post to this group, send email to django...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.

--
...we look at the present day through a rear-view mirror. This is
something Marshall McLuhan said back in the Sixties, when the world
was in the grip of authentic-seeming future narratives. He said, “We
look at the present through a rear-view mirror. We march backwards
into the future.”

http://www.warrenellis.com/?p=14314

[Lachlan Musicman]

On Tue, Jan 8, 2013 at 10:30 AM, Lachlan Musicman <dat...@gmail.com> wrote:
> On Tue, Jan 8, 2013 at 9:52 AM, chad petzoldt <cpet...@gmail.com> wrote:
>> Right now my project only has 2 apps that use a database (*real* apps). The
>> rest of the website is composed of many custom views, scattered all over the
>> place. I am new to Django, and I havn't quite figured out how to structure
>> my project directory just yet.

> I think that splitting each article into it's own app sounds like a
> disaster waiting to happen. A massive clusterfuck of disaster. The
> idea of apps is to abstract out any commonalities to make workflow and
> presentation more simple. It is possible, but you need to do the
> mental abstracting. Personally I use trial and error - implement
> obvious solution, make changes as more complexity is required. My
> sites are simple. For more complex apps, you might want to sit down
> with a text/UML editor and flesh it out first. Get other eyeballs on
> the abstraction too!

I'm glad Mario was so positive. I didn't mean to be so negative. I was
imagining over 10-20 articles. At which point *I* see it as
unmanageable. *I* see it. Others might be happy with more than twenty
or so apps. It's a matter of preference. What happens if there are
over 100 apps?

Cheers
L.

[chad petzoldt]

Tie them neatly together

There are some cases where they send me Indesign, but when I export to HTML some of the layouts break. So then I have to manually rig it until I feel its close enough (but then it turns out it wasn't, so I mod again). Some imagemaps made from sliced up Photoshops with some cute rollover effects added on. And then some of the Articles form sequential "stories", so they need linking in between to make them feel seamless.

The major point is that I have small bundles of files that are very interconnected, and keeping them close and organized makes the reviewing/editing/handling process much easier. Most of my hacking has been from KomodeIDE, and then VIM/SSH (and thats even when Im doing the design work!). Im having a hard time breaking away from the direct editing.

 

> I think that splitting each article into it's own app sounds like a
> disaster waiting to happen.

 
I agree, thats why I posted here. I am already starting to see the problems... I have started breaking the DRY principle, and its not looking much better. The good news right now, is that my repeats are within close proximity on the filesytem, so Im not rooting all over the place just yet. The only major burden is a project-wide navigation map that has to be properly updated and sequenced (still doin that one by hand).

On a good note:
I did find an App called django-media-tree that made me feel a little better. It should be easy enough for my guys to feed the projects in there, and maybe I implement an Article-Bundler to help the push/pull process. And then when I need to do my editing, I can set up a temporary staging directory that checks out a bundle, I edit, and then it parses it back in.

But I still find myself creating bare apps with some crazy model defined that only has 1 table row.

[Lachlan Musicman]

>
> There are some cases where they send me Indesign, but when I export to HTML
> some of the layouts break. So then I have to manually rig it until I feel
> its close enough (but then it turns out it wasn't, so I mod again). Some
> imagemaps made from sliced up Photoshops with some cute rollover effects
> added on. And then some of the Articles form sequential "stories", so they
> need linking in between to make them feel seamless.
>
> The major point is that I have small bundles of files that are very
> interconnected, and keeping them close and organized makes the
> reviewing/editing/handling process much easier. Most of my hacking has been
> from KomodeIDE, and then VIM/SSH (and thats even when Im doing the design
> work!). Im having a hard time breaking away from the direct editing.

So it sounds like a model like this would work:

def BundleFiles(models.Model):
file = models.FileField()
articlw = models.ManyToMany(Article)
# or maybe article = models.ForeignKey(Article)

def Article(models.Model):
html = models.TextField() #for the end product
# or html = models.FileField()
cssfile = models.FileField() #this can be abstracted out to it's own
model if used across many articles
previous = models.FK(Article, blank = True, null= True)
next = models.FK(Article, blank = True, null= True)

See how these are more like folders to hold each set of info. For the
final product/page, you would need to write appropriate views and
methods that extracted the important bits and put them in the
appropriate places.

>>
>> > I think that splitting each article into it's own app sounds like a
>> > disaster waiting to happen.
>>
>
> I agree, thats why I posted here. I am already starting to see the
> problems... I have started breaking the DRY principle, and its not looking
> much better. The good news right now, is that my repeats are within close
> proximity on the filesytem, so Im not rooting all over the place just yet.
> The only major burden is a project-wide navigation map that has to be
> properly updated and sequenced (still doin that one by hand).
>
> On a good note:
> I did find an App called django-media-tree that made me feel a little
> better. It should be easy enough for my guys to feed the projects in there,
> and maybe I implement an Article-Bundler to help the push/pull process. And
> then when I need to do my editing, I can set up a temporary staging
> directory that checks out a bundle, I edit, and then it parses it back in.
>
> But I still find myself creating bare apps with some crazy model defined
> that only has 1 table row.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/django-users/-/mMaEbFr1I4QJ.

[Sanjay Bhangar]

hey all,

This seems to be an itch I've had to scratch myself in the past - let
me see if I understand the situation vaguely correctly:

You have a site that uses Django / database features for some things,
but then you also have a lot of content that's being designed in some
design tool and needs to be manually crafted into precise (sometimes
hackish) html and uploaded, and you almost want to just work with
plain html files on a file-system so you can use the tools you're used
to - Komodo / vim, version control, etc. This content is in the form
of "article bundles" which need to have some sequencing between pages.

The 'new app for every article' definitely sounds like a clusterf....
it maybe an interesting experiment, but it just *feels* wrong at so
many levels .. you do want some way to generalize this into a single
app .. but my guess is you also are unable to wrap things into common
templates and editing the html in the database / django admin is just
too messy .. even if you let individual "articles" or "article
bundles" have their own custom css that's specified some-how in the
db, and their own custom html that you some-how populate from html
files you upload. It may work, and it is perhaps the correct way to do
this. I just know the situation where you're fighting with some silly
html generated by a design tool and actually want simple html files
and css resources, and the database "structure" can feel like its
getting in the way ..

So, here is my radical (?) suggestion :) -

Have a simple model like this:
class Item(models.Model):
title = models.CharField(..)
slug = models.SlugField(..)
... any extra metadata fields you need for the "article bundles"

Then, for each item "slug" name, you create a new folder on the
file-system in some path you define in settings.py or so like
CUSTOM_HTML_PATH or anything - so, say you have a series of articles
on "Why not to do webdesign in Indesign" with a slug like
"why-not-indesign", you create a folder inside CUSTOM_HTML_PATH with
the name "why-not-indesign". Then create the various articles in that
"bundle" like 1.html, 2.html, 3.html, etc. and include all the css /
js files that specific bundle needs inside a static folder in that
directory.

Then you just need a view which takes <item_slug>, looks for the
appropriate folder on the file-system, and renders 1.html inside
whatever master template you may have. The view can also check for all
the <integer>.html files in the directory to handle next / previous.
You would probably need to do a bit of magic with STATICFILE_DIRS or
so to make sure each of the static file folders inside the directories
are collected by collectstatic so you can still serve the static files
through the web server and not through django.

Of course, I don't know the details of your work-flow so I maybe
missing something, but I do in general find the idea of storing some
things just on the file-system quite interesting, and the code to
manage it is generally pretty straightforward.

Of course, this may also be a terrible idea, but I'd also love to hear why .. :)

Cheers,
Sanjay

[Amirouche]

Héllo Chad,

I'd like to help but I need more infos if you don't mind and is granted to provide them.

On Monday, January 7, 2013 9:52:35 PM UTC+1, chad petzoldt wrote:

Right now my project only has 2 apps that use a database (*real* apps). The rest of the website is composed of many custom views, scattered all over the place. I am new to Django, and I havn't quite figured out how to structure my project directory just yet.

What do this two apps ?

The other *issue* is related to my design/develop workflow. Working completely through an internet browser just isnt possible for 80% of the project cycle. We have many programs which want to work directly on the filesystem (on an offline machine), mostly coming from the Adobe Creative Suite.

I don't understand this part, are you programming in the browser ? or is it just about the design phase of each «Articles/bundle», work prior or in parallel of programming the site ?

 

There are all kinds of media documents and markups forming "Articles". These Articles are passed around as bundles during design, and then when they get to me I have to tie them together very neatly and push them onto the website. The one thing I do have control over is the Url layout, and the navigation of these articles.

Should I be splitting each of these "articles" into their own apps (a directory with little more than 'templates/' and 'static/').

 

I'm not sure but what you describe looks like a (e)magazine with several issues (you call them Articles/bundle) where each issue has it's own layout/design - and probably navigation but it seems also there is a global menu - a cross issues menu - present on all issues.

This makes synchronization easy with the server, there is no database interaction required;

I assume by «synchronization» you mean «putting the new "Articles" in production».

 

the only task left after the sync is for me to update my navigation widgets (which I currently just do by hand). 

Can you describe this navigation widget ? Is it available over all apps/Articles/issues ?

Do the Articles gets modified after the first synchronization ? Do you have somekind of administration to edit them ?

In the other mail you say «But I still find myself creating bare apps with some crazy model defined that only has 1 table row.», could you describe one of this table/row and generalize to the other similar tables ?

Regards,

Amirouche

[chad petzoldt]

Sanjay, you were hitting things pretty close. I think that making sure slug names match up to real *static* locations is the key. I am hosting with Apache, and I thought about using some configurations to cheat a little bit, and get some of the static-files burden off of Django and let Apache resolve any requests that point to actual files on disk (with some security in mind).

What about a view that can determine if it should be pointing at a file on disk, and if so, do a custom Http_Response, and read the actually binary contents from disk by hand. Im just curious on this one; it could be an alternative means of file storage and recognition. I understand this may have some performance implications, but this is not going to break Django, correct? Is this bad Django etiquette?

Amirouche, a different way to describe what I am trying to achieve might be this: I would like to bypass the admin as much as possible when it comes to these "Articles". But when rendered, all of these articles do need at least a small HTML wrapper, and probably some kind of of global template that helps with navigation. If I could handle navigation without the admin, i would be very excited, but it does seem that I will need to use the admin for at least this purpose. The navigation would probably help render hierarchical  "menus" for use as templates within the Articles. I would like to keep this navigation as light as possible.

Thanks for the feedback.

[Amirouche]

Héllo again,

I don't see where Django makes things easier for you at all. Why no just use static files and include a menu via Javascript and serve all the thing like static files instead of using Django ?

Regards

[Lachlan Musicman]

It had occurred to me that Django wasn't the right tool for this job -
not everything is a nail :)

cheers
L.

On Sun, Jan 13, 2013 at 10:21 PM, Amirouche
<amirouche...@gmail.com> wrote:
> I don't see where Django makes things easier for you at all. Why no just use
> static files and include a menu via Javascript and serve all the thing like
> static files instead of using Django ?
>

[Sanjay Bhangar]

On Sat, Jan 12, 2013 at 12:04 AM, chad petzoldt <cpet...@gmail.com> wrote:
> Sanjay, you were hitting things pretty close. I think that making sure slug
> names match up to real *static* locations is the key. I am hosting with
> Apache, and I thought about using some configurations to cheat a little bit,
> and get some of the static-files burden off of Django and let Apache resolve
> any requests that point to actual files on disk (with some security in
> mind).
>
> What about a view that can determine if it should be pointing at a file on
> disk, and if so, do a custom Http_Response, and read the actually binary
> contents from disk by hand. Im just curious on this one; it could be an
> alternative means of file storage and recognition. I understand this may
> have some performance implications, but this is not going to break Django,
> correct? Is this bad Django etiquette?
>

Yea, hooking up slug-name to folder-name is brittle - if you're just
reading the binary contents of files and serving them, you could do it
in a django view, but you're probably better off doing it directly
through apache - there's no reason to use Django for something its not
good as :) - I think this really depends on how much these 'static'
html pages you're serving need other components of your django app -
if they're completely separate, then they're probably better off being
served simply as static files through apache :)

What I have done in the past is read template files off disk through
some name in the URL .. so like http://foo.com/home looks into a
directory, looks for home.html, and renders the template .. if you do
need to nest these static html files of yours into a django template
that has some other logic, I think this is a perfectly fine logic to
use and has worked well for me .. if you just need to serve the binary
files off disk though, you're probably better off just using apache to
serve them ...

Hope that makes sense / helps ..
-Sanjay

> Amirouche, a different way to describe what I am trying to achieve might be
> this: I would like to bypass the admin as much as possible when it comes to
> these "Articles". But when rendered, all of these articles do need at least
> a small HTML wrapper, and probably some kind of of global template that
> helps with navigation. If I could handle navigation without the admin, i
> would be very excited, but it does seem that I will need to use the admin
> for at least this purpose. The navigation would probably help render
> hierarchical "menus" for use as templates within the Articles. I would like
> to keep this navigation as light as possible.
>
> Thanks for the feedback.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/django-users/-/ja-o_pBFAT4J.

[chad petzoldt]

It had occurred to me that Django wasn't the right tool for this job -
not everything is a nail :)

I am embedding these files within a template, so they are not direct static serves. But the content must be inserted within the template "as-is" from the filesystem.

I suck at Javascript.

Perhaps still, Django is not proper for the job, but I do know that I need some server-side logic, and I want to do it in Python. Any recommendations on another framework?
There are aspects of Django that are growing on me, this one paradigm is where I am struggling, and I would not like to abandon just yet.

To recap one of my original statements; I do believe what I am really looking for is a content management system, I just don't feel ready to commit (maybe I need to get over that). I want a content management system that focuses more on the "client experience" in the browser. It needs to be picky about layouts, and aware of embedded media. Any suggestions for starters?

NOTE: My site layouts are not "liquid" at all. They are very absolute; from dimensions to positioning. Its not just about getting all the content on the page in a certain order.

 

[Amirouche]

On Monday, January 14, 2013 3:46:54 PM UTC+1, chad petzoldt wrote:

It had occurred to me that Django wasn't the right tool for this job -
not everything is a nail :)

I am embedding these files within a template, so they are not direct static serves. But the content must be inserted within the template "as-is" from the filesystem.

If you use django templates of course you need django they are other solutions, like Jinja2 which as no dependency on Django. The question is what do you really need to use in Django.

 

I suck at Javascript.

It can stay as is but it can also change ;)

 

Perhaps still, Django is not proper for the job,

 

but I do know that I need some server-side logic, and I want to do it in Python.

Yes, but what is this server side logic ? forms ? even if they are forms on your site you don't need to use Django to serve static files via templates.

 

Any recommendations on another framework?

Maybe, but your usecase is vague to me.

 

There are aspects of Django that are growing on me,

What are those ?

 

this one paradigm is where I am struggling, and I would not like to abandon just yet.

Don't try to feet a solution to any problem, your problem might have better solutions outside of Django, the thing is I don't understand your project enough to say, yes or no to Django for you.

 

To recap one of my original statements; I do believe what I am really looking for is a content management system

A CMS where there is no admin and editing ? 

 

I just don't feel ready to commit (maybe I need to get over that).

You are wrong about that, getting a CMS with Django is not that a endeavor especially if the CMS is for particular use case...

I want a content management system that focuses more on the "client experience" in the browser.

What does it mean ? You want javascript things ?

 

It needs to be picky about layouts, and aware of embedded media.

This can be done with both specific models or a Wysiwyg editors but I though there were no editing after the initial templating which makes the model or wysiwyg approach useless, except if you are looking for making it possible for the designer and editors to build a website like it's done in jetstrap. But this is not a CMS, it's a website creator.

 

NOTE: My site layouts are not "liquid" at all. They are very absolute; from dimensions to positioning. Its not just about getting all the content on the page in a certain order.

This has nothing to do with layouts. The question is: is there any patterns in theses layouts ? What are those patterns ? 

If all you need is a template engine and want to avoid javascript which might not be accessible, you might create your own static site generator based on Jinja2. Look at Pelican for an example app that does static blog generation from rst files and templates files. Of course if you don't need the rst part you can skip it.

Best regards,

Amirouche

[Sanjay Bhangar]

On Mon, Jan 14, 2013 at 8:16 PM, chad petzoldt <cpet...@gmail.com> wrote:
>> It had occurred to me that Django wasn't the right tool for this job -
>> not everything is a nail :)
>
>
> I am embedding these files within a template, so they are not direct static
> serves. But the content must be inserted within the template "as-is" from
> the filesystem.
>

here is a view i have from a project (simplified):

def viewPage(request, page):
filename = page + ".html"
html_file = join(HTML_PATH, filename)
try:
txt = open(html_file).read()
except:
return HttpResponse("page not found")
context = RequestContext(request, {'html': txt})
return render_to_response("page.html", context)

And the corresponding url.py:
(r'(?P<page>.*)', 'foo.views.viewPage'),

And then page.html is something like:
<!doctype html>
<!-- any other stuff ... -->
<body>
{{ html }}
</body>

My use case was different tho, but just to highlight that its fairly
straightforward to read the contents of some file on disk and stick it
into a django template ..
In your case, each "page" is going to need its own corresponding set
of css / js, so I imagine this would be a bit more work .. but I'm
still fairly certain you can come up with a scheme that allows you to
have your html and css / js on the file-system and then use python
code to put it together into a master template / integrate with other
app logic you have ..

It does seem like a bit of a hack, but hey, moulding InDesign exports
into actual webpages is always going to be, a bit ;-)

> I suck at Javascript.
>

ok, this shouldn't affect this problem / use-case.

> Perhaps still, Django is not proper for the job, but I do know that I need
> some server-side logic, and I want to do it in Python. Any recommendations
> on another framework?
> There are aspects of Django that are growing on me, this one paradigm is
> where I am struggling, and I would not like to abandon just yet.
>

What is the paradigm, exactly ... ? Do you have an idea of how you
would have implemented the same thing using php or anything else? It
may give a better idea of your use case / what you're thinking.

> To recap one of my original statements; I do believe what I am really
> looking for is a content management system, I just don't feel ready to
> commit (maybe I need to get over that). I want a content management system
> that focuses more on the "client experience" in the browser. It needs to be
> picky about layouts, and aware of embedded media. Any suggestions for
> starters?
>

Welll .. the most important question here is: does the client /
non-tech people need to be able to edit content themselves? Or will
you / someone "techie" always be handling the editing of the html? If
people do need to edit themselves, then you generally do need to
structure things a little - an all-singing-all-dancing WYSIWYG editor
for end-users, is I'm afraid, still a bit of a pipe dream (someone
please correct me if things have changed :)) .. going down the path of
trying to build a system that's going to accommodate for radically
different designs that you have no way of pre-empting, is generally a
recipe for disaster .. I think you've got the right idea of trying to
get the system to adapt to your work-flow rather than the other way
around .. and if it is only you editing the HTML files, build
something that helps you manage that, rather than a "content
management system", although, er, anything that you use to manage
content is a content management system :)

The other option, of course, is to tell the client that they need to
stick to a consistent format, and then just put stuff into different
fields in the database and render it in a template like normal :)

hope my response didn't add to the confusion, and hope you find a
solution that works for you.
happy hacking!
-Sanjay

> NOTE: My site layouts are not "liquid" at all. They are very absolute; from
> dimensions to positioning. Its not just about getting all the content on the
> page in a certain order.
>
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/django-users/-/hLQsR_8-pOYJ.