Re: new to Django and building web applications. Advice with postgresql

[Bill Freeman]

In addition to installing PostgreSQL you must use tools that come with the postgres installation to 1. create a database (a postgres installation has more than one), 2 create a user (unrelated to user account on the operating system) for django to use (though some people cheat and use the pre-defined "postgres" user), 3 set its password, and 4. grant it at least table creation permissions on the database you created.  If you want to connect to the database from other than the local machine, you will need to play with a configuration file (hba_conf, if memory serves).  pgadmin3, by the way, is a very useful tool that requires separate installation.  There are postgres tutorials on the web, and the postgres on line documentation is very good.  I think that there is at least one page in the django documents that details one sequence of commands for setting up an installed postgres for django.

 You will also need the python package psycopg2, elect to use it in settings.py, and, also in settings.py, specify your database, database user, and password.

But you don't have to do that all at once.  Learning about django with sqlite (included in modern pythons) won't leave you surprised when you start using postgres.  (Though if you start to put significant, that is, not just for test, content in your site it is time to move, since transferring content between them is not trivial.  It's a useful skill, but you will have enough on your plate.)

It seems that some people successfully develop django sites without a significant understanding of python.  But I would advise learning python as well and as soon as you can.  It will be valuable to you even if you stop using django.  (My current job has me working with tornado and the Apache Qpid implementation of AMQP, but I still do mostly python.)  You may already be aware of these, but I'd like to offer two starter tips for those coming to python from other languages:

  1. In most languages, including C++, a variable is a named piece of storage.  When you assign to it, the value is copied into that storage.  The storage has to be the right size to hold whatever you are storing (so the variable must be declared with a type).  The declaration also tells the compiler how to interpret the collection of bits in that storage.  In python, those things that look like variables all hold only (extended types of) references.  The "objects" referred to all have self evident types (that is, if you have one, you can tell what type it is), which is why declarations are unnecessary, and a "variable" can hold any type.  Assignment copies only the reference.

 2. To edit python use an editor that is aware of, or has been configured for, python.  No end of frustration results from editors using the ASCII tab character (as distinct from the Tab key on the keyboard) for indentation.  Since indentation is significant in python, it is vital that the indentation looks the same to you (and others) exactly how python will interpret it.  While python can deal with tab characters, and can even be told to interpret them as other than moving to the next multiple of 8 columns, someone else's editor, or printer output, etc., may display them differently.  So it is safest if your editor interprets the Tab key as meaning that it should insert the correct number of spaces.  Any IDE claiming to support python should do this.  Many python installations come with IDLE, or it can be added, and its editor gets this right.  vi or vim can be configure this way.  Modern emacs (my choice) versions come with a python mode, and it can be added to older ones.  Editors that don't think of themselves as being for programmers (notepad, wordpad, etc.) are not likely to even be able to be configured to do this right (fortunately many of the previously mentioned winners are free open source tools).

On Fri, May 17, 2013 at 1:16 AM, <d.gu...@gmail.com> wrote:

Hi, 

I'm new to building web applications, django, backend work, etc... I have experience in C++ up to A.P. computer science level if that means anything and I'm beginning to get a grasp on the python language. I've done all of the exercises on code academy and have watched fairly a lot of videos on building with django and python. I have somewhat of a grasp on the overall design of files in the terminal that are involved (url.py, views.py, etc...). I've been following http://www.djangobook.com/en/2.0/chapter02.html. My question is It was suggested by this free online open source book to install postgresql, so I did but I had no idea how to use it and kind of gave up on it and setting up the backend. Can anyone suggest a resource a book perhaps that can guide me in the right direction or just give some friendly advice. My goal by the end of the summer is to be able to build a substantial dynamic web application with python in Django that will have the feel of a Khan Academy or code academy (a dynamic website that is able to hold lots of content and have a user-interface with accounts, logins, rewards systems and much more) and be launced on the web for the world to see and use. The only other true experience I have with web design is with static web pages and basic HTML and CSS. Yes, it is a big goal, but I think I can do it with the right amount of guidance. I have all summer to learn. Hell, you have to start somewhere right? All suggestions are welcome.

Thanks.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Artem Zinoviev]

you want learn django or postgres ? :-) postgres is cool stuff but it not requited... try this course - https://www.udemy.com/coding-for-entrepreneurs/, (in reddit you can find only 1$)

пятница, 17 мая 2013 г., 8:16:59 UTC+3 пользователь d.gu...@gmail.com написал:

[Artem Zinoviev]

this is the best way - https://django.2scoops.org/, and you can use something like this django template project for your task -https://github.com/nigma/django-modern-template, it have memcached integrated AWS S3 for static files and so on...

[Kelvin Wong]

You don't have to become an expert with postgres to use Django. You can do most of the db development using SQLite and hold off on postgres until you are ready to deploy.

I'd get a Linux server running in VirtualBox. Install Apache and postgres. Configure postgres. Install phpPgAdmin.

If you are going to be looking for a job doing Django, your employer will want you to know Django and Python really well, not postgres.

K

[Scott Anderson]

On Sunday, May 19, 2013 11:18:07 AM UTC-4, WongoBongo wrote:

You don't have to become an expert with postgres to use Django. You can do most of the db development using SQLite and hold off on postgres until you are ready to deploy.

I highly recommend *against* waiting for PostgreSQL until deployment. There are significant differences between SQLite, PostgreSQL, and MySQL. If you wait until deployment to test on your production database you will find yourself fixing and changing things at the last minute. Start testing on your production deployment database as soon as possible.

Even if you are using an ORM there are enough differences between the databases (and/or their drivers!) that you can run into issues. Four examples of MySQL vs. SQLite, but the same recommendation holds true for PostgreSQL as well:

1) MySQL truncates DateTimeFields to the second; SQLite stores the milliseconds.

2) SQLite can re-use keys if a row is deleted; MySQL will not. This can affect unit tests.

3) Each database may use a different natural order when no ORDER BY is used.

4) SQLite allows 0 as a PK, MySQL does not.

Also, you'll benefit from the exercise of setting up and using the same database you'll see in production.

Regards,
-scott

[Tom Evans]

On Sun, May 19, 2013 at 11:51 PM, Scott Anderson
<scottan...@gmail.com> wrote:
> On Sunday, May 19, 2013 11:18:07 AM UTC-4, WongoBongo wrote:
>>
>> You don't have to become an expert with postgres to use Django. You can do
>> most of the db development using SQLite and hold off on postgres until you
>> are ready to deploy.
>
>
>
> I highly recommend *against* waiting for PostgreSQL until deployment. There
> are significant differences between SQLite, PostgreSQL, and MySQL. If you
> wait until deployment to test on your production database you will find
> yourself fixing and changing things at the last minute. Start testing on
> your production deployment database as soon as possible.

The purpose of using an ORM is to make your application database
agnostic. You should not find that changing DB engine is overly
taxing.

>
> Even if you are using an ORM there are enough differences between the
> databases (and/or their drivers!) that you can run into issues. Four
> examples of MySQL vs. SQLite, but the same recommendation holds true for
> PostgreSQL as well:
>
> 1) MySQL truncates DateTimeFields to the second; SQLite stores the
> milliseconds.
>
> 2) SQLite can re-use keys if a row is deleted; MySQL will not. This can
> affect unit tests.
>
> 3) Each database may use a different natural order when no ORDER BY is used.
>
> 4) SQLite allows 0 as a PK, MySQL does not.
>
> Also, you'll benefit from the exercise of setting up and using the same
> database you'll see in production.
>

Not all of these statements are wholly accurate, mysql will re-use
keys when necessary (but doesn't try to 'back fill' the set of used
ids), and mysql is also perfectly happy with 0 as a primary key, but
will replace it with the next auto increment id if the field is an
auto increment field. There is no 'natural ordering' of un-ordered
results, the order is undefined, and undefined behaviour is undefined.

The point is, installing and learning the ins and outs of an RDBMS is
not necessary to using Django. If you think you are stuck trying to
install or understand postgres, just ignore it and use sqlite. You can
always change at a later date.

Cheers

Tom

[Kevin Daum]

On Sunday, May 19, 2013 6:51:09 PM UTC-4, Scott Anderson wrote:

I highly recommend *against* waiting for PostgreSQL until deployment. There are significant differences between SQLite, PostgreSQL, and MySQL. If you wait until deployment to test on your production database you will find yourself fixing and changing things at the last minute. Start testing on your production deployment database as soon as possible.

I'm going to echo Scott's sentiment. I ran into enough difficult-to-troubleshoot (or just annoying) issues resulting from inconsistencies between sqlite and postgres that I no longer allow my developers to use sqlite at all. We now use vagrant to install debian VMs running postgres on all our developer machines. We have a shell script that automatically creates and configures a database for the project, so it doesn't end up taking too much longer to set up than sqlite.

That said, if you're just getting started with django, you probably don't want to deal with postgres yet. Feel free to tinker and learn against sqlite, but once you start writing code that you're eventually going to put into production, I'd start using whatever database you're going to use in production.

Kevin 

[Simon Riggs]

On 21 May 2013 13:04, Tom Evans <teva...@googlemail.com> wrote:
> On Sun, May 19, 2013 at 11:51 PM, Scott Anderson
> <scottan...@gmail.com> wrote:
>> On Sunday, May 19, 2013 11:18:07 AM UTC-4, WongoBongo wrote:
>>>
>>> You don't have to become an expert with postgres to use Django. You can do
>>> most of the db development using SQLite and hold off on postgres until you
>>> are ready to deploy.
>>
>>
>>
>> I highly recommend *against* waiting for PostgreSQL until deployment. There
>> are significant differences between SQLite, PostgreSQL, and MySQL. If you
>> wait until deployment to test on your production database you will find
>> yourself fixing and changing things at the last minute. Start testing on
>> your production deployment database as soon as possible.
>
> The purpose of using an ORM is to make your application database
> agnostic.

ORM stands for Object Relational Mapper. My thinking is that if its
role was solely to make applications database agnostic it would be
called something like Common Database Interface, or Database Agnostic
Layer. Such things do exist, yet so do ORMs. ISTM that ORMs help you
write good applications, not just make things agnostic.

> You should not find that changing DB engine is overly
> taxing.

Hmm, well. A YMMV moment if ever there was one.

> The point is, installing and learning the ins and outs of an RDBMS is
> not necessary to using Django.

Often, yes. But mostly, one needs to consider details about the whole
stack in making things work well.

There are lots of good reasons for careful selection of each part of
your stack. Choosing Django was no doubt an informed decision and one
made with a view to the many good things this gives you. Other
products in your stack benefit from similar careful and informed
decision making.

Otherwise we'd all be using microCOBOL on Netware.

> If you think you are stuck trying to
> install or understand postgres, just ignore it and use sqlite. You can
> always change at a later date.

Changing a major architectural component in your stack is not a
trivial thing. Major changes affect the quality of your deliverables.

The most important thing is that a database is a shared resource. If
you write all your programs assuming you'll be the only user then it
likely won't work very well in production.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

[Tom Evans]

On Tue, May 21, 2013 at 2:16 PM, Simon Riggs <si...@2ndquadrant.com> wrote:
> On 21 May 2013 13:04, Tom Evans <teva...@googlemail.com> wrote:
>> On Sun, May 19, 2013 at 11:51 PM, Scott Anderson
>> <scottan...@gmail.com> wrote:
>>> On Sunday, May 19, 2013 11:18:07 AM UTC-4, WongoBongo wrote:
>>>>
>>>> You don't have to become an expert with postgres to use Django. You can do
>>>> most of the db development using SQLite and hold off on postgres until you
>>>> are ready to deploy.
>>>
>>>
>>>
>>> I highly recommend *against* waiting for PostgreSQL until deployment. There
>>> are significant differences between SQLite, PostgreSQL, and MySQL. If you
>>> wait until deployment to test on your production database you will find
>>> yourself fixing and changing things at the last minute. Start testing on
>>> your production deployment database as soon as possible.
>>
>> The purpose of using an ORM is to make your application database
>> agnostic.
>
> ORM stands for Object Relational Mapper. My thinking is that if its
> role was solely to make applications database agnostic it would be
> called something like Common Database Interface, or Database Agnostic
> Layer. Such things do exist, yet so do ORMs. ISTM that ORMs help you
> write good applications, not just make things agnostic.

Opinions. We've all got them. Django's one line description of their ORM:

Django provides an abstraction layer (the “models”) for structuring
and manipulating the data of your Web application.

Django's abstraction layer provides a bunch of features, all of which
operate identically across a variety of database servers. Its an
abstraction layer that provides database agnostic features.

ISTM you're being overly pedantic.

>
>> You should not find that changing DB engine is overly
>> taxing.
>
> Hmm, well. A YMMV moment if ever there was one.

This comment was directed at the OP. The OP is just starting to use
any web development package, anything he does in Django with sqlite
should be easily switched to another database. Do you have some
examples of Django features he would find difficulty with if he
switched from using django/sqlite to django/postgres?

I know you do a lot of Postgresql consulting, I expect some of this
may involve migrating MySQL -> Postgres, and if you have an
application written for MySQL, it will be tricky porting it over.
However, this is not that scenario.

The only queries you need to rewrite on moving django/sqlite ->
django/postgres are raw queries. How many will this beginner have?
Should rewriting raw queries be the thing this beginner worries about?

'YMMV' indeed. Switching or changing DB engines in a Django app is not
hard, unless the app is extremely badly written.

>
>> The point is, installing and learning the ins and outs of an RDBMS is
>> not necessary to using Django.
>
> Often, yes. But mostly, one needs to consider details about the whole
> stack in making things work well.
>
> There are lots of good reasons for careful selection of each part of
> your stack. Choosing Django was no doubt an informed decision and one
> made with a view to the many good things this gives you. Other
> products in your stack benefit from similar careful and informed
> decision making.
>
> Otherwise we'd all be using microCOBOL on Netware.

Yes. I'm sure the OP is also making a pro/con list to decide which of
gunicorn and Tornado to use.

>
>> If you think you are stuck trying to
>> install or understand postgres, just ignore it and use sqlite. You can
>> always change at a later date.
>
> Changing a major architectural component in your stack is not a
> trivial thing. Major changes affect the quality of your deliverables.

The most important thing as a beginner developer is to start
developing things, and don't get distracted by irrelevancy. Messing
about getting the perfect Postgresql install will not get you any
further with being able to develop websites in Django, and seeing as
that is what the OP wanted to do, if postgresql is a problem for him,
he should ignore it, and start the work he wanted to do.

You do not need postgresql to develop in Django, just like you don't
need Apache/mod_wsgi. The simplest, easiest and most straightforward
way is to use sqlite as your database, and to serve pages using
runserver.

[Simon Riggs]

On 21 May 2013 15:00, Tom Evans <teva...@googlemail.com> wrote:

> The only queries you need to rewrite on moving django/sqlite ->
> django/postgres are raw queries. How many will this beginner have?
> Should rewriting raw queries be the thing this beginner worries about?
>
> 'YMMV' indeed. Switching or changing DB engines in a Django app is not
> hard, unless the app is extremely badly written.

Changing the engine is not hard, but that doesn't mean it will work
exactly as expected.

There are many operational aspects to consider and changing a major
component at the last minute isn't good practice, whether that be
Postgres, sqlite or any other component.

Whatever you plan to use, please use it from the start. If you can.

If there are blockers to doing that for Postgres, please mention it to
me, other community members or on the Postgres lists, so we can help.

[Scott Anderson]

On May 21, 2013, at 8:04 AM, Tom Evans wrote:
>
> The purpose of using an ORM is to make your application database
> agnostic. You should not find that changing DB engine is overly
> taxing.

Theoretically, yes. The difference between theory and practice is very small in theory, and not so much in practice. ;-)

Theoretically, the ORM should protect you from differences between databases. In practice, this doesn't always happen.

There *are* real world differences between running different databases on the same ORM. Quoting the purpose of an ORM and telling someone they "should not" have a problem is naive, sorry.

While some of these differences can be avoided, doing so requires the developer to know that they exist in the first place, thus negating the purpose of using an ORM. Given the character of the question by the original poster, I suspect he might have issues with this.

PostgreSQL is very strict by default. SQLite is very forgiving.

You can also run into problems with South migrations working on one and not the other.

> Not all of these statements are wholly accurate, mysql will re-use
> keys when necessary (but doesn't try to 'back fill' the set of used
> ids), and

The statements don't need to be "wholly accurate". The behavior is different in practice.

> mysql is also perfectly happy with 0 as a primary key, but
> will replace it with the next auto increment id if the field is an
> auto increment field.

class Moo(models.Model):
name = models.CharField(max_length=50)

moo.json:
[
{
"pk": 0,
"model": "test.moo",
"fields": {
"name": "Test"
}
}
]

Running as MySQL:
anderson$ python manage.py loaddata moo.json
ValueError: Problem installing fixture '/Users/anderson/src/.../fixtures/moo_genre.json': The database backend does not accept 0 as a value for AutoField.

Running as Sqlite:
anderson$ python manage.py loaddata moo.json
Installed 1 object(s) from 1 fixture(s)

If your real world dataset includes a row with 0 as the primary key, you need to be aware of this.

> There is no 'natural ordering' of un-ordered
> results, the order is undefined, and undefined behaviour is undefined.

In theory, yes. However, the naive developer may observe that this ordering is actually implementation-specific under the hood and write code based on that fact, and in doing so tie the code to a particular implementation. Yes, you should always include an order_by if your use case depends on ordering. No, not everyone realizes that.

> The point is, installing and learning the ins and outs of an RDBMS is
> not necessary to using Django. If you think you are stuck trying to
> install or understand postgres, just ignore it and use sqlite. You can
> always change at a later date.

And my point is that there remain ins and outs of the various database backends that prevent seamless migration between them.

At some point you will need to install and test on your production database. Why not do it first and avoid possible incompatibilities between dev and production down the way?

If you're just learning, that's fine. No need. But as soon as you're on an actual project, do yourself a favor and develop on the platform you plan on using in production.

Regards,
-scott