Help deciding ORM VS raw SQL trade-off in Complex scenario

[Krishnakant Mane]

Hello.

I am seasoned SQLAlchemy user and quite good in node's sequelise ORM.

But I am new to the one with Django.So here's my situation.

I am developing an accounting (book keeping ) automation software service.

So there are accounting rules (Debit = Dr and credit = Cr) for double
entry book keeping.

Every transaction will have 2 or more amounts, at least 1 each for dr or
Cr.

These entries are called vouchers.

We also store retail bills, receipts and payments again all in different
tables.

But the bills and receipt&payment tables are connected to the voucher
table.

The software generates reports such as cash flow, meaning day's opening
balance, total Drs, total crs, and final closing balance (DRs - Crs).

then there are Profit and Loss as well as balance sheet reports.

All this needs a lot of aggregations (sum and counts ) and also joining
of invoice + voucher and recept&payment + voucher tables.

so here are my questions.

1: given the fact that I have created materialised views in Postgresql,
should I even care to model them and use the ORM syntax instead of raw
query?  What would perform better?

2: datasets are going to be huge some times in terms of shear rows (all
transactions aka vouchers ) or some times sum and count will be used in
complex queries on a huge dataset.

Again, should I rely on raw queries or will ORM plan the queries for me
better?  Should I instead create stored procedures and call them from my
REST API?

talking of which,

3: I am using Django REST Framework and serialising records is an option
to get json output.

Should I use it or just go with raw queries and convert output to JSON
as required?

Again performance is a question.

Tip, My team is very proficient in SQL and yours truely can modestly
call himself an expert in the same, so maintenance is not an issue here.

Regards.

Krishnakant.

[Enock Deghost]

🙄

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/097a6e55-c30e-491e-bf43-86e4c672faa4%40gmail.com.

[eric paul]

In whatever way possible use the Django ORM for security purposes and also efficiency . I won't recommend to use Raw queries if you don't know what you are doing

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAA2jrmJ0TtbxmfXeSCq5S9p8XsrPjJBf6_gKMRY_MSuTagFt4Q%40mail.gmail.com.

[Sam Brown]

Im sure there are performance metrics out there to prove the ORM will not be the bottleneck. But I’ve never seen it slow things down when I’ve employed a timer on operation

Also, ive recently ran into some of the limitations of drf and am looking into moving to an api that can be less coupled with orm. Django-ninja looks promising. 

[Gulshan Yadav]

If you use ORM QUERY THEN IT IS VERY EASY TO SOLVE THIS ONE BY USING FOREIGN KEY OR MANY TO MANY RELATIONSHIP AND WHEN IF USER NOT RAGISTER THEN WE NEED TO ADD NON RAGISTER USER UNIQUE KEY FOR IDENTIFICATION AND AMOUNT WHICH ONE TAKE RECIEVE THE MONEY STORED IDENTIFICATION KEY  IN PAYMENT TABLE AND CONNECT MORE TABLE

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAA2jrmJ0TtbxmfXeSCq5S9p8XsrPjJBf6_gKMRY_MSuTagFt4Q%40mail.gmail.com.

[Krishnakant Mane]

I am looking into ninja too.

I guess I may be able to use SQLAlchemy seamlessly with it.

Regards.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALmEK1sEN%2BLjwcrT-7eYbQkrdkU2bynGehBo4BjvkW1NEBVUjQ%40mail.gmail.com.

[Singhal._.paras]

Bro ,

Just go for it !

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f1a8a082-61c9-449c-9361-d1c8d77c0b07%40gmail.com.

[Krishnakant Mane]

Hi Eric.

Can you explain a bit more on the "performance " aspect you refered to?

I have already mentioned my scenario and given the case study which I am working on.

In that reference, how using the ORM will benefit more than raw queries working with materialised views?

Regarding security, I (and the team working on this ) are totally aware of the pitfalls and the way around them so that's out of the way.

Regards.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALv%3Dr_nWJjrSk%3DLbFh-pfL5Ni%2B%2ByZu0qH%3DRU7o7mnpD-eJHcqw%40mail.gmail.com.

[eric paul]

According to your scenario an accounting system is a more complex system that requires high performance. It can be written in any language but as the system grows some programming languages will experience performance issues python being one of them take a case like Dropbox which was originally written in python but because of python speeds and data handling limitations it led the team to introduce other languages (Go, Rust) . Since you've decided to go with Django then it's best you use raw queries and concurrency etc inorder to achieve high performance.It also comes down to your hardware and code practices . Take a look at the Django ledger repo an accounting system. About the API you can explore Graphql.

[ReynardSec]

Hello,

On Sunday 14 July 2024 at 09:03:38 UTC+2 eric paul wrote:

In whatever way possible use the Django ORM for security purposes and also efficiency .

Offtopic: There are certain issues worth keeping in mind, even if you are using an ORM: https://www.elttam.com/blog/plormbing-your-django-orm/

Cheers,

ReynardSec

https://reynardsec.com/ 

[Krishnakant Mane]

Talking about Ninja.

I am actually trying to figure out how I can distribute my crud APIs in seperate python modules (one per model ) and make a package in my app.

Then How all the routs can be imported in the project urls.py at once.

Any suggestions?

Regards.

On 7/14/24 21:35, Sam Brown wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALmEK1sEN%2BLjwcrT-7eYbQkrdkU2bynGehBo4BjvkW1NEBVUjQ%40mail.gmail.com.