Data model design questio: graph in database
112 views
Skip to first unread message
Sébastien Hinderer
Feb 23, 2024, 5:11:02 PMFeb 23
to django...@googlegroups.com
Dear all,
Using the mailing list rather than the forum because, like many visually
impaired people, I find mailing lists way easier to use than forums, I
hope the list is still active.
I need to store in database and manipulate graphs of which both
vertices and edges can each be of several types.
At the moment I am modelling this as follows:
class Vertex(models.Model):
pass
class Vertex_type_1(Vertex):
# Data specific to vertices of type 1
...
class Vertex_type_2(Vertex):
# Data specific to vertices of type 2
...
class Edge(models.Model):
src = models.ForeignKey(Vertex,
on_delete=models.CASCADE, related_name="+")
dst = models.ForeignKey(Vertex,
on_delete=models.CASCADE, related_name="+")
class Meta:
unique_together = ['src', 'dst']
class Edge_type_1(Edge):
# Data specific to edges of type 1
...
class Edge_type_2(Edge):
# Data specific to edges of type 2
...
I will have to wwite algorithms that work at the graph level, e.g.
to find paths. However, once a path has been found, at some point it
will become necessary to see which types of vertices and edges it is
that are involved in the path that has been found and I am wondering how
I will do that.
More concretely, with the current model, if an edge is found,
does Django have a way to also find out which type of edge it is,
i.e. which child class has given rise to that edge? Or do I need to add
type fields to the Vertex and Edge classes so that it becomes possible
to go from the parent to the children?
I feel unsure because on the one hand it feels to me it is necessary
with the current model to add such type fields, but on the other hand I
feel this is kind of wrong, almost like a code smell suggesting that I
am modelling things in a wrong way.
Any insight would be warmly appreciated as I am a total beginner with
Django.
Many thanks in advance,
Seb.
Using the mailing list rather than the forum because, like many visually
impaired people, I find mailing lists way easier to use than forums, I
hope the list is still active.
I need to store in database and manipulate graphs of which both
vertices and edges can each be of several types.
At the moment I am modelling this as follows:
class Vertex(models.Model):
pass
class Vertex_type_1(Vertex):
# Data specific to vertices of type 1
...
class Vertex_type_2(Vertex):
# Data specific to vertices of type 2
...
class Edge(models.Model):
src = models.ForeignKey(Vertex,
on_delete=models.CASCADE, related_name="+")
dst = models.ForeignKey(Vertex,
on_delete=models.CASCADE, related_name="+")
class Meta:
unique_together = ['src', 'dst']
class Edge_type_1(Edge):
# Data specific to edges of type 1
...
class Edge_type_2(Edge):
# Data specific to edges of type 2
...
I will have to wwite algorithms that work at the graph level, e.g.
to find paths. However, once a path has been found, at some point it
will become necessary to see which types of vertices and edges it is
that are involved in the path that has been found and I am wondering how
I will do that.
More concretely, with the current model, if an edge is found,
does Django have a way to also find out which type of edge it is,
i.e. which child class has given rise to that edge? Or do I need to add
type fields to the Vertex and Edge classes so that it becomes possible
to go from the parent to the children?
I feel unsure because on the one hand it feels to me it is necessary
with the current model to add such type fields, but on the other hand I
feel this is kind of wrong, almost like a code smell suggesting that I
am modelling things in a wrong way.
Any insight would be warmly appreciated as I am a total beginner with
Django.
Many thanks in advance,
Seb.
Mike Dewhirst
Feb 25, 2024, 2:37:34 AMFeb 25
to django...@googlegroups.com
In the base class you need a method like this ...
def which_class(self):
return self.__class__
... which will be inherited by each child class and when called will reveal the class name of the instance.
To test this in development, in the base class add a save method like this ...
def save(self, *args, **kwargs):
print(f"\n class name = {self.which_class()}")
super().save(*args, **kwargs)
Whenever you save a model its name should print in stdout. I tried it in one of my models and this is the result ...
<class 'chemical.models.chemical.Chemical'>
... where "Chemical" is the class name.
So if you have a method which understands what the child class names mean, you can put your algorithm in there and execute calls to which_class() for each instance involved.
That might be enough to get you started.
Cheers
Mike
def which_class(self):
return self.__class__
... which will be inherited by each child class and when called will reveal the class name of the instance.
To test this in development, in the base class add a save method like this ...
def save(self, *args, **kwargs):
print(f"\n class name = {self.which_class()}")
super().save(*args, **kwargs)
Whenever you save a model its name should print in stdout. I tried it in one of my models and this is the result ...
<class 'chemical.models.chemical.Chemical'>
... where "Chemical" is the class name.
So if you have a method which understands what the child class names mean, you can put your algorithm in there and execute calls to which_class() for each instance involved.
That might be enough to get you started.
Cheers
Mike
Many thanks in advance, Seb.
-- We recommend signal.org Signed email is an absolute defence against phishing. This email has been signed with my private key. If you import my public key you can automatically decrypt my signature and be sure it came from me. Your email software can handle signing.
Sébastien Hinderer
Feb 25, 2024, 4:15:06 AMFeb 25
to django...@googlegroups.com
Dear Mike,
Many thanks for your suggestion!
I find it significantly superior to what I had in mind, in the
sense that I would have had to handle the class types manually, with
the obvious drawbacks (error-prone, not so easy to extend), which
your solution completely avoids.
I will thus definitely use it to go ahead because I do think it does
unblock me and, as you concluded, it's definitely enough to get me
started and even going.
In the long run and for my culture, I do reamin interested in feedback,
especially on whether my approach is flawed or is state of the art and
idiomatic in this environment. If it's flawed, I'd be really curious and
grateful to hear about any other possible alternative.
Many thanks again to you Mike and in advance to any other person who
would participate to this discussion.
Best wishes,
Seb.
Many thanks for your suggestion!
I find it significantly superior to what I had in mind, in the
sense that I would have had to handle the class types manually, with
the obvious drawbacks (error-prone, not so easy to extend), which
your solution completely avoids.
I will thus definitely use it to go ahead because I do think it does
unblock me and, as you concluded, it's definitely enough to get me
started and even going.
In the long run and for my culture, I do reamin interested in feedback,
especially on whether my approach is flawed or is state of the art and
idiomatic in this environment. If it's flawed, I'd be really curious and
grateful to hear about any other possible alternative.
Many thanks again to you Mike and in advance to any other person who
would participate to this discussion.
Best wishes,
Seb.
Sébastien Hinderer
Feb 25, 2024, 5:47:12 AMFeb 25
to django...@googlegroups.com
Dear Mike,
I will definitely play aroudn with your nice suggestion, but I would like
to share a concern I have. At this stage I am actually unsure how this
will work.
Indeed, suppose a vertex is given as input and one tries to query the
database about all the edges that originate from this vertex. The answer
will be under the form of a list of edges, but those will be of class
Edge and I assume that their type method will return the Edge class
and not the class associated to the actual child.
Likewise, once given the list of edges, one has access to the list of
vertices that are neiighbourgs of the original one, but under the form
of a list of objects of the base class, Vertex, with still no way to find
the type of the child object that has given rise to that Vertex one.
Am I perhaps missing something here?
Many thanks in advance and apologies for th likely naïve question,
Seb.
I will definitely play aroudn with your nice suggestion, but I would like
to share a concern I have. At this stage I am actually unsure how this
will work.
Indeed, suppose a vertex is given as input and one tries to query the
database about all the edges that originate from this vertex. The answer
will be under the form of a list of edges, but those will be of class
Edge and I assume that their type method will return the Edge class
and not the class associated to the actual child.
Likewise, once given the list of edges, one has access to the list of
vertices that are neiighbourgs of the original one, but under the form
of a list of objects of the base class, Vertex, with still no way to find
the type of the child object that has given rise to that Vertex one.
Am I perhaps missing something here?
Many thanks in advance and apologies for th likely naïve question,
Seb.
Mike Dewhirst
Feb 25, 2024, 7:32:12 AMFeb 25
to django...@googlegroups.com
Seb
Your assumption is too pessimistic.
The class name of the (child) instance will always be returned from obj.__class__
If instead your method returned the type(obj) you would see the base class name instead of the child class name.
Perhaps you need to establish some tests which assert what you expect. I find such an approach very comforting.
Cheers
Mike
--
(Unsigned mail from my phone)
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ZdsadQngSnjTBimQ%40om.localdomain.
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ZdsadQngSnjTBimQ%40om.localdomain.
Mike Dewhirst
Feb 25, 2024, 7:59:31 PMFeb 25
to Django users
Re-reading your question perhaps I was too brief in my earlier response. Let me assume all your actual edges and vertices are objects of child classes. I also assume your base Edge and Vertex classes are abstract for inheritance purposes only.
It doesn't matter whether instances of edges and vertices are in lists. You can call obj.which_class() on any object which has (inherited) that method to determine whether an edge (or vertex) is type_1 or type_2 (using our previous terminology)
You cannot query the database using the return value from which_class() because that operates on instances not at the database level. Therefore you have to retrieve edges and vertices from the database with a query which fetches related rows and then filter the results afterwards. Perhaps like this ...
vertex_src = result_of_vertex_finder(...)
vertex_det = result_of_vertex_finder(...)
if vertex_src and vertex_dst:
edges_1 = Edge_type_1.objects.filter(src=vertex_src, dst=vertex_dst) or []
edges_2 = Edge_type_2.objects.filter(src=vertex_src, dst=vertex_dst) or []
all_edges = [edges_1].extend[edges_2]
if all_edges:
edges_type_1 = [edge for edge in all_edges if edge.which_class() == "type 1"]
edges_type_2 = [edge for edge in all_edges if edge.which_class() == "type 2"]
On the other hand, if you want to keep all edges in the same database table it might be better/simpler to add a field 'type' to the table as you were originally thinking.
Cheers
Mike
Jason
Feb 25, 2024, 8:28:55 PMFeb 25
to Django users
Question about this, why focus on using a rdbms for this functionality vs using an actual graph db like neo4j with neomodel?
Sébastien Hinderer
Feb 26, 2024, 3:25:50 AMFeb 26
to django...@googlegroups.com
Dear Jason,
Many thanks!
Jason (2024/02/25 17:28 -0800):
Oh simply because I was absolutely not aware of the existence of such
solutions, having absolutely no culture in the domain!
Many, many thanks for this suggestion, which I will definitely
investigate!
Seb.
Many thanks!
Jason (2024/02/25 17:28 -0800):
> Question about this, why focus on using a rdbms for this functionality vs
> using an actual graph db like neo4j with neomodel
> <https://github.com/neo4j-contrib/django-neomodel>?
> using an actual graph db like neo4j with neomodel
Oh simply because I was absolutely not aware of the existence of such
solutions, having absolutely no culture in the domain!
Many, many thanks for this suggestion, which I will definitely
investigate!
Seb.
Sébastien Hinderer
Feb 26, 2024, 4:01:45 AMFeb 26
to django...@googlegroups.com
Many thanks Mike for your answer below!
I will simply experiment, trying to put aside my preconceptions. :)
One thing I did realise in the process: it should not be possible to add
elemnts from the base classes from the admin interface because only
those elemnets from the child classes do make sense. I will thus need to
find a way to disable in the admin interface the possibility to work
directly on the Vertex and Edge classes, but I expect the documentation
will tell me how to achieve this.
Thanks agian,
Seb.
Mike Dewhirst (2024/02/25 23:31 +1100):
> SebYour assumption is too pessimistic. The class name of the (child) instance will always be returned from obj.__class__If instead your method returned the type(obj) you would see the base class name instead of the child class name.Perhaps you need to establish some tests which assert what you expect. I find such an approach very comforting.CheersMike--(Unsigned mail from my phone)
> -------- Original message --------From: Sébastien Hinderer <Sebastien...@ens-lyon.org> Date: 25/2/24 21:46 (GMT+10:00) To: django...@googlegroups.com Subject: Re: Data model design questio: graph in database Dear Mike,I will definitely play aroudn with your nice suggestion, but I would liketo share a concern I have. At this stage I am actually unsure how thiswill work.Indeed, suppose a vertex is given as input and one tries to query thedatabase about all the edges that originate from this vertex. The answerwill be under the form of a list of edges, but those will be of classEdge and I assume that their type method will return the Edge classand not the class associated to the actual child.Likewise, once given the list of edges, one has access to the list ofvertices that are neiighbourgs of the original one, but under the formof a list of objects of the base class, Vertex, with still no way to findthe type of the child object that has given rise to that Vertex one.Am I perhaps missing something here?Many thanks in advance and apologies for th likely naïve question,Seb.-- You received this message because you are subscribed to the Google Groups "Django users" group.To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ZdsadQngSnjTBimQ%40om.localdomain.
>
I will simply experiment, trying to put aside my preconceptions. :)
One thing I did realise in the process: it should not be possible to add
elemnts from the base classes from the admin interface because only
those elemnets from the child classes do make sense. I will thus need to
find a way to disable in the admin interface the possibility to work
directly on the Vertex and Edge classes, but I expect the documentation
will tell me how to achieve this.
Thanks agian,
Seb.
Mike Dewhirst (2024/02/25 23:31 +1100):
> SebYour assumption is too pessimistic. The class name of the (child) instance will always be returned from obj.__class__If instead your method returned the type(obj) you would see the base class name instead of the child class name.Perhaps you need to establish some tests which assert what you expect. I find such an approach very comforting.CheersMike--(Unsigned mail from my phone)
> -------- Original message --------From: Sébastien Hinderer <Sebastien...@ens-lyon.org> Date: 25/2/24 21:46 (GMT+10:00) To: django...@googlegroups.com Subject: Re: Data model design questio: graph in database Dear Mike,I will definitely play aroudn with your nice suggestion, but I would liketo share a concern I have. At this stage I am actually unsure how thiswill work.Indeed, suppose a vertex is given as input and one tries to query thedatabase about all the edges that originate from this vertex. The answerwill be under the form of a list of edges, but those will be of classEdge and I assume that their type method will return the Edge classand not the class associated to the actual child.Likewise, once given the list of edges, one has access to the list ofvertices that are neiighbourgs of the original one, but under the formof a list of objects of the base class, Vertex, with still no way to findthe type of the child object that has given rise to that Vertex one.Am I perhaps missing something here?Many thanks in advance and apologies for th likely naïve question,Seb.-- You received this message because you are subscribed to the Google Groups "Django users" group.To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ZdsadQngSnjTBimQ%40om.localdomain.
>
Sébastien Hinderer
Feb 26, 2024, 6:23:20 AMFeb 26
to django...@googlegroups.com
Hi again,
Jason (2024/02/25 17:28 -0800):
I'm actually a bit nervous about giving up completely the relational
model. So I'm wondering whether there are graph extensions to MySQL or
PostgreSQL that we could use, which would give us the best of both
(graph and relational) worlds.
Does Apache AGE for instance work well with Django? Or is there another
similar system that would have good Django support?
Jason (2024/02/25 17:28 -0800):
> Question about this, why focus on using a rdbms for this functionality vs
> using an actual graph db like neo4j with neomodel
> <https://github.com/neo4j-contrib/django-neomodel>?
> using an actual graph db like neo4j with neomodel
I'm actually a bit nervous about giving up completely the relational
model. So I'm wondering whether there are graph extensions to MySQL or
PostgreSQL that we could use, which would give us the best of both
(graph and relational) worlds.
Does Apache AGE for instance work well with Django? Or is there another
similar system that would have good Django support?
Ryan Nowakowski
Feb 26, 2024, 6:40:43 PMFeb 26
to django...@googlegroups.com
You might consider using GenericRelation. That would allow you to query the edges that have a certain vertex type.
https://docs.djangoproject.com/en/5.0/ref/contrib/contenttypes/#generic-relations
https://docs.djangoproject.com/en/5.0/ref/contrib/contenttypes/#generic-relations
Mike Dewhirst
Feb 27, 2024, 1:11:32 AMFeb 27
to django...@googlegroups.com
On 26/02/2024 8:00 pm, Sébastien
Hinderer wrote:
I will thus need to find a way to disable in the admin interface the possibility to work directly on the Vertex and Edge classes, but I expect the documentation will tell me how to achieve this.
In fact you need to enable the classes you want - as
per the docs - so simply omitting the base classes will do the
trick for you.
Sébastien Hinderer
Feb 27, 2024, 1:17:59 AMFeb 27
to django...@googlegroups.com
Dear mike,
Mike Dewhirst (2024/02/27 17:10 +1100):
Seb.
Mike Dewhirst (2024/02/27 17:10 +1100):
> On 26/02/2024 8:00 pm, Sébastien Hinderer wrote:
> > I will thus need to find a way to disable in the admin interface the
> > possibility to work directly on the Vertex and Edge classes, but I
> > expect the documentation will tell me how to achieve this.
>
> In fact you need to enable the classes you want - as per the docs - so
> simply omitting the base classes will do the trick for you.
You are totally right! Many thanks!
> > I will thus need to find a way to disable in the admin interface the
> > possibility to work directly on the Vertex and Edge classes, but I
> > expect the documentation will tell me how to achieve this.
>
> In fact you need to enable the classes you want - as per the docs - so
> simply omitting the base classes will do the trick for you.
Seb.
Sébastien Hinderer
Feb 29, 2024, 2:21:40 PMFeb 29
to django...@googlegroups.com
Dear Ryan,
'Ryan Nowakowski' via Django users (2024/02/26 17:39 -0600):
I'll definitely try that, too!
Best wishes,
Seb.
'Ryan Nowakowski' via Django users (2024/02/26 17:39 -0600):
> You might consider using GenericRelation. That would allow you to query the edges that have a certain vertex type.
>
> https://docs.djangoproject.com/en/5.0/ref/contrib/contenttypes/#generic-relations
That looks indeed very powerful and useful, thaks!
>
> https://docs.djangoproject.com/en/5.0/ref/contrib/contenttypes/#generic-relations
I'll definitely try that, too!
Best wishes,
Seb.
Reply all
Reply to author
Forward
0 new messages