How can I access field values generically?

[Mike Dewhirst]

Consider a chemical mixture with a bunch of ingredients. Both mixture
and ingredients are instances of the same class so they have the same
fields. They also have many related models, including 1:1, 1:n and n:m.

Each related model will have none or many TextField's.

The objective is to programmatically fill empty mixture text fields with
concatenated content from the ingredients. The concatenated content
would be separated by ingredient name titles for the user to deal with
the content more easily.

I don't necessarily need all mixture text fields filled this way but it
certainly makes sense for some. With a couple of related models I'd
concatenate all text fields, with most though I'd like to pick and
choose by field name and I'd ignore some models completely.

The following model method is working properly as described but it is
mostly boiler-plate. It also only covers the first few of a large number
of related models with text fields. If I keep using this technique it
will add hundreds of LOC. Yuk.

The question is how can I refactor this and make it generic? Perhaps
using the _meta API?

Any guidance appreciated


(In the abstract ancestor class of the Solid, Liquid and Gas classes)
def concat_fields(self, ingredients): """ ingredients is a queryset of
substance-to-substance m2m records. A substance has one physical state
object being gas, liquid or solid each of which inherits from
core_fields and the fields *here* we wish to concatenate text from (at
the moment) all come from core_fields. """ assert ingredients # populate
the list of ingredient physical state objects state_objs = list() for
m2m in ingredients: # substance has physical state so it knows which
model to return
state_objs.append(m2m.ingredient.get_physical_state_object()) # get the
text concatenated if not self.stability_comment: comment = "" for obj in
state_objs: if obj.stability_comment: name = obj.substance.name text =
obj.stability_comment comment = "{0}\n{1}: {2}".format(comment, name,
text) comment = comment.strip() if comment: self.stability_comment =
comment # and repeat if not self.reactivity: comment = "" for obj in
state_objs: if obj.reactivity: name = obj.substance.name text =
obj.reactivity comment = "{0}\n{1}: {2}".format(comment, name, text)
comment = comment.strip() if comment: self.reactivity = comment # ad
nauseam if not self.reaction_hazards: comment = "" for obj in
state_objs: if obj.reaction_hazards: name = obj.substance.name text =
obj.reaction_hazards comment = "{0}\n{1}: {2}".format(comment, name,
text) comment = comment.strip() if comment: self.reaction_hazards =
comment if not self.avoid: comment = "" for obj in state_objs: if
obj.avoid: name = obj.substance.name text = obj.avoid comment =
"{0}\n{1}: {2}".format(comment, name, text) comment = comment.strip() if
comment: self.avoid = comment if not self.incompatibilities: comment =
"" for obj in state_objs: if obj.incompatibilities: name =
obj.substance.name text = obj.incompatibilities comment = "{0}\n{1}:
{2}".format(comment, name, text) comment = comment.strip() if comment:
self.incompatibilities = comment

Thanks

Mike

[Avraham Serour]

please format the code at the bottom

Mike


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/592563df-d9b9-3d87-fcdc-9c68d18ab196%40dewhirst.com.au.
For more options, visit https://groups.google.com/d/optout.

[Mike Dewhirst]

Sorry about that ... here it is again.
- - - - - - - - - - - - - - - - - - -

state_objs.append(m2m.ingredient.get_physical_state_object())

# get the text concatenated
if not self.stability_comment:
comment = ""
for obj in state_objs:
if obj.stability_comment:
name = obj.substance.name
text = obj.stability_comment
comment = "{0}\n{1}: {2}".format(comment, name, text)
comment = comment.strip()
if comment:
self.stability_comment = comment

if not self.reactivity:
comment = ""
for obj in state_objs:
if obj.reactivity:
name = obj.substance.name
text = obj.reactivity
comment = "{0}\n{1}: {2}".format(comment, name, text)
comment = comment.strip()
if comment:
self.reactivity = comment

[Mike Dewhirst]

Bumping this question again, I have done all the individual
concatenations with the following model method ...

def concat_fields(self, ingredients):
""" ingredients is a queryset of substance:substance m2m records
with the second FK to substance in a field called "ingredient"
Objective is concatenate text from all text fields into the mixture
"""
if ingredients:
objects = list()
ellip = "..."
for m2m in ingredients:
obj = m2m.ingredient.get_health()
if obj:
objects.append(obj)
if objects:
# first of seven text fields in this model
comment = self.ototoxic_comment or ""

comment = comment.strip()
if comment:

comment = "{0}\n".format(comment.strip())
if not comment or ellip in comment:
for obj in objects:
if not obj.substance.name in comment:
if obj.ototoxic_comment:
comment = "{0}{1}: {2}\n".format(comment,
obj.substance.name, obj.ototoxic_comment)
if comment:
self.ototoxic_comment =
comment.replace(ellip, "")
# next of seven text fields in this model and so on
...

There are 90 occurrences of this pattern in 18 concat_fields() methods
in 18 models which are all much the same.

This offends me but I don't know how to start on the necessary "meta"
programming to make it somewhat more elegant.

Here is the on-screen help text for the user ...

"For mixtures, expandable blank fields "\
"below will be populated with ingredient data from the same fields. "\
"Edit as required. To retrieve that data again add an ellipsis (...) "\
"somewhere in the field and click [Save]"

Any advice would be appreciated. And appreciation might involve red wine.

Thanks

Mike

--
PLEASE NOTE OUR NEW LANDLINE IS +61 (0) 3 9034 3977

Climate Pty Ltd
PO Box 308
Mount Eliza
Vic 3930
Australia +61

T: 03 9034 3977
M: 0411 704 143

[C. Kirby]

Mike, I've done a lot of work with Model meta, and I'm pretty sure I can give you at least the bones of a solution, but I can't really get my head around the problem. Could you post a set of related models and what you would expect the result to look like? 

[Mike Dewhirst]

On 21/12/2016 9:10 PM, C. Kirby wrote:
> Mike, I've done a lot of work with Model meta, and I'm pretty sure I
> can give you at least the bones of a solution, but I can't really get
> my head around the problem. Could you post a set of related models and
> what you would expect the result to look like?

I have a main model (substance) with a bunch of child models
(properties, toxicity, health, spill etc) containing TextFields with
advice for that substance. There are obviously lots of substances and
they carry varying advice in their child models. Two of them are MMA
and Styrene.

With a new substance (eg Base Resin CK90) which is a mixture of those
two substances, the chemical manufacturer needs to assemble advice in
those same TextFields in the CK90 mixture as the ones in the child
models carrying the varying advice. In some cases that advice (eg spill
advice) can come out of an expert's head and just be typed in. In some
cases the expert may prefer to see the spill advice from all the
ingredient substances first. Like so ... [1]



... editing out the ingredient names - in this case MMA and Styrene -
and combining the advice for the mixture, getting rid of redundant or
repetitive advice.

The concat_fields method which sources the advice only works for a
mixture (ie substance has ingredients). It only works on a blank field
or (when signalled) replaces advice from one or more ingredients. The
user deletes one (or more) of the ingredient names and inserts an
ellipsis in the CK90 field. The advice from that/those ingredient(s)
will be appended. If another ingredient is added to the mixture and an
ellipsis is inserted in the CK90 mixture TextField, the new ingredient
advice is appended.

The models are pretty standard ... there are more than the 1:1 (Spill)
and 1:n (Exposure) shown here.

Substance (main model)
ingredients = models.ManyToManyField('self', symmetrical=False,
blank=True,
through='Substance_Ingredients',)

Substance_Ingredients (m2m 'through' table)
substance = models.ForeignKey('Substance')
ingredient = models.ForeignKey('Substance')
proportion = models.DecimalField()

Spill (1:1 model)
substance = models.OneToOneField('Substance')
small = models.TextField(verbose_name="Small spill",
help_text="Response recommended for spills around 25 litres.")

Exposure (1:n model)
substance = models.ForeignKey('Substance', related_name='route')
symptoms = models.TextField(help_text="Potential adverse health
effects and symptoms from "
"the first at lowest exposure through to consequences of severe
exposure.")

Thanks CK

Mike

[1] For those not seeing the embedded image, it is a TextField
containing the following words:
Methyl methacrylate monomer: Soak up with inert absorbent material (e.g.
silica gel, acid binder, universal binder, sawdust). Keep in suitable,
closed containers for disposal.
Styrene: Soak up with inert absorbent material (e.g. sawdust). Keep in
closed containers for disposal.

>
> On Wednesday, December 21, 2016 at 4:55:49 AM UTC+2, Mike Dewhirst wrote:
>
> Bumping this question again, I have done all the individual
> concatenations with the following model method ...
>
> def concat_fields(self, ingredients):
> """ ingredients is a queryset of substance:substance m2m records
> with the second FK to substance in a field called "ingredient"
> Objective is concatenate text from all text fields into the
> mixture
> """
> if ingredients:
> objects = list()
> ellip = "..."
> for m2m in ingredients:
> obj = m2m.ingredient.get_health()
> if obj:
> objects.append(obj)
> if objects:
> # first of seven text fields in this model
> comment = self.ototoxic_comment or ""
> comment = comment.strip()
> if comment:
> comment = "{0}\n".format(comment.strip())
> if not comment or ellip in comment:
> for obj in objects:
> if not obj.substance.name

> <http://obj.substance.name> in comment:

> if obj.ototoxic_comment:
> comment = "{0}{1}:
> {2}\n".format(comment,

> obj.substance.name <http://obj.substance.name>, obj.ototoxic_comment)

> <http://obj.substance.name>

> > text = obj.stability_comment
> > comment = "{0}\n{1}: {2}".format(comment, name,
> text)
> > comment = comment.strip()
> > if comment:
> > self.stability_comment = comment
> >
> > if not self.reactivity:
> > comment = ""
> > for obj in state_objs:
> > if obj.reactivity:
> > name = obj.substance.name

> <http://obj.substance.name>

> > text = obj.reactivity
> > comment = "{0}\n{1}: {2}".format(comment, name,
> text)
> > comment = comment.strip()
> > if comment:
> > self.reactivity = comment
> >
> > if not self.reaction_hazards:
> > comment = ""
> > for obj in state_objs:
> > if obj.reaction_hazards:
> > name = obj.substance.name

> <http://obj.substance.name>

> > text = obj.reaction_hazards
> > comment = "{0}\n{1}: {2}".format(comment, name,
> text)
> > comment = comment.strip()
> > if comment:
> > self.reaction_hazards = comment
> >
> > if not self.avoid:
> > comment = ""
> > for obj in state_objs:
> > if obj.avoid:
> > name = obj.substance.name

> <http://obj.substance.name>

> > text = obj.avoid
> > comment = "{0}\n{1}: {2}".format(comment, name,
> text)
> > comment = comment.strip()
> > if comment:
> > self.avoid = comment
> >
> > if not self.incompatibilities:
> > comment = ""
> > for obj in state_objs:
> > if obj.incompatibilities:
> > name = obj.substance.name

> <http://obj.substance.name>

> > text = obj.incompatibilities
> > comment = "{0}\n{1}: {2}".format(comment, name,
> text)
> > comment = comment.strip()
> > if comment:
> > self.incompatibilities = comment
> >
> > Thanks
> >
> > Mike
> >
> >
>
> --

> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to django-users...@googlegroups.com
> <mailto:django-users...@googlegroups.com>.

> To post to this group, send email to django...@googlegroups.com

> <mailto:django...@googlegroups.com>.

> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-users/54e68516-af14-41f4-a47b-0088a6ce023d%40googlegroups.com
> <https://groups.google.com/d/msgid/django-users/54e68516-af14-41f4-a47b-0088a6ce023d%40googlegroups.com?utm_medium=email&utm_source=footer>.

[C. Kirby]

Ill be honest, I'm still not getting how your models are structured. That is ok though, I can give you some pointers and hopefully  that will be sufficient. I'll also be very explicit in describing the steps. Several are probably better as single orm calls. _meta docs at https://docs.djangoproject.com/en/1.10/ref/models/meta/

Hope this helps. If you have any clarifying questions I am happy to chime in more.

from django.db.models import TextField
#First get all ingredients you are interested in (I am assuming that
#mixtures are only one level deep, if not then build a recursive call)
ingredients = Substance_Ingredients.objects.filter(substance=<parent substance>).values_list('ingredient.pk', flat=True)
#Get the substance instances for these
child_substances = Substance.objects.filter(pk__in = ingredients)
target_text = ''
for cs  in child_substance:
    #Get all OneToOne and OneToMany fields

    models =  [
        (f, f.model if f.model != (Substance or SubstanceIngredient) else None)
        for f in Substance._meta.get_fields()
        if (f.one_to_many or f.one_to_one)
        and not f.auto_created and not f.concrete
    ]
   
    ingredient_text = ''
    for field, model in models:
        #Check the model for TextFields
        tfs = [f for f in model._met.get_fields() if isinstance(f, TextField)]
        #get the textfield attributes for your substance field
        field_text = [getattr(field, tf, '') for tf in tfs]
        #compile you text strings at the necessary levels
        ingredient_text = '{}, {}'.format(ingredient_text, '.'.join(field_text))

    child_text = '{}: {}'.format(cs.substance_name, ingredient_text)
    target_text = '{}\n{}'.formate(target_text, child_text)

#Save you target text to the original substance

[Mike Dewhirst]

Very generous. Thank you. It looks as though you did get the model
structure. Your first comment nails it because mixtures can have other
mixtures as ingredients. I'm right into recursion nowadays :)

Mike

> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users...@googlegroups.com
> <mailto:django-users...@googlegroups.com>.
> To post to this group, send email to django...@googlegroups.com
> <mailto:django...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-users/b39f85e3-1ecc-4a2f-93e7-f6f426a63520%40googlegroups.com
> <https://groups.google.com/d/msgid/django-users/b39f85e3-1ecc-4a2f-93e7-f6f426a63520%40googlegroups.com?utm_medium=email&utm_source=footer>.

[C. Kirby]

Happy to help