Dynamic API: Getting the structure field name from a Reader

[JF Gauvin]

Hi,

If we have this structure:

struct Date {
  year @0 :Int16;
  month @1 :UInt8;
  day @2 :UInt8;
}


Is it possible to dynamically get the name of the month field ("month") associated to dateReader.getMonth()?

Or, is it possible to know that getMonth() is associated with field 1 of the structure? Then, it would be possible to get the Date schema, get field 1 and get its name.


Thanks.

JF

[Farz Hemmati]

I think if you get the DynamicStruct associated with Date, then you can get its fields by looking at the getProto associated with it's schema. 

I do this in https://github.com/chainreactionmfg/capnp_generic_gen/blob/master/json.c++

Either as dateReader.getAs<DynamicStruct>() or dateReader.as<DynamicStruct>(), depending on the exact type of dateReader. It's likely the latter as<>() construct.

Once it's a DynamicStruct, you can do struct.getSchema.getFields() and loop over them to find the one with 'field.getProto().getName() == "month"', then you can look at anything else you need in the field's proto.

[JF Gauvin]

Yes, I was able to do this. But, I just want to get the name, for logging purposes. So, I dont want to search for "month" if I'm in fact looking for this. Otherwise, I would hard code this string, which I want to avoid.

[Farz Hemmati]

Well, what information do you have? The pointer to the function itself? Or the name "getMonth"? The proto has the field name, id, type, etc.

On Wed, Nov 26, 2014 at 4:57 PM, JF Gauvin <jfga...@gmail.com> wrote:

Yes, I was able to do this. But, I just want to get the name, for logging purposes. So, I dont want to search for "month" if I'm in fact looking for this. Otherwise, I would hard code this string, which I want to avoid.

--

You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at http://groups.google.com/group/capnproto.

[Kenton Varda]

It sounds like what you're asking for is something like a set of constants pointing to the StructSchema::Field for each field? E.g.:

    const capnp::StructSchema::Field Date::MONTH_SCHEMA;

Is that correct?

We could maybe add this, though there are two issues:

- Is the code bloat worth it?

- We don't want to #include capnp/schema.h in every generated header. We'll have to find a hack around that.

-Kenton

[JF Gauvin]

Exactly. For example, with a union part of a struct, I'm actually able to get the field name being used, using fields[reader.which()].

[JF Gauvin]

Another thing that I could do is to have an annotation for each struct fields. But, the information would be duplicated and I think it would be slower to get the annotation value each time it's required.

[JF Gauvin]

Well, I think that even this way I would need the constant to access the annotation of the schema struct field.

[JF Gauvin]

We could also do one of those things instead of adding constants.


1. Create an enum for each struct in the capnpc generated files, like it's done for unions (which enum).

For example:

struct Date {
  Date() = delete;

  class Reader;
  class Builder;
  class Pipeline;
  enum Field: uint16_t {
    YEAR,
    MONTH,
    DAY
  };

So, it's possible to get a StructSchema::Field from a StructSchema::FieldList by doing "fields[Date::YEAR]".


2. For each struct field, create a get<FieldName>Offset() function which returns the same offset used when getting a field.

inline uint16_t Date::Reader::getYearFieldOffset() const {
  return 0;
}


What do you think?

JF

Le mercredi 26 novembre 2014 20:05:11 UTC-5, Kenton Varda a écrit :

[Kenton Varda]

I think we want something like:

// in generated-header-support.h

template <typename ContainingType>

class FieldIdentifier {

public:

  explicit constexpr FieldIdentifier(uint index): index(index) {}

private:

  uint index;

  friend class Schema;

};

// in schema.h

template <typename ContainingType>

StructSchema::Field Schema::from(FieldIdentifier<ContainingType> field) {

  return Schema::from<ContainingType>().getFields()[field.index];

}

Then each struct type should declare a bunch of static constexpr members of type FieldIdentifier.

So you'd get a field like:

    StructSchema::Field field = Schema::from<MyType::FIELD_NAME>();

This is nicely type-safe, while not causing generated code to depend directly on the Schema API.

But I still worry that this is bloat-y...

-Kenton

[JF Gauvin]

I tested both cases and the second option one was better. Using an enum for field isn't easy to code the application because we always need to get the right structure and field. So, it's easy to do errors.

I made a patch for option 2, but after reading issue 158 (https://github.com/kentonv/capnproto/issues/158), related to offset calculation in structures. So, I think It's not the offset that I need to return, but the equivalent of the enum value of case 1.

(I will have a look at your post)

[JF Gauvin]

When you're saying that "Then each struct type should declare a bunch of static constexpr members of type FieldIdentifier.", this would be done by the compiler? You're right, it's seems bloaty.

[JF Gauvin]

I fixed my problem. I was using the field offset instead of the field index.

Here is what I'm doing. I'm using a macro, but I think it's quite efficient to do all the stuff I have to do, in a safe way :)


---------------------------------------------

/* Utility header file */

typedef std::list<capnp::StructSchema::Field> FieldList;
typedef std::list<capnp::StructSchema::Field>::const_iterator FieldListConstIterator;

#define UTIL_CONCAT3(a, b, c) UTIL_CONCAT2(UTIL_CONCAT2(a, b), c)
#define UTIL_CONCAT2(a, b)    UTIL_CONCAT2_(a, b)
#define UTIL_CONCAT2_(a, b)   a##b

#define UTIL_FIELD_HAS(reader,    field) UTIL_CONCAT2(reader.has, field())
#define UTIL_FIELD_READER(reader, field) UTIL_CONCAT2(reader.get, field())
#define UTIL_FIELD_INDEX(reader,  field) UTIL_CONCAT3(reader.get, field, FieldIndex())

#define UTIL_SET_COND(fct, reader, field, fieldList, ...)                                      \
    ({                                                                                         \
    bool success = true;                                                                       \
    if (UTIL_FIELD_HAS(reader, field))                                                         \
    {                                                                                          \
        success = Util::pushBackFieldList(fieldList, reader, UTIL_FIELD_INDEX(reader, field)); \
        if (success)                                                                           \
        {                                                                                      \
            success = fct(UTIL_FIELD_READER(reader, field), fieldList, ##__VA_ARGS__);         \
            Util::popBackFieldList(fieldList);                                                 \
        }                                                                                      \
    }                                                                                          \
    success                                                                                    \
    })


---------------------------------------------

/* Utility source file */

void Util::pushBackFieldList(FieldList &fieldList, const DynamicStruct::Reader &reader, uint16_t fieldIndex)
{
    StructSchema::Field field;
    if (getField(reader, fieldIndex, field))
    {
        fieldList.push_back(field);
        return true;
    }
    else
    {
        return false;
    }
}


void Util::popBackFieldList(FieldList &fieldList)
{
    KJ_ASSERT(fieldList.size() > 0);
    if (fieldList.size() > 0)
        fieldList.pop_back();
}


void Util::getField(const DynamicStruct::Reader &reader, uint16_t fieldIndex, StructSchema::Field &field)
{
    auto schema = reader.getSchema();
    auto fields = schema.getFields();

    KJ_ASSERT(fieldIndex < fields.size());
    if (fieldIndex < fields.size())
    {
        field = fields[fieldIndex];
        return true;
    }
    else
    {
        return false;
    }
}


---------------------------------------------

/* Configuration source file */

bool setConfig(const Config::Reader &reader)
{
    FieldList fieldList;

    bool success = UTIL_SET_COND(setDate, reader, Date, fieldList);

    /* ... */

    return success;
}

---------------------------------------------

In setDate, if an error is detected, I'm able to get the error path using the fieldList. So, I would be able to report it at higher level, and the JSON client would receive something like this:

{
    "code": 1,
    "dataPath": "/config/date/hour"
}

Also, I'm able to add an event in a log using this same path (e.g. /config/date/hour 22).