Re: Vdcd Build

[Lukas Zeller]

Hi Mustafa,

for some reason the forward of your last post on the plan44_vdcd mailing list to my inbox failed, so I did not notice it. Sorry for that - but please still use the mailing list, normally it works. And in case you get no answer again, just ping me after a few days. It's best for the project if discussions go on in the open on the mailing list.

Now for the question at hand:

> On 30.05.2016, at 13:01, mustaf...@armakom.com wrote:
>
> At last i resolved the problem. Now when i run "make all" command i am getting
>
> In file included from src/p44utils/jsoncomm.hpp:27:0,
> from src/p44utils/jsoncomm.cpp:22:
> src/p44utils/jsonobject.hpp:27:18: fatal error: json.h: No such file or directory

The problem with json-c is that it's not same in all operating system versions, so names and paths vary.
So it is essential that the autoconf step really works, because that's the mechanism that figures out the right paths to json-c in your installation.

I assume you are compiling according to the "docs/How to build vdcd (and vdsm) on Linux - in particular Raspian on P44-DSB-X.md" file.

Did the "autoconf -i" step work ok? If not, try "autoconf -fi".

Please also make sure that you are using current master (dc701eb). In older versions, the configure.ac script was broken and did not find json-c automatically.
When you still get the error with dc701eb, check the make output. The compiler lines should have a "-I" option pointing to the place on your system where json.h can be found. In my ubuntu test system it is "/usr/include/json-c".

Best Regards,

Lukas

PS: Could you solve the SSL problem with mongoose in the meantime? The reason for that is that mongoose does not link the SSL libraries, but loads them programmatically. For that, it needs "libssl.so" and "libcrypto.so" to be present. That's not always the case, depending on the system these libs are present with version infomration only, like "libcrypto.so.1.0.0", "libssl.so.1.0.0".
To make mongoose work, just add softlinks like:

cd /usr/lib
ln -s libssl.so.1.0.0 libssl.so
ln -s libcrypto.so.1.0.0 libcrypto.so

This is a bit ugly but I haven't found a better way yet. Changing the library names in mongoose would be even worse, because it would add a hard version dependency in the code.

Best Regards,

Lukas

[Mustafa Yuce]

Hi Lukas,

Thanks for your reply. I have resolved my problem. I made a mistake. After I ran "autoconf -i" command , build

is successfull. But now build vdcd doesnt work. It crashes. I dont know why?

Note : I solved my ssl error by using latest version of mongoose.cpp file. In previous versions there was an error about mg_download function. But now it has been fixed.

30 Mayıs 2016 Pazartesi 16:02:40 UTC+3 tarihinde luz yazdı:

[Lukas Zeller]

Hi Mustafa,

> On 30.05.2016, at 15:46, Mustafa Yuce <mustaf...@gmail.com> wrote:
>
> Thanks for your reply. I have resolved my problem. I made a mistake. After I ran "autoconf -i" command , build
> is successfull. But now build vdcd doesnt work. It crashes. I dont know why?

How do you start vdcd?
Does it crash already when you just use --help?

vdcd --help

Anyway, try calling it with gdb to see where it crashes

gdb --args vdcd --help

Best Regards,

Lukas

PS: Note that I just pushed a few commits to reflect the 1.6.0.2 production version of the P44-DSB. I don't think that it changes anything regarding the crash you see, but still, please pull the changes so we're talking about the same. After pulling vdcd, please don't forget to "git submodule update" to get the latest p44utils as well.

[Mustafa Yuce]

Hi Lukas,

It doesnt crashes when I run  "vdcd --help" command. Actually after I build vdcd I move this build file in to usr->bin folder. Then I disable ola command in /etc/service/vdcd/run file. Because it runs olavdcd not vdcd. May be it is the reason of crashing but i am not sure really. Log file is like this;

And I didnt understand how to use  gdb --args vdcd --help command

**************** vdcd exited with status 0, waitpid status 0 ********************

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

root         2  0.0  0.0      0     0 ?        S    11:23   0:00 [kthreadd]

root         3  0.0  0.0      0     0 ?        S    11:23   0:10  \_ [ksoftirqd/0]

root         5  0.0  0.0      0     0 ?        S<   11:23   0:00  \_ [kworker/0:0H]

root         7  0.2  0.0      0     0 ?        S    11:23   0:28  \_ [rcu_preempt]

root         8  0.0  0.0      0     0 ?        S    11:23   0:00  \_ [rcu_sched]

root         9  0.0  0.0      0     0 ?        S    11:23   0:00  \_ [rcu_bh]

root        10  0.0  0.0      0     0 ?        S<   11:23   0:00  \_ [khelper]

root        11  0.0  0.0      0     0 ?        S    11:23   0:00  \_ [kdevtmpfs]

root        12  0.0  0.0      0     0 ?        S<   11:23   0:00  \_ [netns]

root        13  0.0  0.0      0     0 ?        S<   11:23   0:00  \_ [perf]

root        14  0.0  0.0      0     0 ?        S    11:23   0:00  \_ [khungtaskd]

.....

30 Mayıs 2016 Pazartesi 17:00:44 UTC+3 tarihinde luz yazdı:

[Mustafa Yuce]

30 Mayıs 2016 Pazartesi 17:00:44 UTC+3 tarihinde luz yazdı:

Hi Mustafa,

[Mustafa Yuce]

Hi Lukas;

When I wrote event in to log like this;

int main(int argc, char **argv)

{

LOG(LOG_ERR, "1");

  // prevent debug output before application.main scans command line

  SETLOGLEVEL(LOG_EMERG);

LOG(LOG_ERR, "2");

  SETERRLEVEL(LOG_EMERG, false); // messages, if any, go to stderr

LOG(LOG_ERR, "3");

  // create the mainloop

  MainLoop::currentMainLoop().setLoopCycleTime(MAINLOOP_CYCLE_TIME_uS);

LOG(LOG_ERR, "4");

  // create app with current mainloop

  static P44Vdcd application;

LOG(LOG_ERR, "5");

  // pass control

  return application.main(argc, argv);

}

I see just "1" in the log file.

30 Mayıs 2016 Pazartesi 17:07:55 UTC+3 tarihinde Mustafa Yuce yazdı:

[Lukas Zeller]

Hi Mustafa,

> On 30.05.2016, at 16:36, Mustafa Yuce <mustaf...@gmail.com> wrote:
>
> Hi Lukas;
>
> When I wrote event in to log like this;
>

> [...]

> I see just "1" in the log file.

Hmm. What is the command line you start vdcd with?

Before replacing the existing vdcd in /usr/bin, I would recommend to do the testing as described in "docs/How to build vdcd (and vdsm) on Linux - in particular Raspian on P44-DSB-X.md", under "Run vdcd (and vdsm if needed) for experimenting".

> And I didnt understand how to use gdb --args vdcd --help command

gdb is the gnu debugger, maybe you need to "apt-get install gdb" first.

And then you can just type what I wrote:

gdb --args /dir/where/you/have/built/your/vdcd --help

This runs "/dir/where/you/have/built/your/vdcd --help" under control of the debugger, and when vdcd crashes, it will give a reason why.

> It doesnt crashes when I run "vdcd --help" command. Actually after I build vdcd I move this build file in to usr->bin folder. Then I disable ola command in /etc/service/vdcd/run file. Because it runs olavdcd not vdcd. May be it is the reason of crashing but i am not sure really. Log file is like this;
>

> **************** vdcd exited with status 0, waitpid status 0 ********************

This does not look like a crash, but more like a problem with the command line arguments passed, but these are hard to see when the vdcd is autostarted by runit.
So, as said above, you should first test your vdcd by hand, adding the command line arguments you really need one by one.

Note that if you are not using olavdcd, you cannot use the --ola command line switch. Also check --ledchain, which is also not compiled in by default.

Best Regards,

Lukas

[Mustafa Yuce]

Hello Lukas;

At last I have run the application with arguments "~/ds/vdcd/vdcd --sqlitedir ~/ds/data --externaldevices 8999 --externalnonlocal --cfgapiport 8090 --webuiport 80 --sshport 22 --icondir /usr/share/www/icons --hostname "deneme" ". But application doesn't work with runit under /usr/bin folder. I have attached my "run" script under etc\service\vdcd folder. May be this might be incorrect.

Anyway I have changed externaldevicecontainer.cpp as I can add custom channels to the output. In previous versions you 

have a CustomChannel class but now it doesnt exist. Now I have changed externaldevicecontainer.cpp and added this CustomChannel class. This is my injection code in externalDeviceContainer.cpp at line 610;

................................................................................................................................

................................................................................................................................

  else if (outputType=="basic") {

    if (primaryGroup==group_variable) primaryGroup = group_black_joker;

    if (outputFunction==outputFunction_custom) outputFunction = outputFunction_switch;

    // - use simple scene settings

    installSettings(DeviceSettingsPtr(new SceneDeviceSettings(*this)));

    // - add generic output behaviour

    OutputBehaviourPtr o = OutputBehaviourPtr(new OutputBehaviour(*this));

    o->setHardwareOutputConfig(outputFunction, outputFunction==outputFunction_switch ? outputmode_binary : outputmode_gradual, usage_undefined, false, -1);

    o->setHardwareName(hardwareName);

    o->setGroupMembership(primaryGroup, true); // put into primary group

    o->addChannel(ChannelBehaviourPtr(new DigitalChannel(*o)));

    addBehaviour(o);

  }

  else {

    // no output, just install minimal settings without scenes

    installSettings();

  } 

....................................................................

....................................................................

//And here is my code part;

if (aInitParams->get("channels", o) && output) {

 

  for (int i = 0; i < o->arrayLength(); i++) {

  JsonObjectPtr o2 = o->arrayGet(i);

  JsonObjectPtr o3;

  

  int buttonId = 0;

  DsChannelType channelType = channeltype_default;

  string channelName = "";

  double aMin = 0;

  double aMax = 100;

  double aResolution = 0;

  double aDimPerMs = 7;

  double aMinDim = 0;

  

  if (o2->get("channeltype", o3)) channelType = (DsChannelType)o3->int32Value();

  if (o2->get("channelname", o3)) channelName = o3->stringValue();

  if (o2->get("amin", o3)) aMin = o3->doubleValue();

  if (o2->get("amax", o3)) aMax = o3->doubleValue();

  if (o2->get("aresolution", o3)) aResolution = o3->doubleValue();

  if (o2->get("adimperms", o3)) aDimPerMs = o3->doubleValue();

  if (o2->get("amindim", o3)) aMinDim = o3->doubleValue();   

  

   output->addChannel(ChannelBehaviourPtr(new CustomChannel(*output,   

  channelType,

  channelName.c_str(), 

  aMin, 

  aMax, 

  aResolution, 

  aDimPerMs, 

  aMinDim)));

  }

  }

And this is my custom channel class that I have added to channelBehaviour.hpp file;

  class CustomChannel : public ChannelBehaviour

  {

    typedef ChannelBehaviour inherited;

    DsChannelType channelType;

    string name;

    double min;

    double max;

    double resolution;

    double dimPerMS;

    double minDim;

  public:

    CustomChannel(

      OutputBehaviour &aOutput,

      DsChannelType aChannelType,

      const char *aName,

      double aMin,

      double aMax,

      double aResolution,

      double aDimPerMS,

      double aMinDim

    ) :

      inherited(aOutput),

      channelType(aChannelType),

      name(aName),

      min(aMin),

      max(aMax),

      dimPerMS(aDimPerMS),

      minDim(aMinDim)

    {};

    virtual DsChannelType getChannelType() { return channelType; };

    virtual const char *getName() { return name.c_str(); };

    virtual double getMin() { return min; };

    virtual double getMax() { return max; };

    virtual double getDimPerMS() { return dimPerMS; };

    virtual double getMinDim() { return minDim; };

  };

What is your opinion about this injection? If you improve your application by adding this part it will be grateful for me.  

Kind Regards,

Mustafa Yuce.

30 Mayıs 2016 Pazartesi 23:03:46 UTC+3 tarihinde luz yazdı:

[Mustafa Yuce]

Hi Lukas,

I ran this custom vdcd succesfully and identified external device. I see external and other device on vdcd web site. But when i look at the digital strom (dss) web site I  cant see these devices. When ı run custom vdcd I have an error that says ;

[2016-05-31 14:37:28.701 N] avahi: starting client

[2016-05-31 14:37:28.720 E] Failed to create client: Daemon not running (-26)

[2016-05-31 14:37:28.720 E] Failed to create client: Daemon not running (-26)

Can it be the reason of this? 

Kind Regards,

30 Mayıs 2016 Pazartesi 23:03:46 UTC+3 tarihinde luz yazdı:

Hi Mustafa,

[Lukas Zeller]

Hello Mustafa,

> On 31.05.2016, at 15:44, Mustafa Yuce <mustaf...@gmail.com> wrote:
>
> I ran this custom vdcd succesfully and identified external device. I see external and other device on vdcd web site. But when i look at the digital strom (dss) web site I cant see these devices. When ı run custom vdcd I have an error that says ;
>
>
> [2016-05-31 14:37:28.701 N] avahi: starting client
> [2016-05-31 14:37:28.720 E] Failed to create client: Daemon not running (-26)
> [2016-05-31 14:37:28.720 E] Failed to create client: Daemon not running (-26)
>
> Can it be the reason of this?

Yes - dSS (so far, only testing feed dSS!) discovers vdcd via Avahi (aka Bonjour/mdns).

The vdcd of the P44-DSB-X is built with avahi-core, which means it manages avahi announcements and discovery internally. This has the advantage of a smaller footprint of the entire system (no dbus required), but the disadvantage is that no other process in the system can also use avahi.

On the contrary, a standard build of vdcd (like yours) uses avahi-client, which uses linux dbus to connect to the avahi-daemon process.

Now you can either:

- install the avahi daemon (apt-get install avahi-daemon)

or

- build vdcd with avahi-core. If you call configure with --enable-p44buildrpi:

./configure --enable-p44buildrpi

then you will get a vdcd that includes avahi-core. You need to install libavahi-core-dev before make (apt-get install libavahi-core-dev).

Best Regards,

Lukas

[Mustafa Yuce]

1 Haziran 2016 Çarşamba 17:49:44 UTC+3 tarihinde luz yazdı:

[Lukas Zeller]

Hi Mustafa,

thanks for you input - and sorry for the late resonse. I just haven't found the time yet to really check your code and the implications. At a first glance it looks technically ok.

However I don't yet understand what you want to achieve by just adding channels.

A channel has to be backed by an appropriate behaviour, which has a suitable scene table and can load and save the channels, knows the correct default values etc. The behaviour defines the channels, a channel without a behaviour which manages it, is pretty useless. That's why I did not provide a way to add channels in the external device API, and why I removed the CustomChannel.

So far, the only behaviours where digitalSTROM understands more than a single channel are color lights (V, H, S, Ct, cieX, cieY) and blinds (position, angle). The new dS specification for audio also has multiple channels, but that's not yet fully implemented and still subject to change.

For all other uses, like simple lights, heating valves, electric heaters, relays etc. digitalSTROM uses single channel devices, which can be created as external devices using the "basic" output type.

Could you please explain what is the purpose why you want custom channels? What functions are these channels going to control? That would help me to understand how these extra channels should fit and what behaviour support might be needed for them.

Best Regards,

Lukas

PS: to propose changes in the mailing list, you might want to use git's format-patch command. This produces patches in diff format, which are human readable but can also be applied directly using "git am". For example, to format a single git commit as patch, just use "git format-patch --stdout xxxx^!" (xxxx=commit's SHA).

[Mustafa Yuce]

Hi Lukas,

Thanks for your response. We are implementing the knx protocol for a climate control. We are defining this device as an external device that's output is "heatingvalve".  "heatingvalve" output has one channel that is "HeatingLevelChannel". Here we need more channels. For example this device has some outputs like "Fan Level", "Current Temperature", "Heating/Cooling", etc... We think these properties are the output channels of the device. So we define all of these properties as the channel of the output. When we get request about this device from dSS , we can see all the channels we defined. And channel values(device properties) can be set and read bidirectional. Do you have any other advice about this issue ? And I wonder if we make a mistake in this issue?

Kind Regards,

Mustafa Yuce.

18 Haziran 2016 Cumartesi 11:24:42 UTC+3 tarihinde luz yazdı:

[Lukas Zeller]

Hi Mustafa,

thanks a lot for the information! This makes the whole thing much more clear for me.

> On 20.06.2016, at 09:01, Mustafa Yuce <mustaf...@gmail.com> wrote:
>
> Thanks for your response. We are implementing the knx protocol for a climate control. We are defining this device as an external device that's output is "heatingvalve". "heatingvalve" output has one channel that is "HeatingLevelChannel". Here we need more channels. For example this device has some outputs like "Fan Level", "Current Temperature", "Heating/Cooling", etc... We think these properties are the output channels of the device. So we define all of these properties as the channel of the output. When we get request about this device from dSS , we can see all the channels we defined. And channel values(device properties) can be set and read bidirectional.

Ok. If reading and writing the channel via dSS JSON API is all you need, then indeed you can do it this way. You will need to write your own dSS-App to actually control these extra channels, because the dSS heating app will not touch it. Also, the extra channel's values will not be saved into scenes (only the standard heatinglevel channel will).

So while this technically might work, it's not very dS-style. MOC (multi-output-channel) devices are not meant to just represent a bunch of output values, but a system defined functional model (like color light is).

Therefore, I would recommend to make multiple virtual devices from such a multi-functional device, from your example:

- one device controlling the heatinglevel output and the heating/cooling switchover. Note that cooling is already built into the ClimateControlBehaviour - the "heatinglevel" value can be negative to activate cooling, and there's a property "heatingSystemCapability" that can be 1 for "heating only", 2 for "cooling only" and 3 for "heatingAndCooling". If your device can heat and cool, just make sure you activate cooling for negative values.

- one device for the fan level. This is similar to a dimmer, simply a 0..100% value (just with larger steps if your fan only has 0-5).

- "Current Temperature" on the other hand is definitely not a channel, but a sensor. You can't write the current temperature, only read it. So this should be implemented as a sensor. You can add this sensor to one of the controlling devices (I'd say it would fit better to the heating/cooling control than the fan), or create a separate device for it.

This way your devices will be able to make use of existing (and future) digitalSTROM support for heating/cooling, and you will have scene support.

Best Regards,

Lukas

[Mustafa Yuce]

Hi Lukas,

Thanks for your quick response. 

If we define lots of device for all of the properties it doesnt make any sense. Because these are not seperate devices in reality. These are the output channels of the devices. Why dont you advice these channels. Could you please explain us in more detail?

You wrote ;  "Also, the extra channel's values will not be saved into scenes (only the standard heatinglevel channel will).". How scenes and output channels are related. Is there any document that these informations are explained clearly. 

You wrote ; " Also, the extra channel's values will not be saved into scenes (only the standard heatinglevel channel will). ". Could you please a document that explains the standart output channels. 

I quess you are trying to tell about scene commands. (In scene 1 -> Output value 1, In scene 2 -> Output value 2, .... etc). We do not need any relation with output channel and scene. We will show just output channels and the user will adjust values of the channels those we defined. (If we have misunderstanding about logic, please correct us)

PS : Our main problem here is that we can not find document that explains the digital strom technical side clearly. 

Kind Regards, 

Mustafa Yuce.

20 Haziran 2016 Pazartesi 13:00:44 UTC+3 tarihinde luz yazdı:

[Lukas Zeller]

Hi Mustafa,

> On 20.06.2016, at 12:45, Mustafa Yuce <mustaf...@gmail.com> wrote:
>
> If we define lots of device for all of the properties it doesnt make any sense. Because these are not seperate devices in reality. These are the output channels of the devices. Why dont you advice these channels. Could you please explain us in more detail?

My idea (maybe not suitable for your devices) was to break down a multifunction device into pieces that actually exist in the digitalstrom world. A cooler/heater exist. A temperature sensor exists. A fan can be modelled like a dimmer without loosing much.

> You wrote ; "Also, the extra channel's values will not be saved into scenes (only the standard heatinglevel channel will).".

This relates to your extension in the vdcd source - just adding channels does not automatically create the mechanisms to save/load in scene tables. You could create a specialized subclass of ClimateControlBehaviour to implement scene table for more channels.

> How scenes and output channels are related.

The whole idea of digitalSTROM is built on scenes on the device level! A scene can be seen as a snapshot of all the output (channel) values, that can be created with "saveScene" and recalled with "callScene". That's the main mechanism for operating devices in digitalSTROM.
For regulation-type operations, there's the "setControlValue" call, which is currently used by the heating app for distributing the "heatingLevel" value to devices.

However, directly setting output channel values is something that is indended *only* for configuration/testing, and should never be used to operate a device (see Rule 10, page 23 of http://developer.digitalstrom.org/Architecture/ds-basics.pdf).

> Is there any document that these informations are explained clearly.

There's quite some basic information on https://www.digitalstrom.org/allianz/entwickler/developer-resources, in the "System documentation" and "System APIs" sections.

> I quess you are trying to tell about scene commands. (In scene 1 -> Output value 1, In scene 2 -> Output value 2, .... etc).

Yes.

> We do not need any relation with output channel and scene.

I realize that - but I think this way, all you get is a pretty expensive solution to wire a climate device to a remote control ;-)

You can do that technically, but you bypass the entire digitalSTROM architecture. None of the built-in apps will be able to do anything useful with your devices, no scene responder, no heating/cooling controller, no timers, no presence controller logic etc. These all rely on the scene concept.

> We will show just output channels and the user will adjust values of the channels those we defined.

How will the user do that? Will you build your own dSS App or smartphone apps for that?

> (If we have misunderstanding about logic, please correct us)
> PS : Our main problem here is that we can not find document that explains the digital strom technical side clearly.

My own knowledge about the overall system design (and especially plans to extend it) are limited - I'm not a digitalSTROM employee. If the linked documents can't answer the questions, I'd recommend to contact the actual digitalSTROM developers on the dSS mailing list (http://forum.digitalstrom.org/cgi-bin/mailman/listinfo/dss-developer).

Best Regards,

Lukas

[Mustafa Yuce]

Hi Lukas;

Ok it is clear that why you advise this way. I just wonder where scene -> output value pair is recorded ? (vdsm ? vdcd ? ....)

I looked at your vdcd codes and i couldnt see anywhere that you save these pair values.

Kind Regards,

20 Haziran 2016 Pazartesi 17:10:56 UTC+3 tarihinde luz yazdı:

[Lukas Zeller]

Hi Mustafa,

> On 23.06.2016, at 16:38, Mustafa Yuce <mustaf...@gmail.com> wrote:
>
> [...] I just wonder where scene -> output value pair is recorded ? (vdsm ? vdcd ? ....)

The scene table is a property of the digitalSTROM device - for digitalSTROM terminal blocks, it is in fact stored in the block itself.

For virtual devices, the scene table is part of the vdcd (which is where virtual devices are created as software objects, controlling third-party hardware)

> I looked at your vdcd codes and i couldnt see anywhere that you save these pair values.

The mechanism is a bit complicated, because it also manages behaviour-specific default values for the scene table, and because it does not hold the complete scene table in memory nor in the database. The database only contains those scenes the user has actually changed, all other scenes (with their values) are created from defaults on-the fly when needed.

Another reason why the scene save/load might not be so obvious is because it's not always an 1:1 save of output values. For example the color lights have 6 channels, but only three or two of them are active at the same time. You either store colors as HSV, or CIE x/y or color temperature, but not all of them together.

The actual mechanisms for that are in the DsScene/SceneDeviceSettings/OutputBehaviour class hierarchies.

DsScene descendants have the code to save a single scene to the database - see SimpleScene for a simple single-value example, and ColorLightScene an example how 6 channels are mapped to 3 saved values as mentioned above.

Each DsScene subclass also has a SceneDeviceSettings subclass which contains the factory method to instantiate the scene and initialize it with default values.

And multi-channel scenes also need a OutputBehaviour subclass that can actually handle the channels and the required digitalSTROM behaviour (apply color, blink and dim for lights, move to specific positions and angles for blinds, etc.). In particular, the behaviour must provide applyScene() and captureScene(), as well as loadChannelsFromScene() and saveChannelsToScene().

All of this is not really made to be extended just for a specific device. It's core functionality of the vdcd, which gets new behaviours/scenes/channels when the digitalSTROM architecture evolves and actually provides specifications.

Of course, climate control is an area where digitalSTROM needs to evolve. There are recent steps with the support for simple cooling. If you have ideas what would be needed on the conceptual level beyond what's there, let us know, so there's a chance it gets added.
For the moment however I'd recommend to map functionality onto single-channel devices, because that's the only thing really supported throughout the dS system right now.

Best Regards,

Lukas