Channels hierarchy planning

36 views
Skip to first unread message

Matjaž Lipuš

unread,
Sep 8, 2014, 5:48:35 PM9/8/14
to pv...@googlegroups.com
Hi,

I am planning to use PVLng, but I am figuring out what the channel hierarchy should be.
I have one three phase inverter with two strings. Additional to that I have powerbox on each panel with (MPPT).
Standard sensors on inverter are: power, current, voltage, frequency, temperature
Besides this standard data inverter API supports (lifetime energy, this year energy, this month energy, today energy, current power, power 15 min resolution). I receive export energy reading once per month from electricity company, how could I include this data (previous, new kWh - incrementing)? So I can calculate internal consumption based on difference between inverter and reading energy.

I want to create channel hierarchy which would enable me to create different calculations and graphs such as relative powers, estimate etc.

Is there a resettable meter? for example: daily produced energy.
How do you get Mode and Error states?

Here is my current idea of channels hierarchy.

Power Plant
- E-total
\
 Inverter (three phase)
 - Pac
 - Iac1
 - Uac1
 - Freq1
 - Iac2
 - Uac2
 - Freq2
 - Iac3
 - Uac3
 - Freq3
 - Temperature
 - Iac - sum
 - Uac - sum
 - Idc - sum
 - Udc - sum
 - Pdc - sum
 - Freq - avg
 - Performance ratio
 - Estimate Wh
 - Estimate rel. - kWh/kWp
 - Health
 - E-total - kWh
 \
  String 1
  - Pdc - sum
  - Idc - sum
  - Udc - sum
  - Energy today - sum
  \
    powerbox 1
    - Pdc
    - Idc
    - Udc1 (panel)
    - Udc2 (optimizer)
    - Energy today
  \
    ...
 \
  String 2
  ...
- Wunderground
 ... template


Also I would appreciate if you could share your hierarchy and charts definitions.

Thanks for your comments!

--
Matjaž

KKoPV

unread,
Sep 10, 2014, 2:43:52 AM9/10/14
to
Hi,


Am Montag, 8. September 2014 23:48:35 UTC+2 schrieb Matjaž Lipuš:

I am planning to use PVLng, but I am figuring out what the channel hierarchy should be.

Jo, that's not so easy at first glance...
 
I have one three phase inverter with two strings. Additional to that I have powerbox on each panel with (MPPT).
Standard sensors on inverter are: power, current, voltage, frequency, temperature

These channels are the simple ones :-)

Which inverter model do you have? 
Which channels / data are accessible via API?
This info is needed to propose a channel hierarchy.

Besides this standard data inverter API supports (lifetime energy, this year energy, this month energy, today energy, current power, power 15 min resolution).

I suggest (and also use from my inverters) the lifetime consumption.
From this all other can be calculated (http://kk.log.solar/?chart=tagesertrag) This my daily production calculated from lifetime data.
It aggregates on daily basis and draw the "period" values (each day)
 
I receive export energy reading once per month from electricity company, how could I include this data (previous, new kWh - incrementing)?

Also the lifetime value would be good, what the electricity meters shows.
 
So I can calculate internal consumption based on difference between inverter and reading energy.

Correct.
  
Is there a resettable meter? for example: daily produced energy.

Yes, meter channels can adjust itself, see http://pvlng.com/Direct_models#Meter
 
How do you get Mode and Error states?

My inverters deliver them as plain texts and I store them on a Switcher-Channel.
 
Here is my current idea of channels hierarchy.
...

Looks good so far, but for better understanding I would like to know the real channels from your equipment.

Also I would appreciate if you could share your hierarchy and charts definitions.

Ok, but I have at the moment abt. 200 channels (real ones, calculated and aliases) and my channel tree have a length of 296...

Here are some examples:

  • The collected SMA Tripower "real" channels

  • Performance ratio based on Pac and the sum of Pdc

  • Recalulation into kWh

  • Estimated production forecast based on production last days

  • Average grid voltage

  • Estimated production for day based on PVGIS



Knut

Matjaž Lipuš

unread,
Sep 10, 2014, 9:11:30 AM9/10/14
to pv...@googlegroups.com
Hi,

I have SolarEdge inverter and power optimisers. API docs are accessible here http://www.solaredge.com/files/pdfs/se_monitoring_api.pdf

In general these are data I am interested in API:

1.
Current overview
{
    lastUpdateTime: "2014-09-07 11:41:11",
    lifeTimeData: {
        energy: 12300,
        revenue: 100
    },
    lastYearData: {
        energy: 1230
    },
    lastMonthData: {
        energy: 530
    },
    lastDayData: {
        energy: 312
    },
    currentPower: {
        power: 3000
    }
}



2.
Daily energy:
{
    timeUnit: "DAY",
    unit: "Wh",
    values: [{
        date: "2012-12-01 00:00:00",
        value: null
    }, {
        date: "2012-12-02 00:00:00",
        value: null
    }...]
}


3.
15 min resolution power-plant power
{
    power: {
        timeUnit: "QUARTER_OF_AN_HOUR",
        unit: "W",
        values: [{
            date: "2012-12-02 00:00:00",
            value: null
        }, {
            date: "2012-12-02 00:15:00",
            value: null
        }, {
            date: "2012-12-02 00:30:00",
            value: null
        }...]
    }
}


4.
Inverter telemetries (for specified time frame) at 5 min resolution.
{
    count: 1222,
    telemetries: [{
        date: "2014-09-01 05:01:15",
        totalActivePower: 0,
        dcVoltage: 12,
        temperature: 20.2801,
        L1Data: {
            acCurrent: 0,
            acVoltage: 241,
            acFrequency: 50
        },
        L2Data: {
            acCurrent: 0,
            acVoltage: 237,
            acFrequency: 49.99
        },
        L3Data: {
            acCurrent: 0,
            acVoltage: 242,
            acFrequency: 50
        }
    }]
}



These is some data accessible through API, but monitoring portal also supports some additional data. For example
Powerbox telemetries (every measurement recorded), data includes (time (per minute), Pdc, Idc, Udc1 (panel), Udc2 (optimizer), energy (from previous measurement)).

So in nutshell additional to data available from API, these are raw sensors/meters available:

Inverter
  • Pac
  • Iac1
  • Uac1
  • Freq1
  • Iac2
  • Uac2
  • Freq2
  • Iac3
  • Uac3
  • Freq3
  • Temperature

power optimizer
  • Pdc
  • Idc
  • Udc1 (panel)
  • Udc2 (optimizer)
  • Energy today

I have over 50 power optimisers, 2 strings & 1 inverter. That's roughly 280 sensors/meters. Without any calculated channels.

I don't have inverter consumption data. I only have difference between inverter energy and electricity company energy meter (available for previous month). I have this data from start, when meter was installed.
If you see anything I have missed, please let me know.

Thank you for all you effort for building PVLng & help. I would like to make a small donation so please let me know your Paypal account.

Thank you!
Matjaz

KKoPV

unread,
Sep 10, 2014, 11:10:54 AM9/10/14
to pv...@googlegroups.com
Hi,


Am Mittwoch, 10. September 2014 15:11:30 UTC+2 schrieb Matjaž Lipuš:

I have SolarEdge inverter and power optimisers. API docs are accessible here http://www.solaredge.com/files/pdfs/se_monitoring_api.pdf

Ok, that will generate a lot of data...
 
In general these are data I am interested in API: 
1.Current overview
 
From this we need only 
  - E-Total (lifeTimeData)
  - Pac (currentPower)

All other can be calculated.

2. Daily energy:
3. 15 min resolution power-plant power

Not used by PVLng
  
4.Inverter telemetries (for specified time frame) at 5 min resolution.

  - Udc (dcVoltage)
  - Iac1 (L1Data.acCurrent)
  - Uac1 (L1Data.acVoltage)
  - Freq1 (L1Data.acFrequency)
  - Iac2 (L2Data.acCurrent)
  - Uac2 (L2Data.acVoltage)
  - Freq2 (L2Data.acFrequency)
  - Iac3 (L3Data.acCurrent)
  - Uac3 (L3Data.acVoltage)
  - Freq3 (L3Data.acFrequency)
  - Temp (temperature)
 
Whats about Idc? and Pdc? ?!
Have we them to summarize from the optimizers, if needed?!

These is some data accessible through API, but monitoring portal also supports some additional data. For example
Powerbox telemetries (every measurement recorded), data includes (time (per minute), Pdc, Idc, Udc1 (panel), Udc2 (optimizer), energy (from previous measurement)).

So in nutshell additional to data available from API, these are raw sensors/meters available:

Inverter
  • Pac
  • Iac1
  • Uac1
  • Freq1
  • Iac2
  • Uac2
  • Freq2
  • Iac3
  • Uac3
  • Freq3
  • Temperature
 
Ok, I have the same:
  - E-Total (lifeTimeData)
  - Pac (currentPower)
  - Udc (dcVoltage)
  - Iac1 (L1Data.acCurrent)
  - Uac1 (L1Data.acVoltage)
  - Freq1 (L1Data.acFrequency)
  - Iac2 (L2Data.acCurrent)
  - Uac2 (L2Data.acVoltage)
  - Freq2 (L2Data.acFrequency)
  - Iac3 (L3Data.acCurrent)
  - Uac3 (L3Data.acVoltage)
  - Freq3 (L3Data.acFrequency)
  - Temp (temperature)

1 meter and 12 sensors if you use all of them.

power optimizer
  • Pdc
  • Idc
  • Udc1 (panel)
  • Udc2 (optimizer)
  • Energy today
Ok, also 5 channels, 4 sensors and 1 meter (auto-adjustable, will afterwards store also lifetime data. With lifetime energy for each optimizer you can later plot all of them at once and see the good and the bad ones over time...)
 
I have over 50 power optimisers, 2 strings & 1 inverter. That's roughly 280 sensors/meters.

Ok, lest put it together:

- Inverter (generic group)
  - ... (13 channels from above)
  - String 1 (generic group)
    - Whats about RAW string specific data, any?
    - Optimizer 1
      - ... 5 channels from above
    ...
    - Optimizer n
  - String 2 (generic group)
  ...
 
Without any calculated channels. 

Next question, do you need an channel from all optimizers summarized on a higher level (string), e.g. Pdc?!

Then, below string we will have also 

  - String
    - ∑ Pdc (Accumulator)
      - Pdc (optimizer 1)
      ...
      - Pdc (optimizer n)

I don't have inverter consumption data.

No, I have consumption also not from the inverter, one house consumption comes from electricity meter via S0... and the other one via D0 (IEC 62056-21).

I only have difference between inverter energy and electricity company energy meter (available for previous month). I have this data from start, when meter was installed.

Fine, lifetime data are ok.
Problem with a value "only" each (last day of?) month will be, that all charts using this meter channel with aggregation period lower 1 month are not so good...

If you see anything I have missed, please let me know.

I'll build some templates and test the logic in the demo installation, give me a day ;-) ...
 
Thank you for all you effort for building PVLng & help. I would like to make a small donation so please let me know your Paypal account.

Wow, sounds good, thank you very much! Link is now here: http://pvlng.com/Main_Page#Contribute :-)

Knut

Matjaž Lipuš

unread,
Sep 10, 2014, 3:45:50 PM9/10/14
to pv...@googlegroups.com
Yes, this should be summed from optimisers. 
I need to correct myself here. This is energy in last hour not total today. for example 7-8: 40Wh, 8-9: 60Wh,...12-13: 200Wh 13-14: 140Wh 14-15:100Wh...
 
I have over 50 power optimisers, 2 strings & 1 inverter. That's roughly 280 sensors/meters.

Ok, lest put it together:

- Inverter (generic group)
  - ... (13 channels from above)
  - String 1 (generic group)
    - Whats about RAW string specific data, any?
string is just a group of optimisers. (can have calculated channels, Idc, Pdc,.) 
    - Optimizer 1
      - ... 5 channels from above
    ...
    - Optimizer n
  - String 2 (generic group)
  ...
 
Without any calculated channels. 

Next question, do you need an channel from all optimizers summarized on a higher level (string), e.g. Pdc?!

yeah, API does not provide this data on inverter. 

Then, below string we will have also 

  - String
    - ∑ Pdc (Accumulator)
      - Pdc (optimizer 1)
      ...
      - Pdc (optimizer n)

I don't have inverter consumption data.

No, I have consumption also not from the inverter, one house consumption comes from electricity meter via S0... and the other one via D0 (IEC 62056-21).
I don't have S0 or D0 meters. :( I don't have access to energy meters.

I only have difference between inverter energy and electricity company energy meter (available for previous month). I have this data from start, when meter was installed.

Fine, lifetime data are ok.
Problem with a value "only" each (last day of?) month will be, that all charts using this meter channel with aggregation period lower 1 month are not so good...
I understand, I don't have access to more fined grained data at the moment. 

If you see anything I have missed, please let me know.

I'll build some templates and test the logic in the demo installation, give me a day ;-) ...
 
Thank you for all you effort for building PVLng & help. I would like to make a small donation so please let me know your Paypal account.

Wow, sounds good, thank you very much! Link is now here: http://pvlng.com/Main_Page#Contribute :-)

Knut

Thanks! 

KKoPV

unread,
Sep 11, 2014, 2:16:52 AM9/11/14
to pv...@googlegroups.com
Hi,

just for clarification (based on API desc) the URLs and responses.

These exact responses are needed, because the PVLng channel, which your Inverter is based on, uses a kind of JPath to find its sub channel data.
It scans its (direct!) child channels, uses whose "channel" attribute and search the delivered JSON data for matching values.

Site Overview (page 11)

URL: /site/{siteId}/ overview

Example:
{
  "overview": {
    "lastUpdateTime": "2013-10-01 02:37:47",
    "lifeTimeData": {
      "energy": 761985.75,
      "revenue": 946.13104
    },
    "lastYearData": {
      "energy": 761985.8,
      "revenue": 0
    },
    "lastMonthData": {
      "energy": 492736.7,
      "revenue": 0
    },
    "lastDayData": {
      "energy": 0,
      "revenue": 0
    },
    "currentPower": {
      "power": 0
    }
  }
}

Channel example (will be done by template):

- Inverter
  - E-Total - Must have maintained with channel attribute: overview->lifeTimeData->energy

Inverter actual data (page 12)

URL: /equipment/{siteId} /{serialNumber}/data

To get the "last" data only, the request should send &startTime=<now minus 5 minutes>

Example:
{
  "data": {
    "count": 1,
    "telemetries": [
      {
        "date": "2014-09-01 05:01:15",
        "totalActivePower": 0,
        "dcVoltage": 12,
        "temperature": 20.2801,
        "L1Data": {
          "acCurrent": 0,
          "acVoltage": 241,
          "acFrequency": 50
        },
        "L2Data": {
          "acCurrent": 0,
          "acVoltage": 237,
          "acFrequency": 49.99
        },
        "L3Data": {
          "acCurrent": 0,
          "acVoltage": 242,
          "acFrequency": 50
        }
      }
    ]
  }
}

Optimzer data

With which API call can you fetch the optimizer data?

What is the exact JSON return?

Knut

Matjaž Lipuš

unread,
Sep 11, 2014, 3:07:34 AM9/11/14
to pv...@googlegroups.com
That is correct. 
Optimzer data

With which API call can you fetch the optimizer data?

There is no API call for optimiser data. I can download multiple CSV files, where rows are time, and columns are optimiser's data (power or panel voltage or optimiser voltage or current or energy).
What is the exact JSON return?
csv can be transformed to any format that would best fit import.

I have attached those CSV example files (numbers are not real).
NOTEs:
- energy is calculated every full hour (for previous hours)
- csv has in general rows for every minute.


Knut
energy-example.csv
current-example.csv

KKoPV

unread,
Sep 11, 2014, 10:18:40 AM9/11/14
to pv...@googlegroups.com
Hi,

With which API call can you fetch the optimizer data?

There is no API call for optimizer data. I can download multiple CSV files, where rows are time, and columns are optimizer's data (power or panel voltage or optimizer voltage or current or energy).

Ok, then we need a kind of "CSV-Importer" model like then JSON based one.
 
What is the exact JSON return?
csv can be transformed to any format that would best fit import.

Not really necessary, the model will handle this.
 
I have attached those CSV example files (numbers are not real).
NOTEs:
- energy is calculated every full hour (for previous hours)
- csv has in general rows for every minute.

What does
"P1.0.1 I (A)","P1.0.2 I (A)","P1.0.1 V (V)","P1.0.2 V (V)","OP1.0.1 V (V)","OP1.0.2 V (V)"
resp.
"P1.0.1 E (Wh)","P1.0.2 E (Wh)","P1.0.1 P (W)","P1.0.2 P (W)"

really mean? Something like P<Optimizer>.0.<String> ?

Have all of these files such "holes", it looks a bit like randomized data :-)

Here the channels and their sources as I understood so far:

 Inverter
==========


E-Total         /site/{siteId}/overview                     overview->lifeTimeData->energy
Pac             /site/{siteId}/overview                     overview->currentPower


Pdc             /equipment/{siteId}/{serialNumber}/data     data->telemetries->totalActivePower
Idc             /equipment/{siteId}/{serialNumber}/data     data->telemetries->dcVoltage


Iac1            /equipment/{siteId}/{serialNumber}/data     data->telemetries->L1Data->acCurrent
Iac2            /equipment/{siteId}/{serialNumber}/data     data->telemetries->L2Data->acCurrent
Iac3            /equipment/{siteId}/{serialNumber}/data     data->telemetries->L3Data->acCurrent


Uac1            /equipment/{siteId}/{serialNumber}/data     data->telemetries->L1Data->acVoltage
Uac2            /equipment/{siteId}/{serialNumber}/data     data->telemetries->L2Data->acVoltage
Uac3            /equipment/{siteId}/{serialNumber}/data     data->telemetries->L3Data->acVoltage


Freq1           /equipment/{siteId}/{serialNumber}/data     data->telemetries->L1Data->acFrequency
Freq2           /equipment/{siteId}/{serialNumber}/data     data->telemetries->L2Data->acFrequency
Freq3           /equipment/{siteId}/{serialNumber}/data     data->telemetries->L3Data->acFrequency


Temperature     /equipment/{siteId}/{serialNumber}/data     data->telemetries->temperature

 Powerbox (same as optimizer???)
==========

Pdc                   ID in file?
Idc                   ID in file?
Udc1 (panel)          ID in file?
Udc2 (optimizer)      ID in file?
Energy today          ID in file?

 Optimizer
===========

???



Can you please review, I would take this for the templates...

Knut

BTW, thanks for donation

Matjaž Lipuš

unread,
Sep 11, 2014, 12:24:32 PM9/11/14
to pv...@googlegroups.com


On Thursday, September 11, 2014 4:18:40 PM UTC+2, KKoPV wrote:
Hi,

With which API call can you fetch the optimizer data?

There is no API call for optimizer data. I can download multiple CSV files, where rows are time, and columns are optimizer's data (power or panel voltage or optimizer voltage or current or energy).

Ok, then we need a kind of "CSV-Importer" model like then JSON based one.
yes, but keep in mind this csv is accessible only from monitoring portal. (so poking with session cookies is necessary).
 
What is the exact JSON return?
csv can be transformed to any format that would best fit import.

Not really necessary, the model will handle this.
ok. I was thinking of building csv -> csv tranformed -> call PVLngPUTBatch.
But having model similar to SMA would be great.
 
I have attached those CSV example files (numbers are not real).
NOTEs:
- energy is calculated every full hour (for previous hours)
- csv has in general rows for every minute.

What does
"P1.0.1 I (A)","P1.0.2 I (A)","P1.0.1 V (V)","P1.0.2 V (V)","OP1.0.1 V (V)","OP1.0.2 V (V)"
resp.
"P1.0.1 E (Wh)","P1.0.2 E (Wh)","P1.0.1 P (W)","P1.0.2 P (W)"

really mean? Something like P<Optimizer>.0.<String> ?
If I decoded correctly:
P=panel 
OP=optimiser

(O?)P<Inverter>.<String>.<Optimizer> (I|V|U) (unit)



Have all of these files such "holes", it looks a bit like randomized data :-)
in any row there is at least one non-empty cell. it can be many.

Here the channels and their sources as I understood so far:

 Inverter
==========


E-Total         /site/{siteId}/overview.json                     overview->lifeTimeData->energy
Pac             /site/{siteId}/overview.json                     overview->currentPower->power


Pdc             /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->totalActivePower
Udc             /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->dcVoltage


Iac1            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L1Data->acCurrent
Iac2            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L2Data->acCurrent
Iac3            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L3Data->acCurrent


Uac1            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L1Data->acVoltage
Uac2            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L2Data->acVoltage
Uac3            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L3Data->acVoltage


Freq1           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L1Data->acFrequency
Freq2           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L2Data->acFrequency
Freq3           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->L3Data->acFrequency


Temperature     /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[x]->temperature

 Optimizer
==========

Pdc                   P1.0.1 P (W)
Idc                   P1.0.1 I (A)
Udc1 (panel)          P1.0.1 V (V)
Udc2 (optimizer)      OP1.0.1 V (V)
Energy last hour      P1.0.1 E (Wh)

I have updated structure. current power is an object. telemetries is an array of data/time points.
I have sent you an email (to pv at kk de) with full responses.


So, I receive:

from grid operator (via email - so it could be automated) readings for two meters:
1. meter - total produced energy by plant (kWh)  (two readings, VT, MT - tariffs)
2. meter - total energy sent to electrical grid (kWh)  (same)

from electricity company (PX3 scheme)
3. meter - total energy received from electrical grid (kWH) (VT, MT tariffs) (mostly at night)

so basically this are 6 values per month.


Can you please review, I would take this for the templates...

Knut

BTW, thanks for donation

Thank you! 

KKoPV

unread,
Sep 12, 2014, 3:50:05 AM9/12/14
to pv...@googlegroups.com
Hi,

 Inverter
==========

E-Total               /site/{siteId}/overview.json                     overview->lifeTimeData->energy
Pac                   /site/{siteId}/overview.json                     overview->currentPower->power

Pdc                   /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->totalActivePower
Udc                   /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->dcVoltage

Iac1                  /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L1Data->acCurrent
Iac2                  /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L2Data->acCurrent
Iac3                  /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L3Data->acCurrent

Uac1                  /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L1Data->acVoltage
Uac2                  /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L2Data->acVoltage
Uac3                  /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L3Data->acVoltage

Freq1                 /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L1Data->acFrequency
Freq2                 /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L2Data->acFrequency
Freq3                 /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L3Data->acFrequency

Temperature           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->temperature

 Optimizer
===========


Pdc                   P1.0.1 P (W)
Idc                   P1.0.1 I (A)
Udc1 (panel)          P1.0.1 V (V)
Udc2 (optimizer)      OP1.0.1 V (V)
Energy last hour      P1.0.1 E (Wh)
"P1.0.1 I (A)","P1.0.2 I (A)","P1.0.1 V (V)","P1.0.2 V (V)","OP1.0.1 V (V)","OP1.0.2 V (V)"
resp.
"P1.0.1 E (Wh)","P1.0.2 E (Wh)","P1.0.1 P (W)","P1.0.2 P (W)"

P  = panel
OP = optimizer

(O?)P<Inverter>.<String>.<Optimizer> (I|V|U) (unit)
I have updated structure. current power is an object. telemetries is an array of data/time points.

Ok, then the JSON model can't handle this so far, even if it could handle arrays, it don't know where the time stamp should come from for each array...
I'll extend the JSON model to handle this.

Definition for such a (sub) channel then must be:


  data->telemetries[]->totalActivePower||data->telemetries[]->date


Part before double pipe is "location" of measuring value in JSON structure, the other part for the location of timestamp to be used.
(Without timestamp, the server timestamp would be used for all arrays!)

Nevertheless, for later live loading I suggest a cron job each 5 min and an API request with startTime=<now - 5min>
This should respond the "latest one" metrics only (data->count = 1)...

For the Optimizers we have to decide
  1. Split the CSV before upload into value and its timestamp
  2. Upload the whole CSV
Disadvantage of 2. is the definition, how the model should find the data, when it should be universal as possible.

Have the CSV all the same structure with header line?!

I have sent you an email (to pv at kk de) with full responses.

Got it, will test with that.
 

So, I receive:

from grid operator (via email - so it could be automated) readings for two meters:
1. meter - total produced energy by plant (kWh)  (two readings, VT, MT - tariffs)
2. meter - total energy sent to electrical grid (kWh)  (same)

from electricity company (PX3 scheme)
3. meter - total energy received from electrical grid (kWH) (VT, MT tariffs) (mostly at night)

so basically this are 6 values per month.

Ok, this could be in simplest case sources for batch update.

Next I'll make an extra installation which we can use for test.

Knut

Matjaž Lipuš

unread,
Sep 12, 2014, 8:43:44 AM9/12/14
to pv...@googlegroups.com
I was thinking that JSON model should accept "list" property (in this example: data->telemetries), which is expected to be array. and then all sub-channels should be relative from list. (example: totalActivePower, L1Data->acCurrent). Similar for time. sub channel would have value of "date". but if date is outside of list it should be possible to absolutely addressed (from root) (for example: "->data->date").
So maybe it's less to type for channel definition, but it's more complicated to understand.

Nevertheless, for later live loading I suggest a cron job each 5 min and an API request with startTime=<now - 5min>
This should respond the "latest one" metrics only (data->count = 1)...
yeah. but if a request fails I should handle this.
Does PVLng detect duplicates (same time, value in one channel)?

For the Optimizers we have to decide
  1. Split the CSV before upload into value and its timestamp
That is simpler approach and it will work pretty well.
  1. Upload the whole CSV
Disadvantage of 2. is the definition, how the model should find the data, when it should be universal as possible.
Here, before importing, a mapper from column to channel would do the trick (simple php array in config?)
But in one CSV I can have 150 columns with just 10 rows. 

Have the CSV all the same structure with header line?!
yes, first row cells are always (O?)P<Inverter>.<String>.<Optimizer> (I|V|U) (unit)
For example
A1 is always Time
B1 is P1.0.1 E (Wh)
A2 is date time (local timezone - CEST): 06/30/2014 17:01
B2 is value: 15.4124
...

I have sent you an email (to pv at kk de) with full responses.

Got it, will test with that.
 

So, I receive:

from grid operator (via email - so it could be automated) readings for two meters:
1. meter - total produced energy by plant (kWh)  (two readings, VT, MT - tariffs)
2. meter - total energy sent to electrical grid (kWh)  (same)

from electricity company (PX3 scheme)
3. meter - total energy received from electrical grid (kWH) (VT, MT tariffs) (mostly at night)

so basically this are 6 values per month.

Ok, this could be in simplest case sources for batch update.

Next I'll make an extra installation which we can use for test.

Knut

Thanks! 

KKoPV

unread,
Sep 12, 2014, 4:16:18 PM9/12/14
to
Hi,

Definition for such a (sub) channel then must be:


  data->telemetries[]->totalActivePower||data->telemetries[]->date


Part before double pipe is "location" of measuring value in JSON structure, the other part for the location of timestamp to be used.
(Without timestamp, the server timestamp would be used for all arrays!)
 
I was thinking that JSON model should accept "list" property (in this example: data->telemetries), which is expected to be array. and then all sub-channels should be relative from list. (example: totalActivePower, L1Data->acCurrent). Similar for time. sub channel would have value of "date". but if date is outside of list it should be possible to absolutely addressed (from root) (for example: "->data->date").

Sounds more difficult...
 
So maybe it's less to type for channel definition, but it's more complicated to understand.

That's the point: You have to type it only once (or never when you use a template :-)

I would prefer the more "general" logic, this makes integration of new sources easier.
 
Nevertheless, for later live loading I suggest a cron job each 5 min and an API request with startTime=<now - 5min>
This should respond the "latest one" metrics only (data->count = 1)...
yeah. but if a request fails I should handle this.
Does PVLng detect duplicates (same time, value in one channel)?

Yes, by default new reading must have timestamp  min. 5 seconds after last reeading. (Entry in database: pvlng_config -> DoubleRead)
 
For the Optimizers we have to decide
  1. Split the CSV before upload into value and its timestamp
That is simpler approach and it will work pretty well.
  1. Upload the whole CSV
Disadvantage of 2. is the definition, how the model should find the data, when it should be universal as possible.
Here, before importing, a mapper from column to channel would do the trick (simple php array in config?)
But in one CSV I can have 150 columns with just 10 rows. 

Have the CSV all the same structure with header line?!
yes, first row cells are always (O?)P<Inverter>.<String>.<Optimizer> (I|V|U) (unit)
For example
A1 is always Time
B1 is P1.0.1 E (Wh)
A2 is date time (local timezone - CEST): 06/30/2014 17:01
B2 is value: 15.4124
...

 Ok, I think the handling of such a "CSV with header line" is much easier in PHP.

But if you define a number of optimizers (with sub channels) and a file contains data for more than 1 optimizer, you must push the file to all optimizers and they will extract the data for their sub channels...

Logic is, that a specialized grouping channel only scans its direct sub channels, not deeper. (Reason is, that this group can also have grouping channels as sub channels...)

Knut

Matjaž Lipuš

unread,
Sep 13, 2014, 5:54:03 AM9/13/14
to pv...@googlegroups.com


On Friday, September 12, 2014 10:16:18 PM UTC+2, KKoPV wrote:
Hi,

Definition for such a (sub) channel then must be:


  data->telemetries[]->totalActivePower||data->telemetries[]->date


Part before double pipe is "location" of measuring value in JSON structure, the other part for the location of timestamp to be used.
(Without timestamp, the server timestamp would be used for all arrays!)
 
I was thinking that JSON model should accept "list" property (in this example: data->telemetries), which is expected to be array. and then all sub-channels should be relative from list. (example: totalActivePower, L1Data->acCurrent). Similar for time. sub channel would have value of "date". but if date is outside of list it should be possible to absolutely addressed (from root) (for example: "->data->date").

Sounds more difficult...
 
So maybe it's less to type for channel definition, but it's more complicated to understand.
That's the point: You have to type it only once (or never when you use a template :-)

I would prefer the more "general" logic, this makes integration of new sources easier.
I agree. 
 
Nevertheless, for later live loading I suggest a cron job each 5 min and an API request with startTime=<now - 5min>
This should respond the "latest one" metrics only (data->count = 1)...
yeah. but if a request fails I should handle this.
Does PVLng detect duplicates (same time, value in one channel)?

Yes, by default new reading must have timestamp  min. 5 seconds after last reeading. (Entry in database: pvlng_config -> DoubleRead)
 
For the Optimizers we have to decide
  1. Split the CSV before upload into value and its timestamp
That is simpler approach and it will work pretty well.
  1. Upload the whole CSV
Disadvantage of 2. is the definition, how the model should find the data, when it should be universal as possible.
Here, before importing, a mapper from column to channel would do the trick (simple php array in config?)
But in one CSV I can have 150 columns with just 10 rows. 

Have the CSV all the same structure with header line?!
yes, first row cells are always (O?)P<Inverter>.<String>.<Optimizer> (I|V|U) (unit)
For example
A1 is always Time
B1 is P1.0.1 E (Wh)
A2 is date time (local timezone - CEST): 06/30/2014 17:01
B2 is value: 15.4124
...
 Ok, I think the handling of such a "CSV with header line" is much easier in PHP.

But if you define a number of optimizers (with sub channels) and a file contains data for more than 1 optimizer, you must push the file to all optimizers and they will extract the data for their sub channels...

ok, no problem. I can split csvs to contain only 1 optimiser each. 

KKoPV

unread,
Sep 13, 2014, 7:48:42 AM9/13/14
to pv...@googlegroups.com
Hi,

Definition for such a (sub) channel then must be:


  data->telemetries[]->totalActivePower||data->telemetries[]->date


Part before double pipe is "location" of measuring value in JSON structure, the other part for the location of timestamp to be used.
(Without timestamp, the server timestamp would be used for all arrays!)

 
After some testing to find the correct position I came to the result, that it is not that easy as I wish...

I think, I'll make a specialized JSON based model, which handles Solar Edge responses.
The same I still have for the SMA Webbox, there it was the same problem.

So for "site" call the channel will have as address the path below "overview" and for the "equipment" calls the path below "metrics".

 Inverter
==========

Frontend name  API call                                         JPath in response                          Channel attribute

E-Total        /site/{siteId}/overview.json                     overview->lifeTimeData->energy             lifeTimeData->energy
Pac            /site/{siteId}/overview.json                     overview->currentPower->power              currentPower->power

As timestamp for the next channels will be used: data->telemetries[]->date

Pdc            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->totalActivePower      totalActivePower
Udc            /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->dcVoltage             dcVoltage

Iac1           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L1Data->acCurrent     L1Data->acCurrent
Iac2           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L2Data->acCurrent     L2Data->acCurrent
Iac3           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L3Data->acCurrent     L3Data->acCurrent

Uac1           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L1Data->acVoltage     L1Data->acVoltage
Uac2           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L2Data->acVoltage     L2Data->acVoltage
Uac3           /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L3Data->acVoltage     L3Data->acVoltage

Freq1          /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L1Data->acFrequency   L1Data->acFrequency
Freq2          /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L2Data->acFrequency   L2Data->acFrequency
Freq3          /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->L3Data->acFrequency   L3Data->acFrequency

Temperature    /equipment/{siteId}/{serialNumber}/data.json     data->telemetries[]->temperature           temperature

 Optimizer
===========


Pdc                   P1.0.1 P (W)
Idc                   P1.0.1 I (A)
Udc1 (panel)          P1.0.1 V (V)
Udc2 (optimizer)      OP1.0.1 V (V)
Energy last hour      P1.0.1 E (Wh)
 
I can split csvs to contain only 1 optimiser each.

This is then a must!

Knut
Reply all
Reply to author
Forward
0 new messages