ESTCube requirements and SpaceSystemModel

[Lauri Kimmel]

There are some requirements from ESTCube mission for SpaceSystemModel:

1. Dynamic limits – something operator can change from UI. Eg. parameter X has limit Y in the model and it has to be changed to the Z later during the mission. How will Hummingbird handle this case? Editing space system model definition, building new model and publishing it looks like current idea. It's not very user friendly and operator has to have some "programming" skills to do it.

2. Time to live for parameter value. Timestamp which holds creation/receiving time can be added to Parameter interface and components can validate this against some predefined value. More generic way would be to add TTL limit into the model.

3. Hard and soft limits – I suppose it's possible in current SpaceSystemModel. 

4. Codecs for string based input – incoming data in XML, JSON, RSS, etc format. This is case for weather data which is important for ground station operations. Are Hummingbird and SpaceSystemModel suitable for this use case at the moment?

5. Are there any "special" responses planned for TC in current design? For example some kind of simple ACK response or similar. 

Please let me know what you think.

[Mark Doyle]

1) Application level. Gert has something for this I think. Essentially it becomes part of an Enterprise route We had a tiny test we knocked up on the road trip to the Rocket launch for calibration. It was routed like this:

TM--->frame/packet decoding--->payload codec--->calibration app--->GUI

2) Timing does need work. Johannes added some support a few months back. I think it's populated when the data arrives at hbird. Generation time is definitely required although I assumed that would come down as part of either the frame/packet header (if the protocol supports it) or the packet payload.

3) Sounds like application level. Transport should be as ignorant as possible to other concerns.

4) Nothing special has been discussed. Would it not just be a case of picking up a specific piece of telemetry, i.e. the response? If we are talking about G/S reception we could use camel to create a request response route. That's just an EIP though, i.e. not a camel configuration though and only the commanding app would need to support dealing with a response...which in this case it would since it drove the requirement!

[Mark Doyle]

Apparently gmail has a keyboard shortcut for send which I mashed by accident. the last sentence should read:

That's just an EIP though, i.e. not a concern for the hbird components. For example, we'd just use camel to build the route and only the commanding app would need to support dealing with a response...which in this case it would since it drove the requirement! 

[Gert Villemos]

Answers to a select set of the questions (the ones that is related to the business tier);

In general I think way to much focus is put on the SSM. The space system model (SSM) is used for encoding / decoding in the transport tier and when available configuration of the business tier. The bussines tier and at least some of the displays does not demand you to have a SSM. And if you want you can create a transport tier without a SSM. It is just a standard way to define the space system, intended to be a help. If it doesnt help, dont use it.

1. The limits can be changed at anytime simply by injecting a parameter with a name such as '[limitName]_VALUE'. This will change the limit to the new value. Limits can be enabled/disabled by injecting a boolean parameter with the name '[limitName]_SWITCH'. You can of course change the name of the parameters used to change the limits.

In general anything can be changed at anytime. You can add new commands, new arguments to commands, new parameters, new limits, new tasks, a new state of a command, change existing ones, etc, without having to change the configuration or stop the system.

The transport tier is special as it needs to know how to encode/decode things, 

3. You can define any number of limits for each parameter. Whether you intepretate some or all as soft or hard is a question of how you setup the system. You can for example have a parameter 'PARA1' that has the soft limit 'PARA1<1' and the hard limit 'PARA1<0' and a ultra hard limit 'PARA1<-1'. The violation of each limit will be marked in the system by a state transition; a state parameter is issued. The state transition can trigger what ever you want; warnings, emails, SMS, ...

5. No, there are no 'predefined' command states. You define the states you want. The command state handling works like follows;

a. Command is created and injected.

b. A validation of prerelease conditions is done. This is the validation that certain values are within their limits, preconditions for the command to be released. If all conditions parses, then a state parameter '[command]_PV' is set to true. A state parameter is a boolean parameter that has an attribute 'isStateOf' set to the name of the command.

c. Command is released. Which sort of validation states you do depends on your processing. You can have a 'encoding validation', a 'station reception validation', a 'radiation validation', a 'onboard schedule validation', a 'execution start validation', 'execution ack validation'. You may have some for some commands and not for others. Each validation state is a state parameter with the 'isStateOf' pointing to the command.

The state of the command at any point in time is the sum of all state parameters applicable to the command. The specific states are not fixed. The number is not fixed.

As an example; If you want to manually mark a command as failed, because you as a human know it failed even if the system say its fine, then you create a state parameter 'IKnowTheCommandFailed=false; isStateOf=[command]' and inject it in the system. The command is now failed.

Cheers,

Gert.

[Lauri Kimmel]

At the moment there are no business components available in github and it is bit too hard to get bigger picture how to setup hbird based system.

Let's take simple real life use case from ESTCube project – operator has to have overview of the weather conditions to operate ground station safely. Strong wind and extreme temperatures can cause damages to radio antenna.

We have several sources available for up to date weather data. All of them are providing info over HTTP in some sort of format XML, JSON etc. Logical step would be to add WeatherStation component to the hbird based system. This component polls all the available sources in predefined interval, parses HTTP responses and sends data as Parameters to the system. All the sources have their own format for data. Eg. temperature can be presented as "Temperature", "temperature", "temp" etc. There is no point to forward N+1 different Parameters holding temperature to the system. Instead WeatherStation will use single unified Parameter for temperature – let's name it "WeatherStation.tm.Temperature" for example. Each weather ParameterGroup will have extra Parameter "WeatherStation.tm.Source" to identify the actual source of data.

Now it gets unclear to me. How does WeatherStation know the exact names of needed Parameters ("WeatherStation.tm.Temperature" and "WeatherStation.tm.Source" in this case)? They can be defined in SSM but WeatherStation has no idea how to pick the correct ones. So there has to be some kind of constants available in sense of Java source code. Most likely constants have to be available to other components also. For example in the UI there maybe need for dedicated view for temperature etc. Now there is problem – SSM and constants have to be synchronized. Otherwise there will be all sorts of troubles in case only one of them is modified and other is unchanged.

How will LimitChecker function? As I understand there will be Parameters, Limits and some kind of OutOfLimit events/objects. When new Parameter with value is received from the system LimitChecker will validate it against all related Limits and fire OutOfLimit event for each violation. First question in my mind is – where are the limits stored? LimitCheker can go offline in any moment and come back later on. So the limits have to be stored persistently somewhere. Second question is how does LimitChecker know which Limit is related to which Parameter? First solution what comes in to my mind is to include Parameter name in the Limit. At the moment there are no Limit and OutOfLimit objects in code base.

If we are talking about triggering events based on limit checking there comes another set of domain object into my mind – Rules and Actions. They also need to be persistently stored some place. Do you have any thoughts about this?

As I see hbird based system needs some (pre)defined domain specific model. Is it SSM or something else I don't know yet. Updates in runtime are good things to have. In the other hand it's not good to have hardcoded magic value in several places in the system like mentioned "WeatherStation.tm.Temperature" in WeatherStation and UI component.

Whatever is the solution for the Model, it has to be made available in the system. There is component called space-system-publisher. Current implementation is simple Java class. What is the future plan for it – standalone component in the system working over JMS/etc or something else?

Additionally few thought about Parameters. There is lot of static information in Parameter – name, qualified name, long and short description.  Only thing which is changing is value and it's changing in time. Maybe it makes sense to have two classes something like this

class ParameterDefinition<T> {

    String name;

    String qualifiedName;

    String shortDescription;

    String longDescription;

    T defaultValue;

}

class ParameterValueInTime<T> {

    ParameterDefinition<T> definition;

    T value;

    Long timestamp;

}

What do you think?

[Gert Villemos]

Hi Lauri,

Excellent comments. See general answers below as well as specific comments embedded in your email.

First of all; have you taken a look at the new website at www.hbird.de (will be moved to .org as soon as Johanness get around to update the name server)? It contains a much more general information than the old site. Still far from perfect, but closer.

I have completed the migration of the business tier modules and will commit them tonight. Systematic testing is still missing, but I think its more important to give you the code. I am adding tests as well.

Now, more comments embedded below... I will turn your questions and my answers into a 'Guide' on the website for how to do this as I think its an excellent example.

Let's take simple real life use case from ESTCube project – operator has to have overview of the weather conditions to operate ground station safely. Strong wind and extreme temperatures can cause damages to radio antenna.

We have several sources available for up to date weather data. All of them are providing info over HTTP in some sort of format XML, JSON etc. Logical step would be to add WeatherStation component to the hbird based system. This component polls all the available sources in predefined interval, parses HTTP responses and sends data as Parameters to the system. All the sources have their own format for data. Eg. temperature can be presented as "Temperature", "temperature", "temp" etc. There is no point to forward N+1 different Parameters holding temperature to the system. Instead WeatherStation will use single unified Parameter for temperature – let's name it "WeatherStation.tm.Temperature" for example. Each weather ParameterGroup will have extra Parameter "WeatherStation.tm.Source" to identify the actual source of data.

Now it gets unclear to me. How does WeatherStation know the exact names of needed Parameters ("WeatherStation.tm.Temperature" and "WeatherStation.tm.Source" in this case)? They can be defined in SSM but WeatherStation has no idea how to pick the correct ones. So there has to be some kind of constants available in sense of Java source code. Most likely constants have to be available to other components also. For example in the UI there maybe need for dedicated view for temperature etc. Now there is problem – SSM and constants have to be synchronized. Otherwise there will be all sorts of troubles in case only one of them is modified and other is unchanged.

This illustrates nicely that you need a 'protocol tier' that is not just XTCE based. 

1. Create a HTTP Camel 'Consumer Endpoint' (use for example the http://www.villemos.com/?q=ispace/component/httpcrawler Camel component, ahem!). This is logically part of the transport tier (you are interfacing and talking with some external, propiatary protocol). But there is NO reason to use XTCE or similar. 

2. The endpoint 'consumes' data from the weather station. It extract the data and inject it as parameters into the system as a 'Parameter'.

Its important that you notice that the Parameter in the exchange tier and business tier is NOT the same as the one in the protocol tier. When I say Parameter I mean the type that has as base type the type 'Named'. We are working on integration...

3. The business tier will accept and route ANY Parameter you create. There is no need for predefined 'types' in static schemas. So you create a parameter as shown below;

'Parameter temperature = new Parameter("weatherStation", "WeatherStation.tm.Temperature", "The temprature at the weather station", 18, "degree")' 

Notice the first parameter 'weatherStation'. It identifies the SOURCE of the parameter.  Generation timestamp is added automatically and set to NOW (... but you can change it if you need).

You inject the paramter by doing something like (this code assumes you did the smart thing and created a proper Camel Endpoint for the extraction)

'Exchange exchange = new DefaultExchange(context);

exchange.getIn().setBody(WeatherStation.tm.Temperature);

producer.send("activemq:topic:Parameters", exchange);'

Result of these steps is that you create and inject a new Parameter into the system. The parameter will be routed and can be displayed (it is self contained, i.e. holds all data needed to display it). It can be stored.

But there is no processing defined for it yet, thats the next step...

 

 

How will LimitChecker function? As I understand there will be Parameters, Limits and some kind of OutOfLimit events/objects. When new Parameter with value is received from the system LimitChecker will validate it against all related Limits and fire OutOfLimit event for each violation. First question in my mind is – where are the limits stored? LimitCheker can go offline in any moment and come back later on. So the limits have to be stored persistently somewhere. Second question is how does LimitChecker know which Limit is related to which Parameter? First solution what comes in to my mind is to include Parameter name in the Limit. At the moment there are no Limit and OutOfLimit objects in code base.

4. Create a limit and send it to the system.

'Limit limit = new Limit()'

[Gert Villemos]

... I send this too early. Embedded comments were NOT completed. 

I will send a new version tonight.

Cheers,

Gert.

[Johannes Klug]

Hi Lauri,

I'll try to respond point-by-point, please find my answers below.

Am 2012-04-02 16:31, schrieb Lauri Kimmel:
> At the moment there are no business components available in github
> and
> it is bit too hard to get bigger picture how to setup hbird based
> system.

I think the best way would be for Gert to publish the application-layer
parts he's written under his GitHub account, maybe under a name like
"hbird-application", or something like it. Users like ESTcube could then
pull in dependencies from the application layer by pointing to the
artifacts in there.

I think the answer to this one is that the parameter names should be
set in configuration. Once the parameter name is set, it needs to be
configured in the applications that need to use it. Compare this to the
MMI with the world wind globe: It gets all kinds of parameters, but it
needs to know which ones carry latitude, longitude, and altitude. There
is no way around setting it. If you change the parameter name somewhere,
or make a typo, your application can't do anything about it. As you
said, it can't guess which parameters it needs.

>
> How will LimitChecker function? As I understand there will be
> Parameters, Limits and some kind of OutOfLimit events/objects. When
> new Parameter with value is received from the system LimitChecker
> will
> validate it against all related Limits and fire OutOfLimit event for
> each violation. First question in my mind is – where are the limits
> stored? LimitCheker can go offline in any moment and come back later
> on. So the limits have to be stored persistently somewhere. Second
> question is how does LimitChecker know which Limit is related to
> which
> Parameter? First solution what comes in to my mind is to include
> Parameter name in the Limit. At the moment there are no Limit and
> OutOfLimit objects in code base.
>
> If we are talking about triggering events based on limit checking
> there comes another set of domain object into my mind – Rules and
> Actions. They also need to be persistently stored some place. Do you
> have any thoughts about this?

Well, my thoughts about a limit checker are this:
Make a data structure that has a parameter name, upper limit, and lower
limit. This will be fairly compact, and will easily fit into memory even
for thousands of parameters. Also, such a structure is trivial to
persist to disk or update on-line in case it's changed - just overwrite
the parameter's entry if the limits shall change. Such an application
should only take an afternoon to write, but we haven't got one yet. I
remember Gert laughed at this design (because it's "monolithic") when I
presented it a year ago or so, but I still believe it's viable and easy
to implement.

> As I see hbird based system needs some (pre)defined domain specific
> model. Is it SSM or something else I don't know yet. Updates in
> runtime are good things to have. In the other hand it's not good to
> have hardcoded magic value in several places in the system like
> mentioned "WeatherStation.tm.Temperature" in WeatherStation and UI
> component.

Let's assume we had such a model - you'd still have to tell your
parameter injector, limit checker, displays and so forth exactly which
parameter in that model they should deal with. At some point you'll have
to "get real" and configure your system. As long as that happens in
config files, I'm happy with it. Magic values in the code of course are
a different matter and should not exist.

> Whatever is the solution for the Model, it has to be made available
> in
> the system. There is component called space-system-publisher. Current
> implementation is simple Java class. What is the future plan for it
> – standalone component in the system working over JMS/etc or
> something else?

It is just a simple Java class, that's true, but with Camel you can
wire its output to, for example, ActiveMQ, and have all your
applications listen to the publisher for updates. So in that sense, it
is already a standalone-application, just that most of the code is part
of Camel.

> Additionally few thought about Parameters. There is lot of static
> information in Parameter – name, qualified name, long and short
> description. Only thing which is changing is value and it's changing
> in time. Maybe it makes sense to have two classes something like this
>
> class ParameterDefinition<T> {
> String name;
> String qualifiedName;
> String shortDescription;
> String longDescription;
> T defaultValue;
> }
>
> class ParameterValueInTime<T> {
> ParameterDefinition<T> definition;
> T value;
> Long timestamp;
> }
>
> What do you think?

The idea here was that all properties should be contained within a
single parameter sample, so that you don't need to look up a parameter's
name when you receive one of its samples - the name and all metadata
comes with the parameter sample. This is due to experience with a
mission control system that requires lots of lookups like that. The
other reason is that, again, you could change parameters' metadata and
keep the system running, requiring zero downstream applications to
restart. This is a design decision made to make it possible to use
Hummingbird in an EGSE context, where definitions might change rapidly.
Looking back at your parameter archive, you'd see the evolution of
metadata, as it's all preserved per sample. Of course this is a massive
overhead, but we decided to go that route in the early days of
Hummingbird.
I agree that your proposed approach is more normalised and more
efficient in terms of data transmission and storage, but it is less
flexible and doesn't allow modifications like described above. Of course
we haven't ever tested this under high load, so we have no idea how much
of a price we pay for this flexibility. It might well turn out that
lookups aren't as bad after all, and we shouldn't transmit the same
descriptions over and over, just because someone might want to look at
an earlier version of the "shortDescription". I'm open for debate, but
we should do some benchmarking first. These decisions were made by gut
feeling, and are not backed by hard facts.

Cheers,
Johannes

[Gert Villemos]

Hi Johanness and Lauri,

My answers below. Johanness answers are not completly correct (sorry to be so blunt Johanness...). 

Point is that the business tier is completely and utterly dynamic. Stop thinking static schemas and predefined-what-ever. Doesnt mean you dont need to define meta-data, but you dont put it into DBs / static schemas that you distribute and fight to keep consistent.

Answers below. These also override the previous email I send that was only 1/2 finished.

Cheers,

Gert.

At the moment there are no business components available in github and
it is bit too hard to get bigger picture how to setup hbird based
system.

I think the best way would be for Gert to publish the application-layer parts he's written under his GitHub account, maybe under a name like "hbird-application", or something like it. Users like ESTcube could then pull in dependencies from the application layer by pointing to the artifacts in there.

Ok, I will created a new repository. I think we should merge and not split further, but perhaps this is the first step towards merging...

Yes, and no.

The names are conventions that you are responsible for maintaining in a consistent state. The problem of consistency will not change if you dont use the name; if you use a DB ID, what happens if you change the ID one place?... etc, etc. So a parameter name is as good an identifier as anything else.

Most of the business tier ccept and process any parameter injected. This includes; distribution, storage, and display. There is no need to hardcode the parameter or define it statically. 

You might create specific chains of dependencies where you will need to process specific parameters. For example the command 'lock states' are specific parameters whose value the command releaser checks. But in general static schemas are not needed. This is configuration that you create (more later).

A display should be flexible enough to display what is send. The Parameter type, as used in the Exchange and Business Tier, is generic enough. It holds, besides name and value, unit and description as well as the name of the source. Thats all that most displays needs to display the parameters. 

For your example (I will create a Guide on the website to explain exactly this example in detail.);

1. Create a HTTP capabel Camel Endpoint; a consumer endpoint. The endpoint access the wheather station and reads the Parameter(s). This component is part of your Transport Tier! It is a protocol layer integration mechanism between a Hummingbird system and 'something'. And it has no SSM and no XTCE. Because XTCE and SSM solves specific problems when talking with certain classes of spacecrafts.

2. The consumer endpoint should create Parameters in the following way;

/** Constructor; Parameter(String issuedBy, String name, String description, Object value, String unit)*/

Parameter temperature = new Parameter("WheatherStation", "WeatherStation.tm.Temperature", "The outside temperature", currentTemp, "celcius");

Notice that the first parameter is the source. There is no need to inject that as a separate parameter.

3. The consumer endpoint should then inject the parameter into the system. There is many different ways of doing that.

 

How will LimitChecker function? As I understand there will be
Parameters, Limits and some kind of OutOfLimit events/objects. When
new Parameter with value is received from the system LimitChecker will
validate it against all related Limits and fire OutOfLimit event for
each violation. First question in my mind is – where are the limits
stored? LimitCheker can go offline in any moment and come back later
on. So the limits have to be stored persistently somewhere. Second
question is how does LimitChecker know which Limit is related to which
Parameter? First solution what comes in to my mind is to include
Parameter name in the Limit. At the moment there are no Limit and
OutOfLimit objects in code base.

If we are talking about triggering events based on limit checking
there comes another set of domain object into my mind – Rules and
Actions. They also need to be persistently stored some place. Do you
have any thoughts about this?

Well, my thoughts about a limit checker are this:
Make a data structure that has a parameter name, upper limit, and lower limit. This will be fairly compact, and will easily fit into memory even for thousands of parameters. Also, such a structure is trivial to persist to disk or update on-line in case it's changed - just overwrite the parameter's entry if the limits shall change. Such an application should only take an afternoon to write, but we haven't got one yet. I remember Gert laughed at this design (because it's "monolithic") when I presented it a year ago or so, but I still believe it's viable and easy to implement.

No Johanness, your memory is wrong. Your design was a centralized limit checker that did all limit checking in a 'stov-pipe' fashion based on a static schema.

The design you describe above is exactly what the limit check implementation is and does. It already exist and has existed for almost the start of the system (it was one of the first class to be written). It is what I have presented numorous times. The relevant classes are named 'LowerLimit' and 'UpperLimit' in the business tier.

For the example at hand, do the following;

4. The Consumer Endpoint at startup injects a limit 'metadata' request. This is done as shown below.

ConfigurationRequest = new LimitComponentRequest("WeatherStation.tm.Temperature", " WeatherStation.tm.Temperature.limit", "The limit of the temperature", "below 10");

5. This request is injected into the system. The configurator component, somewhere in the system, creates the necesarry limit classs and all routes needed to 

a. Change the limit.

b. Enable/disable the limit.

c. Check the limit.

  

As I see hbird based system needs some (pre)defined domain specific
model. Is it SSM or something else I don't know yet. Updates in
runtime are good things to have. In the other hand it's not good to
have hardcoded magic value in several places in the system like
mentioned "WeatherStation.tm.Temperature" in WeatherStation and UI
component.

Let's assume we had such a model - you'd still have to tell your parameter injector, limit checker, displays and so forth exactly which parameter in that model they should deal with. At some point you'll have to "get real" and configure your system. As long as that happens in config files, I'm happy with it. Magic values in the code of course are a different matter and should not exist.

Consider your assembly data your schema. Why do you want another DB / format? Consistency is always an issue, but it will be the same whether you put the values in your assembly or in your DB. Somewhere ends must meet. And if you configure that wrong your in trouble. No matter how you represent the data. 

The main difference is that you will have to maintain two sets of data; your assembly and your DB.

 

Whatever is the solution for the Model, it has to be made available in
the system. There is component called space-system-publisher. Current
implementation is simple Java class. What is the future plan for it
– standalone component in the system working over JMS/etc or
something else?

It is just a simple Java class, that's true, but with Camel you can wire its output to, for example, ActiveMQ, and have all your applications listen to the publisher for updates. So in that sense, it is already a standalone-application, just that most of the code is part of Camel.

The same functionality is in the business tier offered by the 'ObjectPublisher' in the 'metadata publisher' module.

 

Additionally few thought about Parameters. There is lot of static
information in Parameter – name, qualified name, long and short
description. Only thing which is changing is value and it's changing
in time. Maybe it makes sense to have two classes something like this

class ParameterDefinition<T> {
 String name;
 String qualifiedName;
 String shortDescription;
 String longDescription;
 T defaultValue;
}

class ParameterValueInTime<T> {
 ParameterDefinition<T> definition;
 T value;
 Long timestamp;
}

What do you think?

The idea here was that all properties should be contained within a single parameter sample, so that you don't need to look up a parameter's name when you receive one of its samples - the name and all metadata comes with the parameter sample. This is due to experience with a mission control system that requires lots of lookups like that. The other reason is that, again, you could change parameters' metadata and keep the system running, requiring zero downstream applications to restart. This is a design decision made to make it possible to use Hummingbird in an EGSE context, where definitions might change rapidly. Looking back at your parameter archive, you'd see the evolution of metadata, as it's all preserved per sample. Of course this is a massive overhead, but we decided to go that route in the early days of Hummingbird.
I agree that your proposed approach is more normalised and more efficient in terms of data transmission and storage, but it is less flexible and doesn't allow modifications like described above. Of course we haven't ever tested this under high load, so we have no idea how much of a price we pay for this flexibility. It might well turn out that lookups aren't as bad after all, and we shouldn't transmit the same descriptions over and over, just because someone might want to look at an earlier version of the "shortDescription". I'm open for debate, but we should do some benchmarking first. These decisions were made by gut feeling, and are not backed by hard facts.

I completly agree with Johanness. 

Add to waht he says the problem of keeping different 'versions' of a parameter (I change the description at some point). 

You think normalized. We de-normalize on purpose. To our minds it looks like overhead, but in reality its a massive simplification throughout the system.

And if you want to 'optimize' storage, then update the parameter archive to replace the description to a normalized ID prior to storage and then redo afterwards. If you think saving 10MB storage space is worth that effort...

 

Cheers,
Johannes

[Mark Doyle]

Just to cover or expand upon a few themes:

The Space System Model deals with encoding and decoding parameters and commands for any actual hardware. Data coming from a weather station app doesn't have to exist in the space system model unless your hardware sends it or uses it, which will never be the case.

You can pass data around application level components however you wish, if it's not coming from or going to the space craft use what you like. There are a multitude of tried and tested methods of software component communication out there with almost all of them supported by Camel. Keep it at the application level. There is no reason to assume we can create a parameter based system for inter-process communication that works solely between apps better than experts in the field. We've seen it on another famous system, it's a mess, and you still have to know what to expect and how to decode it! Using real life methods also allows you to swap methodology by hiding it behind Camel. You apps are suddenly decoupled from each other relying only on their own interfaces.

The space system publisher deals with applications that need to display or send data to the actual hardware. It can be updated at runtime, assuming you want to, and publish updates to any interested component. Not really useful outside of a test environment unless some cavalier is holding all the money for the mission :) Works on the GUI right now; it can display telemetry incoming from any hardware wihtout having a single dependency upon the space system model itself. In fact all it knows about is an interface that accepts a Parameter<?>. Nothing else; actually doesn't even know where the parameters come from!

The only thing that has to be correct is the space system model, which is handled (or will be) by a Factory. It's obvious that has to be correct because we have to talk to hardware, often with bit level efficiencies. It doesn't have to be kept in sync with anything as it's the only config for hardware communication. It doesn't have to be static because the publisher can update apps and other  transport components. Apps don't have to be dependant or utilise the entire model because they decide what their interfaces are and what data they require. The telemetry GUI doesn't know anything about payload layout identifiers or even encoding. It exists to display values, a life of telemetry slavery :D

[Gert Villemos]

All,

As suggested I have created the repository 'https://github.com/Villemos/hbird-business' which contains the new business tire and the exchange tier that the new business tier is using. Lauri is 'beta-testing' the installation right now, using the new guides on 'http://www.hbird.de/?q=getstarted'.

I think we need to stop creating new repositories and instead start merging them.

As part of that we finally need to get the exchange tier harmonized, agreeging on the changes Johanness has been proposing and the restructuring into services I have done.

ASAP is a good time for me...

Cheers,

Gert.

---

Von: Gert Villemos <gvil...@googlemail.com>
An: humming...@googlegroups.com
Gesendet: 21:02 Montag, 2.April 2012
Betreff: Re: ESTCube requirements and SpaceSystemModel

[mhawkshaw]

Gert,

If it's a big change, and you'd like input from the Community and so would like another meeting arranged, let me know and I can arrange another poll and post it here.

Cheers,

Michael

On Tuesday, 3 April 2012 15:14:17 UTC+2, Gert Villemos wrote:

All,

As suggested I have created the repository 'https://github.com/Villemos/hbird-business' which contains the new business tire and the exchange tier that the new business tier is using. Lauri is 'beta-testing' the installation right now, using the new guides on 'http://www.hbird.de/?q=getstarted'.

I think we need to stop creating new repositories and instead start merging them.

As part of that we finally need to get the exchange tier harmonized, agreeging on the changes Johanness has been proposing and the restructuring into services I have done.

ASAP is a good time for me...

Cheers,

Gert.

---

Von: Gert Villemos
An: hummingbird-dev

[Gert Villemos]

Well, first of all I would like to hear what people think. People have been branching off repositories for some reason and I would like to know if they still think its a good idea. 

Right now we have the following repositories (... at least)

- Hummingbird Itself (... which is actually not the master branch anymore)

- Business Tier.

- OSGi GUI

- Web GUI

- OpenGL GUI

- Humsat

- Counterbird (Simulator)

- Thunderbird (MCS)

In my oppinion that is way too many. Nobody understands what they need to take. Some repositories are not really maintained. They dont integrate with each other, etc, etc. 

I think we need to merge them and create a consistent set of

- Hummingbird. Contains sub modules with Transport, Exchange, Business, Presentation (all GUIs), Simulators (all simulators) and the system level tests.

- 1 x Repository per mission (Thunderbird, ESTCube, TechDemoSat-1, Strand, ...)

Let me know what you think.

Cheers,

G.

[Lauri Kimmel]

I think Transport, Exchange, Business and system level tests should be in one repo. Probably Simulators also - I'm not familiar with them. All those components are more or less related to the "backend" or "server side". 

Presentation is bit different story. 

Typical RCP app has many plugins/bundles each in standalone module. Adding them all into Hummingbird main repo will make the build process slow and complicated. Target platform(s), jar-s not in OSGi/Eclipse format etc.

WebGUI started by Kait is JavaScript application and has not much to do with rest of the code base. Is there any good reason to put it into the main repo?

I don't know much about OpenGL UI. Most likely it is C/C++ project and building it together with rest of the Hummingbird is not so convenient.

Integrating presentation projects with rest of the system is matter of build process. CI server has build job for Hummingbird base and can have one for each GUI project. Successful base build can trigger GUI builds. Etc. 

Each mission should have own repo – I agree. For example ESTCube has its own repo managed by University of Tartu. 

Bit more about WebGUI. Last summer I had some spare time and I forked it from Kait's account. I added websocket support and did some refactoring. It has been in idle state for last 9 months and it's out dated most likely.

[Mark Doyle]

Here is how I see the partitions that have emerged. I think it's actually a type of emergent architecture as we grasped different concepts and very natural. Split is the wrong word.

I see that there are in essence to type of app. Some that are linked by a common API, some that interface with solely with the transport layer, some that interface with both the transport and other hbird apps, and others that have no direct links with hbird except some kinds of adapter (maybe just a camel type convertor to stick the interfaces together).

I honestly don't see the need to stick everything in the same repo;

• They can become large and unwieldy (SCOS anybody?)
• Present the illusion of integration when it might not exist or even be necessary.

We've built a framework with which to build monitoring and control systems from. It's not something you can check out, build and press go. When a developer uses hbird to build system they will just add a maven dependency, it doesn't matter where it was under source control.

Also, If there is a problem in the transport layer, the developer has clone everything, suddenly they have MBs of plugins, apis, guis and the like. The OSGi gui is in a separate repo to it's own target platform! for the same reasons.

Missions are really just systems built with hbird. They should definitely have their own repository. Anything else would be like creating an enterprise application and putting it under the Apache Camel SVN repo!

In fact, the RCP app is an OSGi application. There is no reason whatsoever to require that plugins developed for it are even in the same source control! 

Keeping a repo simple seems to fit with software best practices to me, single reponsibility, KISS etc.

A quick word on the repo list, Gert. I think a lot of them were just test assemblies, systems created using hbird. We probably don't even need to keep them.

[Lauri Kimmel]

Until there is no common base ground (called exchange AFAIK) for transport and business it's hard to talk about "framework" or define Hummingbird as such. 

For example currently there are at least two different definitions for Paramter available

https://github.com/Villemos/hbird-business/blob/master/exchange/core/src/main/java/org/hbird/exchange/core/Parameter.java

https://github.com/JohannesKlug/hbird/blob/master/src/core/commons/src/main/java/org/hbird/core/commons/tmtc/Parameter.java

I know that first one is used in "business" and second one in "transport", but at the end those two have to work together.

Same goes eventually for all other core/business/exchange (or how ever we will call them) objects like Command, Limit, Task etc.

For Hummingbird user perspective it looks like there is no clear vision what the framework is and how to us it. 

I can build something for business side and something for transport side, but there is no nice way to use them as one system.

What are your thoughts about this?

[Johannes Klug]

Hi Lauri,

you're absolutely right. This split was introduced in October 2010, and we've discussed it plenty of times since, both on this list and in private discussions.

The current situation is: We've got two pieces of software that share a name and some fundamental ideas, but are otherwise very different in scope. On the software side, they aren't compatible, as you pointed out. This is the main reason why I'm against lumping both of them together in one repository: it would give the illusion of a coherent suite of software, which it clearly isn't.

My highest priorities have been and still are in the "transport" part of the software. We've got a pretty decent and robust implementation of a universal, bit-packed payload codec, plus implementations of the CCSDS Transfer Frame and Space Packet standards. We also support a subset of CCSDS XTCE for defining the overall TM/TC structure. We've prototyped TCP and UDP based assemblies, which leave away the CCSDS stuff and feed directly into the payload decoder. We've got an interface to serial line radio hardware like the Aerocomm radio unit that the Dutch team uses on their Thunderbird rockets. In the very near future we'll interface with GnuRadio, and monitor and control our antenna rotor controller for the "Darmstadt Ground Station". In a separate repository, you'll find an Eclipse RCP based GUI.

If you need anything from the list above, you'll find its artifacts in the Sonatype OSS repository [1], and the source code on GitHub [2].

If you'd like to use something in hbird-business, I can't help with that. There just is no "connector" at the moment. I don't usually have a lot of free time these days anyway, but when I do, I spend it on ground station, antenna or radio stuff. Maybe Gert can help, but I reckon he's quite busy himself.

So, that's my view of the current status. My advice for you is to find out what's useful to you, and use it. If you happen to want pieces from both repositories, you'll have to find a way to make them interoperate. If you do, please contribute your modifications to the public source code repositories.

Have fun!
Johannes

[1] https://oss.sonatype.org/index.html#nexus-search;quick~hbird
[2] https://github.com/JohannesKlug/hbird/

[Gert Villemos]

I think that's the wrong answer. Of course the two must be integrated. 

And this is not a reason for keeping the repository separate; it demonstrates why the repositories should never, ever have been split. Segmenting the repositories creates exactly these issues.

As Johannes say; the vision for integration is there (or maybe "was" is better?) and we extra created the "exchange tier" for this. But for some reason the transport tier doesnt use the exchange tier (or the business tier occupied the exchange tier depending on how you look at it). We for a long timer just havent gotten around to do this. I think it has top priority.

I also think we should stop thinking 'Johannes and Mark do the Transport and Gert does the Business'. I has created a very weird separation, to the extend where I start asking whether we really have and want one system. We are busy pointing at the splendor of our own stuff, instead of thinking system. And as Lauri point out; there is little splendor in separated parts.

I think we need one system. Could people please indicate whether they agree.

Cheers,

Gert.

[Mark Doyle]

A connector between the two would be fantastic.

I don't see the need to create a huge monolithic source code repo though.

I wrote a few posts on April 4th (one even had a diagram!) which should cover my views....hopefully.