New proposal on I/O Device Requirements

Cleiton Santoia

unread,

Nov 7, 2014, 12:30:46 AM11/7/14

to i...@isocpp.org

It´s not a proposal yet, please comment, and as soon as possible to champion it, we will deliver to standard.

https://github.com/cleitonsantoia/io/blob/master/io.pdf

BR

Cleiton

Klaim - Joël Lamotte

unread,

Nov 10, 2014, 9:05:57 AM11/10/14

to i...@isocpp.org

Hi, I just wanted to notify that I'm in the middle of this paper and will have questions,

I'm just short of time to provide feedback sooner.

I also am doing some research for a potential proposal which started as a input interface libary.

But I'm far from anything conclusive yet, I'm exploring ideas. Such document might help with that

so thanks for your work.

--
You received this message because you are subscribed to the Google Groups "I/O" group.
To unsubscribe from this group and stop receiving emails from it, send an email to io+unsu...@isocpp.org.
To post to this group, send email to i...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/io/.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/io/898835cd-f6e8-4340-87a6-a5690f111b8f%40isocpp.org.

Cleiton Santoia

unread,

Nov 11, 2014, 12:53:09 AM11/11/14

to i...@isocpp.org

Thanks, I´m looking forward for comments :)

Klaim - Joël Lamotte

unread,

Nov 20, 2014, 7:23:41 AM11/20/14

to i...@isocpp.org

Here are my comments:

1. Shouldn't these kind of recommendations be given only after experience have been built with

implementations following these guidelines? I don't know any implementation following these recommendations.

2. I don't agree with the "device as an associative container" viewpoint because there is a lot of

input devices which basically expose different sets of data (at different times) with different or simmilar types.

At the moment, I believe that:

- input and output devices interfaces, should not have the same requirements

(although they probably would end up with a fez common requirements);

- these devices basically output or input "channels" of data (one device, n channels, each channel

with potentially different types of data);

I'll develop this last point on these forums later if I end up somewhere satisfying with that idea.

Anyway, assuming that one device expose only one kind of data is, in my opinion, incorrect.

3. What if a device configurations have non-discrete values? How would such "possible configurations"

could be enumerated?

Maye there is too much assumptions that I/O devices have to work like a "screen"?

4. It is not clear to me what device "features" are in the context of this paper.

5. I think that "device good" is not an appropriate term. Maybe "usable" or "ready"?

6. The assumption that an event-driven interface would use condition variables or a waiting

mechanism seems very unusual to me. All event-driven interface associated to a device I

have used so far are non-blocking and just assume that the callback implementor take into consideration

that the callback will not be called in a known thread.

In any way, assuming that users will wait have at least one thread for a device to wait for it

don't seem (to me) to reflect any real implementation.

Is there a known example of this that I don't know?

Isn't it better to not specify if a device interface all will wait or not, and let the implementors see what'same

best for the device?

7. I don't understand the meaning of 3.8.

8. The iteration idea is nice but only if you assume that:

- the device expose data in memory (graphics might not);

- the device expose only one set of data;

- the device expose data;

I would have prefered a separate way to acquire the state of a channel of data,

(at least for input devices) and then have iterators on that state.

Cleiton Santoia

unread,

Nov 20, 2014, 11:41:23 PM11/20/14

to i...@isocpp.org

Em quinta-feira, 20 de novembro de 2014 10h23min41s UTC-2, Joel Lamotte escreveu:

1. Shouldn't these kind of recommendations be given only after experience have been built with
implementations following these guidelines? I don't know any implementation following these recommendations.

1.1 Well, me too, I don´t know about a library ( or "standard" ) that has tried a bold task of defining a structure for many different types of devices. The same way at some point in time, nobody have "Container Requirements" each container of each vendor have a different implementation, anyway, we are aware that "nobody uses this" may be a huge no-go, however, for few decades, the hardware and drive implementors focus was C interface and also now Java, without much help from the software architects, we must help them move to C++14 ! And also it's not really new, it's just an attempt to organize what we already have, the only "new" thing is that small wait mechanism.

1.2 One of the biggest projects of the company I work for is a "Deployment streaming or file via internet (realtime or offline) digital media propaganda to your business" in multi-monitor displays, and multi-machine - multi-monitor-displays with video, feeds, flash, web-browsing ( with real-time price changes ) and wherever you want. We got a lot of experience dealing with different showing devices, Tv's, LED display, projectors, array of monitors, video-walls etc. Also input via cam capture, face-recognition, queue-lengh, kinetics, etc. However I must be honest, none of the patterns we showed in the paper resembles what we have in production today, we are not even using C++11 yet in most of the projects, the paper shows what we would do if we could start over again.

2. I don't agree with the "device as an associative container" viewpoint because there is a lot of
input devices which basically expose different sets of data (at different times) with different or simmilar types.
At the moment, I believe that:

- input and output devices interfaces, should not have the same requirements
(although they probably would end up with a fez common requirements);
- these devices basically output or input "channels" of data (one device, n channels, each channel
with potentially different types of data);

I'll develop this last point on these forums later if I end up somewhere satisfying with that idea.
Anyway, assuming that one device expose only one kind of data is, in my opinion, incorrect.

You mean that the "device" should be a collection of "channels" each one with it's type right ? We have considered that, but when we try to define the "channel" we, most of time, reached a similar specification of the device itself, possibly you may need acquire access to channel, one particular channel may have different features and configs of another channel of same device and so on, so we prefer to state this as a "composite-device" instead of device-channel and end-up with some similar definitions for device and channel.

3. What if a device configurations have non-discrete values? How would such "possible configurations"
could be enumerated?
Maye there is too much assumptions that I/O devices have to work like a "screen"?

The two last examples of 3.4: one when you just set some, not all, config values, and the last when you pick a range (min-max) of values for each config value.

And you can be more inventive if you return a map<string, any>, when each config var have a different type;

Or a map<string, vector<any> > for 'enumerate' possible values;

Or even map<string, MyType > where MyType is a complex type.

You can also replace the map with your own "mapping" class if you wish;

4. It is not clear to me what device "features" are in the context of this paper.

We named "the immutable capabilities" as features ( sorry for the poor English, we are open to change the name of everything )

Here some samples:

- In a "Desktop Device" the "memory" and "processor type and speed" are "features" not "configs", since you cannot change them, you cannot just say "ok desktop, now you have 256GB or RAM" !

- In a nVidia GTU card, you may have a "Compatible with DirectX 11" and "2x DVI", "1x HDMI" ports

- In a HDD you may have "speed 7.800 rpm" feature.

5. I think that "device good" is not an appropriate term. Maybe "usable" or "ready"?

( Again, we are open to change the name of everything ), we did that because std::ios, the base class of all iostreams, we have a good() method, this way feels familiar.

6. The assumption that an event-driven interface would use condition variables or a waiting
mechanism seems very unusual to me.

Yes it´s a step forward, in the design of devices ! ( We Hope )

All event-driven interface associated to a device I have used so far are non-blocking and just assume that the callback implementor take into consideration that the callback will not be called in a known thread.

The wait signal is lightweight to the device, if nobody is waiting, the signal will be ignored and the device keep doing it's business, what your program will do with the signal is not problem of the device, and this helps in the case where your app thread become stuck and the device implementation that called your app via callback become stopped too, with the notify signal the device is free to fly !

In any way, assuming that users will wait have at least one thread for a device to wait for it don't seem (to me) to reflect any real implementation.

Is there a known example of this that I don't know?

If you assume a UDP sock as a device, this is almost what the boost::asio does.

- To send you a asynchronous message, io_service has its's own threads.

- When you call async_receive_from it just add a callback handler in the io_service queue,

- When app reach io_service.run() , the io_service starts and keep some threads waiting something happens ( arrive some data ) then send you a callback message.

- Again, same as above, the problem happens when your application become slow, then you are consuming a thread from io_service ( in some cases, consuming all threads ) to do the application job. In the proposal, the device will set some values in an iterator (or some queue) and send a notify via condition var, no app ( alien to device ) code will run under the device thread, no device implementors will need the check for exception in the app code, and also, let the io_service thred clear to work for next event.

And if you think it's important, like a keyboard buffer, you may implement a event queue in the device to keep the last events, so when the app iterator finally move to next event, you just pop out the event of the ( limited ) queue.

Isn't it better to not specify if a device interface all will wait or not, and let the implementors see what'same best for the device?

This is the point ! If the I/O is non-blocking, just use [] or >>.

7. I don't understand the meaning of 3.8.

The "aquire" thing ? or the "wait-acquire" thing ?

8. The iteration idea is nice but only if you assume that:

- the device expose data in memory (graphics might not);
- the device expose only one set of data;
- the device expose data;

I wonder how someone iterate over something that does not exposes data ? :)

All (today) graphics card send a "vertical retrace" event, let´s say your monitor has 70Hz refresh rate at 1024x786 resolution, then your RAM is really faster than that, ok ? So, it´s possible for your favorite game draw a full frame to the RAM, before your monitor ends draw it to the screen, and than your game start the next, and the next, end by the 4th frame, your monitor finishes, in the screen you see four different pieces of four different frames. For this not happens your monitor is drawing the RAM buffer A to the screen, the game must draw in RAM (buffer B) wait the "vertical retrace" event, "screen flip" from A to B and start write next frame to buffer A.

The same way the std::map<> have a find method and std::vector<> don't, one device may have iterators and other don't and the must set the traits accordingly .

We also will replace the iterators to ranges in this section, they are really better and solve few problems.

I would have prefered a separate way to acquire the state of a channel of data,
(at least for input devices) and then have iterators on that state.

I don't know if I understood...

Consider a house fire alarm detector;

1 - You acquire_wait until someone turn-on the thing ...

2 - You set the config sensibility for a factor .01,or whatever config you need to set;

3 - You get an iterator for input and wait. When it detects smoke, it sends you a signal;

Please, tell me your version of acquire and iterate.

Thank you

BR

Klaim - Joël Lamotte

unread,

Nov 21, 2014, 5:32:22 AM11/21/14

to i...@isocpp.org

On Fri, Nov 21, 2014 at 5:41 AM, Cleiton Santoia <cleiton...@gmail.com> wrote:

You mean that the "device" should be a collection of "channels" each one with it's type right ? We have considered that, but when we try to define the "channel" we, most of time, reached a similar specification of the device itself, possibly you may need acquire access to channel, one particular channel may have different features and configs of another channel of same device and so on, so we prefer to state this as a "composite-device" instead of device-channel and end-up with some similar definitions for device and channel.

Interesting. I didn't end up there yet, but I didn't consider it this way.

7. I don't understand the meaning of 3.8.

The "aquire" thing ? or the "wait-acquire" thing ?

I think I use "acquire" in a different way than you. I was considering acquisition like "getting access and control"

of the device, which seems slightly different?

8. The iteration idea is nice but only if you assume that:

- the device expose data in memory (graphics might not);
- the device expose only one set of data;
- the device expose data;

I wonder how someone iterate over something that does not exposes data ? :)

I was assuming that some devices might not expose data, but still expose configurations and way to set them.

I might be wrong though.

I would have prefered a separate way to acquire the state of a channel of data,
(at least for input devices) and then have iterators on that state.

I don't know if I understood...

Consider a house fire alarm detector;

1 - You acquire_wait until someone turn-on the thing ...
2 - You set the config sensibility for a factor .01,or whatever config you need to set;
3 - You get an iterator for input and wait. When it detects smoke, it sends you a signal;

Please, tell me your version of acquire and iterate.

I explained acquire, but for iteration I was considering just a way to get through the data already received,

not a way to wait for it.

I didn't think about it this way though.

Thank you
BR

--
You received this message because you are subscribed to the Google Groups "I/O" group.
To unsubscribe from this group and stop receiving emails from it, send an email to io+unsu...@isocpp.org.
To post to this group, send email to i...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/io/.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/io/64ca58ff-fd42-4f19-9547-83cb1a265265%40isocpp.org.

Cleiton Santoia

unread,

Nov 22, 2014, 2:06:19 AM11/22/14

to i...@isocpp.org

Em sexta-feira, 21 de novembro de 2014 08h32min22s UTC-2, Joel Lamotte escreveu:

I think I use "acquire" in a different way than you. I was considering acquisition like "getting access and control" of the device, which seems slightly different?

Yes that is it, but what if the device is busy acquired by another application, as a scanner or a fullscreen display, you can only acquire for one application at a time, how inform that the application will wait to the device be available ?

I was assuming that some devices might not expose data, but still expose configurations and way to set them.

You are right, some devices are not iterable, and exposes only configs, a desktop usb fan for example, in this one you may only configure the speed ( one, two or three ) in this case, while implementing the device, just set iterable trait as false and suppress the implementation of iterators. But remember that the config will be iterable anyway ( even if there is only one possible ).

I explained acquire, but for iteration I was considering just a way to get through the data already received, not a way to wait for it.
I didn't think about it this way though.

Let´s say you have a TCP Sock device on port 80, and the device implements the iterator<TCPBuffer> this way you can wait until the TCP receive a message, so each event will bring the whole buffer from the message, 1KB to 10KB of data for each event, sure if TCPBuffer is some sort of vector of bytes, it will have a proper set of iterators and sure for each event someone will iterate over the buffer "offline" out of device, but when the next event occurs.