Window creation and graphics context

[Jeremiah Ong]

I posted in the ISO C++ Standards group and am reposting it here since I wasn't aware that the graphics proposal fell under "I/O" (a recent change it looks like)

Regarding the 2D drawing API, I can't help but wonder if we're jumping ahead of ourselves. For the sake of discussion, I believe there are a number of details that need to established and standardized before this will be feasible or useful.

1. Creation of a window/display (resolution, fullscreen, etc)

2. Configuring the renderer (OpenGL, DirectX, software only)

3. Way to allow the enduser to detect input (if the 2D api lets us create a window, how useful can it be if the user can't interact with it?)

Currently, the options for doing this are very murky. For new developers, writing a cross platform graphical application is daunting without a framework in place, venturing far away from core or standard C++. SDL and Qt comes to mind as two frameworks that work with Windows, Linux, OSX, and mobile devices for game developers and application developers respectively. As a result, even getting a simple window to display backed by, say, an OpenGL context is somewhat a rite of passage and requires a lot of patience and trial and error with multiple dependencies. Traditionally, the competing operating systems focused on their own tools for doing this for obvious reasons but in the current software ecosystem, most developers will need to make their creations fully cross platform, so a common API is unlikely to foil any competitive advantage any more.

Before venturing into a 2D api, it seems to me like a lower level abstraction would set us up better in the long run. With lower level abstractions, this would greatly simplify existing user code in the wild (plagued by macros and foreign function calls) and 2D drawing frameworks would certainly crop up in the wild.

My rationale for why doing a low level window/graphics context abstraction first makes sense:

1. For performance, developers will expect hardware acceleration and so, the standard will need to be graphics context aware anyways

2. CAIRO is a multi-backend api for rendering to many targets (pdf, window, png, etc) and wasn't really designed to create interactive applications AFAIK.

3. Creation of a 2D drawing api runs the risk of being too specific to be widely useful (i.e. a person who just wants to make a chart already has easy options in the wild)

4. Historically, applications written using Java's 2D api needed to be rewritten because the api didn't give the user sufficient control. In graphics in particular, predicting what the user wants is challenging.

5. One stated application of the 2D api in the proposal is for games. But modern 2D games need much more than a simple fixed-function drawing calls. They may need hardware-accelerated particles, lighting, sprite sheet handling, texture mapping, etc.

In short, my primary concern is that this proposal will make something simple even simpler but won't make it any easier for the developer to create the difficult things. For games in particular, I can say that if N3888 was implemented verbatim, I probably would not use it in any of my personal projects and I would venture to claim that many other game developers would eschew it as well. There is a reason fixed-function pipelines were rendered obsolete. They simply do not afford the programmer enough flexibility and the programming model is "incorrect" in that it doesn't mirror well what is happening fundamentally in a rendering pipeline. The proposal *does* make an effort to consider the various platforms (see "GPU Resource Management in Various Technologies"), but it would appear that all of that work to unify the api across platforms would be opaque to the programmer. In contrast, if by some miracle, the first 3 points I mentioned above *were* standardized, it would be immediately useful to everybody and many different frameworks for a variety of uses would most certainly crop up immediately as a result. Perhaps those resultant projects would converge on a great API that could be standardized in the future.

Cheers,
Jeremy

[jason zink]

Hi Jeremy,

I think the majority of the points that you brought up are worthy of further consideration, and they have also been discussed to some extent in the other discussions in this group.  The rationale for the proposal contained within N3888 was to have a functional starting point, that we could build on in an incremental fashion.  We are talking about an API here - not an implementation.  Nobody is suggesting to standardize Cairo, but rather to use Cairo as a well known and functional API that we can transform.

Some general comments on some of your points from above:

Standardizing a hardware rendering interface is not likely to happen.  However, there is no reason that hardware acceleration can't be available in the implementation of the library.

You are correct about the need to add input to the proposal, and it is planned / expected to come in a future revision.  I'm not entirely sure that a window specification would be appropriate for this proposal, as there are so many other areas that would be influenced - it should probably be in its own proposal.  My understanding is that previous attempts at a windowing proposal have been met with significant resistance, so we also need to be realistic about what we can achieve in SG13.

Of course, if there is a better way forward, then any competing proposal is welcome to compare and contrast against N3888.  Any proposal should be able to stand on its own merits, and more ideas are likely to lead to a better result in the end.

Jason

--
You received this message because you are subscribed to the Google Groups "I/O" group.
To unsubscribe from this group and stop receiving emails from it, send an email to io+unsu...@isocpp.org.
To post to this group, send email to i...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/io/.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/io/98c198b4-7fa5-4d83-bb69-e732142e7417%40isocpp.org?utm_medium=email&utm_source=footer.

[Jeremy Ong]

Hi Jason,

Thanks for the response.

On Thu, Mar 20, 2014 at 7:12 PM, jason zink <jzi...@yahoo.com> wrote:
> Hi Jeremy,
>
> I think the majority of the points that you brought up are worthy of further
> consideration, and they have also been discussed to some extent in the other
> discussions in this group. The rationale for the proposal contained within
> N3888 was to have a functional starting point, that we could build on in an
> incremental fashion. We are talking about an API here - not an
> implementation. Nobody is suggesting to standardize Cairo, but rather to
> use Cairo as a well known and functional API that we can transform.

Just out of curiosity, how is Cairo used and what is its penetration
like today? I can't help but feel I'm ignoring it's obvious use cases.
Perhaps my judgement was clouded by the fact that the proposal
mentioned "games" in a number of places and I simply couldn't see
using Cairo as a starting point for a game today. Perhaps the actual
goal and use cases should be clarified or corrected?

> Some general comments on some of your points from above:
>
> Standardizing a hardware rendering interface is not likely to happen.
> However, there is no reason that hardware acceleration can't be available in
> the implementation of the library.

Yea this I expected. I only brought it up because it seemed like the
implementations would necessitate it (perhaps controlled by compiler
flags or something). I do know, that if the hardware rendering
interface was being created in the background, I would certainly want
access to it though.

>
> You are correct about the need to add input to the proposal, and it is
> planned / expected to come in a future revision. I'm not entirely sure that
> a window specification would be appropriate for this proposal, as there are
> so many other areas that would be influenced - it should probably be in its
> own proposal. My understanding is that previous attempts at a windowing
> proposal have been met with significant resistance, so we also need to be
> realistic about what we can achieve in SG13.

Understood. I imagine that unless there was a good proposal that
necessitated windowing, any such proposal would not be useful on its
own. Not to mention platform specific considerations (how would it
work on a phone, tablet, or console).

>
> Of course, if there is a better way forward, then any competing proposal is
> welcome to compare and contrast against N3888. Any proposal should be able
> to stand on its own merits, and more ideas are likely to lead to a better
> result in the end.

I'd like to at the very least pin down the audience for such a feature
so we can discuss if that audience is actually being served by the
proposal. It's possible that I am ultimately not somebody for whom
this proposal is ultimately targetting, and I'm OK with that.

Cheers,
Jeremy

> https://groups.google.com/a/isocpp.org/d/msgid/io/1395367957.53032.YahooMailNeo%40web120804.mail.ne1.yahoo.com.

[Olivier Giroux]

About Windowing, the irony is that this is definitely something which is not served well by existing standards.

For example OpenGL is a real standard, covering both 2D and 3D graphics, but to just get at OpenGL you need to write a ton of non-standard code as you pointed out earlier.  If C and C++ had basic windows, then there wouldn't be this annoying gap.

For the past 15 years, most people have used GLUT to fill this gap, at least initially in their process.  Unfortunately GLUT is neither standard, nor even just maintained anymore.  There is no GLUT for iOS or Android, for example.

Olivier

To view this discussion on the web visithttps://groups.google.com/a/isocpp.org/d/msgid/io/CALax_rx1xXy7aViMoQ61JOtU8ggBYSPL2YRWh550%2BULC8FE5CA%40mail.gmail.com.

[Michael McLaughlin]

On Thu, Mar 20, 2014 at 11:27 PM, Jeremy Ong <jerem...@gmail.com> wrote:

Hi Jason,

Thanks for the response.

On Thu, Mar 20, 2014 at 7:12 PM, jason zink <jzi...@yahoo.com> wrote:
> Hi Jeremy,
>
> I think the majority of the points that you brought up are worthy of further
> consideration, and they have also been discussed to some extent in the other
> discussions in this group.  The rationale for the proposal contained within
> N3888 was to have a functional starting point, that we could build on in an
> incremental fashion.  We are talking about an API here - not an
> implementation.  Nobody is suggesting to standardize Cairo, but rather to
> use Cairo as a well known and functional API that we can transform.

Just out of curiosity, how is Cairo used and what is its penetration
like today? I can't help but feel I'm ignoring it's obvious use cases.
Perhaps my judgement was clouded by the fact that the proposal
mentioned "games" in a number of places and I simply couldn't see
using Cairo as a starting point for a game today. Perhaps the actual
goal and use cases should be clarified or corrected?

 

The language in the proposal needs tweaking and modification, certainly. Work is underway on a successor to N3888 that will clean it up and also modify the interface it proposes to better embody modern C++ design.

 

Are there games that use cairo? Probably some, though I doubt it's widespread. SDL would probably be a better choice as things stand now. But GTK+3 uses cairo for its rendering. And Firefox uses it in various places (they have partially moved away from it for performance reasons but A) they still use it quite a bit; B) they have a highly specialized use case (rendering an HTML object graph can get very esoteric with thousands of draw calls and other things that would make a game developer cringe :) ); and C) we plan to address the concerns they had with regard to the number of API calls they needed to make by modernizing the interface and making it better match the realities of modern GPUs. I expect that implementers will address raw performance concerns (for example, I expect that a VC++ implementation would use Direct2D + DirectWrite, possibly with some direct D3D usage as well, whereas cairo currently uses GDI for Windows OSs).

 

> Some general comments on some of your points from above:
>
> Standardizing a hardware rendering interface is not likely to happen.
> However, there is no reason that hardware acceleration can't be available in
> the implementation of the library.

Yea this I expected. I only brought it up because it seemed like the
implementations would necessitate it (perhaps controlled by compiler
flags or something). I do know, that if the hardware rendering
interface was being created in the background, I would certainly want
access to it though.

  

The proposal embraces the concept of native handles (which are successfully used in the STL's threading library) to allow implementers to expose things like OpenGL contexts and D3D devices. We can't force them to do so without getting into implementation details but we expect that a combination of their desire to provide a great product along with the demands of their customers should result in a good outcome. And since we are only standardizing an interface, if a compiler vendor fails to meet their customers' needs, someone could write a third party implementation (placing it in a different namespace and distributing it as a library) that would fill the gap for whichever platform(s) needed a different/better implementation.

 

>
> You are correct about the need to add input to the proposal, and it is
> planned / expected to come in a future revision.  I'm not entirely sure that
> a window specification would be appropriate for this proposal, as there are
> so many other areas that would be influenced - it should probably be in its
> own proposal.  My understanding is that previous attempts at a windowing
> proposal have been met with significant resistance, so we also need to be
> realistic about what we can achieve in SG13.

Understood. I imagine that unless there was a good proposal that
necessitated windowing, any such proposal would not be useful on its
own. Not to mention platform specific considerations (how would it
work on a phone, tablet, or console).

  

The "W" word is taboo. We aren't aiming to standardize windowing, controls, etc. That said, I expect that the input proposal will dovetail with the drawing proposal (resulting in a unified proposal that contains drawing and user input) and that we will end up with a way to create a display surface that gives developers access to user input that is directed at that display surface. By keeping the input surface at a very low, abstract level, implementers should be able to implement it in a way that is meaningful and useful regardless of the form factor the target devices.

 

This is another area where native handles will likely come in handy. We realize that it would be impractical (both technically and "politically") to try to standardize things like windows and gui toolkits. But we recognize that developers may need access to functionality that we don't provide. So we let implementations provide the user with an implementation-specific native handle that can then be used to interact with platform-specific functionality. That native handle might be something like an HWND or a Window (X11), but it might be a struct that contains several different resource pointers/objects. It all depends on what makes sense for the given implementation.

 

>
> Of course, if there is a better way forward, then any competing proposal is
> welcome to compare and contrast against N3888.  Any proposal should be able
> to stand on its own merits, and more ideas are likely to lead to a better
> result in the end.

I'd like to at the very least pin down the audience for such a feature
so we can discuss if that audience is actually being served by the
proposal. It's possible that I am ultimately not somebody for whom
this proposal is ultimately targetting, and I'm OK with that.

  

What we are driving at with this effort is something called a Technical Specification ("TS"). A TS is not something that is officially part of the Standard but it is nonetheless something that is formalized enough that implementers should feel comfortable providing an implementation of it for their users to test and explore. Because they aren't officially part of the Standard, a TS is reasonably malleable. So feedback from users of it will inform and guide its development. It will also ultimately help the ISO C++ Committee decide when and whether the TS should be adopted into the Standard. Our goal is to create something that is useful to the broadest number of users.

 

-Mike