SlimDX 2.0

[Promit Roy]

It's come time that we need to tackle this seriously. What is the current status of the SlimDX 2.0 code on GitHub?

https://github.com/SlimDX/SlimDX 

My priority for a SlimDX update is Windows 8 support. That means Metro/WinRT as well as ARM compatibility. I also think this is a useful point at which to shed old APIs. D3D 11/DXGI, D2D, DWrite, XAudio, and XInput are the only major APIs to carry forward I think, along with the misc Windows, RawInput, D3DCompiler, etc stuff we've got. This also seems like the time to make any architectural revisions. I know there's been discussion of going to a native style explicit AddRef/Release model and a number of other things, but I don't know if there's any actual summary of what's been planned/changed in the current experimental development. My understanding is that the prototype is based around the same basic approach driving SharpDX, parsing DX headers and generating CLI bindings directly for C# code with better performance than what C++/CLI can do. It looks like SharpDX code-gen has substantially improved in recent months, andI would like to know what it is exactly that we're planning on doing differently/better than he has. I'm not interested in doing our own just because of ego/personality clashes.

Effects. Given Microsoft's semi-abandoned approach to the D3D 11 effect compiler, we need to figure out exactly how we're going to handle it. I would love to do a full re-implementation in C#, but I would also love a million dollars cash. 

Assemblies. Last I checked, the dominant thinking was that we are going to have a separate assembly per major API instead of a single super-binary. I missed the explanation of why this is preferable.

I don't have any real interest in further developing the current SlimDX codebase. So the options are to use the GitHub code, write from scratch, or somehow derive from the current iteration of SharpDX. I'm not opposed to any of these on principle, but we need to build a coherent plan for what we're doing and why. It might be interesting to leverage WinRT heavily, for example, which would be a fundamentally different codebase than SharpDX. That's the route I am inclined to take right now, at least.

Lastly, I am curious to know who is going to available to actually code on the new thing. I've CCed a few people who expressed interest in possibly contributing to the project; please join the slimdx-devel mailing list if you are interested:

http://groups.google.com/group/slimdx-devel

Further discussions should be carried out on the list (rather than IRC) for the time being.

-- Promit

[Josh Petrie]

My local fork of the code is ahead of the official repository:  https://github.com/jpetrie/SlimDX 

The trampoline system for generating the P/Invoke calls works well. The parser is functional, but not in an ideal state and I don't think we should bother pursuing that line of development anyway (I'll explain in a moment). The generator can produce usable, fully-managed assemblies from an API description. The toolchain uses a combination of XML and JSON, but JSON is being transitioned out because using XML will allow for XSLT to be leveraged to transform input API descriptions.

The goal, I remind you, was to take an API description in XML form, transform that via a set of rules (such as understanding that "GetFoo" and "SetFoo" methods on an interface can be turned into "Foo" property). 

Given the direction Microsoft is taking with the Direct3D SDK and the difficulty of parsing C++ code, I'm of the opinion that we should drop the parser from the toolchain, because it's not a fun thing to maintain and it has pretty limited utility given that the DX headers will not be changing that frequently. We can run something like GCC-XML over the D3D headers locally, run that through XSL transforms, and check in the resulting XML API description (this will not violate the GPL license of GCC-XML). It can probably be hand-tweaked from there to account for API updates.

That's basically what Mike and I had been up to. In comparison to SharpDX, we intended to have something that was more broadly applicable (the SharpDX code has a lot of hard-coded rules in C# for leveraging transformation and was exceedingly difficult to modify cleanly, we have a more uniform approach implemented or planned and are more amenable to data-driven rules which would allow the v2.0 generator code to be split off into its own project for wrapping other C-like APIs). Most importantly, of course, our approach is legal. SharpDX violates the GPL (by including the gcc-xml binaries, which were modified to compile on modern Windows, without making the source available) and -- at least the last time I checked -- scrapes the MSDN documentation and reproduces it without consent.

Some other commentary and response:

- I agree that old APIs should be dropped, and I think your list is pretty conclusive in terms of what we should bother supporting.

- Using IDisposable is Bad(tm). We should absolutely use an explicit AddReference/ReleaseReference model to deal with the COM reference counts. The end-user can trivially wrap this in a faux-RAII IDisposable wrapper if they so choose.

- A fully-managed re-implementation of the Effects framework is an interesting idea and would certainly provide a way for SlimDX to differentiate itself. Wrapping the Effects11 code is obnoxious anyhow. I'd be interested in this except that it will probably involve parsing and I absolutely hate parsing.

- Multiple assemblies just lets an end-user include only the code they want or need. I think it's a bit cleaner, and it also makes it easier to work with the API description XML since it keeps it more localized. There is no real compelling technical reason though.

So, all of this said... I don't know how much of my discussions with Mike you've seen from IRC, Promit. But for the benefit of these other fellows on the cc-line, I'll repeat the gist of my current state of mind:

I am not all that interested in the direction Microsoft is taking Windows with Windows 8. Not from a user's perspective, and not from a developer's perspective either. I will withhold some measure of judgement until the thing actually ships, but right now I've basically stopped doing Windows development as a hobby. Consequently I'm not that motivated to continue working on SlimDX for the time being. I think the generate-from-API-XML approach has merit and that we've proven, with the current v2 codebase, that it is viable. So I might continue to develop that technology as a separate project regardless of what you do with SlimDX, once my sense of "burn out" goes away. 

But for now you're probably safe assuming that I won't be making too many active contributions to the codebase over the next few months.

Can you elaborate on what the "leveraging WinRT heavily" approach might look like?

-- jmp

--
-- jmp

[Promit Roy]

I am not all that interested in the direction Microsoft is taking Windows with Windows 8. Not from a user's perspective, and not from a developer's perspective either.

 I am inclined to this viewpoint myself, but given that Windows is going to march onwards no matter what happens, I see an opportunity to be present when Win8 tablets roll out. Microsoft's current (and stunningly developer-unfriendly) answer to developers who want to use DirectX 11 is "Use C++". 

Can you elaborate on what the "leveraging WinRT heavily" approach might look like?

I am still working on the implications, but essentially it means writing the new implementation in C++/CX (hand or generated) and being exposed as an API available to .NET, JavaScript, and native C++ clients. I don't know how performance would compare to the current approach or the new PInvoke trampoline, though it would make it possible to run SlimDX in a completely native context. I'm not sure if that is advantageous, and I also wonder to what extent JS/HTML5 will be adopted for development on Win8. This would clearly be the most differentiated approach.

It's probably possible to reshape the existing code from CLI to CX, though what that will do at an architectural level I'm not sure. CX seems to have a very specific attitude on memory management (COM style ref counts, I understand) and it seems like adopting that and letting the frameworks do their thing is the way to go. Again I don't know what that does exactly on the .NET side. Presumably Microsoft has it all neatly drawn up somewhere but I've only just started reading the relevant documentation. Once the technical aspects are more clear, it may be that Win8 support is just a few keyword tweaks away, or it may be that there is serious work to be done.

-- Promit

[Fraser Waters]

To chip in to this, and make myself known.

> I am curious to know who is going to available to actually code on
> the new thing.

Me, I'm happy to work on anything that needs it but I actually quite
like the idea of building up the effects library.
I've been playing around a bit with effects, seeing if it would be
possible to both have a general Effect class similar to the current
one, but also being able to take HLSL effect files and generate typed
C# classes for them. Think how XNA has Effect and then BasicEffect,
SkinnedEffect etc except that it would work with users HLSL code.

I agree with the other comments so far, only thing that concerns me is
the "IDisposable is Bad(tm)" comment. I understand it doesn't play
nice with DirectX but it is the expected way in .NET, making it work
seems better than putting Add/Release responsibility on the users.

> > approach is *legal*. SharpDX violates the GPL (by including the gcc-xml

> > On Sun, Feb 5, 2012 at 12:33 PM, Promit Roy <promit....@gmail.com> wrote:
>
> >> It's come time that we need to tackle this seriously. *What is the
> >> current status of the SlimDX 2.0 code on GitHub?*

[Josh]

IDisposable is a commonly-seen pattern in the .NET world, that is true -- but that doesn't make it the proper pattern to use everywhere. It shouldn't the "expected way" to clean up everything. In particular, the implication of the interface's contract is that one should only dispose of an object one creates. It's not always possible to indicate whether "creation" has occurred when an IDisposable object is returned to the client. In SlimDX, doing things like getting the back buffer surface object were typical ways to run afoul of this. 

Our original implementation thus required you to call Dispose on every reference (even though you got from a property accessor) you ever made to a COM object, which goes against the IDisposable idiom. Later implementations introduced the object table, which was based heavily on the CLR's COM RCW, which tried to enforce a single COM reference to any object SlimDX knew about. This meant when you accessed and object, such as the back buffer, you had to know if you had created this yourself (directly or indirectly) earlier in the code, and call Dispose() or not accordingly. Sometimes this was extremely difficult for somebody who didn't know the inner works of D3D and SlimDX to understand.

Niether solution conforms to the IDisposable idiom correctly; the latter option is easier for the client on a majority of cases, but the cases where it falls apart often become completely intractable and force the client to resort to some pretty ugly hacks to properly maintain the lifetime of objects.

I honestly don't think the concept of reference counting resources is too difficult for users to grasp -- it's certainly less complex than the edge cases they'd have to understand now to properly manage the lifetime of certain COM interfaces. I don't think IDisposable offers any benefit whatsoever ("using" blocks to automatically release a reference can be done with a utility wrapper class, and that pretty much covers it). Making IDisposable work is one of the major sources of complexity in the SlimDX code. Even then, to say it "works" is a bit of a stretch.

Have you looked in to starting doing an effects library at all?

-- jmp

[Fraser Waters]

That's a good enough argument to convince me.

I've started looking into effects, looking at xna and d3dx to try and
see what the interface should be.
I'm assuming we want to support the same syntax and be binary
compatible with fxc? If not then we need to decided on the syntax to
support.

Probably won't have much time to work on this over the next two weeks,
due to exams and then starting an internship at ARM. Gives some time
to make decisions before starting the library in earnest.

> ...
>
> read more »