OpenEmulator's NTSC GPU shader ported to webgl

[Zellyn]

[First posted in comp.emulators.apple2, but I think nobody knows about that one :-) ]

Hi folks,

I finally got around to a long-time goal of mine: porting OpenEmulator's GPU shader to webgl (and in the process, converting from a fixed-function pipeline to the modern vertex shader approach — don't worry if you don't know what that means; I barely do either, but I can hit a nail with stack overflow until it goes in…)

Anyway, here's the (very bare-bones) initial port:

https://zellyn.github.io/apple2shader/

Feel free to clone it, play around with it, whatever. It's GPL, because Marc Ressl's original code is GPL.

I'm happy to answer questions, although I still haven't really tried to understand how it works.

Also feel free to discuss in the #openemulator channel on Slack at retrocomputing.slack.com. (You can sign up at https://retrocomputing.herokuapp.com)

Zellyn

[Thomas Harte]

> I finally got around to a long-time goal of mine: porting OpenEmulator's GPU shader to webgl (and in the process, converting from a fixed-function pipeline to the modern vertex shader approach — don't worry if you don't know what that means; I barely do either, but I can hit a nail with stack overflow until it goes in…)

I have to say I'm very impressed that an earlier version was performing NTSC decoding with only the fixed pipeline. Presumably through some sort of accumulative weighted and repositioned placement of the original and/or careful mip map sampling?

That's fairly academic though; should we expect OpenEmulator in the browser any time soon?

[Zellyn]

On Wednesday, May 16, 2018 at 11:20:11 AM UTC-4, Thomas Harte wrote:
> I have to say I'm very impressed that an earlier version was performing NTSC decoding with only the fixed pipeline. Presumably through some sort of accumulative weighted and repositioned placement of the original and/or careful mip map sampling?

Hmm. It's possible I don't know what I'm talking about :-) I think perhaps it was using the predefined vertex stuff (glLoadIdentity(), glRotatef()) for the vertex transformations, but it definitely uses a fragment shader for the NTSC decoding. Here it is, in my translation, which is easier to read than OpenEmulator's code only because it lacks "\n" everywhere: https://github.com/zellyn/apple2shader/blob/master/screenEmu.js#L304

> That's fairly academic though; should we expect OpenEmulator in the browser any time soon?

I've thought about it, and I guess it probably is possible, using wasm… but I'm not aiming to do that. More likely to try to get other JS emulator authors interested in the NTSC shader, and/or port my (deficient but obviously dear to my heart) personal Apple II emulator code to the web once the Go compiler support for wasm lands.

Zellyn

[Michael 'AppleWin Debugger Dev']

Nice work! I'm an WebGL expert and am impressed with the fine level of controls. I've been meaning to throw Marc Ressl shader code into ShaderToy and reverse engineering for a while but keep getting distracted by other projects. This will definitely help in the that.

Looks like it takes a monochrome image ...
https://zellyn.github.io/apple2shader/images/airheart-560x192.png

... and runs the secret sauce (*) "COMPOSITE_SHADER" to turn it into a proper chroma/luma signal -> RGB.

(*) Technically the secret sauce is the 8 magic constants: c0, c1, c2, c3, c4, c5, c6, c7, c8

c += pixels(q, 1.0 / textureSize.x) * c1;
c += pixels(q, 2.0 / textureSize.x) * c2;
c += pixels(q, 3.0 / textureSize.x) * c3;
c += pixels(q, 4.0 / textureSize.x) * c4;
c += pixels(q, 5.0 / textureSize.x) * c5;
c += pixels(q, 6.0 / textureSize.x) * c6;
c += pixels(q, 7.0 / textureSize.x) * c7;
c += pixels(q, 8.0 / textureSize.x) * c8;

Interesting hack of using back-tick for multi-line code BTW!

Any chance you can allow us to pick a few different test images? While DHGR is interesting and all

i.e.
* Archon -- https://cloud.githubusercontent.com/assets/7876796/5497372/b7ccedce-86c5-11e4-820f-c90dcb72928a.png
* Lode Runner Logo -- https://cloud.githubusercontent.com/assets/22423271/19249449/a2122362-8f03-11e6-9dfa-e761cb8a3fad.png
* All 255 HGR byte values -- https://cloud.githubusercontent.com/assets/7876796/5530575/58664c54-89d6-11e4-95f0-6886b770b7e2.JPG


For a while now we've had accurate colors in various emulators. Unfortunately, they tend to blur the image out to the point of eye strain. i.e. JACE (Love the color accuracy, don't care the extreme blurriness. :-/ On AppleWin we have pretty good color accuracy but still have a tad of blurriness.)

The remaining challenge is to find a good compromise between sharpness, color accuracy, smoothness, and "solid colors" -- which are kind of contradictory goals -- but everyone has their own set of "preferences" for Authenticity vs Readability.

Again, nice work!

P.S.
You might want to change the title on Line 5 of index.html ;-)
"MDN Games: Shaders demo"

[Michael 'AppleWin Debugger Dev']

You can run this locally via web-scraping these 9 files -- nice!

https://zellyn.github.io/apple2shader/index.html
https://zellyn.github.io/apple2shader/screenEmu.js
https://zellyn.github.io/apple2shader/images/airheart-560x192.png
https://zellyn.github.io/apple2shader/textures/Shadow%20Mask%20Inline.png
https://zellyn.github.io/apple2shader/textures/Shadow%20Mask%20Triad.png
https://zellyn.github.io/apple2shader/textures/Shadow%20Mask%20Inline.png
https://zellyn.github.io/apple2shader/textures/Shadow%20Mask%20Aperture.png
https://zellyn.github.io/apple2shader/textures/Shadow%20Mask%20LCD.png
https://zellyn.github.io/apple2shader/textures/Shadow%20Mask%20Bayer.png

Web Server
python -m SimpleHTTPServer 8080

Local URL
http://localhost:8080/index.html

[Michael 'AppleWin Debugger Dev']

Where should we log bugs?

I've come across 2 (minor ones) so far:

shadowMaskDotPitch is const on line 1363 but modified on line 1366.
1363: const shadowMaskDotPitch = this.display.displayShadowMaskDotPitch;
1366: shadowMaskDotPitch = 0.001;

hue is const on line 993 but modified on line 1024.
993: const hue = 2 * Math.PI * this.display.videoHue;
1024: hue = -Math.PI * 33 / 180

Michael

[Thomas Harte]

On Wednesday, 16 May 2018 14:18:25 UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> Nice work! I'm an WebGL expert and am impressed with the fine level of controls. I've been meaning to throw Marc Ressl shader code into ShaderToy and reverse engineering for a while but keep getting distracted by other projects. This will definitely help in the that.
>
> Looks like it takes a monochrome image ...
> https://zellyn.github.io/apple2shader/images/airheart-560x192.png

Minor query: shouldn't the shader take a 561x192 image as input in order to be fully accurate? Wouldn't a single high-resolution image with a final column that had the 90-degree phase shift bit end up pushing a pixel out into what is ordinarily the right border?

> ... and runs the secret sauce (*) "COMPOSITE_SHADER" to turn it into a proper chroma/luma signal -> RGB.
>
> (*) Technically the secret sauce is the 8 magic constants: c0, c1, c2, c3, c4, c5, c6, c7, c8
>
> c += pixels(q, 1.0 / textureSize.x) * c1;
> c += pixels(q, 2.0 / textureSize.x) * c2;
> c += pixels(q, 3.0 / textureSize.x) * c3;
> c += pixels(q, 4.0 / textureSize.x) * c4;
> c += pixels(q, 5.0 / textureSize.x) * c5;
> c += pixels(q, 6.0 / textureSize.x) * c6;
> c += pixels(q, 7.0 / textureSize.x) * c7;
> c += pixels(q, 8.0 / textureSize.x) * c8;

So it's a FIR? Then quick observations:

(i) you can take advantage of the GPU's innate bilinear filtering to halve your number of reads by reading at a suitably-crafted location between each pixels to do a lot of the weighted sum for free.

E.g. take the lines:

c += pixels(q, 1.0 / textureSize.x) * c1;
c += pixels(q, 2.0 / textureSize.x) * c2;

Simplify notationally to c += v, where v = c1*v[1] + c2*v[2]. Then another way of getting exactly the same result is v = ((c1/(c1+c2))*v[1] + (c2/(c1+c2))*v[2]) * (c1+c2).

As (c1+c2)/(c1+c2) = 1, you can do the inner bit in a single sampling:

v = v[c2/(c1+c2)] * (c1+c2)

Because the rule for bilinear filtering is that the pixel at, say, 1.8 is the pixel at location 1 multiplied by 0.2 plus the pixel at location 2 multiplied by 0.8. It's just a linear mixing.

So in your case you'd end up reducing c1 to c8 to just c1 to c4, but the 1.0/ 2.0/ etc would now need to be uniforms rather than literals.

Also, always try to do any steps that apply a fixed offset to a varying in the vertex shader rather than the fragment shader because that allows most GPUs to queue up sampling results a lot more easily.

(ii) much more minor, consider reformulating as a dot product if you can rather than a manual accumulation. Just because some GLSL compilers aren't that intelligent but all GPUs are very good at vectors.

[Michael J. Mahon]

The blurriness is necessary for NTSC fidelity!

Use a smaller window.

Remember that there are only 140 color pixels (plus the interesting
transitions) per horizontal line. The luminance resolution is similar,
since the NTSC TVs Apple graphics were designed for filtered out higher
video frequencies.

Put another way, if it’s sharp, it’s not accurate.

--
-michael - NadaNet 3.1 and AppleCrate II: http://michaeljmahon.com

[Zellyn]

On Wednesday, May 16, 2018 at 2:18:25 PM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> Looks like it takes a monochrome image ...
Yep.

> ... and runs the secret sauce (*) "COMPOSITE_SHADER" to turn it into a proper chroma/luma signal -> RGB.
> (*) Technically the secret sauce is the 8 magic constants: c0, c1, c2, c3, c4, c5, c6, c7, c8

That plus the decoder matrix.

> Interesting hack of using back-tick for multi-line code BTW!

Recommended by https://webgl2fundamentals.org/ — not my idea :-)

> Any chance you can allow us to pick a few different test images? While DHGR is interesting and all
>
> i.e.
> * Archon -- https://cloud.githubusercontent.com/assets/7876796/5497372/b7ccedce-86c5-11e4-820f-c90dcb72928a.png
> * Lode Runner Logo -- https://cloud.githubusercontent.com/assets/22423271/19249449/a2122362-8f03-11e6-9dfa-e761cb8a3fad.png
> * All 255 HGR byte values -- https://cloud.githubusercontent.com/assets/7876796/5530575/58664c54-89d6-11e4-95f0-6886b770b7e2.JPG

Yeah, I plan on adding a menu of images, dhgr and hgr. Unfortunately, the cross-origin browser restrictions won't allow it to take arbitrary URLs, but I was thinking of allowing you to post data: urls into a textarea (they come out to about 5-10 KB)

> P.S.
> You might want to change the title on Line 5 of index.html ;-)
> "MDN Games: Shaders demo"

Oops! :-)

[Zellyn]

Well, sure, if you want to do it the hard way. Or you could just grab it directly from https://github.com/zellyn/apple2shader with a `git clone` command :-p

Zellyn

[Zellyn]

On Wednesday, May 16, 2018 at 2:37:43 PM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> Where should we log bugs?

Second choice: https://github.com/zellyn/apple2shader/issues
Better choice: https://github.com/zellyn/apple2shader/pulls
:-)

Oops. Thanks, fixed.

[Zellyn]

On Wednesday, May 16, 2018 at 3:50:25 PM UTC-4, Thomas Harte wrote:
> Minor query: shouldn't the shader take a 561x192 image as input in order to be fully accurate? Wouldn't a single high-resolution image with a final column that had the 90-degree phase shift bit end up pushing a pixel out into what is ordinarily the right border?

The screenData function actually renders the 560x192 image into a larger screen image. In the case of NTSC it's 755x240, and in the case of DHGR it gets copied in starting at the top left of (85, 19). For HGR, it would be (92, 19).

One of the funniest (to me) parts of implementing DHGR Apple IIe graphics in OpenEmulator was when out of expedience I just put the DGHR stuff to the left of the normal columns, and then realized it would make the whole screen shift left half a normal-width column, and *then* realized that my real IIe did the same thing. I actually spent quite a while sitting with my laptop and my real IIe down in the basement making sure the color tests in a2audit looked the same.

> So it's a FIR? Then quick observations:

I *think* so, but I don't know anything about DSP. The code comments in OpenEmulator point to https://www.dsprelated.com/showarticle/42.php

> (i) you can take advantage of the GPU's innate bilinear filtering to halve your number of reads by reading at a suitably-crafted location between each pixels to do a lot of the weighted sum for free.

Hmmm. I've read that a few times now but I'm losing you somewhere in the details. From what I can tell, it's simply that the chebyshev / lanczos windows are symmetric, so it's sending only half the coefficients, and using the pixels() function to read both behind and ahead of the current pixel, so those 1.0, 2.0, etc. are not weights but x-offsets. Or am I missing something? You are clearly way ahead of me; although this shader-golfing appeals to me, it's brand new :-)

> Also, always try to do any steps that apply a fixed offset to a varying in the vertex shader rather than the fragment shader because that allows most GPUs to queue up sampling results a lot more easily.

How would we do this? Generating the 16 before and behind x-offsets used in pixels() instead as 16 different varyings generated in the vertex shader?

> (ii) much more minor, consider reformulating as a dot product if you can rather than a manual accumulation. Just because some GLSL compilers aren't that intelligent but all GPUs are very good at vectors.

Heh. Yeah, as mentioned before, I'm intrigued enough with this shader optimization to have bookmarked https://twitter.com/erkaman2/status/991291243132080128 a while back, but unfortunately I'm not fluent with it at all!

Thanks for the insights and help.

Zellyn

[Zellyn]

On Wednesday, May 16, 2018 at 8:10:12 PM UTC-4, Michael J. Mahon wrote:
> The blurriness is necessary for NTSC fidelity!

> Put another way, if it’s sharp, it’s not accurate.

Yeah, one of the motivations for porting OpenEmulator code is that after playing with OpenEmulator for so long, I can't stand seeing sharp Apple II graphics. And weird color infidelities between DHGR and HGR/LORES combinations that should produce exactly the same colors. Running a2audit on Virtual ][ still makes me twitchy. Ironically, I don't think this work can help Virtual ][, because the OpenEmulator code is GPL, so it probably can't be used in a closed-source commercial program, and Marc Ressl is too absent to relicense it :-/

Zellyn

[Zellyn]

Thomas,

While I have your attention and obvious NTSC-decoding knowledge, I was curious about one thing in particular in the OpenEmulator NTSC rendering. It actually renders the monochrome input canvas to a decoded canvas of the same size. I was surprised by that. I expected the subtle fringing etc. to require more resolution than one output pixel per input pixel. I guess the scanlines, shadow mask, chroma- and luma-bandwidth blurring obscure it enough to smother the subtleties. I was curious on your opinions, though.

One other thing I was curious about: in your emulator, you're actually decoding the signal as a continuous input, right? Are you doing horizontal and vertical sync with front-porch etc. and color sync by locking to an emulator-produced reference color signal? Are you using the GPU for all that, or just pumping it through some kind of more realtime output filter? I wouldn't know how to go about using a GPU for a continuous signal: OpenEmulator definitely treats the input as a two-dimensional texture, even though the filtering is all horizontal.

Thanks again for your input,

Zellyn

[Thomas Harte]

On Thursday, 17 May 2018 09:06:47 UTC-4, Zellyn wrote:
> Thomas,
>
> While I have your attention and obvious NTSC-decoding knowledge, I was curious about one thing in particular in the OpenEmulator NTSC rendering. It actually renders the monochrome input canvas to a decoded canvas of the same size. I was surprised by that. I expected the subtle fringing etc. to require more resolution than one output pixel per input pixel. I guess the scanlines, shadow mask, chroma- and luma-bandwidth blurring obscure it enough to smother the subtleties. I was curious on your opinions, though.

I'm self-taught, over the last few years, so may nor may not be trustworthy. That disclaimer aside: four samples per colour cycle is by coincidence the recommended capture rate for digital preservation of analogue content.

If you'll accept the hand-waving explanation: Nyquist-Shannon says that fully to preserve the content of a signal of rate n Hz with discrete samples, you need to sample at a rate of 2n Hz. The colour part of a composite signal is actually two signals of n Hz added together though, one offset by 90 degrees from the other. So you need to sample two streams of 2n Hz, each in turn.

Which means sampling at 4n Hz. The rate the Apple produces pixels at.

> One other thing I was curious about: in your emulator, you're actually decoding the signal as a continuous input, right? Are you doing horizontal and vertical sync with front-porch etc. and color sync by locking to an emulator-produced reference color signal? Are you using the GPU for all that, or just pumping it through some kind of more realtime output filter? I wouldn't know how to go about using a GPU for a continuous signal: OpenEmulator definitely treats the input as a two-dimensional texture, even though the filtering is all horizontal.

This is an attempted summary of a much longer, boring version:

Machines produce a continuous input, but it's a series of segments. "I output sync for n units of time", "I output data for m units of time", etc. They're allowed just to say "I output a colour burst of amplitude X and phase Y". The rule is that if it's an accurate but more compact description of the real signal, it's acceptable. So the CRT has to classify syncs because there's only one sync level, but usually gets away with just looking at where it thought a colour burst would need to be and either seeing one already identified or seeing nothing. Which saves some time.

But otherwise, the CPU tracks syncs and bursts and so on, and posts to the GPU a series of runs: a start (x, y), an end (x, y), the colour subcarrier amplitude and phase at the start of that run or else the fact that no colour burst was detected, and a reference to the data that comprises the run.

In the Apple's case, the data is naturally enough 1-bit, seven pixels per byte, double high resolution. So the GPU converts that to full composite, decodes the colours, and paints between the specified coordinates.

I don't think you could helpfully use the GPU for more than that because it's a purely sequential process, rather than a particularly parallel one.

I also don't think you gain anything in terms of a whole frame from doing it this way for the Apple II versus just processing the whole frame already composed in 2d. It's really more useful for machines like the Atari 2600, ZX80/81, Amstrad CPC et al where the programmer has direct control over sync placement and timing.

[Zellyn]

On Thursday, May 17, 2018 at 12:12:44 PM UTC-4, Thomas Harte wrote:
> If you'll accept the hand-waving explanation: Nyquist-Shannon says that fully to preserve the content of a signal of rate n Hz with discrete samples, you need to sample at a rate of 2n Hz.

> Which means sampling at 4n Hz. The rate the Apple produces pixels at.

Seems legit. I'm a bit confused at it seeming to look _ahead_ as well as behind, because the CRT beam for sure can't do that, right?

> This is an attempted summary of a much longer, boring version:

> The rule is that if it's an accurate but more compact description of the real signal, it's acceptable.

That makes sense.

> I also don't think you gain anything in terms of a whole frame from doing it this way for the Apple II versus just processing the whole frame already composed in 2d. It's really more useful for machines like the Atari 2600, ZX80/81, Amstrad CPC et al where the programmer has direct control over sync placement and timing.

I see. I think your approach and that of the original OpenEmulator author are pretty agreeable -- it's a pity y'all didn't overlap in time in terms of emulator-writing activity :-)

Zellyn

[Thomas Harte]

On Thursday, 17 May 2018 12:20:24 UTC-4, Zellyn wrote:
> On Thursday, May 17, 2018 at 12:12:44 PM UTC-4, Thomas Harte wrote:
> > If you'll accept the hand-waving explanation: Nyquist-Shannon says that fully to preserve the content of a signal of rate n Hz with discrete samples, you need to sample at a rate of 2n Hz.
> > Which means sampling at 4n Hz. The rate the Apple produces pixels at.
> Seems legit. I'm a bit confused at it seeming to look _ahead_ as well as behind, because the CRT beam for sure can't do that, right?

No, a real CRT will just introduce a delay. I guess that for shader terms it's really only the difference between outputting the image in the same position or slightly to the right.

> I see. I think your approach and that of the original OpenEmulator author are pretty agreeable -- it's a pity y'all didn't overlap in time in terms of emulator-writing activity :-)

I went way too far with an earlier emulator — the state of the bus is described as a 64-bit integer. Components on the bus get messaged when the bus state changes. They announce their contribution to the bus. Then all the propagation logic has to do is combine all the contributions, check for changes and repeat. One component is the clock generator but it doesn't have elevated privileges.

So many components looked purely functional, and there was no bus owner.

But the actual net effect: an emulator that spends 90% of its time asking for bus contributions and performing calls and returns. So not scalable in any practical sense.

So now I've retreated to doing whatever is the minimum amount of signalling that doesn't actually drop any information, and the bus is something written in code for each platform, with template composition is used to glue bus interaction to components. Pragmatically, buses are entrusted to know enough about a machine to short-circuit interactions subject to the rule that machine specifics may not be built into components.

In practice most of the machines to date have ended up looking almost exactly like a single-machine emulator of old as they effectively piggyback off the processor bus interactions for counting and propagating time. It's not entirely true though, e.g. the Vic-20 with a C1540 is effectively a multiprocessor machine as a 6502 sits in the computer and another sits in the drive, and they share a directly-manipulable serial bus. So that utilises an explicit has-a relationship with a 6502 rather than looking like an is-a.

Without having looked closely at OpenEmulator, I think it sits between the two? A computer is constructed by declaratively combining components rather than programmatically, but messaging is something more efficient than a full accounting of all bus state changes?

[Zellyn]

On Thursday, May 17, 2018 at 4:33:59 PM UTC-4, Thomas Harte wrote:
> Without having looked closely at OpenEmulator, I think it sits between the two? A computer is constructed by declaratively combining components rather than programmatically, but messaging is something more efficient than a full accounting of all bus state changes?

Yeah, pretty much. It's a bit of a weird mix. The xml machine definitions make it look like everything is generic, and to some extent that's true, but many of the components are fairly custom to the computer being emulated. Since most everything on the Apple II communicates by memory address reads or writes, most of it boils down to mappers that dispatch reads and writes to the correct component. Other signals (corresponding to actual wires/connections) are connected to ports defined in the code and attached to each other in the XML.

Zellyn

[Steve Nickolas]

It might be better than my method of having all the modules watch all the
memory map.

-uso.

[Zellyn]

On Thursday, May 17, 2018 at 7:54:16 PM UTC-4, Steve Nickolas wrote:
> It might be better than my method of having all the modules watch all the
> memory map.

Possibly - it uses the AddressDecoder to divide a chunk of memory into blocks. Then each block can be mapped to something else. So you can subdivide the memory map, instead of having everything watch everything.

I took a start at documenting it here: https://github.com/zellyn/OpenEmulator/blob/develop/doc/components.md

Zellyn

[Zellyn]

On Wednesday, May 16, 2018 at 2:18:25 PM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> Any chance you can allow us to pick a few different test images? While DHGR is interesting and all

Done! Check out https://zellyn.github.io/apple2shader/ again. Suggestions welcome.

Zellyn

[Zellyn]

Added a few more suggested test images to https://zellyn.github.io/apple2shader/

The full list is now
- Airheart
- Battle Chess
- Archon
- Archon II
- Lode Runner
- Marty's Family Reader
- Bard's Tale
- DazzleDraw flower
- Rescue Raiders (a)
- Rescue Raiders (b)
- Ancient Legends


It was surprisingly difficult to get 560x192 black-and-white images out of either Virtual ][ or OpenEmulator, and I didn't feel like finishing the Language Card implementation for my own emulator to give it 64k. Luckily, Go's standard library contains everything you need to generate .png images from scratch, and Virtual ][ lets you dump memory ranges easily.

https://github.com/zellyn/apple2shader/blob/master/dumps/hgr.go

Zellyn

[Michael J. Mahon]

Zellyn <zel...@gmail.com> wrote:
> On Thursday, May 17, 2018 at 12:12:44 PM UTC-4, Thomas Harte wrote:
>> If you'll accept the hand-waving explanation: Nyquist-Shannon says that
>> fully to preserve the content of a signal of rate n Hz with discrete
>> samples, you need to sample at a rate of 2n Hz.
>> Which means sampling at 4n Hz. The rate the Apple produces pixels at.
> Seems legit. I'm a bit confused at it seeming to look _ahead_ as well as
> behind, because the CRT beam for sure can't do that, right?

The filters involved in separating the chroma subcarrier from the composite
signal introduce a delay. That delay is compensated for in the luminance
path, so the chroma info remains in sync with the luminance.

Since the chroma output is delayed by several pixels, the “future” does
influence the color of a pixel.

[Michael 'AppleWin Debugger Dev']

Thanks for allowing us to pick the image Zellyn!

Two other suggestions:

* Could you output the new value to the console when the slider changes? This would make it easier to compare screenshots and what values were used. (It isn't clear what the min/max values are.)
* Could you add the original Lode Runner logo screen? Championship Lode Runner is cool but I want to do an apple-to-apples comparison. ;-)

Thanks.

Now that I've taken a closer look at the graphics fidelity -- I have a couple of comments:

* Color accuracy is great!
* Tons of options! This is fantastic for tweaking!
* That Barrel Distortion setting is not practical but from an authenticity POV is pretty cool,
* Chroma Bandwidth needs to be < 25% for accuracy. Between 25% and 50% you get white ringing. Above 50% you get extreme white ringing.
* Luma Bandwidth needs to be > 50% fora accuracy. Best is 100% to minimize "chroma smear." At 0% you get extreme blurriness.

One of the things I was sensitive to when I integrated Sheldon's NTSC work into AppleWin was chroma fringes on white.

Here is a zoomed picture of Archon on real hardware.
https://cloud.githubusercontent.com/assets/7876796/5530971/59c8fff6-89db-11e4-8284-77ff4e1ff3d3.JPG

If we take a look at the black dragon's wing we see some chroma fringes where the black wing meets the white background on the right edges.

Reproducing (only) the (color) bytes we are interested in we can reproduce the chroma fridges on the leading edge of white (2 or 3 adjacent bits):

2000:70 (magenta)
2400:F0 (blue)
2800:60 (green)
2C00:E0 (orange)

I could of swore that when I tried this a few days ago I was only seen the magenta and green fringes.

Its great to see the proper chroma fringes being replicated 100%!

Today I'm seeing these "black holes" between adjacent white. :-(
https://raw.githubusercontent.com/Michaelangel007/apple2_hgrbyte/master/pics/archon_open_emulator_webgl_bug.png

It looks like the source didn't get converted properly?
https://zellyn.github.io/apple2shader/images/archon.png

For comparison this is what it looks like in AppleWin.
https://raw.githubusercontent.com/Michaelangel007/apple2_hgrbyte/master/pics/archon_applewin.png

Anyways the fact that this is running on the GPU is damn impressive!

Michael

[fadden]

On Thursday, May 17, 2018 at 9:12:44 AM UTC-7, Thomas Harte wrote:
> If you'll accept the hand-waving explanation: Nyquist-Shannon says that fully to preserve the content of a signal of rate n Hz with discrete samples, you need to sample at a rate of 2n Hz. The colour part of a composite signal is actually two signals of n Hz added together though, one offset by 90 degrees from the other. So you need to sample two streams of 2n Hz, each in turn.

If I have a signal at 1Hz, and I sample it at 1Hz, I could get the zero crossings at the start and end of the cycle (0, 2*PI).

If I have a signal at 1Hz, and I sample it at 2Hz, I could get the zero crossings at the start, middle, and end of the cycle (0, PI, 2*PI).

Sampling at twice the maximum frequency avoids aliasing, but does not ensure that you get an accurate picture of the signal.

[Zellyn]

On Sunday, May 20, 2018 at 10:36:22 AM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> * Could you output the new value to the console when the slider changes? This would make it easier to compare screenshots and what values were used. (It isn't clear what the min/max values are.)

I'll try. I was actually planning on making it update the URL parameters, but just haven't gotten around to it yet.

> * Could you add the original Lode Runner logo screen? Championship Lode Runner is cool but I want to do an apple-to-apples comparison. ;-)

Done.

> * That Barrel Distortion setting is not practical but from an authenticity POV is pretty cool,

Yeah. Turns out if you distort things as much as they actually were distorted, it looks terrible to people :-)
https://www.gamasutra.com/blogs/KylePittman/20150420/241442/CRT_Simulation_in_Super_Win_the_Game.php

> Today I'm seeing these "black holes" between adjacent white. :-(

Oops. I used a hastily-written converter of my own, which was failing to extend the last pixel of undelayed runs when the next run was delayed. Should be fixed now:
https://github.com/zellyn/apple2shader/blob/master/dumps/hgr.go

> Anyways the fact that this is running on the GPU is damn impressive!

As always, all credit goes to Marc Ressl, the author of OpenEmulator. I just ported it.

Zellyn

[Thomas Harte]

On Sunday, 20 May 2018 13:53:54 UTC-4, fadden wrote:
> If I have a signal at 1Hz, and I sample it at 1Hz, I could get the zero crossings at the start and end of the cycle (0, 2*PI).
>
> If I have a signal at 1Hz, and I sample it at 2Hz, I could get the zero crossings at the start, middle, and end of the cycle (0, PI, 2*PI).
>
> Sampling at twice the maximum frequency avoids aliasing, but does not ensure that you get an accurate picture of the signal.

On the contrary, sampling any continuous signal at twice its bandwidth guarantees no loss of information whatsoever. It's one of the fundamental pieces of sampling theory.

In your example, the facts you're failing to account for are:

1) you have additional knowledge about the true signal that the samples came from: it was a sine wave;
2) you have additional knowledge about the true signal that the samples came from: you have many other samples.

NTSC decoding exploits both of these facts: (2) is used to separate luminance and chrominance. E.g. the posted OpenEmulator code uses sixteen input samples to determine one output. (1) is used by every NTSC decoder in existence; both colour signals are individual amplitude modulations, ninety degrees out of phase (hence 'quadrature amplitude modulation': the two signals are in quadrature, i.e. orthogonally out of phase), and separation of those even after you've isolated the chrominance part of the signal relies counterintuitively on amplitude modulating again and then filtering out the high frequencies. That works because a*sin^2(t) = a * sin(t) * sin(t) = (a/2) * (cos(t - t)- cos(t + 2)) = (1/2)*a + [a bunch of stuff at twice the carrier frequency; so you can just filter this out], and check out the equivalent trig identities for sin*cos and cos*cos to see how the two things, even though combined by simple addition, remain separable as long as phase is known.

[Michael 'AppleWin Debugger Dev']

On Wednesday, May 16, 2018 at 5:10:12 PM UTC-7, Michael J. Mahon wrote:
> Michael 'AppleWin Debugger Dev' wrote:

> > For a while now we've had accurate colors in various emulators.
> > Unfortunately, they tend to blur the image out to the point of eye
> > strain. i.e. JACE (Love the color accuracy, don't care the extreme
> > blurriness. :-/ On AppleWin we have pretty good color accuracy but still
> > have a tad of blurriness.)
> >
> > The remaining challenge is to find a good compromise between sharpness,
> > color accuracy, smoothness, and "solid colors" -- which are kind of
> > contradictory goals -- but everyone has their own set of "preferences"
> > for Authenticity vs Readability.
> >
>

> The blurriness is necessary for NTSC fidelity!

Right. You and I both know that but not everyone does, or even likes it.

> Use a smaller window.

Well, the problem is due to everyone having implemented a broken NTSC converter first and for so long -- with only recently (in the last ~2 years) have we gotten correct implementations -- that users have gotten used to the "wrong" approach and get upset when we do things correctly. You can't please everyone. :-/

i.e.
https://github.com/AppleWin/AppleWin/issues/357

The contradictory nature of the problem doesn't help.

[Thomas Harte]

On Monday, 21 May 2018 08:29:10 UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> On Wednesday, May 16, 2018 at 5:10:12 PM UTC-7, Michael J. Mahon wrote:
> > The blurriness is necessary for NTSC fidelity!
>
> Right. You and I both know that but not everyone does, or even likes it.

I guess you could artificially filter down to relatively-sharp pixels, in a way unlike any TV set or monitor ever, by applying a post-conversion filter that rounds every pixel to the closest of those formally associated with the Apple II? So take the idiomatic low-resolution graphics mode 16, and snap?

It'd be a context-free per-pixel operation, so amenable to GPU implementation as an after-the fact extra step.

Very, very fictional though. Just to appease those looking for an emulator to reproduce the experience of an earlier emulator.

Out of interest, what does the GS do to avoid NTSC? It strikes me that you could do a full NTSC decode using just a six-bit lookup table — a rolling four bits of graphics data plus a two-bit counter for phase — is it anything like that?

[Michael 'AppleWin Debugger Dev']

On Sunday, May 20, 2018 at 5:30:34 PM UTC-7, Zellyn wrote:
> On Sunday, May 20, 2018 at 10:36:22 AM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> > * Could you output the new value to the console when the slider changes? This would make it easier to compare screenshots and what values were used. (It isn't clear what the min/max values are.)
> I'll try. I was actually planning on making it update the URL parameters, but just haven't gotten around to it yet.

No prob. The URL update would be very cool to allow easily sharing of settings.

>
> > * Could you add the original Lode Runner logo screen? Championship Lode Runner is cool but I want to do an apple-to-apples comparison. ;-)
> Done.

Thanks!

Hmm, I don't think I've ever seen those credits at the bottom before:
Aldo Reset, CCB, Laurent Rueil

> > * That Barrel Distortion setting is not practical but from an authenticity POV is pretty cool,
> Yeah. Turns out if you distort things as much as they actually were distorted, it looks terrible to people :-)
> https://www.gamasutra.com/blogs/KylePittman/20150420/241442/CRT_Simulation_in_Super_Win_the_Game.php

Oooh, that's great read!

> > Today I'm seeing these "black holes" between adjacent white. :-(
> Oops. I used a hastily-written converter of my own, which was failing to extend the last pixel of undelayed runs when the next run was delayed. Should be fixed now:
> https://github.com/zellyn/apple2shader/blob/master/dumps/hgr.go

Thanks. Looks great now!

> > Anyways the fact that this is running on the GPU is damn impressive!
> As always, all credit goes to Marc Ressl, the author of OpenEmulator. I just ported it.

Does anyone know if Marc ever wrote up notes on how he implemented his shader? I mean we can read the shader code but what it is doing isn't always obvious.

Michael

[Zellyn]

On Monday, May 21, 2018 at 9:44:21 AM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> > > * Could you add the original Lode Runner logo screen? Championship Lode Runner is cool but I want to do an apple-to-apples comparison. ;-)
> > Done.
>
> Thanks!
>
> Hmm, I don't think I've ever seen those credits at the bottom before:
> Aldo Reset, CCB, Laurent Rueil

Ugh. I bet I got some cracker nonsense mixed in. Point me at a .dsk and I'll turn it into an image.

Zellyn

[Antoine Vignau]

Aldo Reset and Laurent Rueil are French crackers from the Clean Crack Band.

My clean crack @ http://www.brutaldeluxe.fr/crack/LODERUNNER.DSK

av

[fadden]

On Sunday, May 20, 2018 at 6:11:47 PM UTC-7, Thomas Harte wrote:
> On the contrary, sampling any continuous signal at twice its bandwidth guarantees no loss of information whatsoever. It's one of the fundamental pieces of sampling theory.

Hmm. I think we may be talking past each other.

If you're talking about generating samples *from* a signal, i.e. receiving an input and taking periodic samples, then sampling at 2x only guarantees that you won't create aliasing effects. As noted in my example, you can sample a 1Hz test tone at 2Hz and see nothing but zero amplitude.

If you're talking about generating samples *for* a signal, you can fully represent the signal with samples at 2x. Taking the 1Hz test tone example, if you take your first sample at t=0.25 (quarter phase), you'll see the max amplitude (alternating +/-), and can accurately generate the output.

I think you're actually talking about the latter, but I read it as the former.

[Thomas Harte]

On Monday, 21 May 2018 13:17:16 UTC-4, fadden wrote:
> On Sunday, May 20, 2018 at 6:11:47 PM UTC-7, Thomas Harte wrote:
> > On the contrary, sampling any continuous signal at twice its bandwidth guarantees no loss of information whatsoever. It's one of the fundamental pieces of sampling theory.
>
> Hmm. I think we may be talking past each other.

Always a risk with me. Please accept both retrospective and proactive apologies.

> If you're talking about generating samples *from* a signal, i.e. receiving an input and taking periodic samples, then sampling at 2x only guarantees that you won't create aliasing effects. As noted in my example, you can sample a 1Hz test tone at 2Hz and see nothing but zero amplitude.
>
> If you're talking about generating samples *for* a signal, you can fully represent the signal with samples at 2x. Taking the 1Hz test tone example, if you take your first sample at t=0.25 (quarter phase), you'll see the max amplitude (alternating +/-), and can accurately generate the output.
>
> I think you're actually talking about the latter, but I read it as the former.

Got it. You're pointing out that sampling at 2x means not introducing any additional information, whereas I'm arguing that it means not losing it.

I think you're right, and I've improperly conflated frequency and bandwidth in my attempt to reach a compelling hand-waving explanation.

The bandwidth of an AM modulation at x Hz is 2x Hz. Therefore the Nyquist-required sampling is 2*2x = 4x Hz.

So I guess the proper hand-waving explanation is to make that point, then jump to trig identities to explain why the QAM is separable. So your single 4x stream is sufficient for both.

[Michael J. Mahon]

Absolutely. The Nyquist frequency is the *minimum* sampling frequency to
avoid aliasing, but, as you point out, if sampling is synchronous with the
signal, information can be lost.

The requirement that the reconstruction filter be realizable generally
calls for at least a 10% higher than Nyquist sampling frequency.

[Michael J. Mahon]

Zellyn <zel...@gmail.com> wrote:
> On Sunday, May 20, 2018 at 10:36:22 AM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
>> * Could you output the new value to the console when the slider changes?
>> This would make it easier to compare screenshots and what values were
>> used. (It isn't clear what the min/max values are.)
> I'll try. I was actually planning on making it update the URL parameters,
> but just haven't gotten around to it yet.
>
>> * Could you add the original Lode Runner logo screen? Championship Lode
>> Runner is cool but I want to do an apple-to-apples comparison. ;-)
> Done.
>
>> * That Barrel Distortion setting is not practical but from an
>> authenticity POV is pretty cool,
> Yeah. Turns out if you distort things as much as they actually were
> distorted, it looks terrible to people :-)

I used a Panasonic 10” color TV for color graphics and a Zenith 12”
monochrome monitor for text and monochrome graphics, and I never saw
perceptible barrel distortion.

Only truly garbage color TVs had significant barrel distortion, since color
convergence requirements lead to better high voltage regulation (a
fundamental cause of barrel distortion). The other cause was poor electron
optics—again, not a problem for any reasonable quality color TV or monitor.


Monochrome monitors could be induced to cause something similar to barrel
distortion by displaying a bright range of lines surrounded by dark lines,
but for reasonable brightness ranges, the effect was seldom distracting.

[Michael 'AppleWin Debugger Dev']

On Tuesday, May 22, 2018 at 12:42:50 AM UTC-6, Michael J. Mahon wrote:

Thanks for the clarification Michael. I had always wondered if the barrel distortion was just a Fact-of-Life (TM) of CRTs or limited to crappy displays.

I was never a fan of it, but I guess some people might prefer the "Nostalgia".

Michael.

[Nick Westgate]

On Tuesday, 22 May 2018 01:14:17 UTC+12, Thomas Harte wrote:
> On Monday, 21 May 2018 08:29:10 UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> > On Wednesday, May 16, 2018 at 5:10:12 PM UTC-7, Michael J. Mahon wrote:
> > > The blurriness is necessary for NTSC fidelity!
> >
> > Right. You and I both know that but not everyone does, or even likes it.

As a reminder, I did point out in that AppleWin issue that those of us with RGB cards or a IIgs had (relative) crispness.

Fantastic effort Zellyn, and great thread guys.

Cheers,
Nick.

[Ernie Soffronoff]

On Thursday, May 24, 2018 at 2:42:44 AM UTC-4, Nick Westgate wrote:
>
> Fantastic effort Zellyn, and great thread guys.
>
> Cheers,
> Nick.

Agreed! Thanks to everyone for sharing their hard-won experience.

[Michael 'AppleWin Debugger Dev']

On Monday, May 21, 2018 at 9:49:14 AM UTC-6, Zellyn wrote:
> Ugh. I bet I got some cracker nonsense mixed in. Point me at a .dsk and I'll turn it into an image.

GitHub Pull Request sent your way that fixes this and also adds U5. :-)

Michael

[Michael 'AppleWin Debugger Dev']

Michael

Where does the 10% higher sampling rate come from? Is this a Rule-of-Thumb or something else?

In the graphics world W.R.T. antialiasing I’m not sure what percentage is most common but it’s not rare to see 2x, 4x, or even 8x oversampling for (image) quality. The canonical checkerboard at a 90 degrees plane to camera is the worst case. Granted we can calculate partial derivatives in screen-space to minimize “noise” so we can “cheat” from taking a full blown oversampling performance hit.

Michael

[Zellyn]

On Thursday, May 24, 2018 at 10:34:48 AM UTC-4, Michael 'AppleWin Debugger Dev' wrote:
> GitHub Pull Request sent your way that fixes this and also adds U5. :-)
>
> Michael

Thanks! Merged, and should be live :-)

Zellyn

[Thomas Harte]

I can't speak confidently as to the 10% since I'm still arguing for four times based on the mathematics, i.e. amplitude modulation has twice the bandwidth of its carrier frequency, and sampling at twice the bandwidth is the minimum preserves all information. But as before, I'm still on this from the preserving all information angle, not the introducing-no-false information angle. Sampling at a rate of 0 Hz would also produce no aliasing, but it would not preserve all information. So that's a bit of an askew comment.

Re: antialiasing, the graphics world isn't even that cut-and-dried: multisampling is often used rather than supersampling, which means that some things are sampled multiple times for an output pixel not everything is. Usually your fragment shader is run only exactly once, but the work on the underlying geometry happens multiple times. So it's good for edges, irrelevant for fills. You're supposed to have done appropriate things with your mip maps and/or do appropriate anisotropic work to generate a single good colour.

So you've eliminate a lot of the higher frequencies that might cause aliasing in advance; indeed the issue that motivates people to use anisotropic filtering rather than trilinear filtering isn't that the latter introduces too much aliasing, it's that it doesn't preserve enough of the original signal.

[Michael J. Mahon]

Michael 'AppleWin Debugger Dev' <michael....@gmail.com> wrote:
> Michael
>
> Where does the 10% higher sampling rate come from? Is this a
> Rule-of-Thumb or something else?

It’s in the “rule of thumb” category.

Since “brick wall” filters are unrealizable, and a reconstruction filter is
required after any DAC, one can only get as close to the Nyquist
theoretical bandwidth as a practical reconstruction (lowpass) filter can
allow.

Typically, a practical multipole analog lowpass filter can provide 30dB+
attenuation at the theoretical cutoff frequency while still providing just
3dB of attenuation for frequencies 10% below the cutoff frequency.

> In the graphics world W.R.T. antialiasing I’m not sure what percentage is
> most common but it’s not rare to see 2x, 4x, or even 8x oversampling for
> (image) quality. The canonical checkerboard at a 90 degrees plane to
> camera is the worst case. Granted we can calculate partial derivatives in
> screen-space to minimize “noise” so we can “cheat” from taking a full
> blown oversampling performance hit.

The same advantages accrue to audio oversampling, but samples take both
space and time, so usually you do the best you can with as little
oversampling as possible.

BTW, we’re talking about the “playback” side of this issue here, but the
“record” side has a similar requirement to prevent aliasing. The
antialiasing lowpass filter needs to meet a similar requirement: minimal
attenuation in the desired passband and 30dB or more attenuation at and
above half the sampling frequency. Of course, if the maximum signal
frequency is considerably lower than half the sampling frequency then the
filter is much easier to implement (or, on the other hand, the sampling
frequency can be reduced).

> Michael

[Zellyn]

FYI, I finally got tired of not understanding DSP, and found out Coursera has a pretty decent-looking class on it, if anyone on this thread is interested: https://www.coursera.org/learn/dsp

It's free if you don't care about certification nonsense.

Zellyn

[fadden]

On Friday, May 25, 2018 at 6:27:56 AM UTC-7, Zellyn wrote:
> FYI, I finally got tired of not understanding DSP, and found out Coursera has a pretty decent-looking class on it, if anyone on this thread is interested: https://www.coursera.org/learn/dsp

If you prefer the classic dead-tree approach to learning, Lyons' _Understanding Digital Signal Processing_ is an excellent choice. https://www.amazon.com/gp/product/0137027419