Easy Cross-browser Transparency
Janet Denzel
This function just moves through the data looking for where there is no red or blue and where green is larger than some floor value gFloor. Since sometimes the red r and blue b values are populated, we also need to set a ceiling value for these. Anywhere the conditions are true it simply sets the alpha to 0. While most pixels are RGB(0,154,0, 254), with some experimentation, I found setting gFloor to somewhere around 105 and rbCeiling to 80 gave the best general results to remove any artifacts from the segmentation and WebRTC encode/decode processes.
The peer connection encoder changing the stream alpha pixels to seemingly different values had me worried. (ToDo: check the WebRTC encoder code to see how this works). In general, the alpha channel images sent over the peer connected ended up with some green halo artifacts around the segmentation. In one of my initial prototypes, I added a green background to the source image and sent this over the peer connection. It seems like this method is more reliable.
WebGL is supposed to be fast. However, WebGL is not at all like working with the HTML canvas and looks like a beast to learn. Fortunately, James agreed to let me include his code in my sample! It was easy to copy/paste his functions without understanding how they work.
My playground example is helpful for quickly comparing methods. A real application would need to be much more streamlined. WebRTC apps usually show some kind of self-view and would need to add transparency to any incoming streams shown. In addition, most WebRTC apps are designed around elements, not canvases. Therefore, I wanted to use MediaStream inputs and outputs and not rely on visible canvas writes.
So, I made a second, consolidated sample here: transparency.html. This one is fully contained in a single file. It only shows a single transparent element for the sender and one for the receiver. Segmentation and transparency are converted to MediaStreams using Insertable Streams.
The runs the Frame through the segmentation. I take that image and write it to an OffscreenCanvas. Then I use the passed controller to enqueue the processed frame from that canvas where it is added to the segmentStream from the generator.
I had to add an addGreenScreen function with 40 lines of code including its ownMediaStreamTrackProcessor, TransformStream, and TransformStream. After the segmentation process, the stream encoding that takes place in the generator should be the next most expensive operation.
To help with these tests I added the option to run the segmentation at different resolutions. Higher resolutions provide a lot more bits to process, so they are one way to see how efficient the pipeline is running.
This one is easier to measure. I added an frames per second (FPS) meter using stats.js from mrdoob and checked the CPU usage in the browser task manager. The results below are just for my MacBook Pro 2018, 2.9 GHz 6-Core Intel Core i9 running Google Canary (currently 97.0.4687.0) using a Logitech Brio 4K Webcam. These numbers jump around a bit, even running the same parameters from test-to-test, so treat them as just one directional indicator. They were generally in the same ballpark from test to test.
I am curious how well this works across different devices, including mobile web. I tried it quickly and it worked, but the frame rates were well below my MacBook. If you try it, please let me know how it works in the comments.
Great article, with regards to performance, would, using webassembly help on some of the intensive parts of your code. I have only just started to look into wa, so could not offer a possible opinion here, but many who have, claimed that they are seeing anything from 3 to 9 times faster executions.
In the context of global climate change policy and actions, transparency equals trust. This report examines how developing countries are establishing this trust by sharing their successes, challenges, and lessons learned.
Building on the valuable results of the UNFCCC's Consultative Group of Experts of the Subsidiary Body for Implementation, this report highlights the best practices and support needs of 24 countries out of 126 where UNDP-managed projects are addressing enhanced transparency activities in Africa, Asia & Pacific, Europe, Latin America, and the Middle East. The report presents actions within the ETF themes of GHG inventory, mitigation progress, adaptation progress, support needed & received, and cross-cutting issues, making it easy to follow these leading activities and future needs.
The report can serve as a useful guide to identify opportunities for South-to-South knowledge sharing and collaboration to address the Enhanced Transparency Framework. It highlights more than 200 activities related to the ETF that countries have experience in addressing. It also provides detailed information on 23 specific solutions used by countries to tackle a key challenge they faced. Lastly, development partners can use this report to identify areas where targeted ETF support is needed. It identifies over 60 common areas where future support is necessary to enhance transparency.
And not just in politics. If health care reform ever emerges from Congress, it is certain to spread nationally a project to require doctors to reveal to an Internet-linked database any financial interests they may have in any drug company or device manufacturer. Type the name of any doctor into the database, and a long list of consulting contracts, stock ownership, and paid speaking arrangements will be returned to you, presumably to help you avoid doctors with too many conflicting loyalties, and to steer you to doctors who have themselves steered clear of conflicts.
The naked transparency movement marries the power of network technology to the radical decline in the cost of collecting, storing, and distributing data. Its aim is to liberate that data, especially government data, so as to enable the public to process it and understand it better, or at least differently.
Without a doubt, the vast majority of these transparency projects make sense. In particular, management transparency, which is designed to make the performance of government agencies more measurable, will radically improve how government works. And making government data available for others to build upon has historically produced enormous value--from weather data, which produces more than $800 billion in economic value to the United States, to GPS data, liberated originally by Ronald Reagan, which now allows cell phones to instantly report (among other essential facts) whether Peets or Starbucks is closer.
But that is not the whole transparency story. There is a type of transparency project that should raise more questions than it has--in particular, projects that are intended to reveal potentially improper influence, or outright corruption. Projects such as the one that the health care bill would launch--building a massive database of doctors who got money from private interests; or projects such as the ones (these are the really sexy innovations for the movement) to make it trivially easy to track every possible source of influence on a member of Congress, mapped against every single vote that the member has made. These projects assume that they are seeking an obvious good. No doubt they will have a profound effect. But will the effect of these projects--at least on their own, unqualified or unrestrained by other considerations--really be for the good? Do we really want the world that they righteously envisage?
This is a crude but powerful beginning. It points to an obvious future. Even this clarity took an enormous effort to produce, and there are obviously a million other ways in which the data might be inspired to speak. As Congress complies with the clear demands of transparency, and as coders devise better and more efficient ways to mash-up the data that Congress provides, we will see a future more and more inundated with claims about the links between money and results. Every step will have a plausible tie to troubling influence. Every tie will be reported. We will know everything there is to know about at least the publicly recordable events that might be influencing those who regulate us. The panopticon will have been turned upon the rulers.
What could possibly be wrong with such civic omniscience? How could any democracy live without it? Finally America can really know just who squeezed the sausage and when, and hold accountable anyone with an improper touch. Imagine how much Brandeis, the lover of sunlight, would have loved a server rack crunching terabytes of data. As a political disinfectant, silicon beats sunlight hands down.
There is little doubt that the answer to each of these questions is, in some sense and at some time--remember those qualifiers!--yes. In a series titled Speaking Freely, published by the Center for Responsive Politics, you can find testimony from many former members from both parties to support each of those assertions. Everyone inside the system knows that claims about influence are, to some degree, true. It is the nature of the system, as we all know.
But there is also little doubt that it is impossible to know whether any particular contribution or contributions brought about a particular vote, or was inspired by a particular vote. Put differently, if there are benign as well as malign contributions, it is impossible to know for any particular contribution which of the two it is. Even if we had all the data in the world and a month of Google coders, we could not begin to sort corrupting contributions from innocent contributions.
The point is salience, and the assumptions of our political culture. At this time the judgment that Washington is all about money is so wide and so deep that among all the possible reasons to explain something puzzling, money is the first, and most likely the last, explanation that will be given. It sets the default against which anything different must fight. And this default, this unexamined assumption of causality, will only be reinforced by the naked transparency movement and its correlations. What we believe will be confirmed, again and again.
bcf7231420