Selecting video inputs / audio inputs/outputs on publisher
97 views
Skip to first unread message
Robert Berger
May 22, 2020, 4:07:00 AM5/22/20
to Ant Media Server
How can I select the various video inputs / audio inputs/outputs on publisher? There doesn't seem to be a way to set any of these via the WebRTCAdapter other than setting the constraints when initiating the adapter. Seems kind of fundamental. I know you folks are working on the ability to change the video input, but don't see any effort for audio input/output.
mahmut asım Onat
May 22, 2020, 6:37:24 PM5/22/20
to Ant Media Server
Hi Robert,
They have in fact completed writing the method for changing the video input, you can get the latest webrtc_adaptor.js file from here if you haven't:
https://github.com/ant-media/StreamApp/blob/master/src/main/webapp/js/webrtc_adaptor.js
As for audio input, I know that it is going to be developed soon.
Best,
Asim
They have in fact completed writing the method for changing the video input, you can get the latest webrtc_adaptor.js file from here if you haven't:
https://github.com/ant-media/StreamApp/blob/master/src/main/webapp/js/webrtc_adaptor.js
As for audio input, I know that it is going to be developed soon.
Best,
Asim
Robert Berger
May 22, 2020, 8:21:56 PM5/22/20
to Ant Media Server, mahmut asım Onat
Is there a way to update the server image Im running from the Amazon Marketplace to use this? Not sure if there are other changes in the StreamApp that needs to be put on the server.
Thanks
Rob
Thanks
Rob
15550 Wildcat Rdg. Saratoga, CA
--
You received this message because you are subscribed to the Google Groups "Ant Media Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ant-media-serv...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ant-media-server/a2bcb8ae-66e9-4242-aa8b-c64445c24bfb%40googlegroups.com.
Robert Berger
May 23, 2020, 2:53:52 AM5/23/20
to Ant Media Server
I just downloaded that file for my app for now. And I can now switch video sources. Any hints on how to switch the audio inputs would be appreciated.
Robert Berger
May 24, 2020, 2:40:18 AM5/24/20
to Ant Media Server
So actually if I use https://github.com/ant-media/StreamApp/blob/master/src/main/webapp/js/webrtc_adaptor.js, I can switch the video inputs, but I can no longer start a stream. I get an error. If I put the old webrtc_adapter.js back in, it works fine.
Is the new webrtc_adapter.js plug in compatible with previous clients and the current Enterprise server that is deployed with the AWS Marketplace? Or do I need to do something on ether end to make things work?
On the client side in Devtools I see:
react_devtools_backend.js:6 No event.candidate in the iceCandidate event
r @ react_devtools_backend.js:6
WebRTCAdaptor.iceCandidateReceived @ webrtc_adaptor.js:760
thiz.remotePeerConnection.<computed>.onicecandidate @ webrtc_adaptor.js:815
webrtc_adaptor.js:1350 received remote description type for stream id: test1 type: answer
react_devtools_backend.js:6 set remote description is failed with error: TypeError: Cannot read property 'candidate' of undefined
r @ react_devtools_backend.js:6
(anonymous) @ webrtc_adaptor.js:1044
Promise.catch (async)
WebRTCAdaptor.takeConfiguration @ webrtc_adaptor.js:1042
WebSocketAdaptor.wsConn.onmessage @ webrtc_adaptor.js:1352
webrtc_adaptor.js:1091 Uncaught TypeError: Cannot read property 'protocol' of undefined
at WebRTCAdaptor.addIceCandidate (webrtc_adaptor.js:1091)
at WebRTCAdaptor.takeCandidate (webrtc_adaptor.js:1072)
at WebSocketAdaptor.wsConn.onmessage (webrtc_adaptor.js:1345)
On the server side I see:
2020-05-24 06:35:59,953 [http-nio2-0.0.0.0-5080-exec-10] INFO i.a.e.s.WebSocketLocalHandler - Web Socket opened
2020-05-24 06:36:01,537 [vert.x-eventloop-thread-2] INFO i.antmedia.statistic.StatsCollector - -Heartbeat-> System cpu load: 0 Free memory: 958356 KB
2020-05-24 06:36:08,475 [http-nio2-0.0.0.0-5080-exec-8] INFO i.a.e.w.WebSocketEnterpriseHandler - received token:null
2020-05-24 06:36:08,484 [http-nio2-0.0.0.0-5080-exec-8] INFO i.a.e.w.WebSocketEnterpriseHandler - Adding new stream test1 to session b8
2020-05-24 06:36:08,485 [http-nio2-0.0.0.0-5080-exec-8] INFO i.a.e.adaptive.EncoderAdaptor - ServerSettings exist io.antmedia.settings.ServerSettings@74f680f4
2020-05-24 06:36:08,485 [webrtc-publish-test1-0] INFO org.webrtc.Logging - Tag:NativeLibrary - Message:Native library has already been loaded.
2020-05-24 06:36:08,485 [webrtc-publish-test1-0] INFO org.webrtc.Logging - Tag:PeerConnectionFactory - Message:PeerConnectionFactory was initialized without an injected Loggable. Any existing Loggable will be deleted.
2020-05-24 06:36:08,486 [webrtc-publish-test1-0] INFO org.webrtc.Logging - Tag:JavaAudioDeviceModule - Message:createAudioDeviceModule
2020-05-24 06:36:08,486 [webrtc-publish-test1-0] INFO org.webrtc.Logging - Tag:JavaAudioDeviceModule - Message:HW NS will not be used.
2020-05-24 06:36:08,486 [webrtc-publish-test1-0] INFO org.webrtc.Logging - Tag:JavaAudioDeviceModule - Message:HW AEC will not be used.
2020-05-24 06:36:08,807 [http-nio2-0.0.0.0-5080-exec-6] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - before prepare for test1 hash: 451159804
2020-05-24 06:36:08,808 [http-nio2-0.0.0.0-5080-exec-6] INFO io.antmedia.muxer.HLSMuxer - hls time: 2, hls list size: 5
2020-05-24 06:36:08,808 [http-nio2-0.0.0.0-5080-exec-6] INFO i.a.e.adaptive.StreamAdaptor - Stream adaptor scheduled job id 688 stream adaptor:1650673973 for stream:test1
2020-05-24 06:36:08,809 [http-nio2-0.0.0.0-5080-exec-6] INFO i.a.e.adaptive.StreamAdaptor - Stream adaptor scheduled job id 689 stream adaptor:1527637003 for stream:test1
2020-05-24 06:36:08,809 [http-nio2-0.0.0.0-5080-exec-6] INFO i.a.enterprise.adaptive.OpusEncoder - encoder name libopus source sample rate 44100, channel layout 3
2020-05-24 06:36:08,809 [http-nio2-0.0.0.0-5080-exec-6] INFO i.a.enterprise.adaptive.AACEncoder - name : aac
2020-05-24 06:36:08,810 [http-nio2-0.0.0.0-5080-exec-6] INFO io.antmedia.muxer.HLSMuxer - copy codec parameter from context 0 stream index: 0
2020-05-24 06:36:08,810 [http-nio2-0.0.0.0-5080-exec-6] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - after prepare for test1 hash:451159804
2020-05-24 06:36:08,811 [http-nio2-0.0.0.0-5080-exec-6] WARN i.a.e.adaptive.WebRTCEncoderAdaptor - Ice candidate will be added later for stream Id test1. Remote Description not set
2020-05-24 06:36:08,811 [http-nio2-0.0.0.0-5080-exec-6] WARN i.a.e.adaptive.WebRTCEncoderAdaptor - Ice candidate will be added later for stream Id test1. Remote Description not set
2020-05-24 06:36:08,811 [http-nio2-0.0.0.0-5080-exec-6] WARN i.a.e.adaptive.WebRTCEncoderAdaptor - Ice candidate will be added later for stream Id test1. Remote Description not set
2020-05-24 06:36:08,811 [http-nio2-0.0.0.0-5080-exec-6] WARN i.a.e.adaptive.WebRTCEncoderAdaptor - Ice candidate will be added later for stream Id test1. Remote Description not set
2020-05-24 06:36:08,811 [http-nio2-0.0.0.0-5080-exec-6] WARN i.a.e.adaptive.WebRTCEncoderAdaptor - Ice candidate will be added later for stream Id test1. Remote Description not set
2020-05-24 06:36:08,811 [http-nio2-0.0.0.0-5080-exec-6] WARN i.a.e.adaptive.WebRTCEncoderAdaptor - Ice candidate will be added later for stream Id test1. Remote Description not set
2020-05-24 06:36:08,812 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onAddTrack for streamId test1
2020-05-24 06:36:08,812 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onAddTrack for streamId test1
2020-05-24 06:36:08,812 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onAddStream for streamId test1
2020-05-24 06:36:08,813 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onSetSuccess signalling thread for test1 ice candidate queue size: 6
2020-05-24 06:36:08,813 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onSetSuccess iceCandidateQueue for test1
2020-05-24 06:36:08,813 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onSetSuccess iceCandidateQueue for test1
2020-05-24 06:36:08,813 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onSetSuccess iceCandidateQueue for test1
2020-05-24 06:36:08,813 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onSetSuccess iceCandidateQueue for test1
2020-05-24 06:36:08,813 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onSetSuccess iceCandidateQueue for test1
2020-05-24 06:36:08,813 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onSetSuccess iceCandidateQueue for test1
2020-05-24 06:36:08,814 [network_thread - 20265] INFO org.webrtc.Logging - Tag:WebRtcAudioTrackExternal - Message:initPlayout(sampleRate=16000, channels=1, bufferSizeFactor=1.0)
2020-05-24 06:36:08,814 [network_thread - 20265] INFO org.webrtc.Logging - Tag:WebRtcAudioTrackExternal - Message:byteBuffer.capacity: 320
2020-05-24 06:36:08,814 [network_thread - 20265] INFO org.webrtc.Logging - Tag:WebRtcAudioTrackExternal - Message:startPlayout(sampleRate=16000, channels=1)
2020-05-24 06:36:08,814 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - starting playout for stream test1
2020-05-24 06:36:08,816 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onIceConnectionChange CHECKING for streamId: test1 hash:451159804
2020-05-24 06:36:08,817 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onIceCandidate: 0:0:candidate:1696137277 1 udp 2122260223 10.31.254.120 53480 typ host generation 0 ufrag dQlG network-id 1 network-cost 50::UNKNOWN stream id: test1
2020-05-24 06:36:08,828 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onIceCandidate: 0:0:candidate:3830515337 1 udp 1686052607 23.22.34.236 53480 typ srflx raddr 10.31.254.120 rport 53480 generation 0 ufrag dQlG network-id 1 network-cost 50:stun:209.85.144.127:19302:UNKNOWN stream id: test1
2020-05-24 06:36:08,919 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onIceCandidate: 0:0:candidate:731577549 1 tcp 1518280447 10.31.254.120 44017 typ host tcptype passive generation 0 ufrag dQlG network-id 1 network-cost 50::UNKNOWN stream id: test1
2020-05-24 06:36:09,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:09,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:10,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:10,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:11,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:11,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:12,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:12,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:13,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:13,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:14,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:14,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:15,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:15,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:16,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:16,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:17,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:17,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:18,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:18,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:19,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:19,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:20,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:20,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:21,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:21,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:22,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:22,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:23,311 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:23,811 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Waiting for encoder to be ready for test1 hash:451159804
2020-05-24 06:36:23,879 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onIceConnectionChange FAILED for streamId: test1 hash:451159804
2020-05-24 06:36:23,879 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - entering stop() for test1
2020-05-24 06:36:23,879 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - running stop operations in executor for stream: test1 Received audio frame: 0 Processed audio frame: 0Received video frame: 0 Processed video frame: 0
2020-05-24 06:36:23,879 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Publish Stats StreamId: test1 TransferedByte: 0 Stream Duration: 1590302183879 Bitrate: 0 Source IP: null User-Agent: N/A
2020-05-24 06:36:23,888 [vert.x-worker-thread-4] INFO i.a.e.adaptive.StreamAdaptor - Recording set to false in streamadaptor: 1650673973 for stream: test1 stopping scheduler job id: 688
2020-05-24 06:36:23,888 [network_thread - 20265] INFO org.webrtc.Logging - Tag:WebRtcAudioTrackExternal - Message:GetPlayoutUnderrunCount
2020-05-24 06:36:23,889 [vert.x-worker-thread-4] WARN i.a.enterprise.adaptive.H264Encoder - Encoder is not running right now for stream index 0 for stream:test1
2020-05-24 06:36:23,888 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - onIceConnectionChange CLOSED for streamId: test1 hash:451159804
2020-05-24 06:36:23,889 [vert.x-worker-thread-4] INFO i.a.e.adaptive.StreamAdaptor - Flushing video encoder:null for stream adaptor:1650673973 stream id:test1
2020-05-24 06:36:23,889 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - entering stop() for test1
2020-05-24 06:36:23,889 [vert.x-worker-thread-4] INFO i.a.e.adaptive.StreamAdaptor - Flushing audio encoder:libopus for stream adaptor:1650673973 stream id:test1
2020-05-24 06:36:23,889 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - Encoder is already stopped, returning for test1
2020-05-24 06:36:23,889 [vert.x-worker-thread-11] INFO i.a.e.adaptive.StreamAdaptor - Recording set to false in streamadaptor: 1527637003 for stream: test1 stopping scheduler job id: 689
2020-05-24 06:36:23,889 [vert.x-worker-thread-4] INFO i.a.e.adaptive.StreamAdaptor - Flushing audio encoder:aac for stream adaptor:1650673973 stream id:test1
2020-05-24 06:36:23,889 [network_thread - 20265] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - stopping playout for stream test1
2020-05-24 06:36:23,889 [vert.x-worker-thread-11] WARN i.a.enterprise.preview.PngEncoder - video codec context is not initialized on png encoder: 210923299 for stream: test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-11] INFO i.a.e.adaptive.StreamAdaptor - Flushing video encoder:null for stream adaptor:1527637003 stream id:test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-11] INFO i.a.enterprise.preview.PngEncoder - Video codec writing trailer name: null height:480 for stream:test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-11] INFO i.a.e.adaptive.StreamAdaptor - Average thread entrance interval 9ms for stream: test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-11] INFO i.a.e.adaptive.StreamAdaptor - Leaving streamadaptor: 1527637003 for stream: test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-4] INFO i.a.enterprise.webrtc.WebRTCAdaptor - unregistering test1 muxer with 720p and codec: H264 from webrtc adaptor
2020-05-24 06:36:23,890 [vert.x-worker-thread-4] INFO i.a.enterprise.adaptive.H264Encoder - Video codec writing trailer name: null height:720 for stream:test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-4] INFO i.a.enterprise.adaptive.OpusEncoder - writing trailer codec name: libopus bitrate:128000 for stream:test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-4] INFO i.a.enterprise.adaptive.OpusEncoder - Total processing time 0, Average processing time 0 ms per frame on average for stream:test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-4] INFO i.a.enterprise.adaptive.AACEncoder - writing trailer codec name: aac bitrate:128000 for stream:test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-4] INFO i.a.e.adaptive.StreamAdaptor - Average thread entrance interval 9ms for stream: test1
2020-05-24 06:36:23,890 [vert.x-worker-thread-4] INFO i.a.e.adaptive.StreamAdaptor - Leaving streamadaptor: 1650673973 for stream: test1
2020-05-24 06:36:23,899 [webrtc-publish-test1-0] INFO i.a.e.w.WebRTCPublishStreamInfoListener - deleting zombi stream test1
2020-05-24 06:36:23,908 [webrtc-publish-test1-0] INFO i.a.e.adaptive.WebRTCEncoderAdaptor - leaving stop() for test1
Robert Berger
May 24, 2020, 2:58:30 AM5/24/20
to Ant Media Server
Looks like there is a bug in https://github.com/ant-media/StreamApp/blob/master/src/main/webapp/js/webrtc_adaptor.js,
Line 1091 is currently:
protocolSupported = thiz.candidateTypes.includes(event.candidate.protocol.toLowerCase());
But if its changed to (remove the "event"):
protocolSupported = thiz.candidateTypes.includes(candidate.protocol.toLowerCase());
I can send a stream ok.
On Friday, May 22, 2020 at 1:07:00 AM UTC-7, Robert Berger wrote:
Reply all
Reply to author
Forward
0 new messages