Get AppRTCDemo to rotate camera orientation?

[Xiangrong]

Hi,

Currently, the android native app AppRTCDemo is locked in landscape mode.

I wanted to enable device rotation with proper rotation of the video capture.

After unlocking the landscape mode and handling rotation manually (using android:configChanges="orientation|screenSize" in AndroidManifest.xml and overriding method onConfigurationChanged() in AppRTCDemoActivity.java), as expected, the device was able to rotate display on device, but the camera capture orientation does not rotate at all, and hence the display of the captured video (when the device is not at landscape) has the wrong orientation.

To correct this issue, I wanted to set the camera orientation manually, using:

camera.setDisplayOrientation( degreeOfCorrection )

Where camera is the current camera being used (android.hardware.Camera) and degreeOfCorrection is an int representing the degrees (0, 90, 180, or 270) I want the rotate the camera display orientation.

Here are my questions:

1. Will my strategy here result in the remote end having the video orientation manipulated, or does this merely affect the camera display on the android device side.

2. To use the setDisplayOrientation call above, I would need a reference to the current camera being used. How can I get this reference?

The current code does not contain any such reference, and the way the camera is selected is via getVideoCapturer(), which passes a String containing a description of the possible camera to VideoCapturer.create(), which according to VideoCapturer.java, passes that String to another function nativeCreateVideoCapturer(String deviceName).

However, I could not find the definition of nativeCreateVideoCapturer anywhere.

3. By parsing the String that was passed to VideoCapturer.create(), it should be possible for me to find the index of the camera that was used. Is there anyway to get the camera object being used with its index?

I do not know of any such way. If I create a new camera object with that index (either before or after VideoCapturer.create() ), I believe I would have to first close the camera being used, or close the camera I create before calling VideoCapturer.create(). Either way, if I change the display orientation of the camera I don't think the change would be passed on to the camera eventually used to create the video.

4. On a side point, there are many methods with a "native" prefixed. Can anyone explain what this prefixing means, and where the method definitions can be found?

The only other way I can think of to rotate camera orientation is to allow Android to handle device rotation automatically.

However, this will restart the activity and I am investigating how this can be done properly without having to add PeerConnection again.

Thanks in advance for your assistance!

Regards,

Xiangrong.

[Xiangrong]

Hi,

Does anyone knows if AppRTCDemo locks the camera orientation in a landscape direction?

After unlocking the landscape display of AppRTCDemo, I started the app in a portrait orientation.

The display recognised this and the layout was correctly set in a portrait orientation.

However, the video display showed that the camera orientation remained in a landscape direction.

As mentioned in my original email, from what I can see, based on the way VideoCapturer is implemented, there is no handle to the camera given to manipulate the camera being used.

Did I miss something?

Does anyone have the same or different opinion?

Thanks for your feedback.

Xiangrong.

[Xiangrong]

Hi Ami,

In changing the camera display orientation for AppRTCDemo, the challenge seems to be that after VideoCapturer is set (using getVideoCapturer() ),

there is no way to get a reference to the current camera being used.

Is there any way at all that you know of to access the camera used, or to change the camera display orientation (like what setDisplayOrientation( int degrees) would do),

be it directly or indirectly (like forcing screen orientation to certain values, which unfortunately did not change the camera display orientation when I tried it)?

Alternatively, can you confirm that changing camera display orientation is not possible once VideoCapturer is set, in the current Java implementation of the webrtc::PeerConnection API?

Thanks!

Regards,

Xiangrong.

[Ami Fischman]

PeerConnection's API doesn't expose the underlying Android Camera object.  Things should Just Work(tm).

The goal is that camera capture creates a video stream whose "stream-up" corresponds to "world-up" as much as possible.  Bug 1763 has some context on what we did for this for WRD but I guess there's more to be done; I filed 2432 for this (star that issue to be kept updated on it).

Display (as opposed to capture) rotation is possible by hooking whatever you like into your VideoRenderer.Callbacks.renderFrame() impl.  

Cheers,

-a

[Sergio Garcia Murillo]

Hi,

How about using Coordination of Video Orientation as in 3GPP TS 26.114 (http://www.3gpp.org/ftp/Specs/html-info/26114.htm)?

Best regards
Sergio

--
 
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Ami Fischman]

Sergio,

Sorry but I don't know what you mean and am failing to quickly turn your URL & keywords used into a paragraph of spec.  Can you be more explicit?

Cheers,

-a

[Sergio Garcia Murillo]

Hi  Ami,

I think that the spec will explain better than myself :)

Coordination of Video Orientation consists in signalling of the current orientation of the image captured on the sender side to the receiver for appropriate rendering and displaying. When CVO is succesfully negotiated it shall be signalled by the MTSI client. The signalling of the CVO uses RTP Header Extensions as specified in IETF RFC 5285 [95]. The one-byte form of the header shall be used. CVO information for a 2 bit granularity of Rotation (corresponding to urn:3gpp:video-orientation) is carried as a byte formatted as follows:

Bit#                   7                 6                 5                 4               3                 2                 1                 0(LSB)
Definition      0           0           0           0           C          F           R1          R0

With the following definitions:

C = Camera: indicates the direction of the camera used for this video stream. It can be used by the MTSI client in receiver to e.g. display the received video differently depending on the source camera.

    0: Front-facing camera, facing the user. If camera direction is unknown by the sending MTSI client in the terminal then this is the default value used.

1: Back-facing camera, facing away from the user.

F = Flip: indicates a horizontal (left-right flip) mirror operation on the video as sent on the link.

    0: No flip operation. If the sending MTSI client in terminal does not know if a horizontal mirror operation is necessary, then this is the default value used.

    1: Horizontal flip operation

R1, R0 = Rotation: indicates the rotation of the video as transmitted on the link. The receiver should rotate the video to compensate that rotation. E.g. a 90° Counter Clockwise rotation should be compensated by the receiver with a 90° Clockwise rotation prior to displaying.

Table 7.2: Rotation signalling for 2 bit granularity

R1	R0	Rotation of the video as sent on the link	Rotation on the receiver before display
0	0	0° rotation	None
0	1	90° Counter Clockwise (CCW) rotation or 270° Clockwise (CW) rotation	90° CW rotation
1	0	180° CCW rotation or 180° CW rotation	180° CW rotation
1	1	270° CCW rotation or 90° CW rotation	90° CCW rotation

 

CVO information for a higher granularity of Rotation (corresponding to urn:3GPP:video-orientation:6) is carried as a byte  formatted as follows:

Bit#                   7                 6                 5                 4               3                 2                 1                 0(LSB)
Definition      R5          R4          R3          R2          C          F           R1          R0

where C and F are as defined above and the bits R5,R4,R3,R2,R1,R0 represent the Rotation, which indicates the rotation of the video as transmitted on the link. Table 7.3 describes the rotation to be applied by the receiver based on the rotation bits.

Table 7.3: Rotation signalling for 6 bit granularity

R1	R0	R5	R4	R3	R2	Rotation of the video as sent on the link	Rotation on the receiver before display
0	0	0	0	0	0	0° rotation	None
0	0	0	0	0	1	(360/64)° Counter Clockwise (CCW) rotation	(360/64)° CW rotation
0	0	0	0	1	0	(2*360/64)° CCW rotation	(2*360/64)° CW rotation
.	.	.	.	.	.	.	.
.	.	.	.	.	.	.	.
.	.	.	.	.	.	.	.
1	1	1	1	1	0	(62*360/64)° CCW rotation	(2*360/64)° CCW rotation
1	1	1	1	1	1	(63*360/64)° CCW rotation	(360/64)° CCW rotation

 

Maybe it is worthy to add it to webrtc.

Best regards
Sergio

[xr]

Hi Ami,

Thanks for the info and for working on the issue.

I'm currently testing the setRotation() approach at GLSurfaceView.

However, display at the remote end will still not be corrected until issue 2432 is fixed.

When issue 2432 is done, will the local Android device still need to handle display rotation,

or will it also fix the display for local device (in addition to that for the remote peer)?

Regards,

Xiangrong.

[Ami Fischman]

On Thu, Sep 26, 2013 at 7:30 PM, xr <god...@gmail.com> wrote:

When issue 2432 is done, will the local Android device still need to handle display rotation,

or will it also fix the display for local device (in addition to that for the remote peer)?

I'd expect the latter.

Cheers,

-a

[Ami Fischman]

Thanks for clarifying, Sergio.

On Thu, Sep 26, 2013 at 8:45 AM, Sergio Garcia Murillo <sergio.gar...@gmail.com> wrote:

Maybe it is worthy to add it to webrtc.

Maybe, but that sounds like something to take up with the spec groups, not here.

Cheers,

-a

[xr]

Thank you Ami, that would be nice.

Do you have an idea when that might be achieved?

Cheers,

Xiangrong.

[Ami Fischman]

On Thu, Sep 26, 2013 at 8:53 PM, xr <god...@gmail.com> wrote:

Thank you Ami, that would be nice.

Do you have an idea when that might be achieved?

Nope; star the bug and hope ;)

Cheers,

-a

[xr]

I see.

We are all hoping then. :)

Thanks,

Xiangrong.

[Nishant Kumar]

Hi  Xiangrong,

I am also trying to unlock the camera. Did you get any fix?

[xr]

Hi Nishant,

According to Ami above, that camera is not meant to be exposed through PeerConnection's API.

We are still waiting/hoping for Ami to fix it in the issue 2432 filed.

Please star the issue as well. :-)

@Ami, has there been any progress on the bug? Thanks.

Regards,

Xiangrong.

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/H40HgIC3N-o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.

[Nishant Kumar]

Hi Xiangrong,

Thanks for the reply.  I have starred the bug.

I am trying to rotate the camera according to the screen orientation.To start with I have added 

camera.setDisplayOrientation(90);                                 ////            fix the camera orienation to portrait mode                                

in

file:           trunk\webrtc\modules\video_capture\android\java\src\org\webrtc\videoengine\VideoCaptureAndroid.java

method:  tryStartCapture(int width, int height, int frameRate)

line number: 111

 try {

            camera.setParameters(parameters);

            camera.setDisplayOrientation(90);   

        } catch (RuntimeException e) {

            Log.e(TAG, "setParameters failed", e);

            return -1;

        }

method:  StartCapture(int width, int height, int frameRate)

line number: 136

 public int StartCapture(int width, int height, int frameRate) {

        Log.d(TAG, "StartCapture width " + width +

                " height " + height +" frame rate " + frameRate);

        // Get the local preview SurfaceHolder from the static render class

        camera.setDisplayOrientation(90);

        localPreview = ViERenderer.GetLocalRenderer();

But the App camera is still locked to the landscape mode.  Please Help

Regards, 

Nishant

 

On Friday, September 20, 2013 5:13:27 PM UTC+5:30, Xiangrong wrote:

[xr]

Hi Nishant,

As far as I know, the camera object that we declare will have to be closed before the apprtc camera is functional.

In other words, whatever property you set to the camera object will not be available when apprtc creates its camera object for use.

We cannot set the property of the apprtc camera as the PeerConnection API does not give us a handle on that camera.

Regards,

Xiangrong.

[Ami Fischman]

On Thu, Oct 24, 2013 at 7:20 PM, xr <god...@gmail.com> wrote:

@Ami, has there been any progress on the bug? Thanks.

No; when there's progress it'll be on the bug ;)

-a

[Nishant Kumar]

Hi Xiangrong,

I am not creating a new Camera Object. I am just trying to change the camera orientation of the AppRTCCamera Object in the 

file:           trunk\webrtc\modules\video_capture\android\java\src\org\webrtc\videoengine\VideoCaptureAndroid.java

Regards,

Nishant

On Friday, September 20, 2013 5:13:27 PM UTC+5:30, Xiangrong wrote:

[xr]

Thanks, Ami, for the update.

Hope there will be soon. More people are starring it now. :)

--

[Xiangrong]

Hi Nishant,

Sorry, I missed out the line that said which file it was.

I've never modified files in trunk\webrtc, but if I'm not mistaken, the file you referred to is actually for webrtc on the Chrome browser on Android, not the Android native app.

Can somebody validate this opinion? Thanks.

Regards,

Xiangrong.

[Nishant Kumar]

Hi Xiangrong,

These files are in libjingle_peerconnection.jar(org.webrtc.videoengine) and are user for AppRTCDemoActivity.

On Friday, September 20, 2013 5:13:27 PM UTC+5:30, Xiangrong wrote:

[lanqiul...@gmail.com]

Hi, Nishant Kumar :

     as i know, VideoCaptureAndroid.java called by native code c++, could you know how to switch camera(front to back) in apprtcdemo ?     

在 2013年10月29日星期二UTC+8下午12时58分10秒，Nishant Kumar写道：

[Wasin Thiengkunagrit]

Hi,  lanqiul

  You can switch the camera to use back camera in apprtcdemo by editing the file AppRTCDemoActivity.java on method getVideoCapturer()

just switch the "front" and "back" parameter in array

   from String[] cameraFacing = { "front", "back" };   to String[] cameraFacing = { "back", "front" };

[lanqiul...@gmail.com]

Hi, Wasin Thiengkunagrit :

      thanks for your reply,  but we need to switch camera when video is calling, we try to reinit the getVideoCapturer() but has broken.

      how does you switch the camera when video is calling?

      thanks in advance.

在 2014年1月14日星期二UTC+8下午6时38分20秒，Wasin Thiengkunagrit写道：

[balaji sunku]

Very useful information. But where i can change the R0 and R1 values in the WebRTC code to get the change in orientation?? I am actually working on making the AppRTCDemo android application in portrait mode. 

Please help .

1

1

1

<p class="TAL" style="margin-top:3.0pt;text-align:center; mso-pagination:lines-together;tab-stops:70.9pt 5

...

[Christopher Markov]

Hi,

So what happened with this issue? Is there a way to rotate the camera stream orientation in Android?

Best regards,

Chris