Audio recording feature with seperated streams

[Dennis Guse]

Hi,

today I checked the recording feature and yes it works very well (used latest nightly build).
Just for my purpose it would be awesome, if the two audio streams (caller and receiver) would not be merged into one.
I assume it is not that big an issue, isnt it?

Why?
We are going to use for cSipSimple for research in a user study. Recording the phone calls with seperated audio streams allows us to evaluate the structure of a call, e.g. the effect turn-taking during the conversation, depending on the provided call quality, which we can set before a call.

[Régis Montoya]

Hi Dennis,

Yep, it should not be a big problem to make your use case working :
The first thing you have to read is this page :
http://trac.pjsip.org/repos/wiki/Python_SIP/Media

It's for python api but concept are the same for the C one (and so for Java/JNI one).
Then, have a look to where this is done in csipsimple :
http://code.google.com/p/csipsimple/source/browse/trunk/CSipSimple/src/com/csipsimple/pjsip/PjSipService.java#1734

If you have a look at line 1770, you'll see that I connect to recPort (the port of the wave file), both sound from audio device (0) and sound from the remote call (callInfo.getConfPort()).

So in your case, if you want to have two separate files, just create two wav files as done at line 1752; and when connecting ports, connect device port to file1, and call conf port to file2. And normally that's it :). (remember to disconnect the two files/ports when call is ended in disconnect).

It implies to slightly modify csipsimple source code, but not so much (maybe another sparse array to store the new port list).

This two files approach, IMHO is good for testing but probably useless as a mainstream feature.

One other thing that could be introduce in csipsimple as patch for the trunk version is a feature to record in one file, but in two channel of a stereo stream. There is a special port in pjsip that should allow that. It's the stereo_port. It should allow to connect each of our port to each channel of the port (and then, connect this port to the wav file which would be in this case stereo one). This would be an improvement that could benefit mainstream users I think. If you have a proposal patch for that it will be welcome ;).

Regards,
Régis

2012/6/4 Dennis Guse <business.d...@gmail.com>

[Dennis Guse]

Hi Regis,

I love your idea using stereo in a single wav file. I am looking into extending it, but it might take some time...

On Monday, June 4, 2012 11:33:34 AM UTC+2, r3gis wrote:

Hi Dennis,

Yep, it should not be a big problem to make your use case working :
The first thing you have to read is this page :
http://trac.pjsip.org/repos/wiki/Python_SIP/Media

It's for python api but concept are the same for the C one (and so for Java/JNI one).
Then, have a look to where this is done in csipsimple :
http://code.google.com/p/csipsimple/source/browse/trunk/CSipSimple/src/com/csipsimple/pjsip/PjSipService.java#1734

If you have a look at line 1770, you'll see that I connect to recPort (the port of the wave file), both sound from audio device (0) and sound from the remote call (callInfo.getConfPort()).

So in your case, if you want to have two separate files, just create two wav files as done at line 1752; and when connecting ports, connect device port to file1, and call conf port to file2. And normally that's it :). (remember to disconnect the two files/ports when call is ended in disconnect).

It implies to slightly modify csipsimple source code, but not so much (maybe another sparse array to store the new port list).

This two files approach, IMHO is good for testing but probably useless as a mainstream feature.

One other thing that could be introduce in csipsimple as patch for the trunk version is a feature to record in one file, but in two channel of a stereo stream. There is a special port in pjsip that should allow that. It's the stereo_port. It should allow to connect each of our port to each channel of the port (and then, connect this port to the wav file which would be in this case stereo one). This would be an improvement that could benefit mainstream users I think. If you have a proposal patch for that it will be welcome ;).

Regards,
Régis

2012/6/4 Dennis Guse

Hi,

[Dennis Guse]

Hi Regis,

I looked into the PJSUA API and there it is stated that if multiple sources are connected to one recorder, the recorder will mix the audio into the underlying WAV file.
So, the only option would be to use one file per stream - which is in fact ok for us, but not that nice in general.

On Monday, June 4, 2012 11:33:34 AM UTC+2, r3gis wrote:

Hi Dennis,

Yep, it should not be a big problem to make your use case working :
The first thing you have to read is this page :
http://trac.pjsip.org/repos/wiki/Python_SIP/Media

It's for python api but concept are the same for the C one (and so for Java/JNI one).
Then, have a look to where this is done in csipsimple :
http://code.google.com/p/csipsimple/source/browse/trunk/CSipSimple/src/com/csipsimple/pjsip/PjSipService.java#1734

If you have a look at line 1770, you'll see that I connect to recPort (the port of the wave file), both sound from audio device (0) and sound from the remote call (callInfo.getConfPort()).

So in your case, if you want to have two separate files, just create two wav files as done at line 1752; and when connecting ports, connect device port to file1, and call conf port to file2. And normally that's it :). (remember to disconnect the two files/ports when call is ended in disconnect).

It implies to slightly modify csipsimple source code, but not so much (maybe another sparse array to store the new port list).

This two files approach, IMHO is good for testing but probably useless as a mainstream feature.

One other thing that could be introduce in csipsimple as patch for the trunk version is a feature to record in one file, but in two channel of a stereo stream. There is a special port in pjsip that should allow that. It's the stereo_port. It should allow to connect each of our port to each channel of the port (and then, connect this port to the wav file which would be in this case stereo one). This would be an improvement that could benefit mainstream users I think. If you have a proposal patch for that it will be welcome ;).

Regards,
Régis

2012/6/4 Dennis Guse

Hi,

[Régis Montoya]

The 2 file solution is indeed easier,
That was the intent of my first paragraph about how to modify csipsimple to record in two separate files. It's not so hard to do.


The point about stereo was more an extra wish on what could be integrated in csipsimple for mainstream users. I had in mind to use these ports : http://www.pjsip.org/docs/latest-2/pjmedia/docs/html/group__PJMEDIA__STEREO.htm#ga704a744d9f3161bb40d9bb993b301f46 and http://www.pjsip.org/pjmedia/docs/html/group__PJMEDIA__SPLITCOMB.htm
I think that there is some pretty solution to find here by creating and connecting ports split/comb port etc.
It's way more complicate, than the first basic solution to save as two separate file; but I suspect it's possible using the pjmedia api.
In this case it would be
Device port => Chan 0 of Split/comb => Wav file writer opened in stereo
Call port     => Chan 1 of Split/comb __/


Instead of current :
Device port  => File writer opened in mono
Call port      _/

Or in your case :
Device port => File writer number 1 in mono
Call port     => File writer number 2 in mono


Régis

2012/6/6 Dennis Guse <business.d...@gmail.com>

[Dennis Guse]

Can the PJMEDIA API be accessed via PJSUA-JNI?
Is this API exposed to non-C languages?

But I found this thing interesting: http://www.pjsip.org/pjmedia/docs/html/group__PJMEDIA__SPLITCOMB.htm

On Wednesday, June 6, 2012 12:05:12 PM UTC+2, r3gis wrote:

The 2 file solution is indeed easier,
That was the intent of my first paragraph about how to modify csipsimple to record in two separate files. It's not so hard to do.


The point about stereo was more an extra wish on what could be integrated in csipsimple for mainstream users. I had in mind to use these ports : http://www.pjsip.org/docs/latest-2/pjmedia/docs/html/group__PJMEDIA__STEREO.htm#ga704a744d9f3161bb40d9bb993b301f46 and http://www.pjsip.org/pjmedia/docs/html/group__PJMEDIA__SPLITCOMB.htm
I think that there is some pretty solution to find here by creating and connecting ports split/comb port etc.
It's way more complicate, than the first basic solution to save as two separate file; but I suspect it's possible using the pjmedia api.
In this case it would be
Device port => Chan 0 of Split/comb => Wav file writer opened in stereo
Call port     => Chan 1 of Split/comb __/


Instead of current :
Device port  => File writer opened in mono
Call port      _/

Or in your case :
Device port => File writer number 1 in mono
Call port     => File writer number 2 in mono


Régis

2012/6/6 Dennis Guse

Hi Regis,

[Régis Montoya]

By default no, but could be easily exposed.
The pjmedia_tonegen has already been added from pjmedia api to swig export for example. It's not so hard.

Another solution is to expose a wrapper function in csipsimple-wrapper that take only necessary settings as arguments and do the job in C. It's usually easier than using java interface that have complex way to manage pointers. And that's usually the approach I use for this kind of tasks.

Actually everything exposed to java world in csipsimple is done by scripts of swig-glue that are now entirely managed by csipsimple project. So it's possible to add new .h files, to directly enrich with needed pjsip apis etc.

And this swig glue is built at build time of the native library which make these changes easy. But make things more complex for people that don't want to build the native part of the application ;)

[Régis Montoya]

Hi,

I just started something for recording as two different channels of a stereo file using split/comb. It also move recording logic to native part. For some reason when I activate it I get a crash quickly after record start, but the few seconds I get are valid.
It seems something about memory pool not properly managed in my code, but I was not able to find the solution yet. If you want to review, and have an idea on how it could be fixed...; it's available in r1613 (http://code.google.com/p/csipsimple/source/detail?r=1613)

Best regards,
Régis

On 07/06/2012 15:28, Dennis Guse wrote:

[Dennis Guse]

I took a look, but I must admit my C skills are limited.... (will try on the weekend again).

[Dennis Guse]

So, getting back to this old topic ;)

I spent my last night with trying to figure out, why stereo recordering fails (actually crashes).

Your code seems fine and all PJSIP calls return PJ_SUCCESS.

Digging in the source code of PJSIP namely then psjua: pjsua_recorder_create()

http://svn.pjsip.org/repos/pjproject/trunk/pjsip/src/pjsua-lib/pjsua_aud.c

I found that pjsua_recorder...() uses PJSUA_LOCK() and PJSUA_UNLOCK()

I've added this to the stereo recording call (and actually removed the merging recording call), but the SIPStack chrashes without any notification (I believe the complete SIPService).

The behavior looks like either

1) Race-condition issue (shouldn't be as I added the LOCK and UNLOCK)

In fact, my approach might be broken, i.e. I use the locking in a wrong way...

(I am not familar with PJSIP)

2) The SIPService is killed due to a watchdog timer and then restarted

Here is a link to the PJSUA Sample demo, which includes a similar approach (line 6149)

http://www.pjsip.org/pjsip/docs/html/page_pjsip_samples_pjsua.htm

Any suggestions?

Here is my striped off implementation of call_recording_start with code highlighting: http://pastebin.com/HwZE63ZX

and here

PJ_DECL(pj_status_t) call_recording_start(pjsua_call_id call_id, const pj_str_t *file, pj_bool_t stereo){

pj_status_t status = PJ_EINVAL;

pjsua_conf_port_id rec_port;

pjsua_call_info call_info;

// If nothing is recording currently, create recorder

if (file != NULL && file->slen > 0) {

if (css_var.call_recorder_ids[call_id] == PJSUA_INVALID_ID) {

PJSUA_LOCK();

// TODO --- very first implementation -- no error check -- should be done !!!!!

// TODO --- allocation totally unclean !!!!

// Device port => Chan 0 of Split/comb => Wav file writer opened in stereo

// Call port   => Chan 1 of Split/comb __/

struct css_stereo_recorder_data *rec_datas = &css_var.call_stereo_recoders[call_id];

char path[PJ_MAXPATH];

pj_memcpy(path, file->ptr, file->slen);

path[file->slen] = '\0';

rec_datas->pool = pjsua_pool_create("stereo_recorder", 100, 100);

/* Create stereo wave file writer */

status = pjmedia_wav_writer_port_create(rec_datas->pool, path,

pjsua_var.media_cfg.clock_rate, 2,

2 * pjsua_var.mconf_cfg.samples_per_frame,

pjsua_var.mconf_cfg.bits_per_sample, 0, 0, &rec_datas->file_port);

PJ_LOG(4, (THIS_FILE, "Stereo: Wav writer created, %d", status));

/* Create stereo-mono splitter/combiner */

status = pjmedia_splitcomb_create(rec_datas->pool,

pjsua_var.media_cfg.clock_rate /* clock rate */,

2 /* stereo */, 2 * pjsua_var.mconf_cfg.samples_per_frame,

pjsua_var.mconf_cfg.bits_per_sample, 0 /* options */,

&rec_datas->splitcomb_port);

PJ_LOG(1, (THIS_FILE, "Stereo: SplitCombo created, %d", status));

/* Create the master sound for connecting splitcomb and wav writter*/

status = pjmedia_master_port_create(rec_datas->pool, rec_datas->splitcomb_port, rec_datas->file_port, 0,&rec_datas->master_stereo_port);

PJ_LOG(1, (THIS_FILE, "Stereo: Master port created, %d", status));

/* Get each channel for right and left splitcomb port */

// Channel 0

status = pjmedia_splitcomb_create_rev_channel(rec_datas->pool, rec_datas->splitcomb_port,

0 /* ch0 */, 0 /* options */, &rec_datas->splitcomb_chan0_port);

PJ_LOG(1, (THIS_FILE, "Stereo: SplitCombo channel connected [0], %d", status));

status = pjsua_conf_add_port(rec_datas->pool, rec_datas->splitcomb_chan0_port, &rec_datas->splitcomb_chan0_slot);

PJ_LOG(1, (THIS_FILE, "Stereo: Conf port added [0], %d", status));

// Channel 1

status = pjmedia_splitcomb_create_rev_channel(rec_datas->pool, rec_datas->splitcomb_port, 1 /* ch1 */, 0 /* options */, &rec_datas->splitcomb_chan1_port);

PJ_LOG(1, (THIS_FILE, "Stereo: SplitCombo channel connected [1], %d", status));

status = pjsua_conf_add_port(rec_datas->pool, rec_datas->splitcomb_chan1_port, &rec_datas->splitcomb_chan1_slot);

PJ_LOG(1, (THIS_FILE, "Stereo: Conf port added [1], %d", status));

css_var.call_recorder_ids[call_id] = STEREO_RECORDER_ID; //Fake for now

// If a recorded has been created, connect call port to recorder and sound port (0) to recorder.

status = pjsua_call_get_info(call_id, &call_info);

PJ_LOG(1, (THIS_FILE, "Stereo: Check call info, %d", status));

PJ_LOG(1, (THIS_FILE, "Stereo: Start recording call %d", call_id));

status = pjsua_conf_connect(call_info.conf_slot, rec_datas->splitcomb_chan0_slot);

PJ_LOG(1, (THIS_FILE, "Stereo: pjsua_conf_connect: call_info, %d", status));

status = pjsua_conf_connect(0,                   rec_datas->splitcomb_chan1_slot);

PJ_LOG(1, (THIS_FILE, "Stereo:  pjsua_conf_connect: 0, %d", status));

status = pjmedia_master_port_start(rec_datas->master_stereo_port);

PJ_LOG(1, (THIS_FILE, "Stereo: pjmedia_master_port_start, %d", status));

PJSUA_UNLOCK();

return PJ_SUCCESS;

}

}

    return status;

}

[Dennis Guse]

Is there a way to access the PJSIP logging?

[Régis Montoya]

Mmmh interesting.

On java side it's still launched from the Executor thread ?

I ask that because maybe it's about the fact the thread is not registered. (pjsip needs thread to be registered).

Else about the lock :
I'm not sure it's needed to move pjsua lock to wrap all code. But maybe a dedicated css mutex should be added.
As we never use a pjsua private var and always use "pjsua_" method that are properly protected using pjsua lock is maybe not necessary to do our port creation locked.

[Régis Montoya]

Normally it should log in logcat. Just ensure that your log level is set
to 4 (or upper) in the app settings (in expert setting mode in user
interface section).

[Dennis Guse]

@Executor thread I dont know.

@LOCK: actually we also call pjmedia functions. And this might influence the result....

[Dennis Guse]

LOG level 5 and TRACES

But I dont know what to make of this...

12-13 17:48:28.025: D/PjService(8503): Out dir /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-13_174828.wav

12-13 17:48:28.046: D/libpjsip(8503): 17:48:28.059   wav_writer.c !File writer '/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-13_174828.wav' created: samp.rate=16000, bufsize=4KB

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.059 call_recorder.  Stereo: Wav writer created, 0

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.060 call_recorder.  Stereo: SplitCombo created, 0

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.060 call_recorder.  Stereo: Master port created, 0

12-13 17:48:28.046: V/libpjsip(8503): 17:48:28.060     scombdb-dn  Using delay buffer with WSOLA.

12-13 17:48:28.046: V/libpjsip(8503): 17:48:28.060     scombdb-up  Using delay buffer with WSOLA.

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.060 call_recorder.  Stereo: SplitCombo channel connected [0], 0

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.060 call_recorder.  Stereo: Conf port added [0], 0

12-13 17:48:28.046: V/libpjsip(8503): 17:48:28.061     scombdb-dn  Using delay buffer with WSOLA.

12-13 17:48:28.046: V/libpjsip(8503): 17:48:28.061     scombdb-up  Using delay buffer with WSOLA.

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.061 call_recorder.  Stereo: SplitCombo channel connected [1], 0

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.061 call_recorder.  Stereo: Conf port added [1], 0

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.061 call_recorder.  Stereo: Check call info, 0

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.062 call_recorder.  Stereo: Start recording call 0

12-13 17:48:28.046: D/libpjsip(8503): 17:48:28.062    pjsua_aud.c  Conf connect: 2 --> 3

12-13 17:48:28.046: D/libpjsip(8503): 17:48:28.062   conference.c  .Port 2 (sip:333@******) transmitting to port 3 (scomb-rev)

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.062 call_recorder.  Stereo: pjsua_conf_connect: call_info, 0

12-13 17:48:28.046: D/libpjsip(8503): 17:48:28.062    pjsua_aud.c  Conf connect: 0 --> 4

12-13 17:48:28.046: D/libpjsip(8503): 17:48:28.062   conference.c  .Port 0 (Android Audio) transmitting to port 4 (scomb-rev)

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.062 call_recorder.  Stereo:  pjsua_conf_connect: 0, 0

12-13 17:48:28.046: E/libpjsip(8503): 17:48:28.063 call_recorder.  Stereo: pjmedia_master_port_start, 0

12-13 17:48:28.066: D/PowerManager(8503): release flags=0x0 tag=SipWakeLock

12-13 17:48:28.066: V/SipWakeLock(8503): release wakelock: holder count=0

12-13 17:48:28.066: D/InCallActivity(8503): We have a call 0 / 5/1

12-13 17:48:28.066: D/InCallControls(8503): Mode is : 5

12-13 17:48:28.066: D/InCallControls(8503): >> Speaker null

12-13 17:48:28.076: D/InCallInfoGrid(8503): Populate 1 children

12-13 17:48:28.076: V/InCallInfoGrid(8503): Render a grid of 1 x 1

12-13 17:48:28.076: D/InCallCard(8503): Set call state : 5

12-13 17:48:28.076: V/libpjsip(8503): 17:48:28.090   strm0x22528c !Jitter buffer starts returning normal frames (after 17 empty/lost)

12-13 17:48:28.086: V/InCallCard(8503): we start the timer now 

12-13 17:48:28.086: D/InCallInfoGrid(8503): Populate 0 children

12-13 17:48:28.086: D/InCallActivity(8503): Active call is 0

12-13 17:48:28.086: D/InCallActivity(8503): Update ui from call 0 state Aufgebaut

12-13 17:48:28.086: D/InCallActivity(8503): we leave the update ui function

12-13 17:48:28.086: D/CallProximityManager(8503): Horizontal : false and activate for calls true

12-13 17:48:28.116: V/libpjsip(8503): 17:48:28.131   strm0x22528c  Jitter buffer empty (prefetch=0), plc invoked

12-13 17:48:28.216: V/libpjsip(8503): 17:48:28.229     scombdb-up  566 samples reduced, buf_cnt=1994

12-13 17:48:28.236: V/libpjsip(8503): 17:48:28.249     scombdb-up  271 samples reduced, buf_cnt=2043

12-13 17:48:28.276: V/libpjsip(8503): 17:48:28.289     scombdb-up  358 samples reduced, buf_cnt=2005

12-13 17:48:28.296: V/libpjsip(8503): 17:48:28.309     scombdb-up  135 samples reduced, buf_cnt=2190

12-13 17:48:28.316: V/libpjsip(8503): 17:48:28.329     scombdb-up  434 samples reduced, buf_cnt=2076

12-13 17:48:28.316: V/libpjsip(8503): 17:48:28.329      scomb-rev  Pausing media flow on upstream direction (level=14)

12-13 17:48:28.416: V/libpjsip(8503): 17:48:28.428   strm0x22528c  Jitter buffer starts returning normal frames (after 14 empty/lost)

12-13 17:48:28.416: V/libpjsip(8503): 17:48:28.429   strm0x22528c  Jitter buffer empty (prefetch=0), plc invoked

12-13 17:48:28.736: V/libpjsip(8503): 17:48:28.748   strm0x22528c  Jitter buffer starts returning normal frames (after 16 empty/lost)

12-13 17:48:28.736: V/libpjsip(8503): 17:48:28.749   strm0x22528c  Jitter buffer empty (prefetch=0), plc invoked

12-13 17:48:29.097: V/libpjsip(8503): 17:48:29.108   strm0x22528c  Jitter buffer starts returning normal frames (after 17 empty/lost)

12-13 17:48:29.117: V/libpjsip(8503): 17:48:29.127   strm0x22528c  Jitter buffer empty (prefetch=0), plc invoked

12-13 17:48:29.237: I/dalvikvm(8503): threadid=4: reacting to signal 3

12-13 17:48:29.247: I/dalvikvm(8503): Wrote stack traces to '/data/anr/traces.txt'

12-13 17:48:29.247: I/dalvikvm(8552): threadid=4: reacting to signal 3

12-13 17:48:29.247: I/dalvikvm(8552): Wrote stack traces to '/data/anr/traces.txt'

Relevant traces

----- pid 8503 at 2012-12-13 17:48:29 -----

Cmd line: com.csipsimple:sipStack

DALVIK THREADS:

(mutexes: tll=0 tsl=0 tscl=0 ghl=0 hwl=0 hwll=0)

"main" prio=5 tid=1 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x40028280 self=0xd0c0

  | sysTid=8503 nice=0 sched=0/0 cgrp=default handle=-1345006432

  at android.os.MessageQueue.nativePollOnce(Native Method)

  at android.os.MessageQueue.next(MessageQueue.java:119)

  at android.os.Looper.loop(Looper.java:117)

  at android.app.ActivityThread.main(ActivityThread.java:3691)

  at java.lang.reflect.Method.invokeNative(Native Method)

  at java.lang.reflect.Method.invoke(Method.java:507)

  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:907)

  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:665)

  at dalvik.system.NativeStart.main(Native Method)

[<c0293384>] sys_epoll_wait+0x238/0x398

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"android.hardware.SensorManager$SensorThread" prio=5 tid=19 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x405fc130 self=0x236b50

  | sysTid=8637 nice=-8 sched=0/0 cgrp=default handle=2322432

  at android.hardware.SensorManager.sensors_data_poll(Native Method)

  at android.hardware.SensorManager$SensorThread$SensorThreadRunnable.run(SensorManager.java:454)

  at java.lang.Thread.run(Thread.java:1019)

[<c0293384>] sys_epoll_wait+0x238/0x398

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Quit-timer" prio=5 tid=18 WAIT

  | group="main" sCount=1 dsCount=0 obj=0x405d2058 self=0x22aa08

  | sysTid=8636 nice=0 sched=0/0 cgrp=default handle=2085400

  at java.lang.Object.wait(Native Method)

  - waiting on <0x405d2058> (a java.util.Timer$TimerImpl)

  at java.lang.Object.wait(Object.java:358)

  at java.util.Timer$TimerImpl.run(Timer.java:214)

"ClientRecordThread" prio=10 tid=17 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x406022d0 self=0x1ccab8

  | sysTid=8620 nice=-16 sched=0/0 cgrp=default handle=1890008

  at dalvik.system.NativeStart.run(Native Method)

[<c01d7370>] hrtimer_nanosleep+0xb4/0x134

[<c01d7480>] sys_nanosleep+0x90/0xa0

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"AudioTrackThread" prio=10 tid=16 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x405e3de0 self=0x2120f8

  | sysTid=8615 nice=-16 sched=0/0 cgrp=default handle=1988696

  at dalvik.system.NativeStart.run(Native Method)

[<c01e30d8>] futex_wait_queue_me+0x130/0x150

[<c01e3220>] futex_wait+0x128/0x2ec

[<c01e5708>] do_futex+0xb0/0x8dc

[<c01e6074>] sys_futex+0x140/0x154

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Thread-39" prio=5 tid=15 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x405e3370 self=0x210b98

  | sysTid=8613 nice=-19 sched=0/0 cgrp=default handle=1561544

  at android.media.AudioTrack.native_write_byte(Native Method)

  at android.media.AudioTrack.write(AudioTrack.java:912)

  at dalvik.system.NativeStart.run(Native Method)

[<c01e30d8>] futex_wait_queue_me+0x130/0x150

[<c01e3220>] futex_wait+0x128/0x2ec

[<c01e5708>] do_futex+0xb0/0x8dc

[<c01e6074>] sys_futex+0x140/0x154

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Thread-38" prio=5 tid=14 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x405e2fc0 self=0x20fa58

  | sysTid=8612 nice=-19 sched=0/0 cgrp=default handle=1882544

  at android.media.AudioRecord.native_read_in_byte_array(Native Method)

  at android.media.AudioRecord.read(AudioRecord.java:561)

  at dalvik.system.NativeStart.run(Native Method)

[<c01e30d8>] futex_wait_queue_me+0x130/0x150

[<c01e3220>] futex_wait+0x128/0x2ec

[<c01e5708>] do_futex+0xb0/0x8dc

[<c01e6074>] sys_futex+0x140/0x154

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"SipTimers.Executor" prio=5 tid=13 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x405b5fe8 self=0x1f27e8

  | sysTid=8573 nice=0 sched=0/0 cgrp=default handle=1567624

  at android.os.MessageQueue.nativePollOnce(Native Method)

  at android.os.MessageQueue.next(MessageQueue.java:119)

  at android.os.Looper.loop(Looper.java:117)

  at android.os.HandlerThread.run(HandlerThread.java:60)

[<c0293384>] sys_epoll_wait+0x238/0x398

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"UAStateAsyncWorker" prio=5 tid=12 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x40574ca0 self=0x182a18

  | sysTid=8567 nice=0 sched=0/0 cgrp=default handle=1585376

  at android.os.MessageQueue.nativePollOnce(Native Method)

  at android.os.MessageQueue.next(MessageQueue.java:119)

  at android.os.Looper.loop(Looper.java:117)

  at android.os.HandlerThread.run(HandlerThread.java:60)

[<c0293384>] sys_epoll_wait+0x238/0x398

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Binder Thread #4" prio=5 tid=11 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x4052e000 self=0x17d698

  | sysTid=8566 nice=0 sched=0/0 cgrp=default handle=1561480

  at dalvik.system.NativeStart.run(Native Method)

[<c05eaa0c>] binder_thread_read+0x2a8/0xc90

[<c05ede14>] binder_ioctl+0x24c/0x5d8

[<c0269a40>] vfs_ioctl+0x2c/0xac

[<c026a11c>] do_vfs_ioctl+0x56c/0x5cc

[<c026a1b0>] sys_ioctl+0x34/0x54

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"SipService.Executor" prio=5 tid=10 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x405909b8 self=0x1637c8

  | sysTid=8565 nice=0 sched=0/0 cgrp=default handle=1456384

  at android.os.MessageQueue.nativePollOnce(Native Method)

  at android.os.MessageQueue.next(MessageQueue.java:119)

  at android.os.Looper.loop(Looper.java:117)

  at android.os.HandlerThread.run(HandlerThread.java:60)

[<c0293384>] sys_epoll_wait+0x238/0x398

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Binder Thread #3" prio=5 tid=9 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x4058bb30 self=0x15bb20

  | sysTid=8564 nice=0 sched=0/0 cgrp=default handle=1425536

  at dalvik.system.NativeStart.run(Native Method)

[<c05eaa0c>] binder_thread_read+0x2a8/0xc90

[<c05ede14>] binder_ioctl+0x24c/0x5d8

[<c0269a40>] vfs_ioctl+0x2c/0xac

[<c026a11c>] do_vfs_ioctl+0x56c/0x5cc

[<c026a1b0>] sys_ioctl+0x34/0x54

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Binder Thread #2" prio=5 tid=8 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x40531bb0 self=0x95a50

  | sysTid=8519 nice=0 sched=0/0 cgrp=default handle=611696

  at dalvik.system.NativeStart.run(Native Method)

[<c05eaa0c>] binder_thread_read+0x2a8/0xc90

[<c05ede14>] binder_ioctl+0x24c/0x5d8

[<c0269a40>] vfs_ioctl+0x2c/0xac

[<c026a11c>] do_vfs_ioctl+0x56c/0x5cc

[<c026a1b0>] sys_ioctl+0x34/0x54

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Binder Thread #1" prio=5 tid=7 NATIVE

  | group="main" sCount=1 dsCount=0 obj=0x4051d778 self=0x94058

  | sysTid=8516 nice=0 sched=0/0 cgrp=default handle=989784

  at dalvik.system.NativeStart.run(Native Method)

[<c05eaa0c>] binder_thread_read+0x2a8/0xc90

[<c05ede14>] binder_ioctl+0x24c/0x5d8

[<c0269a40>] vfs_ioctl+0x2c/0xac

[<c026a11c>] do_vfs_ioctl+0x56c/0x5cc

[<c026a1b0>] sys_ioctl+0x34/0x54

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"Compiler" daemon prio=5 tid=6 VMWAIT

  | group="system" sCount=1 dsCount=0 obj=0x40518ff8 self=0xefbb0

  | sysTid=8514 nice=0 sched=0/0 cgrp=default handle=974784

  at dalvik.system.NativeStart.run(Native Method)

"JDWP" daemon prio=5 tid=5 VMWAIT

  | group="system" sCount=1 dsCount=0 obj=0x40518f48 self=0xef998

  | sysTid=8512 nice=0 sched=0/0 cgrp=default handle=979552

  at dalvik.system.NativeStart.run(Native Method)

"Signal Catcher" daemon prio=5 tid=4 RUNNABLE

  | group="system" sCount=0 dsCount=0 obj=0x40518e88 self=0xef6c8

  | sysTid=8510 nice=0 sched=0/0 cgrp=default handle=517960

  at dalvik.system.NativeStart.run(Native Method)

[<c0104b40>] save_stack_trace_tsk+0x0/0xc0

[<c02a3e18>] proc_pid_stack+0xec/0x13c

[<c02a51bc>] proc_single_show+0x48/0x84

[<c02791e4>] seq_read+0x264/0x4e0

[<c025ac34>] vfs_read+0xa8/0x150

[<c025ad88>] sys_read+0x3c/0x68

[<c0100ca0>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff

------------------------------

"GC" daemon prio=5 tid=3 VMWAIT

  | group="system" sCount=1 dsCount=0 obj=0x40518de0 self=0xef4d0

  | sysTid=8507 nice=0 sched=0/0 cgrp=default handle=974904

  at dalvik.system.NativeStart.run(Native Method)

"HeapWorker" daemon prio=5 tid=2 VMWAIT

  | group="system" sCount=1 dsCount=0 obj=0x40518d28 self=0xef2e0

  | sysTid=8506 nice=0 sched=0/0 cgrp=default handle=979336

  at dalvik.system.NativeStart.run(Native Method)

----- end 8503 -----

[Régis Montoya]

Mmm, usually when comes with an ANR it's because of a deadlock.

But from ANR trace only active threads are the one to play/record from the device and one timer that is waiting for execution to finish.

So you're probably right, there is something with a timer that goes wrong and lock audio record/play processing.

2012/12/13 Dennis Guse <business.d...@gmail.com>

[Dennis Guse]

I got the recording running, but not using one stereo file, but using two mono files (and it works without deadlocks...).

However, I have one really painful issue:

Since I use two recorders I need two files and thus two filenames. Actually the approach is straight forward...

Creating new pj_str_t's from static c-strings works, but I couldnt get pj_strcat working. Everytime I call it the thread is terminated after 5s.

Some code:

pj_str_t filename_incoming = pj_str("");

PJ_LOG(1, (THIS_FILE, "1 %.*s 1", filename_incoming.slen, filename_incoming.ptr));

pj_strcat(&filename_incoming, &filename); //Filename is the original filename via function call

PJ_LOG(1, (THIS_FILE, "2 %.*s 1", filename_incoming.slen, filename_incoming.ptr));

pj_strcat2(&filename_incoming, "-incoming.wav");

PJ_LOG(1, (THIS_FILE, "1 %.*s 1", filename_incoming.slen, filename_incoming.ptr));

But never reaches the second PJ_LOG...

[Régis Montoya]

pj_str is basically a len + a pointer to a memory buffer.

When you create pj_str(""), buffer point to the char* "". (which in memory only have 1 len).
If buffer size is too short to contain concat string it will memory overflow.
If you are used to cstring it's exactly same api : http://www.cplusplus.com/reference/cstring/strcat/ the target len should be enough to not have buffer overflow.

So, you have basically two solutions :
Solution 1 : allocate a char[] on stack with a fixed len. And use this buffer for your filename pj_str.
Example :

char buf[128];
/* Assert in case a buffer overflow could happen (could raise an error else) */
pj_assert(str1.slen + str2.slen + str3.slen < 128);
pj_str_t filename_incoming = {0, buf};
// your code goes here

Solution 2 : use a pool to allocate dynamically buffer accordingly to the needed filename length.
Example :

/* With pool coming from somewhere (either allocated and released in your code, or available across the module if you need the pj_str to live outside from your method */
pj_str_t filename_incoming;
filename_incoming.ptr = (char*)pj_pool_alloc(pool,
                       str1.slen + str2.slen + str3.slen);
filename_incoming.slen = 0;
// Your code goes here

[Dennis Guse]

Hi Regis,

I moved everything back to Java(where it once was in PjSipService.java) and removed using the call_recorder.

It basically the implementation back before rev 1613, where you implemented the first approach for stereo.

Here you can find my code snippet: http://pastebin.com/VtuUigA9

It uses two files for recording and works...

I created this interface:

    interface RecorderCall {

    boolean prepare();

    boolean startRecording();

    boolean stopRecording();

    void prepareBroadcast(Intent itn); 

    }

and two implementations, one for merged (one file) and one for splitted (two files).

I believe it is a good idea to handle only technical things in the libs and handle the rest in java-code as it is easier to handle (and for most people easier to adapt [like me ;)]).

What do you think?

[Dennis Guse]

So, works well if I trigger it during the call.

The two files are recorded and everything looks pretty neat.

However, if i enable Auto Recording it doesnt. 

The two files are created, but only one file is "growing" - the other stays almost empty (usually less than 10kb).

It is usally the incoming.

I think it is a timing issue, e.g. that one of the audio sources is there and thus the connect works, but not yet ready (probably the port changes or whatever....).

PS: Bug sounds similar to Issue 1649. Using only one codec doesnt solve the problem...

Here is a log of starting with autorecord, stop it and start a second record manually (also stopped by hand

12-17 15:31:26.937: D/PjService(5332): Out dir /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_incoming.wav

12-17 15:31:26.947: D/PjService(5332): Out dir /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_outgoing.wav

12-17 15:31:26.947: D/libpjsip(5332): 15:31:26.954    pjsua_aud.c !.....Creating recorder /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_incoming.wav..

12-17 15:31:26.957: D/libpjsip(5332): 15:31:26.963   wav_writer.c  ......File writer '/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_incoming.wav' created: samp.rate=16000, bufsize=4KB

12-17 15:31:26.957: D/libpjsip(5332): 15:31:26.963    pjsua_aud.c  ......Recorder created, id=0, slot=3

12-17 15:31:26.957: D/libpjsip(5332): 15:31:26.963    pjsua_aud.c  .....Creating recorder /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_outgoing.wav..

12-17 15:31:26.957: D/libpjsip(5332): 15:31:26.971   wav_writer.c !......File writer '/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_outgoing.wav' created: samp.rate=16000, bufsize=4KB

12-17 15:31:26.957: D/libpjsip(5332): 15:31:26.971    pjsua_aud.c  ......Recorder created, id=1, slot=4

12-17 15:31:26.967: D/PjService(5332): Recorder: Starting recording: [I@405d7800 for 2

12-17 15:31:26.967: D/libpjsip(5332): 15:31:26.972   conference.c  ......Port 2 (sip:333******) transmitting to port 3 (/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_incoming.wav)

12-17 15:31:26.967: D/libpjsip(5332): 15:31:26.972   conference.c  ......Port 0 (Android Audio) transmitting to port 4 (/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153126_outgoing.wav)

12-17 15:31:43.914: D/libpjsip(5332): 15:31:43.927    pjsua_aud.c !.....Destroying recorder 0..

12-17 15:31:43.914: D/libpjsip(5332): 15:31:43.927    pjsua_aud.c  .....Destroying recorder 1..

12-17 15:31:43.914: D/PjService(5332): Stop recorder com.csipsimple.pjsip.PjSipService$RecorderCallStereo@4059f648

12-17 15:31:44.975: I/libpjsip(5332): 15:31:44.989 android_jni_de !>> Record thread stopped

12-17 15:31:46.516: I/libpjsip(5332): 15:31:46.527 android_jni_de  .---> Released recorder

12-17 15:31:58.658: I/libpjsip(5332): 15:31:58.665 android_jni_de  ...Min record buffer 6144

12-17 15:31:58.678: I/libpjsip(5332): 15:31:58.689 android_jni_de !<< Enter recorder thread

12-17 15:32:01.000: D/PjService(5332): Out dir /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_incoming.wav

12-17 15:32:01.000: D/PjService(5332): Out dir /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_outgoing.wav

12-17 15:32:01.000: D/libpjsip(5332): 15:32:01.005    pjsua_aud.c !.....Creating recorder /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_incoming.wav..

12-17 15:32:01.010: D/libpjsip(5332): 15:32:01.015   wav_writer.c  ......File writer '/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_incoming.wav' created: samp.rate=16000, bufsize=4KB

12-17 15:32:01.010: D/libpjsip(5332): 15:32:01.015    pjsua_aud.c  ......Recorder created, id=0, slot=3

12-17 15:32:01.010: D/libpjsip(5332): 15:32:01.015    pjsua_aud.c  .....Creating recorder /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_outgoing.wav..

12-17 15:32:01.020: D/libpjsip(5332): 15:32:01.024   wav_writer.c !......File writer '/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_outgoing.wav' created: samp.rate=16000, bufsize=4KB

12-17 15:32:01.020: D/libpjsip(5332): 15:32:01.024    pjsua_aud.c  ......Recorder created, id=1, slot=4

12-17 15:32:01.020: D/PjService(5332): Recorder: Starting recording: [I@4057fb08 for 3

12-17 15:32:01.020: D/libpjsip(5332): 15:32:01.025   conference.c  ......Port 2 (sip:333******) transmitting to port 3 (/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_incoming.wav)

12-17 15:32:01.020: D/libpjsip(5332): 15:32:01.025   conference.c  ......Port 0 (Android Audio) transmitting to port 4 (/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153201_outgoing.wav)

12-17 15:32:07.757: D/libpjsip(5332): 15:32:07.764    pjsua_aud.c !Destroying recorder 0..

12-17 15:32:07.757: D/libpjsip(5332): 15:32:07.764    pjsua_aud.c  Destroying recorder 1..

12-17 15:32:07.757: D/PjService(5332): Stop recorder com.csipsimple.pjsip.PjSipService$RecorderCallStereo@40525c10

12-17 15:32:10.460: D/PjService(5332): Out dir /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_incoming.wav

12-17 15:32:10.460: D/PjService(5332): Out dir /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_outgoing.wav

12-17 15:32:10.460: D/libpjsip(5332): 15:32:10.466    pjsua_aud.c !Creating recorder /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_incoming.wav..

12-17 15:32:10.470: D/libpjsip(5332): 15:32:10.481   wav_writer.c  .File writer '/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_incoming.wav' created: samp.rate=16000, bufsize=4KB

12-17 15:32:10.470: D/libpjsip(5332): 15:32:10.481    pjsua_aud.c  .Recorder created, id=0, slot=3

12-17 15:32:10.470: D/libpjsip(5332): 15:32:10.481    pjsua_aud.c  Creating recorder /mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_outgoing.wav..

12-17 15:32:10.480: D/libpjsip(5332): 15:32:10.490   wav_writer.c  .File writer '/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_outgoing.wav' created: samp.rate=16000, bufsize=4KB

12-17 15:32:10.480: D/libpjsip(5332): 15:32:10.490    pjsua_aud.c  .Recorder created, id=1, slot=4

12-17 15:32:10.480: D/PjService(5332): Recorder: Starting recording: [I@4066ff78 for 3

12-17 15:32:10.480: D/libpjsip(5332): 15:32:10.490   conference.c  .Port 2 (sip:333******) transmitting to port 3 (/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_incoming.wav)

12-17 15:32:10.480: D/libpjsip(5332): 15:32:10.490   conference.c  .Port 0 (Android Audio) transmitting to port 4 (/mnt/sdcard/CSipSimple/records/_sip_333@******__12-12-17_153210_outgoing.wav)

12-17 15:32:15.444: D/libpjsip(5332): 15:32:15.453    pjsua_aud.c !Destroying recorder 0..

12-17 15:32:15.444: D/libpjsip(5332): 15:32:15.454    pjsua_aud.c  Destroying recorder 1..

12-17 15:32:15.444: D/PjService(5332): Stop recorder com.csipsimple.pjsip.PjSipService$RecorderCallStereo@4066ff50

12-17 15:32:17.436: I/libpjsip(5332): 15:32:17.442 android_jni_de !>> Record thread stopped

12-17 15:32:18.948: I/libpjsip(5332): 15:32:18.958 android_jni_de  .---> Released recorder

[Oleksii Vynogradov]

Hi.

i'm try to enable SILK support.

But i have just this list:

12-18 12:34:27.640: E/PjService(5687): Added codec speex/16000/1

12-18 12:34:27.640: E/PjService(5687): Added codec speex/8000/1

12-18 12:34:27.640: E/PjService(5687): Added codec speex/32000/1

12-18 12:34:27.640: E/PjService(5687): Added codec GSM/8000/1

12-18 12:34:27.660: E/PjService(5687): Added codec PCMU/8000/1

12-18 12:34:27.660: E/PjService(5687): Added codec PCMA/8000/1

12-18 12:34:27.660: E/PjService(5687): Added codec G722/16000/1

12-18 12:34:27.660: E/PjService(5687): Added codec AMR/8000/1

12-18 12:34:27.660: E/PjService(5687): Added codec ISAC/16000/1

12-18 12:34:27.660: E/PjService(5687): Added codec ILBC/8000/1

12-18 12:34:27.660: E/PjService(5687): Added codec G729/8000/1

inside initCodecs() function

                nbrCodecs = pjsua.codecs_get_nbr();

return not all list.

Please advise (this check was on alcatel one touch device)

[Dennis Guse]

Found it!

UAStateReceiver.on_call_media_state(...) might get call multiple times during call setup and thus if autorecord is enabled call PjSipService.startRecording(...) also multiple times.

In the original implementation startRecording() would check if a recording is already running and if yes ignore the call.

However, the incoming call_port might change during call setup and thus results in on_call_media_state-calls... For this reason on_call_media_state() always reconnects the incoming and outgoing ports...

I changed my recorder implementation, so that it simply stops the current recording and start recording again (with new files, if the difference is more than one second):

* This will destroy old recorders

* If the time difference between two startRecording calls is less than one second overwrite old files

* Create new recorders

* Connect new recorders

Works well...

[Srimathy]

Hi,

I am trying to create the recorder using the pjsip jni wrappers.

The function pjsua.recorder_create takes a SWIGTYPE_p_int  argument for holding the recorder id. THe SWIGTYPE_p_int constructor is also protected. In that case how to get the recorder id? The program doesnt accept int or int[] argument. It gives compilation error.How to resolve this.

thanks,

srimathy