[Psi-Devel] Jingle A/V

[Justin Karneges]

Tonight I committed the first round of our new Jingle audio/video feature. It
is intended to replace our earlier libjingle-based implementation. I made a
successful test call to a development version of Pidgin, and I'm still kind
of shocked that it worked. :) While Pidgin and Psi are sharing GStreamer
code, the XMPP/ICE layers are not shared, so this is was an excellent interop
test.

It is VERY experimental. Currently only audio is supported, and Psi has to
start the call. You might have to restart Psi to make a call if one fails.
There's also 80 ways to crash it, there's no error checking, Psi doesn't
gracefully terminate calls, etc. Tomorrow (and in time for the XMPP Summit),
my goal is to make the code slightly less craptastic, and perhaps support
video.

To use it, you need a separate checkout of PsiMedia. Then, add
PSIMEDIA_DIR=/path/to/psimedia in Psi's conf.pri file. GStreamer
dependencies are hard-coded. Edit psimedia/gstprovider/gstconf.pri to change
them. With the main window focused, press ctrl+shift+N to configure your
devices. To use STUN, set the PSI_STUNADDR environment variable to the IP
address of a STUN server.

Have fun,
-Justin
_______________________________________________
Psi-Devel mailing list
Psi-...@lists.affinix.com
http://lists.affinix.com/listinfo.cgi/psi-devel-affinix.com

[James Bunton]

On Wed, Feb 04, 2009 at 10:46:22PM -0800, Justin Karneges wrote:
> Tonight I committed the first round of our new Jingle audio/video feature. It
> is intended to replace our earlier libjingle-based implementation. I made a
> successful test call to a development version of Pidgin, and I'm still kind
> of shocked that it worked. :) While Pidgin and Psi are sharing GStreamer
> code, the XMPP/ICE layers are not shared, so this is was an excellent interop
> test.

Wow, great work! :)

Thanks.

---

James

[Justin Karneges]

On Wednesday 04 February 2009 22:46:22 Justin Karneges wrote:
> It is VERY experimental. Currently only audio is supported, and Psi has to
> start the call. You might have to restart Psi to make a call if one fails.
> There's also 80 ways to crash it, there's no error checking, Psi doesn't
> gracefully terminate calls, etc. Tomorrow (and in time for the XMPP
> Summit), my goal is to make the code slightly less craptastic, and perhaps
> support video.

I believe I've resolved most of the problems here. You can now make multiple
calls in succession without having to restart, and call reject/termination
works properly. I've still not tested Psi<->Psi (all of my tests have been
with Maiku and his Pidgin dev version) but it ought to work. We've tested
both PCMU and Speex codecs.

I'm deferring video support until later. For independent one-way
transmissions, PsiMedia works great for both audio and video, however for
two-way transmission (needed for Jingle) it's still mostly broken. It was
difficult working around these shortcomings to even get Speex to work. So I
think I need to complete PsiMedia before we bother about video in Psi.

[Chris Hills]

Great work! Unfortunately I was not able to compile it. Following is the
output (linux-g++-32):-

$ ./configure-jingle --prefix=/home/chaz6/Programs/Psi/git
--enable-jingle --disable-tests
--with-ortp-inc=/home/chaz6/Libraries/ortp/0.15.0/include
--with-ortp-lib=/home/chaz6/Libraries/ortp/0.15.0/lib
Configuring Psi ...
Verifying Qt 4 build environment ... ok
Checking for Qt >= 4.2.3 ... yes
Checking for bundled QCA 2.0 ... yes
Checking for OpenSSL ... yes
Checking for zlib ... yes
Checking for certstore ... /usr/share/ssl/cert.pem
Checking for QDBUS ... yes
Checking for the XScreenSaver extension ... yes
Checking for Aspell support ... yes
Checking for gethostbyname_r() ... yes
Checking for Google File Transfer ... no
Checking for Jingle ...

!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!
LIBJINGLE SUPPORT IS STILL EXPERIMENTAL !!!
USE AT YOUR OWN RISK !!!
yes
Checking for Plugins ... no

Good, your configure finished. Now run /usr/bin/gmake.

$ make
cd third-party/libjingle/ && make -f Makefile
make[1]: Entering directory `/home/chaz6/git/psi/third-party/libjingle'
/usr/local/Trolltech/Qt-4.5.0/bin/qmake -unix -o Makefile libjingle.pro
make[1]: Leaving directory `/home/chaz6/git/psi/third-party/libjingle'
make[1]: Entering directory `/home/chaz6/git/psi/third-party/libjingle'
g++ -c -pipe -O2 -fPIC -Wall -W -D_REENTRANT -DQCA_NO_PLUGINS -DOSSL_097
-DHAVE_OPENSSL -DQCA_SYSTEMSTORE_PATH=\"/usr/share/ssl/cert.pem\"
-DHAVE_XSS -DHAVE_ASPELL -DHAVE_GETHOSTBYNAME_R -DHAVE_SPEEX -DHAVE_ORTP
-DHAVE_ALSA_ASOUNDLIB_H -D__ALSA_ENABLED__ -DHAVE_GLIB -DHAVE_CONFIG
-DPOSIX -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED
-I/usr/local/Trolltech/Qt-4.5.0-20090122/mkspecs/linux-g++ -I.
-I/usr/local/Trolltech/Qt-4.5.0-20090122/include/QtCore
-I/usr/local/Trolltech/Qt-4.5.0-20090122/include/QtGui
-I/usr/local/Trolltech/Qt-4.5.0-20090122/include -I/usr/include/speex
-I../../../../Libraries/ortp/0.15.0/include -I/usr/include/glib-2.0
-I/usr/lib/glib-2.0/include -I. -Italk/third_party/mediastreamer -I. -I.
-o .obj/asynctcpsocket.o talk/base/asynctcpsocket.cc
In file included from ./talk/base/asyncpacketsocket.h:31,
from ./talk/base/asynctcpsocket.h:31,
from talk/base/asynctcpsocket.cc:31:
./talk/base/asyncsocket.h:81: warning: unused parameter ‘socket’
./talk/base/asyncsocket.h:82: warning: unused parameter ‘socket’
./talk/base/asyncsocket.h:83: warning: unused parameter ‘socket’
./talk/base/asyncsocket.h:84: warning: unused parameter ‘socket’
talk/base/asynctcpsocket.cc: In member function ‘virtual int
cricket::AsyncTCPSocket::Send(const void*, size_t)’:
talk/base/asynctcpsocket.cc:84: error: ‘memcpy’ was not declared in this
scope
talk/base/asynctcpsocket.cc: In member function ‘int
cricket::AsyncTCPSocket::SendRaw(const void*, size_t)’:
talk/base/asynctcpsocket.cc:114: error: ‘memcpy’ was not declared in
this scope
talk/base/asynctcpsocket.cc: In member function ‘virtual void
cricket::AsyncTCPSocket::ProcessInput(char*, size_t&)’:
talk/base/asynctcpsocket.cc:128: error: ‘memcpy’ was not declared in
this scope
talk/base/asynctcpsocket.cc:138: error: ‘memmove’ was not declared in
this scope
talk/base/asynctcpsocket.cc: In member function ‘int
cricket::AsyncTCPSocket::Flush()’:
talk/base/asynctcpsocket.cc:155: error: ‘memmove’ was not declared in
this scope
talk/base/asynctcpsocket.cc: At global scope:
talk/base/asynctcpsocket.cc:160: warning: unused parameter ‘socket’
talk/base/asynctcpsocket.cc: In member function ‘void
cricket::AsyncTCPSocket::OnReadEvent(cricket::AsyncSocket*)’:
talk/base/asynctcpsocket.cc:170: error: ‘strerror’ is not a member of ‘std’
talk/base/asynctcpsocket.cc: At global scope:
talk/base/asynctcpsocket.cc:164: warning: unused parameter ‘socket’
talk/base/asynctcpsocket.cc:185: warning: unused parameter ‘socket’
talk/base/asynctcpsocket.cc:193: warning: unused parameter ‘socket’
make[1]: *** [.obj/asynctcpsocket.o] Error 1
make[1]: Leaving directory `/home/chaz6/git/psi/third-party/libjingle'
make: *** [sub-third-party-libjingle-make_default] Error 2

[Remko Tronçon]

> Great work! Unfortunately I was not able to compile it. Following is the
> output (linux-g++-32):-

configure-jingle is the libjingle version. Use the regular configure.

cheers,
Remko

[Chris Hills]

Remko Tronçon wrote:
> configure-jingle is the libjingle version. Use the regular configure.

Regular configure does not have --enable-jingle.

[Remko Tronçon]

> Regular configure does not have --enable-jingle.

You don't need it, Jingle is enabled by default.

cheers,
Remko

[Caemirr]

Remko Tronçon wrote:

>> Regular configure does not have --enable-jingle.
>
> You don't need it, Jingle is enabled by default.
>
> cheers,
> Remko

Hi all,

Where can i get psimedia please ?

[Justin Karneges]

On Friday 06 February 2009 09:52:53 Caemirr wrote:
> Remko Tronçon wrote:
> >> Regular configure does not have --enable-jingle.
> >
> > You don't need it, Jingle is enabled by default.
> >
> > cheers,
> > Remko
>
> Hi all,
>
> Where can i get psimedia please ?

svn checkout https://delta.affinix.com/svn/trunk/psimedia

For best results you should probably build it first, then go build Psi.

-Justin

[Matthieu Volat]

Hello,

I wanted to try it on freebsd, but there were some compilation errors
in the Psimedia module due to the use of parameter (NULL) in
GST_ELEMENT_ERROR and GST_ELEMENT_WARNING : the compiler complained
about the absence of non zero-length string so I put (" ") instead of
(NULL) and it compiled fine (I'm sure anybody will come up with a
better solution).

The demo launched fine, but now, I'm unable to select any audio
input/output. I suspect there is a problem with my gstreamer
installation (I tried using gstreamer as my phonon backend in kde4 ans
it seems to work). I also have all the required plugins...

Does anybody have an idea? Maybe it's just my nasty hacks (did those
functions provided more than debug about errors and warning?)...

Cheers,

- Matthieu

[Justin Karneges]

On Wednesday 18 February 2009 23:18:39 Matthieu Volat wrote:
> Hello,
>
> I wanted to try it on freebsd, but there were some compilation errors
> in the Psimedia module due to the use of parameter (NULL) in
> GST_ELEMENT_ERROR and GST_ELEMENT_WARNING : the compiler complained
> about the absence of non zero-length string so I put (" ") instead of
> (NULL) and it compiled fine (I'm sure anybody will come up with a
> better solution).

What GStreamer version?

> The demo launched fine, but now, I'm unable to select any audio
> input/output. I suspect there is a problem with my gstreamer
> installation (I tried using gstreamer as my phonon backend in kde4 ans
> it seems to work). I also have all the required plugins...

Probably because OSS support is disable in PsiMedia, in favor of ALSA. You
can attempt to enable this by editing psimedia/gstprovider/devices.cpp, and
making the following changes:

1) element_should_use_probe() contains a blacklist of elements that should
not use GStreamer's own device detection. Make sure "osssrc" and "osssink"
are in this list (you can also try without this modification, I'd be
interested to hear if it works).
2) element_name_for_driver() should have a 'driver == "oss"' section, where
the name is osssink for AudioOut and osssrc for AudioIn.
3) devices_list() should put "oss" in the drivers list, for AudioOut and
AudioIn.

-Justin

[Matthieu Volat]

2009/2/19 Justin Karneges <justi...@affinix.com>:

> On Wednesday 18 February 2009 23:18:39 Matthieu Volat wrote:
>> Hello,
>>
>> I wanted to try it on freebsd, but there were some compilation errors
>> in the Psimedia module due to the use of parameter (NULL) in
>> GST_ELEMENT_ERROR and GST_ELEMENT_WARNING : the compiler complained
>> about the absence of non zero-length string so I put (" ") instead of
>> (NULL) and it compiled fine (I'm sure anybody will come up with a
>> better solution).
>
> What GStreamer version?

I'm using 0.10.22, the current version in ports -- which was added 25
january 2009.

>
>> The demo launched fine, but now, I'm unable to select any audio
>> input/output. I suspect there is a problem with my gstreamer
>> installation (I tried using gstreamer as my phonon backend in kde4 ans
>> it seems to work). I also have all the required plugins...
>
> Probably because OSS support is disable in PsiMedia, in favor of ALSA. You
> can attempt to enable this by editing psimedia/gstprovider/devices.cpp, and
> making the following changes:
>
> 1) element_should_use_probe() contains a blacklist of elements that should
> not use GStreamer's own device detection. Make sure "osssrc" and "osssink"
> are in this list (you can also try without this modification, I'd be
> interested to hear if it works).
> 2) element_name_for_driver() should have a 'driver == "oss"' section, where
> the name is osssink for AudioOut and osssrc for AudioIn.
> 3) devices_list() should put "oss" in the drivers list, for AudioOut and
> AudioIn.

I have tried with all of theses (but I will try later without 1,
promise ;)), and between my freebsd computer and a linux (using alsa).

I can configure my devices which are correctly detected, but using
microphones between the two didn't work (no webcam atm, I'll test
later) although the microphones works locally... Yet the Linux pc was
able to stream an ogg file to the freebsd one, while the contrary
didn't work (I did restart the demo application on each machine
between each test).

I'll try a few more times, especially after verifying the microphone
setup on the two machines.

Thanks for your help!

>
> -Justin
> _______________________________________________
> Psi-Devel mailing list
> Psi-...@lists.affinix.com
> http://lists.affinix.com/listinfo.cgi/psi-devel-affinix.com
>

-- Matthieu

[Justin Karneges]

On Wednesday 04 February 2009 22:46:22 Justin Karneges wrote:

> To use it, you need a separate checkout of PsiMedia. Then, add
> PSIMEDIA_DIR=/path/to/psimedia in Psi's conf.pri file. GStreamer
> dependencies are hard-coded. Edit psimedia/gstprovider/gstconf.pri to
> change them. With the main window focused, press ctrl+shift+N to configure
> your devices. To use STUN, set the PSI_STUNADDR environment variable to
> the IP address of a STUN server.

I've changed how this works. Psi now contains a copy of the PsiMedia
front-end library (just a few files, in src/psimedia), and is configured to
use the backend as a plugin rather than static. A/V call support is now
compiled in by default. No need to edit conf.pri.

For A/V calls to actually work, you need to build the psimedia plugin:
svn checkout https://delta.affinix.com/svn/trunk/psimedia
cd psimedia
qconf
./configure
make

(there should be no more need to edit gstconf.pri, as there's now a configure
script to set up gstreamer.)

This will produce the "libgstprovider.so" plugin. Set the PSI_MEDIA_PLUGIN
environment variable to the full path of this file.

Warning: for now, Psi will allow access to the A/V call stuff regardless of
whether or not you have the plugin. Using any of it without the plugin will
cause you to crash, so be careful. My eventual goal is to have the UI
dynamically adjust based on the availability of multimedia support.

[Justin Karneges]

On Wednesday 04 March 2009 23:38:53 Justin Karneges wrote:
> Warning: for now, Psi will allow access to the A/V call stuff regardless of
> whether or not you have the plugin. Using any of it without the plugin
> will cause you to crash, so be careful. My eventual goal is to have the UI
> dynamically adjust based on the availability of multimedia support.

I've done that now. The AVCALL definition is gone. Also, libjingle is
removed from git because we use GStreamer and Iris now.

There's a new PsiEvent, so calls can be queued just like other events.

What's left to do:
- Decide if/how avcall events/popups should be configured. We have these
very granular options for how each event type should be handled, and honestly
I think it's a bit silly. I didn't want to add to the clutter with more
options for avcall without talking about it first.
- Put the mic/camera configuration in the options window, and git rid of
ctrl-shift-N.
- Allow the user to set a STUN server in the account window.
- Allow the user to set a port range in the options window.
- Make the calldlg prettier.
- Make a new sound effect for incoming calls.
- Advertise audio or video via disco only if the user has proper devices?
- Reconcile with existing VoiceCallManager/Dlg code, if that makes sense.

I also still need to clean up the avcall core, as it is currently quite
brittle, as well as fixup video support.

[Justin Karneges]

On Wednesday 08 April 2009 01:01:42 Justin Karneges wrote:
> - Decide if/how avcall events/popups should be configured. We have these
> very granular options for how each event type should be handled, and
> honestly I think it's a bit silly. I didn't want to add to the clutter
> with more options for avcall without talking about it first.

I've decided avcall will do whatever regular messages do, rather than adding
new options.

> - Advertise audio or video via disco only if the user has proper devices?

Since it's not possible to deselect a device in the new UI (and I don't feel
like futzing with it to change this), for privacy reasons disco support will
always be advertised regardless of whether you have the devices.

> - Reconcile with existing VoiceCallManager/Dlg code, if that makes sense.

I'll look at it when I try to prettify the new dialog, but i don't think it
needs to be a task item.

> - Put the mic/camera configuration in the options window, and git rid of
> ctrl-shift-N.
> - Allow the user to set a STUN server in the account window.

Done.

> - Allow the user to set a port range in the options window.

For simplicity, I've changed "Data Transfer Port" to "Data Transfer Base
Port", and the range is hard-coded to 4 ports. By default, Psi will listen
on port 8010 for TCP and ports 8010-8013 for UDP. Note that the port
selection here doesn't actually work yet for avcall. I need to disable UDP
support in S5B (fun trivia: Psi has always listened on port 8010 UDP but does
nothing with it), and then make our ICE code use the port choice.