How to add custom media codec to pjsip

How to add custom media codec to pjsip

Hi, in this post, I will show you how to add a custom codec to pjsip 2.2.1

Introduction:

http://www.pjsip.org
pjsip is a very strong and widely use in voip. you can combine pjsip with kamailio, stund, turn server, freeswitch to build chat application like Skype with many well feature like message, call, conversation.v.v.
http://www.opus-codec.org
opus is a lightweight media codec with very good quality but also cost minimum bandwidth (you can customize it bitrate yourself)
to port opus codec you should choose a codec and follow it config (gsm for example)
(This tutorial I read from http://www.piemontewireless.net/, but now, this link is no longer exist)
A little bit you need to know about pjsip build system (make file)
pjsip use a set of make files to build, if you familiar with gnumake, it’s very easy to understand pjsip. but if you are not, here are some tips to help you easily maintain pjsip and solved bug when buid
  • notice about “build” dir, it’s contain make file you need to config
  • file *.in is input file, config init, run config again and it will generate *.mak file base on *.in
  • autoconf acconfigure.ac > acconfigure

Porting opus with pjsip interface

1. Add opus source code:

1.1 you must write api for opus to work with pjmedia, ex: (view resource file for more)
PJ_DECL(pj_status_t) pjmedia_codec_opus_init(pjmedia_endpt *endpt);
PJ_DECL(pj_status_t) pjmedia_codec_opus_deinit(void);
1.2 after write opus.h and opus.c file for pjmedia. put it in:
header file: pjmedia/include/pjmedia-codec
source file: pjmedia/src/pjmedia-codec
1.3 add opus source to third_party.
1.4 int third_party/build create new opus dir, write Makefile and config for all c file in opus

2. Config pjsip build system, find appropriate place to port this code, often after g7221 config

2.1 aconfigure.ac
line 846:
dnl # Include opus codec
AC_SUBST(ac_has_opus_codec)
AC_ARG_ENABLE(opus_codec,
AC_HELP_STRING([--enable-opus-codec],
[Enable OPUS support (derived from ITU implementation).]),
[ac_has_opus_codec=1]
AC_DEFINE(PJMEDIA_HAS_OPUS_CODEC,1)
AC_MSG_RESULT([Checking if opus codec is disabled...no]),
[ac_has_opus_codec=]
AC_MSG_RESULT([Checking if opus codec is disabled...yes])
)
2.2 pjsip/build.mak.in
ifeq (@ac_has_opus_codec@,1)
APP_THIRD_PARTY_LIB_FILES += $(PJ_DIR)/third_party/lib/libopuscodec-$(LIB_SUFFIX)
ifeq ($(PJ_SHARED_LIBRARIES),)
APP_THIRD_PARTY_LIBS += -lopuscodec-$(TARGET_NAME)
else
APP_THIRD_PARTY_LIBS += -lopuscodec
APP_THIRD_PARTY_LIB_FILES += $(PJ_DIR)/third_party/lib/libopuscodec.$(SHLIB_SUFFIX).$(PJ_VERSION_MAJOR) $(PJ_DIR)/third_party/lib/libopuscodec.$(SHLIB_SUFFIX)
endif
endif
2.3 pjmedia/build/os-auto.mak.in
AC_HAS_OPUS_CODEC=@ac_has_opus_codec@
...
ifeq ($(AC_HAS_OPUS_CODEC),1)
export CFLAGS += -I$(THIRD_PARTY)/opus/include -DPJMEDIA_HAS_OPUS_CODEC=1
export CODEC_OBJS += opus.o
endif
...
2.4 third_party/build/os-auto.mak.in
ifeq (@ac_has_opus_codec@,1)
DIRS += opus
endif
2.5 pjmedia/include/pjmedia-codec.h
#include <pjmedia-codec/opus.h>
2.6 pjmedia/include/pjmedia/codec.h
PJMEDIA_RTP_PT_OPUS = 20
2.7 pjmedia/include/pjmedia-codec/config.h
/**
 * Unless specified otherwise, OPUS codec is not included by default.
 */
#ifndef PJMEDIA_HAS_OPUS_CODEC
#   define PJMEDIA_HAS_OPUS_CODEC    1
#endif
2.8 pjmedia/include/pjmedia-codec/config_auto.h
/* OPUS codec */
#ifndef PJMEDIA_HAS_OPUS_CODEC
#define PJMEDIA_HAS_OPUS_CODEC 1
#endif
2.9 pjmedia/include/pjmedia-codec/config_auto.h.in
/* OPUS codec */
#ifndef PJMEDIA_HAS_OPUS_CODEC
#undef PJMEDIA_HAS_OPUS_CODEC
#endif
2.10 pjmedia/src/pjmedia-codec/audio_codecs.c
#if PJMEDIA_HAS_OPUS_CODEC
    status = pjmedia_codec_opus_init(endpt);
    if (status != PJ_SUCCESS) {
        return status;
    }
#endif /* PJMEDIA_HAS_OPUS_CODEC */

3. run config and build pjsip

autoconf acconfigure.ac > acconfigure
configure --with-ssl --enable-opus-codec
make-dep
make

Comments

  1. hi, thanks for this tutorial.. i'm following your tutorial,
    while compiling (when using make) i'm getting this error message

    Undefined symbols for architecture armv7:
    "_pjmedia_codec_bpus120_init", referenced from:
    _pjmedia_codec_register_audio_codecs in libpjmedia-codec-arm-apple-darwin9.a(audio_codecs.o)
    "_pjmedia_codec_bpus_init", referenced from:
    _pjmedia_codec_register_audio_codecs in libpjmedia-codec-arm-apple-darwin9.a(audio_codecs.o)
    "_pjmedia_codec_opus_init", referenced from:
    _pjmedia_codec_register_audio_codecs in libpjmedia-codec-arm-apple-darwin9.a(audio_codecs.o)
    ld: symbol(s) not found for architecture armv7
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    make[2]: *** [../bin/pjsua2-test-arm-apple-darwin9] Error 1
    make[1]: *** [pjsua2-test-arm-apple-darwin9] Error 2
    make: *** [all] Error 1

    i think i'm missing something.. what is the reason for this error??

    ReplyDelete
  2. this is missing define for these function
    => make sure you have copy opus.c bpus.c into pjproject/pjmedia/src/pjmedia-codec

    ReplyDelete
  3. I have source files in that directory, but still i am getting same error. what are the other possibility for this error. please tell me if you know..

    ReplyDelete
  4. i am getting .a files in third-party/lib folder. do i need to add those(third party/lib) .a files to any library of pjsip(pjmedia) to complete compilation??
    i am still getting same error.. what is the reason please tell me if you know anything.

    ReplyDelete
  5. no, you don't need add *.a file (this is static library, result of build)

    I've just updated tut, remove bpus for simple and correct typo "configure --with-ssl" instead of "configure --with-tls"

    I have upload a completed pjsip with ported opus (on head of this tut), check this and compare with original pjsip or your pjsip project to see differents

    Good luck :)

    (beside, are you build pjsip on mac or linux, or windows ? build in mac and ubuntu is a bit simpler than windows)

    ReplyDelete
    Replies
    1. Hello Guys I'm working on the same problem. I have compiled the pjsip with opus but have some linker errors when trying to init the opus and change some parameters. Do you by any chance still have that completed PJSIP with ported opus so i can compare with my project and maybe find my mistakes ? I would really appreciate that. Thanks

      Delete
    2. Hi Alek, unfortunately, this is 3 years ago, I don't keep the source code. I changed the company, so that I cannot copy any source code with me :) But, the process of porting Opus with PJSIP is just like provide to PJSIP the information it need. So, I think If you can relax a little bit, go into details, check every step you do again, you can fix it.
      Sorry for not helping much

      Delete
    3. Hi Tran tnx anyway. I will follow your steps described. I have just one question. Do you by any means remember what this step means :

      + int third_party/build create new opus dir, write Makefile and config for all c file in opus

      I don't get what am i suppose to do here. The rest is clear.
      Thank you

      Delete
    4. Basically, you create a directory name "opus" in "third_party/build", add opus source code and write Makefile, which is simple list all the *.c files & add some config flags. Sry, I don't remember the details, but you can reference from one of custom codec (speex or G729), in pjsip source code they already custom one codec for example.

      Delete
  6. Thanks tran,
    It is working for me. i compiled for x86_64 and arm7,arm7s,arm64 all are working fine with your dropbox file. but when i am compiling arm7 and arm7s with ssl support it is not working it is throwing on error
    here is the error
    (#define PJ_CONFIG_IPHONE 1 is uncommented in cofig_site.h file)

    $ ./configure-ipone --with-ssl --enable-opus-codec

    Undefined symbols for architecture armv7:
    "_pj_ssl_cert_load_from_files", referenced from:
    _pjsip_tls_transport_start2 in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_close", referenced from:
    _on_accept_complete in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _lis_destroy in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _tls_destroy in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_create", referenced from:
    _pjsip_tls_transport_start2 in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _lis_create_transport in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_get_info", referenced from:
    _pjsip_tls_transport_start2 in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _on_accept_complete in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _lis_create_transport in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _tls_init_shutdown in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _on_connect_complete in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_get_user_data", referenced from:
    _on_accept_complete in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _on_data_read in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _on_data_sent in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _on_connect_complete in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_send", referenced from:
    _tls_send_msg in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _tls_keep_alive_timer in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _on_connect_complete in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_set_certificate", referenced from:
    _pjsip_tls_transport_start2 in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _lis_create_transport in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_set_user_data", referenced from:
    _on_accept_complete in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    _lis_create_transport in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_start_accept", referenced from:
    _pjsip_tls_transport_start2 in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_start_connect", referenced from:
    _lis_create_transport in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    "_pj_ssl_sock_start_read2", referenced from:
    _tls_start_read in libpjsip-arm-apple-darwin9.a(sip_transport_tls.o)
    ld: symbol(s) not found for architecture armv7
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    make[2]: *** [../bin/pjsua2-test-arm-apple-darwin9] Error 1
    make[1]: *** [pjsua2-test-arm-apple-darwin9] Error 2
    make: *** [all] Error 1

    but it is working without tis (if i removes tls from pjlib/include/pj/config_site.h) it is working fine.

    what may be the reason for this error.. anyway one again thank you for help.

    but i still need to find differences between my file and your file.

    :))

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. Hi tran,
    i found the mistakes in my pjproject, i didn't configured aconfigure file. that's why pjsip is giving error (i.e my first comment).
    By comparing your pjproject with my pjproject i found that configuration mistake. Now i am able to configure PJSIP with opus. But ssl support is not working it just giving above error.

    Anyway, Thank you very much for your support .. thanks alot tran.,

    ::))

    ReplyDelete
  9. Hello Sir, Can you help me how I can implement PJSIP in ios project.

    ReplyDelete

Post a Comment

Popular posts from this blog

Level up your log in React-Native with Reactotron