FFmpeg Tutorial (1)

-
FFmpeg is a leading source for encode/decode video, audio and widely use
But,... the only problem is ffmpeg nearly don't have enough documents

I'm currently working with ffmpeg and live streaming, And I'll note here some experience I meet while working and investigate ffmpeg :)

OK, first, this is some useful link:

https://www.ffmpeg.org/

http://dranger.com/ffmpeg/ <- some code on this site is depricated, but this still useful to understand how ffmpeg work, how to display movie, and some issues

https://github.com/chelyaev/ffmpeg-tutorial/issues/11 <- this is same as dranger but updated sourcecode (still have one problem with stereo planar audio, you will hear noise instead of correct audio, using this to resample: https://github.com/illuusio/ffmpeg-tutorial)

And this is some term you should to know 
(google it for better result if you don't understand what I explain:D)


> streaming
+ split video file into frames and send to client buffer

> terms:
+ live streaming: one record and broadcast, one received and display in real time
+ rtmp: real time messaging protocol. it’s tcp-base protocol
+ rtmps: rtmp with tls/ssl connection
+ rtmpe: rtmp encrypted using adobe’s own security
+ rtmp send smooth because it’s send fragments, fragment size is negociate dynamically between server and client. default for audio is 64 and video is 128

> audio:
+ in real world (analog), audio have frequency and volumes. in digital world, we using:
+ sample rate (how frequency sound record) to represented frequency. the higher sample rate, the more accurate the sound describe.
=> how often or how much the sound is describe
+ bit rate (how good sound record) to represented volume. the higher bitrate, the more accuracy we describe how loud the volume is.
=> how well the sound  is describe
+ bit depth: mean how well we describe the volume, i.e: 16bit, mean we have 65536 volumes level to choose
==> when going in audio devices, the audio processor base on sample rate and bit rate to re-produce the analog signal (we can hear digital signal)
(So, how to record sound better ? is improve bitrate or samplerate ? as mention above, sound is both frequency and volume, so, keep the balance of them is the best. Usually using 44.1Khz/16bit, 44.1 is from  Nyquist–Shannon sampling theorem so 44.1/2 = 20.5Khz - human maximum freq)

> audio in ffmpeg:
+ ffmpeg supply samples, but it’ll not give partial sample (even stereo).

> video synced
+ DTS: Decoding Time Stamp => when we need to decode frame
+ PTS: Presentation Time Stamp => when we need to display frame
=> so the decoding order and display order of frames are not identical. then, we need to know our newly decoded frame PTS to display it right time
+ How video are stored: I-Intra, B-Bidirectional, P-Predicted frames
+ I frame contain full image.
+ P frame depend on I frame and previous P frames
+ B frame depend on both previous and next frames

=> FFmpeg reorder the packets so that the DTS of the packet being processed by avcodec_decode_video() will always be the same as the PTS of the frame it returns

I'm working with ffmpeg on MacOS and iOS. So, next post I'll note some build, config project, .v.v and how to using cocoa framework to display raw frame or playing raw audio samples :)

Comments

Post a Comment

Popular posts from this blog

How to add custom media codec to pjsip

-
How to add custom media codec to pjsip Hi, in this post, I will show you how to add a custom codec to pjsip 2.2.1 Introduction: http://www.pjsip.org pjsip is a very strong and widely use in voip. you can combine pjsip with kamailio, stund, turn server, freeswitch to build chat application like Skype with many well feature like message, call, conversation.v.v. http://www.opus-codec.org opus is a lightweight media codec with very good quality but also cost minimum bandwidth (you can customize it bitrate yourself) to port opus codec you should choose a codec and follow it config (gsm for example) (This tutorial I read from http://www.piemontewireless.net/ , but now, this link is no longer exist) A little bit you need to know about pjsip build system (make file) pjsip use a set of make files to build, if you familiar with gnumake, it’s very easy to understand pjsip. but if you are not, here are some tips to help you easily maintain pjsip and solved bug when buid notice abou...
Read more

Level up your log in React-Native with Reactotron

-
Image
A Log - Many Logs Log is the essential part of programming. If you ever worked with mobile programming or react-native, you will find that log is a the essential for monitor your app. In this post, I just want to a share a little bit trick to log in react-native app :) Console.log is bad!!! React-Native come with the default console.log to be able to see the log messages. You have to turn on Debug JS Remotely and open Inspector in chrome to view the message. This is bad because Debug mode is very slow. If you write a long message, it can crash your app The app continues warning about keeping chrome on foreground Reactotron come to rescue!!! https://github.com/infinitered/reactotron Yes!!! with reactotron, you DON’T NEED to turn-on Debug JS to be able to see log, and the log is also FASTER btw. The example below show how to configure Reactotron with redux for simulator and real device import Reactotron from 'reactotron-react-native'; import { reactotronRedux }...
Read more

iOS Push Notification with HTTP2

-
Setup iOS APNS with HTTP/2 Overview: There're one issue while sending iOS notification to multiple devices with old method (send by ftp write stream). Let say you send to 100 devices, ftp write work likes this: 1. connect to APNS Server 2. for each device: send notification by using device_token & payload 3. disconnect In step 2, if there're 1 invalid token (there're many reason make a token invalid: expired, wrong cert, wrong format...etc) the connection will be block without status from APNS Server, if you want to check the status, you have to wait at least 0.5s-1s and using ftp-read to read the status. It's mean you have to check status after every ftp-write and re-connect again if there're an invalid token.  So, to send to 100 devices, you need 50 seconds, that's too long. At WWDC 2015, Apple has introduce a  new notification api by using HTTP/2 This api allow you to send notification as a post request or curl and get the result ...
Read more