Syncronization audio and video

Syncronization audio and video

I need to display stream video using MediaElement in Windwso Phone application.

I'm getting from web-service a stream that contains frames in H264 format AND raw-AAC bytes (strange, but ffmpeg can parse with -f ac3 parameter only).

So, if try to play only one of stream (audio OR video) it plays nice. But I have issues when try it both.

For example, if I report video sample without timestamp and report audio with timestamp, my video plays 3x-5x faster then I need.

MediaStreamSample msSamp = new MediaStreamSample(                     _videoDesc,                     vStream,                     0,                     vStream.Length,                     0,                     _emptySampleDict);                  ReportGetSampleCompleted(msSamp); 

From my web-service I getting a DTS and PTS for video and audio frames in following format:

120665029179960

but when I set it for sample, my audio stream plays too slow and with delays.

Timebase is 90khz.

So, could someone tell me how I can resolve it? Maybe I should calculate others timestamps for samples? If so, show me the way, please.

Thanks.

Answers & Comments...

Answer: 1

Okay, I solved it.

So, what I need to do for sync A/V:

Calculate right timestamps for each video and audio frames using framerate.

For example, for video I have 90 kHz and for audio 48 kHz and 25 frames per second - my frame increments will be:

_videoFrameTime = (int)TimeSpan.FromSeconds((double)0.9 / 25).Ticks; _audioFrameTime = (int)TimeSpan.FromSeconds((double)0.48 / 25).Ticks; 

And now we should add these values for each sample:

private void GetAudioSample() { ...  /* Getting sample from buffer */ MediaStreamSample msSamp = new MediaStreamSample(                     _audioDesc,                     audioStream,                     0,                     audioStream.Length,                     _currentAudioTimeStamp,                     _emptySampleDict);  _currentAudioTimeStamp += _audioFrameTime;  ReportGetSampleCompleted(msSamp); } 

For gettign video frame method will be the same with a _videoFrameTime incrementation instead.

Hope this will be helpfull for someone.

Roman.

by : Roman Golenokhttp://stackoverflow.com/users/762383

---