相比較視訊編碼，音訊編碼要簡單很多，主要就是將採集到的音訊源資料PCM編碼AAC.
MediaPlus中FFmpeg使用的是libfdk-aac編碼器，這裡有個問題需要注意下:FFmpeg已經廢棄了AV_SAMPLE_FMT_S16格式PCM編碼AAC，也就是說如果使用FFmpeg自帶的AAC編碼器，必須做音訊的重取樣（重取樣為:AV_SAMPLE_FMT_FLTP），否則AAC編碼是失敗的。
接下來，看下MediaPlus中是如何採集音訊與AAC編碼的.
在app.mobile.nativeapp.com.libmedia.core.streamer.RtmpPushStreamer的AudioThread獲取AudioRecord採集到的音訊資料並傳入底層：
class AudioThread extends Thread {
public volatile boolean m_bExit = false;
@Override
public void run() {
// TODO Auto-generated method stub
super.run();
int[] dataLength;
byte[] audioBuffer;
AudioCaptureInterface.GetAudioDataReturn ret;
dataLength = new int[1];
audioBuffer = new byte[m_aiBufferLength[0]];
while (!m_bExit) {
try {
Thread.sleep(1, 10);
if (m_bExit) {
break;
}
} catch (InterruptedException e) {
e.printStackTrace();
}
try {
ret = mAudioCapture.GetAudioData(audioBuffer,
m_aiBufferLength[0], dataLength);
if (ret == AudioCaptureInterface.GetAudioDataReturn.RET_SUCCESS) {
encodeAudio(audioBuffer, dataLength[0]);
}
} catch (Exception e) {
e.printStackTrace();
stopThread();
}
}
}複製程式碼具體AudioRecord採集具體實現是avcapture.jar包中，程式碼比較簡單，相關android音視訊採集初始化及API呼叫網上都有相關Demo，這裡不再贅述！
- encodeAudio(audioBuffer, dataLength[0]);將音訊資料傳入底層。
/** * 採集的PCM音訊資料 * * @param audioBuffer * @param length */ public void encodeAudio(byte[] audioBuffer, int length) { try { LiveJniMediaManager.EncodeAAC(audioBuffer, length); } catch (Exception e) { e.printStackTrace(); } }複製程式碼 - JNI層接收到PCM音訊資料，新增到AudioCapture同步佇列中：
JNIEXPORT jint JNICALL Java_app_mobile_nativeapp_com_libmedia_core_jni_LiveJniMediaManager_EncodeAAC(JNIEnv *env, jclass type, jbyteArray audioBuffer_, jint length) { if (audioCaptureInit && !isClose) { jbyte *audioSrc = env->GetByteArrayElements(audioBuffer_, 0); uint8_t *audioDstData = (uint8_t *) malloc(length); memcpy(audioDstData, audioSrc, length); OriginData *audioOriginData = new OriginData(); audioOriginData->size = length; audioOriginData->data = audioDstData; audioCapture->PushAudioData(audioOriginData); env->ReleaseByteArrayElements(audioBuffer_, audioSrc, 0); } return 0; }複製程式碼 - startPushStream的呼叫，會重置AudioCapture::ExitCapture=false;
資料才會被加入到audioCaputureframeQueue對列中./** * 開啟推流 * @param pushUrl * @return */ private boolean startPushStream(String pushUrl) { if (nativeInt) { int ret = 0; ret = LiveJniMediaManager.StartPush(pushUrl); if (ret < 0) { Log.d("initNative", "native push failed!"); return false; } return true; } return false; }複製程式碼
- 重置標記後，audioCaputureframeQueue.push將資料添中到佇列中.
int AudioCapture::PushAudioData(OriginData *originData) {
if (ExitCapture) {
return 0;
}
originData->pts = av_gettime();
LOG_D(DEBUG,"audio capture pts :%lld",originData->pts);
audioCaputureframeQueue.push(originData);
return 0;
}複製程式碼上面這些程式碼與視訊的處理方式都是一樣的流程，在呼叫app.mobile.nativeapp.com.libmedia.core.streamer.RtmpPushStreamer>>startPushStream()，已經開始往音訊佇列中新增資料，緊接著呼叫rtmpStreamer->StartPushStream() ，實際也就是開啟了音視訊的兩個編碼執行緒及推流,推流相關程式碼與視訊一致.
int RtmpStreamer::StartPushStream() {
videoStreamIndex = AddStream(videoEncoder->videoCodecContext);
audioStreamIndex = AddStream(audioEncoder->audioCodecContext);
pthread_create(&t3, NULL, RtmpStreamer::WriteHead, this);
pthread_join(t3, NULL);
VideoCapture *pVideoCapture = videoEncoder->GetVideoCapture();
AudioCapture *pAudioCapture = audioEncoder->GetAudioCapture();
pVideoCapture->videoCaputureframeQueue.clear();
pAudioCapture->audioCaputureframeQueue.clear();
if(writeHeadFinish) {
pthread_create(&t1, NULL, RtmpStreamer::PushAudioStreamTask, this);
pthread_create(&t2, NULL, RtmpStreamer::PushVideoStreamTask, this);
}else{
return -1;
}
return 0;
}複製程式碼- PushAudioStreamTask中從佇列中獲取資料編碼、推流.
rtmpStreamer->audioEncoder->EncodeAAC(&pAudioData);AAC編碼.
rtmpStreamer->SendFrame(pAudioData, rtmpStreamer->audioStreamIndex);推流(與視訊推流一致)
這裡說明下，音訊編碼前獲取編碼器及一些引數的指定：
libmedia/src/main/cpp/AudioEncoder.cpp是音訊編碼的核心類，int AudioEncoder::InitEncode() 方法封裝了音訊編碼器的初始化。
int AudioEncoder::InitEncode() {
std::lock_guard<std::mutex> lk(mut);
avCodec = avcodec_find_encoder_by_name("libfdk_aac");
int ret = 0;
if (!avCodec) {
LOG_D(DEBUG, "aac encoder not found!")
return -1;
}
audioCodecContext = avcodec_alloc_context3(avCodec);
if (!audioCodecContext) {
LOG_D(DEBUG, "avcodec alloc context3 failed!");
return -1;
}
audioCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
audioCodecContext->sample_fmt = AV_SAMPLE_FMT_S16;
audioCodecContext->sample_rate = audioCapture->GetAudioEncodeArgs()->sampleRate;
audioCodecContext->thread_count = 8;
audioCodecContext->bit_rate = 50*1024*8;
audioCodecContext->channels = audioCapture->GetAudioEncodeArgs()->channels;
audioCodecContext->frame_size = audioCapture->GetAudioEncodeArgs()->nb_samples;
audioCodecContext->time_base = {1, 1000000};//AUDIO VIDEO 兩邊時間基數要相同
audioCodecContext->channel_layout = av_get_default_channel_layout(audioCodecContext->channels);
outputFrame = av_frame_alloc();
outputFrame->channels = audioCodecContext->channels;
outputFrame->channel_layout = av_get_default_channel_layout(outputFrame->channels);
outputFrame->format = audioCodecContext->sample_fmt;
outputFrame->nb_samples = 1024;
ret = av_frame_get_buffer(outputFrame, 0);
if (ret != 0) {
LOG_D(DEBUG, "av_frame_get_buffer failed!");
return -1;
}
LOG_D(DEBUG, "av_frame_get_buffer success!");
ret = avcodec_open2(audioCodecContext, NULL, NULL);
if (ret != 0) {
char buf[1024] = {0};
av_strerror(ret, buf, sizeof(buf));
LOG_D(DEBUG, "avcodec open failed! info:%s", buf);
return -1;
}
LOG_D(DEBUG, "open audio codec success!");
LOG_D(DEBUG, "Complete init Audio Encode!")
return 0;
}複製程式碼- 指定獲取libfdk_aac編碼器
avCodec = avcodec_find_encoder_by_name("libfdk_aac");複製程式碼
- 初始化編碼器上下文
audioCodecContext = avcodec_alloc_context3(avCodec);複製程式碼 建立AVFrame，並分配記憶體負責封裝PCM源資料
outputFrame = av_frame_alloc(); outputFrame->channels = audioCodecContext->channels;//通道數 outputFrame->channel_layout = av_get_default_channel_layout(outputFrame->channels); outputFrame->format = audioCodecContext->sample_fmt; outputFrame->nb_samples = 1024;//預設值 ret = av_frame_get_buffer(outputFrame, 0); if (ret != 0) { LOG_D(DEBUG, "av_frame_get_buffer failed!"); return -1; } LOG_D(DEBUG, "av_frame_get_buffer success!");複製程式碼開啟編碼器
ret = avcodec_open2(audioCodecContext, NULL, NULL);複製程式碼以上是編碼前必須要完成的初始化.
int AudioEncoder::EncodeAAC 方法封裝了AAC編碼：
int AudioEncoder::EncodeAAC(OriginData **originData) {
int ret = 0;
ret = avcodec_fill_audio_frame(outputFrame,
audioCodecContext->channels,
audioCodecContext->sample_fmt, (*originData)->data,
8192, 0);
outputFrame->pts = (*originData)->pts;
ret = avcodec_send_frame(audioCodecContext, outputFrame);
if (ret != 0) {
#ifdef SHOW_DEBUG_INFO
LOG_D(DEBUG, "send frame failed!");
#endif
}
av_packet_unref(&audioPacket);
ret = avcodec_receive_packet(audioCodecContext, &audioPacket);
if (ret != 0) {
#ifdef SHOW_DEBUG_INFO
LOG_D(DEBUG, "receive packet failed!");
#endif
}
(*originData)->Drop();
(*originData)->avPacket = &audioPacket;
#ifdef SHOW_DEBUG_INFO
LOG_D(DEBUG, "encode audio packet size:%d pts:%lld", (*originData)->avPacket->size,
(*originData)->avPacket->pts);
LOG_D(DEBUG, "Audio frame encode success!");
#endif
(*originData)->avPacket->size;
return audioPacket.size;
}複製程式碼- *originData->data填充到AVFrame中，
audioPacket就是編碼後的資料了，data是編碼後的資料，size是大小，這樣就完成了編碼.ret = avcodec_send_frame(audioCodecContext, outputFrame); ret = avcodec_receive_packet(audioCodecContext, &audioPacket);複製程式碼
注意：在int AudioEncoder::InitEncode()方法中
avcodec_find_encoder_by_name("libfdk_aac");複製程式碼這裡使用了fdk-aac編碼器，前提是你必須要將libfdk-aac庫，連結到ffmpeg動態庫中，否則是找不到此編碼器的。FFmpeg自帶有AAC編碼器，可以通過：
avcodec_find_encoder(AV_CODEC_ID_AAC);複製程式碼獲取到AAC編碼器,當然如果使用FFmpeg的AAC編碼器，就會涉及到一個問題，就是剛開始文中提到了，AV_SAMPLE_FMT_S16需要重取樣為：AV_SAMPLE_FMT_FLTP的問題，由於FFmpeg廢棄了AV_SAMPLE_FMT_S16格式PCM編碼AAC，那麼在編碼前就需要多一步重取樣的處理.
以下AV_SAMPLE_FMT_S16 PCM音訊資料重取樣相關程式碼僅供參考：
初始化SwrContext,指定輸入輸出引數
swrContext = swr_alloc_set_opts(swrContext, av_get_default_channel_layout(CHANNELS),//輸出通道Layout AV_SAMPLE_FMT_FLTP,//輸出格式 48000,//輸出取樣率 av_get_default_channel_layout(CHANNELS),//輸入通道Layout AV_SAMPLE_FMT_S16,//輸入格式 48000,//輸入取樣率 NULL,//NULL NULL);//NULL ret = swr_init(swrContext);//初始化SwrContext if (ret != 0) { LOG_D(DEBUG, "swr_init failed!"); return -1; }複製程式碼AAC編碼前，將源資料重取樣
for (; ;) { if (encodeAAC->exit) { break; } if (encodeAAC->frame_queue.empty()) { continue; } const uint8_t *indata[AV_NUM_DATA_POINTERS] = {0}; //PCM s16 uint8_t *buf = *encodeAAC->frame_queue.wait_and_pop().get();//PCM 16bit #ifdef FDK_CODEC //fdk-aac無需重取樣 ret = avcodec_fill_audio_frame(encodeAAC->outputFrame, encodeAAC->avCodecContext->channels, encodeAAC->avCodecContext->sample_fmt, buf, BUFFER_SIZE, 0); if (ret < 0) { LOG_D(DEBUG, "fill frame failed!"); continue; } #else //重取樣AM_SAMPLE_FMT_FLTP indata[0] = buf; swr_convert(encodeAAC->swrContext, encodeAAC->outputFrame->data, encodeAAC->outputFrame->nb_samples, indata, encodeAAC->outputFrame->nb_samples); #endif複製程式碼以上程式碼就可以實現音訊重取樣，這樣就可以再使用FFMPEG AAC編碼器完成編碼.
以上簡述了android 採集音訊PCM資料及AAC編碼、AAC編碼涉及的相關初始化、FFmpeg AAC編碼器的重取樣示例.android camera採集、H264編碼與Rtmp推流與本文描述了音視訊採集、編碼過程及如何完成推流，相關文章待續......
版權宣告：本文為原創文章，轉載請註明出處。
程式碼地址：github.com/javandoc/Me…