本例需求：使用H264, H265實現視訊資料的編碼並錄製開始200幀存為檔案.
原理：比如做直播功能，需要將客戶端的視訊資料傳給伺服器，如果解析度過大如2K,4K則傳輸壓力太大，所以需要對視訊資料進行編碼，傳給伺服器後再解碼以實現大資料量的視訊資料的傳輸，而利用硬體編碼則可以極大限度減小CPU壓力，
H264進行編碼，iOS 11 之後，iPhone 7以上的裝置可以支援新的編碼器H265編碼器，使得同等質量視訊佔用的儲存空間更小。所以本例中可以使用兩種方式實現視訊資料的編碼
最終效果如下 ： h264
h265 :
GitHub地址(附程式碼) : H264,H265Encode
簡書地址 : H264,H265Encode
部落格地址 : H264,H265Encode
掘金地址 : H264,H265Encode
實現方式：
1. H264 : H264是當前主流編碼標準，以高壓縮高質量和支援多種網路的流媒體傳輸著稱
2. H265 ：H264編碼器的下一代，它的主要優點提供的壓縮比高，相同質量的視訊是H264的兩倍。
一.本文需要基本知識點
注意:可以先通過H264,H265編碼器介紹, H.264 Data Structure瞭解預備知識。
1. 軟編與硬編概念
- 軟編碼：使用CPU進行編碼。
- 硬編碼：不使用CPU進行編碼，使用顯示卡GPU,專用的DSP、FPGA、ASIC晶片等硬體進行編碼。
- 比較
- 軟編碼：實現直接、簡單，引數調整方便，升級易，但CPU負載重，效能較硬編碼低，低位元速率下質量通常比硬編碼要好一點。
- 效能高，低位元速率下通常質量低於軟編碼器，但部分產品在GPU硬體平臺移植了優秀的軟編碼演算法（如X264）的，質量基本等同於軟編碼。
- 蘋果在iOS 8.0系統之前，沒有開放系統的硬體編碼解碼功能，不過Mac OS系統一直有，被稱為Video ToolBox的框架來處理硬體的編碼和解碼，終於在iOS 8.0後，蘋果將該框架引入iOS系統
- 比較
2.H265優點
- 壓縮比高，在相同圖片質量情況下，比JPEG高兩倍
- 能增加如圖片的深度資訊，透明通道等輔助圖片。
- 支援存放多張圖片，類似相簿和集合。(實現多重曝光的效果)
- 支援多張圖片實現GIF和livePhoto的動畫效果。
- 無類似JPEG的最大畫素限制
- 支援透明畫素
- 分塊載入機制
- 支援縮圖
二.程式碼解析
1.實現流程
- 初始化相機引數，設定相機代理，這裡就固定只有豎屏模式。
- 初始化編碼器引數，並啟動編碼器
- 在編碼成功的回撥中從開始錄製200幀(檔案大小可自行修改)的視訊，存到沙盒中，可以通過連線資料線到電腦從itunes中將檔案(test0.asf)提取出來
2.編碼器實現流程
- 建立編碼器需要的session (h264, h265 或同時建立)
- 設定session屬性,如實時編碼，位元速率，fps, 編碼的解析度的寬高，相鄰I幀的最大間隔等等
-
注意H265目前不支援位元速率的限制
-
- 當相機回撥AVCaptureVideoDataOutputSampleBufferDelegate採集到一幀資料的時候則使用H264/H265編碼器對每一幀資料進行編碼。
- 若編碼成功會觸發回撥，回撥函式首先檢測是否有I幀出現，如果有I幀出現則將sps,pps資訊寫入否則遍歷NALU碼流並將startCode替換成{0x00, 0x00, 0x00, 0x01}
3.主要方法解析
- 初始化編碼器 首先選擇使用哪種方式實現，在本例中可以設定[XDXHardwareEncoder getInstance].enableH264 = YES 或者 [XDXHardwareEncoder getInstance].enableH265 = YES，也可以同時設定，如果同時設定需要將其中一個回撥函式中的writeFile的方法遮蔽掉，並且只有較新的iPhone(> iPhone8 穩定)才支援同時開啟兩個session。
判斷當前裝置是否支援H265編碼，必須滿足兩個條件，一是iPhone 7 以上裝置，二是版本大於iOS 11
if (@available(iOS 11.0, *)) {
BOOL hardwareDecodeSupported = VTIsHardwareDecodeSupported(kCMVideoCodecType_HEVC);
if (hardwareDecodeSupported) {
_deviceSupportH265 = YES;
NSLog(@"XDXHardwareEncoder : Support H265 Encode/Decode!");
}
}else {
_deviceSupportH265 = NO;
NSLog(@"XDXHardwareEncoder : Not support H265 Encode/Decode!");
}
複製程式碼系統已經提供VTIsHardwareDecodeSupported判斷當前裝置是否支援H265編碼
初始化編碼器操作
- (void)prepareForEncode {
if(self.width == 0 || self.height == 0) {
NSLog(@"XDXHardwareEncoder : VTSession need with and height for init,with = %d,height = %d",self.width, self.height);
return;
}
if(g_isSupportRealTimeEncoder) NSLog(@"XDXHardwareEncoder : Device processor is 64 bit");
else NSLog(@"XDXHardwareEncoder : Device processor is not 64 bit");
NSLog(@"XDXHardwareEncoder : Current h264 open state : %d, h265 open state : %d",self.enableH264, self.enableH265);
OSStatus h264Status,h265Status;
BOOL isRestart = NO;
if (self.enableH264) {
if (h264CompressionSession != NULL) {
NSLog(@"XDXHardwareEncoder : H264 session not NULL");
return;
}
[m_h264_lock lock];
NSLog(@"XDXHardwareEncoder : Prepare H264 hardware encoder");
//[self.delegate willEncoderStart];
self.h264ErrCount = 0;
h264Status = VTCompressionSessionCreate(NULL, self.width, self.height, kCMVideoCodecType_H264, NULL, NULL, NULL, vtCallBack,(__bridge void *)self, &h264CompressionSession);
if (h264Status != noErr) {
self.h265ErrCount++;
NSLog(@"XDXHardwareEncoder : H264 VTCompressionSessionCreate Failed, status = %d",h264Status);
}
[self getSupportedPropertyFlags];
[self applyAllSessionProperty:h264CompressionSession propertyArr:self.h264propertyFlags];
h264Status = VTCompressionSessionPrepareToEncodeFrames(h264CompressionSession);
if(h264Status != noErr) {
NSLog(@"XDXHardwareEncoder : H264 VTCompressionSessionPrepareToEncodeFrames Failed, status = %d",h264Status);
}else {
initializedH264 = true;
NSLog(@"XDXHardwareEncoder : H264 VTSession create success, with = %d, height = %d, framerate = %d",self.width,self.height,self.fps);
}
if(h264Status != noErr && self.h264ErrCount != 0) isRestart = YES;
[m_h264_lock unlock];
}
if (self.enableH265) {
if (h265CompressionSession != NULL) {
NSLog(@"XDXHardwareEncoder : H265 session not NULL");
return;
}
[m_h265_lock lock];
NSLog(@"XDXHardwareEncoder : Prepare h265 hardware encoder");
// [self.delegate willEncoderStart];
self.h265ErrCount = 0;
h265Status = VTCompressionSessionCreate(NULL, self.width, self.height, kCMVideoCodecType_HEVC, NULL, NULL, NULL, vtH265CallBack,(__bridge void *)self, &h265CompressionSession);
if (h265Status != noErr) {
self.h265ErrCount++;
NSLog(@"XDXHardwareEncoder : H265 VTCompressionSessionCreate Failed, status = %d",h265Status);
}
[self getSupportedPropertyFlags];
[self applyAllSessionProperty:h265CompressionSession propertyArr:self.h265PropertyFlags];
h265Status = VTCompressionSessionPrepareToEncodeFrames(h265CompressionSession);
if(h265Status != noErr) {
NSLog(@"XDXHardwareEncoder : H265 VTCompressionSessionPrepareToEncodeFrames Failed, status = %d",h265Status);
}else {
initializedH265 = true;
NSLog(@"XDXHardwareEncoder : H265 VTSession create success, with = %d, height = %d, framerate = %d",self.width,self.height,self.fps);
}
if(h265Status != noErr && self.h265ErrCount != 0) isRestart = YES;
[m_h265_lock unlock];
}
if (isRestart) {
NSLog(@"XDXHardwareEncoder : VTSession create failured!");
static int count = 0;
count ++;
if (count == 3) {
NSLog(@"TVUEncoder ： restart 5 times failured! exit!");
return;
}
sleep(1);
NSLog(@"TVUEncoder ： try to restart after 1 second!");
NSLog(@"TVUEncoder ： vtsession error occured!,resetart encoder width: %d, height %d, times %d",self.width,self.height,count);
[self tearDownSession];
[self prepareForEncode];
}
}
複製程式碼1> g_isSupportRealTimeEncoder = (is64Bit == 8) ? true : false;用來判斷當前裝置是32位還是64位
2> 建立H264/H265Session 區別僅僅為引數的不同，h264為kCMVideoCodecType_H264。 h265為kCMVideoCodecType_HEVC，在建立Session指定了回撥函式後，當編碼成功一幀就會呼叫相應的回撥函式。
3> 通過[self getSupportedPropertyFlags];獲取當前編碼器支援設定的屬性，經過測試，H265不支援位元速率的限制。目前暫時得不到解決。等待蘋果後續處理。
4> 之後設定編碼器相關屬性，下面會具體介紹，設定完成後則呼叫VTCompressionSessionPrepareToEncodeFrames準備編碼。
- 設定編碼器相關屬性
- (OSStatus)setSessionProperty:(VTCompressionSessionRef)session key:(CFStringRef)key value:(CFTypeRef)value {
OSStatus status = VTSessionSetProperty(session, key, value);
if (status != noErr) {
NSString *sessionStr;
if (session == h264CompressionSession) {
sessionStr = @"h264 Session";
self.h264ErrCount++;
}else if (session == h265CompressionSession) {
sessionStr = @"h265 Session";
self.h265ErrCount++;
}
NSLog(@"XDXHardwareEncoder : Set %s of %s Failed, status = %d",CFStringGetCStringPtr(key, kCFStringEncodingUTF8),sessionStr.UTF8String,status);
}
return status;
}
- (void)applyAllSessionProperty:(VTCompressionSessionRef)session propertyArr:(NSArray *)propertyArr {
OSStatus status;
if(!g_isSupportRealTimeEncoder) {
/* increase max frame delay from 3 to 6 to reduce encoder pressure*/
int value = 3;
CFNumberRef ref = CFNumberCreate(NULL, kCFNumberSInt32Type, &value);
[self setSessionProperty:session key:kVTCompressionPropertyKey_MaxFrameDelayCount value:ref];
CFRelease(ref);
}
if(self.fps) {
if([self isSupportPropertyWithKey:Key_ExpectedFrameRate inArray:propertyArr]) {
int value = self.fps;
CFNumberRef ref = CFNumberCreate(NULL, kCFNumberSInt32Type, &value);
[self setSessionProperty:session key:kVTCompressionPropertyKey_ExpectedFrameRate value:ref];
CFRelease(ref);
}
}else {
NSLog(@"XDXHardwareEncoder : Current fps is 0");
}
if(self.bitrate) {
if([self isSupportPropertyWithKey:Key_AverageBitRate inArray:propertyArr]) {
int value = self.bitrate;
if (session == h265CompressionSession) value = 2*1000; // if current session is h265, Set birate 2M.
CFNumberRef ref = CFNumberCreate(NULL, kCFNumberSInt32Type, &value);
[self setSessionProperty:session key:kVTCompressionPropertyKey_AverageBitRate value:ref];
CFRelease(ref);
}
}else {
NSLog(@"XDXHardwareEncoder : Current bitrate is 0");
}
/*2016-11-15,@gang, iphone7/7plus do not support realtime encoding, so disable it
otherwize ,we can not control encoding bit rate
*/
if (![[self deviceVersion] isEqualToString:@"iPhone9,1"] && ![[self deviceVersion] isEqualToString:@"iPhone9,2"]) {
if(g_isSupportRealTimeEncoder) {
if([self isSupportPropertyWithKey:Key_RealTime inArray:propertyArr]) {
NSLog(@"use RealTimeEncoder");
NSLog(@"XDXHardwareEncoder : use realTimeEncoder");
[self setSessionProperty:session key:kVTCompressionPropertyKey_RealTime value:kCFBooleanTrue];
}
}
}
if([self isSupportPropertyWithKey:Key_AllowFrameReordering inArray:propertyArr]) {
[self setSessionProperty:session key:kVTCompressionPropertyKey_AllowFrameReordering value:kCFBooleanFalse];
}
if(g_isSupportRealTimeEncoder) {
if([self isSupportPropertyWithKey:Key_ProfileLevel inArray:propertyArr]) {
[self setSessionProperty:session key:kVTCompressionPropertyKey_ProfileLevel value:self.enableH264 ? kVTProfileLevel_H264_Main_AutoLevel : kVTProfileLevel_HEVC_Main_AutoLevel];
}
}else {
if([self isSupportPropertyWithKey:Key_ProfileLevel inArray:propertyArr]) {
[self setSessionProperty:session key:kVTCompressionPropertyKey_ProfileLevel value:self.enableH264 ? kVTProfileLevel_H264_Baseline_AutoLevel : kVTProfileLevel_HEVC_Main_AutoLevel];
}
if (self.enableH264) {
if([self isSupportPropertyWithKey:Key_H264EntropyMode inArray:propertyArr]) {
[self setSessionProperty:session key:kVTCompressionPropertyKey_H264EntropyMode value:kVTH264EntropyMode_CAVLC];
}
}
}
if([self isSupportPropertyWithKey:Key_MaxKeyFrameIntervalDuration inArray:propertyArr]) {
int value = 1;
CFNumberRef ref = CFNumberCreate(NULL, kCFNumberSInt32Type, &value);
[self setSessionProperty:session key:kVTCompressionPropertyKey_MaxKeyFrameIntervalDuration value:ref];
CFRelease(ref);
}
}
複製程式碼上述方法主要設定啟動編碼器所需的各個引數
1> kVTCompressionPropertyKey_MaxFrameDelayCount : 壓縮器被允許保持的最大幀數在輸出一個壓縮幀之前。例如如果最大幀延遲數是M,那麼在編碼幀N返回的呼叫之前，幀N-M必須被排出。
2> kVTCompressionPropertyKey_ExpectedFrameRate : 設定fps
3> kVTCompressionPropertyKey_AverageBitRate : 它不是強制的限制，bit rate可能會超出峰值
4> kVTCompressionPropertyKey_RealTime : 設定編碼器是否實時編碼，如果設定為False則不是實時編碼，視訊效果會更好一點。
5> kVTCompressionPropertyKey_AllowFrameReordering : 是否讓幀進行重新排序。為了編碼B幀，編碼器必須對幀重新排序，這將意味著解碼的順序與顯示的順序不同。將其設定為false以防止幀重新排序。
6> kVTCompressionPropertyKey_ProfileLevel : 指定編碼位元流的配置檔案和級別
7> kVTCompressionPropertyKey_H264EntropyMode ：如果支援h264該屬性設定編碼器是否應該使用基於CAVLC 還是 CABAC
8> kVTCompressionPropertyKey_MaxKeyFrameIntervalDuration : 兩個I幀之間最大持續時間，該屬性特別有用當frame rate是可變
- 相機回撥中對每一幀資料進行編碼
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
if( !CMSampleBufferDataIsReady(sampleBuffer)) {
NSLog( @"sample buffer is not ready. Skipping sample" );
return;
}
if([XDXHardwareEncoder getInstance] != NULL) {
[[XDXHardwareEncoder getInstance] encode:sampleBuffer];
}
}
複製程式碼以上方法在每採集到一幀視訊資料後會呼叫一次，我們將拿到的每一幀資料進行編碼。
- 編碼具體實現
-(void)encode:(CMSampleBufferRef)sampleBuffer {
if (self.enableH264) {
[m_h264_lock lock];
if(h264CompressionSession == NULL) {
[m_h264_lock unlock];
return;
}
if(initializedH264 == false) {
NSLog(@"TVUEncoder : h264 encoder is not ready\n");
return;
}
}
if (self.enableH265) {
[m_h265_lock lock];
if(h265CompressionSession == NULL) {
[m_h265_lock unlock];
return;
}
if(initializedH265 == false) {
NSLog(@"TVUEncoder : h265 encoder is not ready\n");
return;
}
}
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CMTime duration = CMSampleBufferGetOutputDuration(sampleBuffer);
frameID++;
CMTime presentationTimeStamp = CMTimeMake(frameID, 1000);
[self doSetBitrate];
OSStatus status;
VTEncodeInfoFlags flags;
if (self.enableH264) {
status = VTCompressionSessionEncodeFrame(h264CompressionSession, imageBuffer, presentationTimeStamp, duration, NULL, imageBuffer, &flags);
if(status != noErr) NSLog(@"TVUEncoder : H264 VTCompressionSessionEncodeFrame failed");
[m_h264_lock unlock];
if (status != noErr) {
NSLog(@"TVUEncoder : VTCompressionSessionEncodeFrame failed");
VTCompressionSessionCompleteFrames(h264CompressionSession, kCMTimeInvalid);
VTCompressionSessionInvalidate(h264CompressionSession);
CFRelease(h264CompressionSession);
h264CompressionSession = NULL;
}else {
// NSLog(@"TVUEncoder : Success VTCompressionSessionCompleteFrames");
}
}
if (self.enableH265) {
status = VTCompressionSessionEncodeFrame(h265CompressionSession, imageBuffer, presentationTimeStamp, duration, NULL, imageBuffer, &flags);
if(status != noErr) NSLog(@"TVUEncoder : H265 VTCompressionSessionEncodeFrame failed");
[m_h265_lock unlock];
if (status != noErr) {
NSLog(@"TVUEncoder : VTCompressionSessionEncodeFrame failed");
VTCompressionSessionCompleteFrames(h265CompressionSession, kCMTimeInvalid);
VTCompressionSessionInvalidate(h265CompressionSession);
CFRelease(h265CompressionSession);
h265CompressionSession = NULL;
}else {
NSLog(@"TVUEncoder : Success VTCompressionSessionCompleteFrames");
}
}
}
複製程式碼1> 通過frameID的遞增構造時間戳為了使編碼後的每一幀資料連續
2> 設定最大位元速率的限制，注意：H265目前不支援設定位元速率的限制，等待官方後續通知。可以對H264進行位元速率限制
3> kVTCompressionPropertyKey_DataRateLimits : 將資料的bytes和duration封裝到CFMutableArrayRef傳給API進行呼叫
4> VTCompressionSessionEncodeFrame : 呼叫此方法成功後觸發回撥函式完成編碼。
- 回撥函式中處理頭資訊
#pragma mark H264 Callback
static void vtCallBack(void *outputCallbackRefCon,void *souceFrameRefCon,OSStatus status,VTEncodeInfoFlags infoFlags, CMSampleBufferRef sampleBuffer) {
XDXHardwareEncoder *encoder = (__bridge XDXHardwareEncoder*)outputCallbackRefCon;
if(status != noErr) {
NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain code:status userInfo:nil];
NSLog(@"H264: vtCallBack failed with %@", error);
NSLog(@"XDXHardwareEncoder : encode frame failured! %s" ,error.debugDescription.UTF8String);
return;
}
if (!CMSampleBufferDataIsReady(sampleBuffer)) {
NSLog(@"didCompressH265 data is not ready ");
return;
}
if (infoFlags == kVTEncodeInfo_FrameDropped) {
NSLog(@"%s with frame dropped.", __FUNCTION__);
return;
}
CMBlockBufferRef block = CMSampleBufferGetDataBuffer(sampleBuffer);
BOOL isKeyframe = false;
CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, false);
if(attachments != NULL) {
CFDictionaryRef attachment =(CFDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);
CFBooleanRef dependsOnOthers = (CFBooleanRef)CFDictionaryGetValue(attachment, kCMSampleAttachmentKey_DependsOnOthers);
isKeyframe = (dependsOnOthers == kCFBooleanFalse);
}
if(isKeyframe) {
CMFormatDescriptionRef format = CMSampleBufferGetFormatDescription(sampleBuffer);
static uint8_t *spsppsNALBuff = NULL;
static size_t spsSize, ppsSize;
size_t parmCount;
const uint8_t*sps, *pps;
int NALUnitHeaderLengthOut;
CMVideoFormatDescriptionGetH264ParameterSetAtIndex(format, 0, &sps, &spsSize, &parmCount, &NALUnitHeaderLengthOut );
CMVideoFormatDescriptionGetH264ParameterSetAtIndex(format, 1, &pps, &ppsSize, &parmCount, &NALUnitHeaderLengthOut );
spsppsNALBuff = (uint8_t*)malloc(spsSize+4+ppsSize+4);
memcpy(spsppsNALBuff, "\x00\x00\x00\x01", 4);
memcpy(&spsppsNALBuff[4], sps, spsSize);
memcpy(&spsppsNALBuff[4+spsSize], "\x00\x00\x00\x01", 4);
memcpy(&spsppsNALBuff[4+spsSize+4], pps, ppsSize);
NSLog(@"XDXHardwareEncoder : H264 spsSize : %zu, ppsSize : %zu",spsSize, ppsSize);
writeFile(spsppsNALBuff,spsSize+4+ppsSize+4,encoder->_videoFile, 200);
}
size_t blockBufferLength;
uint8_t *bufferDataPointer = NULL;
CMBlockBufferGetDataPointer(block, 0, NULL, &blockBufferLength, (char **)&bufferDataPointer);
size_t bufferOffset = 0;
while (bufferOffset < blockBufferLength - startCodeLength) {
uint32_t NALUnitLength = 0;
memcpy(&NALUnitLength, bufferDataPointer+bufferOffset, startCodeLength);
NALUnitLength = CFSwapInt32BigToHost(NALUnitLength);
memcpy(bufferDataPointer+bufferOffset, startCode, startCodeLength);
bufferOffset += startCodeLength + NALUnitLength;
}
writeFile(bufferDataPointer, blockBufferLength,encoder->_videoFile, 200);
}
#pragma mark H265 Callback
static void vtH265CallBack(void *outputCallbackRefCon,void *souceFrameRefCon,OSStatus status,VTEncodeInfoFlags infoFlags, CMSampleBufferRef sampleBuffer) {
XDXHardwareEncoder *encoder = (__bridge XDXHardwareEncoder*)outputCallbackRefCon;
if(status != noErr) {
NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain code:status userInfo:nil];
NSLog(@"H264: H265 vtH265CallBack failed with %@", error);
NSLog(@"XDXHardwareEncoder : H265 encode frame failured! %s" ,error.debugDescription.UTF8String);
return;
}
if (!CMSampleBufferDataIsReady(sampleBuffer)) {
NSLog(@"didCompressH265 data is not ready ");
return;
}
if (infoFlags == kVTEncodeInfo_FrameDropped) {
NSLog(@"%s with frame dropped.", __FUNCTION__);
return;
}
CMBlockBufferRef block = CMSampleBufferGetDataBuffer(sampleBuffer);
BOOL isKeyframe = false;
CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, false);
if(attachments != NULL) {
CFDictionaryRef attachment =(CFDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);
CFBooleanRef dependsOnOthers = (CFBooleanRef)CFDictionaryGetValue(attachment, kCMSampleAttachmentKey_DependsOnOthers);
isKeyframe = (dependsOnOthers == kCFBooleanFalse);
}
if(isKeyframe) {
CMFormatDescriptionRef format = CMSampleBufferGetFormatDescription(sampleBuffer);
static uint8_t *vpsspsppsNALBuff = NULL;
static size_t vpsSize, spsSize, ppsSize;
size_t parmCount;
const uint8_t *vps, *sps, *pps;
if (encoder.deviceSupportH265) { // >= iPhone 7 && support ios11
CMVideoFormatDescriptionGetHEVCParameterSetAtIndex(format, 0, &vps, &vpsSize, &parmCount, 0);
CMVideoFormatDescriptionGetHEVCParameterSetAtIndex(format, 1, &sps, &spsSize, &parmCount, 0);
CMVideoFormatDescriptionGetHEVCParameterSetAtIndex(format, 2, &pps, &ppsSize, &parmCount, 0);
vpsspsppsNALBuff = (uint8_t*)malloc(vpsSize+4+spsSize+4+ppsSize+4);
memcpy(vpsspsppsNALBuff, "\x00\x00\x00\x01", 4);
memcpy(&vpsspsppsNALBuff[4], vps, vpsSize);
memcpy(&vpsspsppsNALBuff[4+vpsSize], "\x00\x00\x00\x01", 4);
memcpy(&vpsspsppsNALBuff[4+vpsSize+4], sps, spsSize);
memcpy(&vpsspsppsNALBuff[4+vpsSize+4+spsSize], "\x00\x00\x00\x01", 4);
memcpy(&vpsspsppsNALBuff[4+vpsSize+4+spsSize+4], pps, ppsSize);
NSLog(@"XDXHardwareEncoder : H265 vpsSize : %zu, spsSize : %zu, ppsSize : %zu",vpsSize,spsSize, ppsSize);
}
writeFile(vpsspsppsNALBuff, vpsSize+4+spsSize+4+ppsSize+4,encoder->_videoFile, 200);
}
size_t blockBufferLength;
uint8_t *bufferDataPointer = NULL;
CMBlockBufferGetDataPointer(block, 0, NULL, &blockBufferLength, (char **)&bufferDataPointer);
size_t bufferOffset = 0;
while (bufferOffset < blockBufferLength - startCodeLength) {
uint32_t NALUnitLength = 0;
memcpy(&NALUnitLength, bufferDataPointer+bufferOffset, startCodeLength);
NALUnitLength = CFSwapInt32BigToHost(NALUnitLength);
memcpy(bufferDataPointer+bufferOffset, startCode, startCodeLength);
bufferOffset += startCodeLength + NALUnitLength;
}
writeFile(bufferDataPointer, blockBufferLength,encoder->_videoFile, 200);
}
複製程式碼1> 首先在回撥函式中擷取到I幀，從I幀中提取到(h265中新增vps),sps,pps資訊並寫入檔案 2> 遍歷其他幀將頭資訊0000,0001寫入每個頭資訊中，再將該資料寫入檔案即可
二.碼流資料結構介紹
這裡我們簡單介紹一下H264,H265碼流資訊
-
H264流資料是由一系列NAL單元(NAL Unit)組成的。
-
一個NALU可能包含：視訊幀，視訊幀也就是視訊片段，具體有I,P,B幀
- H.264屬性合集-FormatDesc(包含 SPS和PPS)
注意在H265流資料中新增vps在最前。
- H.264屬性合集-FormatDesc(包含 SPS和PPS)
流資料中，屬性集合可能是這樣的：
經過處理之後，在Format Description中則是:
- NALU header 對於流資料來說，一個NALU的Header中，可能是0x00 00 01或者是0x00 00 00 01作為開頭(兩者都有可能，下面以0x00 00 01作為例子)。0x00 00 01因此被稱為開始碼(Start code).所以我們需要在提取的資料中用0x00 00 00 01對資料內容進行替換
總結以上知識，我們知道H264的碼流由NALU單元組成，NALU單元包含視訊影像資料和H264的引數資訊。其中視訊影像資料就是CMBlockBuffer，而H264的引數資訊則可以組合成FormatDesc。具體來說引數資訊包含SPS（Sequence Parameter Set）和PPS（Picture Parameter Set）.如下圖顯示了一個H.264碼流結構：
-
提取sps和pps生成FormatDesc
- 每個NALU的開始碼是0x00 00 01，按照開始碼定位NALU
- 通過型別資訊找到sps和pps並提取，開始碼後第一個byte的後5位，7代表sps，8代表pps
- 使用CMVideoFormatDescriptionCreateFromH264ParameterSets函式來構建CMVideoFormatDescriptionRef
-
提取視訊影像資料生成CMBlockBuffer
- 通過開始碼，定位到NALU
- 確定型別為資料後，將開始碼替換成NALU的長度資訊（4 Bytes）
- 使用CMBlockBufferCreateWithMemoryBlock介面構造CMBlockBufferRef
-
根據需要，生成CMTime資訊。（實際測試時，加入time資訊後，有不穩定的影像，不加入time資訊反而沒有，需要進一步研究，這裡建議不加入time資訊）
根據上述得到CMVideoFormatDescriptionRef、CMBlockBufferRef和可選的時間資訊，使用CMSampleBufferCreate介面得到CMSampleBuffer資料這個待解碼的原始的資料。如下圖所示的H264資料轉換示意圖。