FFmpeg实战项目与代码示例

FFmpeg实战项目与代码示例前置说明核心模块表格模块作用libavformat封装/解封装，文件、rtmp、rtsp、hlsIOlibavcodec编解码器，视频音频编解码libavfilter滤镜：裁剪、缩放、水印、转灰度、混音libswscale图像格式转换、缩放（YUV↔RGB）libswresample音频重采样：采样率、声道、格式转换libavutil工具，内存、日志、帧、时间戳编译链接：-lavformat-lavcodec-lavfilter-lswscale-lswresample-lavutil重要：FFmpeg内存原则：avformat_open_input→avformat_close_input；avcodec_open2→avcodec_close；av_frame_free；av_packet_free，必须释放，否则内存泄漏。时间戳：AVPacketpts/dts，时间基AVRationaltime_base，av_rescale_q做时间换算。实战1：读取本地视频，解封装+解码，输出YUV原始帧（最简解码demo）功能：打开mp4，找到视频流，解码每一帧，输出yuv420praw数据到文件。main.cc运行#include<stdio.h>#include<libavformat/avformat.h>#include<libavcodec/avcodec.h>#include<libavutil/imgutils.h>intmain(intargc,char**argv){if(argc<3){printf("./demoinput.mp4out.yuv\n");return-1;}constchar*in_path=argv[1];constchar*out_yuv=argv[2];AVFormatContext*fmt_ctx=NULL;AVCodecContext*codec_ctx=NULL;constAVCodec*codec=NULL;AVStream*video_stream=NULL;intvideo_idx=-1;AVPacket*pkt=av_packet_alloc();AVFrame*frame=av_frame_alloc();//1.打开输入intret=avformat_open_input(&fmt_ctx,in_path,NULL,NULL);if(ret<0){charerr[AV_ERROR_MAX_STRING_SIZE]={0};av_strerror(ret,err,sizeoferr);fprintf(stderr,"openinputfail:%s\n",err);gotoend;}avformat_find_stream_info(fmt_ctx,NULL);//找视频流video_idx=av_find_best_stream(fmt_ctx,AVMEDIA_TYPE_VIDEO,-1,-1,&codec,0);if(video_idx<0)gotoend;video_stream=fmt_ctx->streams[video_idx];//分配解码器上下文codec_ctx=avcodec_alloc_context3(codec);avcodec_parameters_to_context(codec_ctx,video_stream->codecpar);ret=avcodec_open2(codec_ctx,codec,NULL);if(ret<0)gotoend;FILE*fp_out=fopen(out_yuv,"wb");//循环读包while(av_read_frame(fmt_ctx,pkt)>=0){if(pkt->stream_index==video_idx){//发送packet给解码器ret=avcodec_send_packet(codec_ctx,pkt);if(ret<0){av_packet_unref(pkt);continue;}//接收解码后的framewhile((ret=avcodec_receive_frame(codec_ctx,frame))==0){//YUV420P:Yplane0,Uplane1,Vplane2for(inti=0;i<3;i++){intlinesize=frame->linesize[i];intheight=(i==0)?frame->height:frame->height/2;for(inth=0;h<height;h++){fwrite(frame->data[i]+h*linesize,1,frame->width>>i,fp_out);}}av_frame_unref(frame);}}av_packet_unref(pkt);}//flush解码器，读取缓存剩余帧avcodec_send_packet(codec_ctx,NULL);while(avcodec_receive_frame(codec_ctx,frame)==0){for(inti=0;i<3;i++){intlinesize=frame->linesize[i];intheight=(i==0)?frame->height:frame->height/2;for(inth=0;h<height;h++){fwrite(frame->data[i]+h*linesize,1,frame->width>>i,fp_out);}}av_frame_unref(frame);}fclose(fp_out);printf("decodefinish\n");end:av_packet_free(&pkt);av_frame_free(&frame);avcodec_close(codec_ctx);avcodec_free_context(&codec_ctx);avformat_close_input(&fmt_ctx);return0;}编译命令：bashgccmain.c-odecode_demo`pkg-config--libs--cflagslibavformatlibavcodeclibavutil`./decode_demotest.mp4out.yuv实战2：编码：YUV420P原始帧编码为H264mp4输入原始yuv，编码输出mp4(h264)关键点：分配AVFormatContext输出上下文新建流avformat_new_stream设置编码参数，打开编码器avcodec_send_frame/avcodec_receive_packetav_interleaved_write_frame写包av_write_trailer()结束文件c运行//关键片段（省略错误处理）AVFormatContext*out_fmt_ctx=NULL;avformat_alloc_output_context2(&out_fmt_ctx,NULL,NULL,"out.mp4");AVStream*out_stream=avformat_new_stream(out_fmt_ctx,NULL);constAVCodec*enc_codec=avcodec_find_encoder(AV_CODEC_ID_H264);AVCodecContext*enc_ctx=avcodec_alloc_context3(enc_codec);enc_ctx->width=1280;enc_ctx->height=720;enc_ctx->pix_fmt=AV_PIX_FMT_YUV420P;enc_ctx->time_base=(AVRational){1,25};//25fpsenc_ctx->framerate=(AVRational){25,1};//h264参数av_opt_set(enc_ctx->priv_data,"preset","fast",0);av_opt_set(enc_ctx->priv_data,"crf","23",0);avcodec_open2(enc_ctx,enc_codec,NULL);avcodec_parameters_from_context(out_stream->codecpar,enc_ctx);out_stream->time_base=enc_ctx->time_base;//打开IOavio_open(&out_fmt_ctx->pb,"out.mp4",AVIO_FLAG_WRITE);avformat_write_header(out_fmt_ctx,NULL);//循环送入AVFrame（YUV帧）//avcodec_send_frame(enc_ctx,frame);//avcodec_receive_packet(enc_ctx,pkt);//av_interleaved_write_frame(out_fmt_ctx,pkt);//结束avcodec_send_frame(enc_ctx,NULL);//flushwhile(avcodec_receive_packet(enc_ctx,pkt)==0){av_interleaved_write_frame(out_fmt_ctx,pkt);av_packet_unref(pkt);}av_write_trailer(out_fmt_ctx);avio_closep(&out_fmt_ctx->pb);实战3：完整转码Demo（输入mp4→h264+aac输出mp4）最常用项目场景：转码，解码->可选滤镜->编码->封装输出流程：avformat_open_input读输入创建输出上下文，分别创建视频流、音频流打开输入解码器、输出编码器循环read_frame，解码，时间戳转换av_rescale_q，send_frame给编码器，receive_packet写输出flush解码器、编码器，写trailer提示：完整代码较长，核心难点：pts/dts时间基转换，很多转码花屏、时长错误都是时间戳搞错。时间戳转换核心代码片段c运行//pkt.pts/dts从输入流time_base转到编码器time_baseframe->pts=av_rescale_q(pkt->pts,in_stream->time_base,enc_ctx->time_base);frame->dts=av_rescale_q(pkt->dts,in_stream->time_base,enc_ctx->time_base);实战4：libswscale：YUV转RGB（视频帧转图像，用于截图）场景：解码得到YUV帧，转为RGB24，可保存为bmp；视频截图底层逻辑。c运行#include<libswscale/swscale.h>//源：YUV420P1280x720；目标RGB24SwsContext*sws_ctx=sws_getContext(width,height,AV_PIX_FMT_YUV420P,width,height,AV_PIX_FMT_RGB24,SWS_BILINEAR,NULL,NULL,NULL);AVFrame*rgb_frame=av_frame_alloc();av_image_alloc(rgb_frame->data,rgb_frame->linesize,width,height,AV_PIX_FMT_RGB24,1);//转换：yuvframe→rgbframesws_scale(sws_ctx,(constuint8_t*const*)frame->data,frame->linesize,0,height,rgb_frame->data,rgb_frame->linesize);//rgb_frame->data[0]就是rgb24字节流，可以写bmp文件av_freep(&rgb_frame->data[0]);av_frame_free(&rgb_frame);sws_freeContext(sws_ctx);实战5：libswresample音频重采样场景：输入音频44100stereo→输出48000monoaac编码。c运行#include<libswresample/swresample.h>SwrContext*swr=swr_alloc();av_opt_set_int(swr,"in_sample_rate",44100,0);av_opt_set_int(swr,"out_sample_rate",48000,0);av_opt_set_int(swr,"in_channels",2,0);av_opt_set_int(swr,"out_channels",1,0);av_opt_set_int(swr,"in_sample_fmt",AV_SAMPLE_FMT_FLTP,0);av_opt_set_int(swr,"out_sample_fmt",AV_SAMPLE_FMT_FLTP,0);swr_init(swr);//使用swr_convert()做帧重采样实战6：avfilter滤镜示例（缩放+加黑边）滤镜图示例：[in]scale=640:480,pad=720:480:40:0[vout]适用：视频裁剪、水印、叠加、调色。avfilter流程：创建AVFilterGraph创建输入输出filterparse滤镜字符串把解码frame送入filtergraph，拉取过滤后的frame实战7：流媒体：RTMP推流（把本地mp4推到rtmp服务器）无需解码，直接remux（复用原始码流，只重新封装，性能极高）命令行参考：bashffmpeg-re-itest.mp4-ccopy-fflvrtmp:///live/stream1libavcodec实现remux核心：不解码，直接readpacket，转换pts/dts，av_interleaved_write_frame写入输出rtmp上下文。⚠️-re模拟实时速率；代码中需要自己控制发送速率，否则瞬间推完。实战8：视频截图（C代码）两种方案：方案A：seek到时间点av_seek_frame，解码一帧，swscale转RGB保存图片。av_seek_frame(fmt_ctx,stream_idx,timestamp,AVSEEK_FLAG_BACKWARD);方案B：调用libavfilter的fps滤镜，取1帧输出。坑：seek不一定精准到毫秒；mp4关键帧间隔大，seek会跳到前一个I帧。常用工程项目场景清单（可以直接拿来做项目）媒体转码服务：上传视频，转不同清晰度720p/480p，输出mp4。直播推流网关：读取本地文件/RTSP摄像头→RTMP推流。视频截图服务：对外http接口，输入视频+时间点返回图片。音频转码：mp3/wav→aac。水印服务：给视频叠加文字/图片水印（avfilterdrawtext、overlay）。视频剪辑：截取片段，remux快速裁剪（仅I帧精准）；精确裁剪需要解码再编码。RTSP摄像头取流，解码做AI图像识别：av读取rtsp流，解码YUV帧送给AI模型推理。HLS切片：输入视频，输出m3u8+ts分片。常见坑点总结时间戳pts/dts错误：转码花屏、时长不对、播放器报错。必须av_rescale_q做time_base转换。内存泄漏：avformat_close_input、avcodec_free_context、av_frame_free、av_packet_free必须成对调用。解码器编码器flush：发送NULLpacket/frame才会吐出缓存剩余帧，很多人漏掉导致末尾几帧丢失。RTSP流：设置超时，设置tcp传输：av_dict_set(&opts