最简单的是使用滤镜
# 查看滤镜帮助ffplay -h filter=drawbox# 单个矩形ffplay -i fpx.gif -vf drawbox:x=10:y=10:w=50:h=50:c=red# 多个矩形,中间使用逗号连接ffplay -i fpx.gif -vf drawbox:x=10:y=10:w=50:h=50:c=red,\drawbox:x=100:y=10:w=50:h=50:c=rgb(0,0,255)
现在由 YOLO 算法检测出每帧图片中物体的位置(s-x,s-y,s->w,s->h),需要在 ffplay 源码中嵌入绘制矩形框的函数。首先要参考 ffmpeg-4.3.1/libavfilter/vf_drawbox.c。
图像:
核心代码(仅对 YUV 格式有效):
for (y = FFMAX(yb, 0); y < FFMIN(yb + s->h, frame->height); y++) {row[0] = frame->data[0] + y * frame->linesize[0];for (plane = 1; plane < 3; plane++)row[plane] = frame->data[plane] +frame->linesize[plane] * (y >> s->vsub);for (x = FFMAX(xb, 0); x < FFMIN(xb + s->w, frame->width); x++) {if (pixel_belongs_to_box(s, x, y)) {row[0][x ] = s->yuv_color[Y];row[1][x >> s->hsub] = s->yuv_color[U];row[2][x >> s->hsub] = s->yuv_color[V];}}}
其原理很简单,两个 for 循环遍历矩形的每一个像素点,通过 pixel_belongs_to_box() 函数判断该点是否在矩形框内,如果在,就修改该点的颜色。pixel_belongs_to_box() 函数如下,s->thickness 表示矩形框的厚度。
static int pixel_belongs_to_box(DrawBoxContext *s, int x, int y){return (y - s->y < s->thickness) || (s->y + s->h - 1 - y < s->thickness) ||(x - s->x < s->thickness) || (s->x + s->w - 1 - x < s->thickness);}
在上面的核心代码中,还出现了 s->hsub 和 s->vsub,这是色度二次采样值,和像素格式有关,获取方式如下,pix_fmt 即为帧的像素格式。
const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt);s->hsub = desc->log2_chroma_w;s->vsub = desc->log2_chroma_h;
====================================================================
缺点是扫描整个矩形导致速度太慢,毕竟我们只是绘制矩形框而非整个矩形,所以做出改进:
for (y = FFMAX(yb, 0); y < FFMIN(yb + s->h, frame->height); y++) {row[0] = frame->data[0] + y * frame->linesize[0];for (plane = 1; plane < 3; plane++)row[plane] = frame->data[plane] +frame->linesize[plane] * (y >> s->vsub);if ((y - yb < s->thickness) || (yb + s->h - 1 - y < s->thickness)) {for (x = FFMAX(xb, 0); x < FFMIN(xb + s->w, frame->width); x++) {row[0][x ] = s->dst_color[Y];row[1][x >> s->hsub] = s->dst_color[U];row[2][x >> s->hsub] = s->dst_color[V];}} else {for (x = FFMAX(xb, 0); x < xb + s->thickness; x++) {row[0][x ] = s->dst_color[Y];row[1][x >> s->hsub] = s->dst_color[U];row[2][x >> s->hsub] = s->dst_color[V];}for (x = xb + s->w - s->thickness; x < FFMIN(xb + s->w, frame->width); x++) {row[0][x ] = s->dst_color[Y];row[1][x >> s->hsub] = s->dst_color[U];row[2][x >> s->hsub] = s->dst_color[V];}}}
原理同样是自上而下扫描,但通过对 y 坐标的判断,跳过了矩形框中间的空白部分,速度大大加快!