Nvidia Nano平台deepstream-app视频颠倒调试记录

Nvidia Nano平台deepstream-app视频颠倒调试记录,第1张

1.前言

试图运行deepstream-app与示例

“source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt”

在Jetson Nano上与“deepstream-4.0_4.0-1_arm64.deb” 使用Raspberry Pi V2相机NoIr,

但图像显示面朝上。

修改配置

[source0]
enable = 1
#Type - 1 = CameraV4L2 2 = URI 3 = MultiURI 4 = RTSP 5 = Camera (CSI) (Jetson only)
type = 5
intra-decode-enable = 1
gpu-id = 0
camera-id = 0
camera-width = 1920
camera-height = 1280
camera-fps-n = 30
camera-fps-d = 1
camera-csi-sensor-id = 0
drop-frame-interval = 0

当运行时,图像出现颠倒。

另一方面,当在同一台计算机上执行jetson推理示例时 ~/git/jetson-inference/build/aarch64/bin ./camera-viewer 图像看起来没问题,并在管道中显示 “nvvidconv flip-method = 2”。

那么问题是: 如何在配置文件中指示相机的方向来运行deepstream-app?

JetPack版本:4.2.2

2. 修改create_camera_source_bin

nvvideoconvert中不支持' flip method '属性。

修改create_camera_source_bin()运行如下命令:

nvarguscamerasrc bufapi-version=0 ! nvvidconv flip-method=2 ! video/x-raw,forma=NV12 ! nvvideoconvert ! video/x-raw(memory:NVMM),format=NV12

修改源代码后执行 log:

  • ./deepstream-app -c ../../../../samples/configs/deepstream-app/source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano-csiCam-test01.txt --gst-debug=3 2>&1 | tee debug.txt 0:00:00.242025044 10454 0x557ee9c290 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/mherrera/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS) (deepstream-app:10454): GLib-GObject-WARNING **: 02:31:43.653: cannot register existing type 'GstInterpolationMethod' (deepstream-app:10454): GLib-GObject-CRITICAL **: 02:31:43.654: g_param_spec_enum: assertion 'G_TYPE_IS_ENUM (enum_type)' failed (deepstream-app:10454): GLib-GObject-CRITICAL **: 02:31:43.654: validate_pspec_to_install: assertion 'G_IS_PARAM_SPEC (pspec)' failed 0:00:11.051427092 10454 0x557ef64850 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal: Creating random stream-id, consider implementing a deterministic way of creating a stream-id Creating LL OSD context new Runtime commands: h: Print this help q: Quit p: Pause r: Resume NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source. To go back to the tiled display, right-click anywhere on the window. **PERF: FPS 0 (Avg) **PERF: 0.00 (0.00) ** INFO: : Pipeline ready ** INFO: : Pipeline running Creating LL OSD context new GST_ARGUS: Creating output stream CONSUMER: Waiting until producer is connected... GST_ARGUS: Available Sensor modes : GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: Running with following settings: Camera index = 0 Camera mode = 2 Output Stream W = 1920 H = 1080 seconds to Run = 0 Frame Rate = 29.999999 GST_ARGUS: PowerService: requested_clock_Hz=627200000 GST_ARGUS: Setup Complete, Starting captures for 0 seconds GST_ARGUS: Starting repeat capture requests. CONSUMER: Producer has connected; continuing. nvbuf_utils: nvbuffer Payload Type not supported NvBufferGetParams failed for src_dmabuf_fd nvbuffer_transform Failed gst_nvvconv_transform: NvBufferTransform Failed 0:00:11.977186035 10454 0x557ef64850 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: Internal data stream error. 0:00:11.977212963 10454 0x557ef64850 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: streaming stopped, reason error (-5) ERROR from src_elem: Internal data stream error. Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstNvArgusCameraSrc:src_elem: streaming stopped, reason error (-5) Quitting GST_ARGUS: Cleaning up CONSUMER: Done Success GST_ARGUS: Done Success App run failed GST_ARGUS: PowerServiceHwVic::cleanupResources mherrera@JN01EXPL:/opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/deepstream-app$
  • 它应该是' bufapi-version=FALSE '

    g_object_set (G_OBJECT (bin->src_elem), "bufapi-version", FALSE, NULL);

    更新代码

    g_object_set (G_OBJECT (bin->src_elem), "bufapi-version", FALSE, NULL);

    log:

    ERROR from src_bin_muxer: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)

    作为参考,附加了输出文件和deepstream_source_bin.c。

    尝试了不同的代码更新deepstream_source_bin.c没有成功。

    回到原来的代码,只使用之前的建议进行更新,错误信息是一样的。

    3.使用NvBufSurface api

    下面是一个补丁供参考: sources\apps\apps-common\src\deepstream_source_bin.c:

    include 
    
    +#include "nvbufsurface.h"
    +#include "nvbufsurftransform.h"
     #include "gstnvdsmeta.h"
     #include "deepstream_common.h"
     #include "deepstream_sources.h"
    @@ -62,6 +64,64 @@ set_camera_v4l2_params (NvDsSourceConfig * config, NvDsSrcBin * bin)
       return TRUE;
     }
    
    +static GstPadProbeReturn
    +nvargus_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    +    gpointer u_data)
    +{
    +    GstBuffer *buf = (GstBuffer *) info->data;
    +    GstMapInfo outmap = GST_MAP_INFO_INIT;
    +    gst_buffer_map (buf, &outmap, GST_MAP_WRITE);
    +    NvBufSurface*  surface = (NvBufSurface *)outmap.data;
    +
    +    NvBufSurfTransformRect src_rect, dst_rect;
    +    src_rect.top   = 0;
    +    src_rect.left  = 0;
    +    src_rect.width = (guint) surface->surfaceList[0].width;
    +    src_rect.height= (guint) surface->surfaceList[0].height;
    +
    +    dst_rect.top   = 0;
    +    dst_rect.left  = 0;
    +    dst_rect.width = (guint) surface->surfaceList[0].width;
    +    dst_rect.height= (guint) surface->surfaceList[0].height;
    +
    +    NvBufSurface *dst_surface = NULL;
    +    NvBufSurfaceCreateParams nvbufsurface_create_params;
    +
    +    nvbufsurface_create_params.gpuId  = surface->gpuId;
    +    nvbufsurface_create_params.width  = (gint) surface->surfaceList[0].width;
    +    nvbufsurface_create_params.height = (gint) surface->surfaceList[0].height;
    +    nvbufsurface_create_params.size = 0;
    +    nvbufsurface_create_params.colorFormat = surface->surfaceList[0].colorFormat;
    +    nvbufsurface_create_params.layout = surface->surfaceList[0].layout;
    +    nvbufsurface_create_params.memType = surface->memType;
    +
    +    NvBufSurfaceCreate(&dst_surface,1,&nvbufsurface_create_params);
    +
    +    NvBufSurfTransformParams nvbufsurface_params;
    +    nvbufsurface_params.src_rect = &src_rect;
    +    nvbufsurface_params.dst_rect = &dst_rect;
    +    nvbufsurface_params.transform_flag =  0;
    +    nvbufsurface_params.transform_filter = NvBufSurfTransformInter_Default;
    +
    +    NvBufSurfTransformConfigParams transform_config_params;
    +    NvBufSurfTransform_Error err;
    +
    +    transform_config_params.compute_mode = NvBufSurfTransformCompute_Default;
    +    transform_config_params.gpu_id = surface->gpuId;
    +    transform_config_params.cuda_stream = NULL;
    +    err = NvBufSurfTransformSetSessionParams (&transform_config_params);
    +    // copy to dst_surface
    +    err = NvBufSurfTransform (surface, dst_surface, &nvbufsurface_params);
    +    // rototate 180 degree to original surface
    +    nvbufsurface_params.transform_flag =  NVBUFSURF_TRANSFORM_FLIP;
    +    nvbufsurface_params.transform_flip = NvBufSurfTransform_Rotate180;
    +    err = NvBufSurfTransform (dst_surface, surface, &nvbufsurface_params);
    +    NvBufSurfaceDestroy(dst_surface);
    +
    +    gst_buffer_unmap (buf, &outmap);
    +    return GST_PAD_PROBE_OK;
    +}
    +
     static gboolean
     create_camera_source_bin (NvDsSourceConfig * config, NvDsSrcBin * bin)
     {
    @@ -144,12 +204,16 @@ create_camera_source_bin (NvDsSourceConfig * config, NvDsSrcBin * bin)
         NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");
    
       } else {
    +    GstPad *src_pad;
    
         g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);
    
         gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, bin->cap_filter, NULL);
    
         NVGSTDS_LINK_ELEMENT (bin->src_elem, bin->cap_filter);
    +    src_pad = gst_element_get_static_pad (bin->src_elem, "src");
    +    gst_pad_add_probe (src_pad, GST_PAD_PROBE_TYPE_BUFFER,
    +        nvargus_src_pad_buffer_probe, NULL, NULL);
    
         NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");
       }

    sources\apps\sample_apps\deepstream-app\Makefile

    -24,6 +24,7 @@ APP:= deepstream-app
    
     TARGET_DEVICE = $(shell gcc -dumpmachine | cut -f1 -d -)
    
    +CUDA_VER:=10.0
     NVDS_VERSION:=4.0
    
     LIB_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/lib/
    @@ -41,10 +42,10 @@ PKGS:= gstreamer-1.0 gstreamer-video-1.0 x11
    
     OBJS:= $(SRCS:.c=.o)
    
    -CFLAGS+= -I../../apps-common/includes -I../../../includes -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=4
    +CFLAGS+= -I../../apps-common/includes -I../../../includes -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=4 -I /usr/local/cuda-$(CUDA_VER)/include
    
     LIBS+= -L$(LIB_INSTALL_DIR) -lnvdsgst_meta -lnvds_meta -lnvdsgst_helper -lnvds_utils -lm \
    -       -lgstrtspserver-1.0 -lgstrtp-1.0 -Wl,-rpath,$(LIB_INSTALL_DIR)
    +       -lgstrtspserver-1.0 -lgstrtp-1.0 -Wl,-rpath,$(LIB_INSTALL_DIR) -lnvbufsurface -lnvbufsurftransform
    
     CFLAGS+= `pkg-config --cflags $(PKGS)`

    按照上述修改后

    它起作用了,但只是部分起作用。

    RTSP的一些帧是旋转的,一些不是,它看起来是随机的

    已经将建议的补丁应用到源代码\apps\apps-common\src\deepstream_source_bin.c:

    } else{
    NVGSTDS_LINK_ELEMENT (bin->cap_filter, bin->nvvidconv);
    NVGSTDS_LINK_ELEMENT (bin->nvvidconv, bin->cap_filter1);
    
    /// added here, line 888
    GstPad *src_pad;
    src_pad = gst_element_get_static_pad (bin->nvvidconv, "src");
    gst_pad_add_probe (src_pad, GST_PAD_PROBE_TYPE_BUFFER,
        nvargus_src_pad_buffer_probe, NULL, NULL);
    ///
    
    NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter1, "src");

    以下是deepstream配置文件:

    [application]
    enable-perf-measurement=1
    perf-measurement-interval-sec=1
    #gie-kitti-output-dir=streamscl
    
    [tiled-display]
    enable=0
    rows=4
    columns=2
    width=1280
    height=720
    gpu-id=0
    nvbuf-memory-type=0
    
    [source0]
    enable=1
    #Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
    type=4
    uri=rtsp://192.168.2.216:8554/test
    #num-sources=8
    #drop-frame-interval=2
    latency=200
    gpu-id=0
    # (0): memtype_device   - Memory type Device
    # (1): memtype_pinned   - Memory type Host Pinned
    # (2): memtype_unified  - Memory type Unified
    cudadec-memtype=0
    
    [sink0]
    enable=1
    type=3
    #1=mp4 2=mkv
    container=1
    #1=h264 2=h265
    codec=1
    #encoder type 0=Hardware 1=Software
    enc-type=0
    sync=0
    #iframeinterval=10
    bitrate=2000000
    #H264 Profile - 0=Baseline 2=Main 4=High
    #H265 Profile - 0=Main 1=Main10
    profile=0
    output-file=testf.mp4
    source-id=0
    
    [streammux]
    gpu-id=0
    ##Boolean property to inform muxer that sources are live
    live-source=1
    batch-size=8
    ##time out in usec, to wait after the first buffer is available
    ##to push the batch even if the complete batch is not formed
    batched-push-timeout=40000
    ## Set muxer output width and height
    width=1920
    height=1080
    ##Enable to maintain aspect ratio wrt source, and allow black borders, works
    ##along with width, height properties
    enable-padding=0
    nvbuf-memory-type=0
    ## If set to TRUE, system timestamp will be attached as ntp timestamp
    ## If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached
    ## attach-sys-ts-as-ntp=1

    看起来添加了prob函数nvvideoconvert的源垫,而不是nvarguscamerasrc。

    确保申请了这个

    +    src_pad = gst_element_get_static_pad (bin->src_elem, "src");
    +    gst_pad_add_probe (src_pad, GST_PAD_PROBE_TYPE_BUFFER,
    +        nvargus_src_pad_buffer_probe, NULL, NULL);
    4. 调试gst_element_get_static_pad

    怀疑之前的猜测,有些帧是旋转的,有些没有,是有缺陷的。

    所以进一步测试它

    vbufsurface_params.transform_flip = NvBufSurfTransform_Rotate90;

    事实上,我的怀疑被证明是正确的。 旋转确实起作用,但似乎同一个坐标系被旋转了几次并被推到下游。

    所以不是有一个90度的旋转流,有一些帧旋转了90度,180度,270度,360度, 按照这个特定的顺序。

    请注意,上面的视频发生时,试图修补源代码\apps\apps-common\src\deepstream_source_bin.c:

    src_pad = gst_element_get_static_pad (bin->cap_filter, "src");
    or:
    src_pad = gst_element_get_static_pad (bin->nvvidconv, "src");
    or:
    src_pad = gst_element_get_static_pad (bin->cap_filter1, "src");

    尝试探测src_elem

    src_pad = gst_element_get_static_pad (bin->src_elem, "src");

    但是视频完全没有旋转。 请注意,这是用于RTSP输入,如果有帮助的话。 补丁在这个函数中:

    static gboolean
    create_rtsp_src_bin (NvDsSourceConfig * config, NvDsSrcBin * bin)
    {
       ....
    }

    欢迎分享,转载请注明来源:内存溢出

    原文地址: http://outofmemory.cn/langs/719461.html

    (0)
    打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
    上一篇 2022-04-25
    下一篇 2022-04-25

    发表评论

    登录后才能评论

    评论列表(0条)

    保存