linux 安装codeql环境 (二)codeql database create通过报错分析其流程

linux 安装codeql环境 (二)codeql database create通过报错分析其流程,第1张

尝试过很多解决方案之后无果

决定研究一下它的整个流程
我们首先把他的整个的报错信息拿出来

Initializing database at /home/wuangwuang/Desktop/qemu/test.
Running build command: [/home/wuangwuang/CodeQL/codeql/cpp/tools/autobuild.sh]
[2022-03-16 09:16:34] [build-stderr] + TOOLS_DIR=/home/wuangwuang/CodeQL/codeql/cpp/tools
[2022-03-16 09:16:34] [build-stderr] + /home/wuangwuang/CodeQL/codeql/cpp/tools/detect_source_root
[2022-03-16 09:16:34] [build-stdout] detect_source_root: Using build system found in '.'
[2022-03-16 09:16:34] [build-stderr] ln: 无法创建符号链接'_lgtm_detected_source_root/.': 文件已存在
[2022-03-16 09:16:34] [build-stderr] + '[' -L _lgtm_detected_source_root ']'
[2022-03-16 09:16:34] [build-stderr] ++ readlink _lgtm_detected_source_root
[2022-03-16 09:16:34] [build-stderr] + cd .
[2022-03-16 09:16:34] [build-stderr] + export CXXFLAGS=-fpermissive
[2022-03-16 09:16:34] [build-stderr] + CXXFLAGS=-fpermissive
[2022-03-16 09:16:34] [build-stderr] + configure_prefix_opt=
[2022-03-16 09:16:34] [build-stderr] + '[' -f configure.ac ']'
[2022-03-16 09:16:34] [build-stderr] + '[' -f configure.in ']'
[2022-03-16 09:16:34] [build-stderr] + '[' -f CMakeLists.txt ']'
[2022-03-16 09:16:34] [build-stderr] + '[' -f meson.build ']'
[2022-03-16 09:16:34] [build-stderr] + mkdir -p _lgtm_build_dir
[2022-03-16 09:16:34] [build-stderr] + cd _lgtm_build_dir
[2022-03-16 09:16:34] [build-stderr] + meson ..
[2022-03-16 09:16:34] [build-stderr] /home/wuangwuang/CodeQL/codeql/cpp/tools/do-prebuild:行79: meson:未找到命令
[2022-03-16 09:16:34] [build-stderr] + cd ..
[2022-03-16 09:16:34] [build-stderr] + rm -rf _lgtm_build_dir
[2022-03-16 09:16:34] [build-stderr] + try_configure
[2022-03-16 09:16:34] [build-stderr] + build_dir=.
[2022-03-16 09:16:34] [build-stderr] + root_dir=.
[2022-03-16 09:16:34] [build-stderr] + grep -q 'AC_MSG_.* configure in a separate.* directory' configure.in configure.ac
[2022-03-16 09:16:34] [build-stderr] + for configure in configure configure.gnu
[2022-03-16 09:16:34] [build-stderr] + '[' -x configure ']'
[2022-03-16 09:16:34] [build-stderr] + cd .
[2022-03-16 09:16:34] [build-stderr] + /home/wuangwuang/CodeQL/codeql/cpp/tools/configure-wrapper ./configure
[2022-03-16 09:16:34] [build-stdout] Using './build' as the directory for build output
[2022-03-16 09:16:35] [build-stdout] ERROR: glib-2.56 gthread-2.0 is required to compile QEMU
[2022-03-16 09:16:35] [build-stdout] /home/wuangwuang/CodeQL/codeql/cpp/tools/configure-wrapper: Configure failed
[2022-03-16 09:16:35] [build-stderr] + cd .
[2022-03-16 09:16:35] [build-stderr] + for configure in configure configure.gnu
[2022-03-16 09:16:35] [build-stderr] + '[' -x configure.gnu ']'
[2022-03-16 09:16:35] [build-stderr] + rm -rf _lgtm_build_dir
[2022-03-16 09:16:35] [build-stderr] + for bootstrap in bootstrap.sh bootstrap autogen.sh
[2022-03-16 09:16:35] [build-stderr] + '[' -x bootstrap.sh ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f bootstrap.sh ']'
[2022-03-16 09:16:35] [build-stderr] + for bootstrap in bootstrap.sh bootstrap autogen.sh
[2022-03-16 09:16:35] [build-stderr] + '[' -x bootstrap ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f bootstrap ']'
[2022-03-16 09:16:35] [build-stderr] + for bootstrap in bootstrap.sh bootstrap autogen.sh
[2022-03-16 09:16:35] [build-stderr] + '[' -x autogen.sh ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f autogen.sh ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f wscript ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f Kbuild ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f configure.ac ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f configure.in ']'
[2022-03-16 09:16:35] [build-stderr] ++ echo './*.pro'
[2022-03-16 09:16:35] [build-stderr] + '[' './*.pro' '!=' './*.pro' ']'
[2022-03-16 09:16:35] [build-stderr] + exit 0
[2022-03-16 09:16:35] [build-stderr] + export VERBOSE=1
[2022-03-16 09:16:35] [build-stderr] + VERBOSE=1
[2022-03-16 09:16:35] [build-stderr] + '[' -L _lgtm_detected_source_root ']'
[2022-03-16 09:16:35] [build-stderr] ++ readlink _lgtm_detected_source_root
[2022-03-16 09:16:35] [build-stderr] + cd .
[2022-03-16 09:16:35] [build-stderr] + '[' -d _lgtm_build_dir ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f SConstruct ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f wscript ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f Makefile ']'
[2022-03-16 09:16:35] [build-stderr] + make
[2022-03-16 09:16:35] [build-stdout] changing dir to build for make ""...
[2022-03-16 09:16:35] [build-stdout] make[1]: 进入目录“/home/wuangwuang/Desktop/qemu/build”
[2022-03-16 09:16:35] [build-stdout] make[1]: 离开目录“/home/wuangwuang/Desktop/qemu/build”
[2022-03-16 09:16:35] [build-stderr] make[1]: Makefile: 没有那个文件或目录
[2022-03-16 09:16:35] [build-stderr] make[1]: *** 没有规则可制作目标“Makefile”。 停止。
[2022-03-16 09:16:35] [build-stderr] make: *** [GNUmakefile:11:all] 错误 2
[2022-03-16 09:16:35] [build-stderr] + '[' -f build.ninja ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -d ../_lgtm_build_dir ']'
[2022-03-16 09:16:35] [build-stdout] Semmle autobuild: no supported build system detected.
[2022-03-16 09:16:35] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/home/wuangwuang/CodeQL/codeql/tools/linux64/preload_tracer, /home/wuangwuang/CodeQL/codeql/cpp/tools/autobuild.sh])
[2022-03-16 09:16:35] [build-stderr] + for f in build build.sh
[2022-03-16 09:16:35] [build-stderr] + '[' -x build ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f build ']'
[2022-03-16 09:16:35] [build-stderr] + for f in build build.sh
[2022-03-16 09:16:35] [build-stderr] + '[' -x build.sh ']'
[2022-03-16 09:16:35] [build-stderr] + '[' -f setup.py ']'
[2022-03-16 09:16:35] [build-stderr] + echo 'Semmle autobuild: no supported build system detected.'
[2022-03-16 09:16:35] [build-stderr] + exit 1
A fatal error occurred: Exit status 1 from command: [/home/wuangwuang/CodeQL/codeql/cpp/tools/autobuild.sh]


首先看的到它运行的是codeql/tools/autobuild.sh

autobuild.sh

#!/bin/sh

set -eu

# Directory where the autobuild scripts live.
AUTOBUILD_ROOT="$CODEQL_EXTRACTOR_CPP_ROOT/tools"

"$AUTOBUILD_ROOT/do-prebuild"

"$AUTOBUILD_ROOT/do-build"

长这样
刚开始的set -eu没啥,就是设置了shell的格式

下面的这个AUTOBUILD_ROOT/tools就是当前目录
意思就是又接着运行了当前目录下的do-prebuild 以及 do-build脚本

一个一个分析

#!/bin/bash -x

TOOLS_DIR="${CODEQL_EXTRACTOR_CPP_ROOT}/tools"

"${TOOLS_DIR}/detect_source_root"
#看得出来首先调用了detect_source_root
#detect_source_root在下面分析了
#此脚本创建一个名为 _lgtm_detected_source_root 的符号链接,指向最有可能包含此项目的构建系统的目录。


#下面这个if判断应该就是判断_lgtm_detected_source_root是不是一个软连接
#我们显然判断成功了
#就进入了当前目录
#其实_lgtm_detected_sourch_root就是项目地址
#我们可以用codeql的参数--source-root去指定
#我们不指定就是当前目录
#所以一直到这里就是创建了项目地址的一个软连接
if [ -L _lgtm_detected_source_root ]; then
  cd "$(readlink _lgtm_detected_source_root)"
fi

#就是用了个环境变量 把CXXFLAGS代表了-fpermissive
export CXXFLAGS=-fpermissive

#if 的-f参数指的是如果文件存在并且是一个普通文件
#在我们这个示例中  qemu是根目录
#在根目录只能找到一个configure的bash脚本
#并没有if语句判断的这两个文件
#所以直接向下走
configure_prefix_opt=
if [ -f configure.ac ] || [ -f configure.in ]; then
  # If a project seems to be autoconf-based, add this option to configure. It
  # is required by glibc, and it should do no harm for other projects.
  #翻译过来:如果一个项目似乎是基于autoconf的,请在configure中添加这个选项。它是glibc所要求的,对其他项目应该没有坏处。
  configure_prefix_opt="--prefix=/usr"
fi

#定义了一个try_configure函数

function try_configure() {
  build_dir=.
  root_dir=.
  #在configure.in和configure.ac两个文件中去搜索字符串
  #把标准错误 也就是错误流重定向到了垃圾站
  #也就是如果报错不输出
  #我们前面就判断过根本没这两个文件
  #所以if直接失败
  if grep -q 'AC_MSG_.* configure in a separate.* directory' \
        configure.in configure.ac 2>/dev/null; then
    # Keeping build artifacts separate from the source is always good practice,
    # but it is likely to be the least tested mode for most projects, so we only
    # do it if the configure script seems to require or suggest it. It is
    # required by glibc.
    #翻译过来:将构建工件与源代码分开始终是一种很好的做法,但对于大多数项目来说,它可能是测试最少的模式,因此我们仅在配置脚本似乎需要或建议时才这样做。 glibc 需要它。

    build_dir=_lgtm_build_dir
    root_dir=..
    mkdir -p "$build_dir"
  fi
	
	
  # The name `configure.gnu` is mentioned as an alternative to `configure` in
  # the autoconf manual and is used by Perl 5.
  #翻译过来:名称 `configure.gnu` 在 autoconf 手册中被提及作为 `configure` 的替代品,并且被 Perl 5 使用。
  #也就是前面没找到configure相关文件就找找替代品
  #if -x就是判断是否存在可执行程序configure
  #我们这里的qemu是存在的
  #所以就执行了/home/wuangwuang/CodeQL/codeql/cpp/tools/configure-wrapper ./configure
  #但是我这里它又报错了
  #因为没有库或者其他等原因
  #这里就又失败了
  for configure in configure configure.gnu; do
    if [ -x $configure ]; then
      cd $build_dir
      "${TOOLS_DIR}/configure-wrapper" \
        $root_dir/$configure $configure_prefix_opt  \
        && [ -f Makefile ] \
        && exit 0
      cd $root_dir
    fi
  done
  # In case configure failed, don't leave a broken _lgtm_build_dir that will
  # get picked up by `do-build`.
  rm -rf _lgtm_build_dir
}

#查看有没有CMakeLists.txt文件
#我们好像也没有
if [ -f CMakeLists.txt ]; then
  mkdir -p _lgtm_build_dir
  cd _lgtm_build_dir
  # Previous versions of the lgtm.com docker image for C++ had the CMake
  # wrapper in this location. As long as they may still be in use, we try this
  # location first.
  CMAKE_WRAPPER=/opt/work/autobuild/cmake-wrapper
  if ! [ -x $CMAKE_WRAPPER ]; then
    CMAKE_WRAPPER=cmake
  fi
  $CMAKE_WRAPPER \
    -DCMAKE_VERBOSE_MAKEFILE=ON \
    -DBUILD_DOCS=OFF \
    -DCATKIN_ENABLE_TESTING=OFF \
    -DBUILD_DOCUMENTATION=OFF \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_CXX_FLAGS="$CXXFLAGS" \
    .. \
    && exit 0
  cd ..
  # In case CMake failed, don't leave a broken _lgtm_build_dir that will
  # get picked up by `do-build`.
  rm -rf _lgtm_build_dir
fi

#判断有没有meson.build文件
#这个有
if [ -f meson.build ]; then
	
	#首先递归的创建了文件夹
	#进去之后使用meson命令
	#然后我这里因为没有meson命令报错了。
	#然后再把这个文件夹删掉
  mkdir -p _lgtm_build_dir
  cd _lgtm_build_dir
  meson .. && exit 0
  cd ..
  # In case meson failed, don't leave a broken _lgtm_build_dir that will
  # get picked up by `do-build`.
  rm -rf _lgtm_build_dir
fi
#我的qemu首先在这里报了个错
#我们可以把meson命令装上以后再去试一下
#但是我还是决定将错就错继续分析下去


try_configure
#调用了try_configure
#但是结果是失败了

# The script that invokes autotools is by convention most often called
# autogen.sh, but we also try a few other names. The following are the most
# popular names according to the build instructions in OS X Homebrew, found
# with:
#翻译过来:按照惯例,调用 autotools 的脚本最常称为 autogen.sh,但我们也尝试使用其他一些名称。根据 OS X Homebrew 中的构建说明,以下是最流行的名称,可通过以下方式找到:

# $ find /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula \
#     -name \*.rb | xargs cat \
#     | grep -o 'system .* if build.head' | sort | uniq -c | sort -n

#然后就找文件嘛  但是不巧的是三个文件都没有 这里的for就也结束了
for bootstrap in bootstrap.sh bootstrap autogen.sh; do
  if [ -x $bootstrap ]; then
    ./$bootstrap && try_configure
    break
  elif [ -f $bootstrap ]; then
    bash $bootstrap && try_configure
    break
  fi
done

#下面又找了好几个文件
#但是又都没有
# The Waf "meta build system"
if [ -f wscript ]; then
  # Projects using the Waf build system often have it embedded in the
  # repository. If not, we will copy it in.
  if ! [ -x waf ] && [ -x /opt/work/autobuild/waf ]; then
    cp /opt/work/autobuild/waf .
  fi

  if [ -x waf ]; then
    ./waf configure && exit 0
  fi
fi

if [ -f Kbuild ] && [ -f Kconfig ]; then
  # This is some variation of the Linux kernel build system
  make defconfig && exit 0
fi

# If we've fallen through to here then configure has failed. But if the project
# still seems to use autotools then we run autoreconf, which is a wrapper that
# runs autoconf, automake and releated tools.
if [ -f configure.ac ] || [ -f configure.in ]; then
  # configure refuses to run if this file does not exist. This is a lint check
  # to help the author, not something we should stop the build on.
  # See https://www.gnu.org/software/autoconf/manual/autoconf.html, in the
  # bullet labeled "Macro: AC_PROG_INSTALL".
  touch install-sh

  libtoolize -ci
  autoreconf -i

  try_configure
fi

#就直接来到了最后这个地方
# If there is one or more *.pro files, it may be a Qt project. A successful
# run of QMake will overwrite `Makefile`, so we only try QMake if there is no
# `Makefile` already.
#如果有一个或多个 *.pro 文件,它可能是一个 Qt 项目。成功运行 QMake 会覆盖 `Makefile`,所以我们只在没有 `Makefile` 的情况下尝试 QMake。
#显然我们不是qt项目
#所以它最后啥都没干就退出去了  他就去继续执行do-build脚本去了
if [ "$(echo ./*.pro)" != "./*.pro" ] && ! [ -f Makefile ]; then
  qmake && exit 0
fi

# Nothing worked. Hopefully that just means nothing had to be done.
exit 0

detect_source_root

#!/bin/bash
# This script creates a symlink named _lgtm_detected_source_root, pointing to the directory that most likely contains the build system for this project.

echo

# This function creates the symlink named _lgtm_detected_source_root,
# pointing to the first argument, and exits. Using this function helps to
# ensure that (1) the symlink gets the correct name and (2) we always exit
# after creating the symlink so later stages are not at risk of reading the
# symlink as part of their analysis.
function create_symlink_and_exit() {
  ln -s "" _lgtm_detected_source_root
  echo
  exit
}

# Returns success if the first argument is a directory that appears to contain
# files for a major build system.
function convincingly_has_build_system() {
  ( cd "" &&
    ( [[
        -x configure || \
        -f configure.in || -f configure.ac || \
        -f CMakeLists.txt || \
        -f meson.build || \
        -f wscript || \
        -f SConstruct ]] )
  )
}

# Returns success if the first argument is a directory that appears to contain
# files for some build system, possibly a home-made or minor one.
function may_have_build_system() {
  convincingly_has_build_system "" || \
  ( cd "" &&
    ( [[
        ( -f Kbuild && -f Kconfig ) || \
        -f Makefile || -f makefile || -f GNUmakefile || \
        "$(echo ./*.pro)" != "./*.pro" || \
        ( -x build && -f build ) || \
        -x build.sh || \
        -f setup.py ]] )
  )
}

# Holds if the first argument is a directory that does not appear to contain
# third-party code.
function is_allowed_dir() {
  ! [ -L "" ] && \
    [ -d "" ] && \
    echo "" | grep -vq '/\(thirdparty\|third_party\|third-party\|vendor\|external\|3rdparty\|_vendor\)$'
}

# Takes a list of file names as argument. If that list has length 1, and that
# one argument is a directory for which `is_allowed_dir` returns success, print
# that directory and return success. Otherwise, return failure.
function single_dir() {
  if [ $# -eq 1 ] && is_allowed_dir ""; then
    echo ""
  else
    false
  fi
}

# Takes a directory name as argument traverses down its subdirectories for as
# long as each directory is _trivial_, stopping when it reaches a non-trivial
# directory and printing that. A _trivial_ directory is one containing nothing
# but a single subdirectory (and possibly dot-files).
# This function will not traverse into a directory that seems to contain
# third-party code.

# Takes a directory name as argument traverses down its subdirectories for as long as each directory is _trivial_, stopping when it reaches a non-trivial directory and printing that. A _trivial_ directory is one containing nothing but a single subdirectory (and possibly dot-files). This function will not traverse into a directory that seems to contain third-party code.

function first_nontrivial_dir() {
  curdir=""
  while the_single_dir="$(single_dir "$curdir"/*)"; do
    curdir="$the_single_dir"
  done
  echo "$curdir"
}

# First, find the top directory, which is usually the root of the repository.
#首先,找到顶层目录,通常是存储库的根目录。

top_candidate="$(first_nontrivial_dir .)"
#top_candidate是调用first_nontrivial_dir函数的返回值
#调用函数传入的参数是.  也就是当前目录

#我刚刚跑脚本过程中根据下面的echo出来的信息,找到的top_candidate就是当前目录.
#不知道这个.找的对不对 我们先按照这个来分析
#可以看下面的图1

#may这个函数就是传入的目录里面有建立系统的文件
#我们的当前目录.就满足了条件
#然后调用了下面的create函数
#传入的参数还是当前目录.

#然后通过create_symlink_and_exit创建了一个软连接
#_lgtm_detected_source_root指向.   就退出了
if may_have_build_system "$top_candidate"; then
  echo "detect_source_root: Using build system found in '$top_candidate'"
  create_symlink_and_exit "$top_candidate"
fi

# If we are certain that the top directory cannot possibly contain a build
# system, check each subdirectory to see if there is a single one with a build
# system. We try to be careful not to confuse some random directory with a
# Makefile for the main source directory, so we call
# `convincingly_has_build_system` instead of `may_have_build_system`.

#上面的注释翻译过来是这样的:如果我们确定顶层目录不可能包含构建系统,请检查每个子目录以查看是否有一个具有构建系统的子目录。我们尽量小心不要将一些随机目录与主源目录的 Makefile 混淆,因此我们调用 `convincingly_has_build_system` 而不是 `may_have_build_system`。
#说白了就是找不到代码根目录该咋办
#我们找到了 也就不细分析了

preferred_subdir=""
for below_top_candidate in "$top_candidate"/*; do
  is_allowed_dir "$below_top_candidate" || continue;
  subdir="$(first_nontrivial_dir "$below_top_candidate")"
  if convincingly_has_build_system "$subdir"; then
    if [ -n "$preferred_subdir" ]; then
      echo "detect_source_root: At least these two subdirs have build system files in them:"
      echo "  $preferred_subdir"
      echo "  $subdir"
      echo "To avoid ambiguity, build will be attempted from '$top_candidate'"
      echo "even though no build system was found there."
      create_symlink_and_exit "$top_candidate"
    else
      preferred_subdir="$subdir"
    fi
  fi
done

if [ -n "$preferred_subdir" ]; then
  echo "detect_source_root: Build will be attempted from '$preferred_subdir'"
  create_symlink_and_exit "$preferred_subdir"
else
  echo "detect_source_root: Build will be attempted from '$top_candidate'"
  echo "even though no build system was found there."
  create_symlink_and_exit "$top_candidate"
fi

图1

我们继续向下分析
do-build文件

#!/bin/bash -x

# Enables verbose build output for CMake >= 3.14
export VERBOSE=1
#设定了一个环境变量

#进入了项目文件 现在也就是qemu的根目录
if [ -L _lgtm_detected_source_root ]; then
  cd "$(readlink _lgtm_detected_source_root)"
fi

#如果有这个文件,就进入
#但是我们还记得上面do_prebuild中做的事情
#根本没有建立这个文件
#所以这里就失败了
if [ -d _lgtm_build_dir ]; then
  cd _lgtm_build_dir
fi

#失败之后各种文件都找不到
if [ -f SConstruct ]; then
  scons && exit 0
fi

if [ -f wscript ] && [ -x waf ]; then
  ./waf build && exit 0
fi

if [ -f Makefile ] || [ -f makefile ] || [ -f GNUmakefile ]; then
  make && exit 0
fi

if [ -f build.ninja ]; then
  ninja && exit 0
fi

if [ -d ../_lgtm_build_dir ]; then
  cd ..
fi

for f in build build.sh; do
  if [ -x $f ] && [ -f $f ]; then
    ./$f && exit 0
  fi
done

if [ -f setup.py ]; then
  python setup.py build && exit 0
fi

#那显然没办法输出了最后这个信息
echo "Semmle autobuild: no supported build system detected."
exit 1

欢迎分享,转载请注明来源:内存溢出

原文地址: http://outofmemory.cn/langs/741812.html

(0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
上一篇 2022-04-28
下一篇 2022-04-28

发表评论

登录后才能评论

评论列表(0条)

保存