RoboMaster 2018 Vision Framework

RoboMaster 2018 Vision Framework

RoboMaster is an annual robotics competition for teams of aspiring engineers to design and build next-generation robots for completing difficult tasks and hand-to-hand combat. As the first robot combat elimination tournament in China, RoboMaster has attracted close attention from hundreds of universities, nearly 1,000 high-tech enterprises and tens of thousands of fans from across the country since its debut. The RoboMaster 2018 Competition proposes new tasks and high requirements for vision detection. Faster speed with Faster control.

[VISION TASKS]

The tasks contain as list:
1. Armor Detection. [Same as the competitions before]
2. MNIST and LED Detection. [The RoboMaster 2017 and 2016]
3. Fire MNIST and LED Detection. [The RoboMaster 2018].

Besides we have to apply the vision algorithms on the different robot platforms to gain the better control performance,  I have to rebuild the code and algorithms of vision framework, not using the yolo algorithm framework to detect anymore.

[FILE SYSTEM]

Based on the experiences and knowledges I have learned, I build the RoboMaster 2018 Vision Framework as below.


The code framework for vision contains the subdirectories as list :

[src] This directory contains all the .cpp files and .hpp files of this algorithm framework.
[CMakeLists.txt] This file is the config file of CMake Building System based on different platforms.
[config] This directory contains the file for the parameters’ configuration of different vision tasks.
[COPYRIGHT] This file is the copyright file to show the details about the code owner and backgrounds.
[mnist] This directory contains the file of CNN Model and MNIST Datasets.
[pic] This directory contains some test pictures when the codes running.
[proto] This directory contains the definitions of all the parameters’ configuration of the vision tasks with the protobuf tool.
[python] This directory contains some python files about CNN test on MNIST Datasets (not used).
[README] This file is the instructions about this algorithm framework, when you use it you have to read it first.
[VERSION] This file is used to identify the different version of this algorithm framework.

[PREPARATION]

Platform: Nvidia Jetson TX2
System: Ubuntu 16.04
OpenCV: opencv 3.4.0
Protobuf: protobuf 3.5.1
Eigen: eigen 3.3.4

How to Setup the OpenCV 3.4.0 or other OpenCV3

OpenCV (Open Source Computer Vision Library) is released under a BSD license and hence it’s free for both academic and commercial use. It has C++, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android. OpenCV was designed for computational efficiency and with a strong focus on real-time applications. Written in optimized C/C++, the library can take advantage of multi-core processing. Enabled with OpenCL, it can take advantage of the hardware acceleration of the underlying heterogeneous compute platform.

1. Download the OpenCV3.4.0 package “OpenCV-3.4.0.zip“. The size of image is about 91.04MB.
2. Open the terminal and then setup some required packages as list before you setup the opencv packages:

sudo apt-get install cmake
sudo apt-get install build-essential libgtk2.0-dev libavcodec-dev libavformat-dev
sudo apt-get install libjpeg.dev libtiff4.dev libswscale-dev libjasper-dev

3. unzip the OpenCV3.4.0 package and then enter this directory with the command:

cd ~/Downloads/opencv-3.4.0
mkdir release
cd release
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j8
sudo make install

4. All the process costs about 20 minutes depends on the machine you used, the OpenCV will be installed at the directory /usr/local .
5. Run the command below to check if the install process is success or not.if the command return the version number, the OpenCV is successful setup on this machine.

pkg-config --modversion opencv 

TIPs: If you have setup the OpenCV2 (no matter the version you used) on your ubuntu system, you also can setup the OpenCV3 into the different directory, remember the directory you enter into the command “cmake .. -D CMAKE_INSTALL_PREFIX= [the directory you used]” and then edit the CMakeList.txt ( I will mention that after). I recommend you use the OpenCV3 for building this code framework although I provide the OpenCV2 code in framework which not work prefectly.

How to Setup the Protobuf 3.5.1

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the “old” format.

1. Download the Protobuf 3.5.1 package “Protobuf-cpp-3.5.1.zip“. The size of image is about 5.04MB.
2. unzip the Protobuf 3.5.1 package and then enter this directory with the command:

cd ~/Downloads/protobuf-cpp-3.5.1
./configure --prefix=/usr/local/protobuf
make -j8
make check
sudo make install

3. All the process costs about 40 minutes depends on the machine you used, the make process and the check process cost most. the Protobuf will be installed at the directory /usr/local/protobuf .
4. After the setup process, you should change the environment variables to make the protobuf work. Run the command below:

sudo gedit /etc/profile

5. Enter these lines into the /etc/profile file:

export PATH=$PATH:/usr/local/protobuf/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/protobuf/lib

6. Make the environment variables take effects:

sudo ldconfig

7. Run the command below to check if the install process is success or not.if the command return the version number, the Protobuf is successful setup on this machine.

protoc --version 
How to Setup the Eigen 3.3.4

Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.

1. Download the Eigen 3.3.4 package “Eigen-3.3.4.zip“. The size of image is about 3.05MB.
2. unzip the Eigen 3.3.4 package and there are two ways to use the Eigen packages.
3. One way: You can use right away the headers in the Eigen/ subdirectory. In order to install, just copy this Eigen/ subdirectory to your favorite location. If you also want the unsupported features, copy the unsupported/ subdirectory too.
4. Another way : enter this directory with the command:

cd ~/Downloads/3.3.4
mkdir release
cd release
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local/ ..
sudo make install

[CMAKE]

The vision framework contains three CMakeList.txt files to make the algorithms work : one under the  local directory and others under /src and /proto subdirectories. I will introduce them as details.

Local Directory : CMakeList.txt

This File is core of framework’s cmake system. With checking the necessary packages had installed on this machine system and link the code with the necessary environment variables, the framework need all of the packages as list: OpenCV, Protobuf, Eigen and CUDA.

****CMakeList.txt****
#**********************************************************************************
#
#     Copyright (C) 2018  Mei Xinhu
#
#     RoboMaster 2018 
#     Nanjing University of Science and Technology
#     A.I.R. Artificial Intelligence Robot Team
#
#**********************************************************************************

PROJECT(Alliance)

CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
CMAKE_POLICY(SET CMP0054 OLD)

set(CMAKE_CXX_FLAGS_RELEASE "$ENV{CXXFLAGS} -O3 ")
SET(CMAKE_BUILD_TYPE Release)
#SET(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")

SET(OpenCV_DIR "/usr/local/opencv3.4/share/OpenCV")

# find require opencv package
FIND_PACKAGE(OpenCV REQUIRED)
if(OpenCV_FOUND)
	MESSAGE(STATUS "opencv library found")
	LIST( APPEND ThirdParty_LIBS ${OpenCV_LIBS} )
    INCLUDE_DIRECTORIES( ${OpenCV_INCLUDE_DIRS} )
else()
	MESSAGE(FATAL_ERROR "opencv library is needed but cant be found")
endif(OpenCV_FOUND)

# uncomment this for OpenMP support
if(APPLE)

elseif(UNIX)
	FIND_PACKAGE(OpenMP REQUIRED)
	if(OPENMP_FOUND)
		MESSAGE(STATUS "openmp library found")
		SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
		SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
		SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}")
	else()
		MESSAGE(FATAL_ERROR "openmp library is needed but cant be found")
	endif()
endif()

# find require protobuf package
if(APPLE)
    SET(PROTOBUF_DIR "/usr/local/Cellar/protobuf/3.5.1")
endif()

FIND_PACKAGE(Protobuf REQUIRED)
if(PROTOBUF_FOUND)
    MESSAGE(STATUS "protobuf library found")
else()
    MESSAGE(FATAL_ERROR "protobuf library is needed but cant be found")
endif()

FIND_PACKAGE(CUDA)
set(CUDA_NVCC_FLAGS -gencode arch=compute_53,code=sm_53;-G;-g)

FIND_PACKAGE (Eigen3 3.3 REQUIRED NO_MODULE)
if(EIGEN3_FOUND)
    MESSAGE(STATUS "Eigen3 found")
else()
    MESSAGE(FATAL_ERROR "Eigen3 is needed but cant be found")
endif()

# uncomment this for c++11 support
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")

#SET(CMAKE_VERBOSE_MAKEFILE ON)

SET(PROTO_FILE ${CMAKE_CURRENT_SOURCE_DIR}/proto/param.proto)

ADD_SUBDIRECTORY(proto)

ADD_SUBDIRECTORY(src)

INCLUDE_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR})

发表评论