实战搞定gRPC之移植篇

一、交叉编译protobuf

1.配置交叉编译器

export PATH=$PATH:/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin:/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/lib

source /opt/EC20_crosstool/ql-ol-crosstool/ql-ol-crosstool-env-init

2.编译

1.cd /home/workspace/project/arm_grpc/third_party/protobuf/cmake
2.mkdir arm_build
3.cd arm_build
4.cmake .. -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_CXX_COMPILER:FILEPATH=/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/arm-oe-linux-gnueabi-g++ -DCMAKE_C_COMPILER:FILEPATH=/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/arm-oe-linux-gnueabi-gcc -DCMAKE_BUILD_TYPE:STRING=RELEASE -DCMAKE_INSTALL_PREFIX=/home/workspace/test/arm_install -DBUILD_SHARED_LIBS=ON
5.make -j8
6.make install -j8

编译的时候报错

原因是/usr/local/lib/libz.so是Ubuntu下编译的，因此需要交叉编译libz.so，在EC20编译器中搜索到有这个库。为了图省事，直接将EC20中的libz.so拷贝替换/usr/local/lib/中的libz.so。

编译成功

二、交叉编译grpc

1.mkdir -p cmake/arm_build
2.cd cmake/arm_build
3.cmake ../.. -DCMAKE_CXX_COMPILER:FILEPATH=/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/arm-oe-linux-gnueabi-g++ -DCMAKE_C_COMPILER:FILEPATH=/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/arm-oe-linux-gnueabi-gcc -DCMAKE_BUILD_TYPE:STRING=RELEASE -DCMAKE_INSTALL_PREFIX=/home/workspace/test/arm_install -DBUILD_SHARED_LIBS=ON

执行make报错

/home/workspace/test/grpc/third_party/boringssl-with-bazel/linux-x86_64/crypto/chacha/chacha-x86_64.S: Assembler messages:
/home/workspace/test/grpc/third_party/boringssl-with-bazel/linux-x86_64/crypto/chacha/chacha-x86_64.S:1633: Error: junk at end of line, first unrecognized character is `,'
make[2]: *** [third_party/boringssl-with-bazel/CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o] Error 1
make[1]: *** [third_party/boringssl-with-bazel/CMakeFiles/crypto.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

使用make -j8 HAS_PKG_CONFIG=false PROTOBUF_CONFIG_OPTS="–host=arm-oe-linux-gnueabi --with-protoc=/usr/local/bin/protoc" 错误还是一样。
具体原因还是没找到。放弃cmake的方式进行编译。直接在grpc目录下使用make的方式编译，不通过cmake重新构造Makefile。

export GRPC_CROSS_COMPILE=true
export GRPC_CROSS_AROPTS=“cr --target=elf32-little”

make clean  注意一定要先清除，不然会编译出错。
make -j8 HAS_PKG_CONFIG=false CC=arm-oe-linux-gnueabi-gcc CXX=arm-oe-linux-gnueabi-g++ RANLIB=arm-oe-linux-gnueabi-ranlib LD=arm-oe-linux-gnueabi-ld LDXX=arm-oe-linux-gnueabi-g++ AR=arm-oe-linux-gnueabi-ar PREFIX=/home/workspace/test/arm_install PROTOBUF_CONFIG_OPTS="--host=arm-oe-linux-gnueabi --with-protoc=/usr/local/bin/protoc"

没办法make install，因为是交叉编译，只需将libs/opt中的库文件拷贝到设备中即可。
之前因为没有添加export GRPC_CROSS_COMPILE=true
export GRPC_CROSS_AROPTS=“cr --target=elf32-little”。
导致libs/opt并没有看到交叉编译过的grpc库。


添加之后重新编译，编译成功。


说明arm版本已经编译ok了。

还有一种比较简单的方式判断当前库是交叉编译还是gcc版本。因为我电脑是64位，开发板是32位，所以直接使用objdump来进行判断。例如：
objdump -a libgrpc.a

三、交叉编译demo与设备实测

交叉编译helloworld demo

使用make方式进行编译。只需要交叉编译greeter_server和greeter_client测试基本功能就好了，顺便看看哪些库可以不用拷贝到设备中。

cd /home/workspace/test/grpc/examples/cpp/helloworld

make greeter_server HAS_PKG_CONFIG=false CC=arm-oe-linux-gnueabi-gcc CXX=arm-oe-linux-gnueabi-g++ RANLIB=arm-oe-linux-gnueabi-ranlib LD=arm-oe-linux-gnueabi-ld LDXX=arm-oe-linux-gnueabi-g++ AR=arm-oe-linux-gnueabi-ar PREFIX=/home/workspace/project/grpc/arm_build PROTOBUF_CONFIG_OPTS="–host=arm-oe-linux-gnueabi --with-protoc=/usr/local/bin/protoc"

如果忘记将libs/opt中的库拷贝到交叉编译protobuf 安装的目录/home/workspace/test/arm_install中的话就会报错。

发现只要链接这三个库就可以。于是我只拷贝这几个库（因为编译出来得库都挺大的）。而且还要修改下helloworld目录下的makefile
修改前

修改后

交叉编译成功


将这些库拷贝到设备中

设置好设备的环境变量。
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usrdata/grpc/lib

运行时发现，占用空间都一致应该是同一个动态库，因此为了节省空间创建软连接。


最终在EC20设备上成功运行。

但存在个问题，库太占用空间了。后面得想办法裁剪一下。

过程中遇到的一些坑爹的问题，后面重头操作几次后，总算解决了。顺便记录下自己过程中遇到的奇奇怪怪的问题。

错误:

[HOSTCXX] Compiling src/compiler/cpp_generator.cc
In file included from ./src/compiler/config_protobuf.h:22:0,
                 from ./src/compiler/config.h:22,
                 from ./src/compiler/cpp_generator.h:29,
                 from src/compiler/cpp_generator.cc:21:
include/grpcpp/impl/codegen/config_protobuf.h:30:37: fatal error: google/protobuf/message.h: No such file or directory
 #include <google/protobuf/message.h>
                                     ^
compilation terminated.
make: *** [/home/workspace/project/grpc/objs/opt/src/compiler/cpp_generator.o] Error 1
make: *** Waiting for unfinished jobs....

CXX      google/protobuf/compiler/main.o
  CXXLD    libprotobuf-lite.la
libtool: link: warning: library `/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/../../armv7a-vfp-neon-oe-linux-gnueabi/usr/lib/libstdc++.la' was moved.
  CXXLD    libprotobuf.la
libtool: link: warning: library `/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/../../armv7a-vfp-neon-oe-linux-gnueabi/usr/lib/libstdc++.la' was moved.
  CXXLD    libprotoc.la
libtool: link: warning: `/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/../../armv7a-vfp-neon-oe-linux-gnueabi/usr/lib/libstdc++.la' seems to be moved
libtool: link: warning: library `/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/../../armv7a-vfp-neon-oe-linux-gnueabi/usr/lib/libstdc++.la' was moved.
  CXXLD    protoc
libtool: link: warning: library `/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/../../armv7a-vfp-neon-oe-linux-gnueabi/usr/lib/libstdc++.la' was moved.
libtool: link: warning: library `/opt/EC20_crosstool/ql-ol-crosstool/sysroots/x86_64-oesdk-linux/usr/bin/arm-oe-linux-gnueabi/../../armv7a-vfp-neon-oe-linux-gnueabi/usr/lib/libstdc++.la' was moved.
./.libs/libprotoc.so: error: undefined reference to 'descriptor_table_google_2fprotobuf_2fdescriptor_2eproto'
./.libs/libprotoc.so: error: undefined reference to 'scc_info_FileDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto'
collect2: error: ld returned 1 exit status
make[2]: *** [protoc] Error 1
make[2]: Leaving directory `/home/workspace/project/grpc/third_party/protobuf/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/workspace/project/grpc/third_party/protobuf'
make: *** [all] Error 2

grpc_transport* grpc_create_cronet_transport(void* engine, const char* target,
                 ^
[AR]      Creating /home/workspace/project/grpc/libs/opt/libgrpc_cronet.a
[STRIP]   Stripping libaddress_sorting.a
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libaddress_sorting.a(address_sorting_posix.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libaddress_sorting.a(address_sorting_windows.o)'
[STRIP]   Stripping libgpr.a
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(atm.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(cpu_iphone.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(cpu_linux.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(cpu_posix.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(cpu_windows.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(env_linux.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(env_posix.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(env_windows.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(log.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(log_android.o)'
strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libgpr.a(log_linux.o)'

strip: Unable to recognise the format of the input file `/home/workspace/project/grpc/libs/opt/libupb.a(upb.o)'
[INSTALL] Installing C pkg-config files
[INSTALL] Installing libaddress_sorting.a
[INSTALL] Installing libgpr.a
[INSTALL] Installing libgrpc.a
[INSTALL] Installing libgrpc_cronet.a
[INSTALL] Installing libgrpc_unsecure.a
[INSTALL] Installing libupb.a
[LD]      Linking /home/workspace/project/grpc/libs/opt/libaddress_sorting.so.9.0.0
/usr/bin/ld: /home/workspace/project/grpc/objs/opt/third_party/address_sorting/address_sorting.o: Relocations in generic ELF (EM: 40)
/usr/bin/ld: /home/workspace/project/grpc/objs/opt/third_party/address_sorting/address_sorting.o: Relocations in generic ELF (EM: 40)
/usr/bin/ld: /home/workspace/project/grpc/objs/opt/third_party/address_sorting/address_sorting.o: Relocations in generic ELF (EM: 40)
/usr/bin/ld: /home/workspace/project/grpc/objs/opt/third_party/address_sorting/address_sorting.o: Relocations in generic ELF (EM: 40)
/usr/bin/ld: /home/workspace/project/grpc/objs/opt/third_party/address_sorting/address_sorting.o: Relocations in generic ELF (EM: 40)
/home/workspace/project/grpc/objs/opt/third_party/address_sorting/address_sorting.o: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
make: *** [/home/workspace/project/grpc/libs/opt/libaddress_sorting.so.9.0.0] Error 1

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found ZLIB: /usr/local/lib/libz.so (found version "1.2.11")
-- Using protobuf
CMake Error at CMakeLists.txt:113 (find_package):
  Could not find a configuration file for package "gRPC" that is compatible
  with requested version "".

  The following configuration files were considered but not accepted:

    /usr/local/lib/cmake/grpc/gRPCConfig.cmake, version: 1.28.0-pre3 (64bit)

[CXX]     Compiling src/core/tsi/ssl_transport_security.cc
gnu-configize: 'configure.ac' or 'configure.in' is required
autoreconf: gnu-configize failed with exit status: 1
make: *** [third_party/protobuf/configure] Error 1
make: *** Waiting for unfinished jobs....
src/core/tsi/fake_transport_security.cc:498:34: warning: unused parameter 'self' [-Wunused-parameter]
     const tsi_handshaker_result* self, tsi_peer* peer) {

make  all-recursive
make[1]: Entering directory `/home/workspace/project/grpc/third_party/protobuf'
Making all in .
make[2]: Entering directory `/home/workspace/project/grpc/third_party/protobuf'
make[2]: Leaving directory `/home/workspace/project/grpc/third_party/protobuf'
Making all in src
make[2]: Entering directory `/home/workspace/project/grpc/third_party/protobuf/src'
  CXXLD    protoc
./.libs/libprotoc.so: error: undefined reference to 'descriptor_table_google_2fprotobuf_2fdescriptor_2eproto'
./.libs/libprotoc.so: error: undefined reference to 'scc_info_FileDescriptorProto_google_2fprotobuf_2fdescriptor_2eproto'
collect2: error: ld returned 1 exit status
make[2]: *** [protoc] Error 1
make[2]: Leaving directory `/home/workspace/project/grpc/third_party/protobuf/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/workspace/project/grpc/third_party/protobuf'
make: *** [all] Error 2

-- Found Protobuf: /usr/local/bin/protoc-3.11.2.0 (found version "3.11.2.0")
-- Using protobuf
CMake Error at CMakeLists.txt:113 (find_package):
  Could not find a configuration file for package "gRPC" that is compatible
  with requested version "".

  The following configuration files were considered but not accepted:

    /usr/local/lib/cmake/grpc/gRPCConfig.cmake, version: 1.28.0-pre3 (64bit)



-- Configuring incomplete, errors occurred!

差点整崩溃，后面几次重新解压源码重新流程操作几次与参考网上博客，总算问题解决了。

四、参考资料

交叉编译gRPC_嵌入式_maimang1001的专栏-CSDN博客 https://blog.csdn.net/maimang1001/article/details/100561970

移植protobuf遇到的一点小问题_Johnny_nass_hu的专栏-CSDN博客 https://blog.csdn.net/Johnny_nass_hu/article/details/84138749

grpc arm 交叉编译(ubuntu 16.04)_嵌入式_学习————-CSDN博客 https://blog.csdn.net/qq_35487883/article/details/94398728

记一次grpc arm-hisiv400-linux交叉编译_嵌入式_vc66vcc的博客-CSDN博客 https://blog.csdn.net/vc66vcc/article/details/83614906

Arm下Grpc交叉编译过程记录_嵌入式_深圳趣奇多科技-CSDN博客 https://blog.csdn.net/poolooloo/article/details/97890673