adding perl bindings in embedded system

[peteasa]

I am using yocto to build libzmq3 and the perl bindings.

I get the following:
Probing libzmq via pkg-config ...

• found libzmq 4.1.4
• Detected ZMQ_INCLUDES from pkg-config...
• Detected ZMQ_LIBS from pkg-config...
  Detected the following ZMQ settings:
• ZMQ_HOME = (null)
• ZMQ_H = (null)
• ZMQ_INCLUDES =
• ZMQ_LIBS = -lzmq
• ZMQ_TRACE = (null)

Looks like ZMQ_INCLUDES is not found from the libzmq.pc package file.. but that file looks ok:
prefix=/usr
exec_prefix=/usr
libdir=/usr/lib
includedir=/usr/include

Name: libzmq
Description: 0MQ c++ library
Version: 4.1.4
Libs: -L${libdir} -lzmq
Cflags: -I${includedir}

Anyway I fixed this in yocto by exporting the export ZMQ_INCLUDES location and that works but I still get a failure:
| Probing environment variables:
| Alien::ZMQ not found
| Probing libzmq via pkg-config ...
| + found libzmq 4.1.4
| + Detected ZMQ_LIBS from pkg-config...
| Detected the following ZMQ settings:
| + ZMQ_HOME = (null)
| + ZMQ_H = (null)
| + ZMQ_INCLUDES = /usr/include
| + ZMQ_LIBS = -lzmq
| + ZMQ_TRACE = (null)
| *** Module name IN: /usr/lib/perl/5.22.1/IO.pm
| *** p1: /usr/lib/ p3: perl/5.22.1/ p5: IO.pm
| *** Module name OUT: /usr/lib/perl-native/perl/5.22.1/IO.pm
| # arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a9 --sysroot=/ -O2 -pipe -g -feliminate-unused-debug-types -DDEBIAN -D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector -lpthread -lnsl -ldl -lm -lcrypt -lutil -lc assertlibM2QiFoty.c -I/usr/include -o assertlibtlHxWLPE
| # arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a9 --sysroot=/ -O2 -pipe -g -feliminate-unused-debug-types -DDEBIAN -D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector -lpthread -lnsl -ldl -lm -lcrypt -lutil -lc assertlibipzhaFu0.c -o assertlibhIw7zwaM -I/usr/include -lzmq
| assertlibhIw7zwaM: assertlibhIw7zwaM: cannot execute binary file
| wrong result: 'zmq'

Anyway I tried to build on the target and I get other errors towards the end of the make process:
CHECK: Does your zmq have zmq_ctx_new... YES
Checking if your kit is complete...
Looks good
Warning: prerequisite Task::Weaken 0 not found.
Warning: prerequisite Test::Fatal 0 not found.
Warning: prerequisite Test::Requires 0 not found.
Warning: prerequisite Test::TCP 1.08 not found.
Warning: prerequisite ZMQ::Constants 0 not found.
ExtUtils::MM_Unix::tool_xsubpp : Can't find xsubpp at /usr/lib/perl/5.22.1/ExtUtils/MM_Unix.pm line 3578.

Any assistance or pointers to others who have used yocto and libzmq3 would be much appreciated. Looks like this is a bigger job than I expected!

Peter.

[lestrrat]

For starters, you stated I am using yocto to build libzmq3 but the probe is reporting libzmq 4.1.4. It has been a long time since I stopped following zeromq so I'm not sure if there's a libzmq3-4.1.4 or that just simply means libzmq4.1.4, but my first reaction is that what pkg-config is reporting seems to indicate that you are using libzmq4, not 3. That may have something to do with it.

Secondly, the probe failing to find anything sure looks weird. If you have some spare time, you might want to check the contents of tools/detect_zmq.pl, and see what each command invocation is doing.

[peteasa]

Thanks for your prompt response. I have done a bit more work on this. I built all I needed on the embedded system and it all worked when I build it on the target machine.. so that shows that all the libraries and modules etc are the right version.

Still the same problem with the cross compile build with yocto... I am new to building perl modules but as a general rule when cross compiling (arm-poky-linux-gnueabi-gcc is configured with --host=x86_64-linux --target=arm-poky-linux-gnueabi) the cross compiler does not produce code that would run on the x86_64 build machine. so the run failure is I guess should be expected!.. Now I looked at yocto recipes for other similar perl modules (http://cgit.openembedded.org/cgit.cgi/openembedded-core/tree/meta/recipes-devtools/perl/libxml-parser-perl_2.44.bb?h=master) for example that builds c code as well (https://metacpan.org/pod/XML::Parser contains links to the zip files) but this has no problems.. and builds ok.. so there must be some trick that I dont know about to get this to work.

I find it hard to understand the Makefile.PL file and cant see which bit of the file results in the build and run step. Is it possible to disable the run part and still generate the perl module?

The first part of the build has

*** Module name IN: /home/pas/Projects/meta-yocto-parallella/parallella-yoctobuild/build_parallella/tmp/sysroots/parallella-hdmi/usr/lib/perl/5.22.1/Cwd.pm *** p1: /home/pas/Projects/meta-yocto-parallella/parallella-yoctobuild/build_parallella/tmp/sysroots/parallella-hdmi/usr/lib/ p3: perl/5.22.1/ p5: Cwd.pm *** Module name OUT: /home/pas/Projects/meta-yocto-parallella/parallella-yoctobuild/build_parallella/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/Cwd.pm
The Module name OUT bit worries me because it is trying to output the native ie x86_64 code but I am cross compiling so this should produce arm code not x86_64 and should besaved to places like .. tmp/sysroots/parallella-hdmi/ ..

How can fix up the Makefle.PL file to do the cross compiling???

Thanks for any assistance you can provide!

[peteasa]

Ha! answered my own question.. just commented out # run_check &assertlibs; and it seems I get further with the build.. and it produces a .pm file etc

Will let you know how I progress!

[peteasa]

Ok so very close now I think.. yocto build seems to require libc.so.6 but I am quite sure that it should require an arm version of libc. I think that the Makefile.PL file contains some patch to override the library paths LD_LIBRARY_PATH.. this does not work with yocto when it is doing cross compiles it has some magic to ensure the correct compiler and libraries are found. I guess that I need to disable your patch in some way..

Also yocto requires a license file.. at the moment I am patching in GPLv3 but you might want a different license?.. not important for now just important if I make my layer public for others to use.

[lestrrat]

Let me make sure we're on the same wavelength: you said "cross"-compile? I must admit I'm not familiar with cross-compilation using the Perl build tools, but my first reaction would be "doesn't sound like a good idea".

So when you say cross-compile, what do you do exactly? Build for multiple platforms all at once? What steps/commands do you use?

---

(below is me trying to guess what you are trying to do: feel free to ignore if this does not match your case)
If you are saying the run_check bit is the problem, maybe you are only running perl Makefile.PL once? Normally perl module's rely on Makefile.PL doing all the platform specific magic, and assume that by the time make runs, all of these are magically resolved from the perspective of the module. Perhaps running perl Makefile.PL for every OS/Arch combination of cross-compilation fixes this problem?

[peteasa]

Hmm.. yocto is very complex and I dont want to have to get you involved with that side of things.

Sorry this reply has taken so long.. I have been deep in python zmq.. looks like the python implementation is easier to get going and also has python numpy so is closer to what I need in the long term.

One good point is that I have discovered a mass of perl modules at https://github.com/rehsack/meta-cpan. I searched that site and found most of the perl modules that I need. I have been rebuilding the embedded system every now and then and testing out the latest set of modules and libraries and finding / fixing version issues. Next time I re-install the embedded system I will have another go at the perl modules.

As to the exact steps.. if you look at for example: https://github.com/rehsack/meta-cpan/blob/master/recipes-net/test-tcp-perl/test-tcp-perl_2.15.bb you will see that the recipe to create the perl module is very simple. Most of the work is done in a thing called cpan.bbclass (http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/cpan.bbclass?h=krogoth) that is "inherit cpan" into the recipe and all perl module recipes so that does not change. In theory copy and paste any of any of the test-tcp-perl_2.15.bb perl module recipes, change the name of the file and the SRC_URI to pick up the correct source and that is all you need to do. Then in the yocto environment you run a thing called bitbake and out pops a whole linux distribution with your new perl modules included. Cross compile because my build machine is an x86 machine and the linux distribution runs on an arm processor (the embedded system).

In practice it is never as easy at that. My latest bitbake build still has warnings about libc.so.6 being required for ZMQ-LibZMQ3 as you can see below.. But the perl module is built ok.. I just have to prove it works.. I will let you know my progress in a day or so when I have added the final perl modules that I need and have had time to test the new build.

Peter.

`$ bitbake hdmi-image-debug
WARNING: Host distribution "Ubuntu-16.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |#########################################################################| ETA: 00:00:00
Loaded 2807 entries from dependency cache.
Parsing recipes: 100% |#######################################################################| Time: 00:00:00
Parsing of 1983 .bb files complete (1981 cached, 2 parsed). 2806 targets, 125 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION = "1.30.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "arm-poky-linux-gnueabi"
MACHINE = "parallella-hdmi"
DISTRO = "poky"
DISTRO_VERSION = "2.1.1"
TUNE_FEATURES = "arm armv7a vfp thumb neon callconvention-hard cortexa9"
TARGET_FPU = "hard"
meta
meta-poky
meta-yocto-bsp = "krogoth:7d70e6747973334cccd13a8dfc703cc973d52116"
meta-parallella = "oh-krogoth:ae6e4ddfd4f3a56eda86fab234c9ab463b9764eb"
meta-exotic = "oh-krogoth:314a1da2c2c8f16f87c7e55c04c054aa95eff4d0"
meta-epiphany = "oh-krogoth:eb3fe672b282ba7d3747fb21ef5ce754d6f88145"
meta-oe
meta-python = "krogoth:247b1267bbe95719cd4877d2d3cfbaf2a2f4865a"
meta-cpan = "master:c5f98ca31bf6f8d9b6e315ca3b92b879921feec8"
meta-test = "oh-krogoth-metaparallella:226e64ac7268e9876c942cd957869e9226977564"

NOTE: Preparing RunQueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: ZMQ-LibZMQ3-1.19-r0 do_package_qa: QA Issue: /usr/lib/perl/vendor_perl/5.22.1/auto/ZMQ/LibZMQ3/LibZMQ3.so contained in package ZMQ-LibZMQ3 requires libc.so.6(GLIBC_2.4), but no providers found in RDEPENDS_ZMQ-LibZMQ3? [file-rdeps]
NOTE: Tasks Summary: Attempted 4798 tasks of which 4785 didn't need to be rerun and all succeeded.
No currently running tasks (4796 of 4798)

Summary: There were 2 WARNING messages shown.
`