Qt development on A20

[srxa]

@Ikozic
Ok, I am on business trip right now in Czech Republic, so I will have some time this weekend to write some tutorial and explanations in my hotel room. I will try to explain everything I know so far.

[ikozic]

@Ikozic
Ok, I am on business trip right now in Czech Republic, so I will have some time this weekend to write some tutorial and explanations in my hotel room. I will try to explain everything I know so far.

Ok - as far as I'm concerned just some pointers should be enough. Either way, a huge thanks from me!

[srxa]

So, let's start. First, little bit of introduction, because this will be loooong post. For my project I need hardware which is capable of independent dual screens, fast graphics interface on medium to high resolutions, lots of GPIO pins and other I/O interfaces. Also, hardware need to be compact, without fans, hard disks etc. After little research on net, i found that Olinuxino A20 is perfect for my project. It has NAND storage, dual screen, 1GB of ram, nice processing power, lots of GPIO and other ports, and it is open source hardware and software. And Olimex company is relatively close to me(I am from Serbia), and I do not have to deal with some Chinese manufacturer, and I am pretty sure that after I finish my product, I will be able to order whatever I need from them, and I hope that they will reenable PCB production in few months, so when A20 SOM hits the market, I will be able to order A20 SOM's with my custom board design, everything produced by Olimex  . Until this board all my products has XP embedded systems, so I am new in linux. Until I receive A20 micro I know only  cd and ls and man from linux, and nano also  . After few days I was compiling kernel, then i made Qt work with my platform, and on the end I solved eglfs plugin from Qt for both displays, so my hardware is ready for development now. I must say that many of your need's will not work out of the box with any similar product, but I found that if you know what you need, you will find many communities online which will help you with your problems, and once you get in linux,  you will find that it is pretty intuitive to work with. Just google for everything you need  . So, because I am beginner with linux, do not mind me if in this explanations and tutorial you find that something can be done easier or on better way, please be kind to post your solutions here.

Now few words about hardware on A20. As far as I know (and this phrase is valid for everything that i will write here, so i will not repeat it  ), A20 has mali400MP2 GPU for 2D and 3D accelerations, it is dual core, it works with shared memory and you must reserve some for MALI. MALI did not drive displays, MALI only render everything in memory, and other unit display contents then. On A20 there is also separate 2D acceleration unit called G2D, and video acceleration unit, called CedarX. MALI400 is OpenGL ES 2.0 capable, or compatible, so I decided to use OpenGL ES in my project. Everything you are interested about OpenGL ES you can find at www.khronos.org .

And few words about Qt. My previous projects was based on custom engine made with C backend and javascript. When I search for something that I can use with this platform, and has good support, I found about Qt. It has C++ for my backend where I can drive my hardware, it has QML with embedded javascript engine, where I can work with my GUI, and it is absolutely free and opensource! So, it seems that it is perfect for my needs. Qt has many plugins for different types of "driving" applications on platform, like XCB for x server, linuxfb which uses linux frame buffer, minimalegl, DirectFB which I did not explore enough, and eglfs which I choose. Why eglfs? Eglfs stands for EGL full screen. EGL, as wikipedia says, is "EGL is an interface between Khronos rendering APIs (such as OpenGL, OpenGL ES or OpenVG) and the underlying native platform windowing system. EGL handles graphics context management, surface/buffer binding, rendering synchronization, and enables "high-performance, accelerated, mixed-mode 2D and 3D rendering using other Khronos APIs."
So, because I need for my application to be always full screen, and only running application in system, and i don't need any windowing system, i decided for eglfs. Qt is also cross platform, so if in future I need to port my software solution on some other hardware, it should be easy to do.

Some words about preparing your systems. You can compile Qt on your target device, but it is very long process, and i think that it is easier for development when you have cross compiled Qt, because then you have some remote deployment tools and similar stuff which will make your life easier. For host machine, I am using Ubuntu 13.04 x64 in Parallels on my Mac pro, and on target, because i found on net that MALI drivers version r3p2-01rel2 are much faster and better for my project, i use ssvb's kernel with support for those drivers, and you can find it here: https://github.com/ssvb/linux-sunxi/tree/20130913-mali-r3p2-01rel2 . I will not write about compiling kernel, because everything about that is already on Olimex and linux-sunxi wiki.

So, in order to prepare your systems, you must prepare your HOST and your TARGET machine. Host machine is one where you will cross compile Qt, install Qt creator and use it for development, and target is your device where you will deploy your software.

Preparing  Target:

I assume that you already have ssvb's version of kernel. Next step is to disable login manager. Because I am using Debian rootfs from Olimex latest debian image for A20, it has slim manager. To disable it, go to /etc/X11/default-display-manager and replace only line with /usr/bin/true . Next you need to install or recompile MALI binaries, because I found that if you will use frame buffer device, you must compile MALI with that parameters. So, get the latest mali drivers, go to the directory, and use this:

Code Select Expand

git submodule init
git submodule update
make config ABI=armhf VERSION=r3p2-01rel2 EGL_TYPE=framebuffer
make install
cd test
make

Next, you must load modules, so:

Code Select Expand

modprobe mali
modprobe drm
modprobe mali_drm

Now you can test everything, by running

Code Select Expand

./test

and, if everything is ok, you will see triangle on the display, and if you are running ./test via ssh, you will get next information in console:

EGL Version: "1.4 Linux-r3p2-01rel2"
EGL Vendor: "ARM"
EGL Extensions: "EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_reusable_sync EGL_KHR_fence_sync EGL_KHR_lock_surface EGL_KHR_lock_surface2 EGL_EXT_create_context_robustness "
Surface size: 480x480
GL Vendor: "ARM"
GL Renderer: "Mali-400 MP"
GL Version: "OpenGL ES 2.0"
GL Extensions: "GL_OES_texture_npot GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_depth24 GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth_texture GL_OES_packed_depth_stencil GL_EXT_texture_format_BGRA8888 GL_EXT_blend_minmax GL_OES_EGL_image_external GL_OES_EGL_sync GL_OES_rgb8_rgba8 GL_EXT_multisampled_render_to_texture GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_ARM_mali_program_binary GL_EXT_shader_texture_lod GL_EXT_robustness GL_OES_depth_texture_cube_map"

Now you are ready to install dependencies for Qt. Different Qt modules requires different packages. I will list only few, but if you need some other dependencies for some module, you will need to install them. First, run:

Code Select Expand

sudo apt-get update
sudo apt-get upgrade

Then, install packages. This will be from Ubuntu, but surely there is similar package names for any distro.

Basic Qt dependencies:

• libfontconfig1-dev
• ibdbus-1-dev
• libfreetype6-dev
• libudev-dev

Dependencies for multimedia:

• libasound2-dev
• libavcodec-dev
• libavformat-dev
• libswscale-dev
• libgstreamer0.10-dev
• libgstreamer-plugins-base0.10-dev
• gstreamer-tools
• gstreamer0.10-plugins-good
• gstreamer0.10-plugins-bad

QtWebKit dependencies:

• libicu-dev
• libsqlite3-dev
• libxslt1-dev
• libssl-dev

For QtWebKit you must install flex, bison, gperf, and ruby also.

Install gdbserver because you will need it for debugging purpose.

Code Select Expand

sudo apt-get install gdbserver

Now your target device is ready, let's move to the host.

Preparing Host:

If you are using 64bit linux as I am, you will need to install some libraries to support it, because toolchain is created for 32bit systems. So, on Ubuntu, you will use:

Code Select Expand

sudo apt-get install ia32-libs

Somewhere I use apt-get, but i recommend you to use aptitude instead, because i found that it is pretty good package manager.
Next, you will need toolchain for cross compiling. I recommend you that you use latest linaro toolchain, because everything below version 4.7 is not good for Qt, because Qt has support for C++11, and if you will use C++11 you need to use gcc which is 4.7 or above. Also, newer version of compiler produce faster code, believe it or not . You can use toolchain from here: http://www.linaro.org/downloads/

Then, you need to mount rootfs from your device to host machine. Rootfs must be mounted whenever you use Qt. You can do it in two different ways. First, you can dd your SDcard to image on your host and mount it, but you will need to put it back when you are finished with Qt cross compiling. You can mount your image using command similar to this:

Code Select Expand

mount -o loop,offset=17825792 /home/parallels/a20qt.img /mnt/a20/

To calculate offset, you can use something like:

Code Select Expand

fdisk -lu /home/parallels/a20qt.img

Then you can use start sector from second partition, which is your rootfs partition, and multiply it by sector size. In my case, start sector is at 34816 and there is 512 bytes per sector, so 34816 * 512 =  17825792

Better way is to mount your rootfs via sshfs. Sshfs is fast and reliable, and you don't need to dd your SDcard whenever you change something. Install sshfs on your device, and run something like this:

Code Select Expand

sudo sshfs -o allow_other root@192.168.0.100:/ /mnt/a20/


Next it is time to download some tools and Qt sources. First download some cross compiling tools.

Code Select Expand

git clone git://gitorious.org/cross-compile-tools/cross-compile-tools.git
cd cross-compile-tools

Then apply the fixQualifiedLibraryPaths script. This fixes the symlinks in the mounted image to be relative instead of absolute. The command is: fixQualifiedLibraryPaths target-rootfs path-to-target-toolchain-compiler
In my case it is:

Code Select Expand

./fixQualifiedLibraryPaths /mnt/a20/ /home/parallels/toolchain/gcc-linaro-arm-linux-gnueabihf-4.8-2013.09_linux/bin/arm-linux-gnueabihf-gcc

Download Qt source files from http://qt-project.org/downloads . You can download stable 5.2.1 or 5.3.0 beta versions for now (and all older versions, of course).
Unpack files to some folder. In my case it is /home/parallels/Qt5.2.1/qt-everywhere-opensource-src-5.2.1.
Go to the /qtbase/mkspecs/devices and create new directory called linux-a20olimex-g++
Copy qmake.conf and qplatformdefs.h from /qtbase/mkspecs/devices/linux-beagleboard-g++/ to /qtbase/mkspecs/devices/linux-a20olimex-g++/
Open qmake.conf , delete everything, and replace with:

Code Select Expand

#
# qmake configuration for the A20_OlinuxinO boards
# http://www.olimex.com/

MAKEFILE_GENERATOR      = UNIX
CONFIG                 += incremental gdb_dwarf_index
QMAKE_INCREMENTAL_STYLE = sublib

include(../../common/linux.conf)
include(../../common/gcc-base-unix.conf)
include(../../common/g++-unix.conf)

load(device_config)

QT_QPA_DEFAULT_PLATFORM = eglfs
#EGLFS_PLATFORM_HOOKS_SOURCES = $$PWD/qeglfshooks_a20.cpp
# modifications to g++.conf
QMAKE_CC                = $${CROSS_COMPILE}gcc
QMAKE_CXX               = $${CROSS_COMPILE}g++
QMAKE_LINK              = $${QMAKE_CXX}
QMAKE_LINK_SHLIB        = $${QMAKE_CXX}

# modifications to linux.conf
QMAKE_AR                = $${CROSS_COMPILE}ar cqs
QMAKE_OBJCOPY           = $${CROSS_COMPILE}objcopy
QMAKE_NM                = $${CROSS_COMPILE}nm -P
QMAKE_STRIP             = $${CROSS_COMPILE}strip

COMPILER_FLAGS          = -march=armv7-a -mtune=cortex-a7 -mfpu=neon -DLINUX=1 -DEGL_API_FB=1 -mfloat-abi=hard

#modifications to gcc-base.conf
QMAKE_CFLAGS           += $${COMPILER_FLAGS}
QMAKE_CXXFLAGS         += $${COMPILER_FLAGS}
QMAKE_CXXFLAGS_RELEASE += -O3

QMAKE_LIBS             += -lrt -lpthread -ldl

# Extra stuff (OpenGL, DirectFB, ...)
QMAKE_INCDIR_EGL        = /mnt/a20/usr/include/EGL
QMAKE_LIBDIR_EGL        = /mnt/a20/usr/lib
QMAKE_INCDIR_OPENGL_ES2 = /mnt/a20/usr/include/GLES2
QMAKE_LIBDIR_OPENGL_ES2 = /mnt/a20/usr/lib
#QMAKE_INCDIR_OPENVG     = $${QMAKE_INCDIR_EGL}
#QMAKE_LIBDIR_OPENVG     = $${QMAKE_LIBDIR_EGL}

QMAKE_LIBS_EGL          = -lEGL
QMAKE_LIBS_OPENGL_ES2   = -lGLESv2 $${QMAKE_LIBS_EGL}
#QMAKE_LIBS_OPENVG       = -lOpenVG $${QMAKE_LIBS_EGL}

# Sanity check
deviceSanityCheckCompiler()

load(qt_config) 

Then you need to modify paths for your system, because i did not use variables but fixed path for OpenGL ES driver. Save file and close.

Then we need to patch mali native window which we will use. I did not make qeglfshooks_a20.cpp, instead I directly edit qeglfshooks_stub.cpp. So, go to /qtbase/src/plugins/platforms/eglfs/ , open qeglfshooks_stub.cpp , and change native window function with this code:

Code Select Expand

EGLNativeWindowType QEglFSHooks::createNativeWindow(QPlatformWindow *platformWindow,const QSize &size,const QSurfaceFormat &format)
{
    Q_UNUSED(platformWindow);
    Q_UNUSED(size);
    Q_UNUSED(format);
    //return 0;
    static struct mali_native_window native_window = {
        .width = (short unsigned int)size.width(),
        .height = (short unsigned int)size.height(),
    };
    return &native_window;
}

Finally, we are at configuring Qt before compiling. Important thing is to define sysroot and prefix parameter, because it will make your life easier . Sysroot is rootfs of your target device, and prefix is folder where Qt will be installed. My configure goes like this:

Code Select Expand

./configure -opengl es2 -device linux-a20olimex-g++ -device-option CROSS_COMPILE=/home/parallels/toolchain/gcc-linaro-arm-linux-gnueabihf-4.8-2013.09_linux/bin/arm-linux-gnueabihf- -sysroot /mnt/a20 -opensource -confirm-license -optimized-qmake -release -make libs -prefix /usr/local/qt5olim2 -no-pch -nomake examples -nomake tests -no-xcb -eglfs -v

If everything is ok, check your config.summary.

Code Select Expand

cat qtbase/config.summary

Check for options you need. If something is not all right, change your configuration. You will get something like this:

   Configure summary

Building on:   linux-g++ (x86_64, CPU features: mmx sse sse2)
Building for:  devices/linux-a20olimex-g++ (arm, CPU features: neon)
Platform notes:

            - Also available for Linux: linux-kcc linux-icc linux-cxx
       
qmake vars .......... styles += mac fusion windows DEFINES += QT_NO_MTDEV QT_CFLAGS_DBUS = -I/mnt/a20/usr/include/dbus-1.0 -I/mnt/a20/usr/lib/arm-linux-gnueabihf/dbus-1.0/include   QT_LIBS_DBUS = -L/mnt/a20/usr/lib/arm-linux-gnueabihf -ldbus-1   QT_CFLAGS_GLIB = -pthread -I/mnt/a20/usr/include/glib-2.0 -I/mnt/a20/usr/lib/arm-linux-gnueabihf/glib-2.0/include   QT_LIBS_GLIB = -pthread -L/mnt/a20/usr/lib/arm-linux-gnueabihf -lgthread-2.0 -lrt -lglib-2.0   QMAKE_CFLAGS_FONTCONFIG = -I/mnt/a20/usr/include/freetype2   QMAKE_LIBS_FONTCONFIG = -L/mnt/a20/usr/lib/arm-linux-gnueabihf -lfontconfig -lfreetype   QMAKE_INCDIR_LIBUDEV =   QMAKE_LIBS_LIBUDEV = -ludev   DEFINES += QT_NO_XCB DEFINES += QT_NO_XKBCOMMON sql-drivers =  sql-plugins =  sqlite qmake switches .........

Build options:
  Configuration .......... accessibility alsa audio-backend c++11 clock-gettime clock-monotonic compile_examples concurrent cross_compile dbus egl eglfs evdev eventfd fontconfig full-config getaddrinfo getifaddrs glib iconv icu inotify ipv6ifname large-config largefile libudev linuxfb medium-config minimal-config mremap nis no-harfbuzz opengl opengles2 openssl pcre png posix_fallocate qpa qpa reduce_exports reduce_relocations release rpath shared small-config system-freetype system-png system-zlib xinput2 xlib xrender
  Build parts ............  libs
  Mode ................... release
  Using C++11 ............ yes
  Using PCH .............. no
  Target compiler supports:
    iWMMXt/Neon .......... no/auto

Qt modules and options:
  Qt D-Bus ............... yes (loading dbus-1 at runtime)
  Qt Concurrent .......... yes
  Qt GUI ................. yes
  Qt Widgets ............. yes
  Large File ............. yes
  JavaScriptCore JIT ..... yes (To be decided by JavaScriptCore)
  QML debugging .......... yes
  Use system proxies ..... no

Support enabled for:
  Accessibility .......... yes
  ALSA ................... yes
  CUPS ................... no
  FontConfig ............. yes
  FreeType ............... yes (system library)
  Glib ................... yes
  GTK theme .............. no
  HarfBuzz ............... no
  Iconv .................. yes
  ICU .................... yes
  Image formats:
    GIF .................. yes (plugin, using bundled copy)
    JPEG ................. yes (plugin, using bundled copy)
    PNG .................. yes (in QtGui, using system library)
  journald ............... no
  mtdev .................. no
  Networking:
    getaddrinfo .......... yes
    getifaddrs ........... yes
    IPv6 ifname .......... yes
    OpenSSL .............. yes (loading libraries at run-time)
  NIS .................... yes
  OpenGL / OpenVG:
    EGL .................. yes
    OpenGL ............... yes (OpenGL ES 2.x)
    OpenVG ............... no
  PCRE ................... yes (bundled copy)
  pkg-config ............. yes
  PulseAudio ............. no
  QPA backends:
    DirectFB ............. no
    EGLFS ................ yes
    KMS .................. no
    LinuxFB .............. yes
    XCB .................. no
  Session management ..... yes
  SQL drivers:
    DB2 .................. no
    InterBase ............ no
    MySQL ................ no
    OCI .................. no
    ODBC ................. no
    PostgreSQL ........... no
    SQLite 2 ............. no
    SQLite ............... yes (plugin, using bundled copy)
    TDS .................. no
  udev ................... yes
  xkbcommon .............. no
  zlib ................... yes (system library)

NOTE: Qt is using double for qreal on this system. This is binary incompatible against Qt 5.1.
Configure with '-qreal float' to create a build that is binary compatible with 5.1.

AND FINALLY:

Code Select Expand

sudo make -j5 && sudo make install

Configuring Qt Creator and running some tests, i will describe in my next post, I am writing this whole afternoon, at least 8 hours, so it is time to get some coffee  .

[ikozic]

So, let's start. First, little bit of introduction, because this will be loooong post.

Long and detailed. I bet that many people will praise you now, as this probably deserves a sticky. Hear that admins? Also very structured. Posts like this one actually define a good open-source community.

(I am from Serbia),

You don't say... Probably many of us here are, but we somehow do not even try to figure that out - dunno why though Certainly a nice surprise for me

If I may interrupt with my problems, as it seems that we are more-less in the same state, although I'm without 3D acceleration.
1. You've mentioned this virtual fb. So currently I have modified Qt linuxfb driver to work with layers (video under, GUI over, alpha blended) and this works quite well - it's still a bit buggy, but I will fix it in a week or so. Anyway, this mod includes opening /dev/disp and using Allwinner ioctls to properly work with layers - it's not too complex, but since the driver is quite buggy and there's no docu, I needed quite a while until it worked as expected. If I understood correctly, you said that you did something similar only by increasing framebuffer_num to 4 and using virtual fb. Could you tell me how, provided that I've understood it right of course?
2. Mali is basically mem-to-mem unit, so no output to the screen, I'm familiar with that. In order to actually output something, it would have to go to DEFE->DEBE or directly to DEBE (depending whether you need scaler layer or not). So what I'm not really understanding here is how are DEFE/DEBE initialized in such a case - what initializes them? Because it seems to me that Mali gets some sort of special access to the screen when you modprobe the driver, so it looks as if Mali driver itself is accessing /dev/disp...
3. This eglfs option is quite interesting, but I guess it's not for me, as most of my GUI layer is transparent, because video is underneath. Although what is fullscreen when there is alpha blending? - it can render the whole screen, no problem, but if 90% of the pixels have 0x00 for alpha, then only some objects would be visible, which could in fact be good for my application. Do you know if Mali driver supports working with RGB8888 - 32bit color?
4. Is there any benefit to using -opengl es2 when not using eglfs? Maybe smoother GUI or something similar?

I have a small addition, if you're not using it already - I find it much easier to use TFTP/NFS boot for development phase. SD card gets garbled easily when power-cycling (often when debugging kernel) and it's quite slow to always update it. If you don't need extreme speeds when accessing rootfs, I'd opt for TFTP/NFS - both kernel and rootfs are on your host and when I compile new kernel, I just restart the system and it boots from network. The same for rootfs and Qt - I cross compile everything on host and it's already on target, since rootfs is actually on my host... The only thing on SD is uboot that manages this network boot.
Also quite easy to set up, so if anyone needs any pointers, I'll be happy to help.

[srxa]

@ikozic

You are going too deep to hardware layers, and everything is handled for you if you are using OpenGL ES2... EGL will handle buffers and everything, you don't need to deal with it. Of course that you can use GUI over video with OpenGL ES2, and you will get hardware accelerated GUI with many OpenGL functions , shaders, lights, etc. And in Qt, I think in versions above 5.0, i am not sure, many things are already accelerated and using OpenGL, so if you don't have proper drivers installed, you will not be able to use many of that functions in Qt. Eglfs is only platform for running OpenGL ES2, you can use XCB for x server, you can use QtWayland plugin which is also window manager(but it uses eglfs as a base), and you will get OpenGL ES2 acceleration. But if you are using LinuxFB, there is no acceleration at all...

Here is some example what you can do with OpenGL ES2 with your video in Qt using QML... http://youtu.be/P4kv-AoAJ-Q or this http://youtu.be/Ew114XH7-xc

And for TFTP/NFS .. I did not have ANY problems yet with my SDcard image  ... but i will certainly use it when it comes to fine tuning of kernel, one day when i finish developing my software. Final product must run from NAND, it must be read only with ramfs, and i will have battery backup for safe shutdown, and probably STM32F4 on my daughter board for some critical data, protection of product, i also need true random generator which is embedded to STM32F4 etc...

[ikozic]

@ikozic

You are going too deep to hardware layers, and everything is handled for you if you are using OpenGL ES2... EGL will handle buffers and everything, you don't need to deal with it. Of course that you can use GUI over video with OpenGL ES2, and you will get hardware accelerated GUI with many OpenGL functions , shaders, lights, etc. And in Qt, I think in versions above 5.0, i am not sure, many things are already accelerated and using OpenGL, so if you don't have proper drivers installed, you will not be able to use many of that functions in Qt. Eglfs is only platform for running OpenGL ES2, you can use XCB for x server, you can use QtWayland plugin which is also window manager(but it uses eglfs as a base), and you will get OpenGL ES2 acceleration. But if you are using LinuxFB, there is no acceleration at all...

Here is some example what you can do with OpenGL ES2 with your video in Qt using QML... http://youtu.be/P4kv-AoAJ-Q or this http://youtu.be/Ew114XH7-xc

Well, I am a HW engineer
True, but correct me if I'm wrong - Mali will then use its own layering mechanism instead of DEBE/DEFE, since I hardly doubt it uses Allwinner's display driver (it's really that terrible) - it probably just bypasses it by using one single layer or similar. So, the usual V4L2 flow is to have its buffers exchanged with display framebuffer (in reality this is all done with DMAs in HW). If we use EGLFS, there should be some mechanism to interface it with V4L2 - most of the references from Google go to some sort of GLUT library which is used to handle it. This is however a very different approach to the whole project - instead of using dedicated HW for all the image operations, Mali is used for everything.

It's maybe a better approach, maybe not, either way not really needed for my app. Effects would be nice of course, and I would try it out if I had more time which unfortunately I don't. However, I did mark it as low priority in my roadmap, so I may try it out if I get some time for it. The fact that you're sticking to one platform (GPU+OpenGL) makes it a better candidate on the long run, since DEBE/DEFE and other display drivers will change from generation to generation.

And for TFTP/NFS .. I did not have ANY problems yet with my SDcard image  ... but i will certainly use it when it comes to fine tuning of kernel, one day when i finish developing my software. Final product must run from NAND, it must be read only with ramfs, and i will have battery backup for safe shutdown, and probably STM32F4 on my daughter board for some critical data, protection of product, i also need true random generator which is embedded to STM32F4 etc...

For kernel it's quite useful, but also for rootfs and testing apps - with i.MX6, I had most of the "util" apps source files in my rootfs, edited them on host and build them on host - it was just the matter of typing ./my_app in target's UART console to start it. The same with libs. Basically maybe one step off Qt in automation for normal C files. I also had a lot of dump files which I needed to inspect, so since they were on host it was easy.
But yeah, for kernel development it's a must. Otherwise you'll go crazy with all the copying - developing kernel usually includes a lot of seg faults which effectively block the system. In such situation, when you restart without reboot or poweroff it's quite likely that your SD will corrupt...
How far did you get with NAND? - this week I've reserved for that - you'll also need firmware update with NAND I guess...

[ikozic]

Ahh sorry, just saw how it's done - you're configuring Mali driver for framebuffer (EGL_TYPE=framebuffer), so technically, it goes directly to /dev/fb. /dev/fb has its own layer and it's easy to get a handle for it.

Ok, there actually could be a sort of hybrid solution - V4L2 could run underneath (as I have it like a c library), while Qt could use Mali for regular framebuffer - I could just get a handle on the framebuffer and pull its layer to top (I also have a disp library which runs underneath). Simples.

Still no time for it now. I'll post back as soon as I try this thing out.

[ikozic]

@srxa:

I think I've found a bug with Qt 5.2.1. Let me explain:
- I've compiled Mali drivers in a similar way you did, but with regular kernel (linux-sunxi).
- ./test works as it should.
- when compiling Qt 5.2.1, I've used your qmake.conf, but it had issues with configure script (could not find includes), so I changed your lines:

QMAKE_INCDIR_EGL        = /mnt/a20/usr/include/EGL
QMAKE_INCDIR_OPENGL_ES2 = /mnt/a20/usr/include/GLES2

to:

QMAKE_INCDIR_EGL        = /mnt/a20/usr/include
QMAKE_INCDIR_OPENGL_ES2 = /mnt/a20/usr/include

as Qt EGL files are like:

#include <GLES2/gl2.h>
...

However, as Qt config script is rather buggy, it seems that Qt is pulling all the other libs from this place - stdlib.h for one (as if I've set QMAKE_INCDIR instead of QMAKE_INCDIR_EGL). This introduces A LOT of other issues, so it might be the best to either manually edit these files and clean them from these stupid includes or just copy them to another place and use them from there.

Either way, I dunno how you got past this - it's either different with newer Mali drivers (I'm using r3p0, as they come with the original 3.4 kernel) or you've fixed this somehow but forgot to write it out.

[ikozic]

@srxa:

One more minor correction regarding your hooks implementation:

static struct mali_native_window native_window = {
        .width = (short unsigned int)size.width(),
        .height = (short unsigned int)size.height(),
    };

This cannot be written like this, as it's C++ and we're compiling with g++, if it was gcc it could have been as this syntax is valid in C only:

http://publib.boulder.ibm.com/infocenter/lnxpcomp/v8v101/index.jsp?topic=%2Fcom.ibm.xlcpp8l.doc%2Flanguage%2Fref%2Fstrin.htm

Just change to:

static struct mali_native_window native_window = {
        (short unsigned int)size.width(),
        (short unsigned int)size.height(),
    };

And it'll work as it should.

[srxa]

Yes, I am aware that this is not correct c++ syntax, but it will compile and work  . Otherwise, you will get 0x300 error when you try to start your application, and that error means bad native window. And for MALI drivers, i will check, but i know that there is some difference with ssvb's kernel, because when i was solving the dual screen eglfs option, i found that driver is split in more smaller libraries,  and in your version i think that everything is in libMALI.so . And more about this and some bugs in Qt, and how to work around them, (with distance field and text, for example) I will write in few days, because it will be also time consuming post again  .

[ikozic]

Yes, I am aware that this is not correct c++ syntax, but it will compile and work  .

Hmm... for me it doesn't for some reason - this is why I had to change it in the first place, as it won't compile. Maybe my GCC is too old (I'm still using 4.6).

Otherwise, you will get 0x300 error when you try to start your application, and that error means bad native window.

True This is why I had to make your code compilable on my GCC, hence my post

And for MALI drivers, i will check, but i know that there is some difference with ssvb's kernel, because when i was solving the dual screen eglfs option, i found that driver is split in more smaller libraries,  and in your version i think that everything is in libMALI.so . And more about this and some bugs in Qt, and how to work around them, (with distance field and text, for example) I will write in few days, because it will be also time consuming post again  .

Ok, I need some help here, because currently I'm stuck. I've compiled a new QML project and now thanks to your post I could finally use Quick 2.0 (since it requires OpenGL). However, I don't know how to make Mali render in 32 bits, that is, with transparency. A bit difficult to describe, but I'll try:

I've noticed that, at least in SW, it seems that Mali is using /dev/fb0 for render. I've found this out by playing around with layers and saw that when I move the framebuffer layer to the top, my QML app is displayed. As I said before, video is running under with Alpha value set to 0xff in DEBE.

I've also set a simple rectangle that covers the whole screen in QML and I've set the color to white (0xFFFFFFFF). Once I enable the transparency this is what happens:
1. 0x80FFFFFF (or opacity 0.5, so 50% transparency) - the rectangle gets gray,
2. 0x00FFFFFF (opacity 0, full transparency) - the whole screen is black.
If I move the fb layer with my display support lib down, I can see my video only - this is how I saw that Mali is using default layer.
If on the other hand, I use DEBE's Alpha blender to set the global alpha of the Qt layer to 0.5, I can finally see the video underneath, but it's very black or grey depending on what I set as transparency in Qt layer. However, global alpha is not really usable in Qt, so this is only for debug - I need "pixel-wise" transparency to correctly construct GUI over video.

Ideally, Mali should render in 32 bits, so that when I set transparency in QML to full, that I can see the video underneath without any color distortions. This is what I've tried yesterday, but I have no idea what is wrong - I think I've almost ruled out QML and playing around with Quickview and I'm more and more certain that this is an issue with ELGFS driver in Qt. It could also be something in the kernel driver for Mali, but I think I would be wasting time playing around with the kernel driver since most of the source is closed anyway. I've also disabled FBCON in kernel, thinking it might be this (FBCON uses this default display layer of /dev/fb0 to display console), but other than losing cursor blinking and console, nothing changed. I had a similar problem with SW render in Quick 1.0, so it might be that Qt is painting this black color in the background for some reason, but I couldn't really find out where.

If you have any info on this, it'd be great if you share - I've already posted a similar post to sunxi google group, hoping if someone could help. If I get to the solution first, I'll post back as always.

[srxa]

I think you should use gstreamer and QtMultimedia for your project, and everything will be OK. Gstreamer libs allow you to stream almost every input i think. About other issues in your post, i will write in few days, time is problem for me right now.

[ikozic]

@srxa:

Got it, working great now. Mali is in fact using default framebuffer layer for this (as if it's writing to /dev/fb0). What happens in fact is that it writes to the framebuffer allocated memory (there's references to framebuffer memory in Mali driver (called SUNXI_RESERVED_FB_MEM or something similar)).

Anyway, I've got exactly what I need - video running under, OpenGL accelerated Qt/QML running over it with transparency as needed and directed by Qt/QML.

What needs to be done? Video is pretty straightforward and I already explained it, so for Qt/Mali, in rough steps:
1. Get the framebuffer layer handle by calling FBIOGET_LAYER_HDL_0 (for fb0). This is all we need from kernel to do the changes.
2. Using this handle, call DISP_LAYER_GET_PARA ioctl to get the layer parameters.
3. Modify the parameters for Mali/Qt. This is very important - you need to set the mode to NORMAL, not SCALER (especially if already using scaler for video), alpha_en to 0 and alpha_val to 0. Do not touch the layer_para.fb.addr(x). Set the pipe to the opposite of the video layer (if video layer is 0, set to 1 and the other way around). Scaler for fb layer could also be accidentally set in the fex file, so take care of this.
4. Call DISP_LAYER_SET_PARA ioctl to set the new parameters.
5. Naturally if video is under, you should move its layer, using video layer handle and DISP_LAYER_BOTTOM ioctl, to the bottom, while Qt layer (framebuffer) should be moved to the top (DISP_LAYER_TOP ioctl). If there are some other crazy layers active, move first the video layer to the top, then Qt layer to the top - this will ensure that you have Qt layer on the top, while video just under it.

That's it - works perfectly.

[Polto]

Hi
I make an image (linux shell + qt 4.8 ) work on framebuffer
I tested it on marsboard a20.
If any one could test it on olimex a20 and a10 board and tell me if it works or not.
http://www.remnum.com/myimage+qt.img

[ikozic]

Code Select Expand

modprobe mali
modprobe drm
modprobe mali_drm

Quite strange, but my Mali works without DRM - do you know if there is maybe a downside to this? Basically, I just do:

Code Select Expand


modprobe mali


And it works...