i.MX6Q (ARM Cortex-A9 Quad Core) Embedded Linux-Xenomai RTOS Porting

간만에 삽질좀 했다.
ARM Cortex-A9 쿼드코어 기반 AP인 i.MX6Q 프로세서에 Linux-Xenomai RTOS를 포팅해 봤다.
ARMv4/5기반의 오래된 ARM에만 포팅 해보다 나름 최근(?)의 성능 좋은 프로세서에 포팅작업을 하려다 보니 여러가지 삽질들이 좀 많았다.
우선 리눅스 커널 버전과 Xenomai 버전이 올라가면서 바닐라 커널의 Xenomai패치는 한결 수월해 졌으나, 커널에 Device-Tree라는 요상한게 생기고 Cortex-A9의 하드웨어 FPU인 VFPv3 및 NEON을 제대로 지원하는 툴체인을 빌드하느라 시간을 좀 많이 까묵었다 (툴체인 한번 빌드하는데 3~4시간, 커널 빌드하는데 30~40분).
마침내 커널 부팅되고 램디스크 마운팅 되고 Xenomai 어플리케이션 돌아가고...
드디어 1GHz 쿼드코어에서 돌아가는 RTOS가 만들어 졌다. 움하하...

<> 커널 부팅
------------------------------------------------------------------
Booting Linux on physical CPU 0x0
Linux version 3.10.32-xenokang (root@localhost.localdomain) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0) ) #1 SMP Wed Jun 15 10:43:26 KST 2016
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: Freescale i.MX6 Quad/DualLite (Device Tree), model: Freescale i.MX6 Quad SABRE Smart Device Board
Truncating RAM at 10000000-8fffffff to -7f7fffff (vmalloc region overlap).
Memory policy: ECC disabled, Data cache writealloc
PERCPU: Embedded 10 pages/cpu @815fc000 s18496 r8192 d14272 u40960
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 453136
Kernel command line: console=ttymxc0,115200 root=/dev/ram0 rw --no-log initrd=0x1E000000,16M ramdisk=32768
PID hash table entries: 4096 (order: 2, 16384 bytes)
Dentry cache hash table entries: 262144 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 131072 (order: 7, 524288 bytes)
Memory: 1784MB = 1784MB total
Memory: 1786168k/1786168k available, 40648k reserved, 0K highmem
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    vmalloc : 0xf0000000 - 0xff000000   ( 240 MB)
    lowmem  : 0x80000000 - 0xef800000   (1784 MB)
    modules : 0x7f000000 - 0x80000000   (  16 MB)
      .text : 0x80008000 - 0x80693ff4   (6704 kB)
      .init : 0x80694000 - 0x806ea840   ( 347 kB)
      .data : 0x806ec000 - 0x80717fa0   ( 176 kB)
       .bss : 0x80717fa0 - 0x807f9464   ( 902 kB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Hierarchical RCU implementation.
NR_IRQS:16 nr_irqs:16 16
sched_clock: 32 bits at 66MHz, resolution 15ns, wraps every 65075ms
I-pipe, 66.000 MHz clocksource
I-pipe, 396.000 MHz clocksource
CPU identified as i.MX6Q, silicon rev 1.2
Interrupt pipeline (release #5)
Console: colour dummy device 80x30
Calibrating delay loop... 1581.05 BogoMIPS (lpj=7905280)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
...
...
Switching to clocksource ipipe_tsc
NET: Registered protocol family 2
TCP established hash table entries: 16384 (order: 5, 131072 bytes)
TCP bind hash table entries: 16384 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 16384 bind 16384)
TCP: reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
Trying to unpack rootfs image as initramfs...
rootfs image is not initramfs (no cpio magic); looks like an initrd
Freeing initrd memory: 16384K (8e000000 - 8f000000)
hw perfevents: enabled with ARMv7 Cortex-A9 PMU driver, 7 counters available
I-pipe: head domain Xenomai registered.
Xenomai: hal/arm started.
Xenomai: scheduling class idle registered.
Xenomai: scheduling class rt registered.
Xenomai: real-time nucleus v2.6.4 (Jumpin' Out) loaded.
Xenomai: starting native API services.
Xenomai: starting POSIX services.
Xenomai: starting RTDM services.
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
jffs2: version 2.2. (NAND) ⓒ 2001-2006 Red Hat, Inc.
fuse init (API version 7.22)
msgmni has been set to 3520
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
imx-sdma 20ec000.sdma: initialized
Serial: IMX driver
2020000.serial: ttymxc0 at MMIO 0x2020000 (irq = 58) is a IMX
console [ttymxc0] enabled
...
...
RAMDISK: gzip image found at block 0
VFS: Mounted root (ext2 filesystem) on device 1:0.
devtmpfs: mounted
Freeing unused kernel memory: 344K (80694000 - 806ea000)
init started: BusyBox v1.22.1 (2016-06-15 10:43:42 KST)
starting pid 56, tty '': '/etc/init.d/rcS'

Please press Enter to activate this console.
starting pid 64, tty '': '-/bin/sh'
[root@imx6q /]#
[root@imx6q /]#
[root@imx6q /]# ls -al
drwxr-xr-x   17 0        0             1024 Jan  1 00:57 .
drwxr-xr-x   17 0        0             1024 Jan  1 00:57 ..
-rw-------    1 0        0                7 Jan  1 00:57 .ash_history
drwxr-xr-x    2 0        0             1024 Jun 15  2016 app
drwxr-xr-x    2 0        0             1024 Jun 15  2016 bin
drwxr-xr-x    5 0        0             2840 Jan  1 00:00 dev
drwxr-xr-x    3 0        0             1024 Jun 15  2016 etc
drwxr-xr-x    2 0        0             1024 Jun 15  2016 lib
lrwxrwxrwx    1 0        0               11 Jun 15  2016 linuxrc -> bin/busybox
drwx------    2 0        0            12288 Jun 15  2016 lost+found
drwxr-xr-x    2 0        0             1024 Jun 15  2016 nand
dr-xr-xr-x   64 0        0                0 Jan  1 00:00 proc
drwxr-xr-x    2 0        0             1024 Jun 15  2016 root
drwxr-xr-x    2 0        0             1024 Jun 15  2016 sbin
drwxr-xr-x    2 0        0             1024 Jun 15  2016 sdcard
dr-xr-xr-x   12 0        0                0 Jan  1 00:00 sys
drwxr-xr-x    2 0        0             1024 Jun 15  2016 tmp
drwxr-xr-x    5 0        0             1024 Jun 15  2016 usr
drwxr-xr-x    2 0        0             1024 Jun 15  2016 xenomai
[root@imx6q /]#
------------------------------------------------------------------


<> Xenomai latency 측정
------------------------------------------------------------------
[root@imx6q /xenomai/bin]#
[root@imx6q /xenomai/bin]# ./latency -t 0 -p 1000
== Sampling period: 1000 us
== Test mode: periodic user-mode task
== All results in microseconds
warming up...
RTT|  00:00:01  (periodic user-mode task, 1000 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|     -3.043|     -2.028|      0.108|       0|     0|     -3.043|      0.108
RTD|     -3.152|     -1.927|     12.189|       0|     0|     -3.152|     12.189
RTD|     -3.013|     -1.902|      9.260|       0|     0|     -3.152|     12.189
RTD|     -3.109|     -1.935|     10.295|       0|     0|     -3.152|     12.189
RTD|     -3.006|     -1.879|     12.530|       0|     0|     -3.152|     12.530
RTD|     -3.028|     -1.947|     11.944|       0|     0|     -3.152|     12.530
RTD|     -3.175|     -1.940|     11.242|       0|     0|     -3.175|     12.530
RTD|     -3.102|     -1.975|      9.166|       0|     0|     -3.175|     12.530
RTD|     -3.104|     -1.950|     12.343|       0|     0|     -3.175|     12.530
RTD|     -3.091|     -1.958|     10.580|       0|     0|     -3.175|     12.530
RTD|     -3.089|     -1.955|     11.234|       0|     0|     -3.175|     12.530
RTD|     -3.003|     -1.973|     11.452|       0|     0|     -3.175|     12.530
RTD|     -3.086|     -1.920|      8.444|       0|     0|     -3.175|     12.530
RTD|     -3.124|     -1.925|     12.068|       0|     0|     -3.175|     12.530
RTD|     -3.185|     -1.953|     12.439|       0|     0|     -3.185|     12.530
RTD|     -2.973|     -1.955|     11.510|       0|     0|     -3.185|     12.530
RTD|     -3.079|     -1.955|     11.401|       0|     0|     -3.185|     12.530
RTD|     -3.096|     -1.955|      9.636|       0|     0|     -3.185|     12.530
RTD|     -3.056|     -1.958|     10.664|       0|     0|     -3.185|     12.530
RTD|     -3.081|     -1.877|     11.353|       0|     0|     -3.185|     12.530
RTD|     -2.958|     -1.872|     11.348|       0|     0|     -3.185|     12.530
RTT|  00:00:22  (periodic user-mode task, 1000 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|     -3.099|     -1.958|     11.406|       0|     0|     -3.185|     12.530
RTD|     -3.190|     -1.917|     11.838|       0|     0|     -3.190|     12.530
RTD|     -3.104|     -1.958|     12.045|       0|     0|     -3.190|     12.530
RTD|     -3.033|     -1.940|      9.477|       0|     0|     -3.190|     12.530
RTD|     -2.980|     -1.892|     11.714|       0|     0|     -3.190|     12.530
^C---|-----------|-----------|-----------|--------|------|-------------------------
RTS|     -3.190|     -1.937|     12.530|       0|     0|    00:00:27/00:00:27
[root@imx6q /xenomai/bin]#
[root@imx6q /xenomai/bin]# ./latency -t 1 -p 1000
== Sampling period: 1000 us
== Test mode: in-kernel periodic task
== All results in microseconds
warming up...
RTT|  00:00:01  (in-kernel periodic task, 1000 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|     -5.809|     -4.296|     -1.410|       0|     0|     -5.809|     -1.410
RTD|     -5.299|     -4.212|      7.689|       0|     0|     -5.809|      7.689
RTD|     -5.839|     -4.229|      5.860|       0|     0|     -5.839|      7.689
RTD|     -6.036|     -4.262|      6.257|       0|     0|     -6.036|      7.689
RTD|     -5.807|     -4.261|      4.067|       0|     0|     -6.036|      7.689
RTD|     -5.832|     -4.239|      5.286|       0|     0|     -6.036|      7.689
RTD|     -5.719|     -4.238|      6.799|       0|     0|     -6.036|      7.689
RTD|     -5.164|     -4.241|      6.147|       0|     0|     -6.036|      7.689
RTD|     -5.704|     -4.271|      5.980|       0|     0|     -6.036|      7.689
RTD|     -5.659|     -4.252|      4.008|       0|     0|     -6.036|      7.689
RTD|     -5.422|     -4.270|      3.601|       0|     0|     -6.036|      7.689
RTD|     -5.033|     -4.244|      5.464|       0|     0|     -6.036|      7.689
RTD|     -5.599|     -4.255|      6.290|       0|     0|     -6.036|      7.689
RTD|     -5.844|     -4.233|      5.909|       0|     0|     -6.036|      7.689
RTD|     -5.829|     -4.246|      6.244|       0|     0|     -6.036|      7.689
RTD|     -5.226|     -4.236|      3.805|       0|     0|     -6.036|      7.689
RTD|     -5.741|     -4.213|      4.822|       0|     0|     -6.036|      7.689
RTD|     -5.004|     -4.231|      3.398|       0|     0|     -6.036|      7.689
RTD|     -5.721|     -4.277|      5.842|       0|     0|     -6.036|      7.689
RTD|     -5.095|     -4.245|      6.205|       0|     0|     -6.036|      7.689
RTD|     -5.777|     -4.270|      4.114|       0|     0|     -6.036|      7.689
RTT|  00:00:22  (in-kernel periodic task, 1000 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|     -5.394|     -4.246|      8.788|       0|     0|     -6.036|      8.788
RTD|     -5.677|     -4.195|      4.803|       0|     0|     -6.036|      8.788
RTD|     -5.715|     -4.183|      5.942|       0|     0|     -6.036|      8.788
RTD|     -5.798|     -4.256|      6.323|       0|     0|     -6.036|      8.788
RTD|     -4.904|     -4.132|      3.240|       0|     0|     -6.036|      8.788
RTD|     -5.564|     -4.200|      7.257|       0|     0|     -6.036|      8.788
RTD|     -5.779|     -4.227|      4.143|       0|     0|     -6.036|      8.788
RTD|     -5.693|     -4.237|      5.726|       0|     0|     -6.036|      8.788
RTD|     -5.572|     -4.254|      6.375|       0|     0|     -6.036|      8.788
RTD|     -5.067|     -4.142|      3.623|       0|     0|     -6.036|      8.788
^C---|-----------|-----------|-----------|--------|------|-------------------------
RTS|     -6.036|     -4.235|      8.788|       0|     0|    00:00:32/00:00:32
[root@imx6q /xenomai/bin]#
[root@imx6q /xenomai/bin]# ./latency -t 2 -p 1000
== Sampling period: 1000 us
== Test mode: in-kernel timer handler
== All results in microseconds
warming up...
RTT|  00:00:01  (in-kernel timer handler, 1000 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|     -8.207|     -8.152|     -6.869|       0|     0|     -8.207|     -6.869
RTD|     -8.207|     -8.094|     -2.076|       0|     0|     -8.207|     -2.076
RTD|     -8.205|     -8.123|     -2.359|       0|     0|     -8.207|     -2.076
RTD|     -8.205|     -8.105|     -2.311|       0|     0|     -8.207|     -2.076
RTD|     -8.195|     -8.113|     -2.102|       0|     0|     -8.207|     -2.076
RTD|     -8.208|     -8.119|     -0.008|       0|     0|     -8.208|     -0.008
RTD|     -8.203|     -8.123|     -2.276|       0|     0|     -8.208|     -0.008
RTD|     -8.196|     -8.111|     -2.676|       0|     0|     -8.208|     -0.008
RTD|     -8.209|     -8.092|     -1.744|       0|     0|     -8.209|     -0.008
RTD|     -8.196|     -8.111|     -2.721|       0|     0|     -8.209|     -0.008
RTD|     -8.204|     -8.122|     -0.729|       0|     0|     -8.209|     -0.008
RTD|     -8.202|     -8.119|     -0.676|       0|     0|     -8.209|     -0.008
RTD|     -8.202|     -8.108|     -2.487|       0|     0|     -8.209|     -0.008
RTD|     -8.194|     -8.104|     -2.184|       0|     0|     -8.209|     -0.008
RTD|     -8.202|     -8.103|     -1.919|       0|     0|     -8.209|     -0.008
RTD|     -8.210|     -8.125|     -2.677|       0|     0|     -8.210|     -0.008
RTD|     -8.205|     -8.119|     -0.187|       0|     0|     -8.210|     -0.008
RTD|     -8.205|     -8.117|     -2.655|       0|     0|     -8.210|     -0.008
RTD|     -8.208|     -8.107|      0.115|       0|     0|     -8.210|      0.115
RTD|     -8.206|     -8.123|     -2.494|       0|     0|     -8.210|      0.115
RTD|     -8.193|     -8.102|     -2.380|       0|     0|     -8.210|      0.115
RTT|  00:00:22  (in-kernel timer handler, 1000 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|     -8.209|     -8.118|     -0.395|       0|     0|     -8.210|      0.115
^C---|-----------|-----------|-----------|--------|------|-------------------------
RTS|     -8.210|     -8.114|      0.115|       0|     0|    00:00:23/00:00:23
[root@imx6q /xenomai/bin]#
------------------------------------------------------------------

UART, I2C, SPI, PWM, CAN, Ethernet, USB, SDIO, SATA, CAM, HDMI, 2D/3D Video 및 4개 1GHz 코어와 hardware-FPU를 탑재한 원칩 SoC에서 SMP를 지원하는 RTOS가 불과 1W 언저리에서 돌아간다... 우와, 막강이구먼.
안되는 제어가 없겠다.
어디에 쓰면 좋을까?