Thanks for pointing me in the right direction.
It was indeed an issue with the ashmem, the feature somehow didn't work in my first kernel properly (CONFIG_SYSVIPC).
Now I have chosen a different kernel, enabled all features and the error message is gone! By the way, I'm using now the Debian container provided by Maru.
But still I can't get anything as output (screen share or Displaylink). I will again post the output of mflinger/mclient here.
mclient (no error anymore, process is running):
<6>intial screen config: 1920x1080 508mmx286mm
<6>syncing true display resolution...
<6>found supported mode: 1920x1080
<6>found supported mode: 1280x720
<6>setting screen size to 1280x720 338mmx190mm
<6>waiting for ScreenChangeNotify events...
<6>[t=1030041]: screen size changed to 1920x1080 508mmx286mm
<6>waiting for ScreenChangeNotify events...
<6>[t=1030041]: screen size changed to 1280x720 508mmx286mm
mflinger (logcat):
03-13 17:31:00.013 2079 2079 D KeyguardClockSwitch: Updating clock: 5:31
03-13 17:31:02.921 13000 13000 I mflinger: At your service!
03-13 17:31:06.449 2300 2300 D QtiCarrierConfigHelper: WARNING, no carrier configs on phone Id: 0
03-13 17:31:09.231 1490 2855 D WificondControl: Scan result ready event
03-13 17:31:18.015 13000 13000 W mflinger: getDisplayInfo() for eDisplayIdHdmi failed!
03-13 17:31:18.015 13000 13000 W mflinger: Use default display size 1280 x 720 for at last.
03-13 17:31:18.034 542 542 I hwservicemanager: getTransport: Cannot find entry android.hardware.graphics.mapper@3.0::IMapper/default in either framework or device manifest.
03-13 17:31:18.035 13000 13000 W Gralloc3: mapper 3.x is not supported
03-13 17:31:27.145 2300 2300 D QtiCarrierConfigHelper: WARNING, no carrier configs on phone Id: 0
Somehow I'm still missing something. Still happy about your help!