Page tree
Skip to end of metadata
Go to start of metadata

Minute taking rota
Scratchpad etherpad: https://etherpad.net/p/gsha



November 7


Gunnar: Few participants today, let's get clarity on where we stand and if this development will happen in collaboration.  I will email


KPIT provided an excellent report on the investigations of transferring Wayland technology into Android system / build environment.

Gunnar: As discussed last time, you investigated wlroots to see if dependencies are less difficult than the full weston build.
... because we also said we need at first a very simple compositor running

Sunita: Yes, that's correct.

Sunita's input:

1. wlroot dependencies
meson ( understanding of meson build and mapping with Android.bp)
wayland
wayland-protocols
EGL
GLESv2
libdrm
GBM
libinput
xkbcommon
udev
pixman
systemd (optional, for logind support)
elogind (optional, for logind support on systems without systemd)
libcap (optional, for capability support)

2) version we are using - android - 9.0.0_r48
compiled for mini_emulator_x86_64-userdebug

meson NA
wayland Y ( only for libwayland_client, so we complied it for libwayland_server)
wayland-protocols Y ( only for libwayland_client, so we complied it
for libwayland_server)
EGL Y
GLESv2 Y
libdrm Y
GBM N could integrate libminigbm - 2-3 APIs missing
libinput Not Done
xkbcommon N could integrate/build xkbcommon
udev Not Done
pixman N could integarte/build pixman

3) build modules one-by-one, by changing meson.build to Android.bp per module.

wlroot tree structure.
.
├── backend
├── examples
├── include
├── protocol
├── render -Ongoing
├── tinywl
├── types
├── util -Done
├── xcursor
└── xwayland

4) Our Plan :-

4.1) Check if it is already integrated in Android stack
If not integrated
4.1.1 Understand the APIs that constitute the dependency
4.1.2 Comment out / write stub
4.1.3 Compile and verify that is is successful
4.1.4 Implement actual logic inline with android stack for stub in (2.3)
else
4.1.5 Add dependency under shared_libs[] section in Android.bp
end if

5) Could compile util 2 issue that we have to look into,
5.1) currently wayland_header.h & wayland-server-protocol.h file is
copied directly to src folder.
- how to include generated_headers and generates_sources from
libwayland to wlr_util (c.f. genrule)
5.2) shm_open API used in wlr_util needs to be replaced with Android
specific ( ashmem?)

Eugen: New input lib is required  even if libinput or udev can be compiled it basically won't work.

Eugen:But we can try to solve the main problem first, which is the graphics:
...Receive buffer from wayland, push to android without copying.
..Works only if corresponding EGL extensions are supported

We did not find those extensions in the hardware BSP.

Gunnar: We can look into which BSPs support these.  
...Official aasig development platform includes renesas but also imx8 support and as inexpensive option hikey board
... (and we have talked about other options in this call like android emulator and the android-x86 project). 
We could try to find a platform that works so we don't get 
stuck.

Eugen will list those relevant EGL extensions to today's minutes (or send them).  Gunnar could look for them in imx8 BSP, for example.

Eugen:  Note that we can set up the requirements, analysis and provide help but not lead the development.   We need a real demand from the mother companies if this is to be a real development project for ADIT.


Eugen: Reminder to set up the meeting with HV project

Gunnar: Yes, we have been busy with the virtual platform work.  I will add a reminder for future agendas.




October 24


Ozgur: Have discussed with management about technology.  No firm conclusions.   This is still interesting and needed solution, likely it will come into the industry one way or another.

Gunnar: Agree. Still looking to find opportunity for collaboration on making it happen

Ozgur: SoC vendor support is needed.

Gunnar: What is this needed for?  Secure multimedia pipeline.  Anything more?

Ozgur: A solution that is not using containers would need SoC support in order to build Wayland compositor within Android Linux.

SoC - support teams for Android might help, for example to build some utilities (i.e. do the porting work for some fundamental dependencies, libraries (from standard Linux) and include those in the (Android) BSP delivery package. 

Reviewing current status (Weston port would require a lot more work due to dependencies).

Ozgur: Remember also alternative option which is to use containers.  If weston porting is a showstopper, it is not a showstopper for investigating alternative ways (but this is plan B...)

Is funding available?

Gunnar: The limited GENIVI funding budget is typically directed to things that are concrete and known to be used in the industry when they are done (currently Franca-AUTOSAR conversion tooling).  I think it is unlikely to kickstart this research-oriented work with a GENIVI-funded project.

Proposal still possible of course - the GENIVI board is the decision point for funding.

Collabora interested?
Gunnar: In my opinion they could be interested if there is an active coding project, not before.

Ozgur: Still possible to investigate container solution.  I think there are SoC internal details that are relevant there.  
Stephen: Some SoC specific details possibly, but probably a lot are (generic) software questions.

Ozgur: I would like to discuss more on the design, the diagrams we have, etc.  Discussing with ADIT&Bosch colleagues for example.  Introduce the buffer sharing discussion especially.

Sunita:  I have studied the ideas shown so far, both Android on Wayland and the Wayland on Android.  We have a subsidiary/partner company and we would like to work on this.  It is needed, and a common solution would be good.  What can we begin with?

Gunnar: So far investigatations into compiling Weston in an Android environment, but it has been proved to be difficult.

Ozgur: But we really only need (any) Wayland compositor on Android.  If Weston is not a viable way, maybe a minimal Wayland compositor can be introduced instead?

Gunnar: Another alternative is that KPIT takes the lead for the code project (i.e. commit to some coding work etc.) then we could set up repository and start.

Gunnar: Let's sync up with Ajay to see if he can report what he has done.

Gunnar/Ozgur: A simple compositor could be built, for example on top of libraries libwayland or alternative wlroots with this starting example .

Sunita: I can look into how we could start this activity.


Kimmo says hello and synchronizing with current things.
Gunnar: Note the potential re-started interest in Qt & RAMSES combination.  Talk with Miao.

Progress expected via email on Wayland/Android project, and next phone meeting in 2 weeks.

Adjourned


October 10


Participants:

Discussing memory buffer sharing.  Wish to sync up with HV team.
Possibility to use vsock for Waltham transfer to achieve comparable performance?

There is still interest in solving some of the remaining issues.   But because of availability we switch to bi-weekly calls for a while.

New participant from KPIT.  Also want to discuss Android-Wayland combination opportunities.


September 12

Ajay: I have investigated a bit more what it would mean to compile Weston in Android.

I managed to compile some new things like libevdev

libinput and libudev are hard dependencies.  To continue it would require to stub them out or similar to make things run.  Some systems are still using init should be able to run Weston (but this info is old, so maybe not kept up to date)

At least on Ubuntu...

----

September 05

Participants

  • Stephen
  • Ajay
  • Eugen
  • Gunnar

Minutes

Eugen: A lot of other prioritized work this week so we could not progress it, but I expect this will get better.

Gunnar: Also I have not made too much progress.  I have started looking at but not ready to publish a code repo to investigate weston/wayland build on android build system (similar to Ajay's setup).  Too many unknowns still on the dependency side and since I don't have this project set up with Android build system yet, it's hard to really investigate.  (I need to see it for myself, so to speak)

Eugen: Some multiple display support was investigated by Ajay.  There is support in Android, from Jellybean, but they are primary and secondary display concept.  The primary display resolution controls the resolution used for rendering - this might affect the secondary screens.   Need more investigation. 
(This is not the Virtual Display concept) 
With virtual display you can get access to the content, do what you want... (below the abstraction layer).

Ajay:  Two links to look at:

https://developer.android.com/reference/android/app/Presentation
http://variwiki.com/index.php?title=Dual_Display

Ajay: Need to use the Presentation class (see above) in order to do this dual display.

Stephen: We have some patch (to Surfaceflinger) to support up to 3 displays.  Not sure why it's a patch and not in upstream.

Eugen: In the Variwiki document it says you need to change the Android framework to support independent mode – assuming then each display can run its own resolution.


We looked at the display code in Surfaceflinger (that these patches modify) and discussed some more. 
The number of displays is a fixed number (set to 2 currently), thus the source code needs to be modified to add more.

See for example:  

DisplayHardware/HWComposer.h:224: size_t mVSyncCounts[HWC_NUM_PHYSICAL_DISPLAY_TYPES]{0, 0};

also, here we can see that one primary and one external is expected (DisplayDevice.h):

enum DisplayType {
  DISPLAY_ID_INVALID = -1,
  DISPLAY_PRIMARY = HWC_DISPLAY_PRIMARY,
  DISPLAY_EXTERNAL = HWC_DISPLAY_EXTERNAL,
  DISPLAY_VIRTUAL = HWC_DISPLAY_VIRTUAL,
  NUM_BUILTIN_DISPLAY_TYPES = HWC_NUM_PHYSICAL_DISPLAY_TYPES,
};


Ajay: For "native" wayland port what should be our strategy (if porting weston)?   Port udev over to Android, or try to replace? 
Eugen: Systemd is interacting too deeply with the kernel.   Too much work (to port systemd to android)...  Maybe some simple abstractions possible.  Need to research a bit more what is available (on android, to replace dependencies).

Eugen: Let's try to disable some features to cut off dependencies.

Ajay: Udev monitor is even used to monitor DRM devices...

Eugen: Yes, this is how Weston would detect new connected displays.   They are also using (systemd) socket based activation.  The wayland communication socket is set up this way.  So Weston doesn't need to boot first, but on demand.

Gunnar: So the approach would be to say no dynamic detection of displays supported - just hardcode a static configuration.

Eugen: Is there some configuration for weston. to build without systemd?   For example try it by removing all systemd from system, see if configure script can succeed without it.

Stephen: A side note:  Renesas source release includes a network demo app based on the CastScreen project I have mentioned previously.


August 29

Participants

  • Eugen
  • Stephen
  • Gunnar
  • Ajay

Apologies

  • Harsha
  • Özgür

Minutes


Gunnar: How is the actual practical setup of these experiments?

Ajay: I tried to compile weston inside the AOSP tree
...toolchains were hard to find and use
...Was easier to do it using NDK - it includes the tool chains and setup
..just download latest NDK

Ajay: I then downloaded components like wayland, etc. and started looking at what
needs to be done to compile.

...Of course the compilers are otherwise in prebuilt/ directory (of AOSP tree)

Ajay: Some other issues with dependent components is that they have have
different build system (than AOSP): autoconf/automake, meson, ...

Many advise that we should convert the meson or autoconf to android.mk
file (to make it fit into a systemb build).
But for now with NDK you can directly invoke autoconf and meson...

Note that some related things are already in AOSP tree:
external/ hosts various open-source components including
apparently wayland headers here? But where are they used? --> to
investigate after meeting.

Ajay: You can use https://androidxref.com to search for such things.

Gunnar: What are your preferences for setting up a source tree of
components and dependencies, and parts? I am thinking both of the
full AASIG platform and for this project.
Is it to use github submodules? or repo manifest?

Ajay: No immediate preference...
Eugen: Something where we can easily enable/disable this part of code
(because it will be a failing build for a long time).
We also need to modify a few components.
Gunnar: Yes... so that means pointing to forked versions of some components.

Eugen: As you know the manifest file needs all references to git
repositories. From my experience, they need to have a dedicated hash or tag
because pointing to (always changing) master is a pain.

Stephen: I can send the scripting and integration guide for R-Car release
for AOSP which should help.
Gunnar: Thanks!

Gunnar: (recap) So again, what are the parts and steps needed to set up a
build structure for these Wayland/Weston/Android experiments?

Ajay: I followed instructions from Weston website.
Next cross-compiler toolchain. Created weston dir with deps, e.g.
wayland, wayland-protocols, libinput, ... -> libxml2, ...
NOTE: Some instructions are automake instructions but project now uses meson.

Eugen: There is a risk "No one" (i.e. the main weston upstream) will care about
what we will create. This (The fork) will be a kind of "Weston for
Android". I expect it might be quite different and heavily patched and
this will have less interest from upstream, but that means high risk of
divergence. Therefore at some time we must consider how to align with
upstream (in some manner). I think that if dependencies can be simplified,
e.g. the libinput issue as an example. It seemed to have a to udev,
systemd, ... if we can break that chain such a simplification (maybe)
could be of interest to weston upstream.

Gunnar: Quick look at the UML diagrams of the design to remind ourselves.
OK, next is to start setting up some source tree repo examples, and of
course complete the code-project description document, so that stakeholders
might commit to it.

~ 11.09 Meeting adjourned.

Addendum:  (Gunnar)

Checking the references to the word wayland in AOSP source tree:
(looked at approximately master from 2019-08-20, i.e. Android P 9.0.0)

the actual sources (headers) are stored in
/external/.../wayland and of course have themselves lots of references to the word wayland.


Then if searching the rest of the source tree:

prebuilts/ndk/r20/sources/third_party/vulkan/src/include/vulkan/vulkan.h:#include "vulkan_wayland.h"
... 3 more in ndk/vulkan

external/vulkan-headers/include/vulkan/vulkan.h:#include "vulkan_wayland.h"
... 3 more in prebuilts/vulkan-headers

external/swiftshader/include/vulkan/vulkan.h:#include "vulkan_wayland.h"
... 4 more in swiftshader/include/vulkan

external/crosvm/src/linux.rs: let (wayland_host_socket, wayland_device_socket) =

external/skqp/include/third_party/vulkan/vulkan/vulkan.h:#include "vulkan_wayland.h"

external/mesa3d/include/vulkan/vulkan.h:#include "vulkan_wayland.h"
... 9 more related to mesa3d/.../vulkan

external/mesa3d/src/egl/drivers/dri2/egl_dri2.h:/* forward declarations to avoid pulling wayland headers everywhere */
...
external/deqp/framework/platform/lnx/wayland/tcuLnxWayland.hpp:#ifndef _TCULNXWAYLAND_HPP
7 more in deqp

→ In summary, it is only referenced by additional third party libraries, and while those might be used my guess is that nothing in Android truly uses the Wayland references.   The complete source code of these libraries reference wayland features, but those might not be enabled in the way the libraries are actually used in an Android build.


August 22


Ajay: Tried to compile Wayland and Weston in Android environment.

  • libinput → udev → systemd is the basic dependency chain.
  • Dependencies to libudev and udev daemon.  udev has dependency to systemddevice, systemdevents
  • Compiling systemd would likely be a major problem

Gunnar: At least a few years ago libudev claimed to be possible to compile separately - the code was supposed to be in systemd tree but not "merged" into systemd itself.

Ajay: I found dependency to "systemddevice" and "systemdevents" at least.

Gunnar: It might be required to look at other Wayland implementations?

Gunnar: If there are no viable options, at least there are helper libraries such as libwayland and wlroots that may be a stating point?

Ozgur:  Yes, I mentioned the risk that input devices will be a problem area (since the whole software stack down to drivers are different).
I think we need to discuss with the SoC providers.  We need the kernel/driver level discussion also.

Ozgur:  I don't see the input path is critical.  We could run (the compositor) without input support - at least it works in our compositor. Can we simply disable input handling also in Weston?

Harsha: I think it's not (easily) possible.  Modifying the source code in that case.

Ajay: But some of the DRM events seem to also go over udev?

Ozgur: Is it possible to hardcode some things, just use devices that we know exist, bypassing udev?
Harsha: It's an ugly hack, but maybe for a first system.
... original idea is to get the MESA / graphics point of view first.

Ozgur: Yes, I also want to see the graphics part work first.

We can't host both sets of middleware together.   It needs to be using containers.
LibSSL a big difference, so many things.

Gunnar: What parts of our problem are actually solved by namespaces?  E.g. similar code that accesses kernel features and may conflict without namespace?

Harsha: It is also that the names of commands and libraries are the same, while being different.

Gunnar: OK, yes, then it will be difficult libraries might be loaded by name matching. 
(OK, then a filesystem namespace might help).

Ozgur: I think it might be a good investment - to "hack" the minor utility components just to get rid of the problem (in an easy way) so that we can focus on the graphics part which is the interesting one.

  • Final objective: Sharing the outputs from one of the other.
  • Ultimate goal:  Android display as a Wayland client
  • Reuse hardware composer which will be a Wayland client.  This is the final step.
  • Android will be a Wayland client, and other Wayland clients, and the system compositor is a Wayland component
  • To achieve it, we need a working compositor Weston or other.  Then, describe how to hack libraries to achieve that.
  • From the driver level in the EGL abstraction - sometimes MESA drivers or other.  Each SoC might differ there. 
    We need the Wayland protocols to be supported.
  • Forks of dependencies software might be necessary.

Gunnar:  I would like clarification of what the SPURV project achieved so far.  What is new and different, what (value) are we adding? 

Ozgur: It is a container approach.  (And it uses GoogleOS?)

All:  We are uncertain to the availability of the (full SPURV demo) implementation fully (drivers, etc.). 
...But it's ETNAVIV based?

Gunnar: Kimmo - would QtWayland be an option?  After all Qt framework is ported to Android, so most of those dependency issues solved at least?
Kimmo: I'm not sure which dependencies it brings in.

Ozgur: Could we use a native graphics – direct native rendering using KMS application using DRM, so a standard application but also work with a small limited set of wayland protocol.  A very simplified compositor...

Harsha: Yes, but we are experienced with Weston so we would like to try this first (only write something new if we really find it is impossible)

Harsha: I'm concerned how long we can commit to the implementation project - things can change quickly.

Gunnar: Agreed, but only if we start something, only then is it possible for others to continue it.  Of course we must be producing new value, but if we do that then others get the opportunity to collaborate.

Ozgur: Well the HW compositor approach was done already by Collabora project.  But now we are talking about also adapting the Wayland compositor itself and this is a great opportunity.  That work is very good to have as an open-source project:   It can be more difficult to do work [below the HW abstraction layer] in an open-source fashion.  This should be easier to do this way, and if a quick hack is done first, then it will need a lot more work from different places to complete it.

Apologies:  Ozgur not available next 2 weeks. 







August 15, 2019

Small participation (3 people) today so we keep it short.  Brief discussion with Eugen on last week's minutes to update after vacations.


"Android on Wayland"

Mostly recap of last week's discussion to sync up.

Tryout of spurv started, as mentioned.  Make sure to run a Wayland based host system to make this work seamlessly.

Looking at Anbox.io  a container solution which seems X11 based.  Mechanisms of containerizing applications in X system compared to Wayland.  For Wayland, it's necessary to share the communication socket of course, and the ability to share file descriptors representing memory buffers.

Clarification Android simulator : It is about understanding fundamentals, what is being done to get graphics displayed on the host system.  We believe it only supports X11, or does it run also on Wayland based systems?  TBD link to source of simulator?  Is it FOSS licensed and published?

"Wayland on Android"

A related article:  Why Wayland on Android is a hard problem

Gunnar: Info/update: AASIG group defining Android dev system.  So far  [for a number of sub-projects that have started]
Stephen:  On-hardware testing moving along.  Renesas board farm exists. Lava-based system planned and and being implemented now by Renesas&GENIVI.
Gunnar: Agree, it will be integrated to go.genivi.org first (but if individual code projects use Travis/similar we may be able to attach that as well later on).  Either way we can use it for this project if we want later on.

11.00 Adjourned and planning for a larger group sync up next week.

August 8, 2019

Participants

  • Harsha

  • Stephen
  • Philippe
  • Ajay
  • Kimmo

Minutes

  • discussion on Android & Wayland continues:

    Harsha: Ajay was trying out Spurv.
    Ajay : Tried on virtual box, but build instructions are for i.mx8.
    Harsha : Most of the components on SPURV need to be open source except etna_viv. May be we can modify the manifest to build mesa instead of etna_viv drivers.

    Harsha : I am unable to work on Android topics for last few weeks due to other high priority interruptions at work.

    Harsha : Kimmo, Stephen any comments on the architecture diagrams from Eugen on alternative 1 and 2?
    Kimmo : Not as of now. May be next week I can have some inputs. Also QT android and wayland colleagues can join the call next week.
    Stephen : Busy with automated test for last couple of weeks. Yet to take a look.

    Harsha : Kimmo, are you doing any POC OR is it just out of interest, QT colleagues are joining?
    Kimmo : The discussions are of interest for QT colleagues because of QT offering : https://doc.qt.io/QtAutomotiveSuite/

    Ajay : As per earlier discussion we are supposed to focus on alternative 2.
    Harsha : yes, ozgur was looking into alternative 1. Problem of memory handling between wayland and Android is common for both alternatives. Alternative 2 takes time.

    Harsha : Actually I think easier option than SPURV is to use the Android simulator. We can cross compile Wayland, weston to run on Android simulator. For surface flinger, hw_composer wayland (part of SPURV) can be used.
    We can clearly look into mesa EGL for the kind of modification required.

    Ajay : Yes we can check this.

    Harsha : Kimmo, what do you think about using the simulator?
    Kimmo agrees for the option to work with Simulator and cross compiling weston,wayland.

Apologies

  • Gunnar

Next: August 1, 2019

Participants

  • Eugen

  • Stephen
  • Philippe
  • Ajay

Apologies

  • Gunnar

Minutes

  • Ajay: Spurv (from collabora) can run on any system which support wayland backend, another option might be Anbox (Android in the box) here you can run Android system by adding required kernel modules and start "special" Android image. Applications have Android look and feel but are running each in own X11 window.
  • And still waiting for Ozgur's (smile)
  • Group talked about the plans to have Techsummits in the Fall
  • Philipp informed about the launching of a Android related project about vehicle data apis that happened this week

July 25, 2019

Participants

  • Eugen

  • Stephen
  • Philippe
  • Ajay
  • Gunnar

Apologies

  • Harsha, might be away on training (TBD)

Minutes



July 18, 2019

Participants

  • Ozgur
  • Harsha

  • Eugen
  • Ajay
  • Stephen
  • Gunnar

Apologies

...

Minutes

Discussing the Android development platform.  The page already took input from GSHA group (e.g. Emulator proposal).  Additional input from GSHA group?

Ozgur: I am familiar with all three (Renesas, Qualcomm, NXP) but the most important is the support level from the companies.
...maybe when we know that we can make a choice

Gunnar: Agreed, but please give input you have now (or join the SIG meeting), since it is possible that a first choice might be made in the SIG.

The group agreed we should aim for reusing the common development platform

What kind of support are we talking about?


Harsha/Ozgur:  Surely we need BSP support, GPU drivers, etc.  (but there may be more)

Ozgur: ... also the Wayland development matters, graphics libraries and such, some Wayland dependencies might not be fully available right away in an Android platform.  EGL libraries...

Stephen: Waltham?

Gunnar: (The Waltham library itself...) contains only general networking, plain C structures, and Linux programming from what I can see.  Anyone know of particular challenges?  Obscure kernel API calls that might be disabled on an Android kernel?

Harsha: Have not seen any issues.

Stephen: I am thinking of the video encoding perhaps

Harsha: OK, depends on implementation and HAL access to (hardware) codecs.   Ofcourse you could do something with software impl ffmpeg etc. but performance would not be good.

Ozgur: But Waltham is not really part of the currently discussed PoC, right?

(Everyone):  Yes, you are right, agreed.
Gunnar: Still an interesting topic to work on, just not the exact scope here.   There could be some synergies

Eugen:  I have looked deeper and the current diagram (Android display subsystem as a wayland client) does not show everything.  More is needed.

Ozgur: Yes, still good overview.

Eugen: Hopefully the HAL interface of the HW composer hides all DRM access.

Eugen: If the ION interface is using DMA buffer allocation below the API, then this is more easily reusable on the Wayland side of the system.

...More discussion about memory allocation. 

Conclusion:  Check the implementation of the memory allocation, for example gralloc, in a recent Android kernel . 

Harsha found a relevant library:  https://github.com/robherring/gbm_gralloc

Eugen will contact Rob Herring to see if we can bring him into conversation.


Eugen: I will be away most of the week but back next meeting.  Not as much progress expected however.

Gunnar: I will have vacation starting 1-2 weeks from now.  Harsha and Eugen should lead the meeting.


July 11, 2019

Participants

  • Harsha
  • Ajay
  • Gunnar (few minutes)
  • Eugen
  • Kimmo
  • Jaimohan Shrivastava

Apologies

  • Ozgur (vacation)

Minutes

only few people where on time, meeting started around 10:45

Discussion on the Architecture diagram Wayland on Android 

pointed out a potential problematic points:

  • EGL driver implementation of the Wayland and Android EGL's.
  • Implementation and structure of the special Android App which will run the Wayland Compositor, this one has to get the wayland buffers and forward this to the surface flinger, without a copy of course
  • GPU kernel driver could it be the same for Android and Wayland user space libs?

more details are here: Wayland Application on Android

It was discussed to continue the discussion on the Wayland on Android use-case

Jaimohan would like to see similar diagramm for Android on Wayland and commnt on this → Eugen will do this by end of the week


July 4, 2019

Participants

  • Stephen
  • Harsha
  • Ajay
  • Gunnar
  • Eugen
  • Philippe
  • Kimmo

Apologies

  • Ozgur (vacation)

Minutes

Thanks to Ajay for taking minutes


Eugen Provides new idea (Project Idea-4) for Android and Wayland Graphical system collaboration

"Split system" where each software stack is running mostly independently and graphical output is either on separate displays, or combined through GPU Sharing, Display Sharing (hardware layers or other).   This could be implemented with a Hypervisor or using Linux containers.

GA made  edits to Android and Wayland Graphical Architecture  page and added  new option  "Split system" to Project Idea. Graphics Sharing is now termed as Surface sharing in Project Idea option 1   

GA explains Eugen , about below  alternatives discussed in last meeting and asks his opinion about them.

  • Alternative 1A-1 - Multiple virtual displays, each with one app (typically full-screen) This simulates individual Android apps become individual Wayland apps.
  • Alternative 1A-2 - Actually handle apps as individual Wayland clients. This seems to be major SF rewrite or workaround

Eugen shares similar view on alternative 1A-2 as Harsha. In this option we are kind of leaving window manager in Android. If applications are implemented as Wayland clients , we may have to implement the application lifecycle management differently.

GA: why would the application behave differently ?

How about a full screen app and there can be icons on top of it for other apps. (Possibility- One)

GA : can we have a half screen view ?

Eugen: yeah, but this is managed by window manager

GA: yeah, exactly . My feeling is that this is more like a compositor , enabling the possibility of showing two apps at the same time, rather than the app behaving differently. May be there are  certain features those may not be possible .

HM adds , the point when App goes to background and foreground, there might be specific messages required to inform the app that , it is in background and its in fore ground. It looks like there are still more things  which is not known yet. We need to dig in.

GA : then again , there might not be more than a  handful of messages that we need to somehow find a solution

HM: For example in android video streaming ,as far as I see , in window manager there will be ways  how the next frame is rendered.

There are synchronizations , they will originate from surface flinger. The moment we override this with our own compositor getting the app content then we have to do this for the video , it has to run as it runs in android.

GA: yeah,

Eugen: but we should not cancel this point , we have to keep this as an option. Even it might be complex .
May be for single app or full screen app , this would even work . If its complex , we can comment , its complex because of these points ….

Individual client enablement.

HM: we will keep this option but we will not actively look into this option now

Eugen prefers to look at the first option  Alternative 1A-1

GA: option naming has to change .

The first two option does not talk about virtual display , somehow we wanted to hack into the HAL to make it seem like  a normal display

HM: both the options Alternative1 and alternative 1A-1 can be generically summarized in one statement providing ,actual display or a virtual display content to a Wayland compositor

GA mentions Virtual display is a separate concept in android . There is actual API for that  (so that's why the alternative is different)

All options moved over to Android Application on Wayland page and renamed respectively  A, B, C as below

https://at.projects.genivi.org/wiki/display/DIRO/Android+Application+on+Wayland


Options

  • A) Single full display becomes Wayland client/app, assumed 1 app only (whatever is composited as one display by SF becomes a Wayland app).
  • B) Multiple virtual displays, each with one app (typically full-screen) This simulates individual Android apps become individual Wayland apps.
  • C) Actually handle apps as individual Wayland clients. This seems to need modifying SF or simulating some messaging between SF and app to get the right behavior.

This would be our starting point .

GA ask if we can take the steps to move some of the design diagrams into the page .

Eugen agrees and will continue to work on the EA diagram . But whatever design diagrams he provide is definitely not final and it represents his own idea and definitely needs some modification based on feedbacks.

HM agrees to have the diagrams now , as we have at-least the basic idea . It can be put and we can refine as we proceed

GA starts the discussion about Wayland on Android .

A Wayland compositor backend (possibly implemented as a Weston plugin)  acts as an "Android App"

GA asks if its possible to start planning the solution for this as well ?

Eugen: Yes , we just need to provide the Diagram and then we will see I mean even by diagram I already see some points which are critical.

Harsha agreed to add some  design diagram this week.

GA will send a reminder to Ozgur about the progress on focused description on intended scope of this shared  PoC development .
→ [Ozgur is on vacation for 2 weeks so we will follow up when he is back]

GA provides an update from  Android Automotive SIG.

Low cost:  Hikey and dragon board as noted before.  GENIVI is reaching out to Renesas and Qualcom for full automotive dev platform -  officially asking to support defining a reference platform for us  


HM have now started working on Android X86 (virtual box)

  • Could build everything , generate android  image and set it up (all looks good ) 


GA  Agrees that it will be a good place to start early investigation  (I won't assume we will have whole lot of HW specific issues )

Once we get the project up and running we can definitely try to move it to the official development platform later


HM: Yes,  because there is also MESA in place where you can change , play around and there is really good scope for changes , as its

Completely open source, at least from the graphics point of view I see its quite ok for now.


HM will share more on his experience and observations once he starts some hands on development on the Android X86 setup


GA : we can provide a little instruction on how to do the set up in our Graphics sharing page , overtime we can move it to real development platform


Eugen observes that the recent versions of mesa don’t have  framebuffer back end anymore  , its removed

Eugen asks HM , if in x86 Simulator they are using older version of mesa.


HM : its 18.3 in the x86 build what I have. I have not run it yet , once I run I can check the back ends used.

HM sees that we have to use VMWGFX but that does not appear, but once he runs it , he will have full info and he will share it


Ajay confirms that , in Android Studio based emulator framebuffer device  was used  (/dev/graphics/fb0)  (Pixel_C_API_27)


Gunnar asks HM, so how  in practice  these things are set up ? Is it a complete android tree with some additional components?

Are you also doing some modifications to the components in the android tree ?  [ ... which means we would keep some forks of those]


HM: that’s the aim Gunnar. To modify the HAL layer   and experiment few things.

Regarding instructions its very clear on AndroidX86.org


Harsha shares the links to the instructions over chat  (contains instructions , repositories and manifest information)

https://www.android-x86.org/source

https://git.osdn.net/view?a=project_list;pf=android-x86 ()

Harsha is yet to explore the build system


Gunnar:  I'm used to setting up git submodules to get all dependencies in one place.  But of here we need to create/modify a repo manifest of course.

Gunnar will look at the links and figure out how it would look in a github repositoy


Gunnar asks Eugen , How  ADIT can contribute  and share to the open source project here.

Eugen: It’s a good point, but no idea at the moment how much ADIT can contribute and what it can share

             so can't comment on sharing or  maintaining any source code . With the current set up at ADIT it won't be possible


Eugen agrees  that we continue the current activity  of creating some diagrams and architecture

Eugen mentions if we can somehow link this to Google summer of code  and try to get some help from other people.

Gunnar: its already too late for that . It is running now, and a new project cannot be started until next summer

     We could something similar (internship) or we could find a way to convince your customer and make it project related  so to speak.


All agreed that with a project description, the conversation about the open source project can start properly within each company, including ADIT.


Kimmo hoping to see the architecture diagrams , show them to his Wayland guys and ask their opinion 

Gunnar asks Kimmo if they also could provide back some diagrams on how to create the Qt architecture and so on


Harsha Asks Kimmo is they have QT running on Android ?

Kimmo : Yes

Harsha: does it run like any other android app ? Is it possible to share details of the concept behind ?

               In the perspective of  Android app on Wayland and vice versa , when you say QT running on android , what do you mean ?

    is Qt running as a complete UI framework overriding Android or Qt exists in the android ecosystem ?

Kimmo: you can write application with Qt and those run on android so with qt cross compile for android and you can run it on android


Gunnar: you basically have Qt graphical libraries accessible to you when you write your application

Eugen: How is it really integrated in the device ? Usually you would have some JNI interface  ?

Kimmo : yeah, I think under the hood there is JNI to access the actual hardware .

Eugen: if I have two application, does each application includes the complete Qt stack or someway they share the libraries ?

Kimmo : currently libraries are inside  the apk so it has all the libraries . But there used to be this ministro thingi which enables sharing the libraries    so the they can be kept in one location in the device  , and all application can use those from there

But currently by default it is bundled inside the apk


  Eugen: ok got it , but this will be really interesting but I have not seen any real implementation yet

Gunnar: minstro is like an application you can download from the app store and then it will download all the libraries and then application can use them, and they don’t have to bundle the libraries. Guess due to some android security implementation, this got changed?

Kimmo shares the App name and qt documentation

ministro

https://doc.qt.io/qt-5/android.html

Qt documentation is really good and there is some special team who does the documentation.



June 27, 2019

Participants:

  • Ozgur
  • Stephen
  • Harsha
  • Ajay
  • Gunnar


Minutes:

Gunnar: Today we're again a split group with varying participants.  But we can catch up on progress this week and the sync-up between Harsha/Ajay and Ozgur can occur.


HM?: Collabora implementation can be reused.  It is a Wayland backend on surface flinger.  SF simply acts as a Wayland client
OB: Makes sense. They make the changes directly in the HWC part?
Yes it's the hardware composer HAL
OB: Good if they make the changes only in the HAL layer

HH: Support for some memory allocations etc. It looks like this....  A good basis to explore memory handling.

OB: Did you run on it on imx6?
HM: No, only studied code so far
OB: ... or maybe IMX8 supported? We can check this offline

OB: All Android windows can be rendered to Wayland?
HM: Output of SF is the whole display, so it will be treated as one application in the WL compositor.

OB: The compositor needs to know the handles of the buffer. Just for management, not the actual sharing. One will render, one will composite.

HM: Linux DMA buffer protocol can be used - already available and implemented in Weston. Standard Wayland stuff.  EGL: this is where the understanding of the buffers lie

OB: Compositor needs to use DRM/KMS operations to do the actual compositing. This is still standard though. I just mean they need the buffer
identifiers - the business logic is in compositor, only buffer processing is in the EGL/driver level. All agreed.

GA: OK.  To be clear we are discussing the "alt 1" today. This would be called Android on Wayland.  But I wonder if we now have 3 categories or variants.  There's either a full SF output, or individual apps being treated as apps in Wayland.  Does there exist an alternative to get individual apps handled as individual apps in Wayland compositing.

HM: Surfaceflinger is involved here, this is unavoidable (without large reimplementation). The SF can of course put apps in front or in back.  You would reimplement compositor in that case.

GA: Yes, but it can be desired to handle apps at the same time. And some may prefer to implement the compositor using Wayland technology instead.

OB: I agree.  I think also this alternative should be listed separately. There can be interesting (business) opportunities given from this.

HM: I don't see a big business use case for separate app handling like this, but I agree from technical perspective it's an investigation area.

GA: If using one virtual display per app, then SurfaceFlinger need not behave any different. Just configure apps to be full screen, each on their own display.

HM: Yes with that solution it should be possible.  Make each Android app a wayland app is another way, but perhaps difficult.

OB: Using Android connectivity with some information available from Wayland
clients... (for connectivity things).


Gunnar summarizing in minutes:

  • Alternative 1 - Single full display becomes wayland client/app, assumed likely 1 app only (or whatever is composited as one display by SF).
  • Alternative 1A-1 - Multiple virtual displays, each with one app (typically full-screen) This simulates individual Android apps become individual Wayland apps.
  • Alternative 1A-2 - Actually handle apps as individual wayland clients. This seems to be major SF rewrite or workaround.

GA asking Ozgur about supporting a shared repository for development.

OB: Management decision, still.

GA: As mentioned last week, I think we still need a short and focused description of the intended scope of this shared PoC development, which management of different companies can review and give OK to develop on.  Ozgur, can you start/lead?

OB: I will take lead to get started on that, but I want the input from the other participants.

All agreed this is the way forward and agreed to give input.

HM: We ultimately need Eugen to way in on the implementation matter.  Ask him.


Generally agreed that Collabora code base is a starting point and could/should be reused.

OB: There may be some details we want to do different, we should make it flexible to adapt to some other interfaces, but yes let's start there.


Gunnar summarized the intro to GSHA given to the the Android SIG , i.e. projects we have done already and are doing, that relate to Android.

Discussion about organization.  The intention is to have SIG identify and prioritize topics. But the technical work should be done in dedicated child-projects like GSHA already is for all graphics-related stuff.     (Actually the SIG is too big now and need to be split into dedicated groups even for the preparation / requirement setting).

Gunnar: When we have some more results from the current investigations, we can then come back and give update to the SIG.



June 20, 2019

Participants:

  • Eugen (until 11)
  • Harsha
  • Ajay
  • Gunnar
  • Philippe
  • Stephen

Minutes:

Eugen going through some design ideas in UML (for alternative 2 aka WaylandOnAndroid)

Does the Android Backend it fit into Weston plugin architecture?  
Eugen: -- Maybe. Alternatively it can be a special compositor for this purpose

Harsha: Android backend may need to behave as a "Java" application
This would make it a kind of Wayland app, that can be moved to the front or back by Android Window management as usual

EGL native buffers  coming from Wayland GPU libs
Android GPU libs should support that handle.  Then buffer mapping would be possible.  Wayland compositor can then access these buffers.
But if Android uses different memory allocation framework it can be more difficult.  To be investigated.
At least the GPU drivers seem the same

---

Alternative 1   Android apps on Wayland main HMI

Ajay.  Renesas hardware , now also on Android emulator x86
Need to recompile the code of course for x86
We could use a wayland backend 
emulator showed us it was using the framebuffer device
I am taking logs and looking deeper into how surfaceflinger works under the hood

Emulator runs in Android studio.  We can pick any platform/version, also AOSP 
Emulator must start with root.  So that adb can push binaries.

Another x86 project, you can get a VDI or ISO image.  Install into VirtualBox
The ISO you can download from Android x86 project. www.android-x86.org has ISO images

Harsha:  The Android image remains the same.  Backend is the only thing that differs.  So emulator should be representative for a lot of this work.

Good starting point in Collabora's spurv code.  Shows a way to handle memory buffers for Alternative 1.

Gunnar show some work in progress on Whitepaper finalization



June 13 2019

Participants:

  • timo
  • violin
  • sriram
  • ozgur
  • harsha
  • Ajay Kumar Sahoo (Bosch)
  • eugen
  • stephen
  • PR
  • GA

Minutes:

Ajay is doing exploration in idea 2 "Wayland on Android"
...Implementation Idea: "Weston is an Android App."
...Looking How to do it in only native layer, not interacting with application framework.
---Request a window from surfaceflinger, start doing opengl rendering
... easy mock application running on android, it works now
...explore more a bit of internals.

Harsha: Android also has a windowmanager, with surface ordering, similar APIs to LayerManager APIs. Already Android is running with graphical user interface. Right now it overlays on top, we need to implement control.

Harsha: We can produce/show architecture diagram soon, after a bit more investigation.
...We are developing on Renesas.

Gunnar: Which Android version do you need?

Ajay: Primarily version is "O"... It's code delivered from Google...
Harsha: Need to look more carefully. Developers get the platform delivered from an internal platform team (so we don't need to know exactly).

Discussing if Renesas can provide a click-through license for an evaluation versions of the Graphics drivers.

Looking at Android emulator would be useful.

Harsha:  We can try running on the Android emulator and report.

Platform delivery. More discussion on how to set up the shared Android development platform for this investigation.

Gunnar: We have had some input discussion from the Android SIG on this. Some companies prefer a full-fledged Automotive platform, some more cheap and easy dev boards. I think we might have more than one, but the important thing is to get started.

Gunnar: Does it run on AOSP?  [without any latest extensions that are only available to partners]
Ajay/Harsha: Yes, it should...


Ozgur : Main challenge is the memory handling between the graphics layers. When using the emulator there is some virtual hardware...
I don't think it makes sense... If you work on real hardware it will be easier to get the resources.

Violin agrees, this is an interesting idea and interested in discussing the memory handling. Also Eugen the same.
(Some further discussion...
Ozgur is focused on Alt. 1. here)


(Back to alternative 2...)
Gunnar: I'd like to start this as a public repository with FOSS license at the start, rather than first develop and then decide if it can be "released". Also this is to encourage.

Harsha: ADIT decision, let's check. Eugen?
Eugen: I think it can be possible. Need to have a description to discuss with management.

Gunnar: I can set up repository any time. We can start with an empty one and you fill it.

Gunnar:  It requires more than one company to commit to participating... How about Mentor - Ozgur?

Eugen: I would also target the result to go into existing projects. Examples is adding Android backend in Weston. I.e. it should go upstream.

Agreed... Gunnar: (later, at the end of the meeting): To summarize, yes if we create something really useful the aim should be to push it upstream to for example Weston and AOSP wherever it can fit. But there is a lot of investigative work first before it can be packaged that way.

Asking Ozgur about contribution to an open project
Ozgur: I can ask for approval when we have a clearer plan in place

Ozgur: I would like to step through the detailed design like memory management. I am working on this. How to share memory handles between parts for example. How they will share the memory handles.
I can start up a discussion on that later on.

Violin: You mean .... buffers?
Ozgur: Yes, DRM, KMS buffers, ...

Violin: I would be interested in this discussion.

Timo: Yes, also Qt as mentioned last week. I presented this idea of "Android on Wayland" (Alt 1) to management. But we don't have resources to spend time on this at the moment.

Eugen: ADIT looking first into alt. 2. and maybe later on alt. 1.
Takes a few weeks to 1 month to make some progress on this.

Eugen: Idea: For option 1, if it's possible to convince Android to work *only* with a virtual display, then this should be solvable. (Currently I think it fails if not at least one physical display is found...) There is an API to get to the content of the Virtual Display, so he content of could be passed to Wayland compositor quite easily. A good PoC to make.

Actions/Next steps:

AI(Eugen): Start writing up a project scope for code that supports Alternative 2.
AI(Harsha): Check if the PoC software also runs on Android Emulator. (Not guaranteed to have time this week)
AI(Gunnar): Write the "select the right technology" on GSHA whitepaper, based on Eugen's AMM presentation.
AI(Ozgur): Keep investigating memory management for Alt 1, preparing for a discussion later on



June 6 - 2019

Minutes pending


June 4 - 2019 Graphics Sharing Webinar

Participants:

  • 12 participants + GENIVI staff

Recording

Slide deck

May 23 2019

Participants:

  • Gunnar
  • Eugen
  • Stephen
  • Kimmo


AMM Debrief

- Kimmo - In our presentation, quite a full room, really interested audience.
Kimmo - Interest for functional safety interest
HV, GSHA, combines different domains
Android Automotive - not deep detail, technical level.

Kimmo - Hotel, food, WiFi, coffee -- well organized
Demo evening (showcase) -- interesting.

Eugen - interesting to refresh connection
Xen hypervisor talk interesting
Eugen: Interesting that Renesas driving many of these technology areas
Stephen: We support multiple hypervisors and technologies, we don't limit to a particular vendor
I would like ADIT to work more on Xen - community collaboration, also with Android. Discussing this with Vasco.

HMI tooling

Eugen: Altia - good job in general. Implemented distributed use-case.  The magic is implemented in their Deep Screen component. Hardware specific, optimized how to render efficiently. Plane, GPU, and define where to see it. Native code generated, composition done with native code. Interesting approach.

    [Others: It felt like too much of a sales pitch]

Eugen: But all proprietary tools have the same problem - expensive customization.
Eugen: I asked questions, they were able to answer them quite well. The speakers knew the technology well enough to answer.

Eugen: Wayland-Android idea from our workshop. Is it valid use case? Let's investigate.

Eugen: Vulkan presentation was interesting.

Android: GENIVI should try to collaborate with Google.
Gunnar: Yes. I think OEMs need to drive this desire for discussion.

Eugen: I want to implement more code, show explicit use cases. Makes sense to do it. I'm missing the time to research into future technologies. Graphics sharing use cases on Xen would be interesting to implement.

Stephen: Vulkan talk was curious. The title was functional safety -- but the message "you shouldn't use it for functional safety, yet" -- the technical committee need to
GPU rendering is optional.

Vulkan API is not really for 3D rendering? You can have a Vulkan driver which is not implementing the 3D graphics.
Gunnar: Isn't that because it aims to support also general purpose computing on GPUs? (OpenCL style)
Eugen: Mostly it is very low level, it describes the capability of the hardware.

Gunnar: Eugen, can we record a new instance of your "How to choose GSHA technology" talk?

Eugen: Happy to do this.
Eugen: Next week should be quite OK?

Eugen: Live (webinar) presentation would be better.


Qt ideas

Kimmo: Shared interest in Android Auto.  How I see it - Getting App Stores and content into Qt offering.  Graphics sharing from Android - get application surface and show it in a Wayland window.

Or Dalvik VM integrated (separately).

Gunnar: In my experience taking Dalvik / app runtime out of Android has been only moderately successful – difficult challenge and compatibility is never 100%.  It has been done several times over (in the last 10 years or so?).   [ See for example Bluestacks and other example "emulators" for desktops.] 
Has been offered by suppliers for automotive, but I am not sure if it was ever put into automotive production.   
    [ Of course such a system is also unlikely to meet Google certification tests ]

Investigate - how to get Android application surface to Wayland world.

Eugen: It might be important for you to show that you can use legacy applications with Qt, because many feel you must do everything with Qt if you choose Qt.
Kimmo: Using wayland, clients are not tied to Qt.

Eugen: Which Wayland protocols are you using?
Kimmo: I have to check but I think it's only standard Wayland.

Gunnar: That's my feeling as well - but maybe worth checking protocol version or something... 
The difference is that you have a QtWayland component implementing it, so that you can build your own compositors out of it, using all the Qt features.  It is not one specific compositor only.
Kimmo: Exactly

Stephen: On surface sharing in and out of Android. We used an open-source technology with screencasting. I listed one of them in the Wiki page.

Gunnar: How to take Android surface and make it into the surface of a Wayland client talking to.
Eugen: Also to run (show) a standard Wayland client in a Android system.
Stephen: Inter-ECU? Could be both.

Stephen: Thomas Bruss was pleased with the event. We may follow up with him or his team.
Eugen: I would appreciate that also.

Eugen: For the recorded presentation I can Invite some colleagues
Gunnar: Let's do a poll for the time and check interest. Probably doing it in two weeks from now would be better - enough time to prepare.



May 16 – no meeting (AMM)



May 9 2019

Minutes TBD

Apologies

  • Kimmo, traveling 


May 2 2019

Particicpants

  • Stephen
  • Kimmo
  • Gunnar
  • Harsha
  • Philippe

Apologies

  • Eugen (vacation - mentioned last week)

Minutes 

White Paper

  • Going through great new input from Harsha, texts and pictures.
  • GPU sharing... hardware assist.
  • Stephen:  It's also possible to have multiple register sets per OS, so that they are completely separated.
  • Gunnar: How is the OS ID actually managed, i.e. what actually "sets" the OS ID when VMs are switched (to avoid spoofing this must be in some privileged execution mode?, in other words some minimal Hypervisor must be involved to influence this?).
  • Stephen: Not fully sure on the details of that exactly but for some (other) aspects there is some paravirtualization involved, e.g. there are minor changes to libdrm.
  • Gunnar: I'm only missing a description of a full virtualization of GPU, where the hardware has little or no special hardware support for it?   I'm unaware if this is practically done (in embedded/automotive), outside of desktop OS virtualization like VirtualBox/VMWare?
  • Harsha: The virtio-gpu 3D chapter that follows describes the host/guest principle.
  • Gunnar:  OK, yes you are right, this is more what VirtualBox/VMWare on desktop is doing.  There is a clear host & guest role there.
  • Gunnar: But what about if the VMs are on equal standing, just running on some full virtualization / emulated GPU.  Again, do we have practical examples of this - otherwise it could be more of a minor note.
  • Stephen: I  think Intel graphics has some kind of (hardware) master that arbitrates the separation.  But that is a kind of partial hardware assist, in between full and advanced features found in other SoCs, and no hardware virt support at all.
  • Stephen: Integrity has some other implementation, but the end result includes combining graphics into the final whole.  A bit unclear on the implementation.

AMM Workshop

  • Going through AMM workshop topics.  Android↔Wayland is a topic of interest.  But who will lead this discussion?

TODO

  • Gunnar asked all participants to provide more clear topics, presentations, problem statements to discuss for the workshop.

Other TODOs


Apologies


April 25 2019

Minutes TBD


April 18 2019

Participants

  • Gunnar
  • Eugen
  • Stephen

Apologies


Minutes

  • Progress on white paper and other topics

TBC Minor notes taken by Gunnar


April 11 2019

Participants

  • Timo
  • Guru
  • Eugen
  • Stephen
  • Philippe
  • Violin
  • Gunnar

Apologies


Minutes

  • Discussing white paper, in particular GPU sharing chapter and some others
  • Update on GSoC...

More notes pending


April 04 2019

Participants

  • Kimmo
  • Eugen
  • Stephen
  • Harsha
  • Gunnar

Apologies

  • Özgür

Gunnar taking minutes

...Going through the input from Qt Company to the Whitepaper (remote objects).

  • Gunnar: What do you think, is it understandable also for less experienced?
    (... some brief explanations about how Qt & QML works, how properties will cause QML expressions to be recalculated) 
  • Eugen: OK to expect some prior knowledge, especially if we provide reference links
  • Kimmo: I can put in some reference links to official docs.
  • Harsha: We should highlight .rep file is shared between Cluster and IVI in the example.
  • Kimmo: Yes, it's like an IDL file

GSoC

  • Gunnar: Note some ideas around RAMSES.  Note also the published "Helsinki demo", i.e. the rotating 3D graphics navigation demo you may have seen. 
    Code is being published as we speak at:  ramses-citymodel-demo
  • Eugen: Confirming, we can't do this from ADIT side
  • Stephen: There is too little time to figure this out now, before the student application deadline.
  • Kimmo:  Not planned on Qt side.

    Notes from discussion on current situation for graphics sharing technologies

  • Gunnar: QNX - still need either Tier 1s or QNX company (have had discussions but it's not leading to concrete engagement). 
  • Eugen: Understandable because QNX business is the kernel itself.  It might not be in the business of driving graphics sharing standards and implementations (as a product).
  • Gunnar: Yes, so it becomes a Tier-1 / other implementor challenge to do the implementation.
  • Gunnar: Qt is cross-platform (incl QNX) but for a situation where one side is non-Qt → What is the status of QtWayland + Waltham?  
  • Kimmo: No new information
  • Eugen: ADIT update - Working on implementing.  Discussing Waltham/Weston reworking architecture.  This should make it easier to run Waltham backend in compositor.  Examples likely to be implemented on AGL.
  • Stephen: Any effect on Android?
  • Eugen: Not much effect, still following standards Wayland/Waltham protocol, so it should be the same effort to integrate it on Android side.


AMM Workshop ideas

  • Android Wayland introduction
  • Android to QNX graphics frameworks?
    • Situation of graphics exchange standards between different systems (matrix format?)
    • Share our view, "outcome of the project".
    • Conclusions, use cases and solutions (tie on to Eugen's talk)
  • Followup discussion on (each of the) Return-of-experience talks
  • Case-studies (repetition, backup topic)


March 28 2019

Participants

  • Gunnar
  • Eugen
  • Stephen
  • Kimmo
  • Philippe
  • Timo
  • Violin
  • Özgür (for a short time?)


Minutes

  • Walkthrough of whitepaper
  • Discussion on GPU sharing part
  • Stephen: It's mentioned in the Display Sharing tech brief, but not in a lot of detail.
  • Discussion of Wayland on Android
  • Violin:  I have read an article about it.  I can share the link.
  • (Özgür:)  That would be interesting to read.   Discussion of several challenges in implementing this on Android.

Transcribing some of the more detailed minutes, might come later.



March 14 & March 21, 2019

  meetings were skipped due to external conference (ERCC) and board meeting.


March 07 2019

Participants

  • Eugen
  • Harsha
  • Philippe
  • Stephen
  • Kimmo
  • Gunnar

Minutes

  1. Discussion about the white paper

            @Eugen to add use-cases as preparation for the AMM talk

             Harhsa: added use-cases for each sharing technology

             Gunnar: ok, need to look for a good examples

             Harsha: better name for section "Consequences":

             all: proposals: "Design considerations", "Design constrains", "Key factors", "Key considerations points"

             Steve: Design considerations is good, neutral and don't require to be complete list.

             all agreed on "Design considerations" for now

Gunnar/Harsha/Eugen: use-cases should be automotive related, the technology implemented in vbox or vmware still can be interesting and useful to mention/describe

Describe the graphics system in general and graphical composition in the white paper


2. Virtual display tech brief

Eugen: considering changing the diagram of waltham example to power point format to fit the same style as AllGo diagram

Gunnar: don't see it as critical, AllGo one looks a bit better.

Eugen: will rework the diagram and publish a updated version for review


3. Display sharing tech brief

Steve: Display sharing tech brief is published

Gunnar: Good we should mentions this at the working page

All: looked to the Display Sharing tech brief

Gunnar: 2 pages! very good

Overall good impression from the tech brief


4. Genivi AMM March 14-16

Gunnar/Kimmo: general discussion on the AMM presentation, Gunnar discussing with AllGo about the AMM attendance

also Qt company should have a talk to present their approaches

Technical session at AMM: we need to have a right discussion partners, otherwise difficult to have a discussion with audience

Genivi doesn't not have a goal to be a place for the OSS maintained to meet it is more the platform for Automotive industry to discuss the challenges and approached to solve them.

Discussion about the communication challenges with standard network protocols, interesting area to investigate: how to create reliable communication for graphics sharing and how to address this issue if reliable communication is not available

AI for everyone:

    brainstorm on the topics for the technical discussions session at AMM

AMM attendance

  • Kimmo
  • Gunnar
  • Stephen
  • Eugen
  • not Harsha


February 28

Participants

  • Pushpavati Patil
  • Eugen
  • Philippe
  • Gunnar
  • Stephen
  • Violin
  • Guru
  • Harsha

Apologies

  • Magesh Margabandu


Minutes

Reviewing input to White Paper

Harsha has written, in three installments, introduction, and several chapter introductions.

Gunnar: The structure is there to now get the rest of you to fill in missing content!!

→ We agreed to do editing directly on the draft wiki page, and then review diffs.  

...Everyone is free to go in and make edits, large or small.

Gunnar: It's time for others to step up and work on this!

Eugen: When the whitepaper is done I can focus on writing some of this.


Reviewing Virtual Display tech brief

No update since last week.
Eugen: I plan to update the word document with current content, make a few adjustments.  Maybe redraw the picture also.


GSoC:

Violin: How detailed should the project description be?

Gunnar: More detailed than now, but the exact timeline (for the project duration during summer) can be negotiated a bit with the interested student.
At least we must write a bit more, the concrete goal for the project, i.e. what do "we" (student) intend to develop.  Choice of programming language / environment, and so on.
... note that students will search on keywords such as "graphics" or "c++"
...I will take care of copying info into formal "web forms" that Google might require.  Just put the information on the Wiki page for now.
...The link to this page is already public as part of GSoC.  We will fill in the google forms, but this is our official description page!
...later on mentors will be registered with Google so information on what needs to be done will be sent also directly.


AMM

Gunnar:  Looking for a presenter of the GSHA project introduction/readout.

Philippe:  Also a 3-company view on...: "Graphics sharing, real-life experience".  We're thinking Qt, and two more companies, for example AllGo and ... {Renesas? Harman? ...}. 
(ADIT is likely to already have a full speaking slot for Eugen's talk)

Start confirming participation:  Harsha: No (personal reasons, no travel).   Eugen: Yes (speaker).  Stephen: Yes.  Violin: will ask, ...  we did not ask the rest - return to this.

Philippe:  Is there interest and opportunity to do RAMSES on Android - can we present this?

Violin: I don't think we're ready to do that in time to have something demoable for AMM.

Gunnar: That should not prevent anyone else from trying, using the published code!  (But understood the message - BMW do not have time to support this)
... note that it is not uncommon that companies do things internally... and then just show at the (AMM) showcase without warning.
...in other words it is not always possible to know who uses open source code and for what...

Gunnar/Philippe:  Note ongoing preparation for Android SIG.  Android integration is going to be a future focus area for GENIVI Alliance.



February 21 2019

Minutes pending




February 14 2019

Participants

  • Philippe (GENIVI)
  • Timo (Qt Company)
  • Magesh (AllGo)
  • Stephen Lawrence (Renesas)
  • Violin Yanev (BMW)
  • Vivek Galatage (Visteon)
  • Gunnar (GENIVI)
  • Eugen (ADIT)
  • Harsha, joined later

Minutes

Virtual Display Tech Brief

  • Discussion around making the brief shorter
  • AI:
    • Eugen will move the draft wiki text into the brief word template.
    • Magesh will edit in effort to reduce length.
    • Gunnar to upload  logo received via email.

Display Sharing

  • Final review feedback from Gunnar
    • Comment about not needing to invent protocols to pass gfxs between OS could be reworded as no need to have protocols to pass gfxs.
    • Whilst could be replaced with while.
  • Steve: What's the completion process?
    • Gunnar: Pass to marketing. They may make a second pass as it.
    • AI: Steve: OK I will do a final review before asking you to pass it to marketing.
  • Gunnar: Do we have approval? Who has done a final read through?
    • Timo: approve. We took similar approach in Neptune demo.
    • Eugen, Harsha approve.

GSoC

  • Gunnar: Do we have any proposals for GSoC projects? Using RAMSES perhaps?
  • RAMSES
    • Steve: PR for Helsinki demo could be basis for something. Is it self contained?
    • Violin: Yes, but the map data is missing. We are considering how to provide that.
    • Gunnar and Violin discuss the GSoC process.
    • Two proposals to be created.
    • AI: Violin: I can consider some proposals. Need to discuss with mgt about support required.
  • Gunnar: Any other ideas in Graphics? How about focused "slim" compositor for specific cases?
    • Gunnar shows tiled wayland compositor (swaywm.org), highlighting the existence of a reusable library (wlroots) for wayland primitives for compositors to be built on top. Example of things outside of Weston... (Compare the existence of QtWayland)
    • Discussion between Eugen and Violin about Wayland/Weston which suggests possibility for new project for the group. Details in seperate Waltham section)
    • Gunnar: Any other ideas , write or send them.

Waltham/Wayland/Weston extensions

  • Eugen: (The full) Waltham architecture in Weston and (additional) protocol/APIs needs to be redefined before upstreaming, it should provide a base to implement concrete Waltham Server's and Client's. This requires modification (addition?) of the Waltham core protocol and for this we need to collect some idea's.

  • Violin: We also looking for some generic independent way to share video streaming data and could help/participate the work

  • Eugen: very much appreciated! You input/review will be very welcome

  • AI(all): Define more concretely possible collaboration project

Qt, RAMSES

  • Violin: No response to my email asking for other interested parties and on Qt side no resource to do work. So I am considering commercial project to implement as BMW needs the feature. Would hope to OSS result.
  • Gunnar: Shame the industry is still not collaborating on needed technology in all areas. (Meaning, no answers is not always indicating no interest in the tchnology as such or that it won’t be used once created. We have seen this in many areas, for example widespread Franca interest and used behind closed doors)

AMM

  • Gunnar & Philippe mentions call for papers, and deadline. Draft schedule to be published within a week or so.
  • Eugen: Considering one on the GSHA white paper, i.e. the trade offs on choosing between technologies for gfx sharing.
  • Timo: Can’t make it for tomorrow deadline but can discuss and hopefully come back next week.


February 07 2019

Participants

  • Philippe (GENIVI)
  • Timo (Qt Company)
  • Magesh (AllGo)
  • Eugen (ADIT)
  • Harsha (ADIT)
  • Stephen Lawrence (Renesas)
  • Violin Yanev (BMW)
  • Vivek Galatage (Visteon)
  • Gunnar (GENIVI)

Apologies:

  • Eugen
  • Harsha


Minutes

Intro, housekeeping

  • Gunnar reminded everyone to please be on time!
    • (warning) ... of course you are still welcome to join late, better than not joining at all.  But when there is almost no one on time it's very hard to start the meeting properly!

  • Gunnar: Some apologies for today, see above.  Harsha reiterated the intention to write something for whitepaper.
  • Minute taking (tracked here): No one assigned today (Gunnar will write minutes after the meeting).   Stephen volunteering for next week.
  • Welcoming Vivek (joined later) to the meeting.  → AI: Vivek to read through the tech briefs with "fresh eyes"

Display Sharing Tech Brief

  • Display Sharing Tech Brief is now 95% done, on the right template.
  • Stephen: I have a few final tweaks towards the end
  • Kimmo: I read it briefly and found no errors.  I'll read the final copy more carefully.
  • Özgür: No input still. 
    • Gunnar:  I think that would be a useful check if you read it since you had some thoughts on the original text.   Does it now make sense, etc?
    • AI (Özgür): Read and report feedback before next meeting.

Virtual Display Brief

  • Magesh has produced an initial Android case-study text
  • We discussed the ability to transform it a bit from "this is possible on Android" to "this is what we actually did in the AllGo system".  This makes it more of a case-study, it gives visibility to the author (AllGo) and it 
  • Gunnar: At the same time, the text needs to also be shortened to fit in tech-brief format
  • Magesh: Should I add a picture?
  • Gunnar: Yes that is always useful, especially schematic/design picture.  There's a schematic already for the Weston part (see proposed PDF).  It will be hard to fit with two pictures but I think we could make this a slightly longer tech brief (e.g. 3 pages?) if necessary.

 RAMSES/Qt project(s)

  • Violin: We prepared a project introduction (Qt+RAMSES) as discussed.  I sent it in my internal channels first.  I think we should have a bit more technical plan before sending to the larger lists.
  • Gunnar: OK, keep working on it and when ready it can go out to wider mailing lists. 

GSoC

  • Gunnar:  GENIVI has applied for GSoC again.   I feel a RAMSES based project is one interesting idea. (see project proposals)
  • Violin:  I agree.  For example, I have been working on a game that uses RAMSES.  It could be a fun student project.

AMM

  • Reminder - send your proposal for presentations at the All Member Meeting 2019 in Munich.  Just email Philippe Robin  (or Gunnar Andersson).
  • We are opening up for the ability to do full technical presentations (with connection to domain-interaction) in the main track.  
  • It is a great opportunity so please propose, and/or talk to colleagues that may want to present. 



January 31 2019

Participants

  • Eugen (ADIT)
  • Timo (Qt Company)
  • Violin (BMW)
  • Harsha (Bosch)
  • Özgür (Mentor)
  • Gururaja (Bosch)
  • Stephen (Renesas)
  • Gunnar (GENIVI)


Minutes

Gunnar: We should start doing round-robin for minute taking.  It works well in other projects.
See page:  Minute taking schedule

Discussing Display Sharing tech brief.

Content is still basically done. Stephen to transfer it to Word document template.

... we went through the Q&A section.

Stephen: Most of this is addressed. Some we concluded is detailed discussion appropriate for Whitepaper.
Gunnar: OK, make note that we have lots of WP content in this Q&A section

Özgür: Can we have dynamic assignment of the overlays (layers) or is it always static?
...Assume you have a linux system you use sometimes only some of the layers are used. Can the other OS use the free ones.   I admit it is just optimization....

Gunnar: Interesting but very often you need to have static setup in automotive, design for worst case. Especially for things like HMIs. So in this case it means dedicated layers.

Eugen: Normally one or two of the hardware planes are abstracted to look like a display for the OS, by a Virtual Display Manager. DM (in Hypervisor) handles the hardware layers somehow.
...As of today it is normally all different and proprietary.

Gunnar: The Hypervisor standardization we're discussing in the other group would be the place to address this... (if it is feasible or not to create a common interface / abstraction of the hardware planes and virtual display manager)

Eugen: The answer as of now is that layer assignment is normally decided at
boot time, and from there on it can't be changed.


Discussing Virtual Display tech brief

Gunnar (earlier) : I have contact with Magesh now. Don't know concrete plan for Android paragraph yet however.

Eugen has added a diagram and moved everything to the template.  Ready to review all (except Android paragraph)

AI (all): Review and comment on this draft


Qt/RAMSES combination

Violin: I spoke with Miao. We plan another call to decide if we can really
make this a project. As before, we're interested

Gunnar: I think we/you need to present the idea widely. Prepare an email to genivi-projects, Qt developer list, AGL, any other graphics activity?

AI(?): Ask Miao about which Qt list might be appropriate or not

Gunnar: Reminder, what are we trying to do?

Violin: Check the Wiki page that describes the ideas.
Özgur: Is there some specific priority of these two?

Violin: The tool (Qt 3D studio) ability is #1 priority for us. This is something we need and might consider starting a development project.
...But the other (RAMSES integrated into Qt classes and QML) is interesting too.

Discussion about second idea and how it will work (not captured)

[ Özgür/Violin could amend the minutes here ]

AI(Violin): Start drafting email to present project


White paper

Harsha: I will have something written until next week.
Gunnar: We found interesting discussion topics in Q&A.  These need to be written out.

Violin: ...what about benchmarks and performance comparison?

Gunnar: I put benchmarks at the top of the group backlog. Need you all to provide input.
... discussion if valid comparisons can be made, do we need a reference
hardware?

... general conclusion that it's good to provide data, and that we just
make general recommendations on which technology to use when.

Gunnar: I think providing (benchmark) data and letting the reader interpret
and draw conclusions, this provides value in itself.



January 24 2019

No minutes.



January 17 2019

Stephen opened the call.  Limited participation, some discussion.



December 17, 2018

Participants: harsha, violin, eugen, stephen

Virtual display Tech Brief

Discussion of review comments.

Steve: I made some grammer changes. Please review. Given the space constraints I liked the description and how you managed to outline the sender side limitations, whilst explaining the flexbility of receiver side.

Harsha: A diagram may help the explanation.

Discussion of the Weston Remote receiver side.

Eugen: I am still waiting for input from AllGo on their case study

Display Sharing Tech Brief

Steve: I integrated some minor tweaks. Next step is to move to the word template and pushing towards a final draft. Having read Eugen's work on Virtual Display I may attempt to seperate the description and case study further. It's a current priority.

Harsha: re your comment reply you could describe a "hardware layer" as being a unit associated with the DU which can take multiple buffers and combine them.

Discussion of how to simply describe it without it just sounding like a DU or GPU buffer.

White Paper draft

Harsha: I was time limited due to work over xmas so have not progressed it much. Time still a challenge but load may be easing.

Eugen: We should try to reserve at least small time to move it forward each week.

Reviewed outline in wiki.

Harsha: I should be able to allocate time to move abstract forward this week

Eugen articulated approach to writing the "choosing a strategy" section and said he should be able to draft something.




December 6, 2018

Participatns: guru, harsha, violin, gunnar, philippe, stephen

RAMSES, and Qt

Violin: Interesting workshop last week between BMW and Qt including Qt CTO
BMW now considering to spend some resources to help implement things related to Qt. TBC.

Violin: We discussed Qt quick which a kind of 2.5D rendering framework. I think QML is quite nice.
...Qt 3d studio seems also interesting (it is based on a contribution from nvdia)
...One question is to investigate how to export ramses asset from qt 3d studio

Violin: this was a fairly technical discussion, no planning & resourcing done yet
...the GSHA is a good forum to discuss such opportunities
...how to set up a minimalistic prototype but working (should be 2-3 weeks for 2-3 people understanding qt and ramses)

Discussion on the utilization of Qt artefacts

Contact with a guy from APTIV investigating ramses, who asked for Ramses studio

Violin mentioned that CES Ramses demo is "awesome"

Stephen L: Did you discuss Qt implementation in Android ?
Violin: No, not much. It seems android is a different ecosystem in the Qt company
Stephen L: and QNX ?
Violin: Ramses is OS agnostic
Gunnar: And QNX, Integrity... etc. they are all considered fully supported platforms for Qt as far as I know.  (So what is the specific question?) 
...We can ask Qt representatives if any particular feature is not supported on some platform, but otherwise the default should be that all of Qt5 is supported?  

Gunnar: Qt on Android exists but from what I can tell not greatly adopted (so that makes it more of a separate/special thing). Of course it was demonstrated on phone/tablet Android more than Automotive.  I have
not heard of any uptake on Android Automotive but I would be interested to know.

Stephen L: I meant that both android and qnx are in need of standardisation around graphics sharing.
...they need the api remoting topic such as ramses


*Display sharing tech brief*

- Guru will review it this week
- Harsha will look at it next week

*Virtual display*

Eugen: Very small draft so far.  I am too busy...
Gunnar: It's a start.  I'd like to get input also from Magesh (Allgo) with Android focus.




November 29 2018

participants:

Eugen, Harsha, S ergey (left at 11am CET), Stephen L, Gunnar, Philippe

Minutes

TBD

November 22 2018

participants:

  • Eugen
  • Stephen
  • Evgeniy
  • Guru
  • Serguey
  • Timo
  • Kimmo
  • Violin
  • Ozgur
  • Gunnar
  • Philippe

Minutes

Gunnar: Welcome Ozgur (and welcome back to GENIVI projects)

Ozgur: Intro. I have worked on RAMSES implementation during the last
time.  Before that android, drm-backend, wayland-related stuff, etc.

Gunnar: ...Going through to freshen up backlog

Gunnar: OpenCL - let us know if you're interested in this proposal from Artem


Benchmark/performance

Gunnar: Benchmarks? Sergey?

Sergey: We could look at the solutions we've shown...The bottleneck was the ethernet connection between anroid tablets and qnx. We're looking at HV transport now, which should be more performant,
zero-copy So we'd rather focus on that now than put more work into measuring the old.

Gunnar: I would say it's useful with even just a rough confirmation of surface sizes and bandwidth and that reality matches theoretical expectations?

Gunnar: Eugen?

Ozgur: Video encoding is optimized for temporal changes (not optimized for

HMI (sharp corners) -- might create artifacts? How did that work for you?

Eugen: We visually compared, our use case was an Android map. We could not
visually detect any difference at all.

Ozgur: I'd be interested also in some numeric benchmarks.

Eugen: Guidelines what you _should_ measure it useful.

Gunnar: Agree, describe the methods what you should be doing.

Ozgur: Good to have a checklist. Which parameters, constraints should we
look at.

Gunnar: Lots of interest here. Let's work on setting up a workstream/topic
for that. I would ask everyone (e.g. ADIT, Qt, BMW/Mentor/RAMSES and
Harman) to report what benchmark/performance info you can present. If
everyone commmits to provide something then this topic will be successful.

AI(Qt, ADIT, Mentor/BMW, Harman): Consider what info you can bring to a
collection of benchmark/performance knowledge base, and report next week.


Weston-remote

Gunnar: Weston-remote? Update on upstream project status?

?: I think it's coming along.

Gunnar: This is minor topic, mostly sharing with the group the current status and outlook
Stephen: I could ask again internally about status.
AI(Stephen): Ask internally


Android-API mapping topic

Ozgur: What is this

Gunnar: .. Mapping and applicability of wayland/waltham vs surfaceflinger...  Considering if a shared standard can come up here rather than the current ad-hoc solutions in various production projects.

Eugen: We're evaluating this exact topic, but it is taking some time. ... Not sure what will happen with multiple applications. Input events is also a challenge.

Ozgur: Including the HALs, different etc. I would be interested in this.

Ozgur: HALs are mobile-oriented and adaption to automotive is needed, e.g. early-video is a challenge. Such automotitve use cases should be the basis.

Gunnar: I think we need more input before we are at the point of what you ask for. (e.g. needed modification of HALs). Can you provide more input?


SDL

Gunnar: SDL protocol... GENIVI had some recent contact with SDL project again. Common streaming interface. Some of this might be video but I

would conclude that content-protection / DRM is a big challenge for some video apps (Netflix type).
Ozgur: I would like to also have a look at this at any case. If I find time I could look at it and present.


Qt and RAMSES topics?

Gunnar; Qt & RAMSES collaboration?
Timo: Meeting is next week. Let's come back to it.


Tech Briefs

Gunnar: Progress on tech briefs?
Eugen: I promise for next week some input on Virtual Display.

Stephen: Display Sharing. I have written the first draft.

AI(all): Review the Display Sharing tech brief draft ^^^.


AOB

Timo, Kimmo : Apologies for next thursday. Travelling.

Adjourned.




November 15 2018

Participants:

  • Eugen
  • Stephen
  • Guru
  • Evgeniy Prikazchikov (new, Harman)
  • Gunnar A

Apologies:

  • Philippe


Minutes

Presentation of Evgeniy, new participant from Harman.

Evgeniy: I am catching up on the project so I will be quiet for now.


Whitepaper kick-off


GA: Here is the template.  Here's what I'm thinking so far on abstract and chapters.   Which chapters should we have here?

Eugen: Use-case chapter, which describes which technology to use for each. Go through a few and describe which solution fits best.  Maybe not typical for a pure white paper but I think it would work here.

GA:  Yes a straight advantage/disadvantage (characteristics) comparison chapter followed by use-case examples.   The WP can also go a bit deeper into case studies and examples.

GA: Should we include all case studies?  There were _many_ presented in Bangalore.   In my opinion (as always) is if companies contribute to the text then they can also influence it...

Eugen: How to collaborate?  Wiki page?

GA: OK, adding a Wiki page. → GSHA White Paper.  The document template is also attached there.  We can sync the text between wiki and document as it progresses.


Virtual Display (TB)

Eugen: I have not had time to progress it this week.  I think we can start with the definition on GSHA: Virtual Display anyway.

GA (later): You can write the basics here.  I think some case-study details from AllGo demo might fit in the tech brief after the general description.  We should ask Magesh about that.


Display Sharing: (TB)


Stephen: The good news:  I cleared out other things that prevented me from starting (wink).  I should have friday free to work on this now.

Stephen:  What should I describe and how deep should I go?  
Eugen ... also thinking about this.  Consider that the demo is a mix of virt-display and display-sharing [(...and GPU sharing)]

GA: Let's try to avoid GPU sharing examples in Display sharing tech brief.  I mean if D.S. requires virtualization and it also requires GPU Sharing, then the category makes little sense by itself.  
Are there any real systems that actually mimic our description of Display Sharing?

Eugen: There is a case to be made for systems that do Display sharing but don't need GPU sharing.   A system might be a carrier for camera feed for example, going mostly directly to the display hardware.


GA: ... right so it's basically data with minimal processing which is done by a CPU instead of GPU.

Eugen: Yes

GA: Good point,  so this fits also systems that use a hypervisor but don't necessarily do GPU sharing.

Stephen:  GPU Sharing Tech Brief.  Will "HV team" work on this?  Not started yet?

GA:  Right, not started yet.  Regardless who writes it we can conclude that we have that as a separate category.

... a bit more discussion on content

GA: FYI, I have some hooks out for RAMSES and Qt people for various topics.  Let's see if there is something new ready to discuss in the next few week.




November 8 2018

Participants:

  • Eugen
  • Harsha
  • Miao (or a colleague?)
  • Horst
  • Artem
  • Gunnar
  • Bernhard

- Gunnar: This will be a short meeting because of other engagements.
... I want to check on development of our documents
- Eugen: I started the Virtual Display writing a bit. It is not that long, not that much to say. I started with the existing definition on the Wiki page.
- ... I can also add to Display Sharing...
- Gunnar: Good, but Stephen was going to write the first part I think
- Gunnar: I indicated on the web page that we would not hold a meeting today, and then started it anyway (so understandably Stephen and some other people are away)

... a bit more discussion, planning for next HV meeting.

- Horst: I have been away from the meetings a while and need to catch up on what has been done.
- Gunnar: Yes, we're mostly working on the documents and the GPU Sharing topic is heavily discussed in combination with the HV team. If you want, just join either time slot - agendas will be on the project list as usual.




November 1 2018 (public holiday in some countries)

Participants:

  • Timo
  • Gunnar
  • Eugen
  • Violin
  • Stephen

Minutes

Gunnar: Today's a public holiday in some places.  As I wrote in the invite, let's have a short discussion among those that could join anyway.

Qt:

Timo: Me and Kimmo are planning to join alternating each week,
Gunnar: Happy to schedule a more in depth discussioin on Qt approaches
Timo: Alternatives for graphics sharing approaches with Qt are being discussed internally.
Gunnar: OK, let's keep in touch, and let's not forget the cross-technology standards that we also need for interoperability.

Virtual Display Tech Brief

Eugen: Not sure that this one is needed
Gunnar: Why not try a one-pager?  It's only 4 paragraphs.   We could write a simple proposal based on content we already have and then look at the proposal, to see if we think it provides value.

Eugen: OK, we could try. Include a reference to Surface & display sharing.
Gunnar: I see it as possible, with perhaps two main examples: One being the ongoing work in Weston - "weston remote" and the other perhaps AllGo approach.
Gunnar: Who can write a weston-remote introduction?
Eugen: I volunteer to do that --> AI(Eugen)

Display Sharing Tech Brief

Timo: Qt Cockpit demo is a good example of this. It's mostly identical to "Renesas canvas demo" but Qt focused.

Gunnar: Sure, I'd be happy to include a Qt "case study" in one of the publications, we have not done it so far. (But we looked at Renesas for this first).
AI(Stephen): Write a first draft.
Stephen: A few other things in the pipe but I could start on this next week.

API Remoting Tech Brief

Looking at new PDF proposal.  It's between 2-3 pages. without graphics.  Add graphics (3 pages) or try to cut it down further (2 pages)?

Gunnar: Done further edits... It's getting there... I only got review and input from Eugen & Violin.  Here is the latest PDF.

Gunnar: Looks like I keep the action of finalizing this...

Gunnar: Will we co-publish this together with BMW? (Logo)
Violin: I'll look into that.

Timo/Violin: Discussion on sharing the implementation of some integration done on Integrity (RAMSES).
Violin: Yes. I'm still working on figuring out the license situation for that code.

Whitepaper

Gunnar: Once I set up a structure for this it should be easier to start filling it with content.




October 25 2018

  • Magesh
  • Miao
  • Kimmo
  • Timo
  • Philippe
  • Gunnar
  • Stephen (last part)

Apologies

  • Eugen
  • ...

Minutes


... not all discussion was captured ...

Gunnar: We have been looking at the mapping (difference) between Linux/Wayland style APIs and Surfaceflinger.  Is it the right approach?  There's also of course a more direct approach to just try to implement it.

Magesh: I don't think you need to deal with SurfaceFlinger that much. The Virtual display concept should be enough to transfer android (apps) to other
systems. But it depends on use-cases. We should analyze them.

Gunnar: Whatever the solution we're looking for widely applicable standards here. Both sides (different operating systems) need to agree. We had
aimed to understand if Wayland/Waltham would be a basis for that standard. That is for Surface Sharing only of course.
... I try to look at it simply. Each of the 5 graphics sharing methods probably need a (cross-platform) standard protocol?

The team noted that Android has implemented screen-sharing from phone, and (recently?) added this to automotive Android. So the technology is there for graphics transfer of course.

Magesh: Yes generally the encoding methods for the graphics/video are of course there, but not all details fully defined in terms of [the next level] protocol for sharing between systems.

What parts will go into AOSP? E.g. Android Auto (we mean the "projection mode" feature) is not in AOSP it seems.

Gunnar; Nonetheless Android Embedded for Automotive (whatever the most recent name is) seems likely to merge some newly developed technologies into the common codebase of AOSP, don't you think?

Magesh: Yes sure - AOSP code base can already now be built in a few different variations, but from the common code base.

Magesh: OEMs and suppliers don't want to add new proprietary methods to Android.

Gunnar: No doubt, I agree but I would note  1. We have occasional contact with Google Auto team on this topic and last time we spoke they were supportive of finding those cross-platform methods.    2. Also, if a project team like this one suggests certain way, and proves that it works... it is the first step to start influencing it to become the standard way (as opposed to being proprietary).

... not all discussion was captured ...



October 18 2018


Participants:

  • Roman
  • Kimmo
  • Timo
  • Stephen
  • Harsha
  • Gunnar


Gunnar: I think I scared some people away by reminding of the tech brief review?

Roman: Sergey is on vacation

Gunnar: OK, Feedback from tech summit?

Timo: RAMSES tutorial was good.  We now know more about what this is.  Still evaluating what it means for Qt however.

Gunnar: Let's go through what we discussed on working session.
...Since I was leading the discussion and giving microphones to audience etc, I was not fully concentrating on the content.  (Sorry if I missed some details in your presentations)

Safe Renderer

Interesting for further study?
Harsha: Yes I think so.

Gunnar: OK, let's reschedule a full discussion on it, including Luxoft who still want the chance to present & participate
Timo: We can bring in our expert in that area. 

SL: I was asking last week how important it is to put this implementation in a safe execution mode or on a safety OS / safety CPU

Gunnar: Going back to Qt presentation they described in general terms that (in addition to certified implementation) the safe rendering should be in some kind of separate partition using a "safe OS" and/or hypervisor.

SL: Sure.  Maybe interesting to discuss also partitioning into separate core.  E.g. a R7 core might be appropriate.


Misc stuff...

SL: Also on display manager side - what are the requirements.  Thinking of the ability to define a small display that one OS [e.g. "non-safe" OS] sees as just a display, but the result is offset and placed in the final composition by the safe rendering.  Is that a typical and established feature or something unique to certain OSes?

Gunnar: Availability of this feature in different OSes/environments.  Yes, but also perhaps ask if it is a required / needed / popular from requirement point of view?

Harsha: I think it is a feature that should feed into the VIRTIO 2D GPU sharing standard.   [Is it a requirement, and is it supported?]

Harsha: For VIRTIO GPU 3D I asked about batching of GPU commmands...  Is there a followup planned for GPU-sharing?

Gunnar: Yes, we should have a dedicated session with the Graphics & HV teams together.  I already plan it because there was  some disagreement on the feasibility of API standards in 3D that I would like to resolve.


Waltham in practice

Harsha: For my own presentation I asked if the "multiple surfaces" use case and solution.  Is it needed?

Gunnar: I think it describes one solution to multiple surfaces per app.   It is because the surface-transfer (gstreamer pipe) assumes one surface only.  So instantiating multiple virtual displays is one easy straight forward solution to it. 

Gunnar: In for example Android we have seen the Virtual Display concept as central. So there seems to be an easy mapping there to use this approach where each app essentially sees its own (virtual) display.

Harsha: The solution(s) do not scale.   Because each gstreamer pipeline is separate and is often mapped to hardware encoder/decoder as well, which is a limited resource.

Gunnar: But another alternative would be to have a more capable protocol that could multiplex transfer of multiple surfaces. That would be a "proper" remoting of Wayland.  Wayland support- multiple apps, with multiple surfaces, on one display, right?

SL: Weston-remote, yet another solution approach.  It's related.


Going through Backlog on main page.

Request for more hard data and benchmarks.

Gunnar: Harman surface sharing?
Roman: Wait for Sergey to be back

Gunnar: Waltham in practice?
Harsha: Yes we get some measurements out of our experiments.  Still work in progress however.  I'll work on collecting and sharing data that we collect.  We measure, and we use systemtap also to evaluate.

Gunnar: I will add weston-remote deep dive to the backlog (as requested by SL).

Gunnar: The documented requests for Android & QNX examples have been well met now I think, with the case studies.  Always room for more but that's good anyway!




October 11 2018

Normal meeting cancelled due to tech summit.  A working session was held with presentations and discussion.



October 4 2018

Participants:

  • Sergey
  • Harsha
  • Guru
  • Magesh
  • Stephen
  • Gunnar
  • Artem Mygaiev (EPAM)

Apologies:

  • Eugen (vacation)


Minutes

Final tech summit preparation.  

API Remoting

Gunnar: I didn't receive feedback as expected.
Stephen: Sorry, dropped off my radar
Artem: I could read it and give feedback.

No other feedback from other participants.

Artem: That reminds me of work we are doing on remoting API for OpenCL.   OpenCL is for (non-graphical) calculation on GPUs, so at least it is related to GPUs.
Gunnar: Thanks, will keep it in mind - it is not strictly graphics as noted, but some relationship (and we might learn something from the API Remoting approach)

Adjourned ~11.15



September 27 2018

Tech summit prep + API Remoting brief

No minutes posted yet



September 20 2018

Tech summit prep + API Remoting brief

No minutes posted yet



September 13 2018

Tech summit prep + 

No minutes posted yet

----



September 6 2018

Participants

  • Violin & Bernhard (same line)
  • Eugen
  • Subramanian
  • Harsha 
  • Kimmo
  • Magesh
  • Gunnar
  • Roman
  • Sergey

Apologies

  • Philippe R


Minutes

  • Presented overview of subtopics on each presentation slot - page updated
  • Discussed working session topics - page updated
  • Violin: We could shorten the RAMSES tutorial if needed
  • Gunnar: The schedule is fixed now - concentrate on doing a full day workshop pas planned.
  • AI(all): Prepare "your" topic (should be obvious) not only for presentation but also for working session (detailed tech info & appropriate question/problem/challenge to discuss)

  • Magesh repeats overview of AllGo multi-display system.
  • ...how to transfer audio?  (Especially while still meeting Google's requirements (CTS)) 
  • Gunnar: This is a classic question.  Using AvB and/or a streaming network protocol (with added latency).  Even if this is far from new, I still think the industry would benefit from standards work here.
  • Magesh:  Doing work now on multi-user features.
  • ...Input handling.  We considered VNC-like protocol?  We think HID works better now - even understood by Android.



August 30 2018 Project Meeting

Participants:

  • Eugen Friedrich (ADIT)
  • Kimmo Ollila (The Qt Company)
  • Miao Luo (The Qt Company)
  • Subramanian Dhandapani (Alpine)
  • Sergey (Harman)
  • Magesh Margabandu (AllGo)
  • Harsha Mallikarjun (ADIT)
  • Stephen Lawrence (Renesas)
  • Gunnar (GENIVI)
  • Philippe (GENIVI)

Minutes

  • discussion on the organization of the working session
  • need to identify a list of topics we need to dig into more deeply
  • Phil: proposes to move the GSHA status report on Day 1 morning - DIRO overview session and free timeslots inDay 1 afternoon GSHA session for technologies overview, this would leave more time for deep-dive discussion on Day 2 GSHA working session


August 23 2018 Project Meeting

Participants:

  • Eugen Friedrich
  • Marco Residori
  • Kimmo Ollila
  • Miao Luo
  • Subramanian Dhandapani
  • Timo Aarnipuro
  • Xiaoyang Guo
  • Gunnar (GENIVI)
  • Harsha Mallikarjun (ADIT)
  • Roman Leykin

Apologies:

  • Philippe (GENIVI)


Minutes

  • New participant from Neusoft: Xiaoyang Guo - no microphone, said hello via Chat.
  • Kimmo Ollila presenting himself
  • Harsha from ADIT based in India presenting himself:
    • - 8 years experience in IVI. Lately working on graphics video streaming, wayland and weston

Tech Summit planning

  • Harsha will be joined by at least one colleague (working on graphics that is) at the Tech Summit.
  • Kimmo: I sent presentation topic ideas to Gunnar via email
  • Gunnar - let me go through the planned setup for the Graphics Tracks (except RAMSES which is in parallel).
  • ... first day session is project overview.  Each category presented, then with case-studies for each category.  Opportunity for quick introduction and a picture/video.  Invite to visit and ask questions at the demo tables (for those that have demo table).
  • ... second day can go into more detail.  It is a working session.  Therefore - intro case-studies or subjects, with more technical focus, then time to discuss.  Aim for giving information but also some "open questions" and problems for discussion.
  • Gunnar: Interesting, QtWebAssembly - so it's using web assemblly specifically for graphics transfer? (OK, all of web could be considered "graphics", but you know what I mean).
  • Kimmo: Yes, there are some early ideas.
  • Gunnar: Early ideas are perfect for the working session, good for discussion.
  • --> AI(Gunnar): schedule also phone call discussion on this to prepare in the group. \[ I will put it on the Backlog list \]
  • Renesas - Stephen working on some topic, probably Canvas Demo as example of Display Sharing.
  • Subramanian: I have no update or presentation from Alpine, but I will attend the Tech Summit.
    (later) Gunnar: Will you bring colleagues?
    (later) Subramanian: Alpine has no operations office in India.
  • Gunnar: I need to follow up with AllGo on presentation/demo/other

Project Planning:

  • Updating the milestones schedule
  • Gunnar: API Remoting, should be ready well before October
  • Gunnar: Then follows Display Sharing Tech Brief. Can we use canvas demo as example?
  • Stephen: \[Yes, even if it includes many other things too.\]
  • Gunnar: Good, in any case tech brief is about the theoretical content and does not require a particular example necessarily.
  • Gunnar: Then follows Whitepaper - ready by end of year, challenging but we should do it!
  • Volunteers for white paper authoring/proofreading/etc?
    • - Eugen, Stephen, Gunnar
  • Gunnar: All others, you should consider to be a contributor. (Teaching is a great way to learn).

Implementations:

  • Waltham
  • - Eugen: Various plugin work etc. Not that well synced with Weston yet.  Work done in AGL. Expect an impl. to become part of later "AGL release"
  • ...not sure exactly how/when/where (generic) implementations will appear.
  • ...Upstream Weston cannot receive the changes now (change window for next version, V5, is closing soon). Maybe start discussion for next version following after that.
  • ...lots of things moving (in Weston)
  • ...concept of Virtual display needs changing in Weston
  • ...some of the APIs need to change.
  • ...The implementation of output and backend already available
  • ...clarification: The plugin API is OK, just some of the plugins that are implemented now are not meeting our requirements.

Wrapup

  • Miao: Note that 4 guys are coming from Qt Company now. We have colleague in India attending, and also myself.


August 16 2018 Project Meeting

Participants

  • Miao Luo (Qt Company)
  • Timo Aarnipuro (Qt Company)
  • Subramanian.Dhandapani (Alpine)
  • Sergey K (Harman)
  • Roman Leykin (Harman)
  • Bernhard Kißlinger (BMW)
  • Guru N (Bosch)
  • Horst Saier (Mentor)
  • Stephen Lawrence (Renesas)
  • Gunnar Andersson (GENIVI)

Apologies:

  • Magesh Margabandu (AllGo)
  • Eugen Friedrich (ADIT)
  • Philippe Robin (GENIVI)

Last Week / Recap

  • Walking through last meeting minutes. (Almost all participants are different in this and last meeting)
  • Informed new/returning participants about AllGo system and discussion.
    Basic ideas & content repeated (by Gunnar). Possibly we do a recap when Magesh can join again.

Tech Summit

  • Informed new/returning participants about tech summit --> http://genivievents.com
  • Quick discussion on planning (in Philippe's absence) ---> Bernhard - what's the plan so far on RAMSES?
  • Bernhard: Yes we're planning a hands-on. What do you (the group) think should be included?
  • Timo(?): How to port an existing OpenGL application to RAMSES
  • Lets' compare to Munich workshop
  • ...Good: Show & Tell, concept and then code examples
  • ...Improve: Participants wished for ability to do more coding on our own. But this is maybe not feasible in such short timel
  • Gunnar: But I suppose it will be introduction level, because of new participants
  • Bernhard: Yes.

     Harman participation:
  • Roman: We saw the request for trying Android and QNX, so we are developing sharing between an Android tablet and a QNX instrument cluster system, which uses Qt. It will show Navigation sharing. Time plan says it should be ready in time. Let's discuss how we can present it.
  • ...it is a Surface-Sharing tech demo (with only a bit of Shared-State)
  • ... how do we plan the showing of this?
  • Gunnar: I will follow up on this.

  • Stephen: Could we also consider some smaller/shorter presentations?  I can cover some topic, maybe some Weston/VirtualDisplay or other.
  • Gunnar: Splitting slots into more than one speaker is fine, as long as we keep a consistent topic.
  • Gunnar: Sounds good, let's find the appropriate time slot or partial slot


Qt Participation

  • Miao: We could maybe make a presentation on graphics sharing. We have some ideas but need to consider what we might present.
  • Gunnar: Yes. Normally what is being presented should first be processed by the project group - not unannounced/surprise stuff. So we're short on time then, let's get started on that.

Qt technologies

  • Gunnar: So I remember us discussing Waltham into QtCompositor, and the potential for Qt-specific technologies.
  • Miao: We're also interested in how to connect Qt and RAMSES...
  • Miao/Timo: We have the Remote Objects technology. It is basically a convenient solution for synchronizing data between objects. It fits well for implemning the Shared State idea.
  • Timo: The object could be a QImage, for example. Changes to the image would be propagated to the other side.
  • Gunnar: [Interesting...] so in that usage it becomes a kind of "graphics-sharing" automatically?
  • Timo: Yes (but the main intention is still just to distribute data/state)
  • Miao: Then there is the WebGL based technology (we talked about before)
  • ...It distributes OpenGL commands to the receiver which renders them.
  • ...They are rendered in a WebBrowser on the receiver, so it does not need any Qt technology. The receiver works anywhere.
  • Gunnar: The advantage of following standards. So any standards compliant browser will work.
  • Gunnar: There are known security issues with WebGL (on the general Internet), but I suppose if you are in control of who is sending you the graphics commandsd then it can be managed)
  • Miao/Timo: We also developed some convenience [for defining the graphics] of course, such as QtWebGL class
  • Gunnar: Remote objects - it makes me speculate: What if you use QML to define a scene. There are (graphical) Qt objects created in the background. Could these be remote objects? Could it be a way to synchronize graphics
    across the network? [ As opposed to sending actual QML over the network ]
  • Timo: ... not sure on those details
  • Miao: QtCreator has a config file. There is a preprocessing compiler involved in RemoteObjects
  • Timo: Yes, but the dynamic creation of objects is also possible...
  • Stephen: When was RO introduced?
  • Miao: in 5.10. It's tech preview until 5.12 release

  • Gunnar: And Wayland/Waltham?
  • Miao: I think we have done some PoC, but it might have been in a customer project so I don't know if it is accessible for the R&D / Qt open source.
  • ...We also have a general Multi-Display / Digital Cockpit demo. Using a hypervisor also.
  • Gunnar: References? Video? Source code?
  •   --- ->It is open source.
  • AI (Miao+Timo)  To provide references to documentation on RO and graphics-sharing related things , and graphics-sharing demos/proof of concepts.
  • AI: (Gunnar): Investigate Tech Summit Demo-Table potential for Qt.


Additional discussed topics

  • Quick check of API Remoting.  Bernhard has done some more update of RAMSES text
  • Reiterate focus on Implementations, in GDP and other, during Fall
  • AI: Gunnar to transform to Tech Brief format
  • Request (any): Provide a diagram/picture 
  • AI (all): Review the API Remoting text (main part – RAMSES part is optional)


Postponed:

Virtual Display definition.  Whitepaper kickoff.



August 9 2018 Project Meeting

Cancelled


August 2 2018 Project Meeting

Cancelled



July 26 2918 Project Meeting

  • Magesh Margabandu (AllGo)
  • Guru (Bosch)
  • Eugen (ADIT)
  • Gunnar (GENIVI)
  • Philippe R
  • Stephen L (Renesas)]
  • Sergey (Harman)


Minutes

  • Discussion with Magesh about their multi-display implementation (to be added to table in Technology overview & current options)
  • Demo setup shows One Android, two Linux i.MX6 systems.
  • Protocol was created to transfer screen/surface data over ethernet.
  • There is a launcher for each display.  Possibly multi-user.  So one set of apps per user.  It is possible to launch Android apps to be shown on each receiver (client) display.
  • The Linux (client) receiver systems are primarily dumb thin clients
  • On the Android system, Virtual Displays correspond to each receiver, and the applications can be launched on those virtual displays.
  • It follows the Android published APIs. (AI(Magesh): Send documentation links. -- Actually, I found one and followed-up with email to genivi-projects)
    • Update(Gunnar):  I found this documentation for VirtualDisplay - Magesh, please confirm (and add additional info).
  • The implementation uses hardware-assisted encoding of the resulting graphics before sending.
  • Discussion:  What about inputs?  How are they transferred.
  • Magesh: We transfer touch inputs from clients over our protocol over Ethernet and "inject" it into the Android system.  
  • Gunnar: So far, this matches the overall discussions we had with Google's Automotive team previously.
  • Followed some basic discussion about Waltham/Wayland
    • AllGo protocol was created/proprietary, but it needs only do a full Virtual Display transfer (+ input events)
    • Waltham→Wayland is for compositors...
    • Gunnar: We are interested in graphics transfer standards, for example the question if Wayland/Waltham could be the network standard, even for Android-to-other.  (Of course, that is kind of assuming the Apps↔Compositor relationship, more than Virtual Display only)
    • We have stu
    • Eugen: The compositor function in Android is kind of split up between the Window Management, and SurfaceFlinger.
  • Magesh: I'd be interested to know if we can transfer RAMSES data with our methods
  • Gunnar: RAMSES includes network transfer protocol, it's built in
  • Magesh: How is input handled
  • Eugen:  What I remember, it was a bit complicated and might not be included in the first open-source release.  (So it might be handled outside of the framework)
  • Gunnar: Let's loop back with RAMSES engineers to discuss how input event handling is best implemented in systems using RAMSES.
  • Now that we have more companies with experience, we should pick up the thread again to sync with Google on these ideas.  AI(Gunnar).

Tech Summit

  • Preliminary session agenda was shared with the team.  There will be overview presentations, tutorial (RAMSES), and workshops. Looking for more detailed topic to cover in each of them.
    • → AI(All) Brainstorm and suggest topics for presentation or interesting challenges to discuss in the workshop setting
  • Announcement is out.   More details to be sent next week.
  • DEADLINE:  Send proposals to Philippe R, before Monday August 30 so it may be part of the communication for the summit. 



July 19 Project Meeting

  • Gunnar (GENIVI)
  • Subramanian Dhandapani (Alpine)
  • Horst Saier (Mentor)
  • Philippe R
  • Stephen L (Renesas)

Minutes

  • Gunnar: I didn't send a reminder today so we probably lost a few people... I'm keeping track of vacation planning also
  • Reviewed Virtual Display definition
    • (Stephen) Text is saying "full screen content" but this is seen from the sending side.  On the receiving side it could be part of a larger composition.
    • Gunnar: Agreed, should clarify that
    • Horst: Only Cluster display is mentioned, could be RSE or other.
    • Gunnar: It's noted as being only an example, but I can check if it can be clearer
    • Subramanian: Also a very simple system that just receives and displays the content.
    • Gunnar: Sounds like a "thin client" approach.  Yes it might be another example.
    • Gunnar: Next is to get final consensus from all participants on defintion.
  • Gunnar: API Remoting Tech Brief Work [GSHA] will next be transformed into tech brief.  I'd like to see some contribution here - for example drawing the image. 
    For the RAMSES defintion, I will do some back and forth with BMW
  • Gunnar: Do we have API remoting examples
  • Subramanian: I know I have seen some kind of OpenGL transfer/streaming idea... maybe it was a Youtube video.
  • Gunnar: Please find the details 
    → Action(Subramanian) Look for the OpenGL transfer information / video.
  • Gunnar: Have you made any progress in looking what info Alpine can put into the project?  We talked about it a long time back...
  • Subramaniah: Hmm, we did something with hypervisor and shared memory...
    → Action(Subramanian) Talk to management about what can be described about the project.

  •  Gunnar: OK, now that document publication is going well it's time to refocus on code. 
    Present/show what has been done already in the industry and identify if we need to write new code.  What are the most important ones to do?
    • → Waltham, RAMSES...
  •  Is there a (need for) a community project regarding the actual demonstrations of Waltham?
    • It seems spread out...
    • What part of the conversation happens upstream?  (integration in Weston?).  What are the limits of the scope/plans.
    • Previous Collabora PoCs - published code?
    • AGL - demonstration integrated yet or not?

  • Tech summit 


Graphics Sharing and Distributed HMI July 12 2018, Project Meeting

Participants

  • Stephen L (Renesas)
  • Gunnar (GENIVI)
  • Eugen (ADIT)
  • Bernhard Kisslinger (BMW)
  • Jithin Raj (Tata Elxsi)
  • Guru (Bosch)
  • Horst Saier (Mentor)
  • Philippe R

Minutes


Graphics Sharing and Distributed HMI
June 14 Project Meeting

Participants

  • Stephen L (Renesas)
  • Eugen (ADIT)
  • Sneha James (Tataelxsi)
  • Jithin Raj (Tataelxsi)
  • Sergey (Harman)
  • Miao Luo (The Qt Company)
  • Marco (Mentor)
  • Gunnar
  • Philippe

Minutes

  • new participants
    • Jithin (Tataelxsi): interest is with hypervisor & graphics sharing & common gpu sharing, works in same team as Sneha
    • Miao (The Qt Company): works in the automotive product line ("AutoSuite"), domain interaction is a topic the Qt Company has been digging into for one year (demos have been shown)
  • surface sharing tech brief review: link

    • review of Gunnar's comments

    • comprehensive discussion on what GSHA could standardize,work made on  Qt/Wayland was referred to

    • /TODO/ Eugen prepare a new version of the tech brief for Monday 18 Junr taking into account the review comments

    • /TODO/ Stephen L review the tech brief from a native English speaker standpoint next week

    • Philippe: objective is to publish the tech brief before 27 June



Graphics Sharing and Distributed HMI June 7 2018, Project Meeting

Participants

  • Stephen L (Renesas)
  • Eugen (ADIT)
  • Sneha James (Tataelxsi)
  • Sergey (Harman)
  • Bernhard (BMW)
  • Subramanian (Alpine)
  • Horst (Mentor)
  • Gunnar
  • Philippe

apologies

  • Guru (Bosch)

Minutes

  • new participant: Sneha works in a team focusing on display sharing & hypervisor
  • Review of draft tech brief on surface sharing
    • Gunnar: goes through the paper
    • Stephen: looks good
    • Horst: looks suitable and very good at first glance, will provide more comments next week
    • Gunnar: paragraph on display sharing needs to be reworded
    • /TODO/ ALL send your comments on the wiki page please
  • technology overview wiki page
    • walktrough the page
    • display sharing - Stephen L: I added link to video showing CANVAS, I am currently looking for details on CANVAS demo and gstreamer plugin
    • surface sharing - Gunnar: there is still a big white box for HV Linux to non-Linux
    • Eugen: there are an attempt on-going in virtio interfaces, but technical topic under discussion, other approaches are still being discussed, but the support from the linux community is not sure yet
    • Eugen: will add some inputs in the wiki page
    • /TODO/ Eugen update the overview with inputs on surface sharing
  • display sharing tech brief ownership
    • as said in previous calls, 3-display canvas demo shown by Renesas and GHS is a good example
    • Philippe: we still an owner for this tech brief
    • Stephen L and Eugen could share the ownership in the coming weeks
    • Eugen can provide a 1st version to be filled by Stephen L with further technical details
    • Gunnar: will be a shared work then, should rather be captured in the wiki
    • /TODO/ Eugen start a wiki page for display sharing (using the tech brief toc)
    • Gunnar: would GHS be a possible owner for the tech brief ?
    • Stephen L: don't know !
  • remote api tech brief ownership
      • Bernhard: we do not have the capacity to write it now because of the open sourcing process of Ramses, there are refactoring work in-progress in order to separate the open source part from the proprietary "engines"
      • Gunnar: the load of writing the tech brief can be shared, topic taken offline
  • Wayland Evaluation page
      • /TODO/ all who have not it reviewed yet, please review the page and provide comments (Horst, Serguey, Subramanian, Sneha - will do it next week)



Graphics Sharing and Distributed HMI May 31 2018, Project Meeting

Participants

  • Stephen L (Renesas)
  • Eugen (ADIT)
  • Guru (Bosch)
  • Joonhyung (LGE)
  • Michael (ADIT)
  • Gayathri PP (Tataelxsi)
  • Gunnar
  • Philippe

Minutes

  • walkthrough last week's minutes
    • Guru: I can get info about SDL, mySpin is TBC because it might not be in the public domain
    • /TODO/ Guru gather info about SDL graphics exchange (target: two-week time likely)
  • Waltham evaluation : feedback ?
    • Stephen L: this is fine
    • Eugen: text is not short, but gives good explanation
    • Joonhyung: very helpful from someone not familiar with graphics
    • Michael: will provide feedback later
  • tech brief - surface sharing
    • Eugen: will provide the 1st version today (May 31), describes the graphics sharing in general, Waltham is one possible solution (done) 
    • /TODO/ Stephen / Gunnar review the tech brief on surface sharing
  • tech brief - display sharing
    • Gunnar: we have not seen a lot of use cases
    • Eugen: does it come down to surface sharing with one single display on the surface ? if this is a case, it can be handled by the display driver
    • Eugen: the demo I have in mind is the canvas demo made by Renesas at the Spring AMM
    • Stephen L: this demo is actually a mix of display sharing and gpu sharing
    • Eugen: provides an explanation on the control approach for the display sharing
    • Stephen: will try to get more details on the demo design
    • /TODO/ Stephen get details on the design of the demo shown by Renesas at the Spring AMM
  • milestones plan updated online
  • Eugen: would make sense to have a common intro to all tech briefs

Graphics Sharing and Distributed HMI May 24 2018, Project Meeting

Participants

  • Bernard Kisslinger
  • Christian Schulenberg
  • Haronobu Kurokawa
  • Roman Leykin
  • Sergey
  • Gururaja N
  • Eugen Friedrich
  • Philippe Robin
  • Gunnar Andersson
  • Stephen Lawrence

Minutes

  • Some updates to table on Technology overview & current options have been done
  • Walked through the tree-based input documented in Technology overview & current options.  Slightly augmented.
  • Bernhard: Still discussing internally if we can produce some video proof or similar for cross-OS RAMSES use. 
  • Bernhard: Discussing with project team on resources for publishing tech brief.
  • Stephen to seek out Android teams
  • Gunnar asked Roman & Sergey to seek out QNX related projects in this area
  • Guru asked to  document the graphics sharing details of Bosch mySpin (API remoting: device to car)
  • Discussing Waltham evaluation description page.  Asked everyone to read and give feedback.
  • Still need volunteers for progressing Android API comparison
  • Stephen will provide a link to for QNX demo (TBC)


April→May

Minutes not published on page (yet)


Graphics Sharing and Distributed HMI Project Meeting #18 12 April 2018, 10:30am CET

Participants

  • Marco Residori (Mentor)
  • Eugen (ADIT)
  • Roman (Harman)
  • Sergey (Harman)
  • Stephen (Renesas)
  • Bernhard (BMW)
  • Horst (Mentor)
  • Subramanian (Alpine)
  • Gunnar (GENIVI)


Minutes

  • Discussed and defined the 5th category on the main page
  • Roman: I think there are fewer and more fundamental categories - maybe we only have two categories really, (surface or data sharing)...
  • Is API remoting and Shared data approach maybe the same category...
  • After discussion we decided to keep the distinction.
  • Eugen: Possibly we have two characteristics that each category can either have or lack:
    1. Is interaction between HMIs possible?
    2. Am I sharing graphical data (bitmaps/surfaces
  • Eugen: Note that "synchronized rendering" category is often not strictly synchronized.  You are probably not rendering identical frames, also not guaranteed frame sync.
  • Gunnar: Agreed.  (I brought to the agenda to) come up with a name that is not misleading
  • Candidates:  Shared data rendering, Shared state rendering, Shared content rendering, data model, view/model, ....
  • An informal vote was held.  Shared state rendering and Shared state - independent rendering, received 5 votes each. Other options received fewer or zero votes.  The current group lead is Gunnar and therefore his vote was used as tie breaker.
  • Confirmed plan for GSHA at AMM Munich, April 2018
  • Decided to hold one more informal Graphics planning/working meeting during AMM - Stephen to send out proposal for (likely) lunch time Wed/Thu.


Graphics Sharing and Distributed HMI Webex #17 5 April 2018, 10:30am CET

Agenda

  • Discuss Android-graphics share with Harman

  • Working session organization at the AMM
  • Project deliverable plan

Participants (15)

  • Eugen (ADIT)
  • Emre (ADIT)
  • Roman (Harman)
  • Sergey (Harman)
  • Andrey (Harman)
  • Stephen (Renesas)
  • Joonhyung (LGE)
  • Bernhard (BMW)
  • Christian (BMW)
  • Guru  (Bosch)
  • Horst (Mentor)
  • Subramanian (Alpine)
  • Gunnar (GENIVI)
  • Philippe (GENIVI)

Minutes

  • Harman presented their results (principles + details).   The system is a test system with Android IVI communicating with Linux based cluster.
    Gunnar (offline): Noted that the presentation includes good Use-Case and input requirements before talking about the technology.  Most of that is generic for graphics share and good input for all the approaches.
  • Time boxing the AMM session
  • Discussing a 5th category, "Synchronized Rendering" based on Harman approach and previously discussed appproach for phone/device apps (independent app but communicating with car, and through SOTA having synchronized look&feel)




Graphics Sharing and Distributed HMI Webex #16 29 March 2018, 10:30am CET

Agenda

  • organization of working session at AMM
  • project deliverables

Participants

  • Eucan (ADIT)
  • Eugen (ADIT)
  • Albert Kos (Continental)
  • Sergey Klevitskiy
  • Horst Saier (Mentor)
  • Roman Leykin (Mentor)
  • Subramanian (Alpine)
  • Gunnar (GENIVI)
  • Philippe (GENIVI)
  • Stephan Lawrence (Renesas)

Minutes

to be completed

Graphics Sharing and Distributed HMI Webex #15

22 March 2018, 10.30




Graphics Sharing and Distributed HMI Webex #14 15 March  2018, 17:00 CET

  • RAMSES workshop comments
  • Notes from workshop?
  • Can documentation be made availiable?
  • Updates from Renesas (S.Lawrence) on virtio-gpu, waltham demos, ...
  • webex recording (to be added)

Participants

  • Stephen Lawrence (Renesas)
  • Alistair Adams (The Qt Company)
  • Markus (Intel)
  • Andrey (Harman)
  • Magesh (AllGo)
  • Marco (Mentor)
  • Eugen (ADIT)
  • Subramanian (Alpine)
  • Gunnar (GENIVI)
  • Philippe (GENIVI)

Minutes

Debriefing RAMSES workshop and explaining what we learned to

  • Refer to RAMSES workshop minutes below
  • Question:  Since VM is not available, could documentation be released first, while code is being completed?
  • Discussing how open-source project would be governed, if it's placed in GENIVI care.
  • Markus: It's good to have a strong community like this to discuss these matters.
  • Markus: I'm also interested in specific optimizations that might be possible (even if Intel hardware specific)
  • Discussing plugin architecture to facilitate fair/common handling of hardware-specific optimizations.
  • Short discussion with Alistair/Qt:  What is RAMSES, what makes it attractive?
  • Gunnar:  I'm interested in understanding if RAMSES and Qt could be mixed into something really interesting.  I don't see the details clearly yet however.
  • Magesh:  How is the Android support?
  • Group consensus: Exists, but not heavily tested
  • Discussion - how could an Android app use it?  If not integrated in platform, could the libraries be bundled (presumably, yes).
  • Gunnar - I sent Waltham evaluation page to Pekka and Daniel.  Hoping for some response.  Others, please take a look. 
  • Stephen:  Updates:  Be aware of the Collabora work on VirtIO-GPU. There have been blog posts, and code.  Positive response from Collabora on explaining the work to the group.  Also there's a Waltham demo on Renesas hardware - Stephen asking Collabora if it too can be demonstrated.  Gfx from the cloud - Softbank / Epam contact, still working on it but under way.



RAMSES workshop

13  March 2018, 08.30-17.00 CET
BMW, Munich – Planning page

Participants:

  • Refer to planning page

Minutes

  • Violin explained the principles behind RAMSES, quickly diving into code examples for support.
  • Walking through examples 1 to ? with increasing difficulty / new features
  • RAMSES API wraps OpenGL (ES 3.0 currently) with a slightly higher level API.  Familiar concepts, Textures, Shaders are available.  Animation definitions can be defined abstractly.  Variables can be linked.
  • RAMSES maintains the scene state state and transfers over network when needed
  • The network protocol was presented.
  • A lot of Q&A back and forth.
  • A VM was provided with SDK/Compilers/QtCreator/etc. Examples, and Documentation.
    •  N.B. VM had to be deleted at the end of the workshop since the code is not licensed (yet).
  • To run with TCP/IP, the scenemanager needs to be started (a service discovery component.  Once known connections are peer-to-peer between clients/servers)
  • Participants expressed interest and that "it should be open source".  Clearly several are eager to try it out on their own.

Graphics Sharing and Distributed HMI Webex #13 1 March  2018, 10:30 CET

Participants:

  • johan (Luxoft)
  • marco (Mentor)
  • mathias fielder (Mentor) (replaces horst saier for today)
  • sergey (Luxoft)
  • eugen (ADIT)
  • stephen L (Renesas)
  • subramanian (Alpine)
  • christian S (BMW)
  • bernhard K (BMW)
  • guru (Bosch)
  • gunnar (Genivi)
  • philippe (Genivi)

Minutes
A webex recording is available, uploaded already

SDL
johan: found the team based in St Petersburg
they need to commit to a date
gunnar: there is a feature called audio / video transfer in SDL we might be interested in
johan: plan is to do the presentation next Thu

Waltham
set of slides available from AGL meeting, link to AGL meeting presentation: Graphics Sharing Over the Domain - Wataru Mizuno, ADIT
Eugen: by the end of the month, we should see waltham client and transmitter communicating
there are some activities in-progress on xen and renesas boards
TODO all review the wiki page named Waltham evaluation

Graphics Sharing and Distributed HMI Webex #12 22 February 2018- 10.30 am CET

Participants:

  • Philippe Robin
  • Gunnar Andersson
  • Christian Schulenberg
  • Bernhard Kisslinger
  • Subramanian
  • Guru
  • Roman Leykin
  • Sergey Klevitskiy
  • Andrey Bukirev
  • Stephen Lawrence
  • Horst Saier
  • Marco Residori

Agenda

  • Recap after discussion with Google / AndroidAuto
    • concrete next steps
  • Walkthrough of current topics
  • Waltham code evaluation (notice/intro)
  • Discussed & updated use case list
  • Asked/discussed more CE-device, clarity among options
  • Future planning



Graphics Sharing and Distributed HMI Webex #11 15 February 2018- 1700 / 5 PM CET / 0800 AM, PST

– minutes coming soon



Graphics Sharing and Distributed HMI Webex #10 8 February –

...


Graphics Sharing and Distributed HMI Webex #9 01 February, 2018 - 10.30 am CET

Agenda

QTWayland presentation from Luxoft - slides DIROv4.pdf

webex recording

Additional information

there is some activities in Chromium project virtio wl
Initial google work: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/chromeos-4.4/drivers/virtio/virtio_wl.c
Collabora adding a window server support to virtio  drm: https://lists.freedesktop.org/archives/dri-devel/2017-December/160309.html
Phoronix news: https://www.phoronix.com/scan.php?page=news_item&px=VirtIO-DRM-Window-Server

Participants
Subramanian (Alpine)
Johan Thelin (Luxoft)
Thomas Senyk (Luxoft) via audio only
Sergey K (Harman)
Roman Leykin (Harman)
Stephen Lawrence
Eugen (ADIT)
Kwangsub (Qt)
Bernhard Kisslinger (BMW)
Christian Schulenberg (BMW)
Marco Residori (Mentor)
Guru (Bosch)
Joonhyung Kim (LGE)
Philippe Robin (GENIVI)
Gunnar Andersson (GENIVI)

QTWayland
Johan: Presentation is not only on qt wayland, it is also on Qt design and
how that can support distributed function

slide: Abstractions
points to on-line documentation

Qt in distributed scenarios
hand over to Thomas (Luxoft)

Qt examples
hand over back to Johan

Q&A.
Stephen L: Could Qt benefit from the work done by the Waltham community (i.e. work done in Weston)?
Johan: No work on this yet
Gunnar: Interop might imply one side based on Qt, the other side based on something else

Eugen: QT supports Android, how is this handled.  (Are libraries bundled with app)
Thomas: provides a fairly comprehensive answer (go to recording)

AOB
F2F workshop on Ramses
Gunnar: we could have it hosted in Munich
Eugen: ADIT will join next AMM (Mon 6 April)
Gunnar: this idea was to schedule this workshop before
Johan: I could consider it, but I have a tight travel schedule in the coming weeks but spending some time in Germany
Stephen: I need a travel approval, I would ask the Renesas graphics guys to join (perhaps from Japan)

*TODO* Gunnar create a participation list for this workshop in the GSHA wiki
Update: Not done - I had not noticed I had the action.  But Christian has now taken over the action

Planning for next couple of meetings
Gunnar: we have been through the technologies we wanted to cover, however commercial vendors have not joined the calls yet
Gunnar: @Harman - do you have any thoughts on the Graphics Sharing or on work you would like to share ?
Roman: it would be important to look into Android interaction, need to sync with Kyle on this
Gunnar: FYI the channel with Google is open, we still need to find a timeslot to talk with them, we are working on it
Roman: the Chromium project is important, they implemented virtIO Wayland to couple multiple VMs, this is based on Collabora work
Gunnar:  can you provide a few links to the team ?
*TODO* Roman add links about this work in progress in the meeting minutes DONE (see above)
Roman: we are still looking for use cases identified by OEMs
Gunnar: can you get it started by listing a few points ?  Points Roman to the wiki

Johan: presentation of SDL API remoting approach is possible in two weeks,
Johan spotted the experts within luxoft

Demonstrator part
Gunnar: were are not terribly concrete on this yet, please asked within your project teams for available demos

recording conversion to mp4 is in-progress



Graphics Sharing and Distributed HMI Webex #8 25 January, 2018 - 10:30am CET

agenda

 re-starting activities

Short Notes
Participants

  • Gunnar
  • Bernhard
  • Christian
  • Gianpaolo
  • Horst
  • Johan
  • Subramanian
  • Yonhee
  • Philippe R
  • Stephen

Main topic was the identification of available demos

Ramses:
Bernhard.Kisslinger: BMW is considering organizing a F2F workshop on Ramses in Q1, 2018. The purpose is to show the technology, maybe work out some examples of applications,  and overall understand the community interest. Date and location are TBD
Gunnar: The workshop is before you do the work to fully open-source the code.  If we develop some examples in the workshop, I'd like to understand if participants can take back something that can be demonstrated, i.e. to be run (before the full program is open source?)

Waltham:
Horst Saier: Mentor is interested in evaluating Waltham [in the context of graphics distribution]
Gunnar: Let's talk more about how that work can be done in the project group setting.
Stephen: There is a Waltham demonstrator on Renesas boards available at Collabora (2 starter kits).
Gunnar: It seems to me it's the same companies mentioned [in different constellations], such as ADIT, Collabora.  Why don't we try to get some common demonstration

Graphics Sharing:
LGE: We had a demo at CES, we could  show it in the GSHA project (in the upcoming Spring AMM ?)

SDL:
Gunnar: SDL is a kind of API remoting for all other commands, but for graphics we could see it as Surface Sharing (as far as I know).
Question to Luxoft / Johan & Renesas / Stephen:
   Could you support or seek out a deeper explanation of the SDL graphics sharing methods, and demonstrations.

Next week's call
QtWayland: Johan confirmed this topic is at the agenda, will double check with the intended presenter's manager to make sure the presenter is available. Use cases are  in-progress, will sync offline with Philippe

Other CE device projects?
Gunnar: We all know about Android projection mode & Apple CarPlay, but are there other potentials here?
Gunnar: We are in contact with Google, working out a time slot.  Would be interesting to understand Waltham usage on Android, but what ideas are actually worked on in Android.
Gunnar: [For CE device] Rightware have presented in showcases a technology/product, including CE-device interaction.  No response to the request to work in this group.  www.rightware.com/ (a UI design software and HMI development company):

Webex #7 was cancelled

Graphics Sharing and Distributed HMI Webex #6

21 December, 2017 - 9am CET

agenda 

  • review of TODOs from last week's minutes
  • scheduling of calls and content for January 2018

minutes


Graphics Sharing and Distributed HMI Webex #5 14 December, 2017 - 9am CET

agenda

  • Waltham planning, Q&A
  • Populating wiki, outreach to organizations
  • slide deck (mostly to drive the agenda)

minutes ← follow the link

Graphics Sharing and Distributed HMI Webex #4

7 December, 2017

Waltham & Ramses debriefing

slide deck (mostly to drive the agenda)

Graphics Sharing and Distributed HMI Webex #3 - Materials

30 November, 2017

The main FOCUS of the meeting is a more detailed presentation on RAMSES

slide deck (agenda)

Ramses slide deck

Webex recording (please start at: 4.01)

Graphics Sharing and Distributed HMI Webex #2 - Materials

23 November, 2017

slide deck

ADIT slide deck

Webex recording

Graphics Sharing and Distributed HMI Kickoff - Materials

16 November, 2017

slide deck

Waltham slide deck

Webex recording

*Participants*
Subramanian     subramanian.dhandapani@alpine.de
Johan Thelin (luxoft)   jthelin@luxoft.com
Thomas (luxoft) via audio only
Sergey K        sergey.klevitskiy@harman.com developer working with Roman
Roman Leykin    Roman.Leykin@Harman.com
Stephen Lawrence        stephen.lawrence@renesas.com
eugen   efriedrich@de.adit-jv.com
Kwangsub (Qt)   kwangsub.kim@qt.io
Bernhard Kisslinger     bernhard.kisslinger@bmw.de
Christian Schulenberg   christian.schulenberg@bmw.de
Marco Residori  marco_residori@mentor.com
Guru    Gururaja.N@in.bosch.com
Joonhyung Kim   joonhg.kim@lge.com
Philippe Robin  philippe.robin@technoveo.com
Gunnar Andersson        gandersson@genivi.org

*Agenda*
QTWayland
AOB

*QTWayland*
Johan: presentation is not only on qt wayland, it is also on Qt in 
distributed scenarios

slide: abstractions
points to on-line documentation

qt in distributed scenarios
hand over to Thomas (Luxoft)

qt examples
hand over back to Johan

Q&A.
Stephen L: could Qt benefit from the work done by the Waltham community ?
Johan: no work on this yet
Gunnar: interop might imply one side based on Qt, the other side based on 
something else

Eugen: QT supports Android, how is this handled
Thomas: provides a fairly comprehensive answer (go to recording)

*AOB*

*F2F workshop on Ramses*
Gunnar: we could have it hosted in Munich
Eugen: ADIT will join next AMM (Mon 6 April)
Gunnar: this idea was to schedule this workshop before
Johan: I could consider it, but I have a tight travel schedule in the 
coming weeks
Stephen: I need a travel approval, I would ask the Renesas graphics guys to 
join (perhaps from Japan)

*TODO* Gunnar create a participation list for this workshop in the GSHA wiki

*planning for next couple of meetings*
Gunnar: we have been through the technologies we wanted to cover, however 
commercial vendors have not joined the calls yet
Gunnar: @Harman - do you have any thoughts on the Graphics Sharing or on 
work you would like to share ?
Roman: it would be important to look into Android interaction, need to sync 
with Kyle on this
Gunnar: FYI the channel with Google is open, we still need to find a 
timeslot to talk with them, we are working on it
Roman: the Chromium project is important, they implemented virtIO Wayland 
to couple multiple VMs, this is based on Collabora work

  • No labels