Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Update next meeting

(star) What is this?  → Please refer to the projects overview page for a quick introduction, then for more details, the Kickoff slides and kick-off recording (see below).

Table of Contents
maxLevel2

Graphics Sharing and Distributed HMI - Next Call

...


Next Meeting

We are currently on a bi-weekly schedule.

(green star) Thursday, October 24, 10:30 AM CE(S)T

Zoom Meeting Join Link, (register first - you can do that before to prepare) (star) Password: gsha

Agenda

  • discussion on Android & Wayland


...


Backlog

Topics to be (re)scheduled:

    • Benchmarks and performance knowledge.   (Pool together results from ADIT, Qt, Harman, BMW/Mentor/RAMSES, AllGo, and all...)
    • Provide & discuss details on Weston-remote
    • Android IVI use cases & Android's SurfaceFlinger and Wayland API's Mapping
      • HAL role and features?
    • API Remoting of OpenCL (Artem/EPAM).  (This is not for graphics, but it's GPU and it's API Remoting.  Might be useful.)
    • Coordinate "GPU sharing" topic with the Hypervisor Project
    • Deep-dive?  Web assembly as graphics transfer technology?  (QtWebAssembly) (revisit in February)
    • The Study the graphics sharing part of SmartDeviceLink protocol and compare (need volunteer – possibly Özgür Bulkan )

Requests/actions:

    CES 2019, Las Vegas, USA. 
    • (red star) Example/evidence of graphics distribution involving other graphics / HMI tools including various "commercial HMI tools"
    • (DONE, but feel free to share more) (star) Use-case / requirements workstream.  Write down use cases, scenarios and requirements for graphics sharing & distributed HMI.
    • More example/evidence of Android-based graphics sharing - (especially distributed network, since Hypervisor-based is common) (DONE, but feel free to share more) 
    • Example/evidence of QNX-based graphics sharing
    • (star) Use-case / requirements workstream.  Write down use cases, scenarios and requirements for graphics sharing & distributed HMI.

Upcoming Events

    • sharing (DONE, but feel free to share more) 
    • Example/evidence of graphics distribution involving other graphics / HMI tools including various "commercial HMI tools" (error)

Previous Events

...

Definitions

...

Overall (project scope)

  • Graphics Sharing
    = 1. Graphics in the form of bitmap, scene graph or drawing commands generated on one ECU, transferred for display from another ECU (or between virtual machine instances)
        2. GPU sharing in a virtualized setup
  • Distributed HMI Compositing
    = Methods and technologies to turn a multi-ECU system into what appears and acts as a single user-experience. 

Results

The 5 Categories of Graphics Sharing technologies

(as developed by the group so far)

  • GPU sharing
    The GPU can be used from multiple operating systems, so it is shared. Concurrent access to the physical GPU has to be controlled by the hypervisor, hardware or other means which are implementation specific.

  • Display sharing
    The physical display can be shared across multiple operating systems. HW compositor unit composites final display buffer from HW Layers of each OS. This requires virtualization of the display controller hardware.
  • Surface sharing
    Operating systems exchange graphical (bitmap) content. Then, each OS has full flexibility to use this content. In some cases, the compositor API is made available remotely, e.g. Wayland->Waltham.
  • API Remoting
    Transfer API calls, corresponding to "drawing commands", or other abstract graphics representation from one ECU to another. Commands or scene representation to be executed on the GPU of the receiving ECU.
  • Shared state, independent rendering
    • Each system has independent graphics systems and bitmap information. The systems only synchronize their internal state and exchange abstract data. Based on this shared data, each system independently render graphics to make it appear like they are showing the same or related graphics.
      • Example:  An appearance of synchronized map rendering could be achieved by drawing maps independently, and exchanging only the GPS position, scale of map, etc.
      • Comment: This differs slightly from API Remoting as follows:  API Remoting implies active control from a primary to secondary system, i.e. a command exchange such as "Draw this", whereas Shared state is more akin to "here is the data you need - but you control how/what to draw" (still with the expectation that the appearance will match the "common" HMI).
    • (green star) See Shared State tech brief

  • GPU sharing
    The GPU can be used from multiple operating systems, so it is shared. Concurrent access to the physical GPU has to be controlled by the hypervisor, hardware or other means which are implementation specific.

Project Goals

  • All project participants gain thorough understanding of available choices
  • Produce technology demonstrators, newly created or (if exists already) found and highlighted.
  • Publish hard data on learning:  Performance, resource needs.
  • Seek industry acceptance & alignment among Linux distributions, as well as across operating systems and domains
  • Seek alignment on solutions and protocols among proponents of “closed” alternatives  – commercial HMI-tools, etc.
  • Promote open standards and implementations across industry
  • Separately identify and describe Hypervisor-based opportunities, how they differ, characteristics, advantages and disadvantages.
  • Summarize and create (implementation) documentation for recommended choices

Milestones

Define Graphics Sharing Categories
(tick)

Publish Tech Brief - Shared State
(tick) April 2018 – DONE (see page)

"Complete" Table-based overview
→ Work in progress.  Decide on completion.

Publish (tick) Define Graphics Sharing Categories

(tick) Publish Tech Brief - Shared State

  • April 2018 – DONE (see page)

(tick) Publish Tech Brief - Surface Sharing(tick) July

  • July 2018 – DONE (see page)

Publish (question) Publish Tech Brief - GPU Sharing?

  • We will first put it onto the agenda of the HV project to seek experts/volunteers among that group.

Publish (tick) Publish Tech Brief - API Remoting (or with case study: RAMSES?)

  • API Remoting working page for detailed definition and tech brief.
  • (star) Time Time plan: October Tech Summit

Publish (tick) Publish Tech Brief - Display Sharing

  • (star) Time Time plan:  Write tech brief for mid-November. Ongoing, ready early February.
  • 3 display canvas demo as example? → GPU virtualization with HV, and display sharing (HW supported).
  • Prepare analysis Analysis on separate Wiki PageWorking page for , tech brief draft work here

...

  • PUBLISHED – see page

(green star) Publish Tech Brief - Surface Sharing: Virtual Display?

  • Do we wish to publish a tech note on the Surface Sharing sub-category "Virtual Display"?
  • Technologies known to group:
    • Surface remoting via video streaming.
    • Extending Weston to support virtual remote displays
    • AllGo demo (standard Android APIs + proprietary surface transfer/control protocol)

...

(star) Publish Project Report and/or Whitepaper on all technologies

  • Time Updated time plan:  Release at end-of-year 2018Q1 2019 → delay → Q2
  • Compare technologies - make clear use-case and constraints that lead to decision.
    • Volunteers (writing / copy-editing):
      • Eugen, Harsha, Stephen, Gunnar.

...

      •  
        Qt Company provided short text for Remote Objects
        Others hopefully once document structure is up.


(tick) RAMSES Open Source Release
Autumn 2018...

Use cases

Here we list some real-world functions (ideally from user perspective), to anchor the technical discussions to.

...