Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: added link to virtual display page

...

  • Surface sharing
    • Operating systems exchange graphical (bitmap) content. Then, each OS has full flexibility to use this content. In some cases, the compositor API is made available remotely, e.g. Wayland→Waltham.
    • Studied technologies: 
      • Waltham
      • Consider: SmartDeviceLink (graphics sharing for Navigation and similar apps)
      • Qt Remote Objects are not designed specifically for graphics - more for generically mirroring any data object.  It was noted however, that if the object is a QImage, the feature should in fact synchronize bitmap data. (The feature is unlikely to be optimized for graphics transfer).
      • Renesas Weston compositor virtual-display and gst-recorder plugin...
        • Expand
          titledetails...
          • Renesas Weston compositor virtual-display and gst-recorder plugin
            • virtual-display allows the creation of a composited display window the correct size which can then be remoted
            • gst-recorder GStreamer plugin in Weston Compositor encodes the virtual display using H.264 and transmits. Receiver uses GStreamer to receive and decode the stream, which can then be composited as required.
            • Pros:
              • No modification required to Weston on receiver side.
              • If layer control is required the decoded stream can be transformed as required on receiver side in gfx framework of your choosing.
              • Video encoding minimises network data and bandwidth
              • Virtual display means display size can be made only as large as needed.
              • Technology has multiple applications, e.g. has been used in Waltham implementation for gfx data transfer.
            • Cons:
              • Delay of about 2-3 vsync to encode, transmit, receive, decode and composite the stream.
              • Large display containing a lot of animated objects will impact network bandwidth
            • R-Car M3 PoC in AGL demonstrator. Open source: links TBA
              • Two M3 ECUs. Navigation and mapview app encodes mapview image using gst-recorder and transfers via ethernet (udp). Receiver mapview image, decodes the data and displays moving map image between dials of cluster display.
            • See the slides presented at the ALS 2018 talk "Remote Access and Output Sharing Between Multiple ECUs for Automotive"
    • (Sub-category): Virtual Display
      • some more details needed here...
  • API Remoting
    Transfer API calls, corresponding to "drawing commands", or other abstract graphics representation from one ECU to another, to be executed on the GPU of the receiving ECU.
    • Studied technologies: 
      • RAMSES
      • Qt WebGL streaming is somewhat similar.  The sending side has an OpenGL representation (typically created using Qt abstractions), which is transferred to a WebGL capable renderer.  (blog post, blog #2 - cinematic experience example, feature merged into Qt 5.10
      • Note: SmartDeviceLink.  While SDL design is generally about making remote API calls, the graphics exchange is surface sharing, as far as we know.  A kind of API-remoting for operations as opposed to graphics)

  • Shared state, independent rendering
    • Each system has independent graphics systems and bitmap information. The systems only synchronize their internal state and exchange abstract data. Based on this shared data, each system independently render graphics to make it appear like they are showing the same or related graphics.
    • Studied technologies/examples:

      • Harman presentation at AMM, also in Tech Brief

      • Qt Remote Objects provide a generic way to mirror/synchronize internal state.  If both sides are using Qt, it would be a natural tool.

...