<Intro, the need for unified HMI in vehicles, etc.>
Five different primary techniques for distributing graphics and HMI have been identified in the GENIVI Graphics Sharing and Distributed HMI project. This paper describes the technique that was named API Remoting.
API Remoting method
API Remoting is the technique of taking an existing graphics API and making it accessible remotely, thus enabling one system to essentially call the graphics/drawing API of another system.
In some cases an existing programming API, such as that of a local graphics library is simply modified or extended to provide network-capability it did not have before, in other words to translate directly a local programming API to into an RPC (Remote Procedure Call) interface. In other cases, new programming interfaces might be created that are specially tailored for the remote graphics use case.
Many programming interfaces are networked interfaces, and there is a huge amount of technologies making programming APIs network-capable and independent of their location (local/remote). The sister project "Generic Protocol Evaluation" deals with analyzing some of those methods.
What we consider distinctive to Graphics Sharing through API Remoting, as opposed to similar things, is that the API in question is graphics/drawing related. We are dealing with essentially specific "drawing commands" invoked by system A on system B and the actual creation of images is executed on the remote system B. Thus, it's different from surface sharing wherein the drawing/rendering of an image would be completed on system A, before the image is transferred.
The API in question could be as low-level as "Draw line/box/circle/fill area at this coordinate". For some cases, such as a drawing a simple Navigation map, sending such drawing commands as opposed to images could be more bandwidth efficient, which is why API remoting deserves to be highlighted as a distinct and useful approach.
The remote API can also be more capable and "higher level", such as the definition of 3D objects, textures, animations and relationships. This will require a more advanced processing on the receiving side that renders the graphics and it therefore involves more advanced library algorithms and of course use of a GPU on the remote side .
System A <---> System B
Handheld device displays
Smart phones and similar devices are an important interaction method to modern vehicles and should be considered part of the extended HMI of the car. Whereas many smart phones solutions might use simply the Shared State - Independent Rendering method, it's also likely that some API remoting could be used for the car to define and display an HMI on a mobile device. This requires a framework or standard in which the car and the mobile phone agree on the distributed graphics interface.
RAMSES is a framework for defining, storing and distributing 3D (can be also 2D) scenes for HMIs. From users perspective, RAMSES looks like a thin C++ abstraction of OpenGL commands. RAMSES offers the same feature level as OpenGL but allows higher levels constructs such as widgets to be built on top of it. In addition to basic OpenGL, RAMSES supports higher level relationships between objects such as a hierarchical scene graph, content grouping, off-screen rendering, animations, text, and compositing. All of those features follow the principle of API remoting, i.e. to define the state of an object/animation and execute it on a different machine.
RAMSES distinguishes between content creation and content distribution; the same content (created with RAMSES) can be rendered locally, on one or more network nodes, or everywhere. RAMSES handles the distribution of the scenes, and offers an interface to control the final composition - which scene to be sent where, shown at which time, and how multiple scenes and/or videos will interact together. The control itself is subject to application logic (HMI framework, compositor, smartphone App etc.).
The data pipeline and timing are "immediate", i.e. the state of each scene is simultaneously pushed to all network participants. RAMSES provides a set of mechanisms to control framerate, timings and avoid jitter and frame drops as much as possible, but in order to achieve smooth experience on all network participants it needs careful system design and content optimization.
Usage in virtualized environments
Since API Remoting deals with simply commands sent over a network it should apply to virtualized environments with minimum adaption necessary. Whatever (virtual) network capability, the system provides will likely be transparent to the RPC system that is selected, and possibly some efficient techniques such as shared memory is employed in the implementation of the networking/RPC capability itself. As always, the impact of timing should be evaluated and system behavior may need to be confirmed with the real API Remoting framework and real HMI in place before correct behavior is fully confirmed.