<Intro, the need for unified HMI in vehicles, etc.>
Five different primary techniques for distributing graphics and HMI have been identified in the GENIVI Graphics Sharing and Distributed HMI project. This paper describes the technique that was named API Remoting.
API Remoting method
API Remoting is the technique of taking an existing graphics API and making it accessible remotely, thus enabling one system to essentially call the graphics/drawing API of another system.
In some cases an existing programming API, such as that of a local graphics library is simply modified or extended to provide network-capability it did not have before, in other words to translate directly a local programming API to into an RPC (Remote Procedure Call) interface. In other cases, new programming interfaces might be created that are specially tailored for the remote graphics use case.
Many programming interfaces are networked interfaces, and there is a huge amount of technologies making programming APIs network-capable and independent of their location (local/remote). The sister project "Generic Protocol Evaluation" deals with analyzing some of those methods.
What we consider distinctive to Graphics Sharing through API Remoting, as opposed to similar things, is that the API in question is graphics/drawing related. We are dealing with essentially specific "drawing commands" invoked by system A on system B and the actual creation of images is executed on the remote system B. Thus, it's different from surface sharing wherein the drawing/rendering of an image would be completed on system A, before the image is transferred.
The API in question could be as low-level as "Draw line/box/circle/fill area at this coordinate". For some cases, such as a drawing a simple Navigation map, sending such drawing commands as opposed to images could be more bandwidth efficient, which is why API remoting deserves to be highlighted as a distinct and useful approach.
The remote API can also be more capable and "higher level", such as the definition of 3D objects, textures, animations and relationships. This will require a more advanced processing on the receiving side that renders the graphics and it therefore involves more advanced library algorithms and of course use of a GPU on the remote side .
System A <---> System B
Handheld device displays
Smart phones and similar devices are an important interaction method to modern vehicles and should be considered part of the extended HMI of the car. Whereas many smart phones solutions might use simply the Shared State - Independent Rendering method, it's also likely that some API remoting could be used for the car to define and display an HMI on a mobile device. This requires a framework or standard in which the car and the mobile phone agree on the distributed graphics interface.
RAMSES is a framework for defining "3D" scenes for HMIs and act as a thin abstraction of OpenGL commands. Through its C++ API, a lot of control is given to the user (and direct access to OpenGL if needed) in defining the scene graph. It is thus only a small abstraction/simplification of the complexity of OpenGL, without being very high level. RAMSES however allows for defining some higher level relationships between objects in a scene graph, and also dynamic relationships. For example, animations can be defined in terms of how objects should move in relation to each other, and the details of programming this animation over time is solved by the library.
In addition to providing a convenient C++ programming interface towards GPU features, RAMSES also enables the definitions to be rendered on a remote display. An efficient representation of the defined scene graph, along with needed textures and other meta-data such as the animation definitions, is transferred over the network. Following the main characteristic of API Remoting, we can recognize that the final rendering of graphics is done by the GPU on the receiving side.
The sender and receiver libraries work in concert to achieve network transfer only when the data is required. Unlike a more direct API such as "draw this line here (right now)", RAMSES works through defining first the scene graph and rules locally, and the libraries will take care of when and what needs to be transferred, as well as what and when it is required.
Usage in virtualized environments
Since API Remoting deals with simply commands sent over a network it should apply to virtualized environments with minimum adaption necessary. Whatever (virtual) network capability, the system provides will likely be transparent to the RPC system that is selected, and possibly some efficient techniques such as shared memory is employed in the implementation of the networking/RPC capability itself. As always, the impact of timing should be evaluated and system behavior may need to be confirmed with the real API Remoting framework and real HMI in place before correct behavior is fully confirmed.