High quality, low latency in-home streaming of multimedia applications for mobile devices
Today, mobile devices like smartphones and tablets are becoming more powerful and exhibit enhanced 3D graphics performance. However, the overall computing power of these devices is still limited regarding usage scenarios like photo-realistic gaming, enabling an immersive virtual reality experience or real-time processing and visualization of big data. To overcome these limitations application streaming solutions are constantly gaining focus. The idea is to transfer the graphics output of an application running on a server or even a cluster to a mobile device, conveying the impression that the application is running locally. User inputs on the mobile client side are processed and sent back to the server. The main criteria for successful application streaming are low latency, since users want to interact with the scene in near real-time, as well as high image quality. Here, we present a novel application framework suitable for streaming applications from high-end machines to mobile devices. Using real-time ETC1 compression in combination with a distributed rendering architecture we fully leverage recent progress in wireless computer networking standards (IEEE 802.11ac) for mobile devices, achieving much higher image quality at half the latency than other in-home streaming solutions.(original abstract)
- "FOV2GO," http://projects.ict.usc.edu/mxr/diy/fov2go/.
- "IEEE Standard 802.11ac-2013 (Amendment to IEEE Std 802.11-2012)," 12 2013.
- "PlayStation Now," http://us.playstation.com/playstationnow.
- "Simple DirectMedia Layer," http://libsdl.org.
- "YUV420," http://www.fourcc.org/yuv.php\%23IYUV.
- Ahlers E., "Rasante Datenjongleure," c't Magazin, vol. 1, pp. 80-89, 2014.
- Chalmers A. and Reinhard E., "Parallel and distributed photo-realistic rendering," in Philosophy of Mind: Classical and Contemporary Read-ings. Oxford and. University Press, 1998, pp. 608-633.
- Ericsson Labs, "Ericsson texture compression tool etcpack v2.60: ETC2," https://labs.ericsson.com/research-topics/media-coding.
- FFmpeg project, "FFmpeg," http://www.ffmpeg.org.
- Fox G., Williams R., and Messina G., Parallel Computing works!, 1st ed. Morgan Kaufmann, 1994.
- Gaikai Inc., "Gaikai," http://www.gaikai.com.
- Hildebrandt A., Dehof A., Rurainski A., Bertsch A., Schumann M., Toussaint N., Moll A., Stockel D., Nickels S., Mueller S., and Lenhof O., Kohlbacher H. -P., "BALL - Biochemical Algorithms Library 1.3," BMC Bioinformatics, vol. 11, no. 1, p. 531, 2010. doi: 10.1186/1471-2105-11-531
- Huang C. -Y., Hsu C. -H., Chang Y. -C., and Chen K. -T., "GamingAnywhere: An open cloud gaming system," 2013. doi: 10.1145/2483977.2483981 pp. 36-47.
- Intel Corporation, "Intel wireless display," https://www-ssl.intel.com/content/www/us/en/architecture-and-technology/intel-wireless-display.html.
- Löffler A., Pica L., Hoffmann H., and Slusallek P., "Networked displays for VR applications: Display as a Service (DaaS)," in Virtual Environments 2012: Proceedings of Joint Virtual Reality Conference of ICAT, EuroVR and EGVE (JVRC), 10 2012. doi: 10.2312/EGVE/JVRC12/037-044
- MADFINGER Games, "Dead Trigger 2," https://play.google.com/store/apps/details?id=com.madfingergames.deadtrigger2.
- Madhusood A., "Best practices for using Intel Cilk Plus," Intel Corporation, White Paper, 7 2013, http://software.intel.com/sites/default/files/article/402486/intel-cilk-plus-white-paper.pdf.
- Marsalek L., Dehof A., Georgiev I., Lenhof H. -P., Slusallek P., and Hildebrandt A., "Real-time ray tracing of complex molecular scenes," in Information Visualisation (IV), 2010 14th International Conference. IEEE, 2010. doi: 10.1109/IV.2010.43 pp. 239-245.
- Microsoft Corporation, "Remote Desktop Connection," http://windows.microsoft.com/en-us/windows/connect-using-remote-desktop-connection.
- Moll A., Hildebrandt A., Lenhof H. -P., and K. O., "BALLView: a tool for research and education in molecular modeling." Bioinformatics, vol. 22, no. 3, pp. 365-366, 2006. doi: 10.1093/bioinformatics/bti818
- Nave I., David H., Shani A., Tzruya Y., Laikari A., Eisen P., and Fechteler P., "Games@Large graphics streaming architecture," 2008. doi: 10.1109/ISCE.2008.4559473
- Neptune Computer Inc., "Neptune Pine," http://www.neptunepine.com.
- Nvidia Corporation, "Nvidia Shield," http://shield.nvidia.com.
- OnLive Inc., "Onlive," http://www.onlive.com.
- Pohl D., Johnson G., and Bolkart T., "Improved Pre-Warping for Wide Angle, Head Mounted Displays," 2013. doi: 10.1145/2503713.2503752 pp. 259-262.
- Richardson T., Stafford-Fraser Q., Wood K., and Hopper A., "Virtual network computing," IEEE Internet Computing, vol. 2, no. 1, pp. 33-38, 1998. doi: 10.1109/4236.656066
- RmDroider, "CurrentWidget: Battery Monitor," http://code.google.com/p/currentwidget/.
- Shoogee GmbH & Co KG, "Durovis Dive," http://www.durovis.com/index.html.
- Singh J., Gupta A., and Levoy M., "Parallel visualization algorithms: performance and architectural implications," Computer, vol. 27, no. 7, pp. 45-55, 1994. doi: 10.1109/2.299410
- Sony, "PlayStation 4," http://us.playstation.com.
- Splashtop Inc., "Splashtop personal," http://www.splashtop.com.
- Ström J. and Akenine-Möller T., "ipackman: High-quality, lowcomplexity texture compression for mobile phones," vol. 2005, 2005. doi: 10.1145/1071866.1071877 pp. 63-70.
- Taudul B., "etcpak 0.2.1: The fastest ETC compressor on the planet," https://bitbucket.org/wolfpld/etcpak.
- Valve Corporation, "Steam," http://store.steampowered.com.
- Vo D. and Nguyen T., "Quality enhancement for motion JPEG using temporal redundancies," IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 5, pp. 609-619, 2008. doi: 10.1109/TCSVT.2008.918807
- Wang Y., Ostermann J., and Zhang Y., Video Processing and Communications. Prentice Hall, 2002, p. 29.
- Wang Z., Lu L., and Bovik A., "Video quality assessment based on structural distortion measurement," Signal Processing: Image Communication, vol. 19, no. 2, pp. 121-132, 2004. doi: 10.1016/S0923-5965(03)00076-6
- Wiegand T., Sullivan G., Bjøntegaard G., and Luthra A., "Overview of the H.264/AVC video coding standard," IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 560-576, 2003. doi: 10.1109/TCSVT.2003.815165
- Wi-Fi Alliance, "Miracast," http://www.wi-fi.org/wi-fi-certified-miracast\%E2\%84\%A2.
- Woop S. and Benthin C. and Wald I., "Embree ray tracing kernels for the Intel Xeon and Intel Xeon Phi architectures," http://embree.github.io/data/embree-siggraph-2013-final.pdf.