In his essay “The Ultimate Display” from 1965, Ivan E. Sutherland states that “The ultimate display would [...] be a room within which the computer can control the existence of matter [...]“. This general notion of a computer-mediated or virtual reality, in which synthetic objects or the entire virtual environment get indistinguishable from the real world, dates back to Plato’s “The Allegory of the Cave” and has been reconsidered again and again in science fiction literature as well as the movie industry. VR is often used to question whether we truly “know” if our perceptions are real or not. Movies like “The Matrix” or the fictional holodeck from the Star Trek universe are prominent examples of these kind of perceptual ambiguities. Furthermore, in movies like Steven Spielberg’s “Minority Report” or Jon Favreau’s “Iron Man 2″ actors can seamlessly use free-hand gestures in space combined with speech to manipulate 3D holographic projections, while they also perceive haptic feedback when touching the virtual objects. In my talk I will revisit some of the most visually impressive 3D user interfaces and experiences of such fictional ultimate displays. As a matter of fact, we cannot let a computer fully control the existence of matter, but we can fool our senses and give a user the illusion that the computer can after all. I will show how different ultimate displays can be implemented with current state-of-the-art technology by exploiting perceptually-inspired interfaces. However, we will see that the resulting ultimate displays are not so ultimate at all, but pose novel interesting future research challenges and questions.