A Graphical Revolution: Making sense of zeros and ones
Technological Prehistory and Notable Early Influential Developments within Interface Design
As well as looking to art history itself, the rise and history of interaction design also offers interesting perspective. The invention of the Graphical User Interface (GUI, pronounced ‘Gooey’) was a significant turning point in design, hugely influencing how we interact with technology and—consequently—the influence of that technology on our lives. As video games are fundamentally interactive experiences, the advancements of the GUI and other developments within Human Computer Interaction (HCI) play a crucial role in the evolution of game play.
To trace the roots of the GUI, it may seem logical to travel back to the dawn of Personal Computers (PCs). After all, what use would we have for a graphical interface before then; what would we possibly put it on? However, ideas of such an interface can indeed be traced back much further than the personal computer, far before technology was capable of realising them.
1930’s, Vannevar Bush and The Memex
Particularly, the late 1930’s, when Vannevar Bush wrote about a hypothetical device named the Memex, published in an article named As We May Think. His idea would have far reaching influence on interface design, long after it’s time.
The Memex, pictured above, was envisioned as a desk with two touch screen displays, a keyboard and a scanner attached to it. The idea was to allow the user access to all human knowledge using connections very similar to the hyperlinks we are familiar with today. The fact that this idea was conjured as early as the 1930’s is hugely interesting, and the way in which technology has panned out since is remarkably close to Bush’s ideas.
“The irony here, is that a middle-aged army scientist, writing thirty years before the first PC, understood interactivity better than all the Web titans in Silicon Valley. [..] After all, sometimes the best way to understand a technology is to approach it with no expectations, no preconceived ideas. Unhampered by any historical precedent”
—Steven Johnson, Interface Culture
We do, in the present day, have access to a huge expanse of human knowledge through the internet. Whether this is exactly what Vannevar had in mind or not is unclear but the concept is certainly not too far of a stretch — especially considering our method of interacting with that knowledge is through graphical displays, input devices and a system of hyperlinks all very much like what Bush described.
It is innovators like Vannevar Bush who we can thank for paving the way towards the methods of interaction we often take for granted now. In the words of Frank Chimero, innovators “do not stand on the inside of what is possible and push; they imagine what is just outside of what we deem possible and pull us towards their vision of what is better. They can see through the fog of the unexplored spaces and notice a way forward”.
I think this is a quintessential way of describing not only the ideas of Bush, but of all the other hugely influential innovators of interaction, some of whom I will briefly cover in the following sections. In fact, I could go as far as to say that Chimero’s words epitomise the very core sentiment of this entire paper.
1960’s, Douglas Engelbart’s Demo
One of those innovators who pulled us toward their vision with great impact is Douglas Engelbart. Although the Memex wasn’t developed because of the lack of technology at the time, the ideas proved hugely influential later in the century. Douglas Englebart, often considered the ‘father’ of the GUI, began to work on a machine which would serve to improve human intellect. He recalled Vannevar Bush’s essay to conceptualise such a machine, where the user could build models of information graphically and navigate around them dynamically.
In 1962 this was a huge leap of thinking, undoubtedly difficult for most people to comprehend; the computers which existed at this time were room-filling mainframes operated by specialists only. Despite being a difficult concept to persuade, by 1968 his ideas, technology and staff had grown sufficiently and he demonstrated his ideas publicly in front of over a thousand computer professionals.
This was the “public debut of the computer mouse” but this was only “one of many innovations demonstrated that day” [Source]. The mouse was mechanically different to modern mice, however the way which the user interacted with it is virtually identical. This demo would spark the widespread adoption of the GUI and therefore “dramatically changed the way in which humans and computers interact” according to Johnson (1997), who continues: “The visual metaphors that Doug Engelbart’s demo first conjured up in the sixties probably had more to do with popularizing the digital revolution than any other software advance on record”
Johnson’s comments certainly carry weight which can be seen simply by noting the similarities between Douglas’s 1960’s demo and modern technology. Douglas was undoubtedly one of the most influential figures in interface design; his technological advancements provided a solid base for which designers to work upon in the coming years. Although his work was focused more on mechanics and physical technology rather than design, without his intellect GUIs would not have received the foundational technology on which to base themselves, which was necessary in order for them to eventually achieve the result we experience today.
1973, Xerox PARC’s The Alto
Researchers at Xerox PARC were amazed by Douglas’s demonstration, which inspired their creation of the Xerox Alto in 1973. Despite being a commercial failure, the Alto is widely considered a large influence in interaction design, making some important breakthroughs with the design of its GUI. It was widely used for research purposes [Source] and therefore allowed for further developments within Human Computer Interaction.
The Alto began with an interface that resembled a command terminal more than a desktop environment, but eventually resulted in the creation of SmallTalk in 1975. Originally conceived as a programming environment, SmallTalk went on to become the first modern GUI and in turn conceived the earliest use of icons and pop-up menus.
As well as this, the Alto also demonstrated the first use of a diagonal-pointing bitmap cursor pointer which we recognise in modern computing today. What is most notable about this particular cursor is it’s behavior – it alternated between different shapes depending on the task. For example, today the cursor may change into a hand when grabbing; a watch, spinning wheel or hourglass when loading; various different arrows for re-sizing, etc. This is a significant example of early visual feedback, a crucial element of interface design.
A final advancement to note is that the Alto also inspired the creation of Alto Trek, one of the first network-based multi-person video games which was also the first game to utilise the mouse, and would later inspire the creation of Microsoft’s Allegiance [Source]. It is clear that Xerox contributed many important and varied developments within interaction design.
One of the most crucial of all these developments, though, occurred by inspiring the work of another important innovator — Steve Jobs.
1978 Apple Lisa and the 1984 Apple Macintosh
As established, the influences of the Xerox team were far reaching, but their effect on the developments of Apple was one of—if not the—most crucial of all; what Steve Jobs would go on to create from this sparked revelation he experienced during a visit to Xerox PARC would alter the landscape and direction of User Interface and Experience indefinitely.
Development for Apple Lisa began in 1978, with some members of the team being former members of the Xerox PARC group. The project was to design a powerful personal computer with a Graphical User Interface that would be targeted toward business computers.
The Lisa used a desktop metaphor and saw the birth of the first pull-down menu bar, with each menu always appearing horizontally across the top of the screen. This is just one convention created then that still exists, almost entirely unchanged, in Mac OS X today (at the time of writing, the current version is 10.11 OS X El Capitan, as shown in Fig above). The Lisa also introduced many other elements which we take for granted today: check-marks for selected menu items; keyboard command shortcuts; greyed out inactive items; the trash can; the use of icons to represent the entire file system; drag-and-drop; double-clicking — just to name a few. The developments here allowed for progress towards a universal structure for organising information on the screen, in a way that is familiar, versatile and user-friendly.
However, it wasn’t exactly the Lisa itself that went on to make history. Despite being such an advanced machine, sales were limited mostly due to the $10,000 price tag and difficulty of writing software for it. This called for a much more simplified, lower cost version of the Lisa. Steve Jobs took the task upon himself and achieved this goal with the original Apple Macintosh, which was introduced to the world in dramatic and iconic fashion in 1984, retailing for $2,495. It retained most of the GUI features of the Lisa, and even shared some of its low-level code, but the operating software itself was written from scratch to fit in the small memory footprint. It was this machine that would succeed, and mark a significant turning point for interface design.
Questions surrounding who invented what or who stole from who are often hotly debated within the technology industry. However—regardless of personal opinions or accusations—if artists, entrepreneurs and inventors didn’t take influence from one another, then that would be a true hinderance of innovation. Influence is an intrinsic, fundamental element that is necessary for innovation to happen.
This sentiment can, in fact, be best described with a Haiku written by Yosa Buson.
“Lighting one candle
with another candle—
In the words of Frank Chimero,
“Buson is saying that we accept the light contained in the work of others without darkening their efforts. One candle can light another, and the light may spread without its source being diminished.”
As creators, we must accept that creation and innovation is an accumulative effort – one that is ever progressing. Our own ideas along with everyone else’s snowball together to collect new thoughts and developments along the way — a movement that never ceases. As our malleable inspirations travel down the infinite branches of the thoughts of others, they become reshaped — moulded into something new. An improvement here and a new perspective there, the result is perpetual growth and change. At each stage, no one can claim ownership to an idea, for it is the combined product of a hundred others. Did Karl Benz, Edouard Michelin, or Henry Ford steal the wheel from the Sumerian people of the Bronze Age? Equally, did Douglas Engelbart steal from Vannevar Bush? Or did their personal contributions build upon an ever-progressing concept; the summation of the ideas and contributions of many, always pushing the boundaries of what we know, drawing us closer to that which lies beyond what we currently see?
This is how we innovate; we take inspiration and then develop it. This is what distinguishes innovation from thievery—personal, individual development. Steve Jobs took a mouse that cost Xerox $300 to develop and made it cost $15, while also simplifying it and improving the ease-of-use. “If you lined up Engelbart’s mouse, Xerox’s mouse, and Apple’s mouse, you would not see the serial reproduction of an object. You would see the evolution of a concept.” —Malcom Gladwell, Creation Myth [Source]. An idea must be built upon, for it to not be stolen. Developed, adapted, improved.
As Isaac Newton said,
“If I have seen further it is by standing on the shoulders of giants”
But even this was adapted from another,
“Bernard of Chartres used to say that we are like dwarfs on the shoulders of giants”
— John Salisbury.
Creation requires influence (Kirby Ferguson, Everything is a Remix). The forces that shape our lives can’t be attributed to individual owners; we are the product of everyone before us. In order to see beyond what we know, we must stand on their shoulders.