Pixel Counterrr

If there’s one singular aspect in computer imagining to which the credibleness of the image comes down the most, it is how faithfully the behavior of light is depicted, there’s no question about it. To put it in terms used in my thesis, it’s indeed ultimately the simulation of light – or lack thereof – that has historically made computer-generated imagery unconvincing for the human eye. However, there’ve been enormous breakthroughs during the past decade or so in this particular field, which have led to near photo-realistic representations.

In my mind, the most prominent advancement in terms of realistic lighting has been the emergency of so-called global illumination (GI) techniques, which take account, in short, not only the light emitting directly from a light source, but the bounced light, i.e. indirect illumination, as well. Consequently, every surface the light touches becomes a light source in itself ad infinitum, which unsurprisingly makes the situation rather demanding, to say the least, in terms of needed CPU cycles.

Of course, there’s not enough CPU power in the world to calculate the exact results of genuine global illumination, so the process – as in modeling at large – has to be optimized somehow, like limiting the number of bounces and the overall definition of the solution. So, today we do have highly optimized GI algorithms out there that take a reasonable time to render even in real-time, producing somewhat credible results, but which are still, to my knowledge, completely absent outside the tech-demo context.

Since real-time GI is basically out of the question on today’s hardware and use cases, most games use some kind of static indirect illumination to provide that much needed realism to the overall lighting scheme. A game that notoriously ignored indirect illumination altogether was super dark Doom 3 (2004) which was perhaps more of a proof of concept from John Carmack that a game can be realized with a fully dynamic lighting system. As a result, Doom 3 looked exceedingly artificial and, as said, extremely dark making it hard to make sense of what was going on at times. Carmack did backpedal with Rage (2011) of which lighting approach was more of a practical/aesthetical than ideological one, which in part made Rage one of the better looking games of its genre.

However, a direct opposite to Doom 3 and a prime example of beautifully used static GI is Mirror’s Edge (2008) of which aesthetics relied heavily on the effect. In fact, I would argue that the use of the high-quality and quite realistic GI solution, allowed Mirror’s Edge to employ otherwise more abstract and stylistic visuals, such as the completely white foliage and the extremely clean and sterile look overall. The realness of the visuals didn’t stem from the geometry nor the textures, but solely from the indirect illumination, even if being static.

In the light of all this, I consider that, for example, hardware tessellation that allows ultra refined geometry is a completely redundant direction to go as long as there are these fundamental limitations in the field of light simulation. It’s not about polycount anymore and, in a sense, it never was.

It’s increasingly about the need for genuine, dynamic GI solutions, and I can’t wait to see what the next generation has up to its sleeve regarding this. Hopefully something.

There are a few developers in the video game space who go above and beyond the call of duty when putting a game together. I’m referring to developers who radiate deep, otherworldly understanding towards the medium in question, and possess enough ambition, technology, and talent to pull the developer’s vision off, more or less accurately.

One developer who qualifies as such in my eyes no questions asked is Polyphony Digital, responsible of the Gran Turismo series released exclusively on the PlayStation platforms. In short, GT series could be described as one long, meticulously calligraphed love letter to the automobile industry and everything related, or, conversely, an ultimate, yet affordable fantasy for car enthusiasts at large.  And while the people playing the games sometimes tend to cut corners, Polyphony Digital is definitely driven not to, which comes crystal clear when scrutinizing the newest installment in the series, Gran Turismo 5 released on the PlayStation 3.

Granted, GT 5 suffers from a few painful compromises, not least of which being the dreadful decision to include tracks and cars from the previous iteration only to bump up the numbers printed on the cover. Yes, I really hate the solution as it renders the visuals inconsistent, to say the least, and I’m all about consistency. I believe Polyphony Digital CEO Kazunori Yamauchi hated it too.

What wasn’t compromised in GT 5 was the simulation of light, which is, I would argue, very close to optimal the hardware in question can handle. The beauty of it doesn’t lie in the definition of the simulation per se (e.g. shadows in the cockpit can be quite jaggy), but in the profound understanding of how light interacts with various materials and optical apparatuses such as a camera or an eye in terms of exposure. Indeed, when simulating an entity or a phenomenon, everything is done in vain if one doesn’t grasp the core nature of the object being simulated, and GT 5 seems to avoid that shortcoming to a great extent. The lighting obviously knows what it’s doing.

Furthermore, the aspect that strikes immediately with the lighting is the extremely clean, non-obstructive look of it. The thing is, the extensive use of bloom, color grading, and obnoxious lens flares (there are good ones) are usually cheap strategies to salvage the image from the mediocrity. Bad post-processing is indeed like an old, wrinkled lady trying impress people by plastering her face with a tons of make-up. GT 5 stands on, in addition to the excellent modeling and texturing work, such a robust lighting solution, both precalculated and real-time, that it needs no saving make-up to look appealing and, above all, real.

However, the ultimate testament and benchmark for the GT 5’s rock solid lighting model is the Photomode with which one can produce near photorealistic images. The Photomode is like a window to the future and PD’s nod to the gaming community that “If we just had more horsepower available, this would be the level of realism we’d be dealing with now. We understand light. We have the technology.” Indeed, no other game comes even close to GT 5 as far as photomodes go, not even Forza Motorsport 4 by Turn 10 Studios.

As said, the beauty of GT 5 stems obviously from other factors too, such as the texturing and modeling work, but in the end, everything comes down to the simulation of light. There’s no way around it. And it’s exhilarating to see that there are developers who get that exceedingly well.

Given the somewhat stalled rate at which consumer real-time imagery has been evolving the past half-decade or so, it’s hard to imagine now that there was a period of time when one could basically pinpoint a game to a specific year of release by the mere looks of it. Consider, for instance, the consequential jumps in visuals from Wolfenstein 3D (1992) to Doom (1993), and further on to Descent (1994) – all this in a span of 31 months.

For reasons beyond me, Quake released in 1996 is often attributed as the first commercial engine that popularized the full 3D paradigm in a first-person context even though it was Descent that did the same exact thing 18 months prior. Sure, the Quake Engine was undeniably more sophisticated in a number of respects, but Descent was de facto trailblazer of that short period of time between so-called 2,5D (aka pseudo-3D) such as Doom and full 3D such as Quake.

On a side note (and in all fairness), the Ultima Underworld games released in 1992 and 1993 did were very closely in par with Descent judging solely by the list of supported features. However, the UU graphics engine was simply so far ahead of its time that a) there wasn’t hardware available that ran it at aesthetically pleasing speed, and b) the engine itself was a bit of a mess and far from the robustness of the aforementioned games, such as Descent. Therefore, I feel the UU engine was a bit “cheating” its way to the avant-garde position it held a couple of years.

So, not only Descent expanded the simulation of (texture-mapped) space into six degrees of freedom and introduced actual 3D enemies instead of sprites to the first-person genre, Descent employed a dynamic lighting system that was rather mind-boggling at the time. In contrast to Doom and the like, in addition to the static lighting Descent allowed traversing light sources that rendered shooting lasers and missiles quite a spectacle, especially in narrow corridors which Descent was known for.

It was indeed the shadowy tunnels that really showcased the dynamic lighting in action, and I remember reading at least one review of Descent at the time raving about how impressive the setting in question was. Furthermore, I’m pretty sure the exceptionally dark and lenghty tunnels found here and there in Descent were included into the level design precisely to bring forward the then-novel lighting system, and as such, successfully so.

Sometimes there is a distinct moment when a new technology establishes itself and becomes an actual, tangible reality in one’s mind. For me, that dark tunnel in Descent that lighted up as the missile glided through was one of those defining moments, and even if the effect was rudimentary at best, it was a substantial step towards dynamic illumination in a real-time context at large. Light sources simply couldn’t be completely fixed anymore from then on.

As said, 18 months later Quake did ended up pushing both static and dynamic lighting forward making Descent visually obsolete. Funnily enough, it was this time around the static lighting that impressed me the most, particularly the unprecedented, if static, cast shadows in the environments. Interestingly it took in turn almost a decade to make such shadows move and react in real-time.

For me, following and contemplating real-time imagery, and technology in general, is all about moments of realization that something that wasn’t considered feasible is now just that. And it can be a darkest tunnel that shines light on that new reality.

The way an eye adapts to various lighting conditions, such as extreme darkness, I believe that an eye, or a perception, adapts in a similar manner to a certain visual principle, i.e. a means through which a given visual structure is realized. So, let me open this sentiment a little.

Consider, for instance, Lego-made structures. My argument is that the longer one observes, say, a fire truck made out of Legos and becomes familiar with it, the more the “representative fascination” of the object start to wear off and non-representative aspects begin to rise to the surface, like the Lego bricks themselves and details (scratches, Lego logos) related to them. It’s indeed the visual structure behind the representation that starts to emerge when the representational layer loses its illusive charm through familiarity. In the end, one cannot escape the fact that the Lego fire truck is just a collection of Lego bricks put together to make it appear as a fire truck, but not being a fire truck at all, how ever self-evident it may sound.

What, then, my ultimate point is and what led me to this line of reasoning were these numerous stormy sequences in video games of late that have had a particularly profound aesthetic effect on me in terms of visual realism. And it’s not obviously just me: GameTrailers, for instance, used exactly the thunderstorm imagery from Modern Warfare 2 in their review to emphasize the good looks of the game. Also the first Modern Warfare was first demoed by using the stormy intro sequence with a lot of rain and lightning. And the mother of all video game trailers, the Metal Gear Solid 2 reveal video, did the same exact thing back in 2001.

So, I believe it’s really not so much the rain component of the storm that makes such visuals so convincing, but the lightning that illuminates the scenes seemingly in random just for a couple of frames. And it’s the flashing lightning that interrupts the aforementioned adaptation, i.e. process of familiarity of given visual structure. This is especially true in cases like MW 2 in which the lightning casts real-time shadows on the environment, completely altering the visual landscape for a moment. Consequently, the polygons, textures, and shadow maps gain back enough of their representational power to trick an adapted eye, so it’s the fire truck once again instead of Lego bricks, even if just for a little while.

When talking about lighting, or in this case, lightning, it often comes back to the original Doom. As I previously wrote, what made Doom particularly scary was the back-then realistic illumination scheme, and I may add that the situations where Doom’s visuals were most credible had to do with flickering, strobe-like lighting conditions.

Of course, this kind of an effect has to be employed wisely, and in no case overused, otherwise the effect loses its appeal rapidly. Additionally, there should always be a natural phenomenon explaining the effect, such as a malfunctioning fluorescent tube in Doom or a lightning strike in MW 2, in order to work as described.

From time to time a game comes along that simply radiates unconditional love, technical understanding, and pure commitment towards the craft and the medium in a way that borderlines magic, for the lack of a better word. For me, the first Dead Space by Visceral Games was one of those titles and the sequel Dead Space 2, released three years later, pushes even further the brilliance of the first one, at least technology-wise.

If a survival horror game, which Dead Space games very much are, lives or dies by something, it’s the way it handles the lighting. It was indeed the lighting and the semi-dynamic nature of it that made, for instance, the original Doom so effective at the time. Without it, we wouldn’t reminisce Doom the half we do now, that’s for sure.

So, paradoxically, it takes credible lighting to convey credible darkness. And it’s the inherent fear of the dark in all of us that renders darkness so useful and integral to fear-inducing settings: We need to see our surroundings in order to survive, and we fear more than anything our own demise. It’s as primal as that.

Luckily, of all the cool aspects and details both Dead Space games share, above all is the overall lighting scheme, which is simply put excellent. Yes, Dead Space games don’t include any sophisticated real-time radiosity or ambient occlusion solutions, but the point is Dead Space games do the basic stuff exceedingly well with solid technology and proper art direction.

What strikes me the most about the lighting is the degree at which the player can interact with it, meaning almost every light source that looks moveable is indeed moveable. It really is amazing to pick up, for instance, a light emitting flashlight from the scenery that would be in all probability a static object in some other game. In fact, I barely believed my eyes when I did so at the first time and saw how the flashlight kept illuminating and casting well-defined shadows onto the environment, even if it was only a random prop laying around. It’s exactly this kind of attention to detail and consistency that raise a game above the mediocrity.

Then there are lens flares, especially in Dead Space 2, that make one’s eyes water out of pure visual enjoyment. Lens flares can be a distraction when done wrong, but Dead Space 2 pretty much nails the use of them. There are number of nice little touches everywhere in the game, such as the delicate light-streak on the front of the helmet that is visible only from certain angles. One scene in particular in which the lens flares are employed to “blind” the player is really impressive, which is further enhanced with a subtle, barely noticable depth-of-field effect. Beautiful and effective.

And everything look so effortless for the graphics engine to deal with, which is especially true with Dead Space 2 which comes across as one of the most optimized PC games I’ve seen for some time. It seems to escape many developers that beauty and performance goes hand in hand when dealing with real-time imagery, and optimization is definitely something to not overlook. Visceral Games has proven to not to be one of those developers.

In the vain of the previous, somewhat pessimistic, post, it is only apt to discuss now cases where dynamic lighting performs extraordinarily well, one of which being Valve Software’s Portal 2.

The first Portal was a massive critical success and received huge amounts of good will from the gaming community, not least for the virtue of coming basically from nowhere as an “extra” in the Orange Box, taking everyone by surprise with its novel gameplay and a twisty storyline. One can, thus, only imagine the overwhelming pressure to succeed that must have taken place when making the sequel, Portal 2.

Unfortunately, the will and eager to top the first one shows what comes to the storyline (I’ve never cared about scripted stories anyway), but visually, it’s light-years ahead from the first one, as it should.

Indeed, Portal 2 is filled with intriguing visual details and design decisions to the point that it depresses me to single out just one for now, but, as said, one area in which Portal 2 shines the most is the overall illumination scheme. Technically the lighting consists of carefully placed dynamic lights with dynamic shadows and complementary pre-calculated static lighting, and on the surface there’s really nothing super-fancy about it.

So, even though real-time shadows per se in Portal 2 aren’t groundbreaking by any means, the impact they had on me in terms of pure aesthetic pleasure was considerable nevertheless. The intro-sequence in particular is something of a showcase, presumably by design, for the new lighting system (and for the pre-calculated physics) that differentiates Portal 2 immediately from its predecessor, conveying a message that this time Portal 2 is not a cutesy little side project, but a serious full on retail product.

In addition to the dynamic lighting, the geometry in levels, such as platforms and wall plates, are heavily animated this time around, which wasn’t the case with the first Portal. Of course, since Portal barely contained any dynamic lighting, it would have been problematic, to say the last, to move geometry around to the equal extent as in Portal 2. One function of the dynamic lighting in Portal 2 seems to be indeed to obstruct the static lighting in dynamic objects so that they blend in to the environment when they change their orientation and/or position. It’s like aesthetic cement that smoothes everything out in the end.

Besides being enchantingly beautiful in itself, Portal 2’s visual landscape is a testament for what a few, or in Portal 2’s case, usually just one, real-time light source casting shadows can do to a scene when implemented right.  Of course, Portal 2 benefits from the unreal and dreamlike environments, in the sense that the lighting doesn’t have to replicate often-convoluted natural world scenarios, but that of highly contained and stylized indoor settings instead.

And everything would be done in vane if the performance failed to deliver, which isn’t even remotely the case with Portal 2. Thanks to the art direction and partly gameplay mechanics, the levels in Portal 2 are often lightweight in terms of geometry due to their blocky and abstract architecture, which allocates resources to other things, such as real-time lighting and physics. Consequently, Portal 2 runs actually better than the visuals would let one assume, which is always an ideal scenario when dealing with real-time imagery.

It’s almost redundant to state that John Carmack, the co-founder of id Software, is one of the most important creative minds that have ever pushed consumer real-time imagery forward. Of course, Carmack is most known for coding the graphics engines for the Wolfenstein 3D, Doom, and Quake games that each elevated the visual fidelity of simulated real-time space to a new level.

In terms of pure technology, Doom 3 has been to me the most interesting id release yet, although it received mixed reactions back in 2004, to say the least. As a game, sure, it left much to hope for, but as mere real-time imagery, Doom 3 is a pretty fascinating piece of work.

For one, as far as I know, Doom 3 was one of the very, if not the, first commercial games to include a completely dynamic lighting model with real-time shadows, and in fact, still remains to be one of the few. The idea of fully dynamic lighting is indeed beautiful and something to be proud of, but it did come with a cost – a pretty considerable one.

We all know that indirect illumination is a bitch to simulate accordingly in real-time, and since Doom 3 didn’t contain any, it was for that very reason dark as, well,  Hell. In Doom 3 if something wasn’t lit directly by a light source, the area in question remained 100% dark, which is, of course, highly unrealistic and aesthetically almost unbearable. So, even though the lighting model was consistent and performed well on paper, Doom 3 was indeed deemed too dark at the time and mods that attached a torch to the weapons surfaced rapidly, bringing more precious illumination into the darkness.

All things considered, one could say that Doom 3 (or id Tech 4, as the engine is called) was most uncompromised graphics engine that id has produced yet, in that it didn’t “fake” anything but did everything in real-time, which ended up being its weakness.

So, it’s interesting to see id now going back to semi-static illumination with their upcoming title RAGE, considering such lighting was last seen in an id game back in 1999 when Quake 3 was released (Quake 4 was an id Tech 4 game and not developed by id). Consequently, RAGE does look much better on the surface than any id Tech 4 game due to the pre-rendered indirect illumination, but ideologically it’s a step backwards.

The more I think of Doom 3, the more it comes across as a proof of concept. That Carmack had this grande idea of fully dynamic lighting solution and then ran with it, ignoring all the aesthetical problems (and of such related to realism) it created in the process. I do believe dynamic systems are worthy in themselves within real-time imagery, but simulation of light isn’t just there yet – even today, let alone in 2004 – that it could be adequately handled fully in real-time.