Put the Sci in your Fi—Enhance your Image Enhancement

Hello everyone! Welcome to the first Put the Sci in your Fi post of 2019! This is a particularly special entry because it’s the first guest article for this blog topic! So, without further ado, I’m super stoked to introduce illustrator and graphic designer Alex Ferri Land (check out her website for some adorable art). With a decade of experience in the gaming industry, her post will highlight common misconceptions about image analysis as well as discuss the limitations of technology and what alternative options exist.

Enhance your Image Enhancement

The usual offense:

Our Detective / Cop / Space Captain has their game changing surveillance footage / photo / screen Display up. Somewhere in that image is some crucial bit of information. But it’s so small! If only they could see that tiny piece of the picture more clearly…

They zoom into the spot and see a blurry close-up of the area in question. So far so good. Until… “ENHANCE!” Now through some means of magical, non-existent tech, the zoomed in section of image becomes crystal clear! We can now plainly see the criminal’s face / street sign / name on a tiny slip of paper in the pocket of the dude in the reflection of the eyes of the tiny lady in the background…

Here’s an example:

Except, this isn’t a real thing.

But why is this approach wrong?

First let’s understand how digital media works.

Screens display images via pixels. But what are pixels? Imagine your screen has a grid on it. As an example, my laptop’s screen has a grid 2560 squares wide by 1600 squares tall. To make anything appear on my screen, those squares fill with color to form images and text. Those squares are pixels.

Now, there are limitations to how many pixels can be displayed on a screen. Every screen is different, and the number of pixels capable of being displayed on any screen is finite. There’s no adding more pixels to a screen. It’s a physical and concrete limitation. Understanding that makes the rest easy.

Let’s look at some visuals:

100%) This is a 30×30 pixel section, occupying an extremely small section of the screen. Let’s ZOOM IN (not the same as “Enhancing.” We’ll get to that next).

500%) You can start to see what that is. It’s a duck! “Quack!”

3200%) We can really see the duck now. It’s very pixelated. Even the grid is visible. Within that 30 by 30 pixel area, certain pixels are filled in with color to create our duck friend.

Now let’s see the “same” duck but at a higher pixel density:

1. A) Our same 30×30 pixel duck. (Check out that red line grid! 30 squares by 30 squares.)
2. B) Here I recreated Mr. Duck in a 500 x 500 pixel square. Look at the difference: the grid is so tightly packed the red lines all merged together. If you zoom in on his eye, you can see that black area alone is 15 pixels wide!

The point of the image comparison is that in Image A (30×30 pixels) you have to zoom in 3200% to see the same sized duck as Image B (500×500 pixels) at 100%. The higher the pixel density, the more details you can put into an image and the more you can zoom in and still have clear details.

Okay, but what about “Enhancing?”

So now, let’s say a character has obtained an image and some small part of it needs to be zoomed into and examined. The closer the zoom, the more pixelated and blurry the image becomes. With a wave of a wizard’s wand, suddenly the zoomed in section becomes clear! Right? Well…not really.

Let’s explain with visuals:

In both these images, the person circled in red is a vital suspect in a thrilling crime!

But they’re so small! Who are they? Let’s ENHAAAAAANNCE!!!!!

1. A) Who is that figure way in the back there?!
2. B) Let’s zoom in!
3. C) Let’s zoom in further… There he is! The scoundrel!
4. D) He’s blurry, so let’s ENHANCE! Wait… there is no D…

Where’s my clear, de-pixelated, detailed version of the criminal?

No D again… Where is the next image where his face is suddenly clear and detailed?

Well, there isn’t any. Technology simply isn’t there yet. The pixels that make up the image is all the information it has to offer. Those colored squares in that arrangement are as detailed as we can get with that image. There’s no more detail to be had. Sorry, detective!

But what’s implied when super computers “enhance” the images after that blurry zoom on TV? In the fictional world, the computer has some means to figure out what the image would look like if the photo had more detail (pixels) in it than it really does.

Unfortunately, computers are just not that smart yet. Here’s what really happens if a computer tries to fill in the details in a small image that’s made larger.

Here’s a real life Enhance with our buddy, Sir Duckerton:

100% 1) Our original 30 x 30 pixel duck.

100% 2) Instead of zooming this time, I’ve asked my computer to actually turn the 30×30 grid into an 80×80 grid. It’s adding new pixels to the image and filling them in with its best guess of what belongs there. This isn’t a really dramatic enlargement so he still looks alright.

100% 3) Here he is after I’ve asked the computer to take the 30×30 pixel image and make it a 500×500 pixel image. He’s so blurry and sad!

What gives? Computers are just not good at guessing what colors to fill in new pixels with and retain the image’s original integrity. They become blurry messes, and definitely can’t reveal hidden information from a process like this.

The Conclusion / Review:

Taking a zoomed-in section of an image and asking a computer to make it clear, sharp and full of accurate detail (especially a human face or text or some vital plot detail) is not possible with current technology. In other words, “Enhancing” is just not a thing.

Let’s look at TV Enhancing one more time to recap:

And now, with a better understanding of pixels, it becomes clear that this new hidden image doesn’t make sense. C is simply impossible.

Let’s explore some alternatives!

What can I do instead?

Some suggestions include:

• “Zoom In!”

If the “there’s some hidden small detail in this image” trope is essential to the story, make sure to establish that the image/video was taken at a very high resolution. An extremely high pixel count on a tiny screen will allow for tons of zoom into an image while maintaining a nice amount of detail. So, just make sure the hero has a good camera. (That could even be a plot point! That the characters themselves understand the value of high resolution and bought good equipment.)

• Switch up how the characters are looking at the image.

Maybe, when rotated on its side, removing all the color in the image but one, desaturating the image and upping the contrast, or any other number of totally possible photo edits available right now, a hidden detail is revealed. Look at what IS available in technology currently and work within those boundaries.

As an example: There was a great old detective game where one of the antagonist detectives wore a red visor and didn’t notice red bloodstains in the snow until the main character was able to see the scene. It made for an exciting little twist!

• This is the future / It’s space-tech / Shut up, it’s magic.

Hey, maybe the image is being enhanced by aliens with super magic future-space-tech. That’s cool. Just establish that the characters are working outside the realm of modern Earth technology.

I hope this helps create a better understanding about how to use real life “enhancing” photo tech in stories to create a convincing and more satisfying payoff!

Advertisement