What Is 4K Upscaling and How Does It Work?
If you’ve tried shopping for a new TV anytime in the last two or three years, then you probably know that 4K TVs are considered the new standard for home entertainment. Boasting four times the number of pixels of Full HD TVs, 4K TVs can deliver images that are crisper and more detailed.
However, not all of the media you can watch on TV was shot in 4K. Knowing this, is there still any value in buying a 4K TV? We believe so, and that is through the technology of 4K upscaling?
What is 4K upscaling?
In 4K upscaling, an image with a resolution below 4K is enhanced to make it look better on your bigger or higher resolution screen or TV. Although this seems like an impressive piece of technology, upscaling has been around even at the time of 1080p TVs.
At this point, it’s worth distinguishing between content that is native 4K and those that are merely upscaled 4K. Native 4K content refers to content that has been shot, created, and distributed in actual 4K resolution. Upscaled 4K refers to content that is natively at a lower resolution (2K, 1080p, or 720p) and is enhanced by the display device to approximate 4K resolution as closely as possible.
Upscaling fills in the gap between most of the media available today and the clamor to watch things in 4K resolution. This applies to movies, TV shows, and even video games. The availability of upscaling technology provides compelling value to 4K TVs, even if you don’t exclusively watch Blu-ray movies or have a premium Netflix subscription.
Understanding display resolution
To understand 4K upscaling, we must first define display resolution. While it’s easy to describe resolution simply as the number of pixels in your screen, we can look at it from a more nuanced perspective. There are three factors we must consider:
Number of pixels
The simplest way of looking at resolution is through the number of pixels. For 4K TVs, this number is stated as 3840 x 2160. For comparison, a Full HD TV has 1080 x 1920 pixels. With four times the number of pixels, it’s natural to expect 4K TVs to look much better than one at Full HD.
All 4K TVs have the same number of pixels, regardless of size. This means that a 44-inch 4K TV will have the pixels packed more closely together than a 65-inch set with the same resolution. For this reason, the optimal viewing distance for larger TVs is typically farther. For instance, a 75-inch 4K TV is meant to be viewed from about 4.5 feet while a 43-inch 4K TV should only be at around 3 feet.
The optical resolution of a TV is a quality that is difficult to quantify but is more easily assessed by perception. Regardless of a TV’s number of pixels or pixel density, other factors determine how sharp its images are and how well the colors contrast with each other. In some cases, even a 4K TV displays poor optical resolution.
Knowing the different aspects of display resolution is critical to understanding how image quality is assessed. There are other factors, of course, such as dynamic range and how deep blacks are. However, those are less relevant in our discussion of 4K upscaling.
How does 4K upscaling work?
Look back at the number of pixels that a 4K display has. At 3840 x 2160, this corresponds to more than 8 million pixels in total. On the other hand, a Full HD display only has around 2 million pixels. If you’re watching content made for Full HD screens in a 4K display, this leaves a discrepancy of 6 million pixels.
Obviously, displaying 2 million pixels in a screen made for 8 million pixels will drastically reduce the display’s pixel density. To make up for this, built-in algorithms use the existing display information to “fill in the blanks.”
The process of filling in the slots of the empty pixels relies on interpolation. Although there are several interpolation methods, they all serve the same purpose of approximating the value of a variable given the values of the nearby points. This is the same principle of the human mind seeing blanks in established patterns and making intelligent guesses on how to fill in those blanks. In simple terms, interpolation is a sophisticated guessing game.
In most cases, 4K upscaling involves multiple interpolation algorithms. This is because there are multiple aspects of the image that need to be fixed – filling in the colors of the blank pixels, sharpening the outlines of the image, and softening the boundaries between different colors, among others. Contrast and color can also be enhanced with some filters.
The most common algorithms for interpolation are the nearest neighbor, bicubic, and bilinear methods. It is rare to find these algorithms used in isolation in 4K TVs. How well a TV combines these interpolation methods, as well as the parameters used, plays a huge role in the quality of 4K upscaling. As we shall see, not all TVs are equal in terms of how well they implement 4K upscaling technology.
Native 4K vs. upscaled 4K
Given how much of the media available today is still at resolutions lower than 4K, how does upscaled 4K compare to actual 4K content? The good news is that image upscaling technology has gotten so much better in the last couple of years. There is even AI-enabled image-enhancing technology called Deep Learning Super Sampling (DLSS) used in high-end image processors.
To the untrained eye, 4K upscaling should be smooth enough to be barely noticeable. However, putting a native 4K image next to one that has been upscaled will easily highlight key differences. Here are some things you can expect from an upscaled 4K image:
Artifacts of reduced pixel density
In the absence of 4K upscaling, stretching a Full HD image to 4K will result in a huge drop in pixel density. To make up for this, blank pixels are filled in with colors based on interpolation algorithms. Sophisticated as this technology may be, it is still not perfect. Even with 4K upscaling, there may be cases where digital artifacts such as rugged outlines, blurred areas, and poor contrast. To put it briefly, the optical resolution of an upscaled 4K image is lower than that of a native 4K image.
More visible in larger TVs
A higher pixel density is one of the primary benefits of buying a smaller TV, even at 4K resolution. When pixels are packed more closely together, some digital artifacts such as rugged outlines become barely visible. This is ideal for watching upscaled content. If you prefer the premium of having a large TV, make sure to sit at the optical distance from the TV to soften these digital artifacts.
More apparent when applied to lower quality images
All this time, we have been talking about upscaling 1080p content to 4K resolution. But how about 720p content? Can it also be enhanced with 4K upscaling? Yes, 4K TVs are indeed capable of upscaling 720p to 4K. However, this also means interpolating more than 5 million pixels.
As useful as 4K upscaling is, it cannot do miracles. Upscaling 720p to 4K will result in highly visible digital artifacts. This content is still watchable and is still an improvement over the original 720p resolution. It is not going to fool anyone into thinking that it’s actual 4K content, though.
The quality of 4K upscaling is one of the selling points and distinguishing factors between different TV brands. For instance, Sony 4K TVs use either the 4K X-Reality PRO or the X1 Ultimate processors. On the other hand, Samsung uses the UHD Picture Engine. It is through these proprietary upscaling algorithms that 4K TVs set themselves apart from each other.
Can images also be upscaled to 8K?
Taking it a step further, can the same algorithms be applied to upscale content to 8K resolution. In theory, it should be perfectly possible to extend the usual interpolation methods to 8K. However, this can be quite problematic considering the majority of the media available today isn’t even in 4K.
Just like in DLSS, it appears that the future of image upscaling will rely on AI. With AI-aided upscaling, image enhancement no longer needs to follow a fixed procedure. Through the use of machine learning, AI can identify low-resolution objects, compare them to higher-resolution versions, and do the appropriate adjustments to fill in pixels and eliminate artifacts.
As TVs become bigger and become capable of displaying more pixels, we can only expect more advanced image enhancement algorithms to become the new field of competition among TV brands. This will become more important in the absence of content that was shot or created in native 8K resolution. Fortunately, we’re probably still a few years away from 8K TVs being affordable or easily accessible.
If you’re planning on buying a 4K TV, the quality of 4K upscaling is definitely something you will want to pay attention to. Despite 4K being the standard for new content nowadays, you will still almost inevitably have to watch content on your TV that needs to be upscaled.
Unfortunately, it’s tough to tell how good a TV’s upscaling is until you see it for yourself. Don’t let TV displays in shops fool you – those almost certainly display exclusively 4K content at the highest brightness settings. Ask someone at the shop to play a 1080p video for you so you can see exactly how well the TV upscales the content.