Understanding Video Codecs: H.264, H.265, and AV1

What a codec actually does, how the three main ones compare on efficiency and licensing, and why hardware decode support decides everything.

A codec is the thing that makes video small enough to send over the internet. Raw uncompressed 1080p video runs at roughly 1.5 Gbps, which no consumer connection could carry and no reasonable storage could hold. Compression takes that down to something in the region of 5 Mbps, a reduction of about three hundred to one, and it does so while remaining watchable.

Understanding roughly how that trick works explains most of the practical questions people have, including why a file plays on one device and stutters on another.

The Basic Idea

Video compression exploits the fact that consecutive frames are nearly identical. In a two-second shot of someone talking, the background doesn't change at all and the only difference between one frame and the next is a mouth moving.

So rather than storing every frame, a codec stores occasional complete frames and, in between, only the differences. Complete frames are called I-frames or keyframes. The difference frames come in two flavours: P-frames, which reference what came before, and B-frames, which reference both before and after and are therefore the most efficient of the three.

The group of frames between one keyframe and the next is the GOP, and its length is a trade-off. Long GOPs compress better. Short GOPs let a player start decoding sooner and let a stream switch quality more often, which is why adaptive streaming constrains GOP length rather than letting the encoder choose freely.

Within a frame, the codec breaks the image into blocks, describes each block by how it differs from a prediction, and throws away detail the eye is unlikely to notice. That's the lossy part. Push it too far and you see the results: blocking, banding in gradients, and the smeared look that fast motion gets at low bitrate.

Newer codecs are better mainly because they can use more block sizes, more prediction modes, and smarter maths to decide between them. That flexibility costs computation, which is the recurring theme in everything below.

The Three That Matter

Standardised in 2003 and still the most widely deployed video codec in the world. Every device made in the last fifteen years decodes it in hardware. Every browser plays it. Every encoder produces it.

It's noticeably less efficient than what came after, needing perhaps 40 to 50 percent more bitrate than AV1 for comparable quality. Nobody chooses it for efficiency. They choose it because it works absolutely everywhere, and for a lot of services that reliability outweighs the bandwidth cost.

If you're distributing video and can only ship one codec, this is still the one.

Arrived in 2013 with a genuine improvement, roughly 25 to 50 percent better compression than H.264 depending on content and how hard you tune it. It's the codec behind most 4K and HDR distribution, and hardware decode support is now broad across TVs, phones and streaming boxes.

Its problem was never technical. HEVC's patent licensing fragmented across several separate pools with overlapping claims and unclear total cost, and that uncertainty made large distributors wary. It's why browser support took years to materialise and why HEVC never displaced H.264 on the open web the way its technical merits suggested it should.

For 4K delivery to televisions and mobile apps, it's well established and works well.

Released in 2018 by the Alliance for Open Media, a consortium including Google, Netflix, Amazon, Microsoft, Mozilla, Apple and others. Its defining feature is that it's royalty-free, which is a direct response to the HEVC licensing situation.

Compression is around 30 percent better than HEVC and 50 percent better than H.264. That's a substantial saving at scale, and it's why the large streaming services adopted it quickly for their own delivery.

The costs are real, though. AV1 encoding is computationally expensive, historically far more so than HEVC, though encoders have improved a great deal since launch. And hardware decode support only appeared in devices from roughly 2020 onward, which means a large installed base can only decode it in software. Software AV1 decode on an older phone or a cheap TV stick means dropped frames, a hot device and a flat battery.

Comparison

Hardware Decode Is the Whole Story

This is the part that matters for anyone troubleshooting rather than architecting.

Hardware decoding means a dedicated block of silicon handles the video. It's fast, it uses very little power, and it doesn't trouble the main processor. Software decoding means the CPU does the work, and for modern codecs at high resolution that's a heavy load.

The symptoms of software decode are easy to spot once you know them:

• Playback stutters or drops frames, particularly in busy scenes

• The device gets hot

• On a phone or laptop, the battery drains unusually fast

• Fans spin up on a device that's normally silent

• Audio continues cleanly while video falls behind

A device can only hardware-decode the codecs and profiles its chip supports. That's a property of the silicon and cannot be added with a software update. A 2017 TV will never hardware-decode AV1, no matter what firmware it receives.

This is also why a file that plays perfectly on a laptop stutters on a TV. It's the same file. The chips are different.

Profiles and levels are a further wrinkle. A device might support HEVC Main but not Main 10, meaning it plays 8-bit HEVC and fails on 10-bit HDR content. These distinctions cause a lot of confusion because the device's spec sheet says "HEVC" and the failure looks arbitrary.

What This Means In Practice

If you're choosing a device, check what it decodes in hardware rather than what it claims to play. AV1 hardware support is the current differentiator and will matter increasingly over the next few years.

If you're troubleshooting stuttering, and it only affects certain content, suspect the codec before you suspect the network. Network problems don't usually care what you're watching. Codec problems care very specifically.

If you're distributing video, ship more than one codec and let each device pick. The manifest declares codecs per rung and players filter on what they can handle. Practically that means H.264 as the universal fallback, HEVC for 4K to televisions, and AV1 for devices that can take it. It costs more to encode and store, and it saves substantially on delivery for the portion of your audience that can use the newer codecs.

If you're encoding your own files, H.264 for anything that needs to play anywhere, HEVC for 4K where you know the target device, AV1 only if you know the playback device handles it in hardware and you care more about file size than encoding time.

What's Next

VVC (H.266) improves on HEVC by roughly another 30 percent and arrived in 2020, but it carries the same licensing complexity that held HEVC back, and hardware support is barely beginning. It may find a home in broadcast rather than internet delivery.

AV2 is in development under the same royalty-free model as AV1.

Neither will matter for several years, because codec adoption is gated by the replacement cycle of televisions and phones rather than by the standards process. A codec becomes viable roughly five to seven years after hardware decoders start shipping, and not before. That's the real timeline, and it's why H.264 is still, twenty-odd years on, the safe answer.