A few things - this is one step in a long, LONG path. AV2 is currently unusable in its current state (the encoder typically runs at around 1fps on good hardware), and likely will remain so til ~2028 when the first av2 hardware accelerated chips start dropping. Even then, I wouldn't expect AV2 streams to be common til 2030.
IMO, if it were just the efficiency gains on the table (which are substantial - ~20-30% over AV1), I'd say that AV2 isn't worth it. The biggest thing it does add though is multi-stream support, which will be a big win for VR and live sports. The other fun thing is you can send an alpha channel as a separate stream, which the file will then composite for proper transparent video support.
Based on AV1's trajectory, hardware encode isn't necessary (though it is nice). The current encoder is a reference encoder. Now that the spec is finalized, expect significant speed improvements from production encoders (realtime likely won't happen until we get it in hardware though)
I strongly disagree with it not being required. I run a small social news site - AV1 is still prohibatively expensive both for the server and clients for software encoding/decoding. Without hardware encoding, the tradeoff for better compression ratios in exchange for massive battery use + very long processing times for encoding simply isn't worth it. In order to get AV1 out, I have to often process a h264 version of a video first anyway, just so the client isn't left waiting for their video upload to finish encoding. This means to support AV1 I'm not saving anything on the storage side. Even youtube only does AV1 encodes for extremely popular videos - it only makes sense to do at significant scale.
I love AV1, don't get me wrong, and I can't wait til I can switch over to it as a single unified format for both images and video, but for now the cost is too high until hardware acceleration becomes ubiquitous
Hardware encode is required if you want things like video calls, camera recording and such to use it.
It isn’t required for content distribution platforms which aren’t realtime and the cost of encode is easily made up by hundreds of thousands of streams.
One of the interesting usage of AV1 was specifically for low bitrate calls, and software encoding was perfectly fine, even on mobile.
With low enough resolution, framerate and bitrate, you can get a quality stream without significant encoding artifacts compared to any other codec. It is in production right now and has been for a while.
The tradeoff CPU / bandwidth is quite advantageous in situations like this. And no, AV1 HW encoders cannot usually be used, they are not designed for a tight bitrate control or realtime communications like software encoding is usually.
> One of the interesting usage of AV1 was specifically for low bitrate calls, and software encoding was perfectly fine, even on mobile.
You really want hardware decoding on mobile, otherwise you end up with 40 minutes battery life. Fortunately, for typical videoconference resolutions, VP8 and H.264 are just fine. AV1 is nice to have, though, due to excellent support for synthetic content (screen sharing), and for scalable video coding (a much more elegant solution than simulcast, IMHO).
In the world I live in, the general plan is to stick to VP8 and H.264 for the time being, and to skip to AV1 when it's universally available on mobile. I haven't seen any features of AV2 which would justify waiting for it.
Have you said this for Audio Codec I would have agreed. I do not know a single Smartphone Video Conferencing software that uses CPU encoding rather than hardware encoding. Neither WhatsApp or FaceTime, perhaps the largest of the two real time Video Call uses AV1.
Yeah, no production or large scale VC system is running software AV1 encoders on smartphones. You will drain a full phone battery in 1-2 hours of calls.
It just doesn’t make sense and will result in extraordinary power/battery drainage at best, or output that’s worse than hardware encoding.
The only way you could get AV1 to software encode in realtime AND low latency on a mid-range Android chip is by disabling or skipping nearly all of the compression/encoding features that make it good at low bitrate.
> Yeah, no production or large scale VC system is running software AV1 encoders on smartphones. You will drain a full phone battery in 1-2 hours of calls.
Yeah but, half jokingly, Zoom does that (draining the battery in an hour) already :P
So, status remains quo, the commons remain tragic, and glory to H.264 forever?
At least until a better codec has widespread enough hardware support, I think.
[dead]
Anything running on a battery will need hardware acceleration
Even without encoding, as long as decoding is supported for AV2, streaming sites like Youtube can always transcode uploads. The encoder on mobile hardware is more of a nice bonus as long as we have an AV1 encoder available in the meantime.
> The biggest thing it does add though is multi-stream support
I would have thought this would be a part of the container format rather than the video codec?
Google H264 SVC
The way things are going, we can pretty much forget about AV2 hardware encoders in PCs anytime soon. All the newest, best chip capacity is being completely hogged by Apple and AI companies.
Unless chipmakers port the AV2 design to older, cheaper nodes, it’s just not happening for average users. We’ll probably see some Chinese TV chip makers throw in an AV2 decoder just to check a box, but as an actual encoder? I wouldn't count on it anytime soon.
I wouldn't be so pessimistic, Intel and AMD aren't going to stop making CPUs, and if their integrated graphics adds AV2 it will be motivation enough for others to follow.
Ram and Ssd is just the start capitalism dictates profits will suck all production to AI. CPu have 3-5 year production timeline ie cpu prices will start going up a lot more as previous production contracts expire you might even get shortages new consumer cpus won't be affordable or spread as fast as you think.
PCs don’t need hardware encoders. There are no realtime jobs on these. You can let it encode in however long it takes.
You need hardware encoders for things like cameras because they need to encode in real time since the buffer would quickly overflow otherwise.
Use cases for hardware encoders in PCs:
- Video calls
- Screen/webcam recording
- Live streaming
- Real-time transcoding for media servers (don’t know much about this but I’ve heard it’s a thing)
- Game streaming
- Video editing (making exporting less frustrating)
Everything here is really niche, except video calls (and even that...).
In other words, unless on smart phones, don't expect broadly distributed AV2 encoding hardware.
If it does happen on PC, it will be most likely some courtesy of the hardware chip designers.
Screen recording is not niche, anyone who works remotely probably does video calls daily and the others are somewhat more niche but I'd guess that at least 50-70% of people do at least one of these things
Video calls are niche, and offline encoding of video isn't?
Are you sure this isn't just “things I do are commonplace, and things I don't are incredibly niche”?
Looking at how GPU development has been sidetracked for NPU, I worry that this is 2035 target at best. Manufacturers will push for maximising matrix operation silicon area. In the era of trillion dollar investments into datacenters, traffic cost is afterthought. The only benefitors might be YouTube, Netflix and such, but on their scale investment into ISP level caches might be cheaper.
> enabling high-quality video delivery at significantly lower bitrates
> likely will remain so til ~2028 when the first av2 hardware accelerated chips start dropping
This might sound dumb, but whats the point if its intended for slower devices, but those slower devices don't even exist yet?
It's not for slower devices, it's for lower data transfer bills for providers like YouTube.
It's a win for consumers as well if you can get better video quality or more reliable calls on a slower connection.
so that new devices can adopt it
They can't adopt it if it doesn't exist.
And what about old devices? I'm sure someone out there is still using an s5 as a daily driver... Future proofing is great and all, but 240p on modern devices looks like trash, even worse than tube tv.
240p for someone’s webcam thumbnail in Microsoft teams is perfectly sufficient though.
They can keep using whatever they're currently using.
This doesn't take away anything. It's a new standard.
Based on your argument, one should add new safety standards to cars like seat belts, because old cars might not have them.
one shouldn't*
One shouldn't add new safety standards*
writing before coffee...
To be fair, some municipalities do actually require old cars to be fitted with seatbelts... air bags, not so much, because apparently changing a steering column is too hard?
Well, I just noticed I said 'should' and not 'shouldn't' which I wanted to say
But to your point,
While they might not be required to retrofit, one shouldn't stop defining new safety standards.
In HW accelerated chips, what part of the calculations they usually accelerate? Could it be possible to repurpose old HW?
Essentially all of the processing of the video data, barring the container format which the CPU uses to know what part of the data to send to the GPU or the Audio chip or codec.
And HW acceleration is generally a preset baked in version of the encoder or decoder. These are mostly codec specific.
So, no using hardware from previous versions.
Now, you can see some software that tries to use the GPU itself, instead of the dedicated hardware acceleration, to decode, but that isn't the HW accelerated, and may not operate in real time.
At the same time, that will consume much more power, eliminating some of the advantages or the pure HW rendition, especially important for mobile.
I could see an argument being made for encoding, if it is 2x or faster than the CPU, but I haven't looked at any in a while, so don't know the speeds.
One of the biggest gains of having dedicated hardware is that the computation doesn’t happen on the general hardware.
This is what makes it viable on mobile devices where system responsiveness and power efficiency are high priority.
Generally these hardware decoders haven’t been retoolalble.
Typically things like quantization and motion estimation.
I feel like in 2030 it's more likely that we send 480p and just upscale with ai on the other end
480p ? More like 200i. There's a race to the bottom driven by those "up to" codecs.
That's fine and not anything new for codecs, they always take a long time before mass adoption.
Take a look at AV1 itself, you can't even say it's really ubiquitous on all hardware. It's quite well along in adoption compared to early days, but some mobile devices are still lacking hardware acceleration for it.
It is still ironic to me that my Steam Deck has decode AV1 acceleration, on a really old CPU/GPU combo.
AMD added AV1 encoding only in later SoCs though. Next Steam Deck will have both.
> The biggest thing it does add though is multi-stream support
This was supported in H.264 MVC but only saw real use for 3D movies on physical BluRays. With almost no content available outside that.
>........I'd say that AV2 isn't worth it.
Unless they have hardware encoder and decoder design done in parallel, otherwise it would be 2028 before a hardware block design is done and 2030 for the earliest product to ship with it. In reality I think 2031 or 2032 is more likely.
And I have been saying the same for quite some time that 20-30% for a generational codec improvement isn't worth it. I think they originally aimed at 50%, and then 40% and then 30%.
Where do you see information about the efficiency gains over AV1?
It's at the end on the conclusion slide @24:40:
https://www.youtube.com/watch?v=Se8E_SUlU3w&t=1480
> The biggest thing it does add though is multi-stream support, which will be a big win for VR and live sports.
AV1 supports it too ?
Does anyone know what is so costly to calculate in AV2?
In general they just increase the numbers on everything when they go up a generation.
e.g. if you check in 4 directions to see if you can reuse a chunk then make it check in 8 or 16.
Faster encoders will have smart heuristics on when to use these new abilities and when to skip them but the reference encoder will try everything in a dumb way to eke out a tiny win to maximize a theoretical advantage and map out the extreme best case.
I feel like these encoders that require acceleration are a big reason why good hardware obsoletes so quickly.