It would be cool if one day (if not already today?) you could use AV1 as a drop-in replacement for h264 for recording with OBS, smoothly editing without proxy clips and rendering out highly disk size efficient videos that look good.
DaVinci Resolve's free version on Linux does not support h264 but apparently does support AV1. Kdenlive supports both. AV1 sounds like it would be a great solution for Linux if the above is possible.
I have been converting my H265 (HVEC) collection to AV1. I am still trying to find the exact parameters for FFMPEG to produce the optimal output for the devices I use plex on. Its been a little bit of whack a mole so far, but it seems ideal for file size when I get it right.
I've been transcoding my media collection leaving my PC on overnight over months, it's great. My biggest issue is client support for native playback of AV1, naturally.
For what it's worth, AB-AV1 [1] is a pretty awesome tool written in rust which compares random samples from a file at different parameters based on their VMAF score [2] (algorithm from Netflix for human-perceived visual likeness), choosing optimal parameters to save as much space as possible with the loss you're willing to stomach, on a file-by-file basis.
Small plug: I made a nice little python GUI wrapper for ab-av1 [3].
While I agree with you, I find that sometimes the “experience” can improve.
The most common “artifact” of AV1 is to make things slightly more blurry for example. A common H.265 artifact is “blockiness”. I have re-encoded H.265 to AV1 and not only gotten smaller files that playback better on low-end hardware but also display less blockiness while still looking high-resolution and great colour overall.
I always encode 10 bit colour and fast-decode for re-encoding to AV1, even if coming from an 8 bit original.
But then you look at flashback scenes and wonder where the noise has gone.
A lot of movies have purposeful noise, blurriness, snow, and fake artifacts to represent flashback scenes. One level of compression often keeps them okay-ish (like you can tell side by side that it's different, but only when you know what to look for). But these are the scenes that get especially ruined by two layers of compression.
50GB gives assurances that the BluRays are high quality (but not always. I've seen some horrible BluRay encodings...)
As long as you are going from high quality sources, you should be fine. The issue is each transcoding step is a glorified loop-(find something we think humans can't see and delete it)
In other words: the AV1 encoder in your example works by finding 47GBs of data TO DELETE. It's simply gone, vanished. That's how lossy compression works, delete the right things and save space.
In my experience, this often deletes purposeful noise out of animation (there are often static noise / VHS like effects in animation and film to represent flashbacks, these lossy decoders think it's actually noise and just deleted it all changing the feel of some scenes).
--------
More importantly: what is your plan with the 50GB BluRays? When AV2 (or any other future codec) comes out, you'll want to work off the 50GB originals and not off the 3GB AV1 compressed copies.
IMO, just work with the 50GB originals. Back them up, play them as is.
I guess AV1 compression is useful if you have a limited bandwidth (do you stream them out of your basement, across the internet and to your phone or something? I guess AV1 is good for that) But for most people just working with the 50GB originals is the best plan
AV1 is not about throwing away more data that the human can’t see. It’s about having better tools.
1. the prediction tools of AV1 are better than those of h265. Better angular prediction, better neighboring pixels filtering, an entirely new chroma from luma prediction tool, an intra-block copying tool, more inter prediction tools, non-square coding units.
2. If the prediction is better, the residuals will be smaller.
3. Those residuals are converted to frequency domain with better tools for AV1 as well (more options than just DCT), so that you have a better grouping of coefficients close to the DC component. (Less zeros interleaving non-zero values.)
4. Those coefficients compress better, with a better entropy coding algorithm too.
You can have exactly the same video quality for h265 and AV1 yet still have a lower bitrate for the latter and with no additional decision made to “find out what humans can’t see.” The only place in the process where you decide to throw away stuff that humans can’t see is in the quantization of the frequency transformed residuals (between step 3 and 4), but those residuals are already smaller for AV1.
To be clear: you can of course only go down or stay equal in quality when you transcode, due to rounding errors, incompatible prediction modes etc. That’s not under discussion. I’m only arguing about the claim that AV1 is better in general because you throw away more data. That’s just not true.
> In other words: the AV1 encoder in your example works by finding 47GBs of data TO DELETE.
With that reasoning, lossless compression of .wav to .flac destroys >50% of data.
In actuality, you can reconstruct much of the source even with lossy compression. Hell, 320kbps mp3 (and equivalent aac, opus, etc) are indistinguishable from lossless and thus aurally transparant to humans, meaning as far as concerns us, there is no data loss.
Maybe one day we'll get to the point where video compression is powerful enough that we get transparent lossy compression at the bit rates streaming services are offering us.
> In my experience, this often deletes purposeful noise out of animation
AV1 specifically analyzes the original noise, denoises the source then adds back the noise as a synthetic mask / overlay of sorts. Noise is death for compression so this allows large gains in compression ratio.
> AV1 specifically analyzes the original noise, denoises the source then adds back the noise as a synthetic mask / overlay of sorts. Noise is death for compression so this allows large gains in compression ratio.
If said noise still exists after H265.
And there's no guarantee that these noise detection algorithms are compatible with H264, H265, AV1, or future codecs H266 or AV2.
I'd be surprised if there's any noticeable difference between 265 and AV1 when coming from the 265 encoding. 265 has already thrown a lot of stuff away, so there's not much for AV1 to work with.
Maybe if you're going to a lower resolution it would be fine (ie: going from 4k 265 to 720p AV1).
Aside from video, audio compatibility is tricky as well. You can do AAC stereo and most things support that but AAC 5.1 seems to be supported by only some devices so all my video files end up getting stereo AAC, 5.1 AAC, and 5.1 DTS to avoid live transcoding.
IIUC, it's more about the client hardware that determines ability to play without transcoding. You'd have to check the mix of devices you have connecting to it and make a judgement call.
I started to convert a lot of my content in AV1 until I realized that my Nvidia Shield devices won't play AV1. My $30 firestick will play them but I do really prefer the Shield. I guess I'll wait it out and hope for a new Shield (it's been 2019 since Nvidia released one) but i'm not going to hold my breath.
"We built a billion dollar business on top of ffmpeg" - "did you make a donation that you won't even notice on the books, say, a measly one million dollars, to ffmpeg?" - "psh, why would we?"
I wonder what the video editing story is like.
It would be cool if one day (if not already today?) you could use AV1 as a drop-in replacement for h264 for recording with OBS, smoothly editing without proxy clips and rendering out highly disk size efficient videos that look good.
DaVinci Resolve's free version on Linux does not support h264 but apparently does support AV1. Kdenlive supports both. AV1 sounds like it would be a great solution for Linux if the above is possible.
AV1 should work between OBS and Kdenlive.
I have been converting my H265 (HVEC) collection to AV1. I am still trying to find the exact parameters for FFMPEG to produce the optimal output for the devices I use plex on. Its been a little bit of whack a mole so far, but it seems ideal for file size when I get it right.
I've been transcoding my media collection leaving my PC on overnight over months, it's great. My biggest issue is client support for native playback of AV1, naturally.
For what it's worth, AB-AV1 [1] is a pretty awesome tool written in rust which compares random samples from a file at different parameters based on their VMAF score [2] (algorithm from Netflix for human-perceived visual likeness), choosing optimal parameters to save as much space as possible with the loss you're willing to stomach, on a file-by-file basis.
Small plug: I made a nice little python GUI wrapper for ab-av1 [3].
[1] - https://github.com/alexheretic/ab-av1 [2] - https://github.com/Netflix/vmaf [3] - https://github.com/Loufe/AB-AV1-GUI
Because of conversion losses, I have to imagine this is subtly very bad.
Every form of lossy compression deleted data. Yes AV1 is more efficient but only when working off of high quality originals.
H265 already deleted a ton of data. It can never recover the quality loss. Compressing even further can only worsen the image.
While I agree with you, I find that sometimes the “experience” can improve.
The most common “artifact” of AV1 is to make things slightly more blurry for example. A common H.265 artifact is “blockiness”. I have re-encoded H.265 to AV1 and not only gotten smaller files that playback better on low-end hardware but also display less blockiness while still looking high-resolution and great colour overall.
I always encode 10 bit colour and fast-decode for re-encoding to AV1, even if coming from an 8 bit original.
But then you look at flashback scenes and wonder where the noise has gone.
A lot of movies have purposeful noise, blurriness, snow, and fake artifacts to represent flashback scenes. One level of compression often keeps them okay-ish (like you can tell side by side that it's different, but only when you know what to look for). But these are the scenes that get especially ruined by two layers of compression.
What's the optimal strategy then ? 50 GB Blu-ray remux => 3 GB AV1 ?
50GB gives assurances that the BluRays are high quality (but not always. I've seen some horrible BluRay encodings...)
As long as you are going from high quality sources, you should be fine. The issue is each transcoding step is a glorified loop-(find something we think humans can't see and delete it)
In other words: the AV1 encoder in your example works by finding 47GBs of data TO DELETE. It's simply gone, vanished. That's how lossy compression works, delete the right things and save space.
In my experience, this often deletes purposeful noise out of animation (there are often static noise / VHS like effects in animation and film to represent flashbacks, these lossy decoders think it's actually noise and just deleted it all changing the feel of some scenes).
--------
More importantly: what is your plan with the 50GB BluRays? When AV2 (or any other future codec) comes out, you'll want to work off the 50GB originals and not off the 3GB AV1 compressed copies.
IMO, just work with the 50GB originals. Back them up, play them as is.
I guess AV1 compression is useful if you have a limited bandwidth (do you stream them out of your basement, across the internet and to your phone or something? I guess AV1 is good for that) But for most people just working with the 50GB originals is the best plan
AV1 is not about throwing away more data that the human can’t see. It’s about having better tools.
1. the prediction tools of AV1 are better than those of h265. Better angular prediction, better neighboring pixels filtering, an entirely new chroma from luma prediction tool, an intra-block copying tool, more inter prediction tools, non-square coding units.
2. If the prediction is better, the residuals will be smaller.
3. Those residuals are converted to frequency domain with better tools for AV1 as well (more options than just DCT), so that you have a better grouping of coefficients close to the DC component. (Less zeros interleaving non-zero values.)
4. Those coefficients compress better, with a better entropy coding algorithm too.
You can have exactly the same video quality for h265 and AV1 yet still have a lower bitrate for the latter and with no additional decision made to “find out what humans can’t see.” The only place in the process where you decide to throw away stuff that humans can’t see is in the quantization of the frequency transformed residuals (between step 3 and 4), but those residuals are already smaller for AV1.
To be clear: you can of course only go down or stay equal in quality when you transcode, due to rounding errors, incompatible prediction modes etc. That’s not under discussion. I’m only arguing about the claim that AV1 is better in general because you throw away more data. That’s just not true.
> In other words: the AV1 encoder in your example works by finding 47GBs of data TO DELETE.
With that reasoning, lossless compression of .wav to .flac destroys >50% of data.
In actuality, you can reconstruct much of the source even with lossy compression. Hell, 320kbps mp3 (and equivalent aac, opus, etc) are indistinguishable from lossless and thus aurally transparant to humans, meaning as far as concerns us, there is no data loss.
Maybe one day we'll get to the point where video compression is powerful enough that we get transparent lossy compression at the bit rates streaming services are offering us.
> In my experience, this often deletes purposeful noise out of animation
AV1 specifically analyzes the original noise, denoises the source then adds back the noise as a synthetic mask / overlay of sorts. Noise is death for compression so this allows large gains in compression ratio.
> AV1 specifically analyzes the original noise, denoises the source then adds back the noise as a synthetic mask / overlay of sorts. Noise is death for compression so this allows large gains in compression ratio.
If said noise still exists after H265.
And there's no guarantee that these noise detection algorithms are compatible with H264, H265, AV1, or future codecs H266 or AV2.
Thank you for the detailed answer!
Well its sure gonna get the filesize down though, great HECV -> AV1 transcoding success..
I'd be surprised if there's any noticeable difference between 265 and AV1 when coming from the 265 encoding. 265 has already thrown a lot of stuff away, so there's not much for AV1 to work with.
Maybe if you're going to a lower resolution it would be fine (ie: going from 4k 265 to 720p AV1).
I personally use av1an, and use VMAF target quality of 95 to 96. It's been really good and fast.
https://github.com/rust-av/Av1an
Aside from video, audio compatibility is tricky as well. You can do AAC stereo and most things support that but AAC 5.1 seems to be supported by only some devices so all my video files end up getting stereo AAC, 5.1 AAC, and 5.1 DTS to avoid live transcoding.
Do you know what plex clients can handle av1 without the server having to transcode?
I started looking into converting stuff to AV1 but only confirmed that my gpu doesn't support AV1 but does support hevc so I stopped there...
IIUC, it's more about the client hardware that determines ability to play without transcoding. You'd have to check the mix of devices you have connecting to it and make a judgement call.
I started to convert a lot of my content in AV1 until I realized that my Nvidia Shield devices won't play AV1. My $30 firestick will play them but I do really prefer the Shield. I guess I'll wait it out and hope for a new Shield (it's been 2019 since Nvidia released one) but i'm not going to hold my breath.
> my Nvidia Shield devices won't play AV1
Put VLC on them. See if it works for your AV1 videos.
What is the difference in size of files? Around?
"We built a billion dollar business on top of ffmpeg" - "did you make a donation that you won't even notice on the books, say, a measly one million dollars, to ffmpeg?" - "psh, why would we?"
Providing open source free of charge is for suckers. Greed is good.