Libjxl: Photon Noise & Progressive Encoding Issues

Hey guys, let's dive into a couple of interesting quirks we've found when using photon noise with lossy progressive encoding in libjxl. It looks like there are a few bugs causing some funky results, especially when you combine these features. We'll break down what's happening, why it matters, and what it all means for image compression. Let's get started, shall we?

The Blurry LF Noise Texture: A VarDCT Mystery

So, the first issue we're seeing involves photon noise, particularly when it's used with the VarDCT codec in libjxl, and the images is encoded with -p flag for progressive encoding. Basically, when you throw these elements together, the photon noise seems to get applied across both the low-frequency (LF) and high-frequency (HF) components of the image. What does this mean in practice? Well, it leads to a blurry noise texture that's most noticeable in the LF parts of the image. This is not exactly what we want, right? The noise should be more subtle and not create a blurred effect.

This problem is specifically triggered when using --photon_noise_iso along with either -p or -progressive_dc. Keep in mind, --photon_noise_iso is the flag that controls the intensity of the photon noise, and -p enables progressive encoding. The -progressive_dc flag is focusing on progressive encoding of the DC component of the image. It's a bit weird, because it doesn't seem to happen when we use other progressive flags like --progressive_ac or --qprogressive_ac. It is really important to point out these flags, because it is crucial for understanding the specific context and circumstances that cause these bugs to appear. This inconsistency is definitely something that the libjxl team should have on their radar because it can lead to a less-than-ideal image quality with these specific settings.

Let's talk a bit more about what VarDCT is. VarDCT stands for Variable Discrete Cosine Transform. It's a crucial part of libjxl's lossy compression strategy. It works by breaking an image into different frequency components, and it's a core part of how the image is compressed. This technique is really efficient, but it can be very sensitive to noise. The presence of blurry texture with the LF parts of the image is a pretty clear indication that something is going wrong with this process.

For those of us who are into the more technical aspects, this behavior suggests that the application of photon noise is not correctly partitioned between the different frequency components during the encoding or decoding process. It's causing the noise to bleed over in a way that degrades the image quality rather than enhancing it as intended. This issue is probably related to how libjxl handles the progressive encoding of those components when noise is introduced, which could mean there are some mismatches or errors in how the noise is applied during the encoding or decoding steps.

To put it simply, when using these flags together, it's like the photon noise is being amplified in the LF components, producing a distorted texture. These kinds of issues highlight the complexities involved in designing high-performance image compression algorithms, especially when they involve advanced techniques like photon noise. This type of error can make the final image appear more noisy and possibly less visually appealing. This also means more compression artifacts in the end.

So, the next time you are using these settings, remember these findings. We need to stay aware of these inconsistencies. Hopefully, the development team can address it in upcoming updates, improving the overall image quality when using these features.

Photon Noise Amplification: A Progressive Encoding Problem

Now, let's move on to another bug. This one involves photon noise in VarDCT again. This time, the noise seems to be amplified when it is used with progressive encoding. It looks like the noise is being applied multiple times over. So, the overall result is a much stronger, possibly overwhelming, photon noise effect.

There's a key condition that seems to trigger this bug: the image size. It appears to be more noticeable when the images are larger than 2048x2048 pixels. This implies that chunked encoding is in play here. Chunked encoding is a technique used to divide a large image into smaller parts (or chunks) during the encoding process. This can help the encoder manage large files more efficiently and allows for better compression, especially when dealing with large images. The fact that this bug is more likely to appear when chunked encoding is used suggests that there might be some problems in how photon noise is handled when the image is processed in these chunks.

For lossy modular encoding, the problem is even more pronounced. It's always stronger, regardless of whether progressive encoding is specified. This discrepancy between VarDCT and lossy modular suggests that different parts of the libjxl code base handle photon noise in different ways, with varying degrees of success. This inconsistency can be really difficult to debug because it requires developers to consider all the potential interaction points between these functions and how each of these operations interacts with image data.

This amplification issue is especially bad because it can lead to an over-noised image, which, in turn, can hurt the image quality. It's possible that the photon noise is being applied more than once, or in a way that it accumulates across encoding passes, especially in larger images that use chunked encoding. The implication is that progressive encoding isn't playing well with the photon noise implementation. It would be as if progressive encoding and the noise reduction system are not properly communicating. This means an important area for developers to focus on in order to solve the image quality issues.

For users, this means you'll likely get a much noisier image than intended. Also, it could make it harder to properly compress the image because of the strong noise effects. If you are experiencing this issue with larger images, you may want to avoid using photon noise with progressive encoding, or perhaps try different settings to reduce the impact of the noise.

Conclusion and Future Steps

These bugs with photon noise and progressive encoding in libjxl highlight the challenges of balancing cutting-edge compression techniques with image quality. The problems with the LF blurring and the photon noise amplification show that there's still work to be done. We've pinpointed the areas where these problems are most prominent: VarDCT with -p and -progressive_dc flags, and larger images using chunked encoding. It appears that it has more effect when dealing with the lossy modular.

For the libjxl developers, the next steps should be to carefully review how photon noise is applied in VarDCT, especially when used with progressive encoding. They need to make sure the noise is properly handled across different frequency components and across different encoding passes. They should also make sure to test more extensively how it interacts with chunked encoding. For the users, being aware of these problems is key to getting the best results. Make sure to test different configurations, and keep an eye out for updates. Keep those images looking awesome!