Tema: AMaZE: el nuevo algoritmo de interpolación

Hace ya bastante tiempo que os vengo hablando del nuevo algoritmo de interpolación que Emil Martinec estaba desarrollando: AMaZE.

Posteriormente, Paul Lee comenzó a colaborar con Emil y fruto de esa colaboración está surgiendo un nuevo algoritmo de interpolación revolucionario que si todo va bien irá incorporado en la versión 1.0 de perfectRAW. Digo revolucionario porque suma estas características:

• Es capaz de ver texturas finas (Nyquist).
• Ve más allá de la zona inmediatamente cercana al píxel en cuestión (es un algoritmo no local).
• No produce diagonales serradas.
• No produce artefactos.
• No extiende el ruido.
• De momento las implementaciones preliminares en C arrojan unos tiempos de ejecución unas 4 veces mayores que AHD (lo que lo hace mucho más rápido que los últimos algoritmos que se están publicando en la literatura especializada). Sin embargo creemos que se podrá optimizar mucho más que eso y será fácilmente implementable mediante instrucciones SIMD (SSE/SSE2) y en la GPU (en ese caso, gracias a Egon funcionará sin necesitar CUDA, aunque sí tarjeta modernillas, claro).

Como hemos dicho en muchas ocasiones, lo más crítico a la hora de revelar RAWs es interpolar lo mejor posible. Este nuevo algoritmo nos permitirá partir de una imagen casi perfecta.

Como una imagen vale más que mil palabras, os pongo unos ejemplos reales revelados con la versión en Mathematica del algoritmo sin postprocesado ni refinamiento y del algoritmo aún en desarrollo, y por tanto aún capaz de mejorar. Las imágenes son PNGs de 16 bits y por tanto un poco pesadas, pero es el mejor modo de poder compararlas sin perder calidad. Como veréis son ejemplos escogidos por ser especialmente difíciles:



http://75.126.132.154/PRODS/E1DSMK3/...K3hSLI0100.CR2

http://75.126.132.154/PRODS/D700/FUL...0hSLI25600.NEF


Un saludo:

Aquí una comparación con ACR para que se vea en qué mejora el algoritmo de Emil (AMAZE) las zonas complicadas (moiré):

Guillermo, ¿tú a cuantos fps ves?
______________________

Ahora sí .

Ya sabéis todos que ACR utiliza AHD como algoritmo de interpolación.

Un saludo:

thanks manuel that you finally took some time to present the first results from AMAZE.

it is a pity that the images load that long, i only managed to view one image entirely, the other ones stopped loading at the half.

have you tried the new algorithm on some of my G1-images?
on the images you provided some details are indeed rendered nicely, but i wouldn't claim the algorithm to be perfect yet. i did a quick comparison with Lassus' RGE and there are many parts that RGE decodes better.

i attached two examples.

have a look at the brush, the diagonal brush hair is rendered smoother with AMAZE, but the hairtips are better in RGE, AMAZE has some zipper noise here.


in the next crop i would say that RGE decodes the lettering better. there is less zipper noise if you look at "RULER" or "CT".
on the other hand RGE fails to decode the blue diagonal chart-lines, they are much smoother in AMAZE (not visible in this crop).

Iniciado por oluv

thanks manuel that you finally took some time to present the first results from AMAZE.

It has never been a matter of time. I have just waited for Emil and Paul to decide that the algorithm is in an advanced stage.

Iniciado por oluv

it is a pity that the images load that long, i only managed to view one image entirely, the other ones stopped loading at the half.

I am afraid that imageshack servers are going slow. Have you tried downloading them to your machine instead of letting them load in the browser? Maybe that could help.

Iniciado por oluv

have you tried the new algorithm on some of my G1-images?

Time will come for that.

Iniciado por oluv

on the images you provided some details are indeed rendered nicely, but i wouldn't claim the algorithm to be perfect yet. i did a quick comparison with Lassus' RGE and there are many parts that RGE decodes better.

Well, no one has claimed AMaZE to be perfect yet, but I am confident it will be. The important thing to look in those samples is how the algorithm is able to get correctly the most difficult parts of the images in a way no other actual algorithm can. It is true that at this moment getting better one part of the image can make other parts worse. As I have already said the algorithm still needs more development, more postprocessing and more parameters adjustment (despite the fact that it will probably let the user control some interpolation parameters to get the best of any image). Here you have an example (the same in RawTherapee comparison page) of what you can get changing interpolation parameters in the actual AMaZE implementation:


To be true, at this moment AMaZE is able to get any part of the image almost perfectly although the algorithm still needs to make a better use of adaptative parameters in each zone. That is in fact the actual stage of the development process, together with global optimization.

That said, RGE and the rest of Lassus' work is great. I am waiting for Lassus to finish RGE to put it into perfectRAW, of course.

BR,

Those samples are really impressive. I never though a bayer pattern could be demosaiced in that way.

Thank you Oluv for supporting my work, but when AMaZE will be finished none of my algorithms could be compared with it. At this moment what RGE can do better than AMaZE are marginal parts of the image, but difference on the rest are more evident. Maybe RPC (with the integrated refinement) can compete with noisy patterns at this moment, but probably can't in the future.

I should redirect my efforts to another field. Anyhow I will finish a faster-than-AHD RGE version with diagonal support that ensures a high-quality demosaicing in a reasonable developing time and will try to improve FDD to its limits. I will return to coarse noise and those boring things...

Manuel, where did you download the RAW of the second picture? Links at rawmagick.com don't seem to work.

I'm still astonished...

Actualizo de nuevo tras un ajuste de Emil a su algoritmo de interpolación. Insisto en que aún queda por mejorar en el algoritmo. Especialmente Emil está desarrollando dos mejoras importantes que eliminarán los artefactos tipo zipper que quedan en la última imagen:




Un saludo:

manuel, you are making our mouths wet
but when will we finally have the chance to play with AMAZE a bit?

Emil ha terminado su algoritmo de corrección de aberraciones cromáticas durante la interpolación y los resultados son, desde mi punto de vista, espectaculares.

A ver si tengo un rato de subir algún ejemplo por aquí.

Un saludo:

Me quedo a la espera de novedades

Iniciado por detalhe

Me quedo a la espera de novedades

Pues para que no esperes mucho, te pongo unos resultados:


Y en grande se ve mucho mejor cómo mejoran los artefactos zipper al corregir las aberraciones cromáticas:


Lo último que sé de Emil es que ahora iba a incorporar la reducción de ruido al propio algoritmo de interpolación.

Un saludo:

Gracias

Buff.. veo avances espectaculares en la interpolación.

Enhorabuena, Manuel.
Cuando esté del todo acabado, no va a haber competencia para PerfectRaw.

Es todo el mérito de Emil y Paul. Yo solo pongo aquí los ejemplos.

Un saludo:

Hola, es un logro espectacular, cuando podremos probarlo?, saludos

mmmm parec muy interesante , a ver cuando lo podemos probar en su version definitiva

are there many parameters to tweak to achieve such perfect results, or is the algorithm going to be kind of "intelligent" and will be able to give good results with any kind of images?
any idea when we can expect a first demo or beta? i am having thousands of raw-files waiting for conversion. so far not a single algorithm gave me the results i am after. RGE from lassus is very promising, but i think AMaZE could finally fullfill my dreams!

Iniciado por Sipa

Hola, es un logro espectacular, cuando podremos probarlo?, saludos

Todo el mérito es de Emil Martinec y Paul Lee. Yo sólo estoy posteando por aquí sus resultados.

Cuando el algoritmo sea definitivo, Egon y yo intentaremos ayudar a Paul a implementarlo del modo más rápido posible.

Un saludo:

Iniciado por oluv

are there many parameters to tweak to achieve such perfect results, or is the algorithm going to be kind of "intelligent" and will be able to give good results with any kind of images?

The examples posted so far are all with the same, default settings, apart from the chromatic aberration example where of course one has to input the amount of correction for the different chroma channels. It is not yet determined what parameters should be available to the user, though I have some ideas.

any idea when we can expect a first demo or beta? i am having thousands of raw-files waiting for conversion.

I'm not sure what the time scale is. It is also not determined how AMaZE will be integrated into PerfectRAW, and in what dot release it will appear.

The main interpolation algorithm is in final form. Paul is quite busy at the moment, so progress in C coding of the main algorithm has temporarily halted (I am not a C programmer; my skill is to formulate the algorithm, wherein I have had substantial help from Paul in implementing the ideas). The CA correction algorithm is something I worked out on my own, and I am now exploring options for integrated noise reduction. I would also like PerfectRAW to have a deconvolution sharpening routine, but that would naturally be an independent, post-demosaic module in the code.

hello martin!

thanks for keeping us up to date with the latest info.
i am aware of the complexity of your work and i admire what you and emil have already achieved. i am sure the algorithm will make perfect raw an outstanding raw converter.
i am also sure that many of us would already be glad to trying out AMaZE as a simple dcraw implementation. let's call it beta of alpha or whatever. but this way it could be tested on various user-images and as return you could get feedback where the algorithm behaves well and where it might fail.
of course i am willing to donate to support your work.

what i still don't understand is why with some cameras "standard" demosaicing algorithms give quite good results, while with others the output is quite crappy.
i attached an example, it was converted with raw therapee. the new E-P1 is supposed to use the same panasonic sensor as the G1, but as you can see the output is totally different. the E-P1 renders nice smooth diagonals, wheras the G1 output is quite edged with lots of demosaicing artefacts.
why is this?
i like the G1 as it produces very sharp images, but i have lots of problems converting my RAW files, because any algorithm i have tried so far fails with most of my images. as soon as i sharpen a bit, all artefacts are pronounced.

Notice that the AMaZE chromatic aberration (CA) correction example that Manuel posted did not have much CA, but the correction of the CA also made some of the demosaic artifacts go away.

The sensors in the two cameras, G1 and E-P1, might differ by having different CFA filters with different spectral responses, and different strengths of AA filter, which would have some effect on the conversions; however, I suspect the main difference in the two conversions you posted comes from the fact that the G1 image was taken with a lens having a lot more CA than the lens used on the E-P1 image. As a result the G1 image has a lot more demosaic artifacts. This is why I felt it was important to incorporate CA correction into the demosaic process.

to warm up the topic, i wanted to ask how internal CA correction could be implemented before the demosaicing process itself? i think this is quite a serious problem to deal with. i think it is not easy, because CAs are not always 100% predictable, especially with OIS lenses they can differ from one side of the image to the other.
i had an example where i was able to fix all CAs on one side of the image, but i introduced more artefacts to the other side

i also discovered that capture one has some automatic CA-reduction. it analyzes the image first and then applies the correction automatically. i think this is something we will need sooner or later.

here is an example where i think the biggest problem to demosaic it correctly are the CAs. it is a particular focal-length of the G1 kit-lens that produces quite heavy CAs. as you can see in my attached crop, raw therapee has some serious problems with it:


i would like to see how the output from AMaZE will be with this particular image, and if we can expect wonders...

here is the original RAW, feel free to make conversions out of it and post your results:
http://files.getdropbox.com/u/893528/P1030902.RW2

Iniciado por oluv

to warm up the topic, i wanted to ask how internal CA correction could be implemented before the demosaicing process itself?

Look at the dcraw code for a rather simple but ineffective implementation. AMaZE does something a bit more sophisticated.

here is an example where i think the biggest problem to demosaic it correctly are the CAs. it is a particular focal-length of the G1 kit-lens that produces quite heavy CAs. as you can see in my attached crop, raw therapee has some serious problems with it:

i would like to see how the output from AMaZE will be with this particular image, and if we can expect wonders...

I don't know about wonders, but here is what I was able to get after some work:



I didn't fine tune the CA adjustment parameters, but simply chose them by trial and error. Please note that these are a bit time-consuming to work up, I'd rather be spending the time investigating NR algorithms.

The correction works great. But Emil, did you use oluv's image or some RAW file? because it's not only the CA correction but also a big improvement in sharpness (oluv's image seems to be a bit de-structurated).

I wonder if the general colour shift is due to the CA correction or a different WB. Your image is cooler (less red) than oluv's.

BTW don't expect too much activity in this forum during the Summer months. Vacation time in Spain is veeery long, till September.

Regards.

Iniciado por Guillermo Luijk

The correction works great. But Emil, did you use oluv's image or some RAW file? because it's not only the CA correction but also a big improvement in sharpness (oluv's image seems to be a bit de-structurated).

I wonder if the general colour shift is due to the CA correction or a different WB. Your image is cooler (less red) than oluv's.

I used the RAW file oluv linked to at the bottom of his post. The lack of sharpness in oluv's version is due to the misregistration of the color channels -- R and G and B are shifted from one another, and when superposed that creates a blur in the luminance channel, even if the interpolation were perfect. On top of that, the large CA causes RT's algorithm (I'm assuming HPHD here) to misinterpolate a lot, lowering the sharpness further. If you want some fun, open the RAW file in ACR and turn off the color noise removal (I had to run it through DNG converter first because the file is not supported in ACR 4.6); though the ACR conversion is not too bad with the default color blur of 25 on the slider -- at least it doesn't have the interpolation artifacts of the RT version.

As for color, my image is an output of the Mathematica prototype code. I took the WB coefficients from the metadata (I don't recall I changed them from those values, but I might have); other than that there is no color management, in particular no transformation from the RAW 'color space' to a standard one like sRGB, and so that is a major reason why there is a difference in color. It was actually even more than shown, I took the AMaZE output into CS3 and boosted the saturation substantially to make it look at least a little closer, and to show that the lower amount of CA was not due to lower saturation of the AMaZE output.

BTW don't expect too much activity in this forum during the Summer months. Vacation time in Spain is veeery long, till September.

Yes, I suspect that's why it's oluv and me, the two non-Spaniards. And let me take this opportunity to apologize for not having any Spanish whatsoever. I can only participate here through the (often amusing) translations served up by Google. I can only imagine how it renders mine into Spanish...

thanks emil for your effort.
this sample really seems to be a "hard" one. no idea why, but i am quite sure that it is because of the CAs. the image lacks a bit of sharpness, so it is maybe not the best example for detail extraction abilities of various algorithms, but i think most algorithms will nevertheless have some problems rendering it without artefacts.

AMaZE does a much better job than HPHD, but it still struggles with some parts. i wonder how it would look like, if you didn't try to correct CAs?

i thought i would try DCB and capture one again and post the results here, because i was surprised to see how well DCB does with this particular image:


you cannot take capture one's version for serious though, because it is a bit soft due to lens-distortion correction. capture one does a very lousy job, you can really see where the image gets stretched, i already wrote to phaseone support that the interpolation is absolutely low-quality. have a look at this example, you see the horizontal banding because of image-stretching? (R1 stands for raw therapee of course ):


what i don't like about DCB is the way it renders fine detail and noise, especially if you sharpen the image a bit, you get it enhanced quite a lot. here i used a crop from the center of the same previously posted RAW to avoid any image-stretching, and as you can see the noise from capture one looks more natural, it is regular and grainy wheras DCB does this pattern-like thing. it reminds me a bit of van gogh's painting style... (although C1 looks a bit more like a seurat painting )

i think C1 is a nice compromise between artefact-free edge rendering, details and noise-structure. it is by far not perfect, but it does a very good overall job.
they only need to fix this lens-correction issue and i'd stick with it for my conversion work until perfect raw and AMaZE is ready.

I fixed a bug in the CA correction routine, that wasn't properly implementing the correction when the color plane shifts exceeded 1 pixel, as they do in oluv's image. With this bug fixed, there is an improvement:



Many fewer artifacts, and perhaps slightly more detail.

yes, the new one is much better! so at least my example helped to fix a "bug"

which settings did you use for CA correction? i have quite a hard time to find the right values, so i am glad that C1 has an automatic correction implemented. the funny chroma-NR in ACR is another workaround how to deal with it

A los que no entendemos el ingles, que nos den, no?

Saludos

Emil, you last sample looks just perfect!

Iniciado por Josansaru

A los que no entendemos el ingles, que nos den, no?

Hombre, Josansaru, por supuesto éste es un foro en español y lo lógico es escribir en este idioma, pero se trata de un caso excepcional. La gente que entiende inglés se entera de lo que escriben, y los que no lo entienden pues no se enteran, lógicamente. Pero si se les exigiera escribir en español no se enterarían ni unos ni otros, puesto que no podrían hacerlo ya que no tienen ni papa del idioma y por tanto no publicarían aquí. ejmartin es el creador de AMaZE y todas sus intervenciones en este hilo están por tanto justificadas. Serían incluso más que bienvenidas aunque fueran escritas en chino mandarín, que para algo están los traductores online.

Josansaru: este subforo es para el desarrollo de un programa de revelado (PerfectRAW) y la mejora de los algoritmos de interpolación.

En él se llevan a cabo discusiones técnicas de implementación (que a muchos se nos escapan) y muchos de los participantes en el desarrollo de esos algoritmos son extranjeros, y por tanto se expresan en el idioma que conocen (y que conocen los participantes en el desarrollo).

Por suerte dicho idioma es el inglés (aunque no en todos los casos es su idioma nativo, hay franceses y por supuesto españoles).

Creo que las intervenciones son muy interesantes, aunque si estuviesen en español también seguiría sin enterarme de la mitad, pues son discusiones bastante técnicas sobre detalles de implementación.

El foro es evidentemente en español, pero como dice Lassus este apartado es muy particular, pues se centra en los algoritmos informáticos (que aún en español no serán bien entendidos por muchos de los fotógrafos).

Osea, que de todas maneras no me enteraria de nada; pues no he dicho nada.

Hi Oluv!

Here is my effort, my debayer is designed to be viewed at 100%, I do not have a profile for the G1 so all colours are wrong!

Fujicoly aka Olycoly!

http://img22.imageshack.us/img22/581/57746064.png

fujicoly!

i tried to contact you several times after you have presented your demosaicing results on dpreview-forum some time ago... you never responded

good to have you back. tell us more about your algorithm. is it available for testing etc? or will it be in future?

best regards, olaf

Hi Olaf,

Still only matlab code and slowwww....

Here is another attempt with CA correction, as before these are meant to be viewed at 100%!

Feel free to post a link to any other G1 files you would like to see results with.

Colin

colin, i will try to prepare some further RAWs for conversion. your results are quite appealing, but i am missing a bit of extra-detail, or let's call it "pseudo-detail" are your crops sharpened? they look somehow sharper than the other examples.

i would like to see one of your conversions in full-size if possible, maybe you can upload it somewhere or send it to me.

would it be hard to make an executable out of your algorithm? speed is no issue if the quality is good!
_______________________________________________
_______________________________________________
i quickly searched for 2 examples and finally chose to upload the following two RAWs.
the first one completely fails to demosaic in raw therapee.

in the second one i would like to see how all the fine details are rendered. i am not very satisfied how raw therapee does it.

http://dl.getdropbox.com/u/893528/P1050130.RW2


http://dl.getdropbox.com/u/893528/P1040523.RW2

here a new "challenge" if anyone is willing to try. i wonder how the advanced algorithms will deal with the roof-structures and if any algorithm can demosaic them without color-moiree? here an example from luis' RGE:


here the original RAW file:
http://dl.getdropbox.com/u/893528/P1030893.RW2

oluv,

Sorry if I will not have time to work on all your examples. Here is a sample I did work on a few months ago, which many algorithms have a lot of difficulty interpolating without generating a lot of moire as well as interpolation artifacts. It's from a Nikon D70, whose weak AA filter leads to all sorts of aliasing issues.

Here is AMaZE's version:




And here is what ACR 4.6 can do with it:




Some more examples of this image with older versions of other RAW converters can be found at: Compare ALL.

In order to get the best out of AMaZE, I had to make one straightforward change from the default settings.

Emil, I always thought that I would prefer a thin, or non-existent AA filter to maximise sharpness in 99% of images at the cost of having some trouble in 1% of them, but this sample makes me doubt.

What do you think of eliminating the AA filter from a camera? and given the moire problem appears, how difficult would be to apply pre-blurring over the undemosaiced data only in the affected area? would it be difficult to mimic the result of a genuine AA filter? after all the AA filter is just a low pass filter scheme.

Regards.

Iniciado por Guillermo Luijk

What do you think of eliminating the AA filter from a camera? and given the moire problem appears, how difficult would be to apply pre-blurring over the undemosaiced data only in the affected area? would it be difficult to mimic the result of a genuine AA filter? after all the AA filter is just a low pass filter scheme.

The photosite array samples the image at its Nyquist frequency (the inverse pixel spacing). The finite size of the photosite attenuates frequencies in the image beyond Nyquist. However, because of the color filter array (CFA), individual colors are sampled at a lower frequency than Nyquist. This is what gives rise to aliasing in interpolated data -- the band of frequencies between the sampling frequency of individual colors and the Nyquist frequency. If this band is not attenuated by an AA filter, it can alias (shift) into lower frequencies during demosaic, which is seen as the large scale red-blue oscillation in the ACR example. AMaZE does a better job minimizing the aliasing, but it still would have been present in parts of the image if I hadn't made the one modification to the settings that I did.

Once the high frequency data is aliased into lower frequencies, it can't really be blurred away -- the aliased signal is now in low-frequency data and will be passed by any low-pass filter you might wish to apply. The order of the two operations of sampling and filtering matters, which is why the AA filter precedes the discrete sampling done by the photosite array. Doing away with or weakening the AA filter amounts to an assumption that high-frequency data won't be present in ways that ruin the interpolation; it's a gamble. Do you feel lucky?

It's more than 15 years since I studied the sampling theory at the university, but I think the sampling requirements (Nyquist, AA,...) became much less strict if instead of an ideal Dirac delta signal we use a pulsed signal to do the sampling.

And a CFA is in fact a 2D pulsed sampling signal because the photosites average a square area to collect photons. I wonder if this could have by itself an AA low pass filter effect.

It is true that for being, specially the R and B channels, not collected at all positions, we are actually performing an intermediate kind of sampling: not ideal, not 100% averaging, that could make some AA still necessary.

BTW is AA at all necessary on Foveon and monochrome sensors?

Regards

Iniciado por Guillermo Luijk

BTW is AA at all necessary on Foveon and monochrome sensors?

Regards

Not needed and not used. FYI, most medium format backs do not use an AA filter.

Iniciado por ejmartin

FYI, most medium format backs do not use an AA filter.

I knew that, but always wondered if it's because they (at least some of then) can (at user's request) capture all 3 channels for any pixel through 1-pixel shifts, or just because with their very high resolution AA is considered unnecesary?

BR

Hi. Maybe this is not the right thread ton post in, so I ask the moderator to move this post if it is not appropriate.

I haven't seen any news about the development of PerfectRAW. As I use my blog to let people know about it, I's like to know at what stage is it's development and everything people who are waiting for it should know.

Thank you.

John

John Roshka Photography

Hi, Phoreal, be patient.

On august Spain is closed for hollidays

In a few days you will see a lot of movement here.


BR

Hello everyone. I'm still landing!

BR,

Is there any progress made during the last couple of weeks? I'm curious how AMaze will look like with perfectRaw.

Aquí estamos, expectantes

Still working on noise reduction

Current philosophy is two stages of filtering: one to remove impulsive noise and a second step to remove gaussian/Poisson noise. I have been working on the luminance channel first; chroma noise will be tackled when I feel happy with the quality of the luminance noise reduction.

The first stage noise reduction is I think done; the second stage is still a work in progress, as I try to remove noise without flattening detail -- for example in the fabrics in the image below. I would rather leave some noise and keep the low contrast detail than to erase both.

This is a D700 ISO 25600 test image:



The starting point is the output of AMaZE, then impulse noise removal, then a rather subtle filtering with the current multiscale algorithm (which still needs work )