CaLights 3.1.3.00 adding Flats creates Red Cast on all calibrated light images

[John M]

Hi Peter I finally managed to get some imaging done with my new ASI2600MC camera the other evening thanks to some reasonably clear skies.  I then upgraded my Calights to version 3.1.3.00.  But found I have an issue.

If I calibrate my images without Flat files I get the following (as shown in Calibrate Preview)

 

With the Master Flat included I get

If I then continue with the included Master Flat file the resulting Calibrated CALight images all have this Red cast to them and I cannot remove this with my usual processing in DSS and Siril.

I have tried an earlier version of CaLights 3.1.0.90 and it does not result in this issue.

I have uploaded an sample of my Lights, Flats, and Master files to my Google drive.  Which you should still have access to.  If not not let me know and I'll resend an invitation for you to access the Google Drive.

Sample file CaLight_Test2.zip.

Thanks

[wols...@gmail.com]

John,

Congrats on purchasing the ASI2600MC.  That's a lot of camera. 16 bit A/D and lots of pixels. When I look at the files you sent me I see some big issues for you to tackle.

1)Your master BIAS needs to be done again.  Your camera produces fits files that have values that range from 0 to 65,535.  Your masterBIAS indicates that the mean values  were  around a value of 2 to 3 which is far too low.  I suspect there was far to many values being clipped at zero.  You need to increase the OFFSET value for your camera so that the mean values are 100 to 500.  Other ASI2600MC owners may have suggestions.  You should use this OFFSET value for all of your fits files.

2)Your master DARKs need to be done again. Your DARK frames have fallen victim to the same issue as your BIAS frames.  Your DARK frames have mean values around 2 to 3.  The mean values for DARKs should be significantly higher than your BIAS and right now they are roughly the same which I think makes no sense.  This is always the case with DSLR BIAS and DARKs but never happens with dedicated astrocam BIAS and DARKs.

Your procedure should be to first work on your BIAS frames. 

1)Introduce an OFFSET value and take a BIAS.  

2) Load this BIAS frame into "Make Averaged Master" and click on "Create Master"

3)Click on HISTOGRAM and look at the statistics.  You want to make sure that the values for Mean are, at least, 5 to 10 times larger than the Std.Dev. values.  Some experts would suggest a bigger ratio...I don't.

4)Once you decided on what OFFSET value to use you can go ahead and re-do your BIAS frames and DARK frames using this new value for OFFSET.

5)Finally, regenerate your master BIAS and DARK.

                                                                              BUT!!!

Your FLAT and LIGHT frames show a more serious issue that you need to address. Your camera is not properly centered in your telescope's field of view.  There is dramatically less light reaching your camera all along the left edge of your LIGHT and FLAT frames. The dramatic reduction in light along the left edge should not be a problem for CaLIGHTs.  It's all a form of vignetting.  So I am continuing to study your images to see if I can find why the strong reddish hue when using your master FLAT.  CaLIGHTs was designed to calculate mean values for RED, GREEN and BLUE in your master FLAT and use these mean values when determining how to apply a master FLAT. How this resulted in a reddish image is a puzzle.  The bad BIAS and DARK masters may play a role too.

It's looks like both of us have work to do!

Peter

[wols...@gmail.com]

John,

Once you determine a value for OFFSET you should use the same value for all of your images...including FLATs.  I know that sounds like a burden but "with great power comes great responsibility".  Welcome to the world of dedicated astrocams.

Peter

[wols...@gmail.com]

John,

I did find a bug in the FLAT compensation V3.1.3 which explains the reddish colour cast when you use a master FLAT.  I'm not sure how long this bug was in the code and I am surprised that it took this long to detect it.  The code looked ok but after careful study it was obvious.  I will be posting a new version and taking down V3.1.3 immediately.

Peter

[wols...@gmail.com]

John,

You should find that your images no longer have a strong color cast when you use CaLIGHTs v3.1.4.  Thanks for helping me find this bug. 

You still need to regenerate your masters with a higher OFFSET. I look forward to hearing more about that strange dark area along the  left edge of your images.

Peter

[John M]

Peter I have tested V3.1.4 and yes it does remove the red cast I was getting in v3.1.3.   However I am still not completely happy with the images I get when I include Flats and DarkFlats in my processing.  They appear to be somewhat more noisy then when I process without flats.  This is something I can generally fix with additional tools like Topaz DeNoise but I'm not clear why using Flats would increase my noise levels.  And of course without the Flats I have to touch up the dust bunnies etc.

In response to some of your other concerns. 

My camera is not out of adjustment.  The edge issue is  a result of the OAG prism encroaching  on the main image sensor when I have the guide camera oriented on the short edge of the image sensor.  This is not an issue when the guide camera is oriented along the long edge of the image sensor.  I will resolve this once it warms up a bit and I get some time to adjust the prism.  For now I will just fix it in processing.

As for the offset value on the ASI2600mc camera what I have found is that at a gain of 0(unity gain) ZWO recommends an offset of 50.  But the sweet spot for this camera, as far as acquiring the best level of read noise and maximum DR, is to use a gain of 100 or slightly higher.  And the recommended offset at a gain of 100 appears to be 0.  At least this is what is recommended in the Ascom driver configuration for the ASI2600mc. 

All the imaging (including all my Darks, Biases, and Flat files) I have done so far with this camera has been with a gain of 102 and an offset of 0.  I did do one quick run of only 10 images of M81 Thursday evening using a gain of 0 and offset of 50 but I was not happy with the results.

Once we get some clearer weather I will do additional testing using a gain of 102 and various offsets to see what results appear to work best.

Thanks for your help and quick response to the red cast issue

John M..

[Scott Kuchma]

Hi John ,

Just to be clear , all your Calibration Files and your Lights work OK with the previous version of CaLIGHTs v3.1.0.90 ? I really can't remember what changes were made from that version to the current one .

Hopefully Peter will have a dig through again and see what's changed .

Scott..............

[wols...@gmail.com]

John,

You need to prove to yourself that your BIAS and DARKs contain no useful information.  Basically whether you use your BIAS and DARKs or you don't use your BIAS and DARKs the resulting image is the same because your BIAS and DARKs are essentially zero. 

Please use the "Make Averaged Master" routine with only one BIAS frame.  Click on the Create Master button and then look at the RAW pixel Values that are shown.  I suspect you will see that a huge number of the pixels are zero.

Repeat this same test with one DARK frame...you will also see that there is a huge number of zero pixels.

I suggest that you need to use a non-zero offset and I gave you a procedure to determine how to find the best OFFSET.   

Right now...the only master file you have that has any useful information is your master FLAT but your master FLAT needs a master BIAS or master DARK to determine how strongly to correct your vignetting.  Right now that's not possible because your master BIAS and DARK are invalid.  I suspect that without valid master BIAS and DARKs that the master FLAT is being applied too strongly which may be causing the increased noise.  Also please understand that the OAG pick-off mirror was affecting your image to an unacceptable amount and this further complicates your issue.

Peter

P.S. you mentioned that you also have a master DARKFLAT.  If you used the same OFFSET then this master will also be invalid.

[John M]

Sorry Peter I think I'm suffering from information overload.

I've created a number of bias frames with a gain=102,  temp= -15 and varying offsets.  Here's my results

From this it appears to me that an Offset of 4 or 5 would be appropriate.

Do you agree?

Assuming you agree i will use Offset 5  and rebuild all my Bias, Dark , Flat and DarkFlats and hopefully this evening I can do another set of images of M81 with the same offset, gain etc.

Thanks

John M..

[wols...@gmail.com]

John,

Great spreadsheet.  Your OFFSET gives you great control of this situation.  I would say that a value of 4 is the bare minimum.  If you use a different gain you might find that you need to use a different OFFSET.  You might want to try different gains and see if there is an OFFSET value that works best at all gains.  This is what I do and, ironically, for my QHY294C, an OFFSET 0 gives me BIAS frames with no clipping at all gains. Different vendors have different approaches.

In your case, even with an OFFSET of 20, you are only giving up 201 counts to ensure no clipping at zero.  For a 16b camera that's a very, very small sacrifice of dynamic range.  If you can use just one value for OFFSET, then the real win would be that you only have one OFFSET value to worry about, regardless of which camera GAIN you use. Astrophotography is complicated enuf.

Do some more research now and you can put a bow on it.

I would gather the same statistics when you generate your DARK frames.  The Mean and Std. Dev. should be slightly higher but it will give you a great insight into how much Dark Current you astrocam has.

Great Work!

Peter