Playing the devil’s advocate on callibration frames
Below are the results from time series photometry on V757 Cen, an EW/KW eclipsing binary. 218 images were taken over 3.9 hours. Camera - ZWO ASI294MM (cooled); Lens - Canon 200mm f/2.8 L; Exposure - 60 sec; Gain 0 units, 4.0 e-/ADU; Binning - 1x1; Region of interest – one half; Mount – SkyWatcher Star Adventurer star tracker, polar aligned and autoguided (in RA only).
Photometry was performed after calibration (darks and flats), and repeated on the non-callibrated images.
Mean non-transformed V magnitude: variable, callibrated 8.503; non-callibrated 8.506; check, callibrated 8.056; non-callibrated 8.061 (check catalogue V mag 8.047).
Standard deviation of V magnitude: variable, callibrated 0.123; non-callibrated 0.122; check, calibrated 0.007; non-callibrated 0.007.
Why should I use calibration frames?
Roy
Because you don't always know that your results would still be close without calibration. Calibration is part of the due diligence of providing good data.
-Walt
Correct, because I've only posted data so far from one night of observations.
But I do have data from 7 other nights. They are nearly as good, but not quite. For example, for 5 of the 7 nights the range of values for non-cal magnitude minus cal magnitude was 0.015 or less, not 0.01 or less as was the case for the data I posted from.
However, the mount during those 7 nights was unguided. When I have data from several more nights of guided images I'll report back. Might be a while. There has been a lot of unseasonal cloud where I live.
Roy
Roy, does dividing by flats make more of a difference if you have substantial vignetting and your targets (comps, check, vars) are not near the cent of the image?
CMOS cameras take images faster? No time for dark current to grow?
Ray
Its very possible to luck out that the stars don't fall on a bad pixel, dust mote, amp glow region etc, but there's an even better chance one of them will.
Keith
Roy,
If the SNR's in your targets and comps are high enough and you have picked a comp close enough to the target then calibration could have little or no effect, just as you have shown in your experiment. The problem comes if you are working with faint (low SNR) stars, or if your only good comp is far from the target in your image. In these cases calibration would be important.
How do you know if your SNR's are high enough, or the separation of your target and comp is small enough? You could do your experiment on each of the stars you are observing. You might find that for some of your targets the photometry doesn't benefit by calibration. This would be at the cost of the extra work of doing these tests on each of your targets. I think it would be simpler and less work just to calibrate all your images.
Phil
All good points. The…
Phil,
All good points. The experiments are empirical, and not designed to look specifically at those issues. However as part of the analysis I document distances between stars.
Roy
It was looking at…
Keith,
It was looking at dust motes that started me thinking about flats a few years ago. I looked at the profile of counts across dust motes and found that the range of ADUs was quite small. The ADUS were far smaller, by orders of magnitude, than those of stars. I use a small aperture, so am limited to 'brighter' stars.
The difference to the signal after subtracting the background counts in and around a mote was miniscule.
Roy
Meant to add - there…
Keith,
Meant to add - there is no amp glow in my camera.
Roy
I think It's likely…
Ray,
I think It's likely substantial vignetting would make a difference. I choose comp and check stars to be 'near' the centre of the field, and because of that vignetting across the part of the image I use is less than it is across the entire field. Because of the central position of the stars, I can use a region of interest of one half.
I haven't done the relevant experiment.
Roy
Meant to add - dark…
Ray,
Meant to add - dark current in my camera stabilizes between 5 and 10 seconds. My exposures are always longer than that.
Roy
Thanks for clarifying that Roy.
I suppose the other thing is that those who actually use the AAVSO database would feel a lot better about analyzing calibrated and transformed data.
I have not always done that. It would be interesting to me to be able to search my submissions to see what percentage I have transformed.
I did not have transform coefficients when CCDs were beginning to make their mark.
Ray
Ray,
You wrote: "... those who actually use the AAVSO database would feel a lot better about analyzing calibrated and transformed data."
I agree completely. In fact, when I was composing my "devil's advocate" post I thought of including the answer that, if I were submitting a paper of photometry results for publication, the referees would expect to see mention of callibration. Understandably.
However, if something is regarded as standard practice or absolutely necessary, it should be possible to find numerical data to show why.
I didn't make the original post just to tilt at windmills. I had a situation a while back where I wanted to re-analyse photometry from old images, but realized the callibration frames had been lost. Because they were for old science images I could not do them again. The question then arose how much accuracy/precision would be lost without callibration?
Roy
Roy,
I calibrate because it promotes consistency between observer's data. This is the same reason I transform my data when I am trying to maximize consistency in magnitude estimates when magnitude is an important parameter. Consistency tends to maximize the fit of data to models of change when data are collected by several observers.
Ed
Callibration would…
Ed,
Callibration would improve consistency if it improved accuracy.
My one little experiment (using a guided mount) did not show any improvement in accuracy, but one experiment doesn't prove anything.
Roy
Roy,
I agree that you “got away with it” in your experiment. (I am not being pejorative.) I also agree with you that one experiment cannot be generalized as it now stands. I even remember one former Board member telling me that you can “get away with it” under certain circumstances (former Board member to remain anonymous). But that is not the point.
Your one-night accuracy is judged by conforming to the light curve you produced, since there is little different in magnitude estimates between calibrated and uncalibrated images. Fair enough. You have shown that in this one instance you can dispense with calibration.
Consistency is another matter. Consistency obtains when different observers arrive at similar results. In your case, consistency would be judged by whether a ToM generated by your light curve conformed to or did not conform to a model of the histories of all ToMs taken to different observers. That is: the ephemeris predicts an expected ToM on the night of your observations. Your ToM might conform (within errors) and thus “you got away with it.” But what happens if your ToM does not conform? You might have found some significant behavior in the evolution of that binary. But your data are suspect because you did not calibrate your images.* Maybe you have found something significant, maybe not; your results are suspect all the same.
I have no doubt that your experiment showed what it showed; calibration may not be necessary in all cases. But protecting data integrity is the higher standard.
Ed
*Amusingly, Word thinks “data” is singular.
I agree completely (in…
Ed,
I agree completely (in principle) with everything you have written.
You raised a hypothetical problem if a TOM of mine (from uncallibrated frames) was inconsistent with those of other observers. Errors due to lack of callibration frames would not shift a light curve left or right (in time). If my TOM was incorrect it would be caused by some other problem: incorrect computer clock synchronization, transcription (typographical) error, incorrect time conversion, inconsistent time standard (e.g., JD not HJD), time stamp from beginning not the middle of the exposure, for example.
A more difficult problem would be variable star magnitude measurements different from those of other observers.
Roy
Roy,
Yes, in your case you are correct about your ToM, but my example was a hypothetical, as you note. I suspect that publishing ToMs using uncalibrated images would be frowned on by reviewers. But, I've only published a few papers in this field.
Ken Menzies and I published a paper on some of these questions (JAAVSO, 2022, V. 50) but we did not include uncalibrated images as I (speaking only for myself) never thought one would use uncalibrated images. In short: transformation promotes consistency in magnitude estimates while ensemble methods promote more realistic uncertainty estimates.* The assumption, of course, is that the results from measuring Landolt standard stars also applies to variable stars, at least in terms of consistency of observations between observers. I think that is a reasonable assumption given that we did our tests using two different telescopes with different sensors that were widely seperated geographically as well as several different Landolt fields.
Ed
*"Accuracy" in the case of variables is accuracy relative to a model of change that is considered valid; this is not the "accuracy" Wiley and Menzies adopted where the magnitude of the Landolt stars were considered known, like the speed of light is considered known. "Accuracy" in the case of variables might better thought of as "consistency." Consistent data results in more predictive models.
Again, I agree.
I…
Ed,
Again, I agree.
I routinely callibrate with darks and flats. But I do not transform, nor do I usually use ensembles of comp stars. There are specific reasons why I do these things.
Currently, my targets are southern EW type EBs, therefore with short periods of several hours. My aim is to determine TOMs of primary and secondary eclipses. The data goes to Variable Stars South for future publications. I have one target each observing night, and capture up to a few hundred images per night for time series photometry. I usually have one check and one comp star. I try to choose a comp star with a colour close to that of the variable (EW EBs don't vary much in colour during the cycle) so that I do not have to worry about atmospheric extinction. I use a V filter, and my Tv_bv is -0.054. Therefore if the var/comp B-V difference is 0.2 or less, the error of non-transformed magnitudes due just to the var/comp colour difference should be 0.01 mag or less. I have a 12 bit camera, and try to choose comp and check stars with magnitudes close to those of the comp to optimise the signal for all stars.
Measuring only 3 stars often allows me to keep them close together. For example, the var/comp and check/comp separations for the target I have chosen for tonight are no more than 30', and I image through a 200mm lens to a sensor about the dimension of an APS-C sensor. There is virtually no vignetting across this distance of 30' (which is, I suspect, why I 'got away with' no callibration). My field width is 163' and height 111' after setting the ROI (region of interest) to 1/2.
I have a mono camera, a filter wheel and V and B filters. Therefore theoretically I could take images in B and V and transform the data, but transforming data when the magnitude of the variable is changing rapidly is to me problematical and my B and V filters are not parfocal.
If I used an RGB sensor, I would transform the magnitudes.
Roy
Hi Roy
I only recently figured out how to set filter offsets in MaxIm DL. A great way to get B and V. It can wear out your filter wheel if the mechanics are not set up just right.
Don't know if other controls software is capable.
My FW8 has rotated approximately 100,000 times since I changed the O-ring. I got lucky and set the tension just right. If you can get a BV sequence every minute you may get a TOM for a two hour EB.
Ray
I wish I could do…
Hi Ray,
I wish I could do that, but I image through a camera lens, and must focus manually.
I've tried to find an adapter that controls the lens focus electronically, but the only one I am aware of (Astromechanics, ASCOM-controlled) is made in Russia. The company had to cease exporting because of the conflict with Ukraine.
Roy
I indicated previously that more data would be posted when available. It can be found here:
Cal Non-Cal Study.png
The equipment used is described in the first post in this thread, but essentially comprised a 12 bit mono CMOS camera, a filter wheel with Johnson V filter, and a 200mm Canon camera lens at f/2.8.
The targets are all EBs with short periods, and each line in the tables represents V magnitudes averaged from time series observations taken during one night. Aperture photometry for each field measured the variable star, one comp star and one check star. I think the column headings are self explanatory.
The results from 2023 were from images taken from an unguided mount. The mount was guided for the 2024 images.
I believe it is likely that the (perhaps surprisingly) small differences between the results from calibrated (darks and flats) and non-callibrated images are due to two main facts: the small separation between measured stars (the largest was 62’ and the smallest 18’) and the very limited vignetting across this central part of the field. Also, flats showed few or no dust motes.
Roy