Having multi-filter CCD observations transformed into the standard system is important: it makes our data cross-observer comparable and useful to the professional astronomers.
In 2014 we developed the tools to document and make this easier to do. Check out the documentation here.
And now M67, an excellent calibration target, is in the perfect position for evening observations.
What are you waiting for? Now is the time to move to the next level!
And there is help available: volunteers are listed below to help you learn each step of the process.
Goal of the campaign:
- Increase the number of observers who are submitting transformed data to WebObs.
24 out of 134 observers submitted transformed observations in Jan-Feb 2015
Let's see if we can improve this metric!
- Use the tools developed this last year for transformation
TG: Transform Generator
TA: Transform Applier version 2.30 or better
Process:
--- Get those M67 images! Its a convenient evening target in the month of March. Use best practice to get them BDF calibrated.
--- Extract the instrumental mags.
VPHOT is the most convenient way to do this. It automatically identifies the standard stars in the field and TG is prepared to work with its output directly.
Ken Menzies is available to help with questions:
If you use some other tool for extracting the mags then the issue will be using the proper labels for the stars. AUID's are always preferred and are available
with the photometry from VSP
--- Use TG to compute your transform coefficients
Installation and process details are available here.
Gordon Myers is available to help with questions.
You can get your coefficients from TG in a format compatible with TA (INI file) when you save your results.
--- Use TA to apply your transform your data
- You prepare you WebObs submission as you already do. The only adjustments to that process:
- Comp and Check stars should be identified by AUID. You may be able to use the VSP labels, but AUID's are preferred.
- The ChartID in your observation records needs to be a photometry page, not the picture chart.
Installation and process details are avaiable at here.
- Get the latest version of TA here.
George Silvis is available to help with questions.
Before you start submitting TA transformed data, you should review the results. TA has a feature built in to help you. If you check "Test TC" the transform process will be applied to your Check Star data. Review the results in the Report tab. If your observations can reliably match the transformed standard magnitude of the Check star, then you can submit your variable star measurements with confidence.
--- Tell us how you're doing by posting comments to this thread. What needs to be changed to this process to make it easier? We'll fix it!
Cheers,
George
SGEO
George found a bug in TG last week - the labels being shown on the plots in TG are incorrect. This does not affect the transform values, but does confuse everyone.
The new file will be on the aavso web site later this week.
If you want the update immediately, email me at gordonmyers@hotmail.com and I will send you the .new version 5.6 .pyw file.. This system will not let me attach the file to this post.
To install, just copy the .pyw file to your c:/Anaconda/Photometry directory. Then set up a new icon to launch - just as you've done with previous versions.
Apologies for the error...
Gordon
Circling back here almost 18 months later (I only dig into transforms every now and then), I'm reading through the posts, and I want to first thank Ken, George, and Gordon for all their support and assistance! These guys have clearly put in a LOT of time and effort to create tools and help us use and understand them.
I did want to respond to Ken's response to something James wrote a while back:
James (#39, above): "One thing that is very annoying, is that VPhot uses transform coefficient names like T(subscript R), T(subscript V), and T(subscript I) which has no equivalent in TG5.5. I'm guessing that TR = Tr_ri when doing the IR(Tri) transform and TV = Tv_bv when doing the BV(Tbv) transform?"
Ken's response (#41 in this thread): I can only offer that I do not find it "very annoying". I think it is common to see Tv [rather] than Tv_bv for the magnitude coeff in many equations in texts. The Tv-bv is more "complete" definition since it tells the reader which pair of filters were used to calculate the slope to generate that coeff. Multiple pairs can be used to generate the Tv coeff. In fact, VPhot knows that and in your telescope setup page you can enter a value for your Tv (or other) coeff for each pair of filters. This would permit VPhot to use the correct pair depending on which pair of filters you used for your images. Yes, VPhot shortens the coeff label to Tv rather than e.g., Tv_bv. I suspect the driving reason was not only space but the expectation that most people would use that pair. So yes, you do need to read something into the label but I don't feel that is a big problem? That is subjective, of course.
Golly, I gotta respectfully admit, I'm with James on this one!
Maybe not "annoying", but certainly frustrating. I appreciate the longer, and perhaps more accurate term Tv_vb, rather than the shorthand, especially when it comes to then inputting those figures into VPhot or some other program. Ideally, we'd all have such a complete understanding of all these terms and their derivation that this translation (conversion?) would become second nature, but its a bit like intermingling metric and imperial units, or worse, perhaps more like Greek and French for someone new to all this -- very hard to know which value to put where.
The ideal situation might be that the AAVSO CCD Photometry Manual, VPhot, TG, TA, Photometrica, and other software would all use the same terms, but I'm coming to understand that AAVSO is an almost all-volunteer organization, so some of these tweaks and programming changes don't always rise to the top of the pile. Some may not feel that any changes are needed.
I freely admit that as a newbie to transforms, my knowledge of these terms is not very deep, and I'm aware of the dangers of magic black box GI-GO issues when it comes to science. We certainly don't want folks submitting sloppy data just because some program spit it out (therefore it must be true!), but if we are careful to come up with good transform coefficeints, it does seem like a better solution if we are all using the same language. Just one fella's 2 cents, and worth every penny of it!
Circling back here almost 18 months later (I only dig into transforms every now and then), I'm reading through the posts, and I want to first thank Ken, George, and Gordon for all their support and assistance! These guys have clearly put in a LOT of time and effort to create tools and help us use and understand them.
I did want to respond to Ken's response to something James wrote a while back:
James (#39, above): "One thing that is very annoying, is that VPhot uses transform coefficient names like T(subscript R), T(subscript V), and T(subscript I) which has no equivalent in TG5.5. I'm guessing that TR = Tr_ri when doing the IR(Tri) transform and TV = Tv_bv when doing the BV(Tbv) transform?"
Ken's response (#41 in this thread): I can only offer that I do not find it "very annoying". I think it is common to see Tv [rather] than Tv_bv for the magnitude coeff in many equations in texts. The Tv-bv is more "complete" definition since it tells the reader which pair of filters were used to calculate the slope to generate that coeff. Multiple pairs can be used to generate the Tv coeff. In fact, VPhot knows that and in your telescope setup page you can enter a value for your Tv (or other) coeff for each pair of filters. This would permit VPhot to use the correct pair depending on which pair of filters you used for your images. Yes, VPhot shortens the coeff label to Tv rather than e.g., Tv_bv. I suspect the driving reason was not only space but the expectation that most people would use that pair. So yes, you do need to read something into the label but I don't feel that is a big problem? That is subjective, of course.
Golly, I gotta respectfully admit, I'm with James on this one!
Maybe not "annoying", but certainly frustrating. I appreciate the longer, and perhaps more accurate term Tv_vb, rather than the shorthand, especially when it comes to then inputting those figures into VPhot or some other program. Ideally, we'd all have such a complete understanding of all these terms and their derivation that this translation (conversion?) would become second nature, but its a bit like intermingling metric and imperial units, or worse, perhaps more like Greek and French for someone new to all this -- very hard to know which value to put where.
The ideal situation might be that the AAVSO CCD Photometry Manual, VPhot, TG, TA, Photometrica, and other software would all use the same terms, but I'm coming to understand that AAVSO is an almost all-volunteer organization, so some of these tweaks and programming changes don't always rise to the top of the pile. Some may not feel that any changes are needed.
I freely admit that as a newbie to transforms, my knowledge of these terms is not very deep, and I'm aware of the dangers of magic black box GI-GO issues when it comes to science. We certainly don't want folks submitting sloppy data just because some program spit it out (therefore it must be true!), but if we are careful to come up with good transform coefficeints, it does seem like a better solution if we are all using the same language. Just one fella's 2 cents, and worth every penny of it!
[quote=FJQ]
Btw, here are my latest coefficients (from TGA5.5)
[/quote]
James, your transformation coefficients seems to be a bit large. E.g. Tb_bv = 0.495 and Tbv=1.649. First two values are very large compared to expected 0.0 and 1.0 ones, some others deviate quite a bit, too. What kind of equipment (filters, camera) are you using?
Best wishes,
Tõnis
Thanks George for confirming TV = Tv_bv. Ken, I had to re-run the sequence I made in VPhot becz my M67 field from 30-Aug-14 were getting different numbers of comp stars. I ended-up using 36 comp stars that were common to all fields of my 12 images (3 per BVRI). My numbers got more consistant with what I calculated earlier using AIP4WIN and TGA4:
I checked to see if I get "valid" R magnitudes when transforming my U Gem data from 25Feb15. It still shows an invalid magnitude for R when using a Tvr value of 1.110 and Tr_vr(TR in VPhoto) value of -0.039 for transforms using V & R data ; see:
Before you mention it, I did try Tri and Tr_ri (TI in Vphoto) and it yielded about the same meaningless result for R magnitude.
I have 6 fields of NGC 7790 I'm going to get transform coefficients using Vphoto and TGA5.5 to make sure I'm getting a better average trans-coeffcients before trying to transform any of my variable star data in Vphoto. If this again proves fruitless in transforming R magnitudes, I'm going to go through the extra step and import them into TA. Thanks again for the advice guys!
James
James:
Ahhhh! This issue relates to the lack of a standard comp magnitude in the R filter? Note that your single comp star 114 shows 0.000 for the R catalog value. You need a comp with a valid R magnitude or the target value will clearly be wrong. Take a look. Select another comp(s) with both a V and R mag.
Ken
Thanks Ken for pointing that out.....I re-did a sequence for U Gem and got comps with R-mags for the U Gem field. I'm still not satisified with the results I'm getting, doing a two star transform in Vphot:
I'm not going to use this tool if I can only transform 2 at a time. This is pretty useless for us folks who have 60-200 time series at a time to do. I tried using the Vphot time series function to output my B-data for U Gem taken over 4 nights. While this worked and outputed to AASVO extended format .txt-file, TA would not transform it; see attached links. I'm not sure if this is because I used one check and one comp star besides the object, or I don't have enough transform coefficients loaded; see link below. I know I loaded the 13 coefficients derrived in TGP5.6, but there were alot of gaps in TA's coefficient screen.
There doesn't seem to be any problems for getting transform coefficients out of Vphot to process in TGP, however, I tried loading my NGC7790 files to Vphoto and cannot get them on the image screen; downloaded them 4-10x each! While i see them in the Server Processing Queue for awhile, they disappear and never re-appear in my images.
Until this process becomes easier or more transparent, I WILL NOT transform my data. While Vphot seems like a good program to process our AASVO data I don't want to waste any more hours getting coefficients, waiting for files to upload (uploading NGC 7790 more than 10X), and not being able to use data that another program wont read (both AASVO approved) seems to tell me the process is not yet done. I know I could buy the latest and greastest version of Maxim 6 to process photometric data into a format TA would recognize, but I'll wait til this stove-pipe processing of Vphot data has matured or I find an extra $600 somewhere...whichever comes first!
James
http://www.astroimage.info/images/AAVSOReport_U-Gem_B_20150310-1.txt
http://www.astroimage.info/images/TA.jpg
http://www.astroimage.info/images/TA-COE.jpg
A maintenance release of TG - Version 5.6 has been released. Visit www.aavso.org/tg
There are threee changes -
Hi guys,
I'm just preparing to shoot M67 (hopefully tonight!) and begin the TG/TA process. (I've been submitting photometry observations to both AAVSO and BAA for a while now and reckon its time I 'transformed').
I use AIP4WIN and have a question about labelling the stars and generating the appropriate file for use in TG...
1. What is the star/line labelled "M67" in the TG demo file? Is it a 'random variable'? I get the idea of using the comp stars' values (that I've just generated from VSP= 'EV Cnc') but what do I allocate as the Variable star "Target" when AIP4WIN asks for one?
2. Presumably there's one AIP4WIN file generated for each B, V, R & I image I shoot, and I submit these separately to TG after calibration?
PS. I loaded Anaconda & TG onto an XP PC. Opening the TG file using PythonW.exe doesn't work - but does when I use Python.exe.
Regards
Tony
I just noticed, in Transform Applier (TA) there is no transform coefficient block for Ti_ri that TG outputs.
There are also a large number of blank coefficient blocks in TA that require Tbr, Tbi, Tb_br, Tb_bi, Tr_bv, and Ti_bv coefficients. Since these are not outputted by TG I wonder if we're supposed to manually calculate them and that why I'm getting a error in transforming the AASVO extended format from Vphot?
I wish that TG would output definative Tb, Tv, Tr, and Ti numbers instead of us guessing, Tb=Tb_bv, Tv=Tv_vb, Tr=Tr_ri, Ti=Ti_ir for the (B-V) & (R-I) systems. This makes transforming more difficult when it need not be, especially when using a cloud-based program (Vphot) and two desk-based programs, TG and TA. Thinks I need to go back to school and study more complex anaylisis before tackling photometric transforms! :)
James
James, let me address some of the issues you are running up against:
- Ti_ri
There are lot's of coefficients that can be computed. TA performs the AAVSO recommended transforms using the following coeffcients (from the TA Help)
- The AAVOS recommended transforms that the TA performs require the following coefficients:
for U B V R I
UBVRI Tu_ub Tb_bv Tv_bv Tr_vi Ti_vi
UBVI Tu_ub Tb_bv Tv_bv Ti_vi
UBV Tu_ub Tb_bv Tv_bv
BVRI Tb_bv Tv_bv Tr_vi Ti_vi
BVR Tb_bv Tv_bv Tvr
BVI Tb_bv Tv_bv Ti_vi
VRI Tv_vi Tr_vi Tvi
BV Tb_bv Tv_bv
VR Tv_vr Tr_vr
VI Tv_vi Tvi
Ti_ri is not requried. You don't have to fill in all the blanks in the TA Coeffcient tab, just the ones that are going to be used.
- Coefficient naming. The Tx_yz pattern completely defines the coeffcient, how its derived and how it is applied. VPhot's Tx pattern is incomplete; that's why the AAVSO is recommending the new pattern. In VPhot you can define in Admin a Tb as Tb_bv by setting the color. You just have to remember that Tb in VPhot is Tb_bv and not Tb_vi or whatever. I would like to see VPhot update its nomenclature.
- In an earlier post you showed TA failing to transform. It looks like you had loaded only the B observations. You can't transform with only one filter. Load into TA all of the VPhot files that were produced, B,V R and I,
Feel free to contact me offline with any other issues: SGEO@GASilvis.net. Send your TA INI file and the extended format data you are trying to transform.
George
Tony,
First, I strongly suggest you consider using VPHOT instead of AIP4WIN. It saves literally hours of time hand selecting the stars - and occassionaly making mistakes if you're like me. I love AIP4WIN and still use it all the time for time series, but selecting 30 - 60 standard field stars in M67 is tough.
If you are gonig to use AIP4WIN, here are the answers to your questions -
1. The M67 line in the demo file is not used by TG. It's in the file because I wanted the comp star ids AIP4WIN generates to match the comp star numbers of the Boulder ids. So I named the target star M67 and picked a star at random.
2. No, you should just have just one AIP4WIN file. Load all the images for all the filters before selecting the comp stars. Then generate a single report - look at the figure in section 4.4.1.5 of the TG User's Guide for setting up the report options. AIP4WIN will generate a file you can directly load into TG.
I can't explain why XP prefers the python.exe, but glad it works!
To George,
RE:"You can't transform with only one filter. Load into TA all of the VPhot files that were produced, B,V R and I,"
Your probably right but it seemed like I had a format problem because of the insufficiency of the program message.
I'll try to do this again with the other U Gem Colors. If fhis works I owe you a tall one!
James
O.k. When Vphot B & V are used together in TA I got transformed data. This took in WebObs so I assume everything is correct. I next'd try to do R & I together in TA and I got an error message. I didn't want to just add my R&I data to my previous B&V data since I sued different comp/check stars.
I reprocessed my V data, using the same comp/check star sequence I used for my R & I data with Vphot timeseries. I then took the resulting V data and mixed it with R (V&R) ran it through TA and got transformed AASVO submittible data. I erased the V data from the upload, since I already uploaded this with B using different Comp/Check stars. Did the same process for V data mixed with I (V&I) through TA and got transformed AASVO submittible data for I.
Well now that this has been worked out, I still have not ben able to successfully upload any other CCD images into Vphot. I erased my 60 U Gem CCD images, thinking I might have taken-up too much space, but the problem persists. Aw well 1 step forward, 2 steps back......story of my life! :)
James
James
The correct process for transforming BVRI data is to do it as a single group. It should not be broken up into pairs of filters. You will note that there is a group field in the extended format record. When a set ot observations are transformed together they are assigned the same group identifier. That's so when data in AID (where the WebObs data goes) is analysed it will be understood how the transform was done, which obs were transformed together.
Actually, the group field is meant to be used in the opposite fashion: you are supposed to assign the observations to groups. But that's a whole extra job step that most will skip. If TA doesn't see user assignments then it will attempt to form the groups for you, hoping that the set of observations submitted are related. You see this in the resulting records.
Note also that there is no requirement for observations in a transform group to have the same comp star. So your original data was fine for a BVRI group.
By using 3 pairs of transforms as you described (BV, VR, VI) you computed 3 different transformed values for V. Which one is correct? For transformed data, only the one accompanied with its group buddy, the V in BV, which you submitted. The I and R transforms are not really valid without their versions of V which should/could not be submitted because we are not supposed to submit the same observation more than once.
So, If you have a set of observations that can be logically grouped (ie same night, same sky, same airmass+/-) then transform them together. It's easier too!
George
Hi James:
I'll take on the other issue: "I still have not been able to successfully upload any other CCD images ".
Don't worry about 60 images in your image list. I upload about 1000 after every observing night. I have 10^4 or more in my list until they are finally deleted by VPhot on a regular schedule (few months?). Yesterday there were more than 1400 images in the queue at one time. They did get reduced although it took a while (hours?).
When you say you cannot upload any other images, do you mean U Gem images or other target images? There is a known but yet not understood bug related to "duplicate/similar" images. There is a QC filter in VPhot to prevent duplicate images from being submitted to AID. Sometimes (?) it gets too ambitious! It really has a problem with DSLR images where the three RGB images have the same name and time. So duplicate/similar (and I mean VERY similar) images occasionally have a problem? It clearly occurs when repeated uploading of the same target images with different filters occurs.
Any way, are they more U Gem or not? If not, let's try to see what the issue is? I need some more input here. A few examples and discussion would be helpful. Let's get this resolved so your backward steps are reduced and the forward steps win!
Ken
A few days ago we had a debate on the possibility to calculate transformation coefficients in Sloan type filters. It became clear that still is not possible to use AAVSO data base to automatically create VPhot sequences for Sloan filters. Simply the data from APASS have not yet been integrated. I took 6 series of M67 standard star field with BVIc and g’r’i’ filter sets. I have finished the TG calculations for BVIc filters. I found that to work with TG is easy, and results are impressive. Before TG I was used spreadsheets and believe me TG spends lots of time and efforts.
The next challenge for me was to calculate the transformation coefficients for my Sloan type g’r’i’ filter set.
The important information:
The results of my work are available to anyone who is interested in the applicability of the Sloan g’r’i’ transformations. As attachments one can find:
The next step is to use the mentioned above multi-filter TA (TrasformAplier) procedure for g’r’i’ (George Silvis’s idea) and to prepare WebObs data good for submission.
Any comments and remarks are welcome.
Velimir
Velimir, that's an impressive piece for work!
Now you're ready to try that workaround to get TA to apply a VRI transform to the data.
Remember the first step before jumping to a transform of your data is to try the TCtest option of TA. This will apply the transform process to your check star data. If you see the check star instrumental magnitudes being transformed closer to the standard magnitudes and are getting reasonable error, then you are good-to-go,
Any problems you are welcome to send me, or post to the forum, your TA ini file and input data and I'll help you sort out any issues.
Cheers,
George
Velimir:
I was waiting anxiously for your results. I almost sent you an email yesterday to check.
I'll use some more subjective words to describe your effort: TREMENDOUS, AMAZING, WOW, THANKS!
Ken ;-))
RE:"By using 3 pairs of transforms as you described (BV, VR, VI) you computed 3 different transformed values for V. Which one is correct?" George, the reason I did not want to submit B and V with R and I was that I was afraid that the transforms would not work with image sets using different comp/reference stars. As Ken pointed out, R mags are missing for 1/2 of the comp stars for this U Gem field. Using Comp/Reference stars with magnitudes much less than whag B and V fields offered seemed like it would introduce more error in ghe magnitude estimate. As soon as Vphot comes back on-line, I'll upload some more M67 I shot 2 nights ago.
Thanks for all the help! Ill probably just do BVI photometry since most fields have all these magnitudes represented for comps.
James
Thank you George and Ken,
Unfortunately I am still far away from applauses
I started with TA (TrasformAplier) procedure for g’r’i’. First I have map g’r’i’ filters is BVI (my other Jjonson-Cousins filter set). After several attempts and TA warnings, I decide that calculated by me transformation coefficients are in wrong sequence. I have created updated version of my spreadsheet table, recalculating some of the coefficients and adding one more. At the end I have stacked – one of the coefficients sequence obviously is wrong (if not both). From a certain point, I began to think that I do not understand the things anymore.
I need some urgently help to get over that maze. The two variants of the calculated coefficients are listed below and the second updated variant of a spreadsheet is attached as a file. Please pay special attention to the lines in bold, the first is different (v1 and v2)and the second is a new addition:
var.1. Old file (previous post) \ M67_Transf coeff_gri_38_PVEA stars.xlsx
Tg' (SG-SR) -0.0365 (Slope SG-SR vs SR-r')
Tr' (SR-SI) -0.0477 (Slope SR-SI vs SR-r')
Ti' (SR-SI) -0.3071 (Slope SR-SI vs SI-i')
Tg'r'-slope 0.9457 (Slope SG-SR vs g'-r')
Tg'r' 1.0574 TC=1/Tg'r'_slope
Tr'i'-slope 0.7406 (Slope SR-SI vs r'-i')
Tr'i' 1.3502 TC=1/Tr'i'_slope
var.2. Updated file\ M67_Transf coeff_gri_38_PVEA stars-updated.xlsx
Tg' (SG-SR) 0.0178 (Slope SG-SR vs SG-g')
Tr' (SR-SI) -0.0477 (Slope SR-SI vs SR-r')
Tr' (SG-SR) -0.0365 (Slope SG-SR vs SR-r')
Ti' (SR-SI) -0.3071 (Slope SR-SI vs SI-i')
Tg'r'-slope 0.9457 (Slope SG-SR vs g'-r')
Tg'r' 1.0574 TC=1/Tg'r'_slope
Tr'i'-slope 0.7406 (Slope SR-SI vs r'-i')
Tr'i' 1.3502 TC=1/Tr'i'_slope
Velimir
Seeing how Vphot remains frozen, at least to me, I wondered if I could process output files in my version of MaximDL (5.23) to TA2.30 and have them read and transformed like the VPhot data that contains instrumental magnitudes. My main problem leading to failure using TA with this older version of MaximDL was that I always tried to reduce Ensemble data and not data reduced by using just one Com and one check star. Further, although MaximDL is easy to use and puts out AASVO extended format data for upload, MaximDL does not output instrumental/raw magnitudes but assigns an arbritrary zeropoint to the set of magnitudes being measured.
In another post, Arne states , regarding transformation coefficient(s)...."For differential work (a target and a comparison star in the same field), many of the complications drop out, especially if the target and the comp are roughly the same color. The nightly zeropoint goes away, and first order atmospheric extinction can be ignored if your field of view is small and you are observing at low airmass. That is why we generally try to choose comp stars that are either similar to the target, or at least similar to other comp stars that you might use in an ensemble or later in the star's light curve. " See link to original post:
http://www.aavso.org/calculating-transformation-coefficients-and-use-comp-stars#comment-6982
My spreadsheet seems to bore this out. I was able to transform and upload the MaximDL 5.23 derrived transforms into WEBOBS and compare them to the existing same-time transforms done with VPhot. The average difference btw these 6 pairs of V & B transformed magnitudes were 9.31E-07! (See attached spreadsheet).
http://www.astroimage.info/images/Trans-Test(Vphot-MaxDl523).xls
The typical amount of difference btw the MaximDL and VPhot is 10-20X less than the error of the photometric measurement. The average difference was much less than even this. I guess I can use MaximDL 5.23 after all to get transformed magnitudes that compare very favorably with VPhot, AIPWIN4, and MaximDL6.
James - FJQ
Sure you can use it, The thing I haven't found a way to do in Maxim is to save a sequence that you can apply to plate solved images so you don't have to click on all the stars in an image track it through all images and then check all the images carefully to make sure that all the apertures are on the right stars and not shifted. That and to a lesser extent having the output in a form you don't have t reformat for use in TG are the big time savers using VPhot.
I have the same problem with freezing because I am on DSL which only uploads at about 600 kbit. I have found that if VPhot tells me its load is moderate or heavy I have trouble. If the load is light I don't.
There is no problem using a single comp star for your transforms because you don't care about the accuracy of the magnitude only the slope of the regression curve. Sure there will be some second order extinction effect but it is very small. So if you pick a well positioned comp at around 0.5 < B-V <0.7 and you image at airmass close to 1.0 it should be small compared to your overall uncertainty. Another thing you can do is to take your images in IRVBBVRI sets. Stack te pairs using shift by whole pixels only (or use the averages of the two measurements in each color and the average of the center image times and airmasses) that puts the resulting image times and airmasses as close as possible and the images that are affected most by airmass change are at the center of the set. then each of the resulting measurements from a set get the same group number in your input to TG.
One thing I am not sure of in Maxim is what it does to the image time when you stack them. I am not sure that you get a correct average of the middle-of-integration times and airmass. That was a problem in older versions of Maxim. I don't know if that has been corrected in newer ones but it is easy to check.
Brad Walter
Brad Walter
To: Brad,
RE:"The thing I haven't found a way to do in Maxim is to save a sequence that you can apply to plate solved images so you don't have to click on all the stars in an image track it through all images and then check all the images carefully to make sure that all the apertures are on the right stars and not shifted. "
I agree! Vphot is especially helpful when doing standard fields to get your transform coefficients. However, for the case where there is a variable field to reduce, labelling the variable, Comp, and Check star is very easy if you have the AASVO chart with the same orientation and image scale as the CCD images you processing with MaximDL (I use F scale with north up and lines pointing to each star with a label).
RE:"One thing I am not sure of in Maxim is what it does to the image time when you stack them. I am not sure that you get a correct average of the middle-of-integration times and airmass. That was a problem in older versions of Maxim. I don't know if that has been corrected in newer ones but it is easy to check. "
There are two JD date fits fields in MaximDL 5.23. One is start and the other id mid-point. Using CCD autopilot5, the airmass and JD are recorded in the FITS header with the plate solved RA & DEC. When processing photometic anaylysis in MaximDL, the mid-point JDs are referenced.
James
Thanks, What happens to the mid image date and the airmass to the result of stacking two or more images? Is the mid image date of the resulting image the average of the mid image dates of the stacked images? Is the Airmass the average of the airmass of the stacked images (or even better the airmass that corresponds to the arimass at the averaged date)? I recall in older versions of Maxim, that the individual images had the correct times and airmasses but the image that resulted from stacking did not. That is what I don't know is still true. I use Maxim for image capture and sometimes plate solving. But I don't use it for anything else. I moved to Mira Pro over a decade ago for calibration, aligning and stacking,and photometry.So I am out of touch with Maxim's current capabilites in those areas.
Brad Walter
To: Brad,
Here is a little chart I make for plotting my comp and check stars with the variable in question (X Leo in this case): http://www.astroimage.info/files/X%20Leo%28Oct2015%29.xls
It takes me about 2 mintues to make up one from the AASVO chart making page. Please note that the chart numbers differ btw the image chart and the field photometry table; depending when you click the image, the image chart name changes, no big deal as values are unaffected.
Using this besides an open window of MaximDL in Anylyze, Photometry of your variable field is a piece of cake, especially if your field is small like my ST-10 FOV at 2500mm!
James
If you upgrade your Mac from OS X Mavericks (v. 10.9) to OS X Yosemite (v. 10.10) TG will not run properly.
We will post an update once we sort through the problem.
Gordon
This post become possible because of the great help from George Silvis who corrected my spreadsheet for Sloan type transformations. The corrected file is attached.
The corrections from George Silvis:
Tg' (SG-SR) Slope SG-g' vs SG-SR
Tr' (SR-SI) Slope SR-r' vs SR-SI
Ti' (SR-SI) Slope SI-i' vs SR-SI
Tg'r'-slope Slope g'-r' vs SG-SR
Tg'r' TC=1/Tg'r'_slope
Tr'i'-slope Slope r'-i' vs SR-SI
Tr'i' TC=1/Tr'i'_slope
George created an additional procedure of how to apply these coefficients by TA (TransformApplier). Probably he will explain it later, or I will do when I repeat everything myself and when I become more familiar with it.
Velimir
P.S. The old versions of my spreadsheet should be erased since it contains some inaccuracies.
When we started the campaign on 3/3/2015 24 out of 134 observers submitting data in 2015 had submitted some transformed data.
Two weeks into the campaign we're at 27/137. Making progress. I expect a slow start as there's a lot of things to learn to reduce you M67 observations into an accurate set of transform coefficients. And the CHOICE CCD Photometry class is just reaching chapter 6 which deals with transform theory. And the weather has been miserable in the North East.
Let's press on!
George
To: Brad,
After getting use to processing Maxim DL 5.23 files in TA I've submitted Transformed BVI (and sometimes R) of the following:
GK Per (41), AL Com (102), X Leo (28), & U Gem (60) btw 25Feb15-15Mar15.
As soon As I get another set of Reds of NGC 7790, I'll re-calculate the Transform Coefficients and I'll also include and M67 field I took 1 week ago.
I may have to take another set of standard fields as I switched out my old CFW for a SBIG CFW-10 I got from Paolo Candy in Italy. Although the filters haven't changed, the optical window above the ST-10XME was replaced with the optical window of the CFW-10 so there maybe some variation of dispersion that will impact transform coefficients values.
James - FJQ
Almost fortnight ago I started to learn how to work with Transform Generator (TG) and TransformAplier (TA). I found both of the software very useful.
The following material I have written to myself as memo but later I decided to share it with the AAVSO community. The purpose is to encourage the people who consider the transforms as a very hard work. It is not.
The material is attached as PDF file and mainly refers to the way of working with TG and the determination of the transformation coefficients.
Any remarks and corrections will be highly appreciate.
Velimir
Hi,
I have successfully used TG to get my transformation coefficients. So I now have a few basic questions .
Thanks,
Keith Graham
Keith,
You've made good progress. On your questions -
Yes - those well off the slope should be eliminated.
Not necessarily. In the extreme example you'd eliminate everything down to two points and the linear fit would show zero error. My general approach is to eliminate all the points outside the original 3 sigma lines. Then I look at what remains to see if any clearly appear outside the norm. The actual transform value can go up or down as points are removed. (Also, remember some transforms are the reciprocal of the slope so the results can look "backward" from what you expect.)
Yes, once saved you can not go back. You can, of course, start a new analysis by reloading the original files. I sometimes do this so I can compare results.
There is an easy link. Create the output file in TG (page 11 of the users guide). Then, in TA, click on the Coefficients tab, select the "change" button, and load the TG file. Then you're set to go. Don't be concerned that some coefficients are not filled in - they are not used in calculating the transforms.
Gordon,
Hi Gordon,
Well here is my problem. In TG I create a transform set, and I then eliminate the outliers. I now have the coefficients as I want them and then click SaveTransform Set. I get the message that the set has been saved-but it does not say where it is saved. Now I go to TA, click the Coefficients tab, then click the Change button. A window appears that shows I am in the Anaconda/Photometry, but the window is clear – no files. When I look in Explorer in the Anaconda/Photometry file, there is a file entitled “transform _values.ptgp that has the date and time when I saved the file. When I tried an earlier run, TG saved a .ini file that I could bring up in TA. There is no such file for this latest run. However, I can retrieve this latest set in TG.
So, why can I see the file in TG to retrieve it, but not see it in TA when I want to load the coefficients? And why did TG not save the file as a .ini file? It appears to me that those saved TG .ini files are buried within the transform_values.ptgp file, but how do I access them?
BTW-since I have only 1 file, I am not averaging it with any other file at this time. I see how to save an averaged file (p.11 of the Guide), but I cannot get that far since the yellow table has NA for all TCs.
Cheers,
Keith
Keith,
I suspect the problem is caused by not selecting the small box above the single row of results shown on the average transform page. It must be selected so the "compute average" works. Without that, the create the export ini file won't work. See the attached screen capture.
I need to add an error message if someone doesn't select at least one box to average.... Sorry.
Actually the problem occurrs well before the average option. When I run TG. I first click the Select button and bring up the txt file from the Anaconda/Photometry folder that contains the M67 data. I then click the Calculate Transform button and get the Transform Values. I then run the plots and eliminate the outliers, I click the Save Transform button. A small message appears that reads: Transforms saved at UT 2015-03-28 14:49:15. But this file does not appear in the Anaconda Photometry folder.
Now, when I click the Retrtieve button, that file is shown in a white box on the Review page along with other saved files. A Retrieve Transforms button is under that white box. But none of those files appear in the Photometry folder and there are no .ini files in that folder. So I cannot import the results to TA. I am able to click on any of the listed transforms and bring them up. But I have no idea where they are, and I there are no ini files to use for TA.
So my question is where is the saved ini file that I has the transformed data that I can import into TA? Is it being saved to a different location other than the Photometry folder? I would think it would be saved to the Photometry folder since that is where TA goes to retrieve it.
I am stumped
.But once we get this solved, I will then try the Average function. But first things first.
Cheers,
Keith
Keith,
The initial save is to an internal file the program uses that users don't have access to. The only way to export the data is to create the average and then export it...
Gordon
El Stupido here -
I downloaded TG onto my computer a couple weeks ago and used it once (with satisfactory results). Now I need to use it aagain, but can't find it. It isn't listed under Anaconda, nor does it show up in the program listing. How do I find it?
El Stupido
Thanks ever so much, Ken. I found it under a sublist for Anaconda/Photometry!
El Estupido, CDD*
*CDD = Certified Dumb Donkey
So I
AH HA! Yes. that works.
So I went back to the TG Guide to see what I had missed or misinterpretred. The guide gives the instructions to create the TCs, the plots, elimination of outliers and then click the Save Transforms button. (instruction #5) I figured this meant this was a file I could use inTA. The next instruction (#6) goes directly to Review and Combining Transforms. Since I had only 1 set at the time, I figured since there was nothing to average, I could call up that single saved file in TA. So my hangup was not realizing I MUST average the saved transform set(s) before it can be used in TA. This was not clear to me until instruction #8 and until you pointed it out to me. Perhaps a stratement that transform sets MUST be averaged before they can be saved to a file that is read by TA could be placed in the guide as part of instruction #6. As I think about it, averaging does make sense since we should have at least 3 set of images taken over 3 imaging sessions (at least that is the way I did it when using my own spreadsheets). I see that TG allows for 6 sets of images, so that should be plenty.
Thanks for you hellp andpatience,Gordon.
Cheers,
Keith
I am a newbie at transforming.
I am attempting to develop transformation coefficients for CCD observations. I have used TG and TA. In TA there is a section on extinction coeficients. Unless I have missed it, extinction coefficients do not appear to be discussed in TG, save for the mention in this discussion that since all stars are in the same (or overlapping frames) they will have essentially identical airmass.
That's true, but if you are observing at an air mass of 1.4, the colors will be affected by a considerably different amount. U observations will be significantly affected, but I observations will be barely dimmed. Please don't suggest that I wait for a better time, because it won't happen. 1.4 is the airmass when M67 culminates at -30 degrees (S) latitude. NGC 7790 isn't useable south of the equator.
Do I need to correct for this extinction issue, or am I worrying about something needlessly? If I need to do something, would it make sense to alter the input instrumental magnitudes from VPHOT all by the same amount (different for each filter, of course) depending on the extinction coefficients for each filter?
Uncertain,
Lew
Lew,
You ask a good question. Sorry it took so long to respond. Do your transform coefficients need to be developed with observations that have extinction applied?
I think they do.
Looking at the transform equations, the magnitude coefficients (eg Tv_bv) get applied to the difference of the star and comp reference colors, which are extincted values. In the case of the color coefficients (eg Tbv) you are counting on the coefficient to transform the observed color difference of star and comp to the reference level.
This is fairly easy to do in VPhot: In the Admin/Telescop information you can enter extinction coefficients for each filter. You could use average values for your observing environment and move your data in the right direction.
I haven't been doing this; time to recompute my coefficients too! Just a matter of reprocessing the old observations of M67.
When it comes to running your data through TA you do not need to tell it to apply extinction to your observations because extinction cancels out in the transform equations for the observation values. This is because the star and comp have the same airmass in the small CCD frames.
I'd love to hear more discussion of this. Maybe its time to update the instructions for using the VPhot and TG combination.
George
I do not think extinction needs to be applied to the instrumental magnitudes.
In the transformation coefficient generation process one is plotting graphs of differences, e.g., b-v versus B-V (Tbv), or B-b versus B-V (Tb_bv).
However, the transform coefficients are slope values. Thus, it does not matter what the y values / intercepts are. It does not matter if we shift the magnitudes up or down due to extinction correction. The slope is the same.
This does assume that one collects the images in a short period of time and similar airmass and same small FOV on any given single night. One can average two or three days of independent coeffs because they will have the same slope but not the same intercepts. These are, in fact, the assumptions we always make in generating our transformation coefficients and applying them.
Yo, Arne!?
Ken
Ken,
We're in agreement that for a transformed observation whre the star and comp have the same airmass, the extinction correction drops out.
The question is whether the data used to develop the coefficients needs to be extincted. b-v compared to (b-bextinction)-(v-vextinction) are going to be different because the extinction factors will be different by color.
I'm getting set to recompute my coefficients with and without to see if it is a material difference.
George
I haven't rearranged terms recently but here goes!
(b-bx) - (v-vx) can be rearranged to (b-v) - (bx-vx)
I think this says that there is only a constant/identical shift in all y values (b-v) in the plot because when one applies extinction to the b and v magnitude, the b mag changes by a constant (extinction value for b times airmass), and the v mag changes by a constant (extinction value for v times airmass (same airmass)). Since the airmass and the b or v extinction values are constant (small FOV/change in airmass) for all comps and the difference is calculated, the shift/correction is the same for all comps?
Thus no change in slope??
Ken
Ken, you're right.
The derivation of the transform coefficients should not be affected by the extinction factor if the standard field you are analyzing is at the same airmass.
Both the magnitude and color coefficients have reference color on the X axis. The extinction applied to the instrumental values (either X-x or x-y) for the Y axis will be a constant offset, with no impact on the slope which is the coefficient being found.
So, Lew, go ahead and create your coefficients with M67 low in the sky. You can use these to transform your observations.
Thanks Ken.
George
Thanks for the consideration of this question, folks!
BTW, "Low" was an altitude of 45 degrees at transit.
I'm happy now. :-)
What about 2nd order extinction? I know it is small but if you are trying to do really accurate photometry, the second order correction depends on the difference of two instrumental magnitudes such as k"(b-v)X. It is so small I normally don't worry about it, but If you are really trying for very accurate photometry, and you can't image M67 at really low airmass, then perhaps you might want to consider removing second order extinction, which you can do by imaging M67 at the lowest airmass possible, in this case at about 1.4 and then at an airmass of about 1.0 greater and ploting , for example,
∆v = k”v*∆(b-v)*X + ∆vo
and
∆(b-v) = k”bv*∆(b-v)*X + ∆(b-v)o
Of course you need to remove 2nd order extinction from your instrumental magnitudes before you transform them with TA as well if you are shooting for this level of acccuracy. I don't think the extinction coefficient inputs for TA work with 2nd order because it looks like they are constatn coefficients of X for each filter and the coefficients of X are functions of the color index for second order.
It might be interesting to quantify 2nd order if for no other reason that to be sure it is insignificant. If I recall correctly when I compared transformation coefficients at airmass 1 to those at about airmass 1.5-1.7 they were different by a couple of thenths of a percent.
Brad Walter
Brad:
I think you answered your question in the second sentence - "I know it is small" and the last sentence - "different by a couple of tenths of a percent". Since CCD precision is not that good in most observer's skies, IMHO, the impact of second order extinction is too small (in the noise) to worry about. Getting observer's to do a first order transformation of their differential photometry is more useful. Millimag precision and accuracy is also tough when the comps are only good to 0.02 mag.
One response is appreciated but I hope we do not pursue another "precision" discussion in this forum thread.
Ken
I agree with Ken. If I or Brad were living in the Atacama desert, where skies are pristine, it would be worth the extra effort to quanitfy the 2nd order effects like extinction. I dont know about Brad's sky, but my sky 8 miles east of downtown Los Angeles is plagued with differential color gradients, 10 million Lum rotating sky lights and a backround glow whose color is strong orange-red. My backround in the Rc filter is by far the worst with V 1/2 as bad and B and I 1/3's Rc's backround.
My Data from Mt. Pinos, CA (No Atacama, but at 8,000 ft elev, 90mi NW of Los Angeles, sky backrounds 15x darker), would probably benefit if I had good comp star magnitudes. I might do this in the near future.
When I move to darker skies in the next few years before retirement, I'll go the extra mile to quantify these 2nd order errros. Hopefully by then will have better comp star magintudes after Gaia's photometry data is archived.
James