I know there is a VPHOT forum but this post is more meta-VPHOT which might not be noticed by folks not monitoring that forum and I think it merits a wider audience.
There is a bit of background. I took the Uncertainty course and, to my knowledge, there has been no call for feed back from users. That, perhaps, should be another thread but one of my feed back items has to do with VPHOT so I will mention that one here. The course was advertised as being based on VPHOT but we learned the instructor has never used VPHOT, hence could not address questions related to course material and VPHOT. That then raises the topic of this post.
What is VPHOT all about, really? (Let's clear away all the obvious things like it's a tool for measuring star magnitudes, etc.) It has been promoted by staff, featured at meetings, etc. Is VPHOT somehow "officially" recommended (perhaps as a tool to achieve more consistent data for the AID)? If not, why not? I know it is a work in progress but is there some long term plan or vision for it?
There are at least some of us looking to work around VPHOT's current deficiencies to make our own data flows more efficient but, depending upon where VPHOT is going, might that mean duplication of effort? Are efforts to improve VPHOT ultimately doomed?
Hi, Jim. I'm going to reply to your CHOICE comments and let someone else comment on the meatier issue about the strategic role of VPhot. I do not think the Uncertainty about Uncertainty class was ever advertised as being "based on VPhot". What we stated, in the class description, is:
Basically, we use VPhot in the screen shots and that is it. The majority of students of the 3 Uncertainty courses did not use VPhot for their work. I just checked the forum for the latest Uncertainty class (we plan to keep all CHOICE forums permanently available to members of the class) and when a VPhot question was posed, I saw that it was answered.
"Basically, we
Aaron wrote:
"Basically, we use VPhot in the screen shots and that is it"
That seems rather strange since "VPHOT is envisioned as a flagship product for the AAVSO."
I don't know where else to talk about the uncertainty course since that forum is closed (and was closed to non-participants anyway). Are you seeking any kind of feed back?
Hi Jim,
VPHOT is envisioned as a flagship product for the AAVSO, much like VSX or VSTAR. It has capabilities that other photometry packages do not, such as the extensive graphical output to help in quality assessment. We recommend it as the package of choice for those using AAVSOnet, or those that do not wish to purchase vendor software. That said, you can of course produce quality photometry with a variety of packages; you don't have to use VPHOT, but you will get more support from us if you do.
There are a number of features that are clumsy, especially for multifilter, transformed data. We'll be working on that soon. Personally, I'd like to see suggestions for making it better, for adding new features, etc., rather than have you spend the time developing ancillary software to work around its deficiencies.
That said, the feature that is obviously missing is the ability to calculate transformation coefficients. This can be accomplished with a separate spreadsheet, and I'm leaning in that direction since I know of several basic spreadsheets that do this, but it could also be an add-on to VPHOT. Right now, I want to concentrate on the application of known coefficients and make sure that the program does that right, and then add some more bells and whistles such as HJD or MJED calculation. Geir, Matt and I need to sit down and discuss new features soon!
Why would you say "Are efforts to improve VPHOT ultimately doomed?" Explain!
Arne
"Why would you say "Are efforts to improve VPHOT ultimately doomed?" Explain!"
Number one: it was seeking information. You've provided some in your post but I still see challenges.
I use VPHOT exclusively and have had to adapt to its features rather than adapt it to my needs. I think there may be others like me in the same boat. I have made many suggestions to Geir and while Geir has been very responsive, frankly, I think Geir is overworked. It's a big job and I see even more ahead. Is there any thought to open sourcing the effort?
I also see potential competing technologies that could overtake the effort. Dirk wasn't the first to mention an appliance that could be developed by experts, downloaded and used by folks with enough technical savvy to set up a telescope/ccd system in the first place.
I'm thinking BIG here. Imagine the ability of a user to put all his/her calibrated images into a folder, drag that folder to the appliance and get an AAVSO data report out in a few moments from his/her home computer. Given the name of the target in the FITS header, the appliance would "know" (or could easily download) a sequence prepared by experts and apply transformations. The transformation coeffiecients could be developed painlessly if multi-filter images of M67 or selected Landolt fields are taken by the user and put into that folder. Is VPHOT on a trajectory to approach this type of a system?
I gave a presentation at a regional convention of the Astronomical League last weekend. The pretty picture movement is gathering steam and those folks already have enough equipment and dedication to do lots of VSO. I mentioned the concept of the aforementioned appliance and got some positive audience reactions. I think the pretty picture community is the best recruiting ground we can find to promote the mission of AAVSO.
Hi Jim,
You have an interesting idea. Many of the software packages already do much of what you suggest, at least for time series - submit a set of images, get photometry back. The AAVSOnet pipeline does this too - while most users just see the processed images, in actuality, all stars are extracted from all images and the photometry resides on our servers. However, we do that for AAVSOnet-specific images; consistent software, consistent object and file naming; consistent keywords. Doing this for the general case is non-trivial, and is why I got out of the software support business decades ago.
However, for the average photometrist, I highly, highly recommend that you don't just put your images into a box and crank the handle. VPHOT was developed to give you the inspection tools to ensure that the photometry is correct. We have lots of users who submit data without looking at it, and the quality shows the lack of attention.
Full automation might be fine for surveys, where everything is done the same way, night in and night out. For everything else, making the steps simple is ok, but hands-off is not the right approach. I'm happy to support getting iraf to run easily on your computer, or improving VPHOT to be more efficient for the user, but I'd need a strong proposal from a team before I'd go the black box route.
Arne
I have used VPHOT since its Photometrica days at GRAS. First time I used it I gave up using my other programs (not that they were bad, just less convenient). There are some issues, calculating uncertainties is one of them. I have corresponded with Geir about this, but I think Geir needs to talk with Arne, Aaron and others at headquarters before making any changes.
I don’t produce a lot of data, a good night maybe 8-15 variable fields; so I find VPHOT perfectly adequate for my needs. It allows me to do some manual quality checks that I need given that I do not take professional-quality flats, sometime have trails and all the other bugs that amateurs on modest budgets face. VPHOT has many excellent features; I would encourage all AAVSO members to give it a try.
Re: Open sourcing: Speaking only as a member: you don’t do that when you wish to encourage membership by offering services; AAVSO has to stay in business and one way of doing that is to offer quality programs like VPHOT to its members and draw in new members with such programs. In fact, if VPHOT was the only service AAVSO offered I would become a member for that service alone. :)
"Re: Open
WEY wrote:
"Re: Open sourcing: Speaking only as a member: you don’t do that when you wish to encourage membership by offering services"
Hmmm. Is VPHOT (and other services) used to encourage membership? Doesn't seem like the right emphasis to me.
ROE wrote: "Hmmm. Is VPHOT (and other services) used to encourage membership? Doesn't seem like the right emphasis to me."
I have no idea if Headquarters expects the services it provides to encourage membership or if they have any "emphasis" at all. I'm just a member expressing my opinion. As a member I want to see AAVSO thrive and I am glad they offer us members some benefits not available to non-members, like the Carolyn Hurless Online Institute and VPHOT.
You want to be able to process greater volume of data in a quality manner. I think that is an excellent goal and I bet there are programs and pipelines out there that will allow you to reach those goals. I wish you clear skies and good hunting on your software quest.
VPHOT to encourage membership: I have strong doubts on this. I doubt that a "cool piece of software" is going to encourage anyone to take up this rather estoteric occupation. We can't compete with warfare video games ;-)
Handle cranking results: I suppose this is a problem, but I remember the steep learning curve I went thru. It was eight months after installing my system before I turned in my first observations. Long nights with pencil and paper, spreadsheets and manuals trying to understand the science and mathematics of the process. And it still took a patient Arne who generously mentored me in his office and revelaed the bugaboo of RBI to me. Will there be handle crankers, yeah I guess, but they will, it seems to me, not last. It's a bit like January at my gym - all the resolutionists show up for a few weeks but are 90% gone by mid February.
Why don't I use VPHOT more: 5 reasons (most are known)
1. I tend to do long time series thru several filters. VPHOT needs to handle multiple filters
2. Uploads of images need to be faster. I have seen the queue be so long that it takes more than an hour (sometimes two) for images to get loaded. What the answer is I don't know. Data compression of some sort I suppose.
3. Related to 1 and 2: It is not uncommon for me to gereate 300 or more images a night. I can imagine the ire of folks who have a few images getting backed up behind me in the queue. So I tend to process that my self with AIP4Win. The Z Cam stars and others that I follow, I would like to do with VPHOT, but it is easier just to do them with AIP4Win while it's open. I would rather use VPHOT so that 'how the results were obtained' is as consistant as possible. And the images are readily available to others if anomolies are seen.
4. One AAVSO/WEBOps report rather than many: Maybe this can be done and I just don't know it, but I'd like to collect all of my results from VPHOT in one report and submit one larger file than multiple small ones.
5. Loss of my definitions: It seems every time I come back to VPHOT for a look (say every 4 to6 months) my telescope and sequence definitions are gone. The telescope is easy, but I hate redoing the sequences.
Conclusion: Cloud computing is the future; for me i'd like VPHOT to be my future
..george
I have a few suggestions regarding three of the issues you brought up--I'm sure you may have already thought of them and they are not ideal--but just in case it may help in the short run)
I back up all my sequences by copying and pasting them into a backup file--which can be uploaded at another time. (go to Admin--> Sequences--> and select edit for the sequence you want to back up and copy and paste what is there into a file. I just always automatically back up any sequence I create as part of my routine.
I also like to submit one large report so once the reports are generated for one night, I copy and paste everything into one file. Just leave the header stuff and copy and paste the data under the header.
Perhaps I am using time series with many images and different filters in a different way than you--but I have no problem generating multiple filter time series. I just break up the time series by filter and run them separately. For example I bring up all the images in one filter using the "by filter" option in the main menu, run the time series, then go onto the next filter.
If you want to then see the results visually over time, if you combine the results in the time series in different filters into one file and load it into VSTAR...VSTAR will generate multiple light curves in multiple filters on the same graph.
Ken Mogul (MQE)
Perhaps I am
MQE wrote:
Perhaps I am using time series with many images and different filters in a different way than you--but I have no problem generating multiple filter time series. I just break up the time series by filter and run them separately. For example I bring up all the images in one filter using the "by filter" option in the main menu, run the time series, then go onto the next filter."
I think I understand this process but I also think you were replying to a comment of mine about doing transformations on the data? As best I can tell, to transform measurements to the standard colors in VPHOT is a manual, two image at a time process (and currently only B and V images). If you have a long time series of two color pairs you've got a lot of work ahead of you to transform them.
I'm looking at a script idea to take the output of VPHOT, separate out the pairs (or triplets, or ??), perform the transformations at home and re-write the values back into the AAVSO report.
I've
Thanks Ken,
I've backed up my sequences as you outlined.
As for points 2 and 3, I have been using AIP4Win for years and it's just a tad irksome to go thru extra steps to accomplish what I do automatically with AIP4Win. Please dont take these comments as quarelsome complaints - I really believe that VPHOT is the future! I mostly enjoy using it!
There is one other thing I'd like to see - The option for HJD in reports. I work closely with a professional astronomer and he requires HJD. Its a bore to do the conversion - It is something computers do so well! and more accurately!
..george
It occurs to me that there may be a solution to points 2 and 3 in my other post. I use ACP on my system at NMS. As each images comes down from the camera it is calibrated and stored in a folder on DropBox. If that folder were shared with a VPHOT DropBox account the. Images could be uploaded/processed thru out the night.
..George
I don't know ACP in detail but it is possible to have it send images coming off the camera with FTP (this is what they do at iTelescope).
So it is possible to have ACP send images directly to VPHOT.
Geir
"4. One
SGOR wrote:
"4. One AAVSO/WEBOps report rather than many:"
Geir has developed at least a partial solution which I have been beta testing. Until he releases it I can offer a little script that will combine all those little reports into one for uploading to WebObs. You put all your little reports into a particular folder (set the download default for your browser) then just drag that folder to the script. I use a folder name like 20120608 (yyyymmdd) for my calibrated files each night with a subfolder called Reports into which the little reports are deposited by VPHOT. The combined report is saved in the parent folder (in this case 20120608). It would be easy to change that in the script if you want some other folder for the combined report.
[quote=sgor]
1. I tend to do long time series thru several filters. VPHOT needs to handle multiple filters
[/quote]
As Ken mentioned this is easily done running the time series once for each filter.
[quote=sgor]
4. One AAVSO/WEBOps report rather than many: Maybe this can be done and I just don't know it, but I'd like to collect all of my results from VPHOT in one report and submit one larger file than multiple small ones.
[/quote]
Yes, this can be done. If you do single image analysis, click the 'Keep this' link for each analysis you do. Then go to the Analysis Log, select the analysis you want to report and click 'Create AAVSO Report' link. Time series results are automatically stored to the log.
[quote=sgor]
5. Loss of my definitions: It seems every time I come back to VPHOT for a look (say every 4 to6 months) my telescope and sequence definitions are gone. The telescope is easy, but I hate redoing the sequences.
[/quote]
Are you sure this has happened more than once? It did happen with the outage in March, but I don't see how that could happen otherwise. If others are experiencing this as well I would like to know about it.
Geir
I was
Thank you, Geir,
I was unaware of the feature you mention in point 4. I will try it out.
Certainly this last loss was due to the March outage. This is my third look at VPHOT. The first look was when it first popped up on AAVSO. So the first loss would have been sometime during the months following that. That is when I had my second look. I'm backing up sessions now - something I should have been doing anyway!
..george
"However, for the average photometrist, I highly, highly recommend that you don't just put your images into a box and crank the handle. VPHOT was developed to give you the inspection tools to ensure that the photometry is correct. "
I have to mention something here from my past experience that, I believe, has relevance to this comment. Years ago a study showed that teachers in Missouri were paid below the national average (may still be true). Members of the state legislature introduced a bill to raise them to average. A pithy politician said the goal was mediocrity. I've been alert for that ever since.
It seems to me VPHOT is part of a vision to raise the level of "average." My question is, will it go far enough? You often mention that quality is to be desired before quantity. Right. But once quality standards have been met, what's wrong with quantity? Already in my modest setup VPHOT is not scaling to more images per night - there are just too many manual steps, or things that have to be skipped (eg, transformations)(or sleep). I've mentioned elsewhere the bandwidth burden of uploading images which are 90+ % sky.
I have made many suggestions to Geir and he has tried to respond but, again, I think he is overworked. I also suspect he is not a "user," ie, does he take images and then try to process them to quality photometry? Nothing like starting to feel the onset of carpal tunnel syndrome to stir up the creative juices!
[quote=roe]It seems to me VPHOT is part of a vision to raise the level of "average." My question is, will it go far enough? You often mention that quality is to be desired before quantity. Right. But once quality standards have been met, what's wrong with quantity? Already in my modest setup VPHOT is not scaling to more images per night - there are just too many manual steps, or things that have to be skipped (eg, transformations)(or sleep).[/quote]
Ideas for improving automation of VPHOT:
- allow users to define 'auto-process' lists of targets and sequence stars, and create a new option to 'auto/batch process'. VPHOT would look at this auto-process list and compare it against all images selected for 'auto/batch process'.
...or...
- allow a script language to be used in VPHOT to automate many steps that are currently 'manual-only'. The user could create a script file that lists names of images to load (i.e. all images dated UT 2012-05031 and later, containing R_Aql as the object...process using sequence named "R_Aql_ver3"...and yes, transform the data.)
This would be beneficial to users that have experience with 'manual' VPHOT usage, and generate many images that cover many targets.
Yes, the usual caveats apply: bad images in = bad photometry out.
"Why would you say "Are efforts to improve VPHOT ultimately doomed?" Explain!"
Number one: it was seeking information. You've provided some in your post but I still see challenges.
I use VPHOT exclusively and have had to adapt to its features rather than adapt it to my needs. I think there may be others like me in the same boat. I have made many suggestions to Geir and while Geir has been very responsive, frankly, I think Geir is overworked. It's a big job and I see even more ahead. Is there any thought to open sourcing the effort?
I also see potential competing technologies that could overtake the effort. Dirk wasn't the first to mention an appliance that could be developed by experts, downloaded and used by folks with enough technical savvy to set up a telescope/ccd system in the first place.
I'm thinking BIG here. Imagine the ability of a user to put all his/her calibrated images into a folder, drag that folder to the appliance and get an AAVSO data report out in a few moments from his/her home computer. Given the name of the target in the FITS header, the appliance would "know" (or could easily download) a sequence prepared by experts and apply transformations. The transformation coeffiecients could be developed painlessly if multi-filter images of M67 or selected Landolt fields are taken by the user and put into that folder. Is VPHOT on a trajectory to approach this type of a system?
I gave a presentation at a regional convention of the Astronomical League last weekend. The pretty picture movement is gathering steam and those folks already have enough equipment and dedication to do lots of VSO. I mentioned the concept of the aforementioned appliance and got some positive audience reactions. I think the pretty picture community is the best recruiting ground we can find to promote the mission of AAVSO.