Affiliation
American Association of Variable Star Observers (AAVSO)
Sat, 08/04/2018 - 17:57
Let me ask a couple of questions on this topic. If these now, or in the future become something best taken offline rather than answered in this forum, just let me know.
First is the plugin additional resources source code. The link in this section at the bottom on this page does not seem to work. If I look here instead is that the best place to go?
GAIA import is the plugin I am considering. Photometry from GAIA comes in three passbands. The project gives some guidelines on transforming the GAIA passbands into other ones which are supported by VStar. Is there a way to import the GAIA photometry without transforming it into VStar and not have them all show up as unspecified?
thanks
Cliff
David,
I realize we live a half a world apart. Not wanting to wait a day, I went ahead and progressed to compiling and testing the "hello world" type plugins. I had mixed results and would welcome your suggestions. Please see the attached description.
regards,
Cliff
Hi Cliff
Thanks for your interest in writing plugins and for your detailed log and analysis.
The daily timezone based cycle can make life interesting but we can work with it well enough.
Please try this:
jar -cvf my.vstar.plugin.ExampleObSource.jar my
or
jar -cvf my.vstar.plugin.ExampleObSource.jar my/vstar/plugin/ExampleObSource*.class
Both will have the same effect.
The jar file created with the command in https://www.aavso.org/sites/default/files/vstar-plugins/vstar-plugins-d… is okay when there is only one class file to be added to the jar file, which is the case for the ObservationCounter plugin but not for the observation source plugins you are working with.
Let me know how it goes.
It's been awhile since I updated the plugin dev doc. I should fix that.
David
ps - we'll sort out that broken link to the plugin code from the plugin library page too
Yes, that was it. Thx Cliff
Cliff
Happy to talk about the best approach to creating the GAIA plugin.
We can have the plugin assign bands of whatever name we desire for the GAIA observations.
David
David,
I have posted to the Photometry forum to try to get some feedback on the passbands. So far there has been no interest expressed so I might just make the decision to suit myself.
The GAIA web service response is actually a VOTable in XML format. Now I have searched around a bit and do not see where VStar currently parses this format - is that right? However, I note your post where you got so far as to write a sample. However, I can not locate the file you refer to in that post. I'm sure the last thing we want is two different ways to parse VOTables in VStar. What do you recommend?
thanks,
Cliff
Hi Cliff
I saw your post to the photometry forum. Thanks.
The repository URL structure changed since that post apparently and that source file was removed from the latest codebase, however it's still available in the record of commits.
This was for the purpose of obtaining star information by name or AUID from VSX and at least shows how VOTable text could be handled.
Let me know if you have questions.
David
Thanks David,
Unfortunately, the votable from GAIA is binary
<DATA>
<BINARY2>
<STREAM encoding='base64'>
It just seems to me that using a 3rd party package is the way to go with this. I think this is particularly true for a "standard" format. Since there is no other uses of VOTable in Vstar, it seems I should just include this in the plugin jar. Make sense? Any recommendations on the best java packages to consider?
Cliff
Hi Cliff
It does make sense yes.
This Java library looks promising: Starlink Tables Infrastructure Library
Have to take a closer look but there's jar files and source code. We can go either way for a plugin.
There's also this documentation from IVOA that you have probably seen:
http://www.ivoa.net/documents/VOTable/20130920/REC-VOTable-1.3-20130920.html#ToC39
Happy to talk more about all this.
David
Hi Cliff,
Thank you for letting us know about the broken link on the VStar Plugin Library page. It is now fixed.
Best regards,
Sara
Hi Cliff
I think you may be interested in the tool I've made available here:
https://www.aavso.org/ant-based-tool-create-vstar-plug-ins
How is it going? Any progress? Need help?
David
David,
Thanks for checking in. I'll look at the ant tool.
I have made some progress. I created a standalone python script to pull the GAIA photometry and work with it. Python because it is hard to beat the astropy package functionality. The VOTable access is pretty trivial with it.
Anyway, by goal was to implement the GAIA to VRI passband transformation and see how accurate it is. I compared it to the Landolt North standard stars and, for V, to APASS. Across those 300+ stars the transformed GAIA values are pretty good. For V, the GAIA values overall beat APASS. Still there are enough anomalies that I think it is better to import the native GAIA passbands into VStar.
As with all prototyping, I learned something important. As I took a closer look at the GAIA documentation, I realized there is a CSV option that you can add to the URL. So I will soon move on to building the CSV based plugin. I suspect that the existing plugins will give me what I need to know.
So the help I need is on how to handle the GAIA passbands identifiers. They are green:G, blue:Gbp and red:Grp. So do we just decide ourselves and make this VStar specific, pick something that does not conflict with the filter options here? Or should we take it to a wider group for discussion?
thanks very much,
Cliff
Hi Cliff
Good news re: the CSV option. I know what you mean about Python + astropy.
There's red, green, and blue bands we could use. Have a look here starting around line 95:
https://sourceforge.net/p/vstar/code/HEAD/tree/trunk/src/org/aavso/tools/vstar/data/SeriesType.java
Alternatively build and run the example GeneralTool plugin via the ant tool to see them listed in a dialog.
However, I think those bands are intended to be used for visual obs of some sort but they may also be legacy.
So, to your question: I think it makes sense to take it to a broader audience, but these can easily be changed during coding including creating new ones if necessary, as we do for ASAS and other ob src plugins.
I might ask Sara and Bert to look at this topic in case they haven't seen it.
David
I will be the first to admit that I don't have enough experience with AAVSO to make a good judgement on what IDs to use. However, I think an important consideration is that no ground based observer is likely to be able to replicate the passbands of GAIA. The blue and red are actually artifically generated by integrating the spectrum as I understand it. And then, of course there is the space based aspect.
best regards,
Cliff
David,
I have the first cut working. I put aside the decision on filters and just use blue,green,red for now.
When I set it up for InputType.FILE_OR_URL it works as expected. I can import directly from a URL like
http://geadata.esac.esa.int/data-server/data?RETRIEVAL_TYPE=epoch_photo…
or from a file saved from using that URL in a browser.
Now I thought it would be useful to prepopulate the URL with everything through and including "ID=" so the user only needs to add (paste) the GAIA source_id. I thought perhaps that was what the
ObservationSourcePluginBase.getURLs was for. However, that method does not seem to get invoked unless the InputType.URL is used. And then it appears it expects that to be a complete URL which of course fails if I just put in the first part.
Now the GAIA archive (https://gea.esac.esa.int/archive/) queries will supply the URL and anyone likely to use this plugin will be familiar with it. However, they would need to modify the URL they get with the inclusion of "&FORMT=CSV" as I have dodged the VOTable work in this plugin.
Do you have any suggestions?
thanks,
Cliff
Hi Cliff
It's great that you have a first cut working!
The way to do what you are thinking about is to request input via a dialog to get the GAIA source ID in getURLs() and return a list containing a single URL, somewhat like this:
import javax.swing.JOptionPane;
...
public List<URL> getURLs() throws Exception {
List<URL> urls = new ArrayList<URL>();
String id = JOptionPane.showInputDialog("Enter GAIA Source ID");
if (id != null && id().trim().length() != 0) {
urls.add(new URL(BASE_URL + id.trim());
}
return urls;
}
Make sense?
David
yes, it does make sense.
Cliff
David,
See what you think. A few interesting source_ids:
4116592768715278848,6857141688677395840, 3530587602644108800
Cliff
Hi Cliff
This is looking good! Works fine for me.
Minor typo in dialog: the word transformation has some transposed letters.
The R, G, B band types probably should be SeriesType.Blue,Red,Green rather than TG, TR, TB.
David
Thanks for pointing those out. I'll consider it done unless I turn up something as I work with it.
Thanks for all the help. I learned a lot about VStar.
Cliff
Hi Cliff
No worries.
When you think it's ready, we can include it in the standard set of plugins that are available with each release. Making the source available in GitHub or within the VStar Subversion repo will make that easier (especially the latter).
I usually write a short doc to show usage etc.
Don't be afraid to ask others via the forum to help (test, docs). There are people who have expressed interest in helping in recent posts.
David
David,
So my suggestion is to conduct my own tests for a short time and then try to enlist others. Of course I will provide documentation.
It seems this could happen in parallel with getting the documentation and source code to some public place. I don't currently have any reason to use either GitHub or Subversion and so have no preference. Again I would ask for your suggestion and a bit of help.
Do you want the package name changed to org.aavso... or is the gaia.plugin package path fine as is?
Cliff
That all sounds good Cliff.
It's not critical that the package name be changed but it may simplify things with respect to the bulk build process. Likewise, getting it into Sourceforge with Subversion. Happy to help.
I use git for pretty much everything else these days. I started using Subversion for VStar several years ago and have had little reason to change. That may change in future.
David
FWIW Gais DR1 is accessible through the astroquery.vizier module. The module provides pretty flexible output via astropy.table.
I haven't tried to use it. Just found it when I was trying to find a way to access Pan-Starrs..
Jim
Note about Gaia magnitudes - G is like "Gaia" - it is broadband/white light magnitude from astrometric CCD-s. And Gaia's blue and red magnitudes should be just sums of respective, very wide in wavelength scale, blue and red spectra (see https://www.cosmos.esa.int/documents/29201/302420/IN_photometric_instrument-1.pdf/5c78c8d8-35a6-4258-be6e-f42a3d2b624c). So one should be moderately careful when comparing those magnitudes with other systems, even when using conversion equations. ;-)
Thanks Tonis,
I attempted to get some discussion on this topic over in the photometry forum, but got none.
Over there I linked to a graphic that supports your more detailed link. I was encouraged to look at the transformations by Appendix A of this paper. Once I had the transformation working, I compared a few hundred Landolt standards with encouraging results. I ended up implementing the transformation as optional.
So far my impression is that the photometry available on the 500,000 GAIA "variables" is more interesting to me than most others. That's fine.
best regards,
Cliff
During the testing of my plugin I find the clipboard is not operating for my dialog box when I start VStar from webstart. It does work if I start VStar locally. I see that all the other plugin dialog boxes do work. What's the trick?
thanks
Cliff
Hi Cliff
I'm wondering whether this is a security policy setting.
I have plans to move away from WebStart at some point as well.
Having said that, the fact that you don't see the same problem with other plugins is odd.
Once the code is in SF I'll build and test it too to see if I can reproduce what you see.
David
Hi David,
Is it possible to extend SeriesType adding user-defined series types from an ObservationSourcePlugin? Or it is a fixed list?
Best regards,
Max
Hi Max
Yep, SeriesType can be extended with new series. Indeed, most observation source plugins do exactly that.
Further, they appear in Preferences so that colour and point size can be changed.
David
Hi David,
It's great! I will dig in the sources.
Best regards,
Max