SpecDB

Hi  Keith,  maybe this:  

http://specdb.readthedocs.io/en/latest/

Clear Skies !

Roger

Hi All,

Thank you for your interest in SpecDB. I am happy to say that SpecDB exists and is nearing completion. We are not ready to launch just yet, as there are a few checks to do.

In the meantime, I am happy to provide a poster describing the database, and how you will be able to contribute and interact.

Roger - I believe that link is for an SDSS-related project, and not this one through AAVSO. The naming is rather inconvienent!

Please get in touch or comment below with any questions.

Cheers,

John Weaver

Hi John,

Is the data format compatible with the existing databases currently used by amateurs internationally for archiving spectrooscopic data which already use a common format eg

BAA spectroscopic database     https://www.britastro.org/specdb/

BeSS database    http://basebe.obspm.fr/basebe/

ARAS database  http://www.astrosurf.com/aras/Aras_DataBase/DataBase.htm

Thanks

Robin

Robin

Great to hear about the progress on this project. Though I do share Robin's concern on compatibility between database and also existing amateur spectroscopy software. Hopefully these concerns are unfounded.

I notice the poster has a screen shot of some FITS header fields which include AAV_INST, AAV_SITE, AAV_ITRP and Q_CAL. None of which are in the BeSS standard and are not included in the output generated by the existing amateur spectroscopy software. Though it appears the first 3 have equivalent BeSS header fields of BSS_INST, BSS_SITE, BSS_ITRP. Perhaps these are not a requirement and get populated during the upload process?

Best wishes,

Andy Wilson (BAA Spectroscopy Database Manager)

I agree with Robin and Andy's comments.

Any new database should allow the interchange and comparison of data with the very successful existing databases.

Is there a working group establishing standards for amateur spectroscopy files?  High-energy astronomers have standards hashed out by HEASARC to harmonize data files across different satellite missions.  This keeps everyone's format "on the same page," so to speak.  I think such a working group may be needed here.

From the links provided by Robin, I have so far only been able to find one file specification, that for BeSS (perhaps someone can help?).  Looking at the BeSS spec, I'm puzzled about some things.  The keyword BSS_INST seems only to be used if TELESCOP, INSTRUME, and DETNAM are not supplied.  The latter are well-established FITS keywords.  Why would they not be used?  Is BSS_INST really just a shortcut?  If so, I'm not sure that is wise. 

Similarly, are there really no well-accepted FITS keywords for latitude, longitude, and elevation for ground based observatories?  Why were BSS_LAT, BSS_LONG, BSS_ELEV, and their shortcut, BSS_SITE, defined?  Prefixing the former keywords with BSS_ implies that they are specific to the BeSS project, which they aren't.

I'm not trying to start a squabble, and the AAVSO keyword suite may have the very same problems, but I'm not sure the existing formats have been carefully thought out.  As the complexity and quantity of amateur data products grow, it is important that we have well-designed file specifications.

Tom

Hi Tom,

I am all for improved standards but AAVSO are late in the game here and the practical situation is that BeSS has effectively been the de facto standard among amateurs working on pro-am projects for many years so there are already thousands of archived amateur spectra using that format. I had hoped that as a minimum, spectra in the BeSS format would be accepted in the AAVSO database without modification as they are in the BAA database but the lack of feedback to my original question is a little worrying.

Cheers

Robin

Hi Tom,

The BeSS standard was created over 10 years ago by a collaboration between a group of professional and amateur astronomers. So if not a working group by name, then I think it was equivalent. The BeSS database is maintained at the Observatoire de Paris-Meudon and contains over 150,000 Be star spectra acquired by both amateur and professional astronomers.

When we decided to create the BAA Spectroscopy Database we canvassed the opinions of members already performing spectroscopy. It was a straight forward decision to follow the BeSS standard. Though it is not perfect, it has done a good job and has been used successfully by Pro-Am collaborations and databases for over 10 years.

We have links to the BeSS standard at pertinent places on the BAA website, like the below page:

https://www.britastro.org/specdb/submission_requirements.htm

While I say we follow the BeSS standard, there are a few minor tweaks to the standard which we mention on this page. The main one is we require the OBJNAME FITS header is populated. In the standard this can be omitted if the pointing location is defined by RA, DEC, EQUINOX/RADECSYS. This would have added complexity to the BAA database, so we decided to tweak this requirement for our purposes. We also allow other fields to be populated like the ones you mention.

Of course standards should not be immovable, and should change with the times. If the AAVSO are going to have a different standard, then I would hope there would be discussions with the existing global amateur spectroscopy community. Then at least we can work out together how to address differences in standards going forward. Though I still hold out hope that the BeSS format will be acceptable to the AAVSO.

Best wishes,

Andy

I'm not trying to defend anything AAVSO did or didn't do, and I have no association with the SpecDB project.  I'm looking at a data products specification that appears, to me, to have significant drawbacks, and I'm saying the community needs to discuss it.

We're all trying to gather photometry and spectra for the benefit of researchers, those present-day and those decades yet-to-come.  Good metadata are the researcher's friend, and we should be generous providing them.  They become critical as living memory about the data fades.  A specific example: the researcher needs to know what software was used to capture the data and all the processing parameters involved.  I don't understand why the BeSS specification doesn't include this, and that kind of omission made me question the care given to the design.  [Yes, I see that there are keywords for COSM, NORM, TELL, RQVH, and VHEL, but are those really the only degrees of freedom in the reduction?]

Metadata are also needed for making large collections of data searchable.  Hopefully, the spectra everyone is collecting will someday end up in a major database.  What is the BeSS convention for choosing object names?  HR number?  HD?  BD?  What if the object is non-stellar?  How will someone searching the database find what he/she is looking for?

The long-term value of everyone's work will be diminished if we don't provide consistent, informative metadata.   I'm not a scientist, but I've worked on a major NASA mission that produced a large FITS archive.  I'd  be happy to offer what insights I can to any discussion.

Tom

"What is the BeSS convention for choosing object names?  HR number?  HD?  BD?  What if the object is non-stellar?  How will someone searching the database find what he/she is looking for?"

I would expect that to be handled by the database provided a recognised name is used. For example the BeSS database recognises any of the alternative names which are in SIMBAD and will call up all spectra for that object. (try gamma Cas for example which has 63 alternative names) The same is also true for the professional ELODIE archive for example.

http://atlas.obs-hp.fr/elodie/

Cheers

Robin

Hi All,

Thank you for the great discussion! SpecDB has been designed from the very beginning with all of these concerns in mind.

We began with a survey of common reduction software packages (rspec, vspec, isis, to name a few) and designed the header checking algorithms to search for all of the possible (or likely) keywords associated with a given value. For example, the star name can be assigned to the keys "OBJ", "OBJECT", or "TARGET". The system is also designed for very easy access to add or modify keywords if the need arises.

As well, we are fully compatible with BeSS, and several other databases. There will be a full techincal manual released with SpecDB which will detail all of the compatibility and required keywords that must be in your headers. As well, we can even accept aliases which you may have used for other databases, so that you may submit a name under "OBSERVER" which is not your obscode (ie what you would have submtted to BeSS). We just need to link the alias with your obscode first.

Star names are also a concern, of course. We opted to use the VSX system to allow users to submit a variety of star names (ie Vega vs. Alpha Lyrae) in the observation headers.

Finally, all of the published observations will have their headers standardized to a common SpecDB set of keywords, very similar to BESS. All changes will be noted with comment cards in the headers. This way data processing software can be written once, and then applied to a number of observations from different users.

Again, a technical manual with all of the details will be released and linked to the "help" section of SpecDB. If anyone has any further questions, please let me know. I expect to be travelling over the next few days, so please be patient.

Thank you again for the great discussion!

Cheers,

John

Hi John,

Thank you for clarrifying the situation. I am pleased to hear that SpecDB will accept BeSS format spectra. It sounds like it will be very flexible, which is great!

I look forward to hearing more in due course.

Best wishes,

Andy

6 months on.....

Any updates??

Hi All,

I can say that we are making good progress! The delay came by way of finding suitable non-variable stars which we can use for quality-checking. We have identified a few dozen bright stars across both hemispheres with extremely high resolution spectra which we will use to compare with the first "test" submission for a given user.

Briefly, we require that each user submit a spectrum of a "standard" non-variable star which we can check for any issues with technique or equipment. After passing this stage, you may then submit variable star spectra.

Once we have ensured that 1) the webpage is free of bugs and 2) that our documentation is sufficient and complete, we will move ahead with finally open SpecDB to the community.

We are excited to roll this out - and we hope you are too!

Cheers,

John

Hello!
I would be interested to know more about this SpecDB project.
I've been doing photometry for a few years now, with filters B, V, R, Ic.
In spectro side, I made some approaches with a SA-200 (Star analyzer) and treatment on Rspec.
But next week I will get an Alpy-600 with the guiding module and guiding camera and start discovery and learning.

Thank you for telling me more!
JBD

Actually specdb has been running for 18 months and already contains 2500 amateur spectra  ;-)

https://www.britastro.org/specdb

Maybe a different name should be chosen to avoid confusion ?

It's now three months since the last update.....

Any news on the project??

Hi Ken,

In the meantime why not add your spectra to the BAA database where there are now ~2700 spectra of ~600 different objects?

Hopefully if and when the AAVSO database gets up and running it will mirror the BAA spectroscopy database content as is done with the BAA variable star database.

Robin

Robin

Robin,

Yeah, between the BAA and BeSS databases we're pretty well covered, but I'm interested to see what the AAVSO will come up with......

Hi Ken,

I see you have submitted a minor a change request to the BAA database but have not submitted any spectra there and you have only one spectrum in BeSS and none in the ARAS database as far I can see.   Where are you storing your spectra? What is putting you off using the current databases and what would you like to see done  differently with the AAVSO?

Cheers

Robin

Robin,

For various reasons I don't upload many of my spectra. Sometimes to the SASER group.

I'm just looking at the options available to the 1000+ group members I have.

I think the BAA, BeSS and ARAS databases are good.

Ken