# CLEANest Model Values Question

Wed, 09/06/2023 - 22:08

I'm trying to figure out how CLEANest calculates it's Model values so I can produce  light  curves that result from  subsets of the dominant component frequencies found by CLEANest.

Based on a brief review of  the CLEANest code in the AAVSO Website Software Directory,  I assumed  the following equation for a one term CLEANest Model :

CLEANest Model  (t)  = Avg  Mag of Obs  +  a * Cos (2 *Pi * Freq * (t-t0) )  +  b * Sin (2 *Pi * Freq  * (t-t0)  )

Where      a = Amp * Cos(Phase);      b= Amp * Sin(Phase) ;       t = JD of an Observation;     t0 = Avg Time of All Observations (JD)

Values Calculated by  CLEANest:
Freq = 0.002203 (=Period = 453.967 d)       Amp = 0.041543     Phase = 0.1703 rads

Values Calculated by Me:
Avg  Mag of All Observations = 11.823
Avg Time of All Observations  = 2457595.87

Results:

(SORRY I AM UNABLE TO EMBED OR ATTACH THE PLOT COMPARING THE CLEANEST EXPORTED MODEL TO MY CALCULATED LIGHT CURVE.       THEY MATCH EXCEPT FOR A SMALL MAG DIFFERENCE AND A LARGE TIME/PHASE DIFFERENCE)

Differences between CLEANest Model and Calculated Model
Mag Error = -0.004 mag
t0  Error = -91 days  ==  Phase Error = -1.25949 rads =  2*Pi* (-91/453.96)

Results with Mag & t0/Phase Corrections:

(SORRY I AM UNABLE TO EMBED OR ATTACH THE PLOT COMPARING THE CLEANEST EXPORTED MODEL TO MY CALCULATED LIGHT CURVE AFTER  ADD CORRECTION FOR MAGNITUDE AND  TIME/PHASE. THE TWO CURVES MATCH)

My gut feel is that I incorrectly assumed that t0 is the average time of all observations. There are a lot of time index changes going  on in the CLEANest code that I don't follow.

Appreciate any suggestions you might have.

Thanks

Dave Hollinberger

Affiliation
Astronomical Society of South Australia (ASSAU)
CLEANest question

Hi Dave

I'm looking at the details of your question.

In the meantime, pages 71 to 81 of the user manual may help. In addition, the CLEANest unit tests may provide some insight into the behaviour of the code:

David

Affiliation
Astronomical Society of South Australia (ASSAU)
Model

Hi Dave

Perhaps it would be helpful to reason about a particular dataset, DCDFT search parameters, top hits selected for CLEANest, Fourier model creation parameters.

David

CLEANest Modeling Questions

Hi Dave,

Thank you very much for your suggestions back in early September. I'm sorry for my delay in replying.

I took your advice and inputted a set of test signals consisting of a single sinusoids with varying phase into CLEANest implemented by Peranso.                                        The CLEANest phase value outputted to the CLEANest Workbench Detailed Info Output Table is incorrect. The problem is probably caused by the "arctan(-PeakMatrix(i).sIni,PeakMatrix(i).cosi)" function call in Public Sub CLEANest_CalculateDetailedPeakInfo(...).

I think I also sorted out why Ao and To are close but not exactly equal to the average magnitude and average time of occurrence of all the datapoints in the light curve.

I wrote up my results in 5 page whitepaper. Please let me know if you want a copy.

Thanks

Dave Hollinberger

Affiliation
Astronomical Society of South Australia (ASSAU)
Paper

Hi Dave

I would be interested in your paper, yes. Please send when you get the chance via my AAVSO contact page.

Also, I note above that you found a bug in Peranso's CLEANest. Is that right? It would be good to report that to the developer if you haven't already.

Did you try the same operation in VStar? Perhaps your paper covers this.

Thanks.

David

Re: CLEANest Modeling Questions

Hi Dave,

Sorry for the delay in replying - again.

1. I'm sorry but I can't figure out how to attach a document via your AAVSO contact page. I can only send a blog message like this one. Appreciate suggestions.

2. Yes, the bug is in Peranso's CLEANest - that's what my tests were run on. And yes I have already sent my paper to the Peranso developer, Tonny Vanmunster.

3. I have not run similar tests on VStar's CLEANest . I will try to do that in the future to make sure. The reason I believe the same problem may exist in VStar CLEANest is that when I reviewed the CLEANest code dated 3/10/2005 that is posted on the AAVSO website Software Directory, I found code that appears to be causing the problems t I saw while testing Peranso's CLEANest. The reason I was reviewing the AAVSO CLEANest code was that Tonny was unable provide me with the Peranso CLEANest code because he has a policy of not releasing  active software code. He did provide me with a copy of TS12.bas.

Thanks

Dave Hollinberger

Hi David,

Here is the…

Hi David,

Here is the CLEANest paper in plain text. Hope it more readable than the earlier version

Thanks

Dave Hollinberger

------------

Independently Calculating Signal Models Using CLEANest Signal Parameters

“Analyzing Light Curves”  by Grant Foster (2010) (ALC Foster) gives two equivalent inverse Fourier series equations for calculating a model signal using the signal parameters produced by CLEANest:

ALC Foster Eqn 9.5

Model1(t) = Amp(0) + Sum(n=1:NoP)[ (c(n)*cos{2*Pi*F(n)*(t-t(0))} ) +  (s(n)*sin{2*Pi*F(n)*(t-t(0))} )   ]

ALC Foster Eqn 9.8

Model2(t) = Amp(0) + Sum(n=1:NoP)[Amp(n)*cos{(2*Pi*F(n)*(t-t(0)))+ Phase(n)}]

Where:

Amp(0) = ~Average of the magnitude of all datapoints in the time series

t(0)   = ~Average of the time of occurrence of all datapoints in the time series

t    = time

F(n) = Frequency of nth term

c(n) = Fourier series coefficient for nth cosine term

s(n) = Fourier series coefficient for nth sine term

Amp(n) = Amplitude of nth term = Sqrt[c(n)^2 + s(n)^2] ALC Foster Eqn 9.6

Phase(n) = Phase of nth term = arctan[-s(n)/c(n)]      ALC Foster Eqn 9.7

Problem #1 - CLEANest Phase(n) is Incorrect

Private Subroutine CLEANestSmooth uses ALC Foster Eqn 9.5 to compute the CLEANest model time series values.  It appears to do so correctly since the CLEANest model signal agrees well with the original signal, provided that the CLEANest Number of Periods is made large enough.

Table 1 is the Peranso CLEANest Workbench Detailed Info Output Table for Star ASAS-SN-V J002355.01-715729.7 with CLEANest Auto Search Number of Periods set to 3.

n  Freq            FrqErr     Period PerErr Theta  Amp    AmpErr Phase

(c/d)                           (days)                        (mag)               (cycles)

1   0.002203  0.000035  453.97  7.203  79.67  0.0483  0.0049  0.1995

2   0.025250  0.000060  39.60    0.093  56.92  0.0280  0.0049  0.0887

3   0.003074  0.000051  325.31  5.390  49.14  0.0331  0.0049  0.8957

Table 1 – Example Peranso CLEANest  Workbench Detailed Info Output Table

The CLEANest Output Table does not provide the Fourier series coefficients c(n) and s(n) used in ALC Foster Eqn 9.5.  Instead, it provides a value for Phase(n).  As result, you have to use ALC Foster Eqn 9.8 if you want to independently calculate the CLEANest model values.

I tested Peranso CLEANest using a set of synthetic signals that were single frequency sinusoids, all with the same amplitude and frequency. Only the phase of the test signals varied,from 0 to 1 cycle (=0 to 2*Pi rads). Table 2 shows the results.

TEST SIGNAL PHASE    ||     CLEANEST PHASE

A           0    0.000   Q1/Q4             0    0.000   Q1/Q4

B          1    0.003   Q1                    1    0.003   Q1

C        45    0.125   Q1                  45    0.125   Q1

D        89    0.247   Q1                  89    0.247   Q1

E        90    0.250   Q1/Q2            90    0.250   Q1/Q2

F        91    0.253   Q2                271    0.753   Q4

G     135    0.375   Q2                315    0.875   Q4

H      179    0.497   Q2               359    0.997   Q4

I       180    0.500   Q2/Q3          360    1.000   Q4/Q1

J      181    0.503   Q3                   1    0.003   Q1

K      225    0.625   Q3                 45    0.125   Q1

L       269    0.747   Q3                89    0.247   Q1

M      270    0.750   Q3/Q4          90    0.250   Q1/Q2

N      271    0.753   Q4              271    0.753   Q4

O      315    0.875   Q4              315    0.875   Q4

P      359    0.997   Q4              359    0.997   Q4

Table 2 – Results of Testing CLEANest Phase Calculation

When the phase of the test signal was in Quadrant 1 or 4, the CLEANest phase was correct. However, when the phase of the test signal was in Quadrant 2, the CLEANest phase was off by +0.5 cycles (= -0.5 cycles), putting it in Quadrant 4.  And when the phase of the test signal was in Quadrant 3, the CLEANest phase was off by -0.5 cycles ( = +0.5 cycles),putting it in Quadrant 1.

The CLEANest phases were calculated using ALC Foster Eqn 9.7 Phase(n) = arctan[-s(n)/c(n)].  The Visual Basic “arctan” function only returns a value of +/- 0.25 cycle = +/- Pi/2 rads.  I believe that this limitation of the Visual Basic “arctan” software function causes the problems with the CLEANest phase calculation.

Problem #2  CLEANest does not provide values for Amp(0) and  t(0)

The Peranso CLEANest Workbench Detailed Info Output Table does not include the values for Amp(0) (= CLEANest variable DCOEF(0)) and t(0)    (( = CLEANest variable DTZERO). Amp(0) is close (within several millimags) to the average of the magnitude of all datapoints in the time series.  I can’t tell from the calculations in CLEANest what is causing the difference.  t(0) is close (within a 1 day) to the average of the time of occurrence of all datapoints in the time series. The source of the difference is maybe due to  rounding of  DTZERO, though I can’t find where in the CLEANest code DTZERO is determined.

Possible Solutions:

1.    Do Nothing - CLEANest has been around for a long time. It could be that I am the first person who wanted to independently calculate my own signal model using subsets of the CLEANest frequency components.  I have found  a workaround that is less flexible, being limited to breaking the CLEANest frequency  components  into  frequency sub-bands rather than combining  them as I choose in order to investigate effect of possible harmonic components etc. This workaround appears to be less accurate but it is much easier to use.

2.    Revise/Correct the CLEANest Workbench Detailed Info Output Table – In Public Subroutine CLEANest_CalculateDetailedPeakInfo , delete Phase                        (= PeakMatrix(i).Phase) and add c(n) (=PeakMatrix(i).cosi), s(n) (=PeakMatrix(i)sIni), Amp(0) (=DCOEF(0)) and t(0) (= DTZERO) to the Output Table.  These changes would allow independently calculating the signal model using Foster ALC Eqn 9.5.

3.    Correctly calculate PeakMatrix(i).Phase - Revise the code for calculating PeakMatrix(i).Phase based on Visual Basic code that calculating the arctangent that preserves the quadrant information (http: //www.vb-helper.com/howto_atan2.html). Also, in Public Subroutine CLEANest_CalculateDetailedPeakInfo, add Amp(0)(=DCOEF(0)) and t(0) (= DTZERO) to the Output Table. These changes would allow independently calculating the signal model using Foster ALC Eqn 9.8.

Note: CLEANest uses the variable “dphase” in a number of different places with different values and meanings. Changes to the variable “dphase” should be avoided as much as possible.

Re: CLEANest Modeling Questions Paper

Hi David,

Here is the CLEANest paper in plain text. Hope it more readable than the earlier version

Thanks

Dave Hollinberger

------------

Independently Calculating Signal Models Using CLEANest Signal Parameters

“Analyzing Light Curves”  by Grant Foster (2010) (ALC Foster) gives two equivalent inverse Fourier series equations for calculating a model signal using the signal parameters produced by CLEANest:

ALC Foster Eqn 9.5

Model1(t) = Amp(0) + Sum(n=1:NoP)[ (c(n)*cos{2*Pi*F(n)*(t-t(0))} ) +  (s(n)*sin{2*Pi*F(n)*(t-t(0))} )   ]

ALC Foster Eqn 9.8

Model2(t) = Amp(0) + Sum(n=1:NoP)[Amp(n)*cos{(2*Pi*F(n)*(t-t(0)))+ Phase(n)}]

Where:

Amp(0) = ~Average of the magnitude of all datapoints in the time series

t(0)   = ~Average of the time of occurrence of all datapoints in the time series

t    = time

F(n) = Frequency of nth term

c(n) = Fourier series coefficient for nth cosine term

s(n) = Fourier series coefficient for nth sine term

Amp(n) = Amplitude of nth term = Sqrt[c(n)^2 + s(n)^2] ALC Foster Eqn 9.6

Phase(n) = Phase of nth term = arctan[-s(n)/c(n)]      ALC Foster Eqn 9.7

Problem #1 - CLEANest Phase(n) is Incorrect

Private Subroutine CLEANestSmooth uses ALC Foster Eqn 9.5 to compute the CLEANest model time series values.  It appears to do so correctly since the CLEANest model signal agrees well with the original signal, provided that the CLEANest Number of Periods is made large enough.

Table 1 is the Peranso CLEANest Workbench Detailed Info Output Table for Star ASAS-SN-V J002355.01-715729.7 with CLEANest Auto Search Number of Periods set to 3.

n  Freq            FrqErr     Period PerErr Theta  Amp    AmpErr Phase

(c/d)                           (days)                        (mag)               (cycles)

1   0.002203  0.000035  453.97  7.203  79.67  0.0483  0.0049  0.1995

2   0.025250  0.000060  39.60    0.093  56.92  0.0280  0.0049  0.0887

3   0.003074  0.000051  325.31  5.390  49.14  0.0331  0.0049  0.8957

Table 1 – Example Peranso CLEANest  Workbench Detailed Info Output Table

The CLEANest Output Table does not provide the Fourier series coefficients c(n) and s(n) used in ALC Foster Eqn 9.5.  Instead, it provides a value for Phase(n).  As result, you have to use ALC Foster Eqn 9.8 if you want to independently calculate the CLEANest model values.

I tested Peranso CLEANest using a set of synthetic signals that were single frequency sinusoids, all with the same amplitude and frequency. Only the phase of the test signals varied,from 0 to 1 cycle (=0 to 2*Pi rads). Table 2 shows the results.

TEST SIGNAL PHASE    ||     CLEANEST PHASE

A           0    0.000   Q1/Q4             0    0.000   Q1/Q4

B          1    0.003   Q1                    1    0.003   Q1

C        45    0.125   Q1                  45    0.125   Q1

D        89    0.247   Q1                  89    0.247   Q1

E        90    0.250   Q1/Q2            90    0.250   Q1/Q2

F        91    0.253   Q2                271    0.753   Q4

G     135    0.375   Q2                315    0.875   Q4

H      179    0.497   Q2               359    0.997   Q4

I       180    0.500   Q2/Q3          360    1.000   Q4/Q1

J      181    0.503   Q3                   1    0.003   Q1

K      225    0.625   Q3                 45    0.125   Q1

L       269    0.747   Q3                89    0.247   Q1

M      270    0.750   Q3/Q4          90    0.250   Q1/Q2

N      271    0.753   Q4              271    0.753   Q4

O      315    0.875   Q4              315    0.875   Q4

P      359    0.997   Q4              359    0.997   Q4

Table 2 – Results of Testing CLEANest Phase Calculation

When the phase of the test signal was in Quadrant 1 or 4, the CLEANest phase was correct. However, when the phase of the test signal was in Quadrant 2, the CLEANest phase was off by +0.5 cycles (= -0.5 cycles), putting it in Quadrant 4.  And when the phase of the test signal was in Quadrant 3, the CLEANest phase was off by -0.5 cycles ( = +0.5 cycles),putting it in Quadrant 1.

The CLEANest phases were calculated using ALC Foster Eqn 9.7 Phase(n) = arctan[-s(n)/c(n)].  The Visual Basic “arctan” function only returns a value of +/- 0.25 cycle = +/- Pi/2 rads.  I believe that this limitation of the Visual Basic “arctan” software function causes the problems with the CLEANest phase calculation.

Problem #2  CLEANest does not provide values for Amp(0) and  t(0)

The Peranso CLEANest Workbench Detailed Info Output Table does not include the values for Amp(0) (= CLEANest variable DCOEF(0)) and t(0)    (( = CLEANest variable DTZERO). Amp(0) is close (within several millimags) to the average of the magnitude of all datapoints in the time series.  I can’t tell from the calculations in CLEANest what is causing the difference.  t(0) is close (within a 1 day) to the average of the time of occurrence of all datapoints in the time series. The source of the difference is maybe due to  rounding of  DTZERO, though I can’t find where in the CLEANest code DTZERO is determined.

Possible Solutions:

1.    Do Nothing - CLEANest has been around for a long time. It could be that I am the first person who wanted to independently calculate my own signal model using subsets of the CLEANest frequency components.  I have found  a workaround that is less flexible, being limited to breaking the CLEANest frequency  components  into  frequency sub-bands rather than combining  them as I choose in order to investigate effect of possible harmonic components etc. This workaround appears to be less accurate but it is much easier to use.

2.    Revise/Correct the CLEANest Workbench Detailed Info Output Table – In Public Subroutine CLEANest_CalculateDetailedPeakInfo , delete Phase                        (= PeakMatrix(i).Phase) and add c(n) (=PeakMatrix(i).cosi), s(n) (=PeakMatrix(i)sIni), Amp(0) (=DCOEF(0)) and t(0) (= DTZERO) to the Output Table.  These changes would allow independently calculating the signal model using Foster ALC Eqn 9.5.

3.    Correctly calculate PeakMatrix(i).Phase - Revise the code for calculating PeakMatrix(i).Phase based on Visual Basic code that calculating the arctangent that preserves the quadrant information (http: //www.vb-helper.com/howto_atan2.html). Also, in Public Subroutine CLEANest_CalculateDetailedPeakInfo, add Amp(0)(=DCOEF(0)) and t(0) (= DTZERO) to the Output Table. These changes would allow independently calculating the signal model using Foster ALC Eqn 9.8.

Note: CLEANest uses the variable “dphase” in a number of different places with different values and meanings. Changes to the variable “dphase” should be avoided as much as possible.

Re: CLEANest Modeling Questions Paper

Hi David,

I looked at  the VStar User's Manual briefly.

1. Model

On page 67 & 68 of the VStar User's Manual there is an example model equation. It is in the Foster ALC Equation 9.5 form, which does not use phase values. In place of phase values, cosine coefficients c(n) = -0.250292723 etc and sine coefficients s(n)= 0.154254105 etc values are provided. The example model also includes the ~average magnitude = Amp(0) = c(0) = 3.910242905 value. And it separately provides the ~average time = t(0) = zeroPoint = 2455973 JD value.

So it looks to me that VStar already meets the Option 2 solution that I suggested in my paper. Great.

2. Period Analysis (DC DFT)  Table

The example tables on VStar User's Manual Page 63, 73 &  76 include Frequency, Period, Power and Semi-Amplitude values. But  unlike Peranso Period Analysis Tables, the VStar tables do not include Phase values. So it appears that VStar dodged the bad phase bullet here also. Also great.

3. Relative Phases

On page 67 of the VStar User's Manual there is an example "Relative Amplitudes & Phases by fundamental frequency" table given.   I cant really follow the format of the table - If I had to guess, the last 3 values (2.9029, 0.0276 and 6.1310) might be the relative phases for Freq 2, 3 & 4 given in rads.  I have read the  Foster ALC section on Relative Amps & Phase several times but am still mystified by the concept of Relative Phase. So I really cant tell if VStar has a bad phase issue here or not. There is that potential.

Hope this helps

Dave Hollinberger