Data processing of head echoes

Data processing of head echoes

My goal:

    Reliably investigate

  • Individual events
  • Anomalous events
  • Rare sub-populations
  • Large scale statistics
  • Derived data

    • Apply on multiple radar systems

Data processing of head echoes

Most of this is described in:

Kastinen, D. & Kero, J. MNRAS (2022)

(expect some more revelations I have had since)

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Data and experiment description
  • Data and experiment description
  • Data and experiment description

Typical parameters for me

  • Pulsed experiment
  • Binary phase-shift keying
  • 46.5, 53.5, 224, 500, 930 MHz
  • Single channel / multiple channels
  • Sample rates 1 - 20 MHz
  • Transmit powers 500 kW - 2 MW
  • ~1-3 ms inter pulse period
  • ~1-200 $\mu$s pulse length
  • (plus some oddities and special cases)
  • Data and experiment description

The combination of parameters determine the look of the output data

    For example

  • Sky vs system temperature varies with frequency
  • Sampling frequency limits range accuracy
  • Scattering physics change with wavelength vs target size
  • Interferometry depends on channel layout
  • Radar code changes noise sensitivity
  • ...

    • So there are all of these trade-offs

  • Data and experiment description

Binary phase-shift keying? Radar "code"

  • Data and experiment description

The code determines the filter response

  • Data and experiment description

So for example: different code - different noise sensitivity on $r$

  • Data and experiment description

So for example: different code - different noise sensitivity on $r$

but bounded by your sampling rate

  • Data and experiment description

The combination of parameters determine the look of the output data

  • Data and experiment description

The combination of parameters determine the look of the output data

But here pulse-to-pulse phase measurement is the reverse
(which can be used for velocity estimates)

  • Data and experiment description

So now we have an experiment: lets see meteors!

  • Data and experiment description
  • Data and experiment description

We can discuss event search later - for now: we have events!

  • Data and experiment description

Multiple channels?

  • Data and experiment description

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Sensor model

Sensor "output" based on phenomenological parameters

$$ \Psi(...) \in \mathbb{C}^N $$

Phenomena:
Transmission - scattering - return wave

Variables: sample $n$ and channel $j$

Parameters: range $r$, range rate $\dot{r}$, direction $\mathbf{k}$

(can also add in Radar Cross Section $\sigma$)

  • Sensor model

Important independence statements

$$ \Psi_j(n | \mathbf{k}, r, \dot{r}, \sigma) $$
  • $n$ vs $r$, $\dot{r}$
  • $j$ vs $\mathbf{k}$
  • $\sigma$
  • can be treated independently
  • Sensor model

For example

$$ \Psi(j | \mathbf{k}) = A(\sigma) \sum_{l=1}^{N_j} \gamma_{jl}(\mathbf{k}) e^{-i\langle \mathbf{k}, \mathbf{r}_{jl} \rangle_{\mathbb{R}^3}} $$ $$ \Psi(n | r, \dot{r}, \sigma) = B(\sigma) e^{i(\theta_0 + (r + \dot{r} t_n) c^{-1}\omega)} $$

(two way range and range-rate)

And you can transform the data into these forms

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Calibration
$$ T_\mathrm{noise} =\\= T_\mathrm{sky} + T_\mathrm{sys} =\\= \frac{P_\mathrm{noise}}{g} $$

$T_\mathrm{sky}$ vs $T_\mathrm{sys}$
change with frequency

Other ideas?

  • Calibration

Phase calibration options

  • Strong meteors
  • Strong radio sources
  • Calibration signal
  • Known resident space objects
  • Other transmitters

    • Other ideas?

    Won't go into detail here
    still working on implementing/combining the above

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Matched filtering

Model matching 101

  1. Choose a similarity metric: $\mu(\mathbf{x}, \mathbf{y}) = \langle \cdot, \cdot \rangle_{\mathbb{C}^N}$
  2. Define a parametrized model: $\Psi(\mathbf{a})$
  3. Acquire measured signal $\mathbf{x}$
  4. Match to model parameters using metric: $F(\mathbf{a}) = \mu( \mathbf{x}, \Psi(\mathbf{a}))$
  5. Metric says:
    $F$ high $\mapsto$ model $\approx$ signal
    $F$ low $\mapsto$ model $\not\approx$ signal
  6. Get parameters: $\tilde{\mathbf{a}} = \argmax F(\mathbf{a})$

Check the matching level!!!

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Decoding and interferometry

Decoding: find $r, \dot{r}$ with model matching

Interferometry: find $\mathbf{k}$ with model matching

  • Decoding and interferometry

For example: cross-correlation $\mu(\mathbf{x}, \Psi) = |\langle \mathbf{x}, \Psi(r, \dot{r}) \rangle_{\mathbb{C}^N}|$

For example: beam-forming $\mu(\mathbf{x}, \Psi) = |\langle \mathbf{x}, \Psi(\mathbf{k}) \rangle_{\mathbb{C}^N}|$

For example: MUSIC $\mu(\mathbf{x}, \Psi) = \frac{\boldsymbol{\Psi}(\mathbf{k})^\dagger\boldsymbol{\Psi}(\mathbf{k})}{\boldsymbol{\Psi}(\mathbf{k})^\dagger Q Q^\dagger \boldsymbol{\Psi}(\mathbf{k})}$

It all comes down to the model
how well that matches reality
and if the data supports those parameters

  • Decoding and interferometry

Ambiguities

The classic DOA case - very simple:

  • Perfect ambiguity:
    $\Psi(\mathbf{k})$ unique for every $\mathbf{k}$?
  • Practical ambiguity:
    Probability $\mathbb{P}$ that $\xi$ can perturb $\Psi(\mathbf{k}_1)$ to $\Psi(\mathbf{k}_2)$?

Kastinen, D. & Kero, J. AMT (2020)

  • Decoding and interferometry

Ambiguities

What I visualize

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Radar cross section

What we use for solid targets - already super simplified

$$ d = \left( \frac{4\lambda^4}{9 \pi^5} \text{RCS} \right)^{\frac{1}{6}} \forall\; d < \frac{\lambda}{\pi \sqrt{3}}\\ d = \left( \frac{4}{\pi} \text{RCS} \right)^{\frac{1}{2}} \forall\; d \geq \frac{\lambda}{\pi \sqrt{3}} $$
  • Radar cross section

What we use for solid targets - already super simplified

  • Radar cross section

For meteors?

Need formulas based on simulations! These are probably too simple

Great progress by Dimant, Oppenheim, Sugar, Dyrud, Tarnecki, et al
(Im currently porting the DO model from matlab to Python to try it out)

Using RCS to calculate mass is even harder
RCS mass is integrated over observation - this is problematic

Why?

  • Radar cross section

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Phenomenological model fitting
  • Phenomenological model fitting

Bayes' theorem

$$ P(\text{model} | \text{data}) = \frac{P(\text{data} | \text{model}) P(\text{model})}{P(\text{data})} $$

Important part is $P(\text{data} | \text{model})$

$$ \mathcal{P}(D | \mathbf{y}) = \prod\limits_{n} p_{rvk}(\tilde{r}_n, \tilde{v}_n, \tilde{\mathbf{k}}_n | r(t_n), \dot{r}(t_n), \hat{\mathbf{k}}(t_n)) $$
  • Phenomenological model fitting

Bayes' theorem

Lets say $p_{rvk} = p_r p_v p_k$ and that they are Normal

Then the log-posterior becomes a least-squares!

$$ \log(\mathcal{P}(D | \mathbf{y})) = C - \frac{1}{2}\sum\limits_n \left ( \frac{\tilde{r}_n - r(t_n)}{\sigma_r(t_n)} \right )^2 + \left ( \frac{\tilde{v}_n - v(t_n)}{\sigma_v(t_n)} \right )^2 \\ \notag - \frac{1}{2}\sum\limits_n \left ( ((\tilde{\mathbf{k}}_{xy})_n - \hat{\mathbf{k}}_{xy}(t_n))^T \Sigma_k^{-1}(t_n) ((\tilde{\mathbf{k}}_{xy})_n - \hat{\mathbf{k}}_{xy}(t_n)) \right ), $$
  • Phenomenological model fitting

Bayes' theorem

Now $P(\text{model} | \text{data})$ you can maximize, sample or calculate with whatever algorithm (Nelder-Mead, MCMC, ...)

  • Phenomenological model fitting

Bayes' theorem

$$ \mathbf{r}(t) = \mathbf{r}_0 + \hat{\mathbf{v}} \int_0^t v(\tau) \mathrm{d}\tau \\ v(t) = a + b t \\ v(t) = a - b e^{ct} $$

Im currently working on fitting a physical ablation model instead of an deceleration function along a line
need to be careful, easy to become ill determined!

  • Phenomenological model fitting

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination
  • Orbit determination
  • Orbit determination

rebound

  • Orbit determination
  • Orbit determination

Data processing of head echoes

Outline

  • Data and experiment description
  • Sensor model
  • Calibration
  • Matched filtering
  • Decoding and interferometry
  • Radar cross section
  • Phenomenological model fitting
  • Orbit determination

And that's it - how I would bake a meteor head echo piepeline 🥧