Radio Meteor Scatter – Improved Spectrum Lab Conditional Actions

I have been experimenting with meteor detection using radio forward scatter from the GRAVES transmitter and have developed an improved set of conditional actions for identifying and logging meteor events using Spectrum Lab. If you’re interested in trying this for yourself, then everything you need to get started is explained in a couple of articles by Paul Hyde of the BAA Radio Astronomy Group which were published by the Sky at Night Magazine. The articles and configuration files can be freely downloaded from the BAA web site.

Equipment

I won’t repeat the contents of the articles, which comprehensively explain how to build an antenna and receiver equipment.  Note that the original article recommends the use of the FunCube Pro+ SDR Dongle, which costs approximately £125 (GBP) at the time of writing.  In fact, it is entirely possible to undertake this project using a much cheaper RTL2832U based SDR dongle.

I have used the NooElec NESDR SMArt dongle which can be obtained from the manufacturer and other major retailers such as Amazon for approximately £25 (GBP) at the time of writing. This reduces the cost of the entire project to perhaps £60 (GBP) to build an antenna and set up the dongle assuming you have a suitable computer available to run Spectrum Lab.

Details of my setup and some useful information about the science of Meteor Scatter detection can be found in a presentation I have given, Meteor Scatter Detection -Advanced.

Improved Spectrum Lab Conditional Actions

The starter conditional actions available from the BAA are good for testing, but suffer from a number of deficiencies.  In another presentation (pages 25 – 28), Paul Hyde explains that the basic conditional actions set run on a cyclic basis, typically 5 or 10 times per second depending on your Spectrum Lab configuration. The upshot is that the actions run in a different processor thread to the actual FFT process which monitors the data stream for meteor signals. By the time the conditional actions are triggered by the output of a single FFT, multiple further FFTs will have been executed, rendering the readings of signal, noise, peak frequency, etc. taken by the conditional actions invalid.

Cyclic Conditional Actions

Cyclic Conditional Actions

Instead he recommends switching off the cyclic processing of conditional actions and instead triggering them each time a new FFT is run. This overcomes the above problems and produces valid data. A simple example of a few lines is provided in the above article, which I have used as the basis to develop a much more comprehensive setup.

Installation

VERY IMPORTANT: The new conditional actions have been tested and work with Spectrum Lab version 2.93 b8. They definitely do not work with Spectrum Lab version 2.94 b2 and your mileage may vary with other versions. The problem is that the conditional actions rely on being able to read and write particular Spectrum Lab internal variables which are available in the tested version, but disappeared in the later update. The author of Spectrum Lab confirmed that this is a bug and hopefully it will be rectified in a subsequent update!

Download the Spectrum Lab configuration files here.

IMPORTANT: If you have an existing meteor scatter (or other) Spectrum Lab configuration you wish to keep, use the Spectrum Lab ‘File -> Save Settings As..’ option to save them to a .USR file before proceeding. Then make a copy of that file somewhere else as a backup. Spectrum Lab can be a little erratic and it is possible to overwrite an existing configuration unexpectedly simply by closing Spectrum Lab.

In order to use this configuration open the ZIP file and proceed as follows:

  1. Copy Retune.exe to your Spectrum Lab executable folder. By default, the configuration file expects this to be ‘C:\Spectrum\Retune.exe’. If you need to copy the executable file elsewhere, once you have loaded the configuration, you will need to modify the interpreter commands by right clicking the ‘Tune to GRAVES’ and ‘Tune to Test Beacon’ buttons and changing the path at the end of the interpreter command section.
  2. If you (wisely) do not trust the executable I have provided, you can created your own using the ‘Retune.ahk’ file. You will need a copy of AutoHotKey. Load the .ahk file and convert it to a stand-alone executable file and install as described above. The purpose of ‘Retune.exe’ is simply to programatically click the mouse in Spectrum Lab’s ‘vfo’ box and then programatically hit the Enter key. Spectrum Lab’s own interpreter will quite happily change the frequency display in the ‘vfo’ box, but there is no interpreter mechanism to force the SDR dongle to re-tune to that frequency so it is a misleading cosmetic change. The ‘Tune To’ buttons in the supplied configuration file call ‘Retune.exe’ to make the retune actually happen.
  3. Copy ‘Meteor_freqlist.txt’ to the ‘C:\Spectrum\frequencies’ folder (or the frequencies folder inside your spectrum lab installation folder if different). This will show the GRAVES and GB3VHF frequency markers on the spectrum display.
  4. Copy ‘MetScat_Comprehensive_v7.USR’ to an appropriate location where Spectrum Lab can load it, such as your documents folder. Start Spectrum Lab and use ‘File – > Load Settings From..’ to load this file and configure Spectrum Lab.
  5. At this point you will need to check your SDR and other settings to ensure your SDR dongle is still working and correctly configured. It is beyond the scope of this README to explain how that is done. As a bare minimum check under ‘Options -> Audio I/O’ to ensure the correct ‘ExtIO_***.dll’ file is set as the source.
  6. If you have an existing configuration that you wish to retain and only require the improved set of conditional actions, these are available in the ‘MetScat_Conditional_Actions_V7.txt’ file. These can be loaded in to the conditional actions tab using the ‘Load’ button without affecting any of the other settings. Note, there is no need to do this if you have loaded the full configuration at step 4 above.

Configuration

If you open the conditional actions dialog via ‘File -> Conditional Actions…’ you will immediately note that the ‘Cyclic’ option has been disabled and the ‘on FFT’ option enabled. This causes the ‘new_spectrum’ condition to become true at line 18 each time a new FFT is processed. This line captures the key signal and noise data which is them monitored and used by the rest of the conditional actions.

FFT Conditional Actions

FFT Conditional Actions

The script is well documented with comments and uses meaningful variable names so that you can figure out what is going on. (As an aside, it’s the bane of my life that amateur software developers still use meaningless variable names like ‘a’, ‘b’, ‘c’, etc. making it painful to deduce how their code works!)

As the conditional actions model used by Spectrum Lab is headache inducing even for a seasoned professional software developer, the core of this script is a Finite State Machine with a number of other side actions queued or triggered using timers. Basically the ‘State’ variable tells you (and the script) what is going on, whether waiting for a meteor, monitoring one, logging it, etc. The current state is also displayed near the bottom of the programmable button set on the main waterfall display, hopefully simplifying debugging of any problems.

The activity of the script is controlled using the user-definable variables set in line 10. These are documented in the preceding lines but remember that you need to click the ‘”Init”‘ button to re-initialise everything before any changes become valid. The key ones that you may need to change are:

SNR_Threshold: This is set to 15 dB by default. You may need to increase it if you are getting too many false positive detections, or reduce it if you are missing meteors.

Waterfall_Length: The configuration marks the waterfall display once a second along the bottom of the screen. Count the number of seconds that fit on to your display and subtract 20%. This will trigger the screenshot of a meteor when it is close to, but not at, the left hand edge of the waterfall display maximising the chances of capturing the whole event.

Log_Path, Capture_Path, Sound_Path: Set the paths where you want to store your files. Note the use of double backslashes ‘\\’ as Spectrum Lab uses ‘\’ as the escape character in strings. NOTE: The folder must exist and be writeable by Spectrum Lab otherwise capturing / logging will fail silently until you fix the issue and restart Spectrum Lab.

Features

The conditional actions script provides the following features:

  • Captures screenshots of meteor events from the waterfall display. For events that are longer than the width of the waterfall, the script will attempt to capture multiple intermediate screenshots to ensure the whole event is recorded. The script also queues screen captures to capture each meteor event, whereas some scripts would miss subsequent events if one meteor is already scheduled for capture.
  • Captures .wav sound files of meteor events, which can be enabled or disabled via the user defined variables in the script.
  • Creates multiple log formats, logs can be opened by other applications (e.g. Colorgramme) whilst Spectrum Lab is active without file locking issues:
    • RMOB.dat and RMOB-dur.dat formats for upload to RMOB.
    • daily_log.csv and hourly_log.csv formats containing simple daily and hourly meteor counts.
    • event_log.csv format containing comprehensive data for each captured meteor event. The comma separated columns are:
      • Meteor Start Date (YYYY/MM/DD)
      • Meteor Start Time (hh:mm:ss)
      • Current daily meteor count (integer)
      • Peak Meteor Signal (float, dB)
      • Peak Meteor Noise (float, dB)
      • Peak Meteor Signal to Noise Ratio (float, dB)
      • Peak Meteor Frequency (integer, Hz)
      • Meteor Duration (float, seconds)
Programmable Buttons

Programmable Buttons

The programmable buttons on the left of the main waterfall display provide the following functions:

  • Last Event: Last event that occured (no function when clicked).
  • Capture Now: Captures a screenshot of the waterfall display.
  • Pause: Pauses the waterfall display and processing.
  • Tune to GRAVES: Tunes to the GRAVES frequency (right-click to edit the frequency and path to Retune.exe if needed).
  • Tune to Test Beacon: Tunes to the GB3VHF frequency. This is useful for some UK-based observers to test their equipment.  If you are in range, you should see an alternating pattern of continuous single frequency, a dashed (morse code-like) single frequency and then series of dashes shifting over a small range of frequencies.
  • Logging: Toggle logging of meteors, screenshot and sound captures on and off.
  • Signal Gen: Toggle injection of a strong signal in to the FFT. Useful for testing script triggers, etc.
  • Display Up / Down: Nudge the displayed frequency range up or down by a small amount. More accurate than using the standard Spectrum Lab display controls for small adjustments.
  • Offset Up / Down: The tuning of the SDR dongle is affected by its temperature, and these controls change the ‘Radio Frequency Offset’ found on the ‘Spectrum (2)’ tab to compensate. I tune to GB3VHF, wait for the continuous tone and then adjust these buttons to line the blue frequency marker with the actual tone displayed on the waterfall.
  • SNR: Displays the current peak SNR being read from the FFT for diagnostic purposes.
  • DMC/HMC/Q:  Displays the current daily meteor count, the current hourly meteor count and the number of screenshots currently queued for capture.
  • vfo: Displays the current frequency tuned.
  • SF: Displays the low and high frequency range on the waterfall display.
  • State: Displays the current state of the Finite State Machine in the conditional actions.
  • Time: Shows the current system time.

Bugs & Issues

  • Occasionally ‘Retune.exe’ does not work when the ‘Tune’ buttons are clicked. If you suspect this is the case, manually click in the ‘vfo’ box and hit the enter key.
  • The timestamps displayed on the waterfall may be significantly behind the PC’s clock. Whilst a short delay may be expected between capturing data and processing it for display, this drift can sometimes be in the order of minutes. It is not caused by the ‘Time’ slider controlling the waterfall display. It may be linked to having the conditional actions editor dialog open for long periods but this has not been proven. The simplest cure is to stop Spectrum Lab and restart it.
  • Timestamps used to name screenshots and sound captures may not correspond exactly to the timestamps of events in the log files. The limitations of Spectrum Lab’s interpreter make it difficult to create a queue of filenames to be used for each capture so screenshots and sounds are named using the now() timestamp at the point they are created, not the actual time of the event which triggered them. (For the avoidance of doubt, timing of events in log files should be accurate if your PC clock is accurate).
  • Timings for creation of RMOB log files are not defined anywhere I can find easily. The supplied conditional actions create RMOB log entries for the past hour at just before the hour ends, i.e. the count of meteors between say 23:00:00 and 23:59:59 will be timestamped for 23:00. Many, but not all, other observers seem to be using scripts which would stamp that entry 00:00 on the following day, which doesn’t make sense to me! It is easy enough to adjust the log trigger times if you prefer the other convention.

11 Responses to Radio Meteor Scatter – Improved Spectrum Lab Conditional Actions

  1. Paul Hearn 3rd March 2019 at 12:10 #

    Thanks so much Ian for this. I will try and let you know how I get on. This is all very new to me.

  2. John Berman 22nd May 2019 at 08:55 #

    This is a great script – looking forward to the Upload to Website development – Thanks Again

  3. Paul Hearn (UK) 11th June 2019 at 14:14 #

    I have just started using v7, have you developed the program further. All works but I’m unable to get it to write the RMOB files and the Hourly_log, but I’m sure I will get it sorted. Well documented, thanks.

  4. Simon Holbeche 8th November 2019 at 08:49 #

    Hi, Thanks for the conditional actions .txt as that is less of a change to my set up and downstream publishing. Changed folder designations, hard coded SpecFreqMin/Max and changed Current_Time to now in a few locations (this might not have been necessary as did during debug). The upshot is that the capture is operating again fine. Thanks for the good guidance and helpful article.

    • Ian Lauwerys 8th November 2019 at 09:28 #

      Current_Time records the time of the last FFT using the “time” function. It will almost certainly be different than “now”, since there is a delay in processing each FFT. One of the issues in simpler scripts with Cyclic processing and using “now” is that the script interpreter and the FFT process run in different threads and by reading times, signal and noise values, etc. in different lines is that they are constantly changing and so won’t correspond to each other. This script updates all of the Current_* variables once per FFT so the data corresponds and produces valid results.

  5. Steve Dearden 20th April 2020 at 15:18 #

    Hello Ian,
    I am just getting started in radio meteor scattering so I am very much a newbie! I have got Spectrum Lab up and running using Wolfgang Buescher’s Starter configuration file set. I did at first get too many false positives with this setting so I changed Line 4 in the associated Conditional settings for this configuration to a value of 18 or 19 which seems to have solved the problem.
    However, when I load your config and associated CA settings I am back to getting too many false events being captured.
    Any help would be greatfully received. I am using V2.93b8 of SL
    Thanks
    Steve

    • Ian Lauwerys 28th April 2020 at 15:11 #

      Hi Steve, the SNR threshold is set to 15dB in my script, so you will probably want to change it to 18 or 19 if that is what worked in the starter actions. It’s the same measurement so should be comparable. See the post where it says:

      “The activity of the script is controlled using the user-definable variables set in line 10. These are documented in the preceding lines but remember that you need to click the ‘”Init”‘ button to re-initialise everything before any changes become valid. The key ones that you may need to change are:

      SNR_Threshold: This is set to 15 dB by default. You may need to increase it if you are getting too many false positive detections, or reduce it if you are missing meteors.”

  6. Michael German 5th May 2020 at 12:17 #

    Hallo Ian,

    Many thanks for making this well written and useable pair of scripts. After evetually finding a version of Spectrum Lab (V2.94 b5) that has replaced the cfg handling capability, I am starting to use them. Having used SL in a paticular way for four of five years, you have revealed to me new features I hadn’t been aware of. I have the text files being created to suit my filing system so that they are automatically stored in, for example C:\systemA\2020\05 by year and month.

    How should the screenshot capture work from the CA script – I haven’t managed to get it working yet ?:-( Should it be periodic?
    I have reprogrammed the “Capture Now” button and that saves OK to my file structure but nothing happens from the CA script. (Multiple checks of syntax of the script lines – they are finicky – to no avail. Queing changes state)

    • Ian Lauwerys 11th May 2020 at 13:44 #

      The location of screen captures is set by the Capture_Path variable at line 10. You need to specify a valid directory location but you must use double backslashes in the path (per the example in the default config), as the \ character is the escape character in the SL interpreter. This means that when a backslash is encountered SL will actually use a special character determined by the character immediately following the backslash. Thus to get a plain \ you must insert \\ in the text.

      The folder you specify must exist already and be writable by Spectrum Lab (i.e. not in Program Files or some other protected location). If you start the conditional actions and it tries to write to a non-existent or protected folder, then even if you fix the problem, no screenshots will be saved until you exit and restart Spectrum Lab. You also need to set the Waterfall_Length variable to be about 20% less than the width of your screen display (in seconds) so that the screenshot is triggered when the event is near to but not at/past the left edge of the screen.

      Also check the “DMC / HMC / Q” button. This doesn’t do anything but the “Q = x” value shows how many screenshots are currently queued to be written. When a meteor (or other event) is detected, the value of Q should go up from 0 to 1, and when the event reaches near the left edge of the screen a screenshot should be saved and the value of Q go back to 0. (Obviously if several events happen in quick succession Q may go up to 2 or 3 and then tick back down 2, 1, 0). You can use the signal generator button to generate an event to test this.

  7. Mike German 19th May 2020 at 20:33 #

    Hallo Ian,

    Thanks for the advice. I tracked it down to a typo in my capture line

    if( queued_event ) then capture(Capture_Path + str(“YYYY”,now) + “\\” + str(“MM”,now) + “\\” + Log_Name + str(“YYYYMMDDhhmm”,now)) + “.jpg”, 100)

    … too many closing brackets before + “.jpg”

    “Test line ” reported CLI successful and I failed numerous times to properly count the opening and closing brackets. As I said, it is finicky!

    All operating nicely now – thanks again.

  8. John Berman 22nd May 2020 at 07:43 #

    Ian still using the script and it’s a dream. Did you have plans to add website elements to the script. Would happily test for you. John

Leave a Reply