Radio Meteor Scatter – Improved Spectrum Lab Conditional Actions V7

I have been experimenting with meteor detection using radio forward scatter from the GRAVES transmitter and have developed an improved set of conditional actions for identifying and logging meteor events using Spectrum Lab. If you’re interested in trying this for yourself, then everything you need to get started is explained in a couple of articles by Paul Hyde of the BAA Radio Astronomy Group which were published by the Sky at Night Magazine. The articles and a lot of other useful information can be freely downloaded from the BAA web site. Unfortunately the Spectrum Lab configuration files referred to in the original Sky at Night articles no longer appear to be available following reorganisations of both the Sky at Night Magazine and BAA web sites. There is some limited information on setting up Spectrum Lab in the “Techniques for using the RTL Dongle for Detecting Meteors” article linked on the BAA web site.

Equipment

I won’t repeat the contents of the articles, which comprehensively explain how to build an antenna and receiver equipment.  Note that the original article recommends the use of the FunCube Pro+ SDR Dongle, which costs approximately £125 (GBP) at the time of writing.  In fact, it is entirely possible to undertake this project using a much cheaper RTL2832U based SDR dongle.

I have used the NooElec NESDR SMArt dongle which can be obtained from the manufacturer and other major retailers such as Amazon for approximately £25 (GBP) at the time of writing. This reduces the cost of the entire project to perhaps £60 (GBP) to build an antenna and set up the dongle assuming you have a suitable computer available to run Spectrum Lab.

Details of my setup and some useful information about the science of Meteor Scatter detection can be found in a presentation I have given, Meteor Scatter Detection -Advanced.

Improved Spectrum Lab Conditional Actions

The starter conditional actions available from the BAA are good for testing, but suffer from a number of deficiencies.  In another presentation (pages 25 – 28), Paul Hyde explains that the basic conditional actions set run on a cyclic basis, typically 5 or 10 times per second depending on your Spectrum Lab configuration. The upshot is that the actions run in a different processor thread to the actual FFT process which monitors the data stream for meteor signals. By the time the conditional actions are triggered by the output of a single FFT, multiple further FFTs will have been executed, rendering the readings of signal, noise, peak frequency, etc. taken by the conditional actions invalid.

Cyclic Conditional Actions
Cyclic Conditional Actions

Instead he recommends switching off the cyclic processing of conditional actions and instead triggering them each time a new FFT is run. This overcomes the above problems and produces valid data. A simple example of a few lines is provided in the above article, which I have used as the basis to develop a much more comprehensive setup.

Installation

IMPORTANT: The V7 configuration files have now been superseded by an improved V8 configuration.  You can read about the new configuration in this post.  The post you are reading now still contains useful information, but you should refer to the newer post to download, install and use the updated V8 configuration as the V7 configuration will have significant issues with current versions of Spectrum Lab!

The v7 conditional actions were tested and work with Spectrum Lab version 2.93 b8. They definitely do not work with Spectrum Lab version 2.94 b2 due to a bug which erroneously removed access to certain internal variables in the interpreter. The bug has been fixed in currently available versions but some other changes cause problems with the V7 configuration.

Download the Spectrum Lab v7 configuration files here ( retained historical reference only).

IMPORTANT: If you have an existing meteor scatter (or other) Spectrum Lab configuration you wish to keep, use the Spectrum Lab ‘File -> Save Settings As..’ option to save them to a .USR file before proceeding. Then make a copy of that file somewhere else as a backup. Spectrum Lab can be a little erratic and it is possible to overwrite an existing configuration unexpectedly simply by closing Spectrum Lab.

In order to use this configuration open the ZIP file and proceed as follows:

  1. Copy Retune.exe to your Spectrum Lab executable folder. By default, the configuration file expects this to be ‘C:\Spectrum\Retune.exe’. If you need to copy the executable file elsewhere, once you have loaded the configuration, you will need to modify the interpreter commands by right clicking the ‘Tune to GRAVES’ and ‘Tune to Test Beacon’ buttons and changing the path at the end of the interpreter command section.
  2. If you (wisely) do not trust the executable I have provided, you can created your own using the ‘Retune.ahk’ file. You will need a copy of AutoHotKey. Load the .ahk file and convert it to a stand-alone executable file and install as described above. The purpose of ‘Retune.exe’ is simply to programatically click the mouse in Spectrum Lab’s ‘vfo’ box and then programatically hit the Enter key. Spectrum Lab’s own interpreter will quite happily change the frequency display in the ‘vfo’ box, but there is no interpreter mechanism to force the SDR dongle to re-tune to that frequency so it is a misleading cosmetic change. The ‘Tune To’ buttons in the supplied configuration file call ‘Retune.exe’ to make the retune actually happen.
  3. Copy ‘Meteor_freqlist.txt’ to the ‘C:\Spectrum\frequencies’ folder (or the frequencies folder inside your spectrum lab installation folder if different). This will show the GRAVES and GB3VHF frequency markers on the spectrum display.
  4. Copy ‘MetScat_Comprehensive_v7.USR’ to an appropriate location where Spectrum Lab can load it, such as your documents folder. Start Spectrum Lab and use ‘File – > Load Settings From..’ to load this file and configure Spectrum Lab.
  5. At this point you will need to check your SDR and other settings to ensure your SDR dongle is still working and correctly configured. It is beyond the scope of this README to explain how that is done. As a bare minimum check under ‘Options -> Audio I/O’ to ensure the correct ‘ExtIO_***.dll’ file is set as the source.
  6. If you have an existing configuration that you wish to retain and only require the improved set of conditional actions, these are available in the ‘MetScat_Conditional_Actions_V7.txt’ file. These can be loaded in to the conditional actions tab using the ‘Load’ button without affecting any of the other settings. Note, there is no need to do this if you have loaded the full configuration at step 4 above.

Configuration

If you open the conditional actions dialog via ‘File -> Conditional Actions…’ you will immediately note that the ‘Cyclic’ option has been disabled and the ‘on FFT’ option enabled. This causes the ‘new_spectrum’ condition to become true at line 18 each time a new FFT is processed. This line captures the key signal and noise data which is them monitored and used by the rest of the conditional actions.

FFT Conditional Actions
FFT Conditional Actions

The script is well documented with comments and uses meaningful variable names so that you can figure out what is going on. (As an aside, it’s the bane of my life that amateur software developers still use meaningless variable names like ‘a’, ‘b’, ‘c’, etc. making it painful to deduce how their code works!)

As the conditional actions model used by Spectrum Lab is headache inducing even for a seasoned professional software developer, the core of this script is a Finite State Machine with a number of other side actions queued or triggered using timers. Basically the ‘State’ variable tells you (and the script) what is going on, whether waiting for a meteor, monitoring one, logging it, etc. The current state is also displayed near the bottom of the programmable button set on the main waterfall display, hopefully simplifying debugging of any problems.

The activity of the script is controlled using the user-definable variables set in line 10. These are documented in the preceding lines but remember that you need to click the ‘”Init”‘ button to re-initialise everything before any changes become valid. The key ones that you may need to change are:

SNR_Threshold: This is set to 15 dB by default. You may need to increase it if you are getting too many false positive detections, or reduce it if you are missing meteors.

Waterfall_Length: The configuration marks the waterfall display once a second along the bottom of the screen. Count the number of seconds that fit on to your display and subtract 20%. This will trigger the screenshot of a meteor when it is close to, but not at, the left hand edge of the waterfall display maximising the chances of capturing the whole event.

Log_Path, Capture_Path, Sound_Path: Set the paths where you want to store your files. Note the use of double backslashes ‘\\’ as Spectrum Lab uses ‘\’ as the escape character in strings. NOTE: The folder must exist and be writeable by Spectrum Lab otherwise capturing / logging will fail silently until you fix the issue and restart Spectrum Lab.

Features

The conditional actions script provides the following features:

  • Captures screenshots of meteor events from the waterfall display. For events that are longer than the width of the waterfall, the script will attempt to capture multiple intermediate screenshots to ensure the whole event is recorded. The script also queues screen captures to capture each meteor event, whereas some scripts would miss subsequent events if one meteor is already scheduled for capture.
  • Captures .wav sound files of meteor events, which can be enabled or disabled via the user defined variables in the script.
  • Creates multiple log formats, logs can be opened by other applications (e.g. Colorgramme) whilst Spectrum Lab is active without file locking issues:
    • RMOB.dat and RMOB-dur.dat formats for upload to RMOB.
    • daily_log.csv and hourly_log.csv formats containing simple daily and hourly meteor counts.
    • event_log.csv format containing comprehensive data for each captured meteor event. The comma separated columns are:
      • Meteor Start Date (YYYY/MM/DD)
      • Meteor Start Time (hh:mm:ss)
      • Current daily meteor count (integer)
      • Peak Meteor Signal (float, dB)
      • Peak Meteor Noise (float, dB)
      • Peak Meteor Signal to Noise Ratio (float, dB)
      • Peak Meteor Frequency (integer, Hz)
      • Meteor Duration (float, seconds)
Programmable Buttons
Programmable Buttons

The programmable buttons on the left of the main waterfall display provide the following functions:

  • Last Event: Last event that occured (no function when clicked).
  • Capture Now: Captures a screenshot of the waterfall display.
  • Pause: Pauses the waterfall display and processing.
  • Tune to GRAVES: Tunes to the GRAVES frequency (right-click to edit the frequency and path to Retune.exe if needed).
  • Tune to Test Beacon: Tunes to the GB3VHF frequency. This is useful for some UK-based observers to test their equipment.  If you are in range, you should see an alternating pattern of continuous single frequency, a dashed (morse code-like) single frequency and then series of dashes shifting over a small range of frequencies.
  • Logging: Toggle logging of meteors, screenshot and sound captures on and off.
  • Signal Gen: Toggle injection of a strong signal in to the FFT. Useful for testing script triggers, etc.
  • Display Up / Down: Nudge the displayed frequency range up or down by a small amount. More accurate than using the standard Spectrum Lab display controls for small adjustments.
  • Offset Up / Down: The tuning of the SDR dongle is affected by its temperature, and these controls change the ‘Radio Frequency Offset’ found on the ‘Spectrum (2)’ tab to compensate. I tune to GB3VHF, wait for the continuous tone and then adjust these buttons to line the blue frequency marker with the actual tone displayed on the waterfall.
  • SNR: Displays the current peak SNR being read from the FFT for diagnostic purposes.
  • DMC/HMC/Q:  Displays the current daily meteor count, the current hourly meteor count and the number of screenshots currently queued for capture.
  • vfo: Displays the current frequency tuned.
  • SF: Displays the low and high frequency range on the waterfall display.
  • State: Displays the current state of the Finite State Machine in the conditional actions.
  • Time: Shows the current system time.

Bugs & Issues

  • Occasionally ‘Retune.exe’ does not work when the ‘Tune’ buttons are clicked. If you suspect this is the case, manually click in the ‘vfo’ box and hit the enter key.
  • The timestamps displayed on the waterfall may be significantly behind the PC’s clock. Whilst a short delay may be expected between capturing data and processing it for display, this drift can sometimes be in the order of minutes. It is not caused by the ‘Time’ slider controlling the waterfall display. It may be linked to having the conditional actions editor dialog open for long periods but this has not been proven. The simplest cure is to stop Spectrum Lab and restart it.
  • Timestamps used to name screenshots and sound captures may not correspond exactly to the timestamps of events in the log files. The limitations of Spectrum Lab’s interpreter make it difficult to create a queue of filenames to be used for each capture so screenshots and sounds are named using the now() timestamp at the point they are created, not the actual time of the event which triggered them. (For the avoidance of doubt, timing of events in log files should be accurate if your PC clock is accurate).
  • Timings for creation of RMOB log files are not defined anywhere I can find easily. The supplied conditional actions create RMOB log entries for the past hour at just before the hour ends, i.e. the count of meteors between say 23:00:00 and 23:59:59 will be timestamped for 23:00. Many, but not all, other observers seem to be using scripts which would stamp that entry 00:00 on the following day, which doesn’t make sense to me! It is easy enough to adjust the log trigger times if you prefer the other convention.

19 thoughts on “Radio Meteor Scatter – Improved Spectrum Lab Conditional Actions V7”

  1. Thanks so much Ian for this. I will try and let you know how I get on. This is all very new to me.

  2. Paul Hearn (UK)

    I have just started using v7, have you developed the program further. All works but I’m unable to get it to write the RMOB files and the Hourly_log, but I’m sure I will get it sorted. Well documented, thanks.

  3. Hi, Thanks for the conditional actions .txt as that is less of a change to my set up and downstream publishing. Changed folder designations, hard coded SpecFreqMin/Max and changed Current_Time to now in a few locations (this might not have been necessary as did during debug). The upshot is that the capture is operating again fine. Thanks for the good guidance and helpful article.

    1. Current_Time records the time of the last FFT using the “time” function. It will almost certainly be different than “now”, since there is a delay in processing each FFT. One of the issues in simpler scripts with Cyclic processing and using “now” is that the script interpreter and the FFT process run in different threads and by reading times, signal and noise values, etc. in different lines is that they are constantly changing and so won’t correspond to each other. This script updates all of the Current_* variables once per FFT so the data corresponds and produces valid results.

  4. Hello Ian,
    I am just getting started in radio meteor scattering so I am very much a newbie! I have got Spectrum Lab up and running using Wolfgang Buescher’s Starter configuration file set. I did at first get too many false positives with this setting so I changed Line 4 in the associated Conditional settings for this configuration to a value of 18 or 19 which seems to have solved the problem.
    However, when I load your config and associated CA settings I am back to getting too many false events being captured.
    Any help would be greatfully received. I am using V2.93b8 of SL
    Thanks
    Steve

    1. Hi Steve, the SNR threshold is set to 15dB in my script, so you will probably want to change it to 18 or 19 if that is what worked in the starter actions. It’s the same measurement so should be comparable. See the post where it says:

      “The activity of the script is controlled using the user-definable variables set in line 10. These are documented in the preceding lines but remember that you need to click the ‘”Init”‘ button to re-initialise everything before any changes become valid. The key ones that you may need to change are:

      SNR_Threshold: This is set to 15 dB by default. You may need to increase it if you are getting too many false positive detections, or reduce it if you are missing meteors.”

  5. Michael German

    Hallo Ian,

    Many thanks for making this well written and useable pair of scripts. After evetually finding a version of Spectrum Lab (V2.94 b5) that has replaced the cfg handling capability, I am starting to use them. Having used SL in a paticular way for four of five years, you have revealed to me new features I hadn’t been aware of. I have the text files being created to suit my filing system so that they are automatically stored in, for example C:\systemA\2020\05 by year and month.

    How should the screenshot capture work from the CA script – I haven’t managed to get it working yet ?:-( Should it be periodic?
    I have reprogrammed the “Capture Now” button and that saves OK to my file structure but nothing happens from the CA script. (Multiple checks of syntax of the script lines – they are finicky – to no avail. Queing changes state)

    1. The location of screen captures is set by the Capture_Path variable at line 10. You need to specify a valid directory location but you must use double backslashes in the path (per the example in the default config), as the \ character is the escape character in the SL interpreter. This means that when a backslash is encountered SL will actually use a special character determined by the character immediately following the backslash. Thus to get a plain \ you must insert \\ in the text.

      The folder you specify must exist already and be writable by Spectrum Lab (i.e. not in Program Files or some other protected location). If you start the conditional actions and it tries to write to a non-existent or protected folder, then even if you fix the problem, no screenshots will be saved until you exit and restart Spectrum Lab. You also need to set the Waterfall_Length variable to be about 20% less than the width of your screen display (in seconds) so that the screenshot is triggered when the event is near to but not at/past the left edge of the screen.

      Also check the “DMC / HMC / Q” button. This doesn’t do anything but the “Q = x” value shows how many screenshots are currently queued to be written. When a meteor (or other event) is detected, the value of Q should go up from 0 to 1, and when the event reaches near the left edge of the screen a screenshot should be saved and the value of Q go back to 0. (Obviously if several events happen in quick succession Q may go up to 2 or 3 and then tick back down 2, 1, 0). You can use the signal generator button to generate an event to test this.

  6. Hallo Ian,

    Thanks for the advice. I tracked it down to a typo in my capture line

    if( queued_event ) then capture(Capture_Path + str(“YYYY”,now) + “\\” + str(“MM”,now) + “\\” + Log_Name + str(“YYYYMMDDhhmm”,now)) + “.jpg”, 100)

    … too many closing brackets before + “.jpg”

    “Test line ” reported CLI successful and I failed numerous times to properly count the opening and closing brackets. As I said, it is finicky!

    All operating nicely now – thanks again.

  7. I have problem understanding setings in .usr file. There is “SampleRate=2400000” for the soundcard. I have only virtual cable and Samsung HDMI audio out. It does not works. Tried to lower samplerate, but still no luck. Any help, please.

  8. 06:56:00 SDRctrl: First error (FTDI driver failed) detected in line 820.
    06:56:00 SDR (line 823): Couldn’t load the FTDI driver dll.
    06:56:00 SDR : Couldn’t load FTDI driver !
    06:56:02 SDR : Tried to ‘Start’ but port not open.
    06:56:02 SDR-IQ/IP started, ‘running’ = 0, clkin=66666666.667000 Hz, fs=55555.555556 Hz, vfo=143048016.277547 Hz.
    06:56:02 Error configuring SDR-IQ !

  9. Having problem with non existing FTDI converter. Got error messages:

    06:56:00 SDRctrl: First error (FTDI driver failed) detected in line 820.
    06:56:00 SDR (line 823): Couldn’t load the FTDI driver dll.
    06:56:00 SDR : Couldn’t load FTDI driver !
    06:56:02 SDR : Tried to ‘Start’ but port not open.
    06:56:02 SDR-IQ/IP started, ‘running’ = 0, clkin=66666666.667000 Hz, fs=55555.555556 Hz, vfo=143048016.277547 Hz.
    06:56:02 Error configuring SDR-IQ !

    And before that:

    Invalid plot file header had to be corrected !
    Could not map view of file “D:\plots\plotter0.tmp”.

    Please help. What is wrong. Tried to install some FTDI or “USB serial converter”, but on Windows 10 it is not possible. Thanks.

    1. It sounds like hardware issues. The supplied config files were created for a NooElec NESDR RTL2832U based SDR dongle. If you are using different hardware (or possibly even the same hardware) then after loading the config file you will need to go in to the “Options -> Audio settings, IO device selection” dialog and set up your capture hardware and the config will overwrite whatever was there previously. Hard to give specific instructions without knowing what your setup is.

      The plot file error can be fixed by going to “Views/Windows -> Watch list & plot window”. In the “SL Watch List & Plotter” dialog that opens, choose “File -> Change plot file name” and choose a valid folder and file name where you have permissions to write.

  10. Sorry Ian i,m not IT literate , however wanting to put this into operation.
    Using windows 10 , how to you go about putting Retune .ex into Spectrum folder. Do i. just copy and put anywhere in Spectrum folder. Did it so when i open it i can see the Retune .ex H logo and the folder.
    Is this all i need to do or how do i get the config file to look at and load Retune.ex? If that makes sense.
    Showing screen shots would help.
    My screen shows the Spectrum .ex folder then the Settings ini then the Retune.ex folder then the AHK folder. So all are in main Spectrum folder. How do i get it to work?

    1. Brian, you just right-click on the “Retune.exe” file (the one with the H logo) and select “Copy”. Then go in to the Spectrum folder, right-click on a blank area (e.g. to the right of the list of files) and select “Paste”. You don’t need the AHK folder or anything else in there.

  11. Hi Ian,
    I have a possibly air headed question about the S-N value. I’ve set it to various values (12 thru 24) in your fantastic usr-file and I’m still getting several false captures.

    Question: Is there *something* that will reveal what the actual S-N level is in SL? Or the average S-N level? Can that value be displayed? Is it already displayed or available and I just don’t see it?

    Hopefully I’ll be able to see what range the S-N level is as the program is running, and I can then adjust the value in the script accordingly.

    Thanks a million!!

    -john

    1. John, the SNR is printed in the information on the waterfall in each meteor detection. It’s the number immediately after “Peak SNR:” and corresponds directly to the value in the conditional actions settings, so just set it higher than the values on your false detections and they should no longer be detected as events.

Leave a Reply to Simon Holbeche Cancel Reply

Your email address will not be published. Required fields are marked *