Using AI for Data Conversion

I’m sure I’ve never mentioned that I’m an amateur radio operator. Like Vegans and Cross Fitters, we tend to be shy and withdrawn about our predilections. BWA-HA-HA! Ok, like Vegans and Cross Fitters, you can’t get a ham to shut up about playing radio. Anyhoo, I’ve been experimenting with a brand new, and somewhat buggy, radio, the Baofeng DM-32UV. I’m writing this blog post about how I am using an AI for data conversion to make it a little easier to use this radio.

The Problem

The DM-32 is a Digital Mobile Radio (DMR) as well as an analog radio. You can follow the link to understand all that DMR represents when talking radios. I want to focus on the fact that you have to program the behaviors into a DMR radio. While the end result is identical for every DMR radio, how you get there, the programming software, is radically different for every single radio (unless you get a radio that supports open source OpenGD77, yeah, playing radio involves open source as well). Which means, if I have more than one DMR radio (I’m currently at 7, and no, I don’t have a problem, shut up) I have more than one Customer Programming Software (CPS) that is completely different from other CPS formats. Now, I like to set up my radios similarly. After all, the local repeaters, my hotspot, and the Talkgroups I want to use are all common. Since every CPS is different, you can’t just export from one and import to the next. However, I had the idea of using AI for data conversion. Let’s see how that works.

So, for an example, this is the output of export for the Radioddity GD-88 CPS talk groups:

Name	DMR ID	Type	RX Group
ECHO	9990	Private	DMR,Echo
OKTac	31400	Group	DMR,OK Tac
NorthAm	93	Group	DMR,NorthAm

And this is the format expected for the DM-32:

No.	Name	ID	Type
1	Example	123	Private Call
2	Another example	321	Group Call

Yeah, pretty simple example, but I like to start small & build. I have to move data on two different columns, eliminate a column, generate a sequence, and add the word “Call” to each of the Types. Can I use AI for data conversion?

Using An AI for Data Conversion

First thing I tried was simple. I opened up VSCode where I’m running Github Copilot. I gave it the following prompt:

Please convert the data in the CSV file named gd88.csv into the format of the other CSV file, dm32.csv and add a space and the word “Call” to each instance of “Private” or “Group”

Here’s the output I got:

No.	Name	ID	Type
1	ECHO	9990	Private Call
2	OKTac	31400	Group Call
3	NorthAm	93	Group Call

I mean, it just worked. I imported the new data into the CPS and wrote it successfully to the radio. Done. Win!

OK. Well, that was too easy. Let’s try Channels, that’s a true test.

Converting Channel Data

The thing is, I only have 32 Talkgroups that I’m interested in. However, for DMR radio, I have to create a Channel for every frequency, for every talk group. So, let’s say I have two repeaters and my hotspot (no, I’m not explaining what all these things are) that I want Channels for, then I’m creating 96 channels. And there is a lot of extra information included with Channels. Here’s a picture of the GD-88 (too big to fit in a grid on the blog):

And here’s the DM-32:

As you can see… yeah, OK, you can’t really see it, but as you can tell, there are a lot more columns for the DM-32, let alone they’re arranged differently with varying names. Remember, both radios do, mostly, the same things, but they are programmed very differently to get there. I have typed out 96 different channels into a radio before. It’s a giant pain. So… can we use AI for data conversion on something this complex?

I’m going to try with Perplexity this time, just to toss in some variation. I uploaded both files to Perplexity and gave it this prompt:

I need to convert the data in the file gd88channel.csv into the format of the file dm32channel.csv. Can you please take care of that and display three rows of the results

I’m intentionally being kind of obtuse on these prompts. More than I would be otherwise. Well, Perplexity decided that what I needed, wasn’t a conversion of the data, but a script, so I could convert the data automatically. Clever AI. Here’s what I ended up with:

import pandas as pd

def convert_gd88_to_dm32(gd88_file, dm32_file):
    """
    Converts a GD88 channel CSV file to a DM32 channel CSV file format.

    Args:
        gd88_file (str): Path to the GD88 channel CSV file.
        dm32_file (str): Path to the DM32 channel CSV file.

    Returns:
        pandas.DataFrame: A DataFrame containing the converted data.
    """

    # Read the GD88 CSV file into a DataFrame
    gd88_df = pd.read_csv(gd88_file)

    # Create a new DataFrame with the DM32 column structure
    dm32_df = pd.DataFrame(columns=[
        "No.", "Channel Name", "Channel Type", "RX Frequency[MHz]", "TX Frequency[MHz]",
        "Power", "Band Width", "Scan List", "TX Admit", "Emergency System", "Squelch Level",
        "APRS Report Type", "Forbid TX", "APRS Receive", "Forbid Talkaround", "Auto Scan",
        "Lone Work", "Emergency Indicator", "Emergency ACK", "Analog APRS PTT Mode",
        "Digital APRS PTT Mode", "TX Contact", "RX Group List", "Color Code", "Time Slot",
        "Encryption", "Encryption ID", "APRS Report Channel", "Direct Dual Mode",
        "Private Confirm", "Short Data Confirm", "DMR ID", "CTC/DCS Encode", "CTC/DCS Decode",
        "Scramble", "RX Squelch Mode", "Signaling Type", "PTT ID", "VOX Function", "PTT ID Display"
    ])

    # Iterate over each row in the GD88 DataFrame and map the data to the DM32 DataFrame
    for index, row in gd88_df.iterrows():
        channel_type = "Analog" if row["Type"].upper() == "ANALOG" else "Digital"
        rx_freq_mhz = row["RX Freq"] / 1000000.0
        tx_freq_mhz = row["TX Freq"] / 1000000.0

        dm32_df = dm32_df.append({
            "No.": index + 1,
            "Channel Name": row["CH Name"],
            "Channel Type": channel_type,
            "RX Frequency[MHz]": rx_freq_mhz,
            "TX Frequency[MHz]": tx_freq_mhz,
            "Power": row["Power"],
            "Band Width": row["Bandwidth"],
            "Scan List": row["Scan List Name"],
            "TX Admit": "Allow TX",
            "Emergency System": "None",
            "Squelch Level": 3,
            "APRS Report Type": "Off",
            "Forbid TX": 0,
            "APRS Receive": 0,
            "Forbid Talkaround": 0,
            "Auto Scan": 0,
            "Lone Work": 0,
            "Emergency Indicator": 0,
            "Emergency ACK": 0,
            "Analog APRS PTT Mode": 0,
            "Digital APRS PTT Mode": 0,
            "TX Contact": row["Contact Name"],
            "RX Group List": row["RX Group Name"],
            "Color Code": row["RX CC"] if channel_type == "Digital" else 0,
            "Time Slot": row["RX TS"] if channel_type == "Digital" else "Slot 1",
            "Encryption": 0,
            "Encryption ID": "None",
            "APRS Report Channel": 1,
            "Direct Dual Mode": 0,
            "Private Confirm": 0,
            "Short Data Confirm": 0,
            "DMR ID": "Radio 1",
            "CTC/DCS Encode": "None",
            "CTC/DCS Decode": "None",
            "Scramble": "None",
            "RX Squelch Mode": "Carrier/CTC",
            "Signaling Type": "None",
            "PTT ID": "OFF",
            "VOX Function": 0,
            "PTT ID Display": 0
        }, ignore_index=True)

    return dm32_df

# File paths
gd88_file = "gd88channel.csv"
dm32_file = "dm32channel.csv"

# Convert the data
converted_df = convert_gd88_to_dm32(gd88_file, dm32_file)

# Display the first 3 rows of the converted data
print(converted_df.head(3))

I had to get pandas to make it work. And then I realized it wasn’t output to a file, so I added an output file and this code:

converted_df.to_csv(output_file, index=False)

The output from the script looked like this:

But the proof was when I imported it to the DM-32 CPS. Here are the channels:

IT WORKED!!!

Tried the same thing in VSCode and it generated code that restarted the counting of each group of channels, so the No. column repeated instead of acting like a sequence. Only the last three rows made it in. But that’s down to the lack of specificity in the prompt.

Conclusion

Can I use AI for data conversion? Well, yeah. And it’s doing a pretty damned good job with very little to go on. Could I get better results? Sure. With more a much more specific prompt I could see much better results. However, this is pretty cool.

Apologies for the Friday posting. I know I normally post on Monday mornings, but I wanted to get this one out the door so I could share it with my ham pals. Ha!

For my ham pals, it really did work. However, a few things to note. It didn’t get the Receive Groups, so I’ll have to edit that myself. Same goes for scan groups. The defaults were what was in the sample file, so if you want other settings, you’d need to adjust that. Obviously, if you’re coming from other radios into the DM-32, you’ll need to get a different sample file from each CPS. But, you can do this yourselves. Just be very specific with the prompts to the AI engines.