courses

Making the CaribouLite transmit: the transmission_fix branch

The stock CaribouLite does not transmit modulated I/Q. The modem’s IQIFC2.SYNC bit never reaches 1, so the AT86RF215 never locks onto the I/Q stream the FPGA sends it and its DAC is never fed. That is upstream issue #189, open since February 2024. The full measurement trail is on the CaribouLite page.

A fix exists: PR #197, “Transmission of arbitrary waveforms” — closed unmerged, with no review comments, on 5 August 2025. The work is still maintained on the transmission_fix branch of matteoserva/cariboulite.

It works. Built and measured on this bench against head beaf032: IQIFC2.SYNC reaches 1, and an external HackRF Pro sees the emission rise +62.30 dB (76.4 σ) where the stock code managed +0.21 dB. A Morse callsign sent through the I/Q path decoded back off the air intact.

This directory holds what you need to reproduce that, plus the two patches the branch’s own instructions do not mention.

Files

file what it does
patch_smi_txfix.py makes the branch’s kernel driver compile on kernels ≥ 6.4 / ≥ 6.11
gen_tone.py continuous complex tone at a fixed offset, to stdout — the basic “does it radiate” waveform
gen_morse.py Morse-keyed tone to a file, raised-cosine edges — a real CW identification through the I/Q path
gen_amp_steps.py one file stepping through several drive levels, for a self-timing linearity sweep
patch_tx_gain.py makes tx_test’s -g actually set the transmit power (it is parsed and then discarded)
patch_tx_cal.py exposes the modem’s own LO-leakage trim as -I/-Q, and -S/-D to step it mid-transmission
install-persistent.sh DKMS + systemd so the fix survives a reboot
iqcal.py the transmit pre-distortion itself — loads coefficients, applies x + k·conj(x) + d
gen_cal_tone.py single calibrated tone to a file, coefficients from the command line or the saved set
gen_probe_sweep.py several pre-distortion settings inside one self-timed waveform, for calibration
calibrate.py runs on the host with a HackRF watching: measures the coefficients and writes them
sweep_txcal.py runs on the host: searches the modem’s own 6-bit TXCI/TXCQ trim codes

Every generator writes int16 complex interleaved I/Q, which is what tx_test reads. gen_tone.py, gen_morse.py, and gen_cal_tone.py apply the saved calibration by default and take --no-cal to skip it; gen_amp_steps.py is the other way round (--cal to apply it), because its job is to characterise the raw hardware.

Build

Host needs cmake, yosys, nextpnr-ice40, icepack. Everything else is built natively on the Pi, which avoids needing an aarch64 cross toolchain.

git clone https://github.com/cariboulabs/cariboulite.git
cd cariboulite
git remote add matteoserva https://github.com/matteoserva/cariboulite.git
git fetch matteoserva
git worktree add ../cariboulite-txfix -b txfix matteoserva/transmission_fix
cd ../cariboulite-txfix

1. The bitstream, on the host

make firmware          # yosys + nextpnr-ice40 + icepack; meets timing at 80 MHz

The branch deletes the prebuilt top.bin, so this step is mandatory.

2. The generated firmware header

make tx_test cannot work from a clean tree. It compiles cariboulite_setup.c, which #includes cariboulite_fpga_firmware.h — the bitstream as a C array. Nothing in the documented flow generates it: the firmware target builds top.bin only, not the build target that runs generate_bin_blob. The header is in .gitignore, so a fresh clone never has it. The checked-in generate_bin_blob is an x86-64 binary, so rebuild it from source:

gcc -O2 -o /tmp/generate_bin_blob software/utils/generate_bin_blob.c
mkdir -p firmware/h-files
/tmp/generate_bin_blob firmware/top.bin cariboulite_firmware \
    firmware/h-files/cariboulite_fpga_firmware.h
cp firmware/h-files/cariboulite_fpga_firmware.h software/libcariboulite/src/

3. Patch the driver for a current kernel

The driver is written against pre-6.4 kernels. On 6.18 it fails with:

error: too many arguments to function 'class_create'
error: initialization of 'void (*)(struct platform_device *)' from
       incompatible pointer type 'int (*)(struct platform_device *)'

class_create() lost its owner argument in 6.4; platform_driver.remove became void-returning in 6.11. These are the same breakages the stock driver has — patch_smi_txfix.py is ../setup/patch_smi.py with its anchors re-indented for this branch’s tabs. The shims are guarded on LINUX_VERSION_CODE, so the result still builds on older kernels.

python3 patch_smi_txfix.py driver/smi_stream_dev.c

4. Build on the Pi

rsync -a --exclude .git ./ pi:~/cariboulite-txfix/
ssh pi 'cd ~/cariboulite-txfix && scripts/build-standalone.sh all'

ssh pi 'cd ~/cariboulite-txfix && mkdir -p driver/build-native &&
        cp driver/*.c driver/*.h driver/build-native/ &&
        printf "obj-m += smi_stream_dev.o\nccflags-y += -O2\n" > driver/build-native/Makefile &&
        make -C /lib/modules/$(uname -r)/build \
             M=$HOME/cariboulite-txfix/driver/build-native modules'

On-target builds must stay at -j2; the Pi OOMs above that.

Run

flash_fpga and tx_test need no root if you are in the gpio and spi groups and the udev rule puts /dev/smi in group gpio. Only insmod needs root.

./flash_fpga ./top.bin                      # once per power cycle
sudo insmod ./smi_stream_dev.ko             # once per boot

Neither survives a reboot by itself. The ICE40 is configured into SRAM, not flash, and the module reverts to the distribution’s smi_stream_dev on boot. A transmitter that worked yesterday reports FPGA is not programmed (CDONE low) this morning. Run install-persistent.sh (below) so you do not have to think about it.

Persistence

sudo ./install-persistent.sh ~/cariboulite-txfix    # install and enable
sudo ./install-persistent.sh --uninstall            # back to stock

It does the two things the branch does not:

  1. DKMS. Installs the patched driver as smi-stream-dev/1.1 and uninstalls the stock 1.0 — only one DKMS version may provide smi_stream_dev, or boot-time precedence is down to DKMS ordering. AUTOINSTALL="yes", so it rebuilds against future kernels; the 1.0 source stays in /usr/src so --uninstall can restore the vendor driver, and a PROVENANCE file explains the modified source to whoever finds it next.
  2. systemd oneshot cariboulite-fpga.service, Before=multi-user.target. Ordering matters: libcariboulite programs its own stock bitstream whenever it finds the FPGA unconfigured, so anything that touches the radio before this service runs silently undoes the fix. The hook waits up to 30 s for /dev/spidev1.0 and /dev/gpiochip0 (udev may not have made them yet), checks the flasher’s output for FPGA programming - Success rather than trusting its exit status, retries three times, and only then rebinds the driver — flash first, load second, which is the order the fix was verified in.

Verified across a reboot: the service completes two seconds into boot, the patched module loads, IQIFC2.SYNC reads 1, and the callsign transmits and decodes back off the air with no manual step. Receive unaffected, 3/3 PASS.

Caveats: that was a warm reboot, so the FPGA may not actually have lost its SRAM configuration — the test proves the service runs, not that it was needed. And DKMS surviving a real kernel upgrade has not been observed on this board, only configured.

Confirm the fix is actually active before trusting anything downstream — this is the one line that distinguishes a working datapath from the stock failure:

Status 'Is synchronized to incoming I/Q': 1

With the new bitstream but the stock driver it still reads 0. Both halves are required.

Radiate something

# continuous tone 250 kHz above the carrier -> emission at 445.500 MHz
python3 gen_tone.py 2000000 250000 12 0.7 |
    ./tx_test -c 1 -f 445250000 -r 2000000 -g 50 -

# a real CW identification through the I/Q path
python3 gen_morse.py 2000000 250000 "DE AD7KM" /tmp/id.bin 15 0.7
./tx_test -c 1 -f 445250000 -r 2000000 -g 50 /tmp/id.bin

Tuning the carrier below the intended emission and putting the signal at a baseband offset is deliberate: the emission then lands clear of the transmitter’s own LO leakage and of the receiver’s DC spur, so neither can be mistaken for it. Every false positive in this board’s history was a spur at a tuned centre.

Both of those generators apply the transmit calibration from ~/.cariboulite_iqcal.json if it exists, and pass --no-cal to skip it. See Calibration below for how that file is produced — and note that a set measured at another frequency or gain is worse than none.

Two traps

Python cannot feed 8 MB/s. At 2 Msps a per-sample struct.pack loop underruns the transmitter mid-waveform. The first Morse ID attempted this way radiated at 60.9 dB of contrast and decoded as NEE EE E E T NEEE N NEEEE E TEE T E N T. It looked like a radio fault and was a generator fault. gen_morse.py and gen_amp_steps.py use numpy and write files; only gen_tone.py streams, and it does so by repeating one precomputed block.

Check that your capture is complete. hackrf_transfer can drop samples and stall without failing. Compare the output file size against rate × seconds × 2 bytes and reject the run if it is short — otherwise every wall-clock window in the analysis is aligned to the wrong part of the recording.

Calibration

Uncorrected, the transmitter leaks a carrier at the tuned frequency (−26 dBc) and an image of the signal mirrored about it (−36.5 dBc). Both can be pre-distorted away. The impairment is widely linear, y = α·x + β·conj(x) + D, so the correction has the same shape:

x′ = x + k·conj(x) + d        k = −β/α        d = −D/α

calibrate.py runs on the host, with a HackRF watching the board, and measures k and d:

# measure (two runs, so the spread is visible) and install on the Pi
python3 calibrate.py --runs 2 --install

# corrected against uncorrected, in one transmission
python3 calibrate.py --verify

# and again at a lower receive gain, which is not optional
python3 calibrate.py --verify --rx "-a 0 -l 20 -g 8"

# hold a hardware trim code while measuring, and size the probe step to
# the residual it is measuring rather than to the default
python3 calibrate.py --runs 1 --txcal 35:34 --dd 0.002

It writes ~/.cariboulite_iqcal.json, which the generators then load by themselves. Edit the constants at the top of the file for a different frequency, gain, or rate — and re-measure, because the coefficients are valid only for the one combination recorded in the file.

Pick the lower receive gain so it is lower, not useless: -l 12 -g 10 lands this board at ADC 3/127, where the measurement floor is −51 dBc and every deep null reads 4 dB shallower than it is. -l 20 -g 8 gives ADC 13/127 and a −68 dBc floor, which is a real second gain.

Measured here at 445.680 MHz, -g 24, 2.000 Msps, with corrected and uncorrected segments in the same transmission:

segment carrier image
both off −25.8 dBc −36.3 dBc
both on −45.1 dBc −40.7 dBc
k only −25.3 dBc (+0.5 dB) −36.2 dBc (+0.2 dB)
d only −32.6 dBc (−6.8 dB) −38.2 dBc (−1.9 dB)

Those are one transmission’s figures. Repeated over four later runs the corrected row lands anywhere in −39.5 … −44.8 dBc on the carrier and −39.5 … −41.1 dBc on the image — see Which configuration to use for the spread and for how this compares with correcting in hardware instead.

The quadrature correction alone does nothing — +0.2 dB on the image, at both receive gains. What the image responds to is d, which is the opposite of the textbook expectation and took a second set of experiments to pin down; see What the hardware already does. The correction costs 0.38 dB of headroom, because x + k·conj(x) + d peaks at (1 + |k|)·|x| + |d|; the generators scale by the reciprocal so a given amp means the same power either way.

d is an absolute offset, not a relative one. apply_cal adds it after amp has been applied, so the carrier it radiates on its own is 20·log₁₀(|d|/amp) — for the measured |d| = 0.0097 at amp = 0.5 that is −34.3 dBc, and the measurement agrees to 0.1 dB. Two consequences worth keeping in mind: the suppression you can reach in dBc depends on amp, and a d larger than the offset it is cancelling makes the carrier worse. That second one is not hypothetical — it is what happens if you reuse a set measured against one TXCI/TXCQ code at another.

Six ways this measurement lies

Each of these produced confident, wrong numbers before it was caught.

  1. Averaging a complex ratio across blocks. The frequencies are known only to a few Hz, so block phase rotates and a coherent average cancels itself. It under-read the image by 15 dB and inverted its phase; the correction built on it pushed the carrier 13.4 dB up. Average magnitudes only.
  2. Untrimmed analysis windows. The radio leaks its carrier for as long as it is in transmit, which starts before the samples and ends after them. Blocks with leakage but no tone fall inside an envelope-detected window and inflate the ratio. Three identical transmissions read −19.8, −25.0, and −13.5 dBc that way, and −26.0, −26.0, and −28.6 dBc once 0.25 s was trimmed inside each edge.
  3. Probing k and d in the same segments. A DC offset manufactures image-band energy that does not respond to k. gen_probe_sweep.py steps them in separate segments for this reason.
  4. Comparing separate transmissions. The absolute level wanders a dB or two between runs, so every claim here is a ratio measured inside one window, and corrected-versus-uncorrected lives in one waveform.
  5. A probe step the wrong size for the residual. The fit perturbs a residual to estimate it, so a step much larger than the residual buries it. DD = 0.03 suits the ~0.025 offset behind an untrimmed −26 dBc carrier; hold a good trim code and the residual drops 26× to ~0.0012, at which point the same step returns a d eight times too big that pushes the carrier 11.5 dB up. calibrate.py now prints the ratio and names the --dd value to use.
  6. Mistaking the noise floor for a null. A −57 dBc carrier in a 25-code grid turned out to be a weak segment, not a deep null: the tone varied 14.6 dB across that transmission. sweep_txcal.py measures empty spectrum 125 kHz off as a floor column and flags any carrier within 3 dB of it. The floor runs −68 to −76 dBc at a usable receive level and −51 dBc at a bad one.

The fit guards against the third and fifth: it estimates |u|² independently of Re u and Im u, so their mismatch is a free verdict on whether the model fits. calibrate.py prints it as model check; a few tenths of a dB is healthy, several dB means discard the answer. It earned its keep twice — once on the probe design, once on the probe size, where it read +4.6 dB and the answer was indeed wrong.

What the hardware already does

The modem trims its own LO leakage — RG_TXCI and RG_TXCQ, started automatically on each TRXOFF → TX transition, which is why the uncorrected figure is −26 dBc and not −10. There is a pair per radio, 0x125/0x126 for RF09 and 0x225/0x226 for RF24, and the live pair here is RF24’s: tx_test -c 1 is cariboulite_channel_hif, which maps to at86rf215_rf_channel_2400mhz, so 445.68 MHz comes out of the 2.4 GHz radio through the RFFC5072 mixer. Trim the wrong pair and nothing changes. Each value is masked to 6 bits, and at86rf215_calibrate_device() cycles the transition five times and keeps the median, which override_cal then writes back on every transmit.

patch_tx_cal.py exposes those codes as -I/-Q, and sweep_txcal.py searches them — every code inside one transmission, because the modem re-chooses its own on every transition into TX and runs are not comparable. Three things came out of that, and the first two change the story above.

−26 dBc is not the resolution limit; it is a mis-converged code. The modem chose I=35 Q=29. Holding I=35 and sweeping Q found a clean minimum at Q=34, −43.9 dBc against Q=29’s −25.9 — and the modem’s own choice was five codes off. Re-sweeping I at Q=34 gave −54.1 dBc at I=35, with −38.3 one code below and −41.9 one code above. So a single code step near the null is worth 5–16 dB, which bounds the trim at roughly −45 dBc typical; it was the automatic calibration converging badly, not six bits being too few. A/B triplicated in one transmission, 35:29 against 35:34 gave 19.3 ± 0.4 dB.

The trim is a DC control only, and that is what corrects the image claim. Across every swept code — about 45 segments, including ones where the carrier sat at −54 dBc — the image never left −35.8 … −37.5 dBc. Dropping the radiated carrier 28 dB does nothing to the image. A digital d drops it to −49 dBc with the carrier anywhere. So the offset feeding the image sits upstream of the trim, and only a correction in the samples reaches it. The mechanism that fits is a second-order product of the baseband offset with the signal, which lands at 2·f_carrier − f_tone — exactly the image frequency — but that is inference from two measurements, not something measured here.

The hardware and digital corrections are not in the same basis. Both cancel the same carrier, so they are alternatives rather than cumulative: a d measured against the untrimmed leakage, applied at a trimmed code, puts the carrier straight back. Re-measuring d at the good code fixes the image but re-radiates the carrier at 20·log₁₀(|d|/amp). The way out is to re-search the trim with that d held — sweep_txcal.py --cal kr:ki:dr:di — and the answer is not on the axis you would expect. Cancelling a nearly pure-imaginary d needed the I code moved from 35 to 36–37, with Q unchanged at 34.

Which configuration to use

Measured at 445.680 MHz, -g 24, 2.000 Msps, each row a segment of the same transmission, repeated over four runs spanning ADC occupancies of 13, 32, and 66 (plus one floor-limited run at 3, excluded):

configuration carrier image worst spur
modem’s own code, nothing applied −26.1 … −26.5 −36.2 … −36.9 −26 (stable)
modem’s own code + digital k,d −39.5 … −44.8 −39.5 … −41.1 −40 (stable)
hardware code 35:34 alone −42.5 … −53.0 −36.2 … −36.8 −36 (image-limited)
36:34 + k,d re-measured there −39.8 … −51.6 −48.4 … −50.5 −49 typical

Recommendation: the shipped configuration — the modem’s own trim plus the digital k and d — is still the right default. Its worst spur is −40 dBc and it is stable, needs no per-board search, and survives a frequency change with nothing worse than a re-measurement.

Take the last row only if you need the image down. It is worth about 9 dB typical on the worst spur and never does worse than the default, but the carrier null falls between two 6-bit codes, so which of 36:34 and 37:34 wins changes run to run — 37:34 read −60.5 dBc once and −41.6 … −43.2 in the three other runs, while 36:34 read −39.8 once and −48.9 … −51.6 otherwise. The image at −49 dBc is the robust part; the carrier depth is not. Search both codes and pick per session, or accept −48 dBc and stop.

What works, and what does not

The datapath is fixed, not finished. Measured on this bench, with the receiver held out of compression (see the warning below — the first version of these numbers was wrong):

Works. Energy appears where aimed (+62.3 dB, 76.4 σ). Phase is carried faithfully — a Morse-keyed tone decoded back off the air intact. Sample amplitude is carried linearly, to within 0.3 dB over a 19 dB sweep of digital drive:

digital amplitude radiated linear prediction
0.90 +0.0 dB +0.0 dB
0.50 −5.2 dB −5.1 dB
0.25 −11.4 dB −11.1 dB
0.10 −18.8 dB −19.1 dB

So nothing measured here excludes AM, SSB, QAM, or OFDM.

Ordinary zero-IF hygiene, correctable. Carrier leakage −26 dBc, image −36.5 dBc, both stable when the receive gain is dropped a further 14 dB, which is how you know they are the transmitter’s. Nothing else above −39 dBc. See Calibration above: pre-distortion takes the carrier down 17–19 dB and the image down 4.

-g does nothing until you patch it. tx_test.c parses the gain, range-checks it, prints it, and then calls cariboulite_radio_set_tx_power(state.radio, 0) with a hardcoded zero. Run patch_tx_gain.py on software/tx_test/tx_test.c and rebuild:

python3 patch_tx_gain.py software/tx_test/tx_test.c
scripts/build-standalone.sh tx_test

It maps -g the way the Soapy plugin does, tx_power_dbm = gain - 18. After that:

-g implied dBm radiated
6 −12 +0.0 dB
14 −4 +7.8 dB
24 +6 +14.7 dB
32 +14 +17.1 dB

The usable span is about 26 dB, not 69 — set_tx_power clamps to the channel’s real range (−14…+14 dBm S1G, −12…+14 HiF), so -g below 6 or above 32 does nothing. The top few dB compress as the PA runs out of headroom.

⚠️ Do not measure a transmitter with a receiver in compression. The first version of this section reported no amplitude control at all and −13 dBc third-order spurs. Both were artifacts of the HackRF running with its preamp on, LNA 32, and VGA 20 against a signal 68 dB above the noise. A compressed front end flattens its own response and manufactures intermodulation that keys on and off with the signal, so it survives the obvious “is this spur ours?” check. Re-measure at lower receive gain and watch ADC occupancy — target roughly half of full scale, not merely a good SNR. The sweeps in gen_amp_steps.py are the tool for this; run each one at two receive gains and keep only the numbers that do not move.

Transmit clamps sample rates silently, to the modem’s supported set:

requested actually used
1.400 Msps 1.3312 Msps
2.000 Msps 1.9969 Msps
3.000 Msps 3.9951 Msps
4.000 Msps 3.9937 Msps

3 Msps is not a rate the AT86RF215 has, and it was rounded up to 4, so the waveform played 33% fast and every frequency in it came out a third high — with no error reported. Read the rate back before computing anything from it.

Transmitting at all

Everything on this page assumes an amateur licence and a legal frequency. 445.250/445.500 MHz is used here because the ARRL 70 cm plan marks 445.000–447.000 as local option; your coordinator overrides that. 97.119 requires identification at least every ten minutes and at the end of a session, which is why the test waveform is the callsign. See the transmitting section of the course page before keying anything.

References