Time to start a custom protocol

There have been a number of discussion of custom protocols, or a "deviation" protocol. Well, I find myself needing one for my current project. So it's time to start another discussion.

I already know roughly what I want the protocol to look like, but that's for later. For now, I just want to investigate existing solutions. So I've got questions for the protocol gurus, I think from easiest to hardest to answer.

First, the RF module. I'm tempted to pick on the NRF24L01, because it's cheap, popular and well documented. Anyone think I should use a different module, considering I'm going to require an ACK and would like to do telemetry later? If so, which one and why, please.

Second, want to suggest an existing protocol for whatever module they think I should target to either start with, or use as a model for doing things the right way?

Finally, I think the sticky one. Last time I looked closely, protocols came in three basic flavors. Legacy protocols, that were basically digital versions of analog radios, sending N identical channels every packet.

The robotics/maker crowd uses what I think of as a "command" protocol. They send a command - indicating what to change - along with any parameters when things change, but are generally quite otherwise.

Finally, there are hacks that sort of cram the latter into the former. WLToys mods to FlySky are the example I can think of, where pushing a button turns on a bit in another channel.

Do we have protocols that don't fit into one of those categories somehow? If so, can you either tell me which one(s), or summarize how they work?

Thanks,
MIike

I think that the way we send a digital switches using bandwith of a full channel we should stop doing so.
Also for rotary pots aux channels possibly the priority or bandwith may be lower.

Further I think that for first 4 channel, we should have maximum update rate accuracy and minimum latency, 200hz?

So I would vote for a kind of compressed data stream rather than fixed fields.

Or a combination, so fixed fields for first for channels, and some open area to be used for any kind of data commands. For switches and aux and or control commands to enable settings in esc or flight comtrolers via serial line direct.

It may even be some way of configurable, so setup low datarate low speed, or high datarate.
So for
Long range auv plane may choose low speed high range, but for circuit racing or 3d heli, may choose speed but less range?

Maybe also of interest to check with inav developers, the is alos some open protocol and or middelic.

On the chipset, I think the one with best and most relaible range and bandwith may be preferred.
Possibly cc2500, there are lots of rx who can be flashed.

The latest developments is now rx direct in flight controller, a cc2500 direct over SPI connected to the cpu. The RX code is then part of Betaflight and flashed . (No more sbus, ppm etc) Check midelic and or betaflight.

Would be great if Walkera would build a new TX with a 2in1 module to maximize the link stability and still comply with EU legislation.
1 chip for the Devo legacy stuff and 1 chip that allows 100 mW transmit power. Need of LBT depends on band utilization time but would be great if it could be enabled when needed for future ETSI revisions. Ii wonder what Walkera's next modulation hardware will be like.
I like the mentioned idea regarding bandwidth saving in switches but this would not work with values different from -1/0/+1 e.g. for gyro sensitivity,
I don't want to criticize, I appreciate your efforts and you will do it right.

My only use for a Tx is to control intelligent, very configurable flight controllers. In that case, there is really very little to no need for any configuration on the Tx side. In general, the flight controller firmware can configure everything that you can configure on the Tx.

I've often thought that it would be great to just have the Tx send the state of all of the controls to the flight controller, and let the flight controller sort out which one does what. Of course, that would require a compressed protocol to make it practical.

In any case, I think you're looking in the right direction, which is to make a more intelligent protocol and get us as far away from the PWM legacy as we can get and move into the modern world.

Answers depend on your intended use I think. My comments are from the perspective of acro quads and (unskilled) 3d heli.

The NRF has the advantage of low-cost and ease of use. Consider the cyrf because the protocol wouldn't require any hardware mods. Doesn't requiring an ACK limit range since the receiver typically will have lower transmit power.

I think FHSS with a decent number of channels is a requirement, preferably with LBT. I'm not a radio engineer, but it seems to me FHSS wouild be easier with synchronous packet transmission instead of transmit on demand. If you're sending/listening at regular intervals it makes sense to dedicate some number of bits to the TAER channels. Pilots seem to always want the absolute minimum control delay so those ought to be at the max rate the radio can manage. Plus you don't want these channels delayed in a packetized protocol by something like a long data transfer to the receiver. Some dedicated bits would be nice for low-latency switch response.

Take a look at FrskyX. Each transmit packet has some fixed protocol header bytes, 8 channels of data of 12 bits each, 6/8 bytes for a packet data stream. The receive packet has a header, 1 channel of telemetry data of 8 bits, and 8 bytes for packet data stream. The packet data carries the s.port telemetry data. It can also be used for bidirectional serial data (not yet implemented in deviation). There's a couple design choices that would be unnecessary if the packet data was also used to send commands to the receiver (failsafe settings are multiplexed on the channels and use magic channel values, 16 proportional channels are achieved by multiplexing).

CC2500 is a winner. The FrSky protocols are the only ones with telemetry. But it's a solid protocol, and FrSkyX has an EU legal variant to play with. Last time I looked, the selection of boards it's on wasn't as good as the NRF24L01, but it was as broad. And it can be breadboarded, where the NRF chip can't. I'm sold, barring finding a problem when I read the data sheet.

I agree completely about PWM and combined versions of it. I'd already written the full text of the section on converting the data to those formats: "Don't".

The target control platform is indeed a smart control board talking to an RF chip via SPI, or possibly some kind of combined MCU+RF module. But I also want to make using satellite Rx's simple, so the resulting protocol should also work as a serial line protocol, in lieu of something like SBUS or SRXL. We're also going to assume everything but humans will get bigger, faster and wider. So hard limits will be as high as the RF module will let us go, unless the limits are due to human physiology. You should never feel the need to change protocols because you can't get enough of something, hence the name Z Protocol, or ZP for short.

While sending the control settings to the onboard CPU and letting it sort things out is a perfectly reasonable goal, the protocol needs to be flexible enough to support any kind of craft. A race drone needs 4 analog channels and a few switches. A flying camera with a 3-axis gimbal needs 7 analog channels and a bunch of switches. The AMYA has classes that specifically restrict the radio to controlling two functions, which translates to two analog channels. So the transmitter - and protocol - has to be flexible enough to deal with that, even if you don't want the transmitter doing anything.

There's going to be a lot of complexity and flexibility in the protocol. I'd still like the implementation to be simple. So there's minimal bit-packing. Instead of smushing 2 12-bit analog values into 3 bytes in order to get 8 channels in 12 bytes, almost all values will be full bytes. This also means negotiated values will prefer ranges of 0-255 or 1-256.

Most of this has been talked about in some of those other discussion, which it looks like Fernandez has read.

Motivation
Smart control systems - the thing that makes a drone a drone - are basically robots. But conventional RC protocols aren't very good for sending commands to a robot. You typically want to trigger an action of some kind, not have real-time control over the position of some control surface. This can be seen in the protocols used by robots, which usually are commands with an object and verb, like "Close left gripper" or "both motors on full". Updating a 10 or 12 bit value at 50Hz or faster just to start a motor is very wasteful.

On the other hand, those robot protocols aren't very good for controlling an aircraft in flight. 50hz is a minimal acceptable update frequency, and we're always pushing for more resolution on the channels.

Z protocol is designed to meet both of those requirements by taking advantage of the capabilities of the 32-bit processors found in modern hobby-grade transmitters and control systems.

Channel types
There are three channel types, with two widths and two update behaviors. And yes, there is an update/width combination that isn't used.

Analog channels have widths measured in bytes. Cheap transmitters can use 8-bit analog channels, most modern protocols will use 2-byte analog channels. Yes, that wastes a few bits, but we get it back with switch channels.

Switch channels have widths measured in bits. They will fit in a byte, so are at most 8 bits, but can be less. So while using a 2-byte analog channel for a 10-bit value wastes 6 bits, using a switch channel instead of an analog channel for a switch saves 8 bits, as well as the complexity of packing and unpacking bits for every value.

An analog channel can be a continuous channel. These are updated like conventional radio channels: all the continuous channels get sent in every packet.

A switch channel is a command channel. An analog channel can be a command channel. Command channels are sent after the continuous channels, but only after they change. They are sent in every packet until a packet containing their current value is ack'ed by the receiver.

The missing type is continuous switch channels. Those can be sent in a single byte as a command, so you don't save bandwidth by making them continuous channels. And if you have a switch that you're toggling at 50 Hz, I'd like to know what you're doing with it!

A note on acks: remote controlling a craft is essentially a real-time activity. If the remote fails to get a packet, there's no point in sending the missed packet again - the values are almost certainly out of date. Just send the new values. So acks aren't used to improve reliability. They're used to improve throughput by only sending redundant data until the recipient acknowledges receiving it.

Negotiated values
In order to be flexible enough to meet the needs of a variety of craft - from flying robots with gimbal-mounted cameras and manipulators of some kind down to model yachts limited to two channels by function, there needs to be a lot of flexibility in what a connection looks like. This is a two-step process. The transmitter sends a packet describing what it wants. The receiver sends back a packet describing what it's willing to provide, and then waits for data. The transmitter can either accept the values from the receiver and start sending data, or fail to connect.

The negotiated values are:

a) Analog channel width, in bytes, from 1 to 256 (yes, that's 4K bits of channel resolution should you need it. Welcome to Z protocol!).

b) Continuous analog channel count, from 0 to 255.

c) Switch channel width, in bits, from 0 to 8.

d) Switch channel count, from 0 to 256, but constrained by the switch channel width.

There's also an analog command channel count. It can be derived from the two switch channel values. If the the switch channel width is 0, then the switch channel count is 0, and the analog command channel count is 255. If the switch channel width is 8, then the switch channel count is between 0 and 255, and the analog channel count is 255 minus the switch channel count.

Otherwise, the *switch channel max* is the switch channel count times 2 raised to the switch channel width. That value must be at most 256. The analog command channel count is 256 minus that value.

For convenience, we define the switch channel max to be the switch channel count if the switch channel width is 8.

In C, that would be:
Some examples to illustrate:

If the switch channel count is 0, then the switch channel width doesn't matter, as the switch channel max will be 0, and there will be 256 analog command channels.

If the switch channel width is 7, then the switch channel count is either 1 or 2, giving a switch channel max of either 128 or 256, and either 128 or 256 analog command channels.

If the switch channel width is 4, then the switch channel count is at most 16. If it's 10, then the switch channel max is 160, and there are 96 analog command channels.

I believe that 256 command functions is well beyond any number that a human being will be able to use. For that matter, it's liable to be more command channels than any single controlled device needs, unless you start counting complex groups of devices as "a device".

Packet format
Ignoring the ceremony, a packet will consist of the analog channel count of analog channel data, pretty much like a conventional RC radio packet.. Following that will be an arbitrary number of command channels.

A command channel is either single byte for a switch of width less than 8, two bytes for an 8 bit switch, or the analog channel width bytes plus one. The first byte can be used to figure out which case you have.

If the value of the byte is greater than or equal to the switch channel max, then the byte is an analog command channel number, and is followed by analog channel count bytes holding the new value for that channel.

Otherwise, if switch channel width is 8, then the byte is a switch channel number, and is followed by a byte containing the new value for that channel.

Finally, the byte is a switch channel number and value or'ed together. The low switch channel width bits are the value, and the bits above that are the channel number.

So, looking at those examples again:

If switch channel count is 0, switch channel max is 0, and all the command channels are analog command channels.

If switch channel width is 7 and switch channel count is 1, then if the high order bit is on it's an analog command channel. Otherwise, the low order 7 bits are the value for the switch channel 0.

Finally, if the switch channel width is 4 and the switch channel count is 10, then a byte value between 160 and 255 would be an analog command channel number. A value lower than 160 would be have a channel value in the lower 4 bits (nibble), and a channel number in the upper nibble.

Note that the switch channel values can be extracted from the byte by two operations: mask off the upper bits, then shift the upper bits right logically. While some of the upper bits may not be part of the switch channel bits, those bits will all be zero.

I am not a programmer, so I have no idea about excisting technologies.

But what if we would use a fixed nr of "transmission slots" The slots are a repeating sequence, f.i. Every 2 seconds all repeat.

Then a table defines sequence what is send in each slot? The fill in of slots can be depending on the user. You can fill many slots of the 2 sec sequence to use channel 1 - 4 few for switches and some for aux etc or as you like. So you can have different refresh per function.
That needs than a clever tool to "automate and create easily those tables without risks."

The table can be setup per model.


Another thing what came to mind and may be of interest is possibility to boot in a high speed, low range data mode (or bluetooth mode) or what so ever, that way it could be possible to flash receivers over the air, upgrade flight controllers, transfer
Data.

Just brainstorming here......

Your fixed slots are the "continuous channels" in zp. They are negotiated at connection time.

Flashing over the air is an Rx feature, not a protocol feature, so beyond the scope of this project.

Ok, I've got the basics for how I want things to happen down, and I'm waiting on parts (CC2500's aren't breadboardable - forgot they were 1 or 2mm pitch instead of .1 in. Adapters will be here in a couple of days). So now I'm working on telemetry.

Robotics people tend not to do telemetry. I know some of the drone guys want telemetry in the OSD, but not all of them. Personally, I want telemetry recorded in my Tx so I can do analysis/review/find my craft with GPS.

RC telemetry systems tend to have all the math (units & limits) wired into the protocol. That won't do at all. So conversion factors - which com from the hardware - needs to be sent from the Rx to the Tx. Given that, maybe we should negotiate the telemetry: The Rx sends a list of available telemetry values along with conversion info, and the Tx sends back a list of the ones it wants. The user can be shown a selection of telemetry values like "Voltage to 24, current 23; voltage to 7; currently 5.5, Meters to 2000, currently 0, ..." and pick the ones they want. As a bonus, the mechanism for conversion can also be applied to the analog sticks, allowing those of you who want the flight controller to do all the work to get not merely control positions, but raw ADC outputs.

So, anyone have any thoughts about what they'd like?

This kind of setup I think will be the future and could be of interest for new test;
www.rcgroups.com/forums/showthread.php?2...ght-controller-board

So on one hand Deviation TX SPI -"Your new Open Protocol" - SPI Betaflight/Inav etc. So no more RX to flash....No more pwm sbus etc.
We need to wire cc2500 direct over spi to flightcontroller.

This thread was opened some time, but never really came to conclusion.....
www.rcgroups.com/forums/showthread.php?2661163-Open-RC-Protocol

The board is definitely in the target space I'm aiming at.

Midelic's protocol is more RC and/or aircraft centric (24 channel limit?) and doesn't deal with the robotic stuff they way I'd like to. But the thread had a lot of interesting feedback on it.

Thanks.

Integrating the modem on the FC has no bearing on the protocol. The only difference is it reduces the latency slightly because it has one less hop. In the end it's still a serial protocol going over the air/wire.

I've been arguing for years to ditch the traditional RC protocols and just treat the RF link as a raw serial link where the Tx talks to the FC. We've been doing that for years with Revolution/Sparky2/OPLink with the OpenPilot derived projects. The RF hardware should be transparent and interchangeable and have no relationship to the protocol.

The Z Protocol is design specifically ignores the underlying hardware, other than to require that it be relatively modern: no analog BS, and bit rates measured in hundreds of thousands/second. So it could be implemented on any hardware layer, but that doesn't mean the end user gets to ignore the hardware. The Rx & Tx hardware layers have to be able to talk to each other.

Since reliable connections with the associated retries and delays are a bad idea if you have any kind of real time constraint, it's packet-based instead of simple serial protocol.

Yes indeed oplink was as such.

but a pity that the uhf module en protocol was never supported by deviation. Same for the openlrs en ulrs so you still meed ise of ppm.....

Btw might be slightly offtopic.
I follow the diy development, the use of Lora RF modules.
They have very long range. It seems to be technology used in crossfire en also the new frsky lrs?

www.rcgroups.com/forums/showthread.php?2...-LINK-with-telemetry

Especially lowest datarate seems important here for longest range. It conflicts with need for highest speeds for drone racing.

So for open rc protocol, could it be send over whatever module or telemetry link? As long as the rf module and its registers etc is correct handled enabled and link connection is setup correct?

How about some way of compression etc or to get the "unisefull data out of the over air link" such as only transmit changes ?

Even serial connections pretty much have to be packetized, otherwise you just have a string of bits/bytes and don't know where it starts or stops. Of course, ideally the protocol packets would align with the RF packets, but that's not an absolute requirement, as long as you have some sort of error detection/recovery.

"hundreds of thousands/second"? Even at 2.4GHz, that takes up alot of bandwidth, and pretty much eliminates the possibility of LRS. Since you want the protocol to be negotiable, it would be nice to have it bandwidth scale-able as well, at least down to 100k/sec or so. Us old timers flew just fine on 72MHz PWM, and you can cram that into 9.6kbps.

A bit of an old thread, but we see more and more latency discussion appearing. Also new flight controllers are entering the market with onboard SPI CC2500, such as:

www.mateksys.com/?portfolio=f411-one

Deviation may stand out from the Frsky transmitter approach. With offering a full open developed link, which could offer the best for the application. (such as inav/betaflight) and deviation.