Sending raw frames (custom start code) over OLA? (Enttec)

[sdbbs]

I would like to experiment with "raw" DMX-formatted frames, by which I mean DMX-formatted frames with a custom start code (something other than 0x00 for DMX or 0xCC for RDM) - and I would like to do this on Enttec DMX USB PRO attached to a Raspberry Pi running OLA.

The Enttec API documentation https://cdn.enttec.com/pdf/assets/70304/70304_DMX_USB_PRO_API.pdf mentions for:

• "Output Only Send DMX Packet Request (Label=6)"

Size In Bytes: 25 to 513 Description: DMX data to send, beginning with the start code. The overall message size specifies the size of the DMX data to send, ...

• "Send RDM Packet Request (Label=7)"

Size In Bytes: 1 to 513 Description: RDM data to send, beginning with the start code.

So, if I used the Enttec API directly (either via usbserial+ftdi_sio Linux kernel drivers, or using the FTDI proprietary libftd2xx.so driver (conflicts with ftdi_sio)), I could use either of the above commands to send a DMX-formatted packet with a custom start code from the Enttec.

However, even if I want to use the Enttec API directly in a program - if I want to use that program on a Raspberry Pi with an attached Enttec and running OLA (olad) which manages the Enttec (i.e. has the Enttec as a OLA Universe output or input port), I'd assume that olad would in that case take ownership of the device over the kernel drivers, and thus my application would be refused device access - at least until I shut down olad, which I'd rather not.

So, I would like to make a program that uses OLA (that is, olad) to send raw/custom frames over the Enttec - but I cannot find such an OLA API.

First, I took a look at https://docs.openlighting.org/ola/doc/latest/client_tutorial.html - there are no examples there with sending RDM, only DMX, and the API used is ola::client::OlaClient::SendDmx or (probably) ola::client::StreamingClient::SendDmx - both of those use ola::DmxBuffer Class, where it is mentioned:

dmx_buffer.SetFromString("0,1,2,3,4")

The above code would set channels 1 through 5 to 0,1,2,3,4 respectively,

... meaning, there is no way to manipulate the start code of DmxBuffer (which makes sense, after all, given DMX is in the name of this class).

As far as RDM goes, I tried analyzing ola_rdm_set in a debug build, and eventually arrived at this stack trace:

$ LD_PRELOAD="/path/to/ola/common/.libs/libolacommon.so.0.0.0 /path/to/ola/./ola/.libs/libola.so.1.0.1" gdb --args examples/.libs/ola_rdm_set --frames -u 1 --uid 1000:12345678 device_label "My Lamp"
...
(gdb) b main
Breakpoint 1 at 0x12c80: file examples/ola-rdm.cpp, line 588.
(gdb) r
Starting program:  ...
...
Breakpoint 1, main (argc=8, argv=0x7efff134) at examples/ola-rdm.cpp:588
588     int main(int argc, char *argv[]) {
(gdb) b common/io/Descriptor.cpp:350
Breakpoint 2 at 0x76ec9030: file common/io/Descriptor.cpp, line 350.
(gdb) b common/io/Descriptor.cpp:353
Breakpoint 3 at 0x76ec9044: file common/io/Descriptor.cpp, line 353.
(gdb) c
Continuing.

Breakpoint 2, ola::io::ConnectedDescriptor::Send (this=0x31ff0, buffer=0x49040 "0", size=52) at common/io/Descriptor.cpp:350
350         bytes_sent = send(WriteDescriptor(), buffer, size, MSG_NOSIGNAL);
(gdb) bt
#0  ola::io::ConnectedDescriptor::Send (this=0x31ff0, buffer=0x49040 "0", size=52) at common/io/Descriptor.cpp:350
#1  0x76f4e700 in ola::rpc::RpcChannel::SendMsg (this=this@entry=0x327b8, msg=msg@entry=0x7effe564)
    at common/rpc/RpcChannel.cpp:305
#2  0x76f4f728 in ola::rpc::RpcChannel::CallMethod (this=this@entry=0x327b8, method=<optimized out>,
    controller=controller@entry=0x36b30,
    request=0x76f4f728 <ola::rpc::RpcChannel::CallMethod(google::protobuf::MethodDescriptor const*, ola::rpc::RpcController*, google::protobuf::Message const*, google::protobuf::Message*, ola::SingleUseCallback0<void>*)+504>, request@entry=0x7effe8b4,
    reply=reply@entry=0x32848, done=done@entry=0x37fd0) at common/rpc/RpcChannel.cpp:232
#3  0x76f97540 in ola::proto::OlaServerService_Stub::RDMCommand (this=<optimized out>, controller=controller@entry=0x36b30,
    request=request@entry=0x7effe8b4, response=0x32848, response@entry=0x76e59a84, done=0x37fd0)
    at common/protocol/OlaService.pb.cpp:773
#4  0x76e23110 in ola::client::OlaClientCore::SendRDMCommand (this=0x33278, is_set=is_set@entry=true, universe=<optimized out>,
    uid=..., sub_device=0, pid=pid@entry=130, data=data@entry=0x63bd0 "My Lampv\310;\006", data_length=7, data_length@entry=130,
    args=...) at ola/OlaClientCore.cpp:1013
#5  0x76e23840 in ola::client::OlaClientCore::RDMSet (this=<optimized out>, universe=<optimized out>, uid=...,
    sub_device=<optimized out>, pid=130, data=0x63bd0 "My Lampv\310;\006", data_length=7, args=...) at ola/OlaClientCore.cpp:588
#6  0x76e1ab50 in ola::client::OlaClient::RDMSet (this=<optimized out>, universe=<optimized out>, uid=...,
    sub_device=<optimized out>, pid=130, data=data@entry=0x63bd0 "My Lampv\310;\006", data_length=7, args=...)
    at ola/OlaClient.cpp:204
#7  0x000141b0 in RDMController::PerformRequestAndWait (this=this@entry=0x7effec9c, universe=163928, uid=...,
    sub_device=<optimized out>, pid_name="device_label", is_set=true, inputs=std::vector of length 1, capacity 1 = {...})
    at examples/ola-rdm.cpp:494
#8  0x00012f48 in main (argc=<optimized out>, argv=<optimized out>) at examples/ola-rdm.cpp:638

(gdb) p size
$1 = 52
(gdb) p/x buffer[0]@52
$2 = {0x30, 0x0, 0x0, 0x10, 0x8, 0x1, 0x10, 0x0, 0x1a, 0xa, 0x52, 0x44, 0x4d, 0x43, 0x6f, 0x6d, 0x6d, 0x61, 0x6e, 0x64, 0x22, 0x1e, 0x8, 0x1, 0x12, 0x8, 0x8, 0xa1, 0x42, 0x15, 0x66, 0x33, 0x22, 0x88, 0x18, 0x0, 0x20, 0x82, 0x1, 0x2a, 0x7, 0x4d, 0x79, 0x20, 0x4c, 0x61, 0x6d, 0x70, 0x30, 0x1, 0x38, 0x1}

So: in the ola_rdm_set itself, we first start with RDMController::PerformRequestAndWait, which then calls OlaClient::RDMSet](ola::client::OlaClient::RDMSet - and then in a rather complicated descent through the OLA API, I guess this gets turned into a Remote Procedure Call serialized using protobuf, and the final buffer resulting from that is used in a C send to write to a socket, which I guess then propagates this data to olad over the network. Inspecting the final buffer reveals there is no byte 0xCC to be found as a start code, which means that buffer does not contain a representation of an RDM frame, but a representation of the RPC call - and it is olad, after that call/command is received, that decides to build an actual RDM frame with start code 0xCC, and pass it to/send it over the device attached to the requested universe (Enttec in my case). Again, I see no way to change the RDM start code from 0xCC to something else in this case, either.

While I understand that other hardware may behave differently, at least the Enttec DMX USB PRO seems to allow for a custom start code. So is there any OLA API that would allow me to use it? That is, is there an OLA API function that would allow me to specify the full (DMX or RDM) frame on the client side - including the start code - which would also be recognized by olad, and propagated in full (including the custom start code) to the Enttec device?

[sdbbs]

Ok, did some experiments, and it turns out this:

... I'd assume that olad would in that case take ownership of the device over the kernel drivers, and thus my application would be refused device access ...

... is in fact not true, at least on Raspberry Pi 3B+ stretch - in fact, even with running OLA (olad), I could build an example ENTTEC packet with label=7 (Send RDM ...) with custom start code (here 0xBB) like this:

echo "0x7E 0x07 0x08 0x00 0xBB 0x01 0x01 0x01 0x01 0x01 0x01 0x01 0xE7" | xxd -r -p - outrdm.bin

... and then can write this to the USB/serial device node of the Enttec like this:

sudo bash -c 'cat outrdm.bin > /dev/ttyUSB0'

Checked with signal analyzer, this sends a single packet with start cobe 0xBB on wire; if you use label=6 (Output Only Send DMX...) instead, it will start a stream (i.e. packets with custom start code will be output continuously).

Btw, is it possible to somehow get the corresponding device path of ENTTEC (for me, /det/ttyUSB0) from OLA? I know ola_dev_info output something like "Device 2: Enttec Usb Pro Device .... port 0, OUT, patched to universe 1, RDM supported", but there is no /det/ttyUSB0 device path there (I have to look for startup messages in syslog like "olad: plugins/usbpro/WidgetDetectorThread.cpp:215: Found potential USB Serial device at /dev/ttyUSB0" to find the device path).

[peternewman]

What protocol are you actually trying to send @sdbbs ? Unless you know the details of the Martin protocol, then 0xBB seems unlikely:
https://tsp.esta.org/tsp/working_groups/CP/DMXAlternateCodes.php

As you've sort of found out, everything goes via Protobuf. The supported commands are here:

ola/common/protocol/Ola.proto

Lines 401 to 429 in e8c755c

	service OlaServerService {
	rpc GetPlugins (PluginListRequest) returns (PluginListReply);
	rpc ReloadPlugins (PluginReloadRequest) returns (Ack);
	rpc GetPluginDescription (PluginDescriptionRequest) returns
	(PluginDescriptionReply);
	rpc GetPluginState (PluginStateRequest) returns (PluginStateReply);
	rpc GetDeviceInfo (DeviceInfoRequest) returns (DeviceInfoReply);
	rpc GetCandidatePorts (OptionalUniverseRequest) returns (DeviceInfoReply);
	rpc ConfigureDevice (DeviceConfigRequest) returns (DeviceConfigReply);
	rpc SetPluginState (PluginStateChangeRequest) returns (Ack);
	rpc SetPortPriority (PortPriorityRequest) returns (Ack);
	rpc GetUniverseInfo (OptionalUniverseRequest) returns (UniverseInfoReply);
	rpc SetUniverseName (UniverseNameRequest) returns (Ack);
	rpc SetMergeMode (MergeModeRequest) returns (Ack);
	rpc PatchPort (PatchPortRequest) returns (Ack);
	rpc RegisterForDmx (RegisterDmxRequest) returns (Ack);
	rpc UpdateDmxData (DmxData) returns (Ack);
	rpc GetDmx (UniverseRequest) returns (DmxData);
	rpc GetUIDs (UniverseRequest) returns (UIDListReply);
	rpc ForceDiscovery (DiscoveryRequest) returns (UIDListReply);
	rpc SetSourceUID (UID) returns (Ack);
	rpc RDMCommand (RDMRequest) returns (RDMResponse);
	rpc RDMDiscoveryCommand (RDMDiscoveryRequest) returns (RDMResponse);
	rpc StreamDmxData (DmxData) returns (STREAMING_NO_RESPONSE);
	// timecode
	rpc SendTimeCode(TimeCode) returns (Ack);
	}

We could add a send custom ASC RPC, but if you're trying to do say a text packet, or a test packet, it might be easier for us to have a higher level implementation that makes that more user-friendly. But maybe we add the low level one too for other protocols.

FWIW ETC already have a tool to decode some of that sort of stuff if you've got the right interface (but it wouldn't be too hard to add them to our sniffers too):
https://github.com/ETCLabs/ETCDmxTool

Btw, is it possible to somehow get the corresponding device path of ENTTEC (for me, /det/ttyUSB0) from OLA? I know ola_dev_info output something like "Device 2: Enttec Usb Pro Device .... port 0, OUT, patched to universe 1, RDM supported", but there is no /det/ttyUSB0 device path there (I have to look for startup messages in syslog like "olad: plugins/usbpro/WidgetDetectorThread.cpp:215: Found potential USB Serial device at /dev/ttyUSB0" to find the device path).

Not if you can't currently see it on ola_dev_info, and if you could it would probably still be down to parsing a string that might change. Do you want to open a new issue for this as maybe we should extend our device info stuff to have some sort of path (be that on the device or as an IP) as a specific separate piece of info:

ola/common/protocol/Ola.proto

Lines 221 to 228 in e8c755c

	message DeviceInfo {
	required int32 device_alias = 1;
	required int32 plugin_id = 2;
	required string device_name = 3;
	repeated PortInfo input_port = 4;
	repeated PortInfo output_port = 5;
	required string device_id = 6;
	}

[sdbbs]

Many thanks @peternewman :

What protocol are you actually trying to send @sdbbs ? Unless you know the details of the Martin protocol, then 0xBB seems unlikely:

The 0xBB was just an example, I had no specific intent with that example other than to see bytes reproduced on signal analyzer capture. Otherwise, there is some vintage gear I have access to which uses custom start codes, and I'd like to try interfacing with it from the same Raspberry Pi OLA is running on.

Also, thanks for the link:

https://tsp.esta.org/tsp/working_groups/CP/DMXAlternateCodes.php

I was not aware of this:

Manufacturers are encouraged to obtain a two-byte Manufacturer's ID and to use it with Alternate START Code 91h to create a proprietary message in lieu of using a proprietary Alternate START Code.

... probably because I'd mostly been reading ANSI E1.11-2004.

We could add a send custom ASC RPC, but if you're trying to do say a text packet, or a test packet, it might be easier for us to have a higher level implementation that makes that more user-friendly. But maybe we add the low level one too for other protocols.

I understand the user-friendliness approach, but it sure would be nice to have possibility to write a fully custom packet via OLA as well, especially for prototype equipment, where it might not be worth the effort to implement the protocol in OLA itself.

Do you want to open a new issue for this as maybe we should extend our device info stuff to have some sort of path

Sure thing, here it is: #2002