Trouble Sending Packets to CWOP via Cloudflare Workers

[Leo Herzog]

Hello, all.

Cloudflare has a wonderful serverless platform called Cloudflare Workers that allows you to write code without worrying about underlying hardware or software. Recently, they announced that Workers now have the ability to connect directly over TCP sockets. I want to use this feature to start sending an APRS packet to CWOP as a new station.

I have the following function in my code:

export async function sendPacket(packet, server, port) {

  console.log('Opening connection to ' + server + ':' + port);

  const socket = connect({ "hostname": server, "port": port });
  const writer = socket.writable.getWriter();
  const reader = socket.readable.getReader();
  const encoder = new TextEncoder();
 
  // Wait for server's initial message - http://www.wxqa.com/faq.html
  let initialMessage = await reader.read();
  console.log('Received from server: ', new TextDecoder().decode(initialMessage.value));

  // Send login line
  const id = packet.split('>')[0];
  const loginLine = 'user ' + id + ' pass -1 vers cwop.rest 0.1\r\n';
  console.log('Sending to server: ', loginLine);
  let encoded = encoder.encode(loginLine);
  await writer.write(encoded);

  // Wait for server's acknowledgement
  let { value, done } = await reader.read();
  console.log('Received from server: ', new TextDecoder().decode(value));

  // Send packet
  console.log('Sending to server: ', packet);
  encoded = encoder.encode(packet + '\r\n');
  await writer.write(encoded);

  writer.close();
  reader.releaseLock();
 
  let serverResponse = new TextDecoder().decode(value);
  console.log('Received from server: ', serverResponse);

  console.log('Closing connection to ' + server + ':' + port);
 
  return new Response(serverResponse, { "headers": { "Content-Type": "text/plain" } });
 
}

The log shows the following:

Opening connection to rotate.aprs.net:14580
Received from server:  # aprsc 2.1.5-g8af3cdc
Sending to server:  user K9ZOG pass -1 vers cwop.rest 0.1
Received from server:  # logresp K9ZOG unverified, server NINTH
Sending to server:  K9ZOG>APRS,TCPIP*:@181242z4246.58N/0866.22W_119/001g001t048r000p000h64b09968L162appsscriptforwarder
Received from server:  # logresp K9ZOG unverified, server NINTH
Closing connection to rotate.aprs.net:14580

This has been running every five minutes for almost a week now, but my data is still not showing up on findu, aprs.fi, cwopviewer.com, etc. Any ideas what might be going wrong?

---

Leo Herzog

IT Interaction Technologist
Hope College

[googl...@tedlum.com]

One of your problems is here:

Received from server:  # logresp K9ZOG unverified, server NINTH

Normally I would say you're using the wrong servers, but you're a HAM, so you have a choice, but you definitely can't use the ones you're trying to use the way you're trying to use them. The "private" servers you are currently addressing are for licensed HAM's only and require a passcode, which is just a very basic hash of your call sign. You're not sending one, you're sending a '-1', so the server ' NINTH ' is responding appropriately with ' K9ZOG unverified '.

So, you can do one of two things, either get a passcode and start sending it, or, switch to the public servers 'cwop.aprs.net:14580' which do accept 'unverified ' packets. The biggest difference is that your data can only be gated over RF if you're sending ' verified ' packets over the private APRS network, because only a licensed person may cause data to be sent over RF. If you don't care which APRS network you use then the public network is probably less hassle.

I can generate a hash for your call sign and send it to you privately, but it'll take me a little time to do it so I won't unless you specifically respond with a request for it. There were others who could do it but I'm not sure how many are still around or still have the tools... plus it's a very HAM-specific topic so there was always greater knowledge of it in the HAM specific communities.

All that said, and on a slightly different topic, be very careful with the "version strings", like 'appsscriptforwarder ' and ' cwop.rest 0.1 '. The APRS spec does not allow for anything even remotely close to this. I would say the parsers are 'exceptionally tolerant', but anything that deviates from the strict APRS specification will result in "undefined' behavior. The most important parser, the one on findu, uses a nightmare of a regex expression which has been tuned to deal the best it can with the reality of the data flow it sees, but it's not infallible. What trips it up most times are characters that have special meaning in the context of a weather packet, basically [.csgtrpPhblL] appearing in a toxic sequence. It's always easier to understand why something that doesn't work, doesn't work, than predict what won't work - regex is just funny that way.

According to the specification, 'appsscriptforwarder ' would be interpreted as APRS Software = 'a' and WX Unit (Station Model) = ' ppss '. Neither of those values are defined for either of those fields, and the specification in unmaintained and never defined values that would be usable today, so mostly the goal is to get the parsers to just quietly pass along the extraneous data without choking. Once you resolve the server verification dilemma you'll be able to see how the parser behaves.

Regards,

Ted

[Leo Herzog]

Thanks so much, Ted.

My problems were a combination of several things you named! Your insight was really helpful. I started sending to cwop.aprs.net instead of rotate.aprs.net (even though I am a Ham) until I got ahold of a verification passcode. I had an error in my lat/long format that I've since corrected.

I found your details about the version string very interesting! I found some implementations of APRS packet assembly logic such as this and this were adding a simple software name and/or version at the end of the packet, so I figured that was the norm 🤔

The packets are being processed successfully with appsscriptforwarder appended still, so the regex parsers (yeesh 😬) must be tolerant enough to allow for the invalid values.

---

Leo Herzog

IT Interaction Technologist
Hope College

[googl...@tedlum.com]

The APRS specification dates from the early 2000's, and as you probably know, was first and foremost a specification for doing packet radio... coupled with APRS-IS the packets can flow fairly seamlessly over the internet via. land lines in addition to RF.

Weather was but one "application layer specification" [terminology mine, not theirs] that was implemented in the APRS specification (Chapter 12). APRS is really a "meshed publish/subscribe" network [again, terminology mine, not theirs]. HAM's could publish weather data, but who would subscribe to it? That's where CWOP comes in. The MADIS system over at the NWS effectively became a subscriber to the APRS weather data, which it terms the APRSWXNET mesonet - findu is the gateway - among the may things it does - which technically subscribes to the weather data, parses it, archives it, and generates periodic batch files formatted for MADIS. findu is one, but not the only parser - I'll come back to this later.

The early HAM implementers believed that imagining 100 reporting weather stations probably constituted delusions of grandeur... as of this moment, more than 20 year on,  there are ~15,800 stations actively reporting in the last hour, and more than 40,000 station identifiers have been issued - to non-HAMs, who's station identifier is also their call sign.

So, to some extent the APRS specification became a victim of it's unexpected and overwhelming success, and in ways that certainly were never imagined. In the last hour only 1.7% of reporting stations were using the APRS private network, which is "RF capable". So you have a decades old packet radio specification being used for a predominantly non-packet radio application by an audience that is predominantly not licensed and unable to contribute to the original core purpose of the specification. Consequently, little was ever done to update the original specification (de jure) beyond the initial use cases of the target audience, which defines the "APRS Software Type" field as:

d = APRSdos
M = MacAPRS
P = pocketAPRS
S = APRS+SA
W = WinAPRS
X = X-APRS (Linux)

and defines the Weather Unit Type as:

a 2–4 character code uuuu for the type of weather station unit. The following codes have been allocated:
Dvs = Davis
HKT = Heathkit
PIC = PIC device
RSW = Radio Shack
U-II = Original Ultimeter U-II (auto mode)
U2R = Original Ultimeter U-II (remote mode)
U2k = Ultimeter 500/2000
U2kr = Remote Ultimeter logger
U5 = Ultimeter 500
Upkm = Remote Ultimeter packet mode
Users may specify any other 2–4 character code for devices not in this list.

So, you'll see two different de facto standards in use. In the first case people are sticking to the official schema of 'tuuuu'. Davis, for example uses 'eDvs' - they use a Type code not in the list, but constrain it to the single character. The other de facto standard is simply whatever long string the findu parser doesn't choke on. Because the "Weather Unit Type" was defined as a 2-4 character code, that allows someone to implement a new code of their choosing as necessary, so the specification can accommodate new station hardware, however, no such allowance was made for "APRS Software Type", so using anything not is the list is "off book" as it were. Whether you can make up new codes or not is a fairly moot point, though.

So, going back to parsers and a meshed publish/subscribe network, specifications are an absolute necessity if multiple arbitrary subscribers are to interpret what's being published. Even if a specification allows a publisher to make up a value for a field, downstream subscribers have no way of knowing how to interpret made up values whether the specification allowed them to or not, but then there is making up codes within the established schema and there is going totally off-book and violating the schema all together. As a publisher you don't necessarily know who your consumers may be, so you also don't necessarily understand the breadth of the effect of what you're publishing. In this case, findu is one obvious consumer, but it's not the only one. For example, this is Philip Gladstone trying to make sense out of these fields, which is behind the " Station type/software: " field on the WXQC site page

https://weather.gladstonefamily.net/cgi-bin/wxequip.pl

which is a snapshot of just about everything that just about everyone is throwing out there. If you do make something up then it'll land in an "Unknown" bucket, unless and until someone decides to create a rule to categorize it... which isn't likely to happen if when there is a high rate of unique one-off values showing up.

Also, there are certain non-obvious implementation details to know about:

While the packets contain a timestamp findu doesn't use it for MADIS, findu uses the ephemeral timestamp from the APRS-IS server, which is lost when the packets are saved to a file, for example. If you try to replay the data from an archive file, and the APRS packet timestamp differs from the APRS-IS transport timestamp, you'll get a different result.

While the APRS packet MAY contain an optional APRS "position" with optional amounts of resolution, MADIS uses the coordinates that were registered when the ID was created. If you try to replay data from an archive file, and the APRS "position" is different or not present, you'll get a different result.

While the APRS specification allows for "position-less" reports, position-less packets will often be excluded from archive files because the packet is not worth the space it would occupy. What the APRS specification ALLOWS is not necessarily useful in certain contexts. So, even things that are legal don't necessarily work.

For example: http://www.wxqa.com/lum_search.htm

What I'm saying is that in an environment with multiple anonymous consumers right and wrong boils down to what comes closest to making all of them happy, and context matters because some have access to things that others don't.

Regards,

Ted

[Denver]

Just want to say thank you to Ted.  This was very informative, and good background information even while my station is operating without issues.

Denver