Might've found the LED freezing bug: our LEDs should freeze on Oct 3 0422Z

[randomshi...@gmail.com]

I'd had enough of my PiDP11's LED lights freezing after a few weeks, so I pointed Qwen3.8-27B at it. I also gave it a companion, GPT-6-Sol to talk to. The latter mentioned this defect just in passing... and I told them to focus on this.

Qwen takes the stage:

The 32-bit ms epoch wrap. Confirmed in the code:
- `sim_os_msec()` returns **`uint32`** — 32-bit ms, wraps every 49.71 days
  (2³² ms) — `src/02.3_simh/4.x+realcons/src/sim_timer.c:136`.
- `service_next_time_msec` / `service_cur_time_msec` are **`t_uint64`**
  (64-bit) — `src/.../REALCONS/realcons.h:216,218`.
- `realcons.c:22`  `if (service_next_time_msec >= service_cur_time_msec) return;`
  — **64-bit** comparison.
- `realcons.c:71-72`  `service_next_time_msec = sim_os_msec() + interval`
  — **32-bit** addition (both operands 32-bit) assigned into the 64-bit field.

At the wrap: the deadline is a large 64-bit value near 2³² (the 32-bit add
did not overflow), while the current time has wrapped to a small number.
`deadline >= current` is therefore true for ~49.71 days, so
`realcons_service()` skips every cycle. The panel is never updated → LEDs
freeze on the last pattern. No RPC error, no disconnect, IDLED keeps
running, the UDP socket stays open. Matches **every** field symptom
(frozen not off, no message, simh responsive, restart cures, after weeks).

Trigger is **conditional**, not guaranteed every wrap: the CPU (IDLED in
`sim_idle`) must be asleep such that the last service before the wrap sets a
non-overflowed deadline and the wrap falls in the gap before the next
service. IDLED is almost always idle, so it triggers at most wraps.

So if all this is correct, we'll just have to wait a week to see if our LEDs froze.

As much as I like coding, I wasn't going to delve into the weeds for this. This is going to be the best use of AI, hardening hobby projects like this. Somebody has already made an XHCI driver for Windows 9x, maybe one day I'll even get those models porting and updating Nekoware for IRIX one day.

[wjegr...@gmail.com]

This has been an issue in one form or another since the first release. I fixed the first fault, a classic threading issue where a buffer ptr was being checked with no guard, it would eventually get a bad value and die. Now this, and I think there was at least one other problem fixed.

Might be a good time to unleash AI on it for a rewrite. I've had amazing success with Claude Code Opus over in my pidp1-mods project.

But one note, my -11 has been up for over a year 24/7 without a problem having only my threading fix. Did I miss  an update that introduced this?

Bill

[Eric N]

I managed to permanently fix the hanging LED issue a while back, and the epoch wrap in realcons.c was one of the spots I changed.  I had other more frequent hangs earlier but that one looked to be the cause of the 49-day freeze.  There are some other sections where I added mutexes around some calls as well that seemed to fix some of the more regular freezes.

Just one caveat on my fixes:  I'm on an older code base of PiDP.   But if you implement these patches to the modern code base, I think you can finally resolve this issue - I've been LED-freeze free for many months now.

Here are the files I changed, and I've attached the patch diffs for each file.

pidp11-source-mychanges/src/02.3_simh/4.x+realcons/src/REALCONS/realcons.c
pidp11-source-mychanges/src/02.3_simh/4.x+realcons/src/sim_timer.c
pidp11-source-mychanges/src/07.0_blinkenlight_api/historybuffer.c
pidp11-source-mychanges/src/07.0_blinkenlight_api/historybuffer.h
pidp11-source-mychanges/src/11_pidp_server/pidp11/gpio.c
pidp11-source-mychanges/src/11_pidp_server/pidp11/gpio.h
pidp11-source-mychanges/src/11_pidp_server/pidp11/gpiopattern.c
pidp11-source-mychanges/src/11_pidp_server/pidp11/gpiopattern.h

Eric N.

[Anton L.]

Thanks for sharing your patches, Eric N.!

historybuffer.c.patch seems to be a complete file replacement, rather than a diff patch, though -- probably because of the different line endings.

[Eric N]

Thanks for catching that.  Looks like CRLFs in my updated file confused my patch program.  I've attached the corrected set of patches.

[Steven A. Falco]

The historybuffer.c.patch still appears to completely replace the entire file contents. Apparently, the "mychanges" version of historybuffer.c on Eric's system is using CR-LF line endings while the original uses LF line endings. So the diff flags every line as different.

Is anyone else seeing this with the historybuffer.c.patch file?

Steve

> --
> You received this message because you are subscribed to the Google Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pidp-11+u...@googlegroups.com <mailto:pidp-11+u...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/pidp-11/3fa07d63-354a-4310-95e6-f31e4bbabfa2n%40googlegroups.com <https://groups.google.com/d/msgid/pidp-11/3fa07d63-354a-4310-95e6-f31e4bbabfa2n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Bruce Robertson]

Try the --strip-trailing-cr option if your version of diff has it, when
making the diff.  Or use a sed script to strip it.

On 9/29/26 11:12

[Eric N]

So I accidentally re-uploaded the old .zip file.  Here is the right file.

Eric

[randomshi...@gmail.com]

I'm on commit 

26d20be of the official obsolescence/pidp11 repo, so that one should have the LED bug. I'm personally waiting for Oct 3, curious.

Eric, would you mind opening a PR to obsolescence/pidp11?

[Eric N]

I'll look at doing the PR this weekend.  I'm significantly behind the main release, so I'll check to see how much those files have changed since I last checked out.  I might throw in the fflush() line I added to the printer section as well that updates your attached printer text file in real time.

Eric

[Steven A. Falco]

After looking at this some more I thought it would be better to change the service_next_time_msec, service_cur_time_msec, and timer_running_msec variables to uint32_t. They are only ever set by sim_os_msec() which returns a uint32 so it makes no sense to have them as 64-bit variables.

That exposed another bug where service_next_time_msec is initialized to 0 in a few places. It should be initialized to sim_os_msec(); otherwise the lights can stall on startup.

Another bug is that timer_running_msec[] starts out at 0, which is used as a sentinel to indicate that a timer isn't running. But, while rare, it is possible that this line in the pdp11_70.c code:

_this->realcons->timer_running_msec[TIMER_TEST] = _this->realcons->service_cur_time_msec + TIME_TEST_MS;

just might come out as 0, in which case the timer wouldn't run. There are other occurrences with similar problems for other cpu models, so I introduced a macro, REALCONS_SET_TIMER_MSEC, and I use that to enable the timers. If it just so happens that the value calculated is 0, I bump it up by one millisecond. The probability of hitting that condition is 1 in 2^32, but I decided to fix it anyway.

I attached a unified patch of my changes on the client side. The server side (historybuffer and gpios) has been dealt with a while ago. I could work up a unified patch of that stuff too if anyone wants it.

Steve

On 9/30/26 08:16 AM, Eric N wrote:
> I'll look at doing the PR this weekend.  I'm significantly behind the main release, so I'll check to see how much those files have changed since I last checked out.  I might throw in the fflush() line I added to the printer section as well that updates your attached printer text file in real time.
>
> Eric
>
>
> On Tuesday, September 29, 2026 at 7:09:14 PM UTC-5 randomshi...@gmail.com wrote:
>
> I'm on commit

> 26d20be <https://github.com/obsolescence/pidp11/commit/26d20beda0c6b5a1d3a5cb3d7a4dd494e02bff75> of the official obsolescence/pidp11 repo, so that one should have the LED bug. I'm personally waiting for Oct 3, curious.

> >> <https://groups.google.com/d/msgid/pidp-11/3fa07d63-354a-4310-95e6-f31e4bbabfa2n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pidp-11/3fa07d63-354a-4310-95e6-f31e4bbabfa2n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pidp-11+u...@googlegroups.com <mailto:pidp-11+u...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pidp-11/4910f3e8-e22f-468c-9965-6f7009804f1bn%40googlegroups.com <https://groups.google.com/d/msgid/pidp-11/4910f3e8-e22f-468c-9965-6f7009804f1bn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Steven A. Falco]

I've forked the obsolescence/pidp11 github repo, and added the patches that I have authored and collected from others. Please look at the "falco" branch here:

https://github.com/stevefalco/pidp11

Steve

[Eric N]

I opened https://github.com/obsolescence/pidp11/pull/21

A note on the PR:  I first tried checking out the current code and running it on my Pi 3, but PiDP wouldn't run, so instead I surgically applied my fix to the head of your repo.  It's only two files, realcons.c and sim_timer.c, and neither file appeared change from my version.  I have another fix that involves wrapping mutexes around some code, however, they did not resolve the 49-day freeze, so I've left those out for now.  I can add them later if this current PR doesn't resolve the problem 100%.  

I'm not able to test the current repo version on my Pi 3, so I would appreciate anyone able to test this on a Pi 5 or whatever you have.  Rather than wait 49 days, I left a note on how to use GDP to preset the UINT32 to the overflow value to trigger the failure.

Eric N.

On Tuesday, September 29, 2026 at 7:09:14 PM UTC-5 randomshi...@gmail.com wrote:

[Claude Felizardo]

FYI, I bounced my PiDP-11 before the weekend and I just noticed that the LED's are frozen as of Monday 2026-10-05 at 7 pm PDT.

I am running an install from several years ago.  Anything anyone want me to check before I bounce it?

Claude

[Eric N]

When you say "bounced," what do you mean?  Did you install my patch above?

Eric N.

[Anton Lavrentiev]

> our LEDs should freeze on Oct 3 0422Z

Mine did! I don't know exactly what time it happened, but I had to
restart the system on Oct 3 @ 11:25 EDT because the lights were frozen
in the morning.

I know I don't have to restart the Pi for that, technically, but the
old simh holds the DZ listening port, so it can't be reused right away
in the next invocation, and restarting is the best clean way to work
around that.

> To unsubscribe from this group and stop receiving emails from it, send an email to pidp-11+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/pidp-11/8c83ea86-d806-4fa8-b3e9-3f25b6f60b08n%40googlegroups.com.

[Bruce Robertson]

My LEDs have never frozen. 🤷‍♂️

> On Oct 5, 2026, at 9:29 PM, Anton Lavrentiev <anton.la...@gmail.com> wrote:
>
> 

> To view this discussion visit https://groups.google.com/d/msgid/pidp-11/CAAo%3Dyr14G77sLwHGNDf3M4RdsGpfLNsg3MfFT1r747og51TZXQ%40mail.gmail.com.

[Chuck McManis]

Mine only freeze if the office gets above 95F. 

To view this discussion visit https://groups.google.com/d/msgid/pidp-11/3F16DFD8-8AEB-431C-AFAA-AEA9868DA85A%40gmail.com.

[DR]

Bounced?

A term I've not heard in the computer world except to drop a very costly
piece of equipment when no replacement was immediately available.  The
last I heard that was when one of the engineers for our Univac 1108 was
hurrying to replace a board just express delivered to bring us back up
again.


Lots of tears and gnashing of teeth.


So what does bouncing mean?

[Glenn Babecki]

The only usage of "bounced" I've heard applies to either resetting or powering a device off and then on again. 🤷🏻‍♂️

--
You received this message because you are subscribed to the Google Groups "[PiDP-11]" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pidp-11+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pidp-11/8e136751-8e05-4043-9971-cca2187d2aea%40gmail.com.

[Anton Lavrentiev]

Bouncing basically means restarting (or rebooting, to make something bounce back from being down or dead). It's used quite often in the environment I work in even today (and it's quite a large team, of a few hundred ppl altogether, from operation to developers).

On Tue, Oct 6, 2026, 8:53 AM DR <daleea...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "[PiDP-11]" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pidp-11+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pidp-11/8e136751-8e05-4043-9971-cca2187d2aea%40gmail.com.

[Steve Huston]

I've heard and used it all the time, referring to rebooting something
(with a hard bounce being equivalent to a '120 reset' or cold boot).

Internet Archive link since catb isn't responding right now:
https://web.archive.org/web/20250619174054fw_/https://www.catb.org/jargon/html/B/bounce.html

> --
> You received this message because you are subscribed to the Google Groups "[PiDP-11]" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to pidp-11+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/pidp-11/8e136751-8e05-4043-9971-cca2187d2aea%40gmail.com.



--
Steve Huston, W2SRH - https://srhuston.net
“And no one sings me lullabies, and no one makes me
close my eyes, and so I throw the windows wide and
call to you across the sky.” - Pink Floyd, “Echoes”

[Eric N]

Well, I've learned a new term today, and it sounds like I need to also commit my mutex wraps as well.  They are a little more extensive.  I'll do a second PR this weekend.

Eric N.

[Claude Felizardo]

Sorry for the jargon. In my line of work, restarting a computer, especially a VM or a service that can be done quickly is commonly referred to as bouncing and I just used it without thinking about the audience.  This would be different than a full shutdown, wait a few seconds for the caps to lose some charge, fans to stop, then power up the computer which could take minutes.

In this case I was indirectly asking if anyone wanted me to run gdb on anything and confirm a value or something.

Is it possible to kill one or more processes and restart just the emulator without having to reboot the entire PI as I have a couple of things running on it.

As for applying patches, I'm still running an install from probably when I first got it years ago - late 2021 I think.   Is this something I can apply and rebuild in place and restart, or does it require reinstalling a bunch of stuff?  Looks like I'm running Debian 11.11.  The file /etc/debian_version was last modified 9/5/2024 so perhaps some host OS updates.

I have another PiDP-11 kit I was gifted but haven't had a chance to build it and was going to install the latest and greatest at that time then come back and reinstall everything on current one but that was probably over a year ago.

I'll just reboot it for now.

Claude