OpenCBM 0.4.99.100 Windows Binary

[Spiro Trikaliotis]

Hello,

I just uploaded a Windows binary of OpenCBM 0.4.99.100.

I tried to ease the installation as good as possible. There is an
install.cmd script that will install everything in place. It will even
install the (self-signed!) drivers for xum1541 and xu1541.

Additionally, it adds a Link to a cmd prompt that has OpenCBM in its
path. This link is added on the Desktop of the user who started the
install script. That's why you should run it from your everyday user
account!

It also adds the patch from Martin Thierer that should help the people
who can use OpenCBM only once, and on the second command, it stops
working!


In order to use it, just download it, unpack it and run install.cmd for
your normal user account! It will automatically elevate ("UAC") in order
to get the rights!

Hint: For now, I tested it on Windows 10 only! It requires the
Powershell, so everything after Windows 7 SP1 should work, but I could
not test it so far.

It can be found here:

https://spiro.trikaliotis.net/download/opencbm-0.4.99.100/opencbm-0.4.99.100.zip


NOTE:

While installing the USB drivers, a red alert will pop up that tells
you that we are not signed, and what bad things can happen. I cannot do
anything about it as I cannot sign the binary myself.

So, If you do not trust me, do not install OpenCBM!

Regards,
Spiro.

--
Spiro R. Trikaliotis
http://spiro.trikaliotis.net/

[Martin Thierer]

Hi Spiro,

I tried that on my (barely used) win10 install and didn't get it to work. 

This is the error I get when trying to run a "cbmctrl status 8":

error: Cannot open USB device: LIBUSB_ERROR_NOT_SUPPORTED                                                               

error: no xum1541 device found                                                                                          

error: Cannot open USB device: LIBUSB_ERROR_NOT_SUPPORTED                                                               

error: no xum1541 device found                                                                                          

libusb/xum1541:: The handle is invalid.     

I tried uninstalling and installing but that didn't help. 

I can think of multiple things that might be relevant here:

1. I already had libusb1 and a zadig xum1541 driver installed that I used with pyusb. I had no issues installing it and it worked fine at the time. I did *not* uninstall the driver before installing opencbm today (I don't even know how I would do that). After installing opencbm pyusb also doesn't seem to work any more. I haven't tried uninstalling opencbm and re-installing the driver with zadig yet.

2. As I only planned to give it a quick try, I tried to avoid to really install opencbm and be clever instead. Maybe that also screwed something up:

a. First I just unpacked the zip file and tried to run cbmctrl from the amd64 directory. This failed with this error message:

Error loading plugin '(null)': The parameter is incorrect.  (87)

Error loading plugin '(null)': The parameter is incorrect.  (87)

NO PLUGIN DRIVER!: The parameter is incorrect.

b. I know you wrote that powershell is needed but i wasn't sure if I had that installed so next I just ran the install script in an administrator cmd.exe shell. This seemed to work (I noticed no errors), but the resulting install still didn't work. (I *think* it also gave me the "Error loading plugin '(null)'" message and not the "LIBUSB_ERROR_NOT_SUPPORTED" message I get now after the "proper" install, but I might remember wrong).

2. So I finally gave in and installed it via powershell as it was meant to be, but unfortunately that still doesn't work. I uninstalled and re-installed a few times with re-boots in between but that didn't change anything.

I also tried the version you tagged v0.4.99.100 (b18c367b) on linux and it seems to work fine. (That's what I expected, as I think the problem with windows is with the libusb/driver install, but just to be sure).

Martin

[Martin Thierer]

Oh, btw, this cmd shell link is very handy, but it isn't removed when opencbm is uninstalled. I guess that's a bug?

[Martin Thierer]

Hi Spiro,

just a quick heads up before you put any work into this: It works now, but I'm not sure why. I uninstalled the driver from the device manager and reinstalled again, but I also tried that before and it didn't help. I also realized that I wasn't sure which firmware I had on my pro micro, so I flashed a known good firmware, but I can't imagine what other firmware I could have had on the device that could have caused this (especially as it worked fine on linux). I'll keep an eye on that because if there are firmware changes that make it fail on windows but still work on linux that would be interesting.

Now that it works I found that deconfiguring of the device does actually lead to an error on windows, as I already suspected:

USB deconfig device error: -2 LIBUSB_ERROR_INVALID_PARAM      

Don't you see that error on your windows install?

As mentioned last time, I propose to just remove the error check. I've prepared a patch that does that: 

https://github.com/thierer/OpenCBM/tree/no_error_check_on_deconfigure

Martin

[Spiro Trikaliotis]

Hello Martin,

* On Fri, Jun 12, 2020 at 03:34:23AM -0700 Martin Thierer wrote:


> This is the error I get when trying to run a "cbmctrl status 8":
>
>
> error: Cannot open USB device: LIBUSB_ERROR_NOT_SUPPORTED                 
>                                              
> error: no xum1541 device found                                             
>                                             
> error: Cannot open USB device: LIBUSB_ERROR_NOT_SUPPORTED                 
>                                              
> error: no xum1541 device found                                             
>                                             
> libusb/xum1541:: The handle is invalid.     

I had this once, too. Unplugging the ZF, waiting 2 or 3 seconds and
re-plugging helped.

It seems that after installation, it is not always usable directly. I am
not sure if it would have helped if I waited any longer after
installaton.

> 1. I already had libusb1 and a zadig xum1541 driver installed that I used with
> pyusb.

BTW: INFer.exe is a variant of zadic, which is the console version of
zadig (note the end-"c" and end-"g" of the commands!) Thus, it is the
same infrastructure, and the driver is installed almost identical.

Why did I do it this way?
INFer allows me the install the USB drivers for multiple devices at
once. Thus, it installs WinUSB for the ZF, for the different xum1541,
for the xu1541, and for the boot loader of the Atmel uC. Especially
the boot loader is missing if you use zadig yourself, and it results in
problems upgrading the firmware, as xum1541cfg hangs - it cannot detect
the ZF anymore!

There is a workaround, though, if anyone does not want to use my INFer:

1. start the update with xum1541cfg and wait for it failing.
2. start zadig and install the driver for the Atmel Bootloader (I do not
remember the exact name)
3. Run xum1541cfg again.

After this procedure, future updates will succeed.

Of course, using my INFer seems to be the simpler solution to me, at
least, if it works as expected.

> I had no issues installing it and it worked fine at the time. I did
> *not* uninstall the driver before installing opencbm today (I don't even know
> how I would do that). After installing opencbm pyusb also doesn't seem to work
> any more. I haven't tried uninstalling opencbm and re-installing the driver
> with zadig yet.

My suggestion: Unplug the ZF and try the installation again. Then, when
everything has finished, try pluggin the ZF again.

> 2. As I only planned to give it a quick try, I tried to avoid to really install
> opencbm and be clever instead. Maybe that also screwed something up:

Don't try to be clever! ;)

> a. First I just unpacked the zip file and tried to run cbmctrl from the amd64
> directory. This failed with this error message:
>
> Error loading plugin '(null)': The parameter is incorrect.  (87)
> Error loading plugin '(null)': The parameter is incorrect.  (87)
> NO PLUGIN DRIVER!: The parameter is incorrect.

opencbm requires an installation. Currently, it needs the opencbm.conf
in c:\windows\system32 in order to find the plugin. If it is not there
or if it is corrupted, you see the above message.

> b. I know you wrote that powershell is needed but i wasn't sure if I had that
> installed so next I just ran the install script in an administrator cmd.exe
> shell.

1. powershell is automatically installed on Windows 7 SP1 (build 7601) and later.

2. The administrator's cmd.exe does NOT work! To be more precise, it is
not enough! I needed some time to find out that this seems to be the
reason why some people claim that the installation fails!

With that approach, you do not get the UAC prompt, and thus, you do
not have (full) administrator rights! Yes, you are administrator, but
you do not have administrator rights! Weird, huh?

So, the install script will fail copying things to the right
locations.

That's exactly what I needed the powershell for: To elevate the
cmd.exe, so I have the correct rights. I did not find any alternative
to doing it this way.

Of course, you can start the elevated cmd.exe yourself. I think this
article should be good:
https://winaero.com/blog/how-to-open-elevated-command-prompt-in-windows-10/

Note that starting a cmd.exe from an administrator account is NOT
enough! Damn UAC...

> This seemed to work (I noticed no errors), but the resulting install
> still didn't work. (I *think* it also gave me the "Error loading plugin '(null)
> '" message and not the "LIBUSB_ERROR_NOT_SUPPORTED" message I get now after the
> "proper" install, but I might remember wrong).

The "Error loading plugin" message tells me you did not have the rights.
A c:\windows\system32\opencbm.conf does not exist, right?

> 2. So I finally gave in and installed it via powershell as it was meant to be,
> but unfortunately that still doesn't work. I uninstalled and re-installed a few
> times with re-boots in between but that didn't change anything.

Ok. So, what is the exact state at the moment on your side?
I would suggest we try to clean your machine. I think we should do this
off-list, right?

BTW, are you German? In this case, we can also write in German.

> I also tried the version you tagged v0.4.99.100 (b18c367b) on linux and it
> seems to work fine. (That's what I expected, as I think the problem with
> windows is with the libusb/driver install, but just to be sure).

There seem to be 2 issues:
1. WinUSB (!) driver install
2. administrative rights for the shell

[Spiro Trikaliotis]

Hello Martin,

* On Fri, Jun 12, 2020 at 03:36:01AM -0700 Martin Thierer wrote:
> Oh, btw, this cmd shell link is very handy, but it isn't removed when opencbm
> is uninstalled. I guess that's a bug?

Erm... Well... Not a bug, I prefer to call it an "oversight". ;)

[Spiro Trikaliotis]

Hello,

* On Thu, Jun 11, 2020 at 11:03:22PM +0200 I wrote:

> I just uploaded a Windows binary of OpenCBM 0.4.99.100.

Just a note: I removed that binary for now. I will work on it and put a
0.4.99.101 up asap, hopefully today.

[Spiro Trikaliotis]

Hello Martin,

* On Fri, Jun 12, 2020 at 05:02:07AM -0700 Martin Thierer wrote:

> Now that it works I found that deconfiguring of the device does actually lead
> to an error on windows, as I already suspected:
>
> USB deconfig device error: -2 LIBUSB_ERROR_INVALID_PARAM      
>
> Don't you see that error on your windows install?

Unfortunately, I do.

I had to re-create my environment for creating the package and for
testing it. I had a bug in my environment, so I did not test the package
that I just created, but an older one....

I am sorry for this.

> As mentioned last time, I propose to just remove the error check. I've prepared
> a patch that does that: 
>
> https://github.com/thierer/OpenCBM/tree/no_error_check_on_deconfigure

Thanks. But this leaves the question: Will the fix also fix Windows
then, because Windows does not like the deconfiguration?

[Martin Thierer]

> > As mentioned last time, I propose to just remove the error check. I've prepared
> > a patch that does that:
> >
> > https://github.com/thierer/OpenCBM/tree/no_error_check_on_deconfigure
>
> Thanks. But this leaves the question: Will the fix also fix Windows
> then, because Windows does not like the deconfiguration?

No, but as far as I can tell windows doesn't need the fix, because it
never sends extra "set configuration" messages. At least that's the
case on my win 10 install.

If there are in fact configurations on windows where this happens,
then no, I don't think the fix would then work. But I consider the
fact that libusb1 reports an error for a parameter value that is
mentioned in its own documentation a bug in libusb. Maybe I should
report it on their mailing list.

Also, I read somewhere (I think it was either the libusb docs or the
source code) that windows doesn't let applications send "set
configuration" messages at all, only device drivers.

Martin

[Martin Thierer]

Hi Spiro,

thanks for the detailed explanation! Kind of unnecessary for me now,
because I got it to work, but it might help others.

> > This seemed to work (I noticed no errors), but the resulting install
> > still didn't work. (I *think* it also gave me the "Error loading plugin '(null)
> > '" message and not the "LIBUSB_ERROR_NOT_SUPPORTED" message I get now after the
> > "proper" install, but I might remember wrong).
>
> The "Error loading plugin" message tells me you did not have the rights.
> A c:\windows\system32\opencbm.conf does not exist, right?

I can't tell any more if it existed at the time and as I said, it
might well be that I remember the message wrong.

I understand now that an administrator cmd shell isn't enough, but
isn't it possible to detect if commands fail from within your install
script? I guess others will try the same and there should be a big
"You should do as I told you and use powershell" warning...

> Ok. So, what is the exact state at the moment on your side?
> I would suggest we try to clean your machine. I think we should do this
> off-list, right?
>
> BTW, are you German? In this case, we can also write in German.

Yes, I'm german :) But no need for off-list communication right now,
as it seems to work now.

Martin

[Martin Thierer]

> > I just uploaded a Windows binary of OpenCBM 0.4.99.100.
>
> Just a note: I removed that binary for now. I will work on it and put a
> 0.4.99.101 up asap, hopefully today.

Just a quick heads up: I have a pipeline of a few fixes for minor bugs
that I found in the last few weeks (mostly stability improvements in
cases of errors). I started going through them today and rebasing them
on your current HEAD. I should have them ready later today.

If you plan to release a new version anyway, you might want to wait
for them. I'll open a PR in your github repository.

Martin

[Spiro Trikaliotis]

Hello,

* On Fri, Jun 12, 2020 at 06:33:52PM +0200 Martin Thierer wrote:

> If you plan to release a new version anyway, you might want to wait
> for them. I'll open a PR in your github repository.

Ok. I have had a look into some of them, but I am not sure if I
understand them all.

The update to .101 will have to wait a little bit, I will check your
patches and integrate them as I see fit before creating the .101.

As far as I see, there is no machine that is kept broken if I delay the
.101, right? Otherwise, whoever has a problem, please contact me here or
via mail directly!

[Spiro Trikaliotis]

Hello,

I must correct myself:

* On Fri, Jun 12, 2020 at 05:55:48PM +0200 I wrote:

> > USB deconfig device error: -2 LIBUSB_ERROR_INVALID_PARAM      
> >
> > Don't you see that error on your windows install?
>
> Unfortunately, I do.

I installed it multiple times in the same Windows 10 VM. Sometimes, I
get this error, and sometimes, I do not get it. It is important to note
that I am doing exactly the same in both cases! (Going back to the VM
snapshot, installing the same OpenCBM package, doing the tests...)

It's weird...

[Martin Thierer]

If there are in fact configurations on windows where this happens,
then no, I don't think the fix would then work. But I consider the
fact that libusb1 reports an error for a parameter value that is
mentioned in its own documentation a bug in libusb. Maybe I should
report it on their mailing list.

I just opened an issue on libusb's github page. Let's see what they have to say.

 

No, but as far as I can tell windows doesn't need the fix, because it 
never sends extra "set configuration" messages. At least that's the 
case on my win 10 install. 

While there, I found this comment which might be relevant:

One of the reasons we used libusb0.sys was because it was the only back-end (at the time) which supported changing the device configuration. Is that still the case?

Unfortunately it is still the case. I forgot about this use case. On the other hand, this is a very rare use case.

If I understand it correctly, then libusb0.sys is the driver part of libusb-win32, the windows version of libusb0, which was also used by opencbm until not too long ago? This 1. might explain why people were seeing this issue on windows too and 2. in this case de-configuring the device would probably help.

BUT that would also mean that the weird behaviour regarding "set configuration" messages on usb 3 vs usb 2 ports I see on my linux machine would also exist on windows. Which would probably mean that it's either a hardware issue or maybe it is the correct behaviour.

Now I'm really curious what would happen when I install libusb-win32 on my machine. Would your 0.4.99.100 build still work with that version?

Martin

[Martin Thierer]

Now that it works I found that deconfiguring of the device does actually lead to an error on windows, as I already suspected:

USB deconfig device error: -2 LIBUSB_ERROR_INVALID_PARAM      

Don't you see that error on your windows install?

As mentioned last time, I propose to just remove the error check. I've prepared a patch that does that: 

https://github.com/thierer/OpenCBM/tree/no_error_check_on_deconfigure

I now think that removing the whole error check might be a bit overkill. Just ignoring the LIBUSB_ERROR_INVALID_PARAM error should be sufficient for now.

[Spiro Trikaliotis]

Hello Martin,

* On Sat, Jun 13, 2020 at 02:09:28AM -0700 Martin Thierer wrote:

> I just opened an issue on libusb's github page. Let's see what they have to
> say.

I ask myself if it would not make sense to ask about the correct
behaviour of the data flag there. ;)

Please also note this here:
https://www.forum64.de/index.php?thread/103178-opencbm-0-4-99-100/&postID=1532132#post1532132

The claim is that directly after booting, he gets an "previous command
was interrupted, resetting" and then the "USB request for XUM1541 close
failed, continuing: LIBUSB_ERROR_IO"

I think we should get help from people more knowledgeable and
experienced. As you have the most experience from us and know most what
is going on, you would be the perfect candidate to ask.

What I also find very interesting is that almost all examples I find
about using USB do not call setconfiguration at all! I am not sure if it
is because it is problematic, or because it is not needed, or... I do
not know.

> No, but as far as I can tell windows doesn't need the fix, because it 
> never sends extra "set configuration" messages. At least that's the 
> case on my win 10 install. 
>
> While there, I found this comment which might be relevant:
>
> One of the reasons we used libusb0.sys was because it was the only
> back-end (at the time) which supported changing the device
> configuration. Is that still the case?
>
> Unfortunately it is still the case. I forgot about this use case. On the
> other hand, this is a very rare use case.
>
> If I understand it correctly, then libusb0.sys is the driver part of
> libusb-win32, the windows version of libusb0, which was also used by opencbm
> until not too long ago?

Yes.

libusb-win32 used the libusb0.sys driver and implemented the libusb0
interface.

libusbK and WinUSB can be used to implement the libusb1 interface. I
think libusbK also supports the libusb0 interface.

It is not very understandable.

> This 1. might explain why people were seeing this issue
> on windows too and 2. in this case de-configuring the device would probably
> help.

But with libusb1, it should not happen there, right? But how can we
understand the Forum64 result which I linked above?

> BUT that would also mean that the weird behaviour regarding "set configuration"
> messages on usb 3 vs usb 2 ports I see on my linux machine would also exist on
> windows. Which would probably mean that it's either a hardware issue or maybe
> it is the correct behaviour.

I am completely out here...

> Now I'm really curious what would happen when I install libusb-win32 on my
> machine. Would your 0.4.99.100 build still work with that version?

No. For libusb-win32, you need an OpenCBM which is compiled against
libusb0. I had thought about making the dynamic, but I decided against
it as I do not think having too much legacy and options makes the code
any clearer.

If you need it for testing purposes, I can build one for you.

[Martin Thierer]

Hi Spiro,

> I just opened an issue on libusb's github page. Let's see what they have to
> say.

I ask myself if it would not make sense to ask about the correct
behaviour of the data flag there. ;)

Yes, I plan to ask either there or on the linux-usb mailing list at some point. It's just that first I'd like to have a good feeling that I at least understand the problem. And with new evidence popping up all the time I just haven't reached that state yet... :)

 

Please also note this here:
https://www.forum64.de/index.php?thread/103178-opencbm-0-4-99-100/&postID=1532132#post1532132

The claim is that directly after booting, he gets an "previous command
was interrupted, resetting" and then the "USB request for XUM1541 close
failed, continuing: LIBUSB_ERROR_IO"

That's very strange. The xum1541 plugin prints the "previous command was interrupted, ..." message when the firmware signals this in its response to the init message. It's not something the host would come up with on his own. I can't imagine how this would happen without the firmware actually receiving a command before.

So my guess is that somehow some opencbm command gets invoked, maybe from autostart or a frontend (I find it suspicious that he starts the command from a folder called CBMXfer which seems to be a frontend for opencbm?).

I also don't really understand what could cause the "close failed" message. Maybe he's not really running your 0.4.99.101 version or a different executable (maybe the one that started the unfinished command?) is still running and blocking the communication.

Btw. when talking about v07 for the promicro, he's probably referring to this version:

https://github.com/zyonee/opencbm/tree/2e8264cb84d677e48e0a383e5c95b7f12c3250e3/xum1541

I might flash and try it, but I don't think his problem is the firmware. 

I think we should get help from people more knowledgeable and
experienced. As you have the most experience from us and know most what
is going on, you would be the perfect candidate to ask.

What I also find very interesting is that almost all examples I find
about using USB do not call setconfiguration at all! I am not sure if it
is because it is problematic, or because it is not needed, or... I do
not know.

Yes, by now I also tend to think that the best fix would be to check if the device is already configured and only explicitly configure it if not. But as I said, I'd still like to get a better understanding of the problem first.

 

> Now I'm really curious what would happen when I install libusb-win32 on my
> machine. Would your 0.4.99.100 build still work with that version?

No. For libusb-win32, you need an OpenCBM which is compiled against
libusb0. I had thought about making the dynamic, but I decided against
it as I do not think having too much legacy and options makes the code
any clearer.

If you need it for testing purposes, I can build one for you.

That might become useful, but for know I think it would be sufficient if you could point me to a version that would still work with libusb-win32. I'd explicitly like to test with a version that does not have the de-configuration, because I'm curious if it shows the same behaviour regarding usb 2 vs usb 3 ports on my machine. That will not really help to find a solution, but give us a better understanding if the same behaviour I see on linux could have existed on windows before or if the reports you got on windows were caused by different issues.

Btw. what's needed to compile on windows? Does mingw work or do you need vs express? (I once installed vs express and was repelled by the bloat...)

Martin 

[Martin Thierer]

Please also note this here:
https://www.forum64.de/index.php?thread/103178-opencbm-0-4-99-100/&postID=1532132#post1532132

The claim is that directly after booting, he gets an "previous command
was interrupted, resetting" and then the "USB request for XUM1541 close
failed, continuing: LIBUSB_ERROR_IO"

That's very strange. The xum1541 plugin prints the "previous command was interrupted, ..." message when the firmware signals this in its response to the init message. It's not something the host would come up with on his own. I can't imagine how this would happen without the firmware actually receiving a command before.

So my guess is that somehow some opencbm command gets invoked, maybe from autostart or a frontend (I find it suspicious that he starts the command from a folder called CBMXfer which seems to be a frontend for opencbm?).

I also don't really understand what could cause the "close failed" message. Maybe he's not really running your 0.4.99.101 version or a different executable (maybe the one that started the unfinished command?) is still running and blocking the communication.

Btw. when talking about v07 for the promicro, he's probably referring to this version:

https://github.com/zyonee/opencbm/tree/2e8264cb84d677e48e0a383e5c95b7f12c3250e3/xum1541

I might flash and try it, but I don't think his problem is the firmware. 

I saw that the guy on the forum64.de website posted that he used d64copy before trying cbmctrl and that already complained about not being able to close the connection (at least that's how I read it and that would at least explain the "previous command 

was interrupted" message), I went ahead, flashed the v07 firmware and tried with your 0.4.99.101 on windows. The result is I couldn't reproduce the problem; d64copy and cbmctrl close the usb connection just fine, both on an usb3 and an usb2 port.

What I did find, however, which might be related or not, but should in any case be kept in mind for when people report problems, is that after using d64copy *with a parallel cable* the lines on the parallel cable apparently stay in a state that partly keeps the pro micro powered, even after unplugging it from usb! This causes the adapter to not being properly reset (at least that's what I think is happening) and therefore the device most of the time not being recognized after re-plugging it, unless either the serial or the parallel cable to the floppy drive are also disconnected. (Even though I'm pretty sure the pro micro is kept powered through the parallel cable, disconnecting the serial cable also works, probably because it cuts the ground connection).

This is a circuit with a pro micro and a 7406 chip, so it might very well be specific to this hardware combination.

Martin

[Spiro Trikaliotis]

Hello Martin,

* On Sun, Jun 14, 2020 at 09:55:54AM -0700 Martin Thierer wrote:

> What I did find, however, which might be related or not, but should in any case
> be kept in mind for when people report problems, is that after using d64copy *
> with a parallel cable* the lines on the parallel cable apparently stay in a
> state that partly keeps the pro micro powered, even after unplugging it from
> usb!

So, what should the parallel port output? A $00 or $FF? I would assume
$00 to be the right one?