VirtualBox support about vmwmouse HOT 15 CLOSED

FWIW you might find the discussion I had with the dosemu2 devs about the mouse driver interesting: dosemu2/dosemu2#1552

from vmwmouse.

VirtualBox still seems to be using their own custom protocol for mouse still. I don't use VirtualBox, so I probably won't support it here, buuuuuuuut it might not be too difficult for someone to fork this (hint hint) and add VBox support. (I can provide some help if someone wants to try that.)

from vmwmouse.

Found out about this project via Vogons.

Turns out I had just written one such driver for VirtualBox
https://git.javispedro.com/cgit/vbmouse.git/tree/

It's in (Watcom) C . I used to patch mouse.drv before (e.g. for dosbox in Maemo tablets) so I tried something higher-level for a change . VirtualBox protocol is similar in idea but more complicated in execution (it requires physical<->logical address translation and PCI).

Will look into some ideas here such as wheel support :)

from vmwmouse.

Oh, that's very nice! The VirtualBox interface looked a lot more complex with PCI in play.

As for why this one is the way it is: I was considering C, but I wasn't familiar with x86 or the driver space, so basing my driver off the existing one seemed easier than figuring out the very scantly documented Windows drivers in C. Plus I'm more familiar with the Microsoft toolchain, and AFAIK most drivers were based on (and MS intended it to be) DDK samples. Maybe one day I'll rewrite it...

As for wheel support: it does work, but there might be problems like in #22 and #23. I'm tempted to hide it behind an INI option (and hey, maybe also support a control panel from the driver to control that kind of thing - that's definitely possible per the DDK).

WRT your comment about DOS boxes, I think it'd need a VMD to flip it back to normal PS/2 protocol (since VMware at least turns off relative positioning; see #3 ) or a mouse.com implementation (cool, but a lot of DOS software may be hardcoded to expect PS/2 or relative input at minimum; see #15 ).

from vmwmouse.

Sorry, are these issues related to the wheel "hack" , or as you mention elsewhere because the driver is clobbering the high word of the 32-bit registers ? In any case, I remember there where wheel mice for 3.x which included drivers, so it may be a matter of scavenging for one of these and see what the driver did. Unfortunately, I found that the VirtualBox BIOS does not forward the contents of the Z byte/wheel to the mouse handler, even when I set it to 4-byte packets, so I'd need to think of something else.

For the fullscreen DOS box issue as far as I can see it's exactly the same issue as you describe. I think an approach that does not require an extra VxD is to hook int 2Fh and look out for 4001/4002 calls. These were broadcast when Windows enters/leaves fullscreen and e.g. the video driver is supposed to catch them to do its stuff. (Likewise they are also broadcast inside the DOS VMs so that the programs/mouse drivers there would know when to do their stuff). I think WINOS2 does something similar.

Also, yes, most DOS programs/games I know use relative data from the DOS mouse driver, as learnt from the DOSbox days :( .
However at least many of the text mode utilities that came with DOS itself do work with absolute positioning. E.g. QBASIC/EDIT, defrag, etc.
You'll note that on these, the mouse actually works even inside a Win31 windowed DOS box. In fact, windowed DOS boxes have exactly the same problem that we are trying to solve here -- how to seamlessly integrate the host (in this case Windows 3.x) mouse cursor into the DOS box window whose mouse driver usually expects relative coordinates :) MS's solution was a hook in their DOS mouse driver so that Windows could feed it with absolute coordinates. But of course, this still only works in the program relies on the DOS mouse driver for the (absolutely positioned) cursor, and most programs don't.

from vmwmouse.

Sorry, are these issues related to the wheel "hack" , or as you mention elsewhere because the driver is clobbering the high word of the 32-bit registers ? In any case, I remember there where wheel mice for 3.x which included drivers, so it may be a matter of scavenging for one of these and see what the driver did. Unfortunately, I found that the VirtualBox BIOS does not forward the contents of the Z byte/wheel to the mouse handler, even when I set it to 4-byte packets, so I'd need to think of something else.

I'm not entirely sure, because I think it happens without the wheel stuff (though the wheel stuff is also a little rough).

For the fullscreen DOS box issue as far as I can see it's exactly the same issue as you describe. I think an approach that does not require an extra VxD is to hook int 2Fh and look out for 4001/4002 calls. These were broadcast when Windows enters/leaves fullscreen and e.g. the video driver is supposed to catch them to do its stuff. (Likewise they are also broadcast inside the DOS VMs so that the programs/mouse drivers there would know when to do their stuff). I think WINOS2 does something similar.

Is hooking from the mouse driver a good idea? I thought that might be problematic on 386 enhanced mode, hence why VMDs exist.

Also, yes, most DOS programs/games I know use relative data from the DOS mouse driver, as learnt from the DOSbox days :( .
However at least many of the text mode utilities that came with DOS itself do work with absolute positioning. E.g. QBASIC/EDIT, defrag, etc.
You'll note that on these, the mouse actually works even inside a Win31 windowed DOS box. In fact, windowed DOS boxes have exactly the same problem that we are trying to solve here -- how to seamlessly integrate the host (in this case Windows 3.x) mouse cursor into the DOS box window whose mouse driver usually expects relative coordinates :) MS's solution was a hook in their DOS mouse driver so that Windows could feed it with absolute coordinates. But of course, this still only works in the program relies on the DOS mouse driver for the (absolutely positioned) cursor, and most programs don't.

That's interesting, although I'm not sure how to make all these hooks and 33h handling etc. kiss in my own driver. FWIW, I ripped out all the non-VMware code paths from the original driver, which felt like the right thing because the polymorphism was making it really hard to understand.

from vmwmouse.

Is hooking from the mouse driver a good idea? I thought that might be problematic on 386 enhanced mode, hence why VMDs exist.

Well I tried and it seems to work. I don't think it should be much of a problem, as far as I can see this is designed precisely for software running under 386 enhanced mode (such as Windows itself : ). When I get the background notification, it means the system VM is going to the background due to another VM (i.e. a DOS box) getting fullscreen'ed, so I disable the absolute mouse in VirtualBox. And viceversa on the foreground notification.

My DOS mouse driver is MSMOUSE 9.0 . I have noticed that I still need to load the VKD replacement that comes with MSMOUSE for this to work (called MOUSEVKD.386). I have no idea what it is doing , but without it, mouse still goes crazy on fullscreen DOS boxes, as if the PS/2 interface was not correctly synchronized or something.
When I use CuteMouse 1.9, this does not happen either and my change works without the MOUSEVKD.386 . But of course, it is far from ideal since CuteMouse does not have the hook for windowed DOS boxes.
I tried MSMOUSE 8.0, which does not ship a custom VKD, and it does not have this issue either, so I think it's good enough as it is.

Also found another integration mouse for 3.x (and 2.x apparently!), this one is also in OpenWatcom but targets DOSBox-X. Wish I had seen all of this before :)
https://github.com/joncampbell123/doslib/tree/master/windrv/dosboxpi/win3x

from vmwmouse.

Just let you know that DOSBox-X version 0.83.24 (release notes here: https://dosbox-x.com/release-0.83.24.html) now officially supports this VMware mouse driver for Windows 3.1, so that when used along with this driver the mouse will be seamlessly integrated with the host system and can enter/exit the DOSBox-X window without having to be captured or released, similar to VMware or QEMU. DOSBox Staging also plans to support it in a future version.

from vmwmouse.

Didn't Dosbox-X already support the "integration device" exactly for this reason ? The doslib driver above uses that, and made by DOSBox-X guys as far as I can see.

from vmwmouse.

DOSBox-X’s own integration device and VMware guest integration do not cover exact the same scope. For example, the VMware backdoor allows additional functions that the "integration device" does not support. So supporting the VMware backdoor makes things more flexible.

from vmwmouse.

Well, in the future I guess :)

I am now trying a different approach -- an absolute-pointer DOS mouse driver (which most DOSBox forks already have, but not VBox/VMware). Could probably have added support for it to CuteMouse, but I can barely understand the CuteMouse source, and besides reinventing the wheel is much more fun :) (speaking of wheels, I never got the mouse wheel to work with CuteMouse anyway since the VBox BIOS does not passthrough the wheel packets, but I workarounded that in my driver).

It's on branch 'dos' on my repo ( no longer; now it's on its own repo at https://git.javispedro.com/cgit/vbados.git/ ) .
Most of the code is for the "normal" parts of a DOS mouse driver, like cursor drawing -- very messy stuff. I'm not aware of any FLOSS DOS mouse driver in C anyway.

Now edit/qbasic behaves exactly like a DOS window under 3.x/9x. The mouse cursor is rendered by the host, not the guest (the cursor actually renders on top of VBox GUI elements):

And in fact it also works inside a Win3.x DOS window:

For Win3.x, I'm actually using a much smaller mouse.drv that calls into the DOS driver via int33, identical to the dosemu approach. There is quite a level of indirection there: Windows receives mouse events from the DOS driver, and at the same time Windows sends mouse events to the individual DOS driver instances in each DOS window.

Sadly under Win3.x cursor rendering is done by the display driver, so can't take over that.
Notice how once Composer goes fullscreen the DOS driver does take over the cursor rendering.

Also for the games which use relative mouse movements you have to turn it off the mouse integration manually (VirtualBox has an option in the menu for that, precisely for games, and I guess Vmware does too). Surprisingly, some "modern" games (like Warcraft2) actually use the absolute mouse coordinates.

from vmwmouse.

This seems to be a pretty good idea, because trying to cooperate with the VMD so DOS apps would be happy (even just fullscreen) is the hard part with the special hypervisor protocols. Using a DOS-level mouse driver in theory sidestep that, though I wonder wonder what we'd be missing out on if we move the HV protocol handling to the DOS mouse driver instead of in Windows.

from vmwmouse.

This seems to be a pretty good idea, because trying to cooperate with the VMD so DOS apps would be happy (even just fullscreen) is the hard part with the special hypervisor protocols.

The approach I mentioned above (hooking int 2fH) seemed to give good results, in that you could send the hypercalls to disable the integration when a DOS program goes fullscreen. It seemed a bit "incomplete" in that windowed EDIT could use the absolute mouse, but fullscreen EDIT couldn't. The solution for that is either to make VMD a full DOS mouse driver (a 32-bit DOS mouse driver :) ) , or to make a plain old 16-bit DOS driver, which is what I'm trying.

Sidenote: there is a reason to change VKD, actually, because VKD is limiting the rate of PS/2 (mouse) interrupts to 40 per second, and prevents the mouse driver from changing it. (I guess that's what the MS MOUSEVKD.386 is fixing).
In VirtualBox at least it leads to somewhat stuttery mouse which is not present in real mode Windows. This also affects vmware/vmwmouse , but VMware seems faster overall and the stutter is not that noticeable.

Using a DOS-level mouse driver in theory sidestep that, though I wonder wonder what we'd be missing out on if we move the HV protocol handling to the DOS mouse driver instead of in Windows.

One disadvantage i can think off is the waste of "precious" conventional DOS memory. My driver is now at around 10KiB, but probably can make it 7-8KiB if I remove all the debug printfs; while a Windows driver can use practically no DOS memory. If the DOS driver can be kept around ~10KiB it should not be much problem (MSMOUSE uses up to 20KiB...).

Overall I'm not entirely sure what is the best approach. DOS apps are very picky about the mouse driver -- forcing the user to a specific DOS mouse driver can be a lot of compatibility pain. However I hope that with the driver being in C it's easier to add hacks and workarounds for DOS programs/games.

I am trying to also support the VMware backdoor, which is interesting because of BIOS differences. All of VirtualBox/Bochs/qemu BIOSes practically evolve from the same BIOS code, so they are very similar. But the VMware BIOS does not, and this means it shows issues in my code. For example, I didn't preserve the registers during the ps2 mouse callback / "ps2 soft int", and the VirtualBox BIOS is happy with that. But the VMware BIOS goes outright crazy and will give you garbage on the next callback.

Another sidenote: I saw the same VMware disk issue you mentioned at #22 with vmware-player . This was during Windows 3.11 startup with the standard mouse.drv, albeit I had loaded my TSR before in the same session so not sure if memory was not corrupted. Hitting "report to guest" allowed Windows 3.11 to continue loading without visible issues.

from vmwmouse.

Yeah, I could see poor absolute cursor support as an issue with a DOS mouse driver. The way I was going to fix my driver implementation, I'd just let fullscreen DOS sessions have the normal PS/2 mouse (by disabling absolute input on DOS session switch) since they'd be possibly more picky, but it wouldn't help in a window I bet.

That said, another benefit of the DOS mouse driver support is you would get captureless input at DOS - even the keyboard.

from vmwmouse.

Fixed the VMware issue, so it works now.

It seems that on VMware, host rendering of the cursor needs to be configured in the VMware SVGA adapter, which is probably too complicated to do from DOS. So I'm leaving the guest-rendering cursor for now.

That said, another benefit of the DOS mouse driver support is you would get captureless input at DOS - even the keyboard.

I really dislike VMwarePlayer behavior in that it seems that you really need to capture the mouse even to use the keyboard. Even with the mouse integration. Which means I can only write in the window as long as the mouse cursor is on top of it. The moment the cursor leaves the window, it is "ungrabbed" and keyboard input stops working. Really annoying.
On VirtualBox at least I can keep writing on as long as the window is focused.

from vmwmouse.