More spec-compliant Touch Support about fna-mghistory HOT 34 CLOSED

Made some changes that likely don't affect the existing functionality, but should get us pretty close to spec-parity:
BlueLineGames/FNA@ab95e64

The notable variance from the spec is that there is a TouchPanelStateEXT class in the code which isn't part of the spec.

I peeked through some of the code & it seems in general to be trying to do what it should with XNA touch stuff, but I'm not sure how well it did.

The next step is going to be writing some code to USE the Touch functionality (as described by the XNA docs) as if it were implemented correctly, and see how far the current implementation can get us. If it's a total mess, we could just gut the methods, throw away TouchPanelStateEXT and start over.... if it's close, we could consider bugfixing and possibly a rewrite to get rid of TouchPanelStateEXT.

from fna-mghistory.

@flibitijibibo I noticed my commit to my branch messed with indentation a bit because I auto-replace with spaces & that code had tabs in it. Since this is old monogame code & I'm not sure it matches what you normally use... what indentation do you prefer in FNA?

from fna-mghistory.

'\t' tabbing over ' ' tabbing is what I prefer; I'm sure the Touch namespace is messy so feel free to pick a style you can be consistent with and we can sweep through it when it's ready for master.

from fna-mghistory.

Also, if it's any help, feel free to remove anything that's non-spec, even stuff used internally by the library. If it's easier to implement the Touch namespace without that TouchPanelState, feel free to throw it in the dumpster and write the superior solution. With 0 projects using it, there's no harm in compat breakage here ;P

from fna-mghistory.

I wrote some basic usages of the Touch code to see how far along it was. These are related to some of the things I'll need to detect for Khet to use Touch.

                TouchPanelCapabilities tc = TouchPanel.GetCapabilities();
                if (tc.IsConnected)
                {
                    MessageBoxUtil.Notify("Touch connected with " + tc.MaximumTouchCount + " max touches.");
                }
                else
                {
                    MessageBoxUtil.Notify("Touch panel not detected.");
                }

                // Need to enable the Drag and Tap gestures.
                TouchPanel.EnabledGestures = GestureType.HorizontalDrag | GestureType.Vertical Drag | GestureType.Tap;

                // Get the Touch state (we'll use this in several spots below).
                TouchCollection touchCollection = TouchPanel.GetState();

                // Test the handling of just raw press/move/release info. We likely won't
                // use this if we can handle everything using gestures.
                foreach (TouchLocation tl in touchCollection)
                {
                    if (tl.State == TouchLocationState.Released)
                    {
                        MessageBoxUtil.Notify("Released at: (" + tl.Position.X + ", " + tl.Position.Y + ")");
                    }
                }

                // Pre.process all of the gestures. We will probably look for clicks and
                // multi-touch rotations at first (via Drag gestures), then maybe piece dragging also.
                while (TouchPanel.IsGestureAvailable)
                {
                    GestureSample gs = TouchPanel.ReadGesture();
                    switch (gs.GestureType)
                    {
                        case GestureType.Tap:
                            MessageBoxUtil.Notify("Tapped at (" + gs.Position.X + ", " + gs.Position.Y + ")");
                            break;

                        case GestureType.HorizontalDrag:
                            MessageBoxUtil.Notify("HHH - HDrag1 (" + gs.Delta.X + ", " + gs.Delta.Y + ")");
                            MessageBoxUtil.Notify("HHH - HDrag2 (" + gs.Delta2.X + ", " + gs.Delta2.Y + ")");
                            break;

                        case GestureType.VerticalDrag:
                            MessageBoxUtil.Notify("VVV - VDrag1 (" + gs.Delta.X + ", " + gs.Delta.Y + ")");
                            MessageBoxUtil.Notify("VVV - VDrag2 (" + gs.Delta2.X + ", " + gs.Delta2.Y + ")");
                            break;

                        default:
                            // TODO: REMOVE
                            MessageBoxUtil.Notify("Other gesture: " + gs.GestureType.ToString());
                            break;
                    }
                }

I then ran these on a 2-in-1. The current state seems to be this:

• The capabilities are not detected so it does not say a TouchPanel is connected.
• TouchPane.GetState() does seem to return releases at least (presumably presses too, but hadn't looked into that yet).
• No gestures are detected. Even if I comment out the GetState() call above the gesture reading (in case that was "consuming" the press/release or something) the Gestures are still not found. I did drags, pinches, taps, doubletaps, etc.).

from fna-mghistory.

My advice is to assume that absolutely nothing in the Touch namespace works whatsoever and requires a rewrite, unless a specific portion coincidentally works with the other rewritten parts. That will probably be much much easier than debugging the current code, which may not have ever worked to begin with (this includes the touch code in SDL2_GamePlatform!).

from fna-mghistory.

Why do we assume it's so b0rked? Has someone attempted to use it in the past & it was a mess?

from fna-mghistory.

Ran the test by merging the touchsupport branch with my own FNADroid fork (https://github.com/AngelDE98/FNA) and running "Escape from Minimalism" with your code hooked up to it via FNADroid on my Moto G2.

• Some capabilities are detected as it says "Touch connected with 8 max touches.", but my phone supports 10. Maybe it's SDL2 just screwing up - I haven't checked yet.
• I can confirm that releases are working. Presses are working, too.
• I can also confirm that no gestures are detected.

I'll try to help testing your changes as I'm hitting some issues with FNADroid itself (some games don't render / are broken). My FNADroid TODO list isn't even that large yet and Android should be a good platform to test and use this on.

from fna-mghistory.

Why do we assume it's so b0rked?

I assume it's so borked because it's untouched code from MonoGame. Going to be totally honest here.

from fna-mghistory.

@angelde98 that 8 is actually hardcoded in there and will say that for any device. Ran into that tidbit while reading the code other day. Thanks for the help testing this!

from fna-mghistory.

@flibitijibibo: understood

from fna-mghistory.

The fact that it doesn't detect my device as having touch seems to be down in SDL. SDL also does not appear to have any methods for saying the max number of fingers. Hopefully this isn't a big deal in the modern-day since most can process plenty of fingers.

So neither of those issues have obvious fixes. Neither of them are essential to write a game using Touch, but since FNA is geared towards compatibility, this is frustrating because existing XNA games could have used those features & we don't have a great solution for them yet.

There is code to attempt to detect gestures... it doesn't use the SDL gesture stuff though. I'll try to debug why the gestures aren't working, then if that code is too messy, just see if I can throw it away and use the SDL stuff. It appears the reason that there is explicit processing in there (rather than SDL) is that XNA has specific behavior (tap is sent on release, double-tap is sent on the second press) so SDL gestures might give non-compliant results.

from fna-mghistory.

This appears to be all I can see for device enumeration:

https://wiki.libsdl.org/SDL_GetNumTouchDevices

As for the Win32 sources:

https://hg.libsdl.org/SDL/file/183936dd34d5/src/video/windows/SDL_windowsvideo.c#l111
https://hg.libsdl.org/SDL/file/183936dd34d5/src/video/windows/SDL_windowswindow.c#l251
https://hg.libsdl.org/SDL/file/183936dd34d5/src/video/windows/SDL_windowsevents.c#l840

So it appears Windows has no method of detecting touch devices outside of getting an event from the device. Isn't that nice.

from fna-mghistory.

Also, one more function call that may be of help:

https://wiki.libsdl.org/SDL_GetNumTouchFingers

from fna-mghistory.

I dug into GetNumTouchFingers, and while the docs are ambiguous, there is existing code out there that iterates through them, so it appears to be the number of fingers that are currently touching the screen.

We could potentially use this to update the number that are allowed, if more actually appear.

from fna-mghistory.

grep-ing the SDL2 source shows no call to GetSystemMetrics, required to get the value for Windows. I've also tested calling SDL_GetNumTouchFingers via FNADroid, which returns 0. Native calls seem to be required at this point until SDL2 offers proper functionality.

According to the Chromium source code (https://chromium.googlesource.com/chromium/src/+/master/ui/base/touch/), the following platform-dependent code paths are available:

• Windows: Call GetSystemMetrics(SM_MAXIMUMTOUCHES). Easy. (native)
• Android: Approximate the minimum available maximum by testing the device capabilities (0, 1, 2, 2+, 5+). Easy-ish but imprecise. (Java)
• Linux / X11: Pick the XITouchClassInfo with the most points and return it. Probably useful manpage: http://manpages.ubuntu.com/manpages/saucy/man3/XIQueryDevice.3.html (native)
• Linux / Wayland: There was a stub in the Chromium Ozone implementation, but hopefully there's something I've simply missed.
• iOS: It's missing from touch_device_ios.cc, so I don't know.
• OS X: Not even Chromium knows as there's no touch_device_mac.cc or similar.

... ouch :/

Fill in a SDL2 bug report? The Windows one is just a P/Invoke, the Android one is just a P/Invoke + C -> Java wrapper (SDL2 kinda acts as one for minimum functionality; FNADroid offers an unofficial code path). And with Wayland on its way, I'm not even sure about the Linux code path...

from fna-mghistory.

It's it's a problem, it needs to be reported. Pretty simple rule of thumb.

One of the big reasons touch and mobile don't get the attention they need is because the people who actually work on the project (myself included) recognize its market presence enough to put "something" there, but aren't interested in it enough to actually use it with the same frequency as we do all the other subsystems. It's essentially the MonoGame problem of "yeah, we have this platform, even though nobody ships with it ever," but SDL's small enough to make this more of a solvable problem if the reports and patches come in at a reasonable pace. Until slime came in, iOS support wasn't particularly good, but it got redone and should be a whole lot nicer for 2.0.4. If they take any longer to ship 2.0.4 then you might be able to squeeze in decent Android/touch support too.

from fna-mghistory.

I've recently noticed that the above test code sometimes gives me values above 1 (1.125605, 1.000611) on the bottom / right edge of the screen. I've checked and the only proper cause would be a wrong surface size or rounding errors (although the x value seems kinda off) on the Java side of SDL2, so I don't think I could fix that.

SDL also offers the ability to get pressure, which even works on Android (at least in code, not tested yet). This was available in XNA 3.1 when developing for Zune via TouchLocation.Pressure and some Android devices support it (either natively or emulation by detecting the finger size), but XNA 4.0 removed the property. Would it make sense to add it back as extension?

from fna-mghistory.

You could probably do that, sure. iPhone 6s added pressure too, so it'd probably be helpful over there too.

from fna-mghistory.

This commit over at my fork adds PressureEXT as property to TouchLocation, simply using the value SDL2 gives us (for Android, it's the value the touch device gives us).

0x0ade@55640e3

It's tested and it works, returning the same values as the Android "Dev Tools" app and multiple others. I won't issue a pull request due to the code formatting,

from fna-mghistory.

So I just announced FNA's release date:

https://twitter.com/flibitijibibo/status/669685528904601600

This basically puts it at the Steam Winter Sale.

This indirectly puts the Touch namespace at a deadline for that date. If we don't have a functional, spec-compliant replacement by the release date, I'll be deleting the Touch namespace entirely from the official release of the library. This doesn't mean that I just won't support it, but I don't want to give the impression that anything in the current version is in any way usable or even partially functional; as I mentioned before I don't find even a single line of the Touch namespace valuable, so without someone to write that namespace I'll just be cutting it from FNA's list of supported features until the work here is complete.

from fna-mghistory.

It seems traffic for FNA has gone up a fair amount, so to give new users a better impression of what we support now rather than later, I've removed the Touch namespace:

3eac457

Developers still working on this can of course ignore/revert this commit locally, then for the final PR the changes can be cleanly applied to upstream when it's ready.

from fna-mghistory.

Been sick all week so haven't gotten almost anything done on this. I'm hoping to be back on top of things by Monday. Hopefully we can get this wrapped up before the FNA launch. I'll probably need some help testing (and perhaps tweaking) the features that we won't be using in Khet, Hive, etc..

from fna-mghistory.

By default, none of the GestureTypes are enabled. This matches what the documentation says under #1 here: https://msdn.microsoft.com/en-us/library/ff827740.aspx (this is for performance reasons).

If we add the following code to the Khet example (I've updated it above, so you can see a spot where it makes sense):

// Need to enable the Drag and Tap gestures.
TouchPanel.EnabledGestures = GestureType.HorizontalDrag | GestureType.Vertical Drag | GestureType.Tap;

The code correctly detects & reports taps, but still isn't getting Drags yet. It'll require more digging, but my initial suspicion is that it's due to the TapJitterTolerance being 35 and the movements (calculated as Vector2.Distance b/w touch.Position and touch.PressPosition) being very minute (all less than 1 so far). I'm not sure if the positions are being reported too quickly (so the diffs are too tiny) or if they are on some different sort of scale, or what.

from fna-mghistory.

Are touch coordinates reported in "pixels" (screen coordinates) by spec / for you? FNA / SDL returns values between 0 and 1 on Android (IIRC they're dividing touch position by screen dimension) and TapJitterTolerance being 35 would indicate that something's wrong on my end...

from fna-mghistory.

@angelde98 I'm getting it as floats from 0 to 1, I think. This sounds like it's the same thing you're getting?

The spec says it should be in screen coordinates:
https://msdn.microsoft.com/en-us/library/microsoft.xna.framework.input.touch.touchlocation.position.aspx
"Posiiton (sic), in screen coordinates, of the touch location."

I'll see if I can fix this in my branch & see what happens.

from fna-mghistory.

Multiplying by the screen size seems to have caused Gestures to be detected. Things are still very weird & I think maybe my usage code isn't right. I think it spews Horizontl/Vertical/Free drag events every tick (so the deltas are really small) then it sends a DragComplete (which doesn't have any location data... it's basically just like a notification). I'll have to do more testing to make sure I'm understanding it correctly though.

from fna-mghistory.

Ok, good. Judging by the sample code here https://msdn.microsoft.com/en-us/library/ff827740.aspx
it looks like the delta is supposed to be just for the small sample that's being sent constantly. This makes it really easy to do something like scrolling, it just isn't great for something like dragging an object.

I'll commit what I already have. It seems like Gestures are working as far as I've exercised them.

from fna-mghistory.

@flibitijibibo while working on the 2-in-1, I keep getting a crash message here: BlueLineGames/FNA@0fd39be#diff-274024ee06f3dc4563049e5bdb34cb2eR812 the message didn't have any real info, so I just commented it out and everything ran fine. Any idea what causes that crash and/or whether there is something legitimately wrong with this comp or just something triggering this superfluously?

from fna-mghistory.

@angelde98 the stuff I just committed should have working Gestures.

from fna-mghistory.

Was the trace not helpful either?

from fna-mghistory.

I just realized how old your branch is; an update will probably fix whatever it is that's wrong.

from fna-mghistory.

@flibitijibibo I almost never have rebases or updates work well on this project... it should be especially bad since the Touch support has all been deleted, right? Tree Conflicts and such? I'll give it a try later when I have some time to wrestle with it :).

Trace wasn't helpful, it just died in SwapBuffers and it's not the usual problem if FrameBuffers not being supported ( OpenGL Device: Intel(R) HD Graphics 5300
OpenGL Driver: 4.3.0 - Build 10.18.10.3977
OpenGL Vendor: Intel
MojoShader Profile: glsl120)

from fna-mghistory.

This issue has been discontinued in favor of the new repository:

FNA-XNA/FNA#2

from fna-mghistory.