Discuss
  • The Gamepad API assumes you aren't using the gamepad.

The Gamepad API assumes you aren't using the gamepad.

# Carl Smith (9 years ago)

I've been looking at the Gamepad API. It's a nice API, and the scope is sensibly narrow, but I think it is just too limited.

With smart TVs, games consoles, Android boxes and so on, lots of users are using joypads and remote controls to browse the Web. It's pretty clumsy, but when you only want to find an album on YouTube, it works.

On the OUYA, buttons and axes are mapped to browser controls. FireFox on Android maps the shoulder buttons to Back and Forward, and the left axes pan the page. Chromium is similar. I'm not sure the vendors intended for that, but still.

The issue is that the browser, extensions and webapps all want to use the same pad, but the API provides no way to 'take' or 'free' a pad.

Once an app has access to a joypad, the browser will (must) just ignore the state of the pad for so long as the app can reference that pad. It can be difficult for the user to exit an app when the only input device they have to hand must be completely ignored by the browser. Users often want to keep the app open too.

The problem is a lot like the problem of pages using bindings that conflict​ ​with the browser, only here, you have less keys, and the browser must let​ ​the page take over all the bindings.

The Gamepad API should allow each pad to be 'taken' and 'dropped'.

I'd suggest using a stack, with the browser always on the bottom. If an app calls gamepad.take it goes on the stack. It gets popped when it calls gamepad.drop. Whoever's on the top of a pad's stack has the pad all to itself.

With a stack, for example, a browser extension could take the pad from the browser, and a game could then take the pad from the extension. The game could drop the pad once it was done, automatically returning the pad to the extension. The game wouldn't need to expect the extension.

There'd need to be some place to attach two callbacks, so programs could handle the pad being taken and returned to them.

No app should ever try and take the pad in the callback for when it's taken from them, but bad code might hog a pad or get into a fight over it. In any case, I think there should have always been a required, reserved 'menu' button on every mapping that the user can use open the browser menu. From there, they could close tabs etc.

If something like this were to happen, browsers would need to support pads properly, at least allowing gamepad configs to be added by the user that would map ​'​​actions' to browser controls.​​

The browser would ideally abstract actions - stuff like taps, double-taps, long-presses and combos (hold X and do Left, Right, Left)​, though ​extensions could handle that stuff if the ​Gamepad ​API ​were better (and all the browser actions are exposed to extensions).

I tried building something with the Gamepad API and couldn't, so just wanted to offer some feedback really.

One other thing: The inability of apps and extensions to grab the mouse ​prevents using an analogue stick to point and click.​ ​That's​ ​another​ ​thing that can only be fixed at the browser level.

# Florian Bösch (9 years ago)

There is an API called "pointerlock" whose purpose is to allow you to capture the mouse. An equivalent API is (as of yet) lacking in the gamepad API, but it is requested as a feature: w3c/gamepad/blob/gh-pages/FeatureRequests.md

# Carl Smith (9 years ago)

Thanks for the link, Florian. I hadn't seen that. I don't think you can emulate the mouse with pointerlock. It works differently.​

The thing is, I don't think the Gamepad API needs extending. It needs removing from ES6​. It will never work without breaking everything that uses it.

Currently, no extension can use the Gamepad API without conflicting with any other extension that uses it, and the entire Web. And the browser can never offer first-class support for pads for the same reason.

To the extent that we can't break the Gamepad API, we can't use pads in extensions or to control the browser. As we go into a SmartTV era, that will probably be pretty bad.

# Florian Bösch (9 years ago)

On Sun, Oct 25, 2015 at 10:13 AM, Carl Smith <carl.input at gmail.com> wrote:

​Thanks for the link, Florian. I hadn't seen that. I don't think you can emulate the mouse with pointerlock. It works differently.​

You cannot emulate the mouse for two reasons:

  • You can't synthesize the necessary events to make it work (or some of the events synthesized aren't picked up correctly)
  • The mouse pointer is deep in the OS and is drawn at much higher frequency/lower latency than other things, at an interval of 60fps and given the latencies that browsers introduce, a smooth pointer emulation is impossible.

However that does not mean that pointerlock is a pointless API. UAs/OSes can work on getting the latency down, and it is a useful API for some kinds of content.

The thing is, I don't think the Gamepad API needs extending. It needs removing from ES6​.

ES6 is not responsible for the APIs that UAs offer. ES is administrated by ECMA whereas the APIs are administred by W3C and WhatWG. As such ES6 has little say in what API is supported or not.

It will never work without breaking everything that uses it.

Currently, no extension can use the Gamepad API without conflicting with any other extension that uses it, and the entire Web. And the browser can never offer first-class support for pads for the same reason.

To the extent that we can't break the Gamepad API, we can't use pads in extensions or to control the browser. As we go into a SmartTV era, that will probably be pretty bad.

There is a host of content that uses the Gamepad API just fine. There are some improvements that should be made to the API for platforms whose primary interaction with the UA happens trough HIDs other than a mouse and keyboard.

Regardless, the Gamepad API exists because it satisfies a need that (some) web applications do in reality have. I don't think the proper solution to solve gamepads is not to have gamepads.