Using a Mouse and Keyboard for Console Games

Part 4: Conclusions, Next Steps, Lessons Learned

How does it work?


Most of my rig works flawlessly in Xbox One games. All the buttons feel responsive, as does the left stick mapped to WASD. Despite the extra steps involved in getting my input to the game, the latency is really not at all noticeable.

On the flip side, the mouse -> joystick mapping still doesn’t quite feel right, mostly in the Y direction. I fixed this a bit by upping the sensitivity in-game, but it still feels wrong sometimes. When I return the joystick to idle, sometimes it “bounces” back the opposite direction in-game. I suspect this is a problem with the iffy sacrificial controller, since I’m basically setting the resistance digitally.

What’s left to finish?

Right now, I don’t have the Guide button or the D-Pad wired up. There are enough pins on the Teensy to make this work, but the pads on the controller’s board aren’t through-hole, so I can’t solder header pins for them like I did on the other buttons. They’re designed for rubber dome switches and aren’t as straightforward to mess with. (I take that back – they might be, but I lack the knowledge ;)

The hardware layout is on a breadboard, and I have some solderable breadboard that I might transfer it to at some point, assuming I use it enough to warrant the extra effort.

Also, I think it would be interesting to compare my hack to GIMX, since I have the hardware to put together a GIMX adapter. I’m also a bit tempted to grab a Titan One and stack the three of them up and see which one I like best.

As I’ve mentioned a few times in this article, I sacrificed a somewhat iffy controller for this build. I didn’t want to risk my newer, nicer, working-er… controllers just in case I ran into something I couldn’t solve. If Xbox One controllers ever get super cheap, or if I can buy one with some cosmetic or unrelated issues, it might be worth it to upgrade the controller. I made this whole thing modular, so it shouldn’t be a problem to swap out the controller board. Then again, most standard controllers use dome switches for A/B/X/Y/Start/Back, which would create more issues if I switched.

How far can I take this?

Now that I can programmatically alter console game inputs, there are some possible interesting applications to this technology. I could script inputs to the game, or even perhaps use computer vision to capture the game output and create some fully automated game-playing bot. If I take this any further, though, I’ll need to be wary of any terms of service issues I might run into.

Am I satisfied with the results?

I really embarked upon this project out of curiosity. How do controllers work? Is it possible to simulate movement on the analog sticks from a microcontroller? Can I really fool the Xbox One into accepting mouse and keyboard input? How do I map a mouse’s movements to a joystick? I feel like, despite the extra work involved versus buying a commercial solution, I learned a lot. Hopefully, by reading this far, you’ve learned a lot too :)

Prev Page 4 of 4 Next