Started working on AirBender

Hey,

it’s been a while since I touched topics solely devoted to my beloved Sony DualShock 3 controller, mainly because, well, ViGEm 😁 Trace on ScpToolkit has gone cold for a couple months now but a shard of hope I can present for the community. I will split and port the connection-oriented low-level components within the toolkit to its own respective native Windows drivers, including wireless functionalities.

Connecting the DS3 to a standard PC running Windows via Bluetooth has always been an annoyance because the DS3 lacks a compatible SDP layer. I was thinking a lot about a convenient solution where you can keep your existing Bluetooth software stack so the DS3 can connect side by side to your existing gadgets like keyboard, mouse, headset, you name it. Well, unfortunately this is not possible. Now I very rarely use the phrase “not possible” or “impossible” in tech because as a developer you literally have the power to create and change your (virtual) environment as you please but sometimes you have to accept the limitations of the environment you’re dealing with (Windows in this case).

Thing is, you have the abilities to create Bluetooth profile drivers with tools Microsoft provides you with, but once again the DS3 being a non-standard bastard needs adjustments lower in the stack where you can’t easily hack into without risking stability of other devices.

I’ve decided that the best solution is the existing design of sacrificing a dedicated Bluetooth host dongle for the entire purpose of communicating with the DS3, but as a real driver. But a user-mode driver this time! This has several advantages over a kernel-mode function driver:

  • Faster development due to easier testing and debugging
  • An uncaught error only crashes a process and not the entire system
  • Increased security; no more kernel privileges needed
  • Faster and easier signing process

But it’s only fair to also mention the cons:

  • UMDF 2 requires at least Windows 8.1 (thanks, Microsoft 😑)
  • No kernel-mode features like spawning children (PDOs)
  • Slightly but probably unnoticeable less performance

Now I know I’m getting hate for ignoring the still popular Windows 7 demographic but hear me out: a driver written for UMDF 2 can be be ported to a Windows 7 (or even older) compatible KMDF driver at any time. So once I’m finished I might consider going the extra mile and building and releasing it for Windows 7 too. Time will tell, as always 😋

Have a nice day,
c’ya!

PS: last but not least: the project repository 😀

ViGEm v1.13.0.0 released

Hey,

Version 1.13.0.0 is out, containing the following improvements:

  • Fixed freezing the host machine on Windows 7
  • Attaching virtual devices (plugging in children) is now a blocking operation. It’s backwards compatible to software using the deprecated ViGEmUM library. The new ViGEmClient library provides APIs for both blocking and asynchronous plugin.
  • New API documentation.
  • Updated example implementations.

So two big chunks from my previous post are already live and test feedback is mostly positive. More to come, stay tuned!

Cya!

PS: I know the .NET wrapper is still toast, will update soon, promise! 🙃

ViGEm Roadmap-thingy

Yo,

once again I’d like to inform you what’s going on in the development dungeon. Whacking together a quick list of stuff I wanna work on the next couple days/weeks regarding the ViGEm framework.

Redesign user-mode library

Currently ViGEmUM.dll is responsible for interaction with the bus driver and providing developers with access to the emulation sub-system. The library has some serious flaws due to rushed development and bad test coverage. Since fixing most of the stuff I’m unsatisfied with would inevitably break both API and ABI backwards-compatibility I decided to start again from scratch and make it a static library instead since most of my users compile it like that anyways. I’m also investigating some tools to provide decent API documentation, one of the is Read the Docs. Do keep in mind that I’ve never designed an API before this project so I’m learning as I code along 🙃

Redesign .NET stuff

There’s no beating around the bush; my .NET-wrappers are (still) in a pitiful state, mainly caused by their dependency to the native user-mode library which introduces issues with CPU architectures and compatibility between updates. My approach is to port the user-mode functionality entirely to the managed .NET world and implement it a 100% in C# so only one nice and clean assembly is necessary for communication with the bus driver. This would also simplify the creation of a NuGet package.

Blocking device attachment

Actions like adding another virtual device to the bus are inherently asynchronous because once the bus device (FDO) is instructed to report another of its children (PDO) to the system, it’s the Windows PNP-Managers responsibility to bring that new child to live so it becomes available to the system (“boot-up”). This usually takes around 300 to 700 milliseconds. Within this time frame it’s not possible to interact with the PDO through the bus drivers control mechanisms. I never had a problem with this fact but feedback from the community suggests it would be better to have this plugin procedure be a blocking action until the virtual device is truly ready for interaction. I agree. But you need to understand that at the driver level it’s not as easy as it seems because once the bus driver has announced a new child it no more has responsibility or control over it’s boot-up progress. I did some research and found a way of interaction between the FDO and the PDO which could satisfy this need. I’ll keep you updated on this topic.

Generic HID-Device emulation

I receive more and more feature requests to support a dynamic multi-purpose DirectInput-compatible joystick HID device like vJoy can. While I certainly sympathize with the idea behind it due to its complexity I’d rank this a bit further down the list of priorities for now. Time will tell, be assured 😄

Well, that’s it for now,
Cya!

ViGEm v1.10.2.0 released

Ey,

a big update of the bus driver was just released. I’m happy to announce, that I was finally able to fix two of the nastiest long-time bugs in the bus driver:

Fortunately both were linked to the same (stupid) mistake I made when porting some sections over from the legacy ScpVBus driver sub-project of the ScpServer/ScpToolkit. Since it’s a rather subtle yet “hidden in plain sight”-obvious bug I’d like to explain it in greater detail.

When you emulate USB devices, you typically want to mimic the physical devices properties as close as possible in your virtual environment as well. Sometimes (depending on the device and the driver stack it uses) you can get away with either slight or even major simplifications of the original implementation and everything is still working fine (the device might have unused interfaces you can ditch or special power/sleep attributes you don’t have to replicate in an emulated device) but typically – especially when debugging – it’s the best idea to emulate everything as precise as possible. Now why is that important? Let’s dive into more detail on how the Xbox 360 Controller interacts with its USB host.

An XUSB-compatible game pad typically exposes four interfaces, where the first one has two endpoints (Interrupt In and Interrupt Out), the second one has four endpoints (two pairs of Interrupt In and Interrupt Out) and the last two interfaces are unused and non of our concern.

When emulating the X360 pad, only the first interface and its Interrupt In endpoint is used to transfer pad state changes to the XUSB/XInput subsystem (while the Interrupt Out endpoint delivers vibration and LED state changes) which then get reflected to games using the XInput API (and DInput, ofc.). This was also the first logic I ported over and tested when first developing the ViGEm Bus ground up.

Now you have to know, that ViGEm is built using the Windows Driver Frameworks, an abstraction layer for lots and lots of boilerplate and pitfalls from WDM-land, which ScpVBus is written in. Why do I bring this up? Well, the same code you need in WDF to queue an IRP and mark it pending are roughly two, with proper locking, maybe five lines of code whereas in WDM it’s more like thirteen to fifteen lines of code. When hunting down the bus I mentioned I used the most basic option available; compare the code of the legacy yet working project to the brand new shiny yet faulty project. I read the same passages over and over and over and didn’t see any difference so I made the mistake of wasting my time looking in other areas for the issue. Then I came back and read again. And again. And ag… wait a minute. What’s this line doing, I’m not checking the value in ViGEm. Ok, that’s a major fail in plain sight but why is the check which pipe the request is associated to important?

if (Pipe == 0x81)
{
	status = STATUS_PENDING;
	IoMarkIrpPending(Irp);

	KeAcquireSpinLock(&pdoData->PendingQueueLock, &PrevIrql);
	{
		IoSetCancelRoutine(Irp, Bus_CancelIrp);

		if (Irp->Cancel == TRUE)
		{
			IoSetCancelRoutine(Irp, NULL);
			KeReleaseSpinLock(&pdoData->PendingQueueLock, PrevIrql);

			Irp->IoStatus.Status = STATUS_CANCELLED;
			Irp->IoStatus.Information = 0;

			IoCompleteRequest(Irp, IO_NO_INCREMENT);
		}
		else
		{
			PPENDING_IRP le = ExAllocateFromNPagedLookasideList(&g_LookAside);

			le->Irp = Irp; InsertTailList(&pdoData->PendingQueue, &le->Link);
			KeReleaseSpinLock(&pdoData->PendingQueueLock, PrevIrql);
		}
	}
}
else
{
	status = STATUS_PENDING;
	IoMarkIrpPending(Irp);

	KeAcquireSpinLock(&pdoData->PendingQueueLock, &PrevIrql);
	{
		IoSetCancelRoutine(Irp, Bus_CancelIrp);

		if (Irp->Cancel == TRUE)
		{
			IoSetCancelRoutine(Irp, NULL);
			KeReleaseSpinLock(&pdoData->PendingQueueLock, PrevIrql);

			Irp->IoStatus.Status = STATUS_CANCELLED;
			Irp->IoStatus.Information = 0;

			IoCompleteRequest(Irp, IO_NO_INCREMENT);
		}
		else
		{
			PPENDING_IRP le = ExAllocateFromNPagedLookasideList(&g_LookAside);

			le->Irp = Irp; InsertTailList(&pdoData->HoldingQueue, &le->Link);
			KeReleaseSpinLock(&pdoData->PendingQueueLock, PrevIrql);
		}
	}
}

Both sections look identical so what’s this check for? 🤔 Wait a minute:

&pdoData->HoldingQueue

What the hell are you? 😮 I’ve never seen you in my entire life, what’s this queue for?! Are you kidding me, how could I have missed that?

Dammit! Flash of genius! That’s the culprit right there; in ViGEm I simply checked for the transfer direction but didn’t distinguish the pipes which are mapped to different endpoints. So since both IRPs from different pipes ended up in the same queue reserved for input reports only, some packets got randomly (race condition too, yay 😖) sent to the wrong pipe. When you send a D-Pad Up state change, it would look like this on the wire:

0x00 0x14 0x01 ...

Which happens to be one of the “headset got inserted” control codes of the XUSB protocol 😂 Feeding two pipes from one queue also caused every second or even third submit request getting swallowed by the wrong pipe and therefore no state changes in XInput. What a beast!

Thankfully the fix was very easy; just introduce a second queue in ViGEm which holds the control interrupt pipe requests pending until device destruction, distinguish the pipes properly and everything is fine! 😀

’till next time!

ViGEm v1.10.1.0 released

Yo,

a stable release for real this time 😅 I reverted the dummy audio device pseudo-fix I blogged about in the past.

But we have some progress also; the axis value offset calculation in XnaGuardian was faulty due to my impressive math skills 😁 This is also fixed now, meaning that the values submitted via the override functions now get reflected 1:1 on the output like you’d expect.

’till next time!

ViGEm v1.10.0.0 faulty

Ey,

I did a lot of research and testing the last couple days and have to share some results. In the latest release I assumed I fixed the bug where a deaf XBOX 360 headset was appearing on the system when – for some odd reason – the Up button on the D-Pad was pressed.

After an awful lot of searching and reading various threads I came across the IRP_MN_QUERY_DEVICE_RELATIONS request, which seemed to be linked to my particular issue. Since ViGEm uses the WDF I assumed that this request was handled differently than in WDM, the driver model that the ancestor ScpVBus uses. Well, I was partially right; the request does indeed get handled per default by the KMDF internally so i decided to just intercept and fail the BusRelations Request in my driver which at first glance seemed like it did the trick! Sadly due to all the euphoria I slapped together a new build and released it without further testing 🙄

But in reality I wasn’t intentionally fixing the issue with my code like I thought; I just forgot to read all of the documentation which stated that you also need to call IoSkipCurrentIrpStackLocation(Irp) after pre-processing the IRP or else! Well, from what I understand this ruined the IRPs stack in a way that the request got left in a pending state which looked fine after you create and use the emulated device but goes horribly wrong once you try to cleanly unplug it.

Ok, so I know what I have to revert but now we aren’t any closer to a proper solution like we were prior to this release 😕 Further investigation followed. In this case it was once again time for a Wireshark session and a real physical X360 pad. Simply pressing and holding the Up button on the D-Pad really isn’t a challenge and we receive the following expected INTERRUPT_IN traffic:

0000   1b 00 10 10 d5 b1 86 b8 ff ff 00 00 00 00 09 00 
0010   01 01 00 02 00 81 01 14 00 00 00 00 14 01 00 00 
0020   00 5e f5 24 03 20 fa 4f 08 00 00 00 00 00 00    

The last twenty bytes represent the transfer buffer content of one input HID report (well, it isn’t HID-compatible but you know what I mean) which looks just fine. Although this doesn’t differ in any way like my emulation layer does the native pad won’t announce it’s headphone jack while the virtual one does. I’m baffled. And out of ideas. For the moment.

I’ll be back on this! 😠

ViGEm v1.10.0.0 released

I know the latest ViGEm release has bin out for quite some time but I have to drop a few notes on this one. Although I managed to fix an annoying long-term bug it seems I went a bit overboard from what users are reporting. I will investigate and – yet again – fix it and also try to provide some close-to-stable user-mode libraries for the published driver editions as well. Thanks again for the patience and continuous feedback and last but not least the slowly but steadily rising donations 😘

’till next time!

Updated Forums and fixed mailing

Heya,

I just updated the forums to the latest and greatest beta of Flarum and switched over from PHP 5.6.x to PHP 7.0.x. I also changed the mail driver to use PHP’s built-in mailing function and successfully tested it a couple minutes ago. Hopefully you can now finally keep updated on posts via mail notifications. If you don’t, just… well… drop me a quick mail 😁

Thanks for your patience, keep in mind that Flarum is beta software (but I love it! 😍)

Cya!

ViGEm Framework Roadmap

I got asked about a long-term roadmap for the ViGEm project so here it is. It’s sorted by appearance in my head, not priority or chronology or something that’s close to a system 😜

Add/Improve Documentation

I won’t sugarcoat it; I suck at documenting my APIs and ideas 😅 It’s not that I’d dislike documentation but I’d rather spend my time on coding, reverse engineering and solving riddles. Since the projects are intended to provide a framework for other developers it’s crucial to improve this part.

Standardize and harden APIs

I tend to change APIs without version bumps which isn’t best practice at least 😁 I should also mention that before this project I didn’t really design publicly available APIs so I just make up things as I go. Feedback always welcome of course 😉

Provide example applications

When diving into new frameworks or libraries I tend to search for some example implementations first instead of pondering over documentation. I typically whack together some simple console application if I wanna test new features and tend to forget them after success. This should improve massively since nothing beats nicely commented ready-to-copyuse code.

Add Xbox One Controller emulation

Although I successfully created a working proof-of-concept there’s still a long road ahead of me if I wanna provide stable XBONE Controller emulation. Since hunting bugs and reacting to user feedback was my main priority the last couple month I kinda procrastinated on this topic. I don’t want to ditch it though since there is obvious user demand for this feature.

Work on Website and Forums

Coding currently eats up a lot of my time (besides other pesky real-life tasks 😜) so I’m always short on updates and general social-media stuff. I should really rename this to “respond to messages in time” generally. Oh well… 😇

Continue updating the ScpToolkit

Yes, yes, I haven’t forgotten about this delicate topic and I am aware about the community demands. It’s just that the driver development ate up so much time and brain power I didn’t have enough energy left for support, fixes or new features at all. Hope you can understand that 🙃

Update this post

There sure is a lot I forgot to include in this post but that’s all my brain can take at this time so… time for more procrastination! 😅

Cya!