Let me introduce you to the steps that Appium needs to go through.
scan windows and create a source map
apply the object's locator to search
Then, Winappdriver will apply the command on the desktop application.
That's the reason why it takes time when acting like switching windows.