Game is designed to use input device you are using and locks to it outside main menu. When joining friends game via Steam overlay and clicking invitation game locks itself to keyboard and you have to return to main menu to change back to controller.
Workaround for this is to start Steam in Big Picture Mode so you can select invitation with controller. You may also try to join friends game via matchmaking.
The fix was posted on the Steam Community forums, consists of using an hex editor and replacing 6 bytes starting at offset 0x24F2BE with 0x90 (NOP Instruction).
A binary patch with this change applied can be found here, you need bspatch to apply it.
bspatch [original_executable] [patched_executable_to_create] [provided_patch_file]
mv [patched_executable_created] [original_executable]