this is indeed a weird behavior. Unfortunately I don't have an explanation for you. I tried the same on my machines (1809 and 1909) and I don't see this. For me, the Explorer will always be launched nested within the pinned taskbar item.
Running Windows 10 1809. Behaviour is that if I press ⊞+E with focus on the RoyalTS window, NOT in a TS session, then Windows ignores the pinned Explorer icon on the taskbar and opens a new Explorer window on my local PC.
If I take the focus off RoyalTS it will open explorer nested within the pinned taskbar icon on my local laptop.