There is a branching path where, if you go see the scholar, he will sometimes send you fetch an alchemical ingredient. Upon giving it to him, he proposes to teleport you directly on the quest item in the other dungeon.
Problem is: I have been teleported to the wrong spot

Not a huge deal (I can pretend the scholar was absent minded

I have unfortunately overwritten my save before I realized something was off, but I imagine this can be easily replicated.