Two clips (video or audio) can overlap only when they are placed in two different tracks. For a video transition to work, you must place it in between the overlapping section of the two video tracks.
For example, the Picture-in-Picture video transition lets you embed a video present in Track 2 inside the video present in Track 2. Another example is the Key-Color transition that lets you render the color of the video present in Track 1 as transparent. This makes the video in Track 1 appear through the transparent portion of video in Track 2.
For more information please refer to the article "Applying Transitions Correctly" under the "Articles" section of Video Edit Magic's help file. You can also read this article online by visiting the URL:
http://www.deskshare.com/Resources/articles/vem_applying_transitions.aspx
DeskShare - Support