Natural and efficient 3D interaction techniques play one of the key roles in the development of virtual reality systems. Most of these techniques, which are used currently, mainly aim at supporting the implementation of interaction tasks in geometric level. As a result, they usually lack of the sufficient ability for executing those interaction tasks that orient to high-level application. Based on the cognitive principles in the real world, interaction objects in virtual environments not only include some geometric attributes in the visual view, but also own some specific semantic attributes such as interaction rules, constraints and affordances, which are related with interaction process tightly. In this paper these objects are called semantic objects, in the sense that they know how the user can interact with them, giving clues to aid the interaction. Through parsing and interpreting interaction semantics, these semantic objects can help to realize high-level interaction metaphors above “direct manipulation”. Because they hide low-level details about the implementation of interaction techniques, this kind of metaphors evidently improve the efficiency and usability of interaction techniques, and enable user to concentrate more on the high-level interaction control directly related with the special applications.