This tutorial demonstrates how to enable gaze cursor on DPVR M2, E3, P1 HMD using the DPVR Unity plugin.
Unlike traditional games, VR games often use the VR reticle (or gaze cursor) which is located in the center of the screen to interact with the GUI. Unity's Canvas is currently the most widely used GUI system, but since the system is designed for traditional games, many developers have encountered interaction problems more or less.
In VR games, we recommend using Canvas in WorldSpace to incorporate the entire GUI into the game. Poor designed GUI in ScreenSpace will cause ghost effect because of the unmatched binocular vision. Here we only discuss Canvas in WorldSpace.
The tutorial is a simple tutorial based on the DPVR plugin DpnUnity 0.7.2c. For more details, please refer to the GazeCursor scene demo in the plugin.
The tutorial is applicable to HMD interaction only.
Download, unzip, and open the Unity (Unity5.3.0) project TestDpnGraphicRaycaster2, which has a scene called "Tests-DPN-TestDpnGraphicRaycaster". Open the scene.
1. Scene Introduction:
a) Canvas：The Render Mode is WorldSpace. There are four Buttons which can interact with Cursor. When interacting with Cursor, the button color changes.
b) DpnCameraRig: There are 3 Cameras (LeftEyeAnchor, RightEyeAnchor, CenterEyeAnchor), which will move with the HMD. A cursor is added to the CenterEyeAnchor to indicate the vision center.
2. System settings:
In order not to cause conflicts, Peripheral Support should set to None(Figure 2).
In order to interact with Cursor, we added a new Component "DpnGraphicRaycaster" to the Canvas to replace Unity's GraphicRaycaster. So we need to disable Unity's "GraphicRaycaster", as shown in Figure (3) to uncheck or directly delete the Component. Then we drag the CenterEyeAnchor to the Interact Camera of DpnGraphicRaycaster. This way all interactions are between this Interact Camera and the Canvas.
After completing the above steps, you can see that Cursor can interact with these buttons. Button can receive OnMouseEnter, OnMouseOver and OnMouseExit events. The operation effect is as shown below (Figure 4)
The code is actually very simple, and all the work is done in about 100 lines. Mainly the Raycast and Update functions of DpnGraphicRaycaster.
We inherited from the Unity standard class GraphicRaycaster to perform collision detection for Canvas.
public class DpnGraphicRaycaster : GraphicRaycaster
Overload the Raycast function
This function was originally used for collision detection of mouse or touch screen with Canvas. But in VR applications, Cursor is used, so we need to override this function. In addition, when dealing with curved Canvas, you also need to override this function.
It first gets all the Graphic references from Canvas.
IList<Graphic> list = GraphicRegistry.GetGraphicsForCanvas(_canvas);
Then the intersection algorithm of the rectangle and the ray determines whether the ray collides with this rectangle.
Vector3 corners = new Vector3;
Then we sort the Graphic in collision and put them in the return list.
sorted_graphic.Sort((g1, g2) => g2.graph.depth.CompareTo(g1.graph.depth));
At this point, it can be seen that when Cursor enters, removes, and clicks on these Graphics, it will react in appearance. This is the work done internally by Unity3D.
Overload the Update function
The Raycast function has been implemented previously, and the appearance of the Graphic will change with the Cursor. However, these Graphics do not respond to the OnMouseEnter, OnMouseOver, and OnMouseExit events. So we need to overload the Update function.
This function is much simpler. First, use the Raycast function to get the collision Graphic List.
Then compare the previous frame with the list of this frame, find which Graphic is newly added to this frame, and send the "OnMouseEnter" message and the "OnMouseOver message". Only the "OnMouseOver" message is sent to Graphic which is valid for the last two frames.
// enter and over
The "OnMouseExit" message is sent to the Graphic which is valid on the previous frame, and invalid in this frame, .
At this point, you can use the Cursor to simulate the mouse message.
We can add a script to those Graphic to check if we can receive these messages.
public class OnMouseEvents : MonoBehaviour