简体中文 | English

中文

This tutorial demonstrates how to enable gaze cursor on DPVR M2, E3, P1 HMD using the DPVR Unity plugin.

This tutorial applies to DPVR Unity plugin version: DpvrUnity0.7.2 to DpvrUnity0.7.4 ONLY. The DpvrUnity0.7.6 or later plugin uses another way (DpnEventSystem or DpnStandaloneInputModule). Please refer to Unity Non-VR to VR Porting Tutorial or My Very First Unity VR App for more details

Foreword

Unlike traditional games, VR games often use the VR reticle (or gaze cursor) which is located in the center of the screen to interact with the GUI. Unity's Canvas is currently the most widely used GUI system, but since the system is designed for traditional games, many developers have encountered interaction problems more or less.

In VR games, we recommend using Canvas in WorldSpace to incorporate the entire GUI into the game. Poor designed GUI in ScreenSpace will cause ghost effect because of the unmatched binocular vision. Here we only discuss Canvas in WorldSpace.

Tutorial

The tutorial is a simple tutorial based on the DPVR plugin DpnUnity 0.7.2c. For more details, please refer to the GazeCursor scene demo in the plugin.

The tutorial is applicable to HMD interaction only.

Download, unzip, and open the Unity (Unity5.3.0) project TestDpnGraphicRaycaster2, which has a scene called "Tests-DPN-TestDpnGraphicRaycaster". Open the scene.

1. Scene Introduction:

Figure (1)

a) Canvas:The Render Mode is WorldSpace. There are four Buttons which can interact with Cursor. When interacting with Cursor, the button color changes.

b) DpnCameraRig: There are 3 Cameras (LeftEyeAnchor, RightEyeAnchor, CenterEyeAnchor), which will move with the HMD. A cursor is added to the CenterEyeAnchor to indicate the vision center.

2. System settings:

Menu-DPVR-Setting

In order not to cause conflicts, Peripheral Support should set to None(Figure 2).

Click apply.

Figure (2)

3. Script:

In order to interact with Cursor, we added a new Component "DpnGraphicRaycaster" to the Canvas to replace Unity's GraphicRaycaster. So we need to disable Unity's "GraphicRaycaster", as shown in Figure (3) to uncheck or directly delete the Component. Then we drag the CenterEyeAnchor to the Interact Camera of DpnGraphicRaycaster. This way all interactions are between this Interact Camera and the Canvas.

Figure (3)

4. Results:

After completing the above steps, you can see that Cursor can interact with these buttons. Button can receive OnMouseEnter, OnMouseOver and OnMouseExit events. The operation effect is as shown below (Figure 4)

Figure (4)


Code analysis

The code is actually very simple, and all the work is done in about 100 lines. Mainly the Raycast and Update functions of DpnGraphicRaycaster.

We inherited from the Unity standard class GraphicRaycaster to perform collision detection for Canvas.

public class DpnGraphicRaycaster : GraphicRaycaster

Overload the Raycast function

This function was originally used for collision detection of mouse or touch screen with Canvas. But in VR applications, Cursor is used, so we need to override this function. In addition, when dealing with curved Canvas, you also need to override this function.

It first gets all the Graphic references from Canvas.

IList<Graphic> list = GraphicRegistry.GetGraphicsForCanvas(_canvas);

Then the intersection algorithm of the rectangle and the ray determines whether the ray collides with this rectangle.

Vector3[] corners = new Vector3[6];
g.rectTransform.GetWorldCorners(corners);
corners[4] = corners[0];
corners[5] = corners[1];
 
Plane rect_plane = new Plane();
rect_plane.Set3Points(corners[0], corners[1], corners[2]);
 
float enter;
if (false == rect_plane.Raycast(ray, out enter)) continue;
 
Vector3 intersection = ray.GetPoint(enter);
bool is_inside = true;
for (int j = 0; j < 4; ++j)
{
    Vector3 b = corners[j + 1];
    Vector3 ba = corners[j + 0] - b;
    Vector3 bc = corners[j + 2] - b;
    Vector3 bp = intersection - b;
 
    Vector3 cross_ba_bp = Vector3.Cross(ba, bp);
    Vector3 cross_bp_bc = Vector3.Cross(bp, bc);
 
    if (Vector3.Dot(cross_ba_bp, cross_bp_bc) < 0)
    {
        is_inside = false;
        break;
    }
}
 
if (false == is_inside) continue;

Then we sort the Graphic in collision and put them in the return list.

sorted_graphic.Sort((g1, g2) => g2.graph.depth.CompareTo(g1.graph.depth));
 
for (int i = 0; i < sorted_graphic.Count; ++i)
{
    var castResult = new RaycastResult
    {
        gameObject = sorted_graphic[i].graph.gameObject,
        module = this,
        distance = (sorted_graphic[i].worldPos - ray.origin).magnitude,
        index = resultAppendList.Count,
        depth = sorted_graphic[i].graph.depth,
        worldPosition = sorted_graphic[i].worldPos,
    };
    resultAppendList.Add(castResult);
}

At this point, it can be seen that when Cursor enters, removes, and clicks on these Graphics, it will react in appearance. This is the work done internally by Unity3D.

Overload the Update function

The Raycast function has been implemented previously, and the appearance of the Graphic will change with the Cursor. However, these Graphics do not respond to the OnMouseEnter, OnMouseOver, and OnMouseExit events. So we need to overload the Update function.

This function is much simpler. First, use the Raycast function to get the collision Graphic List.

// raycast
List<RaycastResult> list = new List<RaycastResult>();
Raycast(null, list);

Then compare the previous frame with the list of this frame, find which Graphic is newly added to this frame, and send the "OnMouseEnter" message and the "OnMouseOver message". Only the "OnMouseOver" message is sent to Graphic which is valid for the last two frames.

// enter and over
foreach (GameObject co in cur_objects)
{
    if (false == _last_objects.Contains(co))
    {
        co.SendMessage("OnMouseEnter");
    }
    co.SendMessage("OnMouseOver");
}

The "OnMouseExit" message is sent to the Graphic which is valid on the previous frame, and invalid in this frame, .

// exit
foreach (GameObject lo in _last_objects)
{
    if (false == cur_objects.Contains(lo))
    {
        lo.SendMessage("OnMouseExit");
    }
}

At this point, you can use the Cursor to simulate the mouse message.

We can add a script to those Graphic to check if we can receive these messages.

public class OnMouseEvents : MonoBehaviour
{
    void Start()
    {
        Debug.Log("UGUI: Start " + transform.name);
    }
    void OnMouseEnter()
    {
        Debug.Log("UGUI: OnMouseEnter " + transform.name);
    }
    void OnMouseOver()
    {
        Debug.Log("UGUI: OnMouseOver " + transform.name);
    }
    void OnMouseExit()
    {
        Debug.Log("UGUI: OnMouseExit " + transform.name);
    }
}


Copyright © dpvr.cn, All Rights Reserved 沪ICP备15019466号-2 (Shanghai) Shanghai Lexiang Technology Co., Ltd