Appendix 6: Mouse Picking

Mouse picking refers to the ability to use the mouse to interact with objects in 3D space.  Part of the challenge is that the mouse position is in 2D screen coordinate while our 3D space has its own coordinate system.  We need a way to translate 2D screen position to position in 3D space.

Env3D provides the following methods for mouse interaction:

 java.lang.Object getPick(int x, int y)
          Get the closest object that is under the current mouse position.
 java.util.List<EnvPickResult> getPickList(int x, int y)
          Get a list of objects that are under a particular screen position.
 EnvRay getPickRay(int x, int y)

The getPick(int x, int y) method returns the closest object under the screen position (x, y).  Since it returns an Object, you'll need to cast the returned object to the proper class in order to do something useful with it.

The getPickRay(int x, int y) method returns an invisible "ray" that starts at screen coordinate (x, y).  The EnvRay class contains a starting position and a direction, all translated into 3D space.

The getPickList(int x, int y) method returns all of the objects under the screen coordinate (x, y).  The list is sorted by the distance between the object and the screen coordinate in ascending order.

The following is a simple application demonstrating how to use these mouse picking methods:

The Game class

import env3d.Env;
import env3d.EnvRay;

public class Game 
    public void play() 
        Env env = new Env();
        Doty [] objects = new Doty[100];
        // Create a "wall" of doty objects
        for (int x = 0; x < 10; x++) {
            for (int y = 0; y < 10; y++) {
                int i = 10*y+x;
                objects[i]=new Doty(x+0.5, y+0.5, 1);
        Doty selected = null;
        while (env.getKey() != 1) {            
            Doty pick = (Doty) env.getPick(env.getMouseX(), env.getMouseY());
            if (selected != pick) {
                // We have selected a different object, reset the current object
                if (selected != null) {
                // set the new object as selected
                if (pick != null) {
                selected = pick;                
            // Move the selected Doty object close to the pick origin
            // Exercise: move this code into the Doty class.
            if (selected != null) {
                // The pick ray object contains the origin and direction of the
                // mouse location, translated into 3D coordinate.
                EnvRay ray = env.getPickRay(env.getMouseX(), env.getMouseY());
                // Move the object along the direction of the ray.
    public static void main(String args[])
        (new Game()).play();

The Doty class

import env3d.EnvObject;

 * A Doty object that can be "picked" by the mouse
public class Doty extends EnvObject
    double defaultX, defaultY, defaultZ;
    public Doty(double x, double y, double z)
        defaultX = x;
        defaultY = y;
        defaultZ = z;       
     * Simply change the texture when the object is picked.
    public void picked() 
     * Resets the object to its original location
    public void reset()