Picking unsigned integer handling fix.

Julien Gouesse send me this patch to the jogamp ML on 2010-09-09 16:24 Thu +0200:

Picking.java is a provided example of JOGL 2 here:
http://jogamp.org/jogl-demos/src/demos/misc/Picking.java
http://jogamp.org/jogl-demos/src/demos/misc/Picking.java

The both lines above are wrong:
z1 = (float) buffer.get(offset) / 0x7fffffff; offset++;
z2 = (float) buffer.get(offset) / 0x7fffffff; offset++;

Actually, the select buffer contains 32-bits unsigned integers stored into
32-bits signed integers. Then, the last digit is wrongly used as a sign bit.
As Java has no unsigned type, at least 33 bits are required to store these
positive values with their sign. Therefore, the long type (64 bits) should
be used. The both lines above often work correctly except when the depth
values are bigger than 2^31. I assume that they have been copied from an
example written in C++. We should rather do this:
z1 = (float) (buffer.get(offset)& 0xffffffffL) / 0x7fffffff; offset++;
z2 = (float) (buffer.get(offset)& 0xffffffffL) / 0x7fffffff; offset++;

1 files changed, 2 insertions, 2 deletions

diff --git a/src/demos/misc/Picking.java b/src/demos/misc/Picking.java
index 472ed95..5c55272 100755
--- a/src/demos/misc/Picking.java
+++ b/src/demos/misc/Picking.java
@@ -133,8 +133,8 @@ public class Picking
           System.out.println("- - - - - - - - - - - -");
           System.out.println(" hit: " + (i + 1));
           names = buffer.get(offset); offset++;
-          z1 = (float) buffer.get(offset) / 0x7fffffff; offset++;
-          z2 = (float) buffer.get(offset) / 0x7fffffff; offset++;
+          z1 = (float) (buffer.get(offset)& 0xffffffffL) / 0x7fffffff; offset++;
+          z2 = (float) (buffer.get(offset)& 0xffffffffL) / 0x7fffffff; offset++;
           System.out.println(" number of names: " + names);
           System.out.println(" z1: " + z1);
           System.out.println(" z2: " + z2);