last day (15 days later) » 

Stack Exchange
Stack Exchange
8:27 PM
0
Q: OpenGL: What's the best way to convert a screen coordinate to a world coordinate?

TylerFor example, suppose I'm building a first-person shooter and the player pulls the trigger. I want to convert the screen coordinate (x, y) into a world coordinate (x, y, z) to know what they've hit. I've seen some people mentioning a lookup using glReadPixels, but I've heard it's slow and those r...

opengl coordinates depth-buffer
 
user4704
Sounds like you're trying to implement picking. There are several questions about this on the site, including this one, which shows you how to do it via gluUnproject (the references to ES 2 are a red-herring here as this technique applies to "regular" OpenGL as well).
 
user4704
You don't need gluUnproject; you can see from the documentation that all it does is simple matrix multiplication and math. The basic technique is the same, you can just unroll the function (see, for example, this question).
 
Tyler
Tyler
Thank you for the tip, Josh. I do not know a good way to retrieve the depth buffer value in order to perform the reverse projection. I guess I could investigate glReadPixels, but I'm wondering if there is a faster way. If not, I'm wondering if there is a good way to learn how to use this function - I find the official docs heard to learn from. I've also seen references to pixel buffer objects, so I thought they might be another alternative.
 
user4704
You shouldn't need a depth buffer value. Just unproject twice: first (x, y, 0) and then (x,y, 1); this gives you two points in object space you can use to create a ray, and anything that intersects with that ray is something the mouse "hit."
 
Tyler
Tyler
I would like to request that this question be re-opened. I understand that its first version did appear to be a duplicate of the marked question, but that was simply because I had left out some critical details - mainly about which OpenGL version I am using. I have edited the question to clarify why it is different from the currently-linked previous question. I do appreciate the helpful comments. I would still find value in seeing what the gamedev community at large has to say. Thank you.
 
user4704
8:27 PM
I don't see what separates this question from the duplicate or any of the similar questions on doing 3D picking on the site. The answer is still the same. We don't want to collect a bunch of questions that of the form "how can I do X in API Y" and "how can I do X in API Z," in general. I need you to further distinguish the question before I'd feel comfortable unilaterally re-opening. For example, do you want to know explicitly how to do picking using glReadPixels (knowing that this is perhaps not the most optimal solution)?
 
Tyler
Tyler
This question is about how to accomplish a specific task with a specific API. The linked question is about another API. It seems valid to me based on the type of questions suggested in the gamedev tour [1] - specifically, it fits well into "game-specific programming issues (engine architecture, game-related APIs, networking, tools, etc)." Jumping to an abstract level, this discussion makes me feel that my question is seen as "stupid until proven worthy." This kind of interaction pushes away people who otherwise may add a lot of value to the site. [1] gamedev.stackexchange.com/tour
 
user4704
@Tyler it isn't that the question is "stupid," it's a good question.
 
user4704
However, it's a question that has already been asked here -- often.
 
user4704
With only minor variants like "...in OpenGL" or "...in D3D but not using this API."
 
user4704
Which is a problem for the overal quality of the site.
 
Tyler
Tyler
8:28 PM
I have a pretty specific task in mind. I think a good answer would be helpful to other people.
 
user4704
First, it dilutes the value of, and searchability of, each individual question (since SE is about distilling questions to their canonical question-and-answer form)
 
Tyler
Tyler
Maybe there is a duplicate somewhere on this site, but I didn't see it when I looked through the top-ranked questions in the 3d picking link.
 
user4704
Second, it creates a feedback loop of precisely the kind you are experiencing now, where while there are many potential duplicates they're all slightly different enough to confuse, at various time, various people into thinking they aren't.
 
user4704
So, what is specific or particular about your scenario?
 
Tyler
Tyler
Just a minute.
 
user4704
8:30 PM
Sure.
 
Tyler
Tyler
I'll try to phrase things carefully.
Basically, I believe it's possible to retrieve an (x, y, z) world coordinate based on a framebuffer and a pixel (x, y) coordinate.
I could probably use glReadPixels or a pixel buffer object to accomplish this, although I've only read briefly about either one, so I have more to learn.
 
user4704
So you specifically want to get a world coordinate from a frame buffer?
 
Tyler
Tyler
I don't know (a) exactly how OpenGL determines what value to be written in the framebuffer's depth buffer (so I can invert it), and (b) a good way - eg example code - to learn how to use either glReadPixels or a PBO (pixel buffer object) or whatever is appropriate.
Yes, I do want to get a world coordinate from a specific point in a framebuffer.
 
user4704
(a) comes from a shader, and (b) is involved-but-doable (and not always particularly fast).
 
user4704
Okay.
 
Tyler
Tyler
8:34 PM
One pixel.
 
user4704
So my next question then, is, why?
 
user4704
(i.e., is this an XY-problem?)
 
Tyler
Tyler
I'm not sure what you're asking. If you mean, why do I want to know, then it's because the player is looking at something, and I want to efficiently code interaction with what the player is looking at.
Is that what you were asking?
 
user4704
 
user4704
Basically I want to know if you are asking about what you think the solution is (reading world position from a framebuffer) rather than what the actual problem is.
 
Tyler
Tyler
8:36 PM
Well, I am not married to either glReadPixels or pixel buffer objects. I don't know what the best way to do it is. That's why I'm asking.
 
user4704
But "I want to efficiently code interaction with what the player is looking at" suggests that your question is about the former -- your perceived approach to the solution.
 
user4704
There's basically two ways to do object picking, at a high level. One is based on rendering special values to a frame buffer and reading back that frame buffer. The other is based on associating bounding volumes with your objects and checking for intersectoin between those volumes and some ray.
 
Tyler
Tyler
Ok.
 
user4704
Generally the ray-based-approach is better (although this is a bit subjective) because GPU readback is "slow" and unless you have a deferred renderer you usually need an entirely seperate "selection buffer" rendering pass for all objects to get a useful result from the glReadPixels (etc) option.
 
Tyler
Tyler
Ok, I'm thinking about this.
I appreciate you taking the time to speak with me by chat.
 
user4704
8:40 PM
np
 
user4704
I picked the question I did for the dupe-closure because while the question itself is a bit crappy (and mentioned requirements like OpenGL ES and other crud that don't apply to you) the answer to it is actually pretty decent.
 
Tyler
Tyler
Well, I feel like I have a lot of jumping-off points from here, and I'd like to basically pull together the knowledge I can find from the links I have now.
It's helpful to know, for example, that this is called "picking," which has helped me find relevant pages.
 
user4704
It contains what you could take to be pseudo-code (I think it's in Java or something) for a way to compute that "pick ray" -- which is usually the trickiest part of the ray-based picking approach.
 
Tyler
Tyler
I see.
 
user4704
The only problem that answer has for your needs is that it uses gluUnproject, but gluUnproject is just a bunch of trivial matrix math.
 
user4704
8:43 PM
It's basically just (manually) undoing the projection transformation that all 3D points go through on their trip through the GPU.
 
user4704
It's basically result = windows_position * inverse(world * view * projection)
 
user4704
plus some other crap for normalization based on the viewport bounds etc
 
Tyler
Tyler
I see what you're saying about the code performing the reverse transformations.
Ok. I do appreciate your time. I'm going to code/learn from here. Thanks.
 
user4704
np
 
user4704
I'm going back to work but if you have any follow up questions, @-ping me here or in the regular Game Development Chat and I will try to answer when I get back.
 
Lee Jacobs
Lee Jacobs
8:48 PM
does the inverse of the MVP matrix provide a 2x4 matrix? Since the screen coordinates are in <x , y> how do we reasonably expect to do this multiplication?
never mind, read the thread
 
user4704
@LeeJacobs gluUnproject asks for a 3-vector for the input; Z shoudl be depth, but for the purposes of ray picking where you're going to generate an infinite ray anyhow, you can just pick two arbitrary Z values within your view frustum. 0 and 1 for example.
 
user4704
(really leaving now!)
 

last day (15 days later) »