This project has moved. For the latest updates, please go here.

Raycasting normal problems

Topics: Developer Forum
Feb 3, 2011 at 2:38 PM

Me and my friend have been playing around with raycasting. I've been using it to delete things from a map editor, which seems to work quite well. My friend has been using it to find objects infront of our AI characters.

As I understand it, the raycast only finds an object if the ray intersects the object from the outside in, rather than the other way around. Is that correct? Because my deletion code at the moment has a habit of deleting more than the object I click on (It's taken a small distance away from the object to the other side of the object).

My friend is having more trouble when it comes to the normals. He's getting the object fine, but finding that the normal isn't what he thinks it should be. What is the 'normal' in terms of the raycast call back? It seems to be that when he expects it to be (1,0) it returns (0,1) instead. He's not confused as it may look, we know the different between x and y. I was expecting it to be the normal of the surface hit.

Can anyone advise on these issues please?


Thanks in advance.

Feb 3, 2011 at 4:24 PM

Raycasts only return the points on the surface of shapes that it enters. To get the points that it exits, you will have to reverse the raycast.

If you use raycasts to delete objects close to the mouse pointer, I would recommend you use World.QueryAABB() instead.

What version of FPE are you using? The latest version return the normal as expected - take a look at the Testbed test called "Edge Shapes Test". Earlier versions had some problems with the normal returned.

Feb 3, 2011 at 4:36 PM

Hey, im the other guy working on this. 

Its strange. So i have a horizontal wall like:  ________________________

When the character moves from above it in the direction of the wall in a straight line it returns the normal as (0,1). Thats fine. It should be that.

But if the character moves at an angle it returns (1,0). What does the angle of the interaction have anything to do with the normal? Anyone have a similar problem or has anyone used the RayCast in this manner that has any advice? 


Feb 4, 2011 at 1:50 PM

I'm not sure what version of 3.0 we're using (One from the source control). It was the recommended one a few months ago. I'll get the most recent from the downloads page and try that.

I'll let you know how it goes.

Feb 4, 2011 at 3:10 PM

It didnt do much. Im still getting the same problem. 

Basically the direction the character approaches the wall seems to determine the normal. If the character approaches at (-108, -200) then it returns (0,1). But if that goes below the x, so (-109, -200), then i start to get the normal as (1,0). 

Can someone please explain how the direction the character approaching the wall has any effect on the normal, because i do not understand why/how that is possible.

Feb 4, 2011 at 11:04 PM

Those values seem very high. The unit system is in meters, kilograms and seconds. You need to convert your pixel units to meters.

If you are still having problems after conversion to meters, I would like you to put up a quick testbed test (use the testbed project from the downloads tab) that shows the problem.

Feb 7, 2011 at 2:42 PM
Edited Feb 7, 2011 at 3:10 PM

Its just a directional vector though. I dont understand how the direction is having any bearing.

EDIT: How do i convert from pixel to metres? Ive looked on google and there seems to eb a few different methods and i dont know which one is the correct way. Cheers

Feb 7, 2011 at 3:23 PM

Our conversion scale is 100. So basically, the values given are actually -1.09 and -2. We've abstracted over all of the meters stuff so we're working in pixel coordinates but the back-end and physics engine is running in meters so I don't think that's the problem.

Feb 7, 2011 at 5:22 PM

Take a look at the EdgeShapes test from the Testbed. It raycasts against a lot of edge shapes that makes up a terrain. If you zoom in (use x and z), you will see the normal pointing the right way.

Feb 8, 2011 at 8:38 AM

Yeah, I took a look at the test bed stuff (specifically the raycast test) and saw the normal was fine. It may be something to do with the way we've laid out our level or the fact that most of the walls are made up of lots of squares (we're very early on in development). I figure the problem is probably on our end. I'll be able to take a proper look at it on Thursday.

Feb 10, 2011 at 2:14 PM


@GenBox I tried using the QueryAABB to remove object close to the mouse, but it doesn't work very well with rotated objects (deletes them when I click around them) and struggles to delete unrotated boxes as well. Is there a better way to deal with rotated boxes without using AABB queries or ray casting? I suppose I could turn the mouse cursor into a sensor fixture. Any thoughts?

Feb 10, 2011 at 2:47 PM

You could do the following:

1. Use QueryAABB to find fixtures close to the mouse pointer
2. Use the GJK algorithm to check the distance between the mouse pointer and the fixtures (See the DistanceTest from the Testbed on how to use GJK)
3. If the fixtures are within the deletion radius from the mouse pointer, delete the body from the world.

I'm not 100% sure what you are trying to accomplish, so this might not be what you are looking for.

Feb 10, 2011 at 3:11 PM

Basically, it's a map editor. All it's doing at the moment is placing walls by clicking on their start position, and then on the second click the wall is stretched from one point to the other. The deletion thing doesn't have to be perfect because it's only being used for development. We don't plan to release it. So I'll just keep the AABB just now, but it deletes things in the AABB, which is not accurate to the actual object.

I'd upload an image of it but my server is acting up. Instead, enjoy these videos from previous farseer games I've made(Blatent Advertisement):


Enjoy. ;)


Feb 10, 2011 at 3:19 PM

Thanks for the videos. I've linked to them on our videos page, hope you don't mind.

Feb 10, 2011 at 3:51 PM

Not at all.

Feb 10, 2011 at 4:28 PM


Have a question about the original problem.

It seems the problem might be that the raycasting is hitting past the first surface of the wall. So our wall is made up of several square blocks. So the side nearest the character has a normal of (0,1) when coming from the y direction. But the raycast seems to be going through that and hitting one of the sides of the squares that has a normal or (1,0). Is there anyway to specify that the ray cast should check only the nearest surface of an object?


Feb 10, 2011 at 4:40 PM

Regarding your editor problem: Have you tried using something like Fixture.TestPoint() or Vertices.PointInPoly() to just delete the object directly under the mousepointer? I used that for the YuPeng Clipper Testbed sample. You could use AABB to narrow down the objects on which you perform the TestPoint() test. It is pretty fast though anyway and for your editor performance should not be that critical.


Awesome videos btw. Absolutely love the Pop-up book style in the first one :)

Feb 10, 2011 at 4:44 PM

Thanks for the advice and the comments.

The first video was a game for Dare to Be Digital (an event held here in Dundee open to UK Students) and was done in 10 weeks with a group of 5. The second one was made in a week with a much less experienced team which is why it looks a little shoddy, and it was my first time playing with Farseer 3.0.

I'll take a look at more of the test beds and find exactly what I need. I know it'll be in there somewhere.