Farseer + TUIO integration

Topics: Developer Forum, User Forum
Feb 16, 2011 at 11:43 AM
Hello all,

I have been working on a way to integrate the TUIO interface with Farseer. I am using the TUOI protocol to get data from a webcam, which picks up the locations and IDs of individual markers, then depending on the markers' locations, apply forces and torques to the objects accordingly, with the aim of creating a simple physics-based pong game.

The problem I am having is that both bits of code appear to have been written in a way as to make it very easy to have additions made and other software be integrated into them, however integrating them into other software is a lot more difficult.

As I understand it, the Client side of TUIO works by integrating the TuioListener class into the class using it. Similarly, Farseer works by the user overriding relevant functions, to get content on screen.

The problem I have is that I cannot feed the variables from TUIO to the Farseer objects, even by parsing them through a temporary class. No matter what method I have tried, somewhere, somehow values which are given by the TUIO are not stable within the application and whenever they reach the Farseer Object, they appear to be reset to 0, despite the only place they exist being within a stand-alone class which I wrote.

So the path that the data is currently taking is as follows:

Webcam > Reactivision > UDP3333 > TUIO class, integrated into the farseer-based game-screen > My class to store the fiducialID, type of message (add/update/remove object) X, Y and Rotation values.

Up to this point the data travels fine. However from here on, every time that class is called by the same game screen, this time working on the Farseer side of things, the value comes out as 0. As a temporary hack, I have the HandleKeyboardEvent class handling the TUIO signals (basically, told it to look at my class when the key 'Pause' is not pressed, will work it into)

If anyone has as to any clue in how to combine the TUIO and the farseer or box2D (or any other physics engine for that matter!), together it would be of great help. I have been trawling over the supplied documentation for both systems, and of the 10+ different methods I have tried to merge the two, all have failed so far...


Many, many thanks if you could help at all, am more than happy to provide relevant parts of the code, if needed.


Nikolay.
 
---------------------------------------------------------------------------------------
 
here is the code I am using:
 
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using FarseerPhysics.DebugViews;
using FarseerPhysics.DemoBaseXNA;
using FarseerPhysics.DemoBaseXNA.ScreenSystem;
using FarseerPhysics.Dynamics;
using FarseerPhysics.Factories;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Input;
using FarseerPhysics.DemoBaseXNA.Screens;
using FarseerPhysics.Dynamics.Joints;
using FarseerPhysics.Collision.Shapes;
using TUIO;

namespace FarseerPhysics.SimpleSamplesXNA
{
    public class DemoRogerAndNikolay : PhysicsGameScreen, IDemoScreen, TuioListener
    {
        private Fixture ball;
        private Fixture paddle1;
        private FixedPrismaticJoint paddle1SliderJoint;
        private RevoluteJoint paddle1RevoluteJoint;
        private TuioBridge myTuioB = new TuioBridge();

        private const int addObject = 0;
        private const int updateObject = 1;
        private const int removeObject = 2;

        public float[,] TuioXPos = new float[3,9];
        public float[,] TuioYPos = new float[3,9];
        public float[,] TuioAngl = new float[3,9];


        #region IDemoScreen Members

        public string GetTitle()
        {
            return "Roger and Nikolay's Work";
        }

        public string GetDetails()
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendLine("This is our own work");
            sb.AppendLine("A fixture is a combination of a body and a shape.");
            sb.AppendLine(string.Empty);
            sb.AppendLine("GamePad:");
            sb.AppendLine("  -Rotate: left and right triggers");
            sb.AppendLine("  -Move: left thumbstick");
            sb.AppendLine(string.Empty);
            sb.AppendLine("Keyboard:");
            sb.AppendLine("  -Rotate: left and right arrows");
            sb.AppendLine("  -Move: A,S,D,W");
            return sb.ToString();
        }

        #endregion

        public override void LoadContent()
        {
            TuioClient client = new TuioClient(3333);
            DemoRogerAndNikolay tuioListener = new DemoRogerAndNikolay();
            World = new World(Vector2.Zero);
            base.LoadContent();
            client.addTuioListener(tuioListener);
            client.connect();
            LoadBall();
            LoadPaddles();            
        }

        public void LoadBall()
        {
            DebugMaterial ballMaterial = new DebugMaterial(MaterialType.Face)
            {
                Color = Color.Yellow,
                Scale = 3.5f
            };

            ball = FixtureFactory.CreateCircle(World, 2, 0.5f, ballMaterial);
            ball.Body.BodyType = BodyType.Dynamic;
            ball.Restitution = 1.0f;

        }

        public void LoadPaddles()
        {
            #region Paddle1 Slider
            Vector2 posTop = new Vector2(0, 20);
            DebugMaterial paddleMaterial = new DebugMaterial(MaterialType.Circles)
            {
                Color = Color.Gold,
                Scale = 2.5f
            };
            //Fixture paddle1Slider = FixtureFactory.CreateCircle(World, 0.1f, 0, paddleMaterial);
            //paddle1Slider.Body.BodyType = BodyType.Dynamic;
            //paddle1Slider.Body.Position = posTop;
            PolygonShape shape = new PolygonShape(5);
            shape.SetAsBox(0.1f, 0.1f);

            Body paddle1Slider = BodyFactory.CreateBody(World);
            paddle1Slider.BodyType = BodyType.Dynamic;
            paddle1Slider.Position = posTop;

            paddle1Slider.CreateFixture(shape);
            paddle1SliderJoint = new FixedPrismaticJoint(paddle1Slider, paddle1Slider.Position, new Vector2(1.0f, 0.0f));
            World.AddJoint(paddle1SliderJoint);
            #endregion

            #region Paddle1 Body
            paddle1 = FixtureFactory.CreateRectangle(World, 5.0f, 1.0f, 5.0f, paddleMaterial);
            paddle1.Body.BodyType = BodyType.Dynamic;
            paddle1.Restitution = 0.4f;
            paddle1.Body.Mass = 5;
            paddle1.Body.Position = posTop;

            paddle1RevoluteJoint = new RevoluteJoint(paddle1.Body, paddle1Slider, paddle1.Body.GetLocalPoint(paddle1Slider.Position), Vector2.Zero);
            paddle1RevoluteJoint.CollideConnected = false;
            World.AddJoint(paddle1RevoluteJoint);
            #endregion
        }

        public override void HandleGamePadInput(InputHelper input)
        {
            Vector2 force = 50 * input.CurrentGamePadState.ThumbSticks.Left;
            paddle1.Body.ApplyForce(force);

            float rotation = 40 * input.CurrentGamePadState.Triggers.Left;
            paddle1.Body.ApplyTorque(rotation);

            rotation = -40 * input.CurrentGamePadState.Triggers.Right;
            paddle1.Body.ApplyTorque(rotation);

            base.HandleGamePadInput(input);
        }

        public override void HandleKeyboardInput(InputHelper input)
        {
            const float forceAmount = 100;
            Vector2 force = Vector2.Zero;

            #region TUIO Controler
            // temporary hack to keep a constant look at the TUIO inputs
            if (input.CurrentKeyboardState.IsKeyUp(Keys.Pause))
            {
                for (int i = 0; i < 7; i++)
                {
                    TuioXPos[addObject, i] = myTuioB.getTuioXPos(addObject, i);
                    //if (TuioXPos[updateObject, i] != null)
                    //{
                    if (TuioXPos[addObject, i] > 0)
                    {
                        paddle1.Body.ApplyLinearImpulse(new Vector2(1, 0));
                    }
                        //else if (TuioXPos[updateObject, i] > 0.5)
                        //{
                        //    force += new Vector2(forceAmount * 5, 0);
                        //}
                        //TuioXPos[updateObject, i] = 0;
                        //paddle1.Body.ApplyLinearImpulse(new Vector2(100, 0));
                    //}
                }
            }
            #endregion

            #region Keyboard Control
            if (input.CurrentKeyboardState.IsKeyDown(Keys.A))
            {
                if (paddle1.Body.LinearVelocity.X <= 0)
                {
                    force += new Vector2(-forceAmount, 0);
                }
                else
                {
                    force += new Vector2(-forceAmount*5, 0);
                }
            }
            if (input.CurrentKeyboardState.IsKeyDown(Keys.S))
            {
                force += new Vector2(0, -forceAmount);
            }
            if (input.CurrentKeyboardState.IsKeyDown(Keys.D))
            {
                if (paddle1.Body.LinearVelocity.X >= 0)
                {
                    force += new Vector2(forceAmount, 0);
                }
                else
                {
                    force += new Vector2(forceAmount * 5, 0);
                }
            }
            if (input.CurrentKeyboardState.IsKeyDown(Keys.W))
            {
                force += new Vector2(0, forceAmount);
            }

            paddle1.Body.ApplyForce(force);

            const float torqueAmount = 1000;
            float torque = 0;

            if (input.CurrentKeyboardState.IsKeyDown(Keys.Q))
            {
                if (paddle1.Body.AngularVelocity>0)
                {
                    torque += torqueAmount;
                }
                else
                {
                    torque += torqueAmount*5;
                }
            }
            if (input.CurrentKeyboardState.IsKeyDown(Keys.E))
            {
                if (paddle1.Body.AngularVelocity < 0)
                {
                    torque -= torqueAmount;
                }
                else
                {
                    torque -= torqueAmount * 5;
                }
            }

            if (input.IsNewKeyPress(Keys.F9))
            {
                ScreenManager.AddScreen(new PauseScreen(GetTitle(), GetDetails()), 0);
            }

            paddle1.Body.ApplyTorque(torque);

            base.HandleKeyboardInput(input);
            #endregion
        }

        /**
		 * This callback method is invoked by the TuioClient when a new TuioObject is added to the session.
		 *
		 * @param  tobj  the TuioObject reference associated to the addTuioObject event
		 */
        public void addTuioObject(TuioObject tobj) {
            int TuioSymbolID = tobj.getSymbolID();
            int actionID = addObject;
            TuioXPos[actionID, TuioSymbolID] = tobj.getX();
            TuioYPos[actionID, TuioSymbolID] = tobj.getY();
            TuioAngl[actionID, TuioSymbolID] = tobj.getAngle();

            myTuioB.saveData(actionID, TuioSymbolID, TuioXPos[actionID, TuioSymbolID], TuioYPos[actionID, TuioSymbolID], TuioAngl[actionID, TuioSymbolID]);
        }

        /**
         * This callback method is invoked by the TuioClient when an existing TuioObject is updated during the session.
         *
         * @param  tobj  the TuioObject reference associated to the updateTuioObject event
         */
        public void updateTuioObject(TuioObject tobj)
        {
            int TuioSymbolID = tobj.getSymbolID();
            int actionID = updateObject;
            TuioXPos[actionID, TuioSymbolID] = tobj.getX();
            TuioYPos[actionID, TuioSymbolID] = tobj.getY();
            TuioAngl[actionID, TuioSymbolID] = tobj.getAngle();

            myTuioB.saveData(actionID, TuioSymbolID, TuioXPos[actionID, TuioSymbolID], TuioYPos[actionID, TuioSymbolID], TuioAngl[actionID, TuioSymbolID]);
        }

        public void removeTuioObject(TuioObject tobj)
        {
            int TuioSymbolID = tobj.getSymbolID();
            int actionID = removeObject;
            TuioXPos[actionID, TuioSymbolID] = tobj.getX();
            TuioYPos[actionID, TuioSymbolID] = tobj.getY();
            TuioAngl[actionID, TuioSymbolID] = tobj.getAngle();

            myTuioB.saveData(actionID, TuioSymbolID, TuioXPos[actionID, TuioSymbolID], TuioYPos[actionID, TuioSymbolID], TuioAngl[actionID, TuioSymbolID]);
        }

        public void addTuioCursor(TuioCursor tcur)
        {
            Console.WriteLine("add cur " + tcur.getCursorID() + " (" + tcur.getSessionID() + ") " + tcur.getX() + " " + tcur.getY());
        }

        public void updateTuioCursor(TuioCursor tcur)
        {
            Console.WriteLine("set cur " + tcur.getCursorID() + " (" + tcur.getSessionID() + ") " + tcur.getX() + " " + tcur.getY() + " " + tcur.getMotionSpeed() + " " + tcur.getMotionAccel());
        }

        public void removeTuioCursor(TuioCursor tcur)
        {
            Console.WriteLine("del cur " + tcur.getCursorID() + " (" + tcur.getSessionID() + ")");
        }

        public void refresh(TuioTime frameTime)
        {
            myTuioB.preparePreviousState();
        }
    }
}
---------------------------------------------------------------------------------------
And the bridging class I wrote, labled above as myTuioB:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace FarseerPhysics.SimpleSamplesXNA
{
    class TuioBridge
    {
        public float latestTuioXPos;
        public float[,] savedTuioXPos = new float[3, 9];
        public float[,] savedTuioYPos = new float[3, 9];
        public float[,] savedTuioAngl = new float[3, 9];
        public float[,] previousTuioXPos = new float[3, 9];
        public float[,] previousTuioYPos = new float[3, 9];
        public float[,] previousTuioAngl = new float[3, 9];

        public void saveData(int actionID, int TuioSymbolID, float tuioXpos, float tuioYpos, float tuioAngl)
        {
            savedTuioXPos[actionID, TuioSymbolID] = tuioXpos;
            if (tuioXpos != 0.0f)
            {
                latestTuioXPos = tuioXpos;
            }
            savedTuioYPos[actionID, TuioSymbolID] = tuioYpos;
            savedTuioAngl[actionID, TuioSymbolID] = tuioAngl;
            Console.WriteLine("The XPos of " + TuioSymbolID + " saved =" + tuioXpos + ".");
        }

        public void preparePreviousState() 
        {
            previousTuioXPos = savedTuioXPos;
            previousTuioYPos = savedTuioYPos;
            previousTuioAngl = savedTuioAngl;
        }

        public float getTuioXPos(int actionID, int TuioSymbolID)
        {
            if (latestTuioXPos > 0)
            {
                Console.WriteLine("The XPos of " + TuioSymbolID + " used =" + savedTuioXPos[actionID, TuioSymbolID] + ".");
            }
            return latestTuioXPos;
        }

        public float getTuioYPos(int actionID, int TuioSymbolID)
        {
            return savedTuioXPos[actionID, TuioSymbolID];
        }

        public float getTuioAngle(int actionID, int TuioSymbolID)
        {
            return savedTuioXPos[actionID, TuioSymbolID];
        }
    }
}
Developer
Feb 23, 2011 at 7:49 AM

I don't know anything about Tuio and have not read all your code in detail, cause to be honest it looks like it has been carelessly copy pasted together and your question sounds a bit like "can you please write my program for me".

Some remarks anyway: First of Farseer is not the Screensystem, Gamescreen etc. stuff. That is our samples framework, which we use to demonstrate Farseers features. It is ok to look at it to get how things are done, but it is not meant as a general solution. There are a million ways in which Farseer can be used and you have to find out for yourself how it is best integrated in your game.

Regarding the code:

public override void LoadContent()
{
     TuioClient client = new TuioClient(3333);
     DemoRogerAndNikolay tuioListener = new DemoRogerAndNikolay();
     World = new World(Vector2.Zero);
     base.LoadContent();
     client.addTuioListener(tuioListener);
     client.connect();
     LoadBall();
     LoadPaddles();            
}

This looks kinda weird and wrong. You create a new instance of your demoscreen here and pass that to your TuioClient... why? That new instance will of course allways produce empty/zero input, values etc. For starters try this:

public override void LoadContent()
{
     TuioClient client = new TuioClient(3333);
     // DemoRogerAndNikolay tuioListener = new DemoRogerAndNikolay(); /*REMOVE ME*/
     World = new World(Vector2.Zero);
     base.LoadContent();
     client.addTuioListener(this);
     client.connect();
     LoadBall();
     LoadPaddles();            
}

There might be further errors in your code though.