unity3d - Accessing Color Frames with Unity and Tango 'Leibniz' -

i'm starting tinker tango , unity. unfortunately there doesn't seem documentation or examples on how access color data in unity latest release.

i've been using motion tracking example github (https://github.com/googlesamples/tango-examples-unity) starting point, trying read incoming color frames same way pose , depth data read. i'm assuming best way go through "itangovideooverlay" interface , "ontangoimageavailableeventhandler" callback.

all trying right "ontangoimageavailableeventhandler" callback working , can't quite figure out. have "enable video overlay" checked on tango manager , following script connected gui text object debugging.

using system.collections; using unityengine; using unityengine.ui; using tango; using system;  public class videocontroller : monobehaviour, itangovideooverlay {      private tangoapplication m_tangoapplication;      private bool debugb = false;     public text debugtext;      // use initialization     void start () {         m_tangoapplication = findobjectoftype<tangoapplication>();         m_tangoapplication.register(this);     }      // update called once per frame     void update () {         if (debugb)             debugtext.text = "true";         else             debugtext.text = "false";     }      // no unity api     public void ontangoimageavailableeventhandler(tango.tangoenums.tangocameraid id, tango.tangounityimagedata image)     {         debugb = true;     } } 

is there initialization of camera missing? or preferred method still use videooverlaylistener in older code: getting color data in unity

i know it's possible directly access camera through unity (disabling depth). learn "proper way" first.

thank time!

update 04/28/15 - latest version of script, callback works! still needs conversion rgb color

this script written addition google's tango motion tracking example on github. attach script unity camera , link public field "m_viewscreen" mesh object (like plane) video texture display on.

using system.collections; using unityengine; using tango; using system;  public class videocontroller : monobehaviour {     private tangoapplication m_tangoapplication;     private texture2d m_texture;     private material m_screenmaterial;     private meshfilter m_meshfilter;     private bool m_readytodraw = false;      // link mesh object displaying texture     public gameobject m_viewscreen;      // use initialization     void start ()     {         // tango initilization         m_tangoapplication = findobjectoftype<tangoapplication>();         m_tangoapplication.register(this);         m_tangoapplication.registerpermissionscallback(_ontangoapplicationpermissionsevent);          // initialize view object material         m_meshfilter = m_viewscreen.getcomponent<meshfilter> ();         m_screenmaterial = new material(shader.find("mobile/unlit (supports lightmap)"));          // begin texture webcam         m_texture = m_tangoapplication.getvideooverlaytexture();         m_texture.apply();          if (m_screenmaterial != null)         {             // apply texture             m_screenmaterial.maintexture = m_texture;             m_meshfilter.getcomponent<meshrenderer>().material = m_screenmaterial;              // connect texture camera             if (m_tangoapplication.m_useexperimentalvideooverlay)             {                 videooverlayprovider.experimentalconnecttexture(tangoenums.tangocameraid.tango_camera_color, m_texture.getnativetextureid(), _onunityframeavailable);             }             else             {                 videooverlayprovider.connecttexture(tangoenums.tangocameraid.tango_camera_color, m_texture.getnativetextureid());             }         }     }      private void _ontangoapplicationpermissionsevent(bool permissionsgranted)     {         m_readytodraw = true;     }      private void _onunityframeavailable(system.intptr callbackcontext, tango.tangoenums.tangocameraid cameraid)     {         // fun stuff here!      }      void onprerender()     {         if (m_readytodraw)         {             videooverlayprovider.renderlatestframe(tangoenums.tangocameraid.tango_camera_color);             gl.invalidatestate();         }     }      void update ()     {         // other fun stuff here!     } } 

from find, changed whole system it's easier access image, image grabbing handle tango object.

in start after grabbing tango app, try this:

m_texture = m_tangoapplication.getvideooverlaytexture(); m_texture.apply();  if (m_screenmaterial != null) {     // apply texture     m_screenmaterial.maintexture = m_texture;     m_meshfilter.getcomponent<meshrenderer>().material = m_screenmaterial;      // connect texture camera     if (m_tangoapplication.m_useexperimentalvideooverlay)     {         videooverlayprovider.experimentalconnecttexture(tangoenums.tangocameraid.tango_camera_color, m_texture.getnativetextureid(), _onunityframeavailable);     }     else     {         videooverlayprovider.connecttexture(tangoenums.tangocameraid.tango_camera_color, m_texture.getnativetextureid());     }  } 

and you'll need somewhere else callback event:

private void _onunityframeavailable(system.intptr callbackcontext, tango.tangoenums.tangocameraid cameraid) {     // fun stuff here! } 

but doesn't need anything. of course, image still in yuv-nv12 format you'll need convert rgb (or wait til next release should fix it).

edit: oops! forgot need 1 more call update texture on ar material:

in start() after grabbing tangoapp:

m_tangoapplication.registerpermissionscallback(_ontangoapplicationpermissionsevent); 

then:

private void _ontangoapplicationpermissionsevent(bool permissionsgranted) {     m_readytodraw = true; }  void onprerender() {     if (m_readytodraw)     {         videooverlayprovider.renderlatestframe(tangoenums.tangocameraid.tango_camera_color);         gl.invalidatestate();     } } 

hope works now!