Announcement

Collapse
No announcement yet.

NIUI: An OpenNI / Kinect UDK Library

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    NIUI: An OpenNI / Kinect UDK Library



    Hi all!

    So I think that it is time to announce a little project I have been working on called NIUI (pronounced new-ee).

    NIUI is an OpenNI / Kinect interface for the Unreal Development Kit through UnrealScript and C++. It will allow for Natural Interaction within the Unreal Engine, providing access to the depth buffer, image buffer, user label map and user skeleton mapping.

    I intend on releasing an alpha version of NIUI in roughly a months time. That said, I will only release the API when I am happy with it and I feel that I have provided enough examples of how to use the NIUI API.

    In the meantime, here is the current state of NIUI:
    http://www.youtube.com/watch?v=CKCkYuKNnCk

    I will try to post weekly updates on what I'm doing for NIUI and the features that I am coding. I have quite a bit to do (code a c++ backend, an UnrealScript frontend, document the API and provide some demos) so I may forget to keep everyone updated.

    PS:
    If people have suggestions of what they would like to see in the library, feel free to suggest them to me. I won't be adding suggested features until after the alpha is completed (I want to get the core functionality working stably first) but if I am aware of desired features, I can plan ahead and design the API accordingly .

    UPDATE 1.
    Just did a quick post updating everyone on where I stand with the project (see post 13).

    UPDATE 2:
    Bump post updating people on the API. Added a video link of bone rotations being applied in realtime.
    OpenNI to UDK Real-time Bone Rotation Demo (NIUI)

    #2
    Hi,

    Firstly I'd just like to take a moment to say that this is a brilliant idea and I think that this will be used alot once its out there, and that personally I Can't wait to use it myself for possible development within the project I'm currently working on.

    Secondly as a suggestion for later development is maybe facial mapping, for example Microsoft just annouce Avatar Kinect for the Xbox 360, as I'm most people are aware this is a chatroom which you control you're avatar with up to 8 friends and it has facial mapping. Reason I suggest this might be a good way for developers to give the player more control over their characters during a cutscene.

    keep up the great work man, really look forward to seeing more!

    Comment


      #3
      Originally posted by SweptSquash View Post
      Hi,

      Firstly I'd just like to take a moment to say that this is a brilliant idea and I think that this will be used alot once its out there, and that personally I Can't wait to use it myself for possible development within the project I'm currently working on.

      keep up the great work man, really look forward to seeing more!
      All I can say is... This

      Comment


        #4
        Woah... Facial interactions during cutscenes... Never thought of that before...

        Comment


          #5
          Keep up the good work! I'm sure alot of people will find good use for this once you release it

          Comment


            #6
            This is awesome. If you don't mind me asking, what's your general approach for getting the data from C++ into UnrealScript? Does the script just poll at regular intervals? Did you figure out how to get events working with DLLBind? Or are you doing something else entirely?

            Comment


              #7
              Originally posted by Wildebeest View Post
              This is awesome. If you don't mind me asking, what's your general approach for getting the data from C++ into UnrealScript? Does the script just poll at regular intervals? Did you figure out how to get events working with DLLBind? Or are you doing something else entirely?
              I'm trying to keep the majority of the code and processing in c++ as its native code and will run faster than doing the same thing in the Unreal VM. There is a NIUICore UnrealScript object that calls an update each frame that updates the OpenNI device and all of the components of NIUI. As for getting the data from C++ to Unreal, there is just a heap of accessor methods defined in "C" code for this.

              With NIUI, I'm trying to build another level of abstraction in c++ over OpenNI so that it becomes more games specific and easier to interact with from the Unreal side of things.

              The structure of NIUI is basically this:

              NIUI UnrealScript Wrapper.
              ---
              NIUI C Wrapper (provides update calls and accessor methods to the C++ core, aka the dllimport functions).
              ---
              NIUI C++ Core (OpenNI abstraction).
              ---
              OpenNI Library.

              I hope that answered your question.

              Comment


                #8
                **** impressive!

                Comment


                  #9
                  Amazing, truley. Would be awesome to do animation through this. Very impressive. Keep it up!

                  Comment


                    #10
                    great to see someone is doing what we knew was possible. Keep it up. If and when the Kinect driver comes out what happens with this and are you using FAAST by chance or library all you?

                    Comment


                      #11
                      Originally posted by onethought View Post
                      I hope that answered your question.
                      Yup, thanks.

                      Comment


                        #12
                        Originally posted by Hboybowen View Post
                        great to see someone is doing what we knew was possible. Keep it up. If and when the Kinect driver comes out what happens with this and are you using FAAST by chance or library all you?
                        I've been developing this with the Kinect driver floating around on the OpenNI forums, its basically a modification of the PrimeSense driver that was released with OpenNI a short time ago. This version isn't all that easy to use / setup so I would like to swap it over to the proper PrimeSense driver when the WAVI Xtion depth camera is released.

                        In terms of the library that I am using, see OpenNI.

                        Comment


                          #13
                          Hi everyone!

                          I thought it was time for a progress update so here it goes...
                          Nearly all of the c++ backend is done (the only things I haven't done is the buffer streaming).
                          I am currently about 1/4 of the way through the UnrealScript API side (its taking a while to do as I need to test everything as I code it).

                          I am also on track in getting the alpha release of NIUI out when I had hoped for providing that nothing major comes up in the next 2 & half weeks. The alpha should be out roughly mid-feb.

                          And lastly, I will soon be joining forces with the wise SwizzleFish, which will be when the core of the API is done. We will be working together with NIUI to provide some demos / tutorials as soon as I finish the API.

                          Regards,
                          OneThought.

                          Comment


                            #14
                            The structure of NIUI is basically this:

                            NIUI UnrealScript Wrapper.
                            ---
                            NIUI C Wrapper (provides update calls and accessor methods to the C++ core, aka the dllimport functions).
                            ---
                            NIUI C++ Core (OpenNI abstraction).
                            ---
                            OpenNI Library.
                            For those of us who can not read things correctly. Could you clarify if you are using DLLBind or Native DLL coding?

                            And by the way this looks great, good work.

                            Comment


                              #15
                              Hi, I am having the same idea as you do.As I have no license for the Unreal Engine, so I just cannt program the native code, how can you nanaged to do that? Do you have unreal license .Right now I am using the share memory to communicate with unreal engine, do all the heavy computing in C++, after bone caculation is done, then apply those bone transformation to unreal skeleton mesh. I am now reaching the final stage now, and I hope I can finish this by the Feb mid.I am glad to share my work with you !

                              Comment

                              Working...
                              X