Parity

Parity

Parity is a new installation of mine for Auckland Art Week running from 6pm until 9pm, 17th to the 19th of October at Audio Foundation.

Parity aims to be a discourse on the nature of human-computer interaction and perception. The work investigates our relationships with sound, space, virtuality and reality. Perception is the closest thing to reality people have but this is far from the true reality we inhabit if it can be said that such a thing exists.

In essence, Parity consists of a virtual acoustic environment which may only be witnessed via headphones that have been specially altered to track head location and orientation in physical space using AHRS units and 3D cameras. Virtual sound sources are realised via a custom GPU-based digital sound processing extension for Unity3D I have developed named Psycho (working title).

Ingame Debug
A visualisation of Psycho’s spatial determination system including occlusion, binaural head-relative delay and intensity.

Scene View
Psycho’s ‘Audio Walls’ in editor.

Of the senses, sound is arguably the most spatial. By layering a virtual acoustic environment over a physical one and enabling natural traversion of that environment by the audience, people may interact with a synthetic space in a manner which does not feel forced, alien or circuitous. Acoustics is also an area of virtual reality that is oft ignored. Because of this, sound will be the only virtual element in this work.

More info here: Parity

Special thanks to K Road Business Association for their incredible support.

Posted in Art, Unity3D, Virtual Reality, WIP | Leave a comment

VR Jam – Nil

Well, it’s all over. The Oculus IndieCade VR Jam winners have been announced.

Congratulations to all the finalists, and to all those 220 development teams with the willpower to see it through to the end.

My project, Nil, was a sleep depriving nightmare. But in a fit of masochism I’m following through with my goal of continuing its development.

Nil made a number of innovations, many of which I intend to keep as idioms for further VR work. For one, the head tracker was fully associated with a skeletal rig. So orientation also affected position based on bodily constraints. Something that greatly helped immersion and prevented motion sickness. For another, the game explored a more personal approach to first person interaction. When your whole world is replaced with a virtual environment, concepts such as personal space become much more important. In this case, I utilised the horror we feel when our personal space is startlingly invaded. Coupled with a dash of xenophobic revulsion provided by the uncannily faceless creatures I dubbed ‘Angels’, which you are forced to constantly stare at as a gameplay mechanic, the end result was very effective.

Pretty pictures of Nil’s progression from tech demo in week one to game at the end of the 3 week jam below.

Read more about the VR Jam and the finalists here: Oculus IndieCade VR Jam
And download the latest build of Nil (requires an Oculus Rift) here: Nil.zip

If you don’t have a Rift and you will be in Auckland for Digital Nationz this month, I will be at the homegrown booth showing Nil off. Look out for the Running Dimensions time slot!

Scene 3

Scene 1

Better door propagation

Awesome Glitch

Tiled

Excellent

Posted in Virtual Reality, WIP | Leave a comment

Unloading Native Plugins in the Unity Editor

Important note: This solution is intended to ease the stress of debugging on Windows only. It is not a viable solution for a release product or non-Windows platform.

So, after a knuckle gnawing afternoon debugging my native plugin for Unity3D, I decided to have a go at resolving an age old problem with Unity; It is not possible to replace a native dll in the Plugins folder without restarting the editor.

Doing some research in the form of a few Google queries, I found the Kernel32.dll methods for loading and unloading native assemblies. Namely LoadLibrary and FreeLibrary.

These methods alone, however, would not suffice. Attempts at calling methods via standard platform invoke simply did not work, as Unity/Mono still took over the assembly loading process.

The solution was to manually acquire the pointer to any desired method within my native library by defining its signature as a delegate, using GetProcAddress and Marshal.GetDelegateForFunctionPointer to create an instance of that delegate which points to the native assembly method and invoke it using Delegate.DynamicInvoke.

As you can imagine, this is a tedious process. So to make things simpler, I created a helper class called Native (see below).

/* 
 * Native dll invocation helper by Francis R. Griffiths-Keam
 * www.runningdimensions.com/blog
 */

using System;
using System.Runtime.InteropServices;
using UnityEngine;

public static class Native
{
    public static T Invoke<T, T2>(IntPtr library, params object[] pars)
    {
        IntPtr funcPtr = GetProcAddress(library, typeof(T2).Name);
        if (funcPtr == IntPtr.Zero)
        {
            Debug.LogWarning("Could not gain reference to method address.");
            return default(T);
        }

        var func = Marshal.GetDelegateForFunctionPointer(GetProcAddress(library, typeof(T2).Name), typeof(T2));
        return (T)func.DynamicInvoke(pars);
    }

    public static void Invoke<T>(IntPtr library, params object[] pars)
    {
        IntPtr funcPtr = GetProcAddress(library, typeof(T).Name);
        if (funcPtr == IntPtr.Zero)
        {
            Debug.LogWarning("Could not gain reference to method address.");
            return;
        }

        var func = Marshal.GetDelegateForFunctionPointer(funcPtr, typeof(T));
        func.DynamicInvoke(pars);
    }

    [DllImport("kernel32", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool FreeLibrary(IntPtr hModule);

    [DllImport("kernel32", SetLastError = true, CharSet = CharSet.Unicode)]
    public static extern IntPtr LoadLibrary(string lpFileName);

    [DllImport("kernel32")]
    public static extern IntPtr GetProcAddress(IntPtr hModule, string procedureName);
}

So to use this helper, first of all DO NOT build your plugins to the Assets/Plugins folder. Instead, build them to your game project’s root (the same folder as contains the Assets folder).
Then, for each object you wish to make calls to a native dll, follow this format:

using System;
using UnityEngine;

public class Example : MonoBehaviour
{
    static IntPtr nativeLibraryPtr;

    delegate int MultiplyFloat(float number, float multiplyBy);
    delegate void DoSomething(string words);

    void Awake()
    {
        if (nativeLibraryPtr != IntPtr.Zero) return;

        nativeLibraryPtr = Native.LoadLibrary("MyNativeLibraryName");
        if (nativeLibraryPtr == IntPtr.Zero)
        {
            Debug.LogError("Failed to load native library");
        }
    }

    void Update()
    {
        Native.Invoke<DoSomething>(nativeLibraryPtr, "Hello, World!");
        int result = Native.Invoke<int, MultiplyFloat>(nativeLibraryPtr, 10, 5); // Should return the number 50.
    }

    void OnApplicationQuit()
    {
        if (nativeLibraryPtr == IntPtr.Zero) return;

        Debug.Log(Native.FreeLibrary(nativeLibraryPtr)
                      ? "Native library successfully unloaded."
                      : "Native library could not be unloaded.");
    }
}

Take care to ensure your method signature and the parameters you provide match, with the handle provided by Native.LoadLibrary as the first parameter.

Once you are ready to build the project you will want to replace the helper calls with normal platform invoke calls and move the latest version of your plugin to Assets/Plugins. Alternatively, the #if UNITY_EDITOR preprocessor argument is helpful for automation of this process.

And there you have it! You can now press play in Unity’s editor, invoke native methods, stop the game, re-build your native plugin and try again!
Feel free to use this code as you please.

Posted in Code, Unity3D | Tagged , , | Leave a comment