Feddas - Shawn Featherly

OculusSpatializerReflection setup

1. Edit -> Project Settings -> Audio -> spatializer Plugin should be set to OculusSpatializer
2. In the project panel, Right click -> Create -> Audio Mixer
3. Add an OculusSpatializerReflection effect to the Master channel of the AudioMixer that you just created. Then setup that effect to be a very reflective hallway using the following properties:
   GScale: 1 - this value is essentially a multiplier from the Room XYZ
   E.Rflt On: 1 - E.Rflt stands for Early Reflections. They help bring out the spatial aspect of the reverb
   E.Rflt Rev On: 1
   Room X: 16 - Here you want the room to be like a big church. The algorithm doesn't handle long hallways well.
   Room Y: 16
   Room Z: 16
   Left: 0.97
   Right: 0.97
   Up: 0.97
   Down: 0.97
   Behind:0.97
   Front: 0.97
   Shared Rev Min: 1
   Shared Rev Max: 10000 - This seems to be the value corresponding to when echos get cutoff. Maximize this to ensure you get as few echo cutoff as possible.
   
4. Create a new scene, the camera should be left at it's default position of 0, 1, -10
5. Add an empty game object at 0,0,0 that has an AudioSource component with:
   AudioClip: set to any audioclip you want to test with, vocal1.ogg from the OculusNativeSpatializer.unitypackage worked well for this
   Output: Master (This is the channel of the audio mixer you created in step #2)
   Spatial Blend: 1
   Spatialize: checked
   

Goal

How to use Unity3D 2017.x+ to make *.mp4s with overlaid text.

Why I didn’t go with Blender

I wanted to make a Facebook style video to run an ad. Which means take an existing video and overlay some text so that the video had some meaning when it was played without audio. Initially I tried to use Blender as I thought it’d be a great way for me to learn that tool. Plus, there are tons of tutorials on how to use Blender for video editing. Halfway through my first attempt I decided to find a way other than Blender due to:

1. Blenders text overlay feature is difficult to work with. Overlaid strings of text can not be multi-line.
2. I couldn’t figure out how to copy the animated graph window parts text objects, meaning my 4 pieces of text which each had 4 lines of text would require redoing my animated graph fade work 16 times.
3. I once found a way Blender scrubbing a video wouldn’t be choppy on a low end computer, I was unable to re-find this method. Upping Blenders RAM usage wasn’t enough.

Why I went with Unity3D

Unity3D I was already familiar with. Yet I couldn’t find a single guide on how to use it to create *.mp4s. I realized I could use a screen recorder, like ObsStudio, to encode the output to an *.mp4. The more I thought about it the better Unity3D seemed to fit:

1. Unity3D has pretty good UI tools.
2. Unity3D 5.6 introduced a new VideoPlayer component.
3. Unity3D 2017.1 introduced a new TimeLine tool.

Setting up Unity3D

Source Video

All we’re going to do is overlay text ontop of an already composed video. The first step is to make sure our video plays correctly in Unity3D. These steps were taken from a section of this youtube video: https://youtu.be/V8rwCWiRLWI?t=2m53s

1. Start a new Unity3D project / scene
2. Drop an *.mp4 source video file in the Unity3d project window. This will be the background video our text we be overlaid upon.
3. Add a VideoPlayer component to an empty game object.
4. Drag *.mp4 clip onto video player component.
5. On the VideoPlayer component change Render Mode to “Camera Far Plane”.
6. Drag the Main Camera into the Camera slot in VideoPlayer.
7. Add an AudioSource component to the same VideoPlayer game object.
8. Drag that AudioSource component into the Audio Source field of the VideoPlayer component.

Creating Text Objects

I wanted text to fade in over half a second, display for 5.5 seconds, then fade out over half a second.

1. Add GameObject > UI > Text.
2. To set the text to use the full view, open the Rect Transform’s red cross-hair positioning icon. Hold shift+alt and click the popup windows bottom right icon.
3. In the Text component set the font size and Paragraph Alignment to what you prefer. Set the color to white.
4. Add an Outline component to help the text stand out from your video. Set the alpha of the outlines’ black to 255.
5. Add a Canvas Group component to this text.
6. Make sure the Text object is selected in your hierarchy. Then open the Animation window and click “Create”.
7. Click the red circle record button in the Animation window.
8. Make sure the play head is at 0 and change the Texts’ Canvas Group components Alpha value to 0.
9. Move the play head to 0:30 (half a second) and put the Alpha value back to 1.
10. Scroll out until you see 6:30 on the animation track.
11. In the Canvas Group.Alpha row right click under 6:00 and “Add Key”.
12. Move the play head to 6:30 and set the Alpha back to 0.
13. Turn off record mode and press the play preview button to make sure the fade works correctly.
14. Find the saved *.anim file in the Project explorer. Click it to uncheck “Loop Time” in the inspector. This will make sure the same text isn’t shown again after it’s played once.
15. Copy the text component as many times as you need with Ctrl+D. Change each one’s text to match what you want it to say.
16. Disable all text gameobjects, they’ll be enabled by the timeline below.

Setting when text gets shown

The timeline feature of Unity 2017 is great for timing which objects are shown when. With how our text objects are setup all we need to do is enable them at the correct time.

1. In Unity3D go to Window > Timeline
2. Create a new empty gameobject called “Timeline”
3. Click “Create” in the Timeline window
4. Save it as TextTiming.playable
5. Delete its only row, which should have a blue cap on the left (animation track) and contain a reference to the Timeline game object
6. Add an Activation track to the timeline.
7. Click the settings cog in the upper right of the Timeline window and change the view to Seconds instead of Frames.
8. Resize the Activation track’s Active to be 8 seconds long
9. Select the Activation track and press Ctrl+D for each copy of the Text object you created.
10. Drag one Text object to each Activation track
11. Move their Active components to start when the previous ends. It should look like a staircase.
12. Extend the last Texts Active track to be as long as your video. Otherwise all text gameobjects activate.

Now when you press the Unity Editors play button you should see your video play with text being overlaid. You’re ready to record an *.mp4!

Setting up ObsStudio

To setup the video resolution in ObsStudio go to Settings > Video > Base Resolution. Here I also set “Common FPS Values” to “30”.

Next decide how the file will be saved in Settings > Output by setting its’ “Recording Path” and “Recording Format”

Recording from the “game” view in the Unity3D editor worked great for me. I made sure the Unity3D game window was also locked to the correct resolution by changing the resolution from “Free Aspect” to match the value I put in ObsStudios’ “Base Resolution”. More details on the game view control bar at https://docs.unity3d.com/Manual/GameView.html

Under Sources add a Display Capture. This shouldn’t show your entire monitor’s contents as long as you set the Base Resolution smaller than your monitors resolution. You do not want to scale the video with the red box handles, instead you want to pan it to the right part of the monitor by left clicking any part of the display view and dragging.

Recording

1. Made sure the Unity3D game window was bigger than the set video resolution.
2. Set the edge of the game view against the right edge of my monitor.
3. In ObsStudio made sure the game window was centered in the display
4. Pressed the “Start Recording” button in ObsStudio.
5. Pressed play in Unity3D
6. Waited until it played through what I wanted
7. Pressed “Stop Recording” in ObsStudio

Video trimming

I used Windows 10 Photos to trim the video. Open the *.mp4 with Windows 10 Photos then use the edit tools in the upper right to easily define the trim. More details at http://www.tech-recipes.com/rx/56780/trim-videos-windows-10-photos/

Completed video editing unity scene.
Written with StackEdit.

The goal of Unity3D's uNet HL(HighLevel)API is to sync an experience across multiple clients. This differs from uNets' low level networking in that the HLAPI delegates network tasks through a spawned prefab that inherits from NetworkBehaviour. Syncing is done through this prefab created for each player as soon as they join a server. Keeping a shared variable or gameobject sync'd between clients, using the HLAPI, also needs to be accomplished through the prefab spawned in by the HLAPI. The clients prefab sends a message to the server that it wants to update a shared resource. The server then makes an RPC call for all clients to update their copy of the shared resource. To illustrate the flow, lets update a shared variable (instead of a gameobject).

public class NetworkSession : NetworkBehaviour
{
    public static int SharedIntExample;

    void Update()
    {
        // only do something if the local player is doing it. CmdUpdateShare() calling RpcShareUpdate() will make sure everyone else sees it
        if (isLocalPlayer && Input.GetMouseButtonDown(0))
        {
            CmdUpdateShare(SharedIntExample + 1);  // define how the value should be updated, in this case increment by 1
        }
    }

    [Command]
    void CmdUpdateShare(int newValue)
    {
        RpcShareUpdate(newValue);  // use a Client RPC function to update the value on all clients
    }

    [ClientRpc]
    void RpcShareUpdate(int newValue)
    {
        SharedIntExample = newValue;
    }
}

    private void Start()
    {
        CmdNewClient();
    }

    [Command]
    void CmdNewClient()
    {
        RpcShareUpdate(SharedIntExample);
    }

    [Command]
    void CmdNewClient()
    {
        TargetShareUpdate(this.connectionToClient, SharedIntExample);
    }

    [TargetRpc]
    void TargetShareUpdate(NetworkConnection target, int newValue)
    {
        SharedIntExample = newValue;
    }

Before I figured out how to do this correctly, as above, and by using AssignClientAuthority (from stumbling on http://stackoverflow.com/questions/33469930/how-do-i-sync-non-player-gameobject-properties-in-unet-unity5/35504268). I tried to do it an unnecessarily complex way, which is described below, in strike-through, under "Wrong way of doing it".

public class NetworkSession : NetworkBehaviour
{
    [Command]
    public void CmdSetSlide(int toIndex)
    {
        // update servers version of SlideServer.Instance.SlideIndex
        MapStatus.Instance.SlideIndex = toIndex;
    }
}

public class FindIdentity : MonoBehaviour
{
    void Start()
    {
        networkConn = GameObject.FindObjectsOfType()
            .Where(s => s.isLocalPlayer)
            .FirstOrDefault();
    }

    void Update()
    {
        if (networkConn != null // check if this is a client. networkConn should only be null when this code is running on the server.
            && networkConn.CurrentCity != MapStatus.Instance.CurrentCity)
        {
            onCitySyncVarChanged();
        }
        else
        {
            keyInCity();

            // catch city changes Server side
            if (MapStatus.Instance.CurrentCity != oldCity)
            {
                oldCity = MapStatus.Instance.CurrentCity;

                // update map on the server
                mapOfPois.CurrentMap = MapStatus.Instance.CurrentCity;

                // update city syncVar of all sessions
                foreach (var netSession in GameObject.FindObjectsOfType())
                {
                    Debug.Log("netSession.name)" + netSession.name);
                    netSession.CurrentCity = MapStatus.Instance.CurrentCity;
                }
            }
        }
    }
}

    /// 
 Make sure a newly connected client gets the city selection made before they joined
    /// This override only runs on the client when the client has authority, which is after OnStartClient 
    public override void OnStartLocalPlayer()
    {
        base.OnStartLocalPlayer();

        Debug.Log("OnStartLocalPlayer " + this.name + " with " + CurrentCity + ". Asking server to update CurrentCity.");
        CmdReconnect();
    }

    [Command]
    public void CmdReconnect()
    {
        Debug.Log("CmdReconnect " + this.name + " with " + CurrentCity + " updated to " + MapStatus.Instance.CurrentCity);

        // uses the servers copy of CurrentCity to update the SyncVar
        CurrentCity = MapStatus.Instance.CurrentCity;
    }

Full Screen Video

using UnityEngine;

public class ScaleToCamera : MonoBehaviour
{
    public new Camera camera;
    private SpriteRenderer spriteRenderer;

    void Start()
    {
        if (camera == null) camera = Camera.main;

        spriteRenderer = this.GetComponent<SpriteRenderer>();
        if (spriteRenderer != null)
        {
            scaleSprite();
        }
    }

    private void scaleSprite()
    {
        if (camera.orthographic == false)
        {
            Debug.Log(this.name + " can't run ScaleToCamera.cs. Sprite scaling calculation is only implemented for an orthographic camera.");
            return;
        }

        float worldScreenHeight = camera.orthographicSize * 2;
        float worldScreenWidth = worldScreenHeight / Screen.height * Screen.width;

        transform.localScale = new Vector3(
            worldScreenWidth / spriteRenderer.sprite.bounds.size.x,
            worldScreenHeight / spriteRenderer.sprite.bounds.size.y,
            1);
    }
}

Adding Play/Stop control

using UnityEngine;
using System;

[RequireComponent(typeof(Animator))]
public class TransitionSprite : MonoBehaviour
{
    /// <summary> The sprite animation has stopped playing, either from finishing or from running Stop() </summary>
    public event EventHandler<EventArgs> Finished;

    private Animator cachedAnimator;
    private bool debugToggle;

    void Start()
    {
        cachedAnimator = this.GetComponent<Animator>();
    }

    void Update()
    {
        // Note: this if block is used only for debugging purposes,
        if (Input.GetButtonDown("Fire1"))
        {
            debugToggle = !debugToggle;
            if (debugToggle)
                Play();
            else
                Stop();
        }
    }

    /// <summary> Plays the animation from the start to the end. </summary>
    public void Play()
    {
        cachedAnimator.ResetTrigger("Stop"); // clear stop to make sure the animation won't stop as soon as it starts.
        cachedAnimator.SetTrigger("Play");
    }

    public void Stop()
    {
        cachedAnimator.ResetTrigger("Play"); // clear play to make sure the animation won't play as soon as it stops.
        cachedAnimator.SetTrigger("Stop");
    }

    public void RaiseFinished()
    {
        Debug.Log("Finished playing " + this.name);
        if (Finished != null)
        {
            Finished(this, new EventArgs());
        }
    }
}

using UnityEngine;

public class TransitionSpriteBehaviour : StateMachineBehaviour
{
    // OnStateExit is called when a transition ends and the state machine finishes evaluating this state
    override public void OnStateExit(Animator animator, AnimatorStateInfo stateInfo, int layerIndex)
    {
        var sprite = animator.GetComponent<TransitionSprite>();
        if (sprite != null)
        {
            sprite.RaiseFinished();
        }
    }
}

Labels: Unity3D

Labels: Interview, Nintendo

StrangeIoC compared to MVVM in XAML

How StrangeIoC gets started

Building a Unity3D app with rudimentary usage of StrangeIoC

using strange.extensions.context.impl;
public class CubeRoot : ContextView
{
 void Awake()
 {
  context = new CubeContext(this); 
 }
}

using UnityEngine;
using strange.extensions.context.impl;
using strange.extensions.context.api;
public class CubeContext : MVCSContext
{
 public CubeContext(MonoBehaviour view) : base(view)
 {
  view.renderer.material.color = Color.yellow; //verify view loads it's context
 }
}

Following a pattern by using injection

• Context - Only transform and CubeContext.cs components.
• Cube - Has default cube components along with CubeView.cs script.

using UnityEngine;
using strange.extensions.context.impl;
using strange.extensions.context.api;

public class CubeContext : MVCSContext
{
 public CubeContext(MonoBehaviour view) : base(view) { }

 protected override void mapBindings()
 {
  injectionBinder.Bind().To(Color.red); // map binding to a model
 }
}

using UnityEngine;
using strange.extensions.mediation.impl;

public class CubeView : View
{
 [Inject]
 public Color CubeColor {get; set;}

 protected override void Start()
 {
  base.Start();
  this.renderer.material.color = CubeColor;
 }
}

Labels: StrangeIoC, Unity3D

Labels: Android, Unity3D