At my current employer, we moved all of our videos to Azure using Azure Media Services a little under 2 years ago. This allowed us to upgrade them from using a Flash player to an HTML5 based video system, using the Azure Media Player for playback. I can’t say enough good things about using Azure for this. The videos went from looking mediocre and generally only playing on desktop machines to looking crystal clear at multiple resolutions, playable on every desktop and mobile device we throw at it.

We’ve now circled back to fill in a gap we missed at that point in time, captions. (Or subtitles, if you prefer.) Videos without captioning or subtitles excludes a portion of users, and that’s not cool. Since we’re already using Media Services for the video encoding, it made sense to use the Azure Media Indexer to generate the captions for us. However, most of the examples out there around doing this seem to be targeted at doing the indexing when you upload a video. We are certainly doing that moving forward, but there were a significant number of videos already out there which needed to be processed and that doesn’t seem to be a well documented scenario. Hopefully I can fill in that gap a little with this post.

First thing, start with the upload scenario from this link:
https://docs.microsoft.com/en-us/azure/media-services/media-services-index-content

That will get you most of the way, but there are a couple changes when using existing files. The first change is load the existing video Asset using the Asset ID. Replace the function called CreateAssetAndUploadSingleFile with one which looks something like this:

static IAsset LoadExistingAsset(string AssetId)
{
    var matchingAssets = (from a in _context.Assets
                                      where a.Id.Equals(AssetId)
                                      select a);

    IAsset asset = null;
    foreach (IAsset ia in matchingAssets)
    {
        asset = ia;
    }

    return asset;
}

You’ll need to know the Asset Id for the video. Hopefully you’ve been storing those someplace as you’ve encoded videos; we had them in SQL Azure so I pulled them back from there. If you don’t have them, playing around with the LINQ on _context.Assets will probably return them in some way. I haven’t needed to do that myself.

Now that you have a reference to the video asset, you can work your way down the code in RunIndexingJob and update a few things. I would recommend renaming the job to something which uniquely identifies the video, as that will show up in the Jobs section of the media services account in the Azure portal. If it fails, it makes it much easier to figure out which one to redo. Same thing with the indexing task and output asset name, renaming them makes them easier to track in the logs. For the configuration file, follow the Task Preset for Media Indexer link and load the file from wherever seems appropriate to you. I put some string placeholders into the config file for the metadata fields, which I’m replacing with some data pulled from the same database I’m getting the Asset ID from. So that section for me looks like:

<input>
    <metadata key="title" value="{0}" />
    <metadata key="description" value="{1}" />
  </input>

That should get you through RunIndexingJob. This is where the examples really fell flat for me. There are some additional steps required now. I changed RunIndexingJob to return the output media asset, as the caption files now have a different Asset ID than the video. Since Azure Blob Storage underpins Media Services, the Asset ID is actually the blob container name as well. Since the files the indexer generated have a different Asset ID, it means they’re actually in a different container than the video. This is important for actually consuming the captioning file. So rather than returning true like the example code, mine returns job.OutputMediaAssets[0]. There are three steps left to actually be able to use the caption files.

1. Publish the caption asset.
2. Change the blob storage permissions on the Asset ID. (Remember the Asset ID is the same as the blob container name.)
3. Save the path to the published caption file in blob storage.

Publish the Caption Asset

This is really easy, and very similar to publishing the video files after encoding. From code which calls RunIndexingJob:

var asset = RunIndexingJob(AssetId);
ILocator loc = PublishAsset(asset);

The definition for PublishAsset looks something like so:

static ILocator PublishAsset(IAsset asset)
{
    var locator = _context.Locators.Create(LocatorType.Sas, asset, AccessPermissions.Read, TimeSpan.FromDays(35600));
    return locator;
}

The major difference between the video publish and this is the different LocatorType. Using Sas creates a Progressive download locator, whereas OnDemandOrigin creates a streaming locator. If you use the latter, it won’t work. You return the locator back as it has the URL to the container, which will be helpful for the next step.

Change Blob Storage Permissions

Now that the Asset is published, the blob container is out there and available, but requires a SAS token to access it. If that’s what you want, skip this step. I want it to be available publicly, however, so the blob container permissions need a quick update. Since the Asset ID is the same as the blob container name, we’ll use the blob storage API to alter this.

var videoBlobStorageAcct = CloudStorageAccount.Parse(_blobConnStr);
CloudBlobClient videoBlobStorage = videoBlobStorageAcct.CreateCloudBlobClient();
string destinationContainerName = (new Uri(loc.Path)).Segments[1];
CloudBlobContainer assetContainer = videoBlobStorage.GetContainerReference(destinationContainerName);

if (assetContainer.Exists()) // This should always be true
{
     assetContainer.SetPermissions(new BlobContainerPermissions
     {
         PublicAccess = BlobContainerPublicAccessType.Blob
     });
}

I’m setting up the blob container there and grabbing the container name from the locator just to be safe. Then it gets the reference and sets the container access to blob. No more SAS required to get the captions!

Save the path to the caption file

The last thing to do now is build the path to the caption file or files and save them so they can be retrieved and used by the player. I’m only generating the WebVTT format, since I’m only concerned with playing the videos via a website.

string PublishUrl = loc.BaseUri;
string vttFileName = "";

// Loop through the files in the asset to find the vtt file
foreach (IAssetFile file in asset.AssetFiles)
{
    if (file.Name.EndsWith(".vtt"))
    {
        vttFileName = file.Name;
        break;
    }
}

string captionUrl = PublishUrl + "/" + vttFileName;

Now you save the value in captionUrl and you’re good to go! One small additional note which I was stuck on for a little while. If you’re consuming the caption file from a different domain, which is almost guaranteed (unless you’re running your site from static files on blob storage), you’ll need to change the CORS settings for the blob storage account being used by Media Services. The easiest way I’ve found to set this is to use the Azure portal. Browse to the storage account being used via the storage blade and not the media services blade. The storage blade has a nice UI which lets you whip through it in a few seconds. Hope this helps!

(This post refers to Azure Media Indexer v1. At the time of writing, v2 was in preview.)

Extremely easy to do, but extremely easy to mess up. I’m assuming with this post you have already configured Azure Mobile Services with your certificates. If not, there are plenty of tutorials around which can help you through the process. (I personally like this one for APNS.)

The classic Mobile Services examples all involve the Todo List app, which expect you to be using the full Mobile Services backend. It’s certainly slick if you can, but what if you only need the push notification capabilities? This is the problem I was trying to solve. The data for my mobile application is already managed elsewhere, I only need Mobile Services to manage the push subscriptions for my end users. The Javascript API portion of Mobile Services will be called from my backend and by some other applications using C# to generate the push notifications. It’s maybe not a common scenario, but it’s certainly there to allow you to do it. No amount of Google/Bing attempts landed me on a nice tutorial which satisfied my questions, so I’ll do my best.

Go to the API tab in your mobile service in the portal, and either click on the Create option at the bottom or click the big message in the center of the screen. Enter the name you want for the API, I choose Godzilla. (There was a marathon on El Rey last weekend, so I have Godzilla on the brain still.) Set the app permissions appropriately for your API, I’ll to stick with “Anybody with the Application Key” for the example and click the Checkmark button.

Once it’s done creating the API, click on it to view the code. You’ll have the starter API with a Post and a Get.

There’s no need for the Get, so delete it. The Post has a hint of what you need to use on the third line of comments, “var push =…”. Let’s uncomment that line so we can use it to generate the push. (I’m only going to write the code to send pushes to iOS (APNS) to keep the example nice and simple, the code for Android should be fairly trivial to add.) After the line with “var push = …”, paste in the following code over the remaining body of Post.

push.apns.send("atombomb", {
        alert: request.body.msg       
    },{ 
        success: function(result){
            //console.log("Godzilla push success");
       }, error: function(error){
           console.log("Godzilla push failed, error: " + error);
       }
    });
    response.send(statusCodes.OK, "{msg: 'Push executed, check log for results'}");

So what is this doing? It will push a notification to only the users subscribed to the “atombomb” tag. If you want to push to all users who have allowed push notifications for your app, change that parameter to null, without quotes around null. The next parameter is the JSON content which will be the contents of the push notification. (Here’s a great reference for more options you can put in it.) You see many other examples where they only use the payload property, which is certainly nice if you need to send some data with the push, but won’t actually pop a message on the user’s device. Alert is what you need to pop a message up, and the contents of the Alert parameter get displayed on the top of the user’s screen when it is received.

Quick side tangent. If you only provide the Payload property, as I did at first, it’s hugely confusing why there isn’t a notification popping up on the device. The portal will show the push succeeds, but nothing shows up on the device. The reference link above was the light bulb moment when I realized what I had done wrong. If there is no alert property as part of the content parameter, the user will not be notified in any way. You only want to use Payload by itself when you want to push an update in the background, without the user being prompted. You can certainly use both Alert and Payload at the same time, however.

On my alert parameter, you can see I’m actually sending in the content as part of the body of the API call. If you know your push message will always be the same, you can change this out for a string right there, but I think a parameter from the body makes for a better real world example. The final parameter contains the success and error functions, which are really useful for debugging. They will write to the mobile service log using console.log, which I’ll get to next. So here’s the final result:

I also edited the response message, but you can leave it as the default if you like. Now click the save button at the bottom to save your changes. If you want to check the logs once you have called your new API, click the back arrow after saving and it’s the last option in the top menu. You may need to uncomment the body of the success function to get anything to appear there, if it all works right away.

So now we need to call our API. This is when Mobile Services really flexes its muscles, it’s so incredibly simple. I’ve got a fairly reusable method for this which lets me pass in the API I want to call, and the message I want to send. So when invoking the method below, I use the parameters “Godzilla” for Api and “I knew that tuna-eating monster was useless!” for Message. If you’ve swapped out the message in the body for a string as I mentioned earlier, it gets even easier as you can then remove two lines which deal with the body variable.

public async Task SendPushNotification(string Api, string Message)
        {
            JObject body = new JObject();
            body.Add("msg", Message);
	    MobileServiceClient client = new MobileServiceClient(appUrl, appKey);
            return await client.InvokeApiAsync(Api, body);
        }

You can go much further with that method if you want, such as passing in your own custom object rather than using a JObject. If you’re looking to do something like that, I would recommend reading “Custom API in Mobile Services,” it goes nicely into a scenario like that, and includes a little bit on handling authenticated users.

I’m in the middle of upgrading a web site to .Net 4.5 while reworking some architecture. I found myself wanting to use the same site name on my local IIS so that I can continue to maintain the old code until the migration is done, and hopefully there would be a minimal amount of work to switch over the rest of the developers on the team to the new project when I’m done. I had done some coding for IIS 7 a while ago, but the cobwebs were thick so I was struggling. It took a couple of hours of searching but I finally came across how to do this. 

I ended up making a small winform app that I can just click a button and switch between the two solutions as needed. I’ll probably end up sending this to the other developers on the team when we do the change over to to new code just to make it really easy. Firstly, a couple of textboxes to input the paths, and a couple of buttons on it to trigger the change. Next I added a couple of entries to the settings file to hold the default paths that will end up in the textboxes.

Generate the button click events, then head over to the code behind. Now, add a reference to Microsoft.Web.Administration, which lives in windir\system32\inetsrv, and then add a using statement for it in the code behind file.

        public Form1()
        {
            InitializeComponent();

            txt2010.Text = Properties.Settings.Default.Path2010;
            txt2012.Text = Properties.Settings.Default.Path2012;
        }

        private void btn2010_Click(object sender, EventArgs e)
        {
            Properties.Settings.Default.Path2010 = txt2010.Text;            
            Properties.Settings.Default.Save();

            ServerManager oMan = new ServerManager();
            oMan.Sites["www.example.com"].Applications[0].VirtualDirectories[0].PhysicalPath = Properties.Settings.Default.Path2010;
            oMan.CommitChanges();
        }

        private void btn2012_Click(object sender, EventArgs e)
        {
            Properties.Settings.Default.Path2012 = txt2012.Text;            
            Properties.Settings.Default.Save();

            ServerManager oMan = new ServerManager();
            oMan.Sites["www.example.com"].Applications[0].VirtualDirectories[0].PhysicalPath = Properties.Settings.Default.Path2012;
            oMan.CommitChanges();
        }

In the form load, I’m setting the default settings to the textboxes. In the clicks, I’m saving any changes to the setting, setting it to the correct site, then committing it. Really easy, once you find the right documentation!

Continuing on with a previous post where I did a C# version of the Delta-E algorithm, which is really mostly just converting RGB to CIE-L*ab. Getting the delta score at that point is really pretty simple. I’ve had a chance now to convert one of the better algorithms now as well. To use the 1994 version below, just replace the CompareTo function in my original post with the one below. You could also overload it and pass in the K_1 and K_2 values as parameters, or create an enum to pass in as a parameter. I would recommend using a couple of existing values and eyeballing them if you’re not sure if you should use the graphics or textiles ones. Dark blues, teal, and tan were the ones that I used.

There is obviously a performance hit as it’s significantly more complex than the older version. I haven’t done any performance testing to compare the two yet, but I may post those in the future if I get a chance. I may do the CMC and 2000 formulas as well when I get some more time.

 public int CompareTo(ColorFormulas oComparisionColor)
        {
            // Based upon the Delta-E (1994) formula at easyrgb.com (http://www.easyrgb.com/index.php?X=DELT&H=04#text4)
            // Also referenced the python ColorMath project for factoring in textile vs screen: https://github.com/gtaylor/python-colormath/blob/master/colormath/color_diff.py
            double CIE_L1 = CieL;                       //Color #1 CIE-L*ab values
            double CIE_a1 = CieA;
            double CIE_b1 = CieB;
            double CIE_L2 = oComparisionColor.CieL;     //Color #2 CIE-L*ab values
            double CIE_a2 = oComparisionColor.CieA;
            double CIE_b2 = oComparisionColor.CieB;

            double K_1 = 0.048;      // 0.045 graphic arts, 0.048 textiles
            double K_2 = 0.014;      // 0.015 graphic arts, 0.014 textiles

            double K_L = 2;         // 1 default, 2 textiles
            double K_C = 1;
            double K_H = 1;

            double xC1 = Math.Sqrt(Math.Pow(CIE_a1, 2) + Math.Pow(CIE_b1, 2));
            double xC2 = Math.Sqrt(Math.Pow(CIE_a2, 2) + Math.Pow(CIE_b2, 2));

            double S_L = 1;
            double S_C = 1 + K_1 * xC1;
            double S_H = 1 + K_2 * xC1;

            double delta_L = CIE_L1 - CIE_L2;
            double delta_C = xC1 - xC2;
            double delta_a = CIE_a1 - CIE_a2;
            double delta_b = CIE_b1 - CIE_b2;

            double delta_H = 0;
            double deltaHCalc = Math.Pow(delta_a, 2) + Math.Pow(delta_b, 2) - Math.Pow(delta_C, 2);

            // Can't do a sqrt of a negative num
            if (deltaHCalc < 0)
            {
                delta_H = 0;
            }
            else
            {
                delta_H = Math.Sqrt(deltaHCalc);
            }

            // Make double sure that delta_H is non-negative 
            if (Double.IsNaN(delta_H) || delta_H < 0) delta_H = 0;

            double L_group = Math.Pow(delta_L / (K_L * S_L), 2);
            double C_group = Math.Pow(delta_C / (K_C * S_C), 2);
            double H_group = Math.Pow(delta_H / (K_H * S_H), 2);

            double Delta94 = Math.Sqrt(L_group + C_group + H_group);
            return Convert.ToInt16(Math.Round(Delta94));
        }

I’ve had to do some work on color comparison recently. There are some really great algorithms out there for this, particularly at easyrgb.com. Unfortunately, I couldn’t find a good C# version of this anywhere, so I had to convert my own. So for my own future reference, and for anyone that needs it here it is.

public class ColorFormulas
    {
        public double X { get; set; }
        public double Y { get; set; }
        public double Z { get; set; }

        public double CieL { get; set; }
        public double CieA { get; set; }
        public double CieB { get; set; }

        public ColorFormulas(int R, int G, int B)
        {
            RGBtoLAB(R, G, B);
        }

        public void RGBtoLAB(int R, int G, int B)
        {
            RGBtoXYZ(R, G, B);
            XYZtoLAB();
        }

        public void RGBtoXYZ(int RVal, int GVal, int BVal)
        {
            double R = Convert.ToDouble(RVal) / 255.0;       //R from 0 to 255
            double G = Convert.ToDouble(GVal) / 255.0;       //G from 0 to 255
            double B = Convert.ToDouble(BVal) / 255.0;       //B from 0 to 255

            if (R > 0.04045)
            {
                R = Math.Pow(((R + 0.055) / 1.055), 2.4);
            }
            else
            {
                R = R / 12.92;
            }
            if (G > 0.04045)
            {
                G = Math.Pow(((G + 0.055) / 1.055), 2.4);
            }
            else
            {
                G = G / 12.92;
            }
            if (B > 0.04045)
            {
                B = Math.Pow(((B + 0.055) / 1.055), 2.4);
            }
            else
            {
                B = B / 12.92;
            }

            R = R * 100;
            G = G * 100;
            B = B * 100;

            //Observer. = 2°, Illuminant = D65
            X = R * 0.4124 + G * 0.3576 + B * 0.1805;
            Y = R * 0.2126 + G * 0.7152 + B * 0.0722;
            Z = R * 0.0193 + G * 0.1192 + B * 0.9505;
        }

        public void XYZtoLAB()
        {
            // based upon the XYZ - CIE-L*ab formula at easyrgb.com (http://www.easyrgb.com/index.php?X=MATH&H=07#text7)
            double ref_X = 95.047;
            double ref_Y = 100.000;
            double ref_Z = 108.883;

            double var_X = X / ref_X;         // Observer= 2°, Illuminant= D65
            double var_Y = Y / ref_Y;
            double var_Z = Z / ref_Z;

            if (var_X > 0.008856)
            {
                var_X = Math.Pow(var_X , (1 / 3.0));
            }
            else
            {
                var_X = (7.787 * var_X) + (16 / 116.0);
            }
            if (var_Y > 0.008856)
            {
                var_Y = Math.Pow(var_Y, (1 / 3.0));
            }
            else
            {
                var_Y = (7.787 * var_Y) + (16 / 116.0);
            }
            if (var_Z > 0.008856)
            {
                var_Z = Math.Pow(var_Z, (1 / 3.0));
            }
            else
            {
                var_Z = (7.787 * var_Z) + (16 / 116.0);
            }

            CieL = (116 * var_Y) - 16;
            CieA = 500 * (var_X - var_Y);
            CieB = 200 * (var_Y - var_Z);
        }

        ///
        /// The smaller the number returned by this, the closer the colors are
        ///
        ///
        /// 
        public int CompareTo(ColorFormulas oComparisionColor)
        {
            // Based upon the Delta-E (1976) formula at easyrgb.com (http://www.easyrgb.com/index.php?X=DELT&H=03#text3)
            double DeltaE = Math.Sqrt(Math.Pow((CieL - oComparisionColor.CieL), 2) + Math.Pow((CieA - oComparisionColor.CieA), 2) + Math.Pow((CieB - oComparisionColor.CieB), 2));
            return Convert.ToInt16(Math.Round(DeltaE));
        }

        public static int DoFullCompare(int R1, int G1, int B1, int R2, int G2, int B2)
        {
            ColorFormulas oColor1 = new ColorFormulas(R1, G1, B1);
            ColorFormulas oColor2 = new ColorFormulas(R2, G2, B2);
            return oColor1.CompareTo(oColor2);
        }
    }