Building your own REST query language: An experiment with Azure Cosmos Db

For all the REST resources we deploy out every day, one of the most common scenario a developer ends up handling is designing a nice and effective search/filter/query endpoint for these aforementioned resources. Usually in these scenarios conventions play a big part and that starts from query filter parameters in the query string to using a full blown search engine like ElasticSearch, Lucene or Algolia. Vinay Sahni did a phenomenal job for putting out the gist for basic REST practices here. If you are new into REST and want to get yourself started talking REST, it’s worth a look.

Today, we will do a simple yet fun experiment. Our goal today is to create a simple nifty looking query language for a sample REST resource. We will try to mimic a couple of features Twitter Search Api provides and our sample data storage today will be Azure Cosmos Db: Document Db which previously was known as just Azure Document db. Recently it moved to the Azure Cosmos Db family and if you have missed it, here is the full details.

Therefore, the first thing we need is a set of sample data we can use as our REST resource. Since we are going to mimic the twitter public search api, we definitely need some sample data first. And of course github comes to the rescue. Download the sample data and put it to your own document db instance. If you don’t know how to create a document db instance, follow the quick start here. I’m referencing the link to the .net quick start, but there are node.js, Python, Java and Xamarin resources right by the side of it. For those of you who do not want to test your work against an Azure hosted document db and want to test Azure document db locally, you can always opt for the azure document db emulator.

Now, for the sake of this experiment I ported the github twitter sample data to an azure document db instance. It has 10 entries in that small data set and hosted here. If you want to browse the data, azure portal does allow it. But I suggest looking at this open source app here named Azure DocumentDB Studio. You can use the endpoint and the key written in the sample code in github to connect. The database is public just for you guys so you can test the sample code I have hosted in github and will list out in the end of this article. TL;DR people, this is your cue to go to the end of the article, but I highly suggest to stick around.

Features to be built:

Out of all the features twitters search api provides, I will port the to:UserAccount, from:UserAccount and “exact text” search capabilities. That means, We will be able to search tweets sent from one account to other and we will also be able to search tweets by mentioning a string we like to be present in the tweets. For an example if we want to find tweets

  1. from an user account named @terminator
  2. to another user account named @robocop 
  3. or any tweet where the words “I’m back”
  4. or the hashtag #HastaLaVista is present,

our sample query language excerpt will be

http://our-awesome-api/search/q?to:robocop AND from:terminator OR “I’m back” OR #HastaLaVista”

If you have an eye for detail, you will notice that this is not exactly like the twitter search api since doesn’t use an and operator. But for the sake of the simplicity in this example this should be enough.

Tools we are going to use:

Our api stack will be written in asp.net core. We will use ANTLR as our query language lexer-parser. If you need an ANTLR primer and another experiment I did with it, please have a look here where I tried to gobble up a simple scripting language. If you are not really accustomed to any of .net stacks, fret not. All of these are totally doable in any other tech stack you will possibly prefer to use. Since ANTLR has targets for multiple languages, api can be built in basically any language and even for our sample storage solution azure document db here, you have access to multiple client SDKs.

The data:

As I mentioned above, we have a little data set of 10 entries. I’m posting a sample entry here so the rest of the tutorial makes sense. One entry points to a single tweet made by a sample user.

{
  "entities": {
    "user_mentions": [
      {
        "indices": [
          3,
          15
        ],
        "id_str": "178253493",
        "screen_name": "mikalabrags",
        "name": "Mika Labrague",
        "id": 178253493
      }
    ],
    "urls": [],
    "hashtags": [ "#Confused" ]
  },
  "in_reply_to_screen_name": null,
  "text": "RT @mikalabrags: Bipolar weather #Confused",
  "id_str": "210621130703245313",
  "place": null,
  "retweeted_status": {
    "entities": {
      "user_mentions": [],
      "urls": [],
      "hashtags": []
    },
    "in_reply_to_screen_name": null,
    "text": "Bipolar weather",
    "id_str": "210619512855343105",
    "place": null,
    "in_reply_to_status_id": null,
    "contributors": null,
    "retweet_count": 0,
    "favorited": false,
    "truncated": false,
    "source": "http://ubersocial.com",
    "in_reply_to_status_id_str": null,
    "created_at": "Thu Jun 07 06:29:39 +0000 2012",
    "in_reply_to_user_id_str": null,
    "in_reply_to_user_id": null,
    "user": {
      "lang": "en",
      "profile_background_image_url": "http://a0.twimg.com/profile_background_images/503549271/tumblr_m25lrjIjgT1qb6nmgo1_500.jpg",
      "id_str": "178253493",
      "default_profile_image": false,
      "statuses_count": 13635,
      "profile_link_color": "06544a",
      "favourites_count": 819,
      "profile_image_url_https": "https://si0.twimg.com/profile_images/2240536982/AtRKA77CIAAJRHT_normal.jpg",
      "following": null,
      "profile_background_color": "373d3a",
      "description": "No fate but what we make",
      "notifications": null,
      "profile_background_tile": true,
      "time_zone": "Alaska",
      "profile_sidebar_fill_color": "1c1c21",
      "listed_count": 1,
      "contributors_enabled": false,
      "geo_enabled": true,
      "created_at": "Sat Aug 14 07:31:28 +0000 2010",
      "screen_name": "mikalabrags",
      "follow_request_sent": null,
      "profile_sidebar_border_color": "08080a",
      "protected": false,
      "url": null,
      "default_profile": false,
      "name": "Mika Labrague",
      "is_translator": false,
      "show_all_inline_media": true,
      "verified": false,
      "profile_use_background_image": true,
      "followers_count": 214,
      "profile_image_url": "http://a0.twimg.com/profile_images/2240536982/AtRKA77CIAAJRHT_normal.jpg",
      "id": 178253493,
      "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/503549271/tumblr_m25lrjIjgT1qb6nmgo1_500.jpg",
      "utc_offset": -32400,
      "friends_count": 224,
      "profile_text_color": "352e4d",
      "location": "Mnl"
    },
    "retweeted": false,
    "id": 2.106195128553431E+17,
    "coordinates": null,
    "geo": null
  },
  "in_reply_to_status_id": null,
  "contributors": null,
  "retweet_count": 0,
  "favorited": false,
  "truncated": false,
  "source": "<a href=\"http://blackberry.com/twitter\" rel=\"nofollow\">Twitter for BlackBerry®</a>",
  "in_reply_to_status_id_str": null,
  "created_at": "Thu Jun 07 06:36:05 +0000 2012",
  "in_reply_to_user_id_str": null,
  "in_reply_to_user_id": null,
  "user": {
    "lang": "en",
    "profile_background_image_url": "http://a0.twimg.com/profile_background_images/542537222/534075_10150809727636812_541871811_10087628_844237475_n_large.jpg",
    "id_str": "37200018",
    "default_profile_image": false,
    "statuses_count": 5715,
    "profile_link_color": "CC3366",
    "favourites_count": 46,
    "profile_image_url_https": "https://si0.twimg.com/profile_images/2276155427/photo_201_1_normal.jpg",
    "following": null,
    "profile_background_color": "dbe9ed",
    "description": "protège-moi de mes désirs  23107961 ☍",
    "notifications": null,
    "profile_background_tile": true,
    "time_zone": "Singapore",
    "profile_sidebar_fill_color": "ffffff",
    "listed_count": 2,
    "contributors_enabled": false,
    "geo_enabled": true,
    "created_at": "Sat May 02 13:55:49 +0000 2009",
    "screen_name": "yoursweetiethea",
    "follow_request_sent": null,
    "profile_sidebar_border_color": "91f50e",
    "protected": false,
    "url": "http://yoursweetiethea.tumblr.com",
    "default_profile": false,
    "name": "Althea Arellano",
    "is_translator": false,
    "show_all_inline_media": false,
    "verified": false,
    "profile_use_background_image": true,
    "followers_count": 306,
    "profile_image_url": "http://a0.twimg.com/profile_images/2276155427/photo_201_1_normal.jpg",
    "id": "37200018",
    "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/542537222/534075_10150809727636812_541871811_10087628_844237475_n_large.jpg",
    "utc_offset": 28800,
    "friends_count": 297,
    "profile_text_color": "fa3c6b",
    "location": "Christian's Heart"
  },
  "retweeted": false,
  "id": "210621130703245300",
  "coordinates": null,
  "geo": null
}

The ANTLR grammar:

The ANTLR grammar we are going to use here is:

grammar Search;

expr: term (op term)*;
term: exactText | hashText | toText | fromText;
op: AND | OR;

toText: 'to:'ID;
fromText: 'from:'ID;
hashText: '#'ID;
exactText: EXACTTEXT;

// lexer rule
EXACTTEXT: '"' ~'"'* '"';
OR: 'OR';
AND: 'AND';
ID: [a-zA-Z_] [a-zA-Z0-9_]*;
WS: [ \n\t\r]+ -> skip;

It’s a small and simple grammar like the one we used in our scripting language tutorial. Since our REST resource query is essentially an expression, our entry rule will be expr. The railroad diagram for expr looks like:

expr_rest
Rule expr

This means we can have a single search term or multiple search term chained by a op  rule. The op rule is nothing but the two relational operator we support: AND and OR.

relop
Rule op

Along side of the op rule we also need to know what the term rule stands for. The term rule stands for the search term format we are allowed to use. In our sample query stated above, we have terms like  from:terminatorto:robocop, “I’m back” or a hashtag like #HastaLaVista. That’s why we have 4 rules defining all of these cases and the term rule is a OR relationship between them.

term_rest
rule term

I’m not going to post the railroad diagram for toText, fromText, hashText and exactText rules since they are pretty self-explanatory if you have a cursory look at the grammar.

So, what are we waiting for? Let’s start writing our little codebase that will parse this query string and translate it to an azure document db SQL that we can use to fetch the tweets. For that, we need a small repository that will connect to our desired database and collection in document db and will let us fetch some items. I only added methods that will allow us to read the tweets and connect to the database. I ignored the rest since you can always have a look at those in the quick start for azure document db.

Here’s our small rough database repository. Please remember to keep your endpoint and key strings somewhere secret and safe in production environment, since this is a tutorial, I went with what is easy and fast to go for a proof of concept.

namespace TweetQuery.Lib
{
    using System;
    using System.Collections.Generic;
    using System.Threading.Tasks;
    using Microsoft.Azure.Documents.Client;
    using Microsoft.Azure.Documents.Linq;

    public class CosmosDBRepository<T> where T : class
    {
        private readonly string Endpoint = "https://tweet.documents.azure.com:443/";
        private readonly string Key = "fjp9Z3qKPxSOfE0KS1aaKvUY27B8IoL347sdtMBMjkCQqPmoaKjGXoyltrItNXNN6h4QjAYLSY5nyb2djWWUOQ==";
        private readonly string DatabaseId = "tweetdb";
        private readonly string CollectionId = "tweets";
        private DocumentClient client;

        public CosmosDBRepository()
        {
            client = new DocumentClient(new Uri(Endpoint), Key);
        }

        public async Task<IEnumerable<T>> GetItemsAsync(string sql)
        {
            if (string.IsNullOrEmpty(sql))
                throw new ArgumentNullException(nameof(sql));

            FeedOptions queryOptions = new FeedOptions { MaxItemCount = -1 };

            var query = this.client.CreateDocumentQuery<T>(
                 UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId),
                 sql, queryOptions)
                 .AsDocumentQuery();

            List<T> results = new List<T>();
            while (query.HasMoreResults)
            {
                results.AddRange(await query.ExecuteNextAsync<T>());
            }

            return results;
        }
    }
}

I opted for executing a SQL instead of a linq expression since constructing SQL is easier and simpler for a tutorial. Plus it decouples the query structure from compile time POCOs that we use as our models too.

I created a DocumentDbListener class based off the SearchBaseListener which was auto-generated from our ANTLR grammar. The sole purpose of this class is to generate a simple SQL against our search expression. To search inside nested arrays, I used a user defined function for azure document db. All of these are very crudely written, so forgive my indecency. Since this is just a tutorial, I tried to keep it as simple as possible.

function matchArrayElement(array, match) {
    for (var index = 0; index < array.length; index++) {
        var element = array[index];

        if (typeof match === "object") {
            for (var key in match) {
                if (match.hasOwnProperty(key) && element.hasOwnProperty(key)) {
                    var matchVal = match[key];
                    var elemVal = element[key];

                    return matchVal == elemVal;
                }
            }
        }
        else {
            return (element == match)
        }
    }

    return false;
}

All this method does is it tries to find nested array elements based on the match we send back. You can achieve the same result thorough JOINs in Azure Document Db or Array method ARRAY_CONTAINS, but I preferred a user defined function since it serves my purpose easily.

Constructing SQL from the query expression:

To understand how the SQL is generated from the query expression, let’s begin with the to:UserAccount expression. Since we start with the rule expr, let’s override the SearchBaseListener method EnterExpr first.

namespace TweetSearch.CosmosDb.DocumentDb
{
    using Antlr4.Runtime.Misc;
    using TweetSearch.CosmosDb.Util;

    public class DocumentDbListener : SearchBaseListener
    {
        private string projectionClause = "SELECT * FROM twt";
        private string whereClause;

        public string Output
        {
            get { return projectionClause + " " + whereClause; }
        }

        public override void EnterExpr([NotNull] SearchParser.ExprContext context)
        {
            this.whereClause = "WHERE";
        }
    }
}

The approach I took here is essentially the simplest. I handle the events fired the moment ANTLR enters a specific rule and I keep appending the SQL string to the whereClause. Since, entering the expr rule means that I will need a where SQL clause, I initialized it with “WHERE”. The thing to notice here is instead of concatenating I chose to initialize it because I expect this event to be fired exactly once since that is how the grammar is designed.

Following the same trail the next thing to handle will be the EnterTerm event. But, term is nothing but an OR relationship between 4 other rules. Handling them specifically gives me the edge since they produce simpler and smaller readable methods. For example, if we want to handle the to:UserAccount expression, a simple method like following should be sufficient for our use case.

        public override void EnterToText([NotNull] SearchParser.ToTextContext context)
        {
            var screenName = context.GetText().Substring(3).Enquote();
            this.whereClause = string.Concat(whereClause, " ", $"udf.matchArrayElement(twt.entities.user_mentions, {{ \"screen_name\" : {screenName} }} )");
        }

This is where our user defined function also comes in play though. I’m trying to find any tweet that has an user mention to the parsed user account I fetched from the query.

By following the same rule I completed the rest of the four rules and my full listener class looks like:

namespace TweetSearch.CosmosDb.DocumentDb
{
    using Antlr4.Runtime.Misc;
    using TweetSearch.CosmosDb.Util;

    public class DocumentDbListener : SearchBaseListener
    {
        private string projectionClause = "SELECT * FROM twt";
        private string whereClause;

        public string Output
        {
            get { return projectionClause + " " + whereClause; }
        }

        public override void EnterExpr([NotNull] SearchParser.ExprContext context)
        {
            this.whereClause = "WHERE";
        }

        public override void EnterFromText([NotNull] SearchParser.FromTextContext context)
        {
            var screenName = context.GetText().Substring(5).Enquote();
            this.whereClause = string.Concat(whereClause, " ", "twt.user.screen_name = ", screenName);
        }

        public override void EnterOp([NotNull] SearchParser.OpContext context)
        {
            var text = context.GetText();
            this.whereClause = string.Concat(this.whereClause, " ", text.ToUpper());
        }

        public override void EnterToText([NotNull] SearchParser.ToTextContext context)
        {
            var screenName = context.GetText().Substring(3).Enquote();
            this.whereClause = string.Concat(whereClause, " ", $"udf.matchArrayElement(twt.entities.user_mentions, {{ \"screen_name\" : {screenName} }} )");
        }

        public override void EnterHashText([NotNull] SearchParser.HashTextContext context)
        {
            var hashtag = context.GetText().Enquote();
            this.whereClause = string.Concat(whereClause, " ", $"udf.matchArrayElement(twt.entities.hashtags, {hashtag})");
        }

        public override void EnterExactText([NotNull] SearchParser.ExactTextContext context)
        {
            var text = context.GetText();
            this.whereClause = string.Concat(whereClause, " ", $"CONTAINS(twt.text, {text})");
        }
    }
}

We got our listener ready! Now, all we need is a context class that will bootstrap the lexer and parser and tokens so the input expression is transpiled and the output SQL is generated. Just like our last work on ANTLR, the TweetQueryContext class will look like the following:

namespace TweetSearch.CosmosDb.DocumentDb
{
    using Antlr4.Runtime;
    using Antlr4.Runtime.Tree;

    public class TweetQueryContext
    {
        private DocumentDbListener listener;

        public TweetQueryContext()
        {
            this.listener = new DocumentDbListener();
        }

        public SearchParser.ExprContext GenerateAST(string input)
        {
            var inputStream = new AntlrInputStream(input);
            var lexer = new SearchLexer(inputStream);
            var tokens = new CommonTokenStream(lexer);
            var parser = new SearchParser(tokens);
            parser.ErrorHandler = new BailErrorStrategy();

            return parser.expr();
        }

        public string GenerateQuery(string inputText)
        {
            var astree = this.GenerateAST(inputText);
            ParseTreeWalker.Default.Walk(listener, astree);
            return listener.Output;
        }
    }
}

Whew! That was easy, right?

Bootstrapping the api layer:

We have all we need except the api. Thanks to asp .net core, that is two clicks away. Open Visual Studio and open a .net core api project. Our TweetsController class looks like the following:

namespace TweetQuery.Controllers
{
    using Microsoft.AspNetCore.Mvc;
    using System.Threading.Tasks;
    using TweetQuery.Lib;
    using TweetQuery.Lib.Model;
    using TweetSearch.CosmosDb.DocumentDb;

    [Route("api/[controller]")]
    public class TweetsController : Controller
    {
        private CosmosDBRepository<Tweet> repository;
        private TweetQueryContext context;

        public TweetsController(CosmosDBRepository<Tweet> repository)
        {
            this.repository = repository;
            this.context = new TweetQueryContext();
        }

        [HttpGet("search")]
        public async Task<IActionResult> Search([FromQuery] string q)
        {
            if (string.IsNullOrEmpty(q))
                return BadRequest();

            var querySql = this.context.GenerateQuery(q).Trim();
            var result = await repository.GetItemsAsync(querySql);
            return Ok(result);
        }
    }
}

I reused the db repository we created earlier and as you see that is dependency injected in the controller which you have to configure in the ConfigureServices method in your Startup  class. I’m not adding that specific code here since it is already in the sample code and doesn’t belong to the scope of this tutorial. Same goes for the model class Tweet and the classes it uses inside.

Time to test!

The project is hosted here in github. Clone the code, and build and run it from your visual studio. As a sample query try the following:

http://localhost:5000/api/tweets/search?q=to:hatena_sugoi AND from:maeta_tw OR %23HopesUp OR "Surely June is a summer"

 

I url-encoded the hashtag here just to be nice on the REST client you might use. I highly suggest Postman if you don’t want anything heavy.

I only took the minimalists way of using ANTLR here, you can build your own expression tree based on the auto-generated listener and can do so much more if you want. A perfect example will be LinqToQuerystring written by Pete Smith. It generates the necessary LINQ expression for any IQueryable and thus allows you to write database agnostic Odata driven queries and it’s tons faster and lighter than the one Microsoft ships.

I hope this was fun. Happy RESTing!

I thought I knew C# : Garbage collection & disposal. Part-II

Okay, we are back in to collect “garbage” properly. If you haven’t the first part of this post you might want to read it here. We are following norms and conventions from the first post here too.

 Starting things from where we left off:

This write up continues after the first one where we ended things with SafeHandle class and IDisposable  interface. I do see how intriguing this is of course. But before that, like a unexpected, miserable commercial break before the actual stuff comes on your TV, let’s go have a look on something called finalization.

What you create, you “might” need to destroy:

If you know a little bit of C++ you’d know that there is a thing called destructor and the name implies it does the exactly opposite of what a constructor does. C++ kind of needs it since it doesn’t really have a proper garbage collection paradigm. But that also raises the question of whether C# has a destructor and if it does what does it do and why do we even have one since we said managed resourced would be collected automatically.

Lets jump onto some code, shall we?

class Cat
{
    ~Cart()
    {
        // cleanup things
    }
}

Wait a minute, now this is getting confusing. I have IDisposable::Dispose() and I have a destructor. Both looks like to have same responsibility. Not exactly. Before we confuse ourselves more, lets dig a bit more in the code. To be honest, C# doesn’t really have a destructor, it’s basically a syntactic sugar and inside c sharp compiler translates this segment as

protected override void Finalize()
{
    try
    {
        // Clean things here
    }
    finally
    {
        base.Finalize();
    }
}

We can pick up to things here. First thing is the destructor essentially translates to a overridden Finalize() method. And it also calls the Finalize() base class in the finally block. So, it’s essentially a recursive finalize(). Wait, we have no clue what is Finalize().

Finalize, who art thou?

Finalize is somebody who is blessed to talk with the garbage collector. If you have already questioning where the topics we discussed in last post comes in help, this is the segment. Lets start getting answers for all our confusions on last segment. First, why Finalize is overridden and where does it gets its signature. Finalize gets its signature from Object class. Okay, understandable. What would be the default implementation then? Well, it doesn’t have a default implementation. You might ask why. A little secret to slip in here, remember the mark step of garbage collector. He marks a type instance for finalization if and only if it has overridden the Finalize() method. It’s your way of telling the garbage collector that you need to do some more work before you can reclaim my memory. Garbage collector would mark this guy and put him in a finalization queue which is essentially a list of objects whose finalization codes must be run before GC can reclaim their memory. So, you are possibly left with one more confusion now. And that is, you understood garbage collector uses this method to finalize things, i.e. doing the necessary cleanup. Now, then why do we need IDisposable where we can just override finalize and be done with it, right?

Dispose, you confusing scum!

Turns out dispose intends to point out a pattern in code that you can use to release unmanaged memory. What I didn’t tell you about garbage collector before is you don’t really know when he would essentially do that, you practically have no clue. That also means if you write an app that uses unmanaged resources like crazy like reading 50-60 files at a time, you are using a lot of scarce resources at the same time. And in the end you are waiting for GC to do his job but that guy has no time table. So, hoarding these resources is not a good idea in the meantime. Since releasing unmanaged resources are developers duty, putting that over a finalize method  and waiting for GC to come and invoke that is a stupid way to go. Moreover, if you send an instance to a finalization queue, it means, GC will essentially do the memory cleanup in the next round, he will only invoke finalize this round. That also means GC has to visit you twice to clear off your unused occupied memory which you kinda need now. And the fact that you might want to release the resources NOW is a pretty good reason itself to not wait for GC to do your dirty work. And, when you actually shut down your application, Mr. GC visits you unconditionally and takes out EVERYONE he finds. I hope the need of Dispose() is getting partially clear to you now. We need a method SomeMethod() that we can call which would clean up the unmanaged resources. If we fail to call that at some point, just to make sure garbage collector can call that we will use that same method inside Finalize() so it is called anyhow. If I have not made a fool out of myself at this point, you have figured out the fact that SomeMethod() is Dispose(). Okay, so we know what we are going to do. Now we need to act on it. We will implement the dispose pattern we have been talking about. The first thing we would do here is we would try to write code that reads a couple of lines from a file. We would try to do it the unmanaged way, then move it to slowly to the managed way and in the process we would see how we can use IDisposable there too.

Doing things the unsafe way:

I stole some code from a very old msdn doc which describes a FileReader class like the following:

using System;
using System.Runtime.InteropServices;
public class FileReader
{
    const uint GENERIC_READ = 0x80000000;
    const uint OPEN_EXISTING = 3;
    IntPtr handle;
[DllImport("kernel32", SetLastError = true)]
    static extern unsafe IntPtr CreateFile(
            string FileName,                    // file name
            uint DesiredAccess,                 // access mode
            uint ShareMode,                     // share mode
            uint SecurityAttributes,            // Security Attributes
            uint CreationDisposition,           // how to create
            uint FlagsAndAttributes,            // file attributes
            int hTemplateFile                   // handle to template file
            );
[DllImport("kernel32", SetLastError = true)]
    static extern unsafe bool ReadFile(
            IntPtr hFile,                       // handle to file
            void* pBuffer,                      // data buffer
            int NumberOfBytesToRead,            // number of bytes to read
            int* pNumberOfBytesRead,            // number of bytes read
            int Overlapped                      // overlapped buffer
            );
[DllImport("kernel32", SetLastError = true)]
    static extern unsafe bool CloseHandle(
            IntPtr hObject   // handle to object
            );
    public bool Open(string FileName)
    {
        // open the existing file for reading
        handle = CreateFile(
                FileName,
                GENERIC_READ,
                0,
                0,
                OPEN_EXISTING,
                0,
                0);
if (handle != IntPtr.Zero)
            return true;
        else
            return false;
    }
    public unsafe int Read(byte[] buffer, int index, int count)
    {
        int n = 0;
        fixed (byte* p = buffer)
        {
            if (!ReadFile(handle, p + index, count, &amp;n, 0))
                return 0;
        }
        return n;
    }
    public bool Close()
    {
        // close file handle
        return CloseHandle(handle);
    }
}

Please remember you need to check your Allow Unsafe Code checkbox in your build properties before you start using this class. Lets have a quick run on the code pasted here. I don’t intend to tell everything in details here because that is not the scope of this article. But we will build up on it, so we need to know a little bit. The DllImport attribute here is essentially something  you would need to use an external dll (thus unmanaged) and map the functions inside it to your own managed class. You can also see that’s why we have used the extern keyword here. The implementations of these methods doesn’t live in your code and thus your garbage collector can’t take responsibility of clean up here. 🙂 The next thing you would notice is the fixed statement. fixed statement essentially link up a managed type to an unsafe one and thus make sure GC doesn’t move the managed type when it collects. So, the managed one stays in one place and points to the unmanaged resource perfectly. So, what are we waiting for? Lets read a file.

static int Main(string[] args)
{
    if (args.Length != 1)
    {
        Console.WriteLine("Usage : ReadFile &lt;FileName&gt;");
        return 1;
    }
    if (!System.IO.File.Exists(args[0]))
    {
        Console.WriteLine("File " + args[0] + " not found.");
        return 1;
    }
    byte[] buffer = new byte[128];
    FileReader fr = new FileReader();
    if (fr.Open(args[0]))
    {
        // Assume that an ASCII file is being read
        ASCIIEncoding Encoding = new ASCIIEncoding();
        int bytesRead;
        do
        {
            bytesRead = fr.Read(buffer, 0, buffer.Length);
            string content = Encoding.GetString(buffer, 0, bytesRead);
            Console.Write("{0}", content);
        }
        while (bytesRead &gt; 0);
        fr.Close();
        return 0;
    }
    else
    {
        Console.WriteLine("Failed to open requested file");
        return 1;
    }
}

So, this is essentially a very basic console app and looks somewhat okay. I have created a byte array of size 128 which I would use as a buffer when I read. FileReader returns 0 when it can’t read anymore. Don’t get confused seeing this.

while (bytesRead > 0)

It’s all nice and dandy to be honest. And it works too. Invoke the application (in this case the name here is  TestFileReading.exe) like the following:

TestFileReading.exe somefile.txt

And it works like a charm. But what I did here is we closed the file after use. What if something happens in the middle, something like the file not being available. Or I throw an exception in the middle. What will happen is the file would not be closed up until my process is not closed. And the GC will not take care of it because it doesn’t have anything in the Finalize() method.

Making it safe:

public class FileReader: IDisposable
{
    const uint GENERIC_READ = 0x80000000;
    const uint OPEN_EXISTING = 3;
    IntPtr handle = IntPtr.Zero;
    [DllImport("kernel32", SetLastError = true)]
    static extern unsafe IntPtr CreateFile(
            string FileName,                 // file name
            uint DesiredAccess,             // access mode
            uint ShareMode,                // share mode
            uint SecurityAttributes,      // Security Attributes
            uint CreationDisposition,    // how to create
            uint FlagsAndAttributes,    // file attributes
            int hTemplateFile          // handle to template file
            );
    [DllImport("kernel32", SetLastError = true)]
    static extern unsafe bool ReadFile(
            IntPtr hFile,                   // handle to file
            void* pBuffer,                 // data buffer
            int NumberOfBytesToRead,      // number of bytes to read
            int* pNumberOfBytesRead,     // number of bytes read
            int Overlapped              // overlapped buffer
            );
    [DllImport("kernel32", SetLastError = true)]
    static extern unsafe bool CloseHandle(
            IntPtr hObject   // handle to object
            );
    public bool Open(string FileName)
    {
        // open the existing file for reading
        handle = CreateFile(
                FileName,
                GENERIC_READ,
                0,
                0,
                OPEN_EXISTING,
                0,
                0);
        if (handle != IntPtr.Zero)
            return true;
        else
            return false;
    }
    public unsafe int Read(byte[] buffer, int index, int count)
    {
        int n = 0;
        fixed (byte* p = buffer)
        {
            if (!ReadFile(handle, p + index, count, &amp;n, 0))
                return 0;
        }
        return n;
    }
    public bool Close()
    {
        // close file handle
        return CloseHandle(handle);
    }
    public void Dispose()
    {
        if (handle != IntPtr.Zero)
            Close();
    }
}

Now, in our way towards making things safe, we implemented IDisposable here. That exposed Dispose() and the first thing I did here is we checked whether the handle is IntPtr.Zero and if it’s not we invoked Close(). Dispose() is written this way because it should be invokable in any possible time and it shouldn’t throw any exception if it is invoked multiple times. But is it the solution we want? Look closely. We wanted to have a Finalize() implementation that will essentially do the same things if somehow Dispose() is not called. Right?

Enter the Dispose(bool) overload. We want the parameter less Dispose() to be used by only the external consumers. We would issue a second Dispose(bool) overload where the boolean parameter indicates whether the method call comes from a Dispose method or from the finalizer. It would be true if it is invoked from the parameter less Dispose() method.

With that in mind our code would eventually be this:

    public class FileReader: IDisposable
    {
        const uint GENERIC_READ = 0x80000000;
        const uint OPEN_EXISTING = 3;
        IntPtr handle = IntPtr.Zero;
        private bool isDisposed;

        SafeHandle safeHandle = new SafeFileHandle(IntPtr.Zero, true);

        [DllImport("kernel32", SetLastError = true)]
        static extern unsafe IntPtr CreateFile(
              string FileName,                  // file name
              uint DesiredAccess,              // access mode
              uint ShareMode,                 // share mode
              uint SecurityAttributes,       // Security Attributes
              uint CreationDisposition,     // how to create
              uint FlagsAndAttributes,     // file attributes
              int hTemplateFile           // handle to template file
              );

        [DllImport("kernel32", SetLastError = true)]
        static extern unsafe bool ReadFile(
             IntPtr hFile,                // handle to file
             void* pBuffer,              // data buffer
             int NumberOfBytesToRead,   // number of bytes to read
             int* pNumberOfBytesRead,  // number of bytes read
             int Overlapped           // overlapped buffer
             );

        [DllImport("kernel32", SetLastError = true)]
        static extern unsafe bool CloseHandle(
              IntPtr hObject   // handle to object
              );

        public bool Open(string FileName)
        {
            // open the existing file for reading
            handle = CreateFile(
                  FileName,
                  GENERIC_READ,
                  0,
                  0,
                  OPEN_EXISTING,
                  0,
                  0);

            if (handle != IntPtr.Zero)
                return true;
            else
                return false;
        }

        public unsafe int Read(byte[] buffer, int index, int count)
        {
            int n = 0;
            fixed (byte* p = buffer)
            {
                if (!ReadFile(handle, p + index, count, &amp;n, 0))
                    return 0;
            }
            return n;
        }

        public bool Close()
        {
            // close file handle
            return CloseHandle(handle);
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool isDisposing)
        {
            if (isDisposed)
                return;

            if (isDisposing)
            {
                safeHandle.Dispose();
            }

            if (handle != IntPtr.Zero)
                Close();

            isDisposed = true;
        }
    }

Now if you focus on the changes we made here is introducing the following method:

protected virtual void Dispose(bool isDisposing)

Now, this method envisions what we discussed a moment earlier. You can invoke it multiple times without any issue. There are two prominent block here.

  • The conditional block is supposed to free managed resources (Read invoking Dispose() methods of other IDisposable member/properties inside the class, if we have any.)
  • The non-conditional block frees the unmanaged resources.

You might ask why the conditional block tries to dispose managed resources. The GC takes care of that anyway right? Yes, you’re right. Since garbage collector is going to take care of the managed resources anyway, we are making sure the managed resources are disposed on demand if only someone calls the parameter less Dispose(). 

Are we forgetting something again? Remember, you have unmanaged resources and if somehow the Dispose() is not invoked you still have to make sure this is finalized by the garbage collector. Let’s write up a one line destructor here.

 ~FileReader()
 {
    Dispose(false);
 }

It’s pretty straightforward and it complies with everything we said before. Kudos! We are done with FileReader.

Words of Experience:

Although we are safe already. We indeed forgot one thing. If we invoke Dispose() now it will dispose unmanaged and managed resources both. That also means when the garbage collector will come to collect he will see there is a destructor ergo there is a Finalize() override here. So, he would still put this instance into Finalization Queue. That kind of hurts our purpose. Because we wanted to release memory as soon as possible. If the garbage collector has to come back again, that doesn’t really make much sense. So, we would like to suppress the garbage collector to invoke Finalize() if we know we have disposed it ourselves. And a single line modification to the Dispose() method would allow you to do so.

public void Dispose()
{
    Dispose(true);
    GC.SuppressFinalize(this);
}

We added the following statement to make sure if we have done disposing ourselves the garbage collector would not invoke Finalize() anymore.

GC.SuppressFinalize(this);

Now, please keep in mind that you shouldn’t write a GC.SupperssFinalize() in a derived class since your dispose method would be overridden and you would follow the same pattern and call base.Dispose(isDisposing) in the following way:

class DerivedReader : FileReader
{
   // Flag: Has Dispose already been called?
   bool disposed = false;

   // Protected implementation of Dispose pattern.
   protected override void Dispose(bool disposing)
   {
      if (disposed)
         return; 

      if (disposing) {
         // Free any other managed objects here.
         //
      }

      // Free any unmanaged objects here.
      //
      disposed = true;

      // Call the base class implementation.
      base.Dispose(disposing);
   }

   ~DerivedClass()
   {
      Dispose(false);
   }
}

It should be fairly clear to now why we are doing it this way. We want disposal to go recursively to base class. So, when we dispose the derived class resources, the base class disposes its own resources too.

To use or not to use a Finalizer:

We are almost done, really, we are. This is a section where Im supposed to tell you why you really shouldn’t use unmanaged resources whenever you need. It’s always a good idea not to write a Finalizer if you really really don’t need it. Currently we need it because you are using a unsafe file handle and we need to close it manually. To keep destructor free and as managed as possible, we should always wrap our handles in SafeHandle class and dispose the SafeHandle as a managed resource. Thus, eliminating the need for cleaning unmanaged resources and the overloaded Finalize() . You will find more about that here.

“Using” it right:

Before you figure out why I quoted the word using here, let’s finally wrap our work up. We have made our FileReader class disposable and we would like to invoke dispose() after we are done using it. We would opt for a try-catch-finally block to do it and will dispose the resources in the finally block.

FileReader fr = new FileReader();
try
{
    if (fr.Open(args[0]))
    {
        // Assume that an ASCII file is being read
        ASCIIEncoding Encoding = new ASCIIEncoding();
        int bytesRead;
        do
        {
            bytesRead = fr.Read(buffer, 0, buffer.Length);
            string content = Encoding.GetString(buffer, 0, bytesRead);
            Console.Write("{0}", content);
        }
        while (bytesRead &gt; 0);
        return 0;
    }
    else
    {
        Console.WriteLine("Failed to open requested file");
        return 1;
    }
}
finally
{
    if (fr != null)
        fr.Dispose();
}

The only difference you see here that we don’t explicitly call Close() anymore. Because that is already handled when we are disposing the FileReader instance.

Good thing for you is that C# has essentially made things even easier than this. Remember the using statements we used in Part-I? An using statement is basically a syntactic sugar placed on a try-finally block with a call to Dispose() in the finally block just like we wrote it here. Now, with that in mind, our code-block will change to:

using (FileReader fr = new FileReader())
{
    if (fr.Open(args[0]))
    {
        // Assume that an ASCII file is being read
        ASCIIEncoding Encoding = new ASCIIEncoding();
        int bytesRead;
        do
        {
            bytesRead = fr.Read(buffer, 0, buffer.Length);
            string content = Encoding.GetString(buffer, 0, bytesRead);
            Console.Write("{0}", content);
        }
        while (bytesRead &gt; 0);
        return 0;
    }
    else
    {
        Console.WriteLine("Failed to open requested file");
        return 1;
    }
}

Now you can go back to part-I and try to understand the first bits of code we saw. I hope it would make a bit better sense to you now. Hope you like it. Hopefully, if there’s a next part, I would talk about garbage collection algorithms.

You can find the code sample over github here.

I thought I knew C# : Garbage collection & disposal. Part-I

Yeah, that’s right, time to be a garbage guy. And if this line didn’t make you laugh, you probably know how bad of a stand up comedian I ever would make.

How things goes when anyone asks about these:

Now this is going to be long. And I’m going to jump into what I want to talk about right away. Let’s start with a regular Joe who writes C#. Let’s tell him to write a “safe” looking block of code that would essentially open a gzip file, read it as byte array, decompress the byte array using a buffer, write it over a memory stream and return it when he is done doing the whole thing.

This is something you might expect in return. This would be the block where the aforementioned byte array gets decompressed:

    public class Solution
    {
        public static byte[] Decompress(byte[] gzip)
        {
            using (MemoryStream memoryStream = new MemoryStream())
            using (GZipStream gzipStream = new GZipStream(memoryStream, CompressionMode.Decompress))
            {
                const int size = 4096;
                byte[] buffer = new byte[size];
                using (MemoryStream writeStream = new MemoryStream())
                {
                    int count = 0;
                    do
                    {
                        count = gzipStream.Read(buffer, 0, count);
                        if (count &amp;amp;amp;amp;amp;gt; 0)
                        {
                            memoryStream.Write(buffer, 0, count);
                        }
                    } while (count &amp;amp;amp;amp;amp;gt; 0);
                    return writeStream.ToArray();
                }
            }
        }
    }

And this would be a possible segment where you see the file being read:

    class Program
    {
        const int _max = 200000;
        static void Main()
        {
            byte[] array = File.ReadAllBytes("Capture.7z");
            Solution.Decompress(array);
        }
    }

And yes, the code sample credit goes to dotnetperls . The reason I started with an example before any explanations is I want to build up on this. I believe when you have a place to go, sometimes the fact that you know where you would end up reinforces what you will learn in the process. But it is very important here to know why you would end up here.

Breaking the code bits:

Let’s start breaking the code up into bits. Now, the first question you would ask the regular Joe like me is how do you claim this code block is “safe” and what do you mean when you say this is “safe”. The first answer would essentially be that there is a beautiful using block there which would essentially dispose the resources when it is done being used.

I focused three words here, throughout this articles I would bold out words like this and whenever I do that, if you don’t know exactly what I’m talking about, put these words in a dictionary in your brain and in turns I will explain all these. That also means if you think you know all of these, your journey ends here.

Words of wisdom:

The first word of wisdom here is dispose. And I would essentially start with some basics for it. Before going into dispose, we need to dive back on some proper backgrounds on .net garbage collection process. If you are totally new to this, garbage collector is an automatic memory manager, it lets you develop your application without having the need to free the memory every time. If I drove you inside more confusion, that means you need to know what happens when you allocate some memory, essentially which happens every time you declare and initiate any value or reference you write when you are writing C#.

Every time you write the new keyword to initialize an object in C#, you essentially allocate the object in a managed heap. And garbage collector automatically deallocates the objects that are not being used anymore from the managed heap “some time in the future”.  If you are already curious what is a managed heap, fret not. I will explain that too. But before that, lets talk about some fundamentals on memory.  When we essentially write C#, we essentially use a virtual address space. Since you have a lot of processes in the same computer who shares the same memory (read your RAM here) and you would essentially need them not to overlap with one another. Each process then needs to address a specific set of the memory for them and thus you have your virtual address space mapped for each process. By default 32 bit computers has around 2GB user mode virtual address space. When you are actually allocating memory, you allocate memory on this virtual address space, not the physical memory. For this, the garbage collector works on this virtual space and frees up this virtual memory for you automatically. Neat, huh?

I need memory:

What actually happens when we essentially write something like the following:

    static void Main()
    {
        Cat cat = new Cat("Nerd");
    }

Looks like we are initiating a harmless cat with the name Nerd. When you compile this C Sharp compiler will generate a common intermediate language (IL/CIL) code so the JIT compiler in CLR can compile those for any possible machine configuration. You see, I said a lot of jargons, I didn’t bold them out because I’m not going to talk about them here. Now the intermediate code that is being generated here kind of looks like this:

IL_000a:        newobj instance void CilNew.Cat::.ctor (string)

It looks about right, we only care about the newobj instruction here. This specific instruction needs to do three things.

  1. Calculate the total amount of memory you require for the object.
  2. Look for space in the managed heap for space.
  3. When the object is created, return the reference to the caller and advance the next object pointer  to the next available slot on the managed heap.

Im quiet sure number 1 is very very easy to understand here. Why would we need to look for space in the managed heap then? Lets look at this first.

managed-heap

If you look at the example here, now it should be pretty clear to you what I meant. If the next object pointer doesn’t find enough space to fit the next object in, you would expect a OutOfMemoryException . This can also happen when you don’t have enough physical memory either. This picture also can mislead you. I will come to that now. You might think now Virtual address space is contiguous always. Well, it’s not. Virtual address space can be fragmented. This means that there are free blocks or holes among used blocks and the virtual memory manager has to find a big enough free block to allocate so you can instantiate your variable. So, even if you have 2GB virtual address space, this does not mean you have 2GB contiguously. If you ask the virtual memory manager for 2GB of space, it could fail due to the fact you dont have that amount of contiguous address space. But for regular explanations that picture will suffice well.

Now we know how objects are allocated and we spent some time on what is the managed heap and how objects are allocated on the managed heap. The reason we discussed about this is to make you understand why you need garbage collection and when it is triggered.

States of virtual memory:

There are three states of the virtual memory. Free state says this block of memory is available for allocation. When you request for allocation, it goes to Reserved state. Much like booking a hotel. Now your memory block is reserved for you but not used yet. And no one else can use this block either because you reserved it. When you finally use it, it goes to Committed state. In this state, the block of memory has a physical storage association.

The garbage collector kicks in:

There are definitely multiple conditions which are responsible for garbage collection. And you already know the very first one now. When you run out of space for a new allocation in the virtual address space. We are going to jump in and see what actually the garbage collector does in a very basic level.

Garbage collection happens in two stages. Mark and Sweep. The mark essentially searches for managed objects that are referenced in managed code. It will attempt to finalize objects that are unreachable. That is the first thing to do on sweep stage. The last work to do on sweep stage is to reclaim the memory of the unreachable objects now.

I know you are thinking what is managed code. We would come back to this in the journey. Don’t worry. For now, keep in your mind that garbage collector can only deal with managed code.

So, the technique is essentially to mark objects the program might be using and just clean off the rest one. But, how the garbage collector would know which objects it needs to clean? How would it decide which objects are unreachable. It does it using something called Object Graph which is not essentially under the context of this article. But I do have a nice representation to go with.

object-graph-pr-before-gc

Lets assume this is the situation in the heap. You have a managed heap like this and lets assume the garbage collector kicks in due to less memory. It would essentially look like the following after the collection.

object-graph-pr-after-gc

Now it should be evident to you what basically happens in a garbage collection from a birds eye view. Marked man needs finalization and then it leaves.

I still didn’t properly explain how you essentially get these marked objects.  To understand that properly, we need to understand about generations.

Generations inside the heap:

Generations inside the heap essentially dictates how long the object would be essentially needed. And thus it is divided into long-lived and short-lived objects. There are three generations here and the indexing starts from zero:

  1. Generation 0: This is the youngest generation and contains short-lived objects. Temporary and newly allocated objects live here. This is the part of the heap where garbage collection happens very frequently.
  2. Generation 1: This is essentially a buffer between generation 0 and generation 2. Generation 2 contains long-lived objects. Generation 1 essentially holds the objects who is still looking to be short-lived but survived generation 0.
  3. Generation 2: This is the generation of long-lived objects. These objects are usually objects that stays for long time in the process. Statics come first in mind. And a new object can be allocated straight to generation 2 instead of generation 0 if it’s really big. Like a big array with a lot of space allocation.

Garbage collections are generation specific but the collection is recursive up until the younger generation. So it clears generation 1, it would also clear generation 0. If GC clears generation 2, it would also go down to clear generation 1 and generation 0.

I used the word survive  a moment ago. What I wanted to say is if an object doesn’t get reclaimed/cleaned up during a sweep operation over a generation it gets promoted to the next generation. If survival rate is higher in a generation, GC tries to increase the threshold of allocation of that specific generation. So in the next cleanup the application gets a big size of memory freed.

One more thing to remember here. Garbage collector would stop any managed thread to work. So, it has to be quick and efficient unless you are looking at performance penalties.

Back to the code bits:

If you have survived up until now, you deserve to go back to the code bit at the beginning. The first thing I’m going to clear up is the managed vs unmanaged resources. Managed resources are directly under the control of the garbage collector. It is a result of managed code which would eventually compile to intermediate language. Unmanaged resources are resources your garbage collector don’t really know about. That includes, open files, open network connections and of course unmanaged memory. Now, if you are using C# classes to do these, most of the times these are almost managed. That means the managed code does the “dirty work” inside and you don’t have to clean up these yourselves. The garbage collector would clean up the managed wrapper and the managed wrapper would clean up the unmanaged code in the disposal process.

Now, lets go back to the word dispose here. How would I dispose something off my code. Is there a method somewhere, something I could use? Indeed there is. A dispose method implementation has essentially two variations. Since your garbage collection can’t handle unmanaged resources, you need to wrap them. The first technique is to wrap them under any class derived from SafeHandle class and use IDisposable  interface to make it properly disposable. This very interface would expose the dispose() method you need and you would use that to dispose the resources yourself.

I explained in details how to do that in the next part.

To switch or not to switch

Now, the first thing that comes to mind when you are reading the title of this blog is a very “insightful” question that potentially dictates my reluctance to learn switch properly in C# even after all these years. Fret not, I’m the same old uncool dude who learns stuff late and in the process gets his ass kicked.

Today I want to go through the age old switch statement we all have been using. There’s a good number among us who prefers switch over an annoying if-else block. And I claim no crime there, not at all. Up until a couple of days ago I was a happy chump to use switch whenever I’m handling compile time constants like enums and writing if else blocks for my logical operation checks. Simple, happy as ever. All of that changed when my skyrim-ridden dull brain asked “Why there are two prominent branching paradigm here in this statically typed language” and kind of made a good point to find that out. I don’t remember what point my brain made at that time, sorry. But it’s my brain and I can’t deny much like deadpool cant.

deadpool

Let’s stay on focus, shall we? Lets go ahead and write a simple switch snippet like the following:

    public class Program
    {
        enum WhoGivesACrap
        {
            I_do,
            Nope_I_dont,
            Maybe_NotSure
        }
        static void Main(string[] args)
        {
            WhoGivesACrap whoGivesACrap = WhoGivesACrap.I_do;

            switch(whoGivesACrap)
            {
                case WhoGivesACrap.I_do:
                    System.Console.WriteLine($"{whoGivesACrap}");
                    break;
                case WhoGivesACrap.Maybe_NotSure:
                    System.Console.WriteLine($"{whoGivesACrap}");
                    break;
                case WhoGivesACrap.Nope_I_dont:
                    System.Console.WriteLine($"{whoGivesACrap}");
                    break;
            }
        }
    }

Now, this is definitely the first day at C# programming grade code. Dull, blasphemous and quiet frankly not worth of any attention. Same thoughts as mine to be honest. Thus I kept looking, booted up ildasm and asked what we can we see inside. Now, I can post the full de-assembled MSIL code here but that won’t make much of sense. Lets look on the parts we really want to look.

//000014:
//000015:             switch(whoGivesACrap)
    IL_0003:  ldloc.0
    IL_0004:  stloc.1
    .line 16707566,16707566 : 0,0 ''
//000016:             {
//000017:                 case WhoGivesACrap.I_do:
//000018:                     System.Console.WriteLine($"{whoGivesACrap}");
//000019:                     break;
//000020:                 case WhoGivesACrap.Maybe_NotSure:
//000021:                     System.Console.WriteLine($"{whoGivesACrap}");
//000022:                     break;
//000023:                 case WhoGivesACrap.Nope_I_dont:
//000024:                     System.Console.WriteLine($"{whoGivesACrap}");
//000025:                     break;
//000026:             }
//000027:         }
//000028:     }
//000029: }
    IL_0005:  ldloc.1
    IL_0006:  switch     (
                          IL_0019,
                          IL_0049,
                          IL_0031)
    IL_0017:  br.s       IL_0061

    .line 18,18 : 21,66 ''
//000018:                     System.Console.WriteLine($"{whoGivesACrap}");

I opted for dumping with my C# source code. And the thing that catches my eye is the switch  invocation with the three jump locations. And as my enums were adjacent it makes sense. CIL switch essentially create a jump table. The three arguments it takes are essentially jump locations which will be compared against my enums. Cool, at least now I have an answer why it is different than if-else-if-else. Remember I didn’t say if-else block because it makes more sense to do if-else in this fashion than checking else for no apparent reason.

If you are still not bored enough why don’t you go and have a look here?

Now, I also thought this would be the end of it. But the voices in my head reminded me to do one more thing. And that’s using non-adjacent values. Thus I modified my enum in the following fashion.

        enum WhoGivesACrap
        {
            I_do = 17,
            Nope_I_dont = 57,
            Maybe_NotSure = 945
        }

And hooked up ildasm again to figure out what happened this time. Now, my values are non adjacent. It doesn’t really make sense anymore to create around 900+ entry jump table.

//000014:
//000015:             switch(whoGivesACrap)
    IL_0004:  ldloc.0
    IL_0005:  stloc.1
    .line 16707566,16707566 : 0,0 ''
//000016:             {
//000017:                 case WhoGivesACrap.I_do:
//000018:                     System.Console.WriteLine($"{whoGivesACrap}");
//000019:                     break;
//000020:                 case WhoGivesACrap.Maybe_NotSure:
//000021:                     System.Console.WriteLine($"{whoGivesACrap}");
//000022:                     break;
//000023:                 case WhoGivesACrap.Nope_I_dont:
//000024:                     System.Console.WriteLine($"{whoGivesACrap}");
//000025:                     break;
//000026:             }
//000027:         }
//000028:     }
//000029: }
    IL_0006:  ldloc.1
    IL_0007:  ldc.i4.s   17
    IL_0009:  beq.s      IL_001e

    IL_000b:  br.s       IL_000d

    IL_000d:  ldloc.1
    IL_000e:  ldc.i4.s   57
    IL_0010:  beq.s      IL_004e

    IL_0012:  br.s       IL_0014

    IL_0014:  ldloc.1
    IL_0015:  ldc.i4     0x3b1
    IL_001a:  beq.s      IL_0036

    IL_001c:  br.s       IL_0066

    .line 18,18 : 21,66 ''
//000018:                     System.Console.WriteLine($"{whoGivesACrap}");

And I wasn’t wrong. Since the values are non-adjacent now CSC opted for loading the enums separately and used beq.s which stands for branch-if-equal in short form. Technically this looks like a vanilla if-else-if block. Although I can’t possibly say CSC would generate opcodes following this and this only.

I know this whole thing might sound insanely boring while performance wise this would only make a minuscule difference. Still, it’s always fun to answer the voices inside my head and they are much happier when they have something to hold against when they ask why.

Until next time!

Porting your WebApi 2.2 app to Azure Service Fabric

Now, to start talking about this, you gotta know why this is spawned off instead of spawning off a “hello world” to service fabric. I’m back to writing after a long time and it’s only fitting I do share what I have been doing meanwhile.

Usually in a production scenario you’d end up with a case where you have your applications lying around in several tiers. To be honest, I’m pointing reference to a production environment which is microservices driven. If you’re reading this and questioning why I did that you probably want to go down here to get yourself started with the basic concepts of Azure Service Fabric.

Now, I hope you have familiarized yourself with service fabric by now and I can start talking about how you can port your existing IIS hosted Asp.net Web Api 2.2 application to a self-hosted statless web api service in Azure Service Fabric.

To get yourself started please install Azure Service Fabric local development environment in your machine from here and go through the set of instructions to get yourself started. When you’re done with these, you’d see Visual Studio come up with a new set of project types for service fabric.

And it would pretty much like the following:

servicefabricprojecttype

Now, before we click the that elusive OK button, we need to understand what are the tasks at hand here. We have to do two things here.

1. Make sure our web api 2.2 app is compatible with self host as it was using IIS.

2. Make sure we can salvage the same Visual Studio project we used for our web project.

Making sure the existing web api 2.2 app is compatible of self hosting:

  1. For number one, one might think it should be farely easy to port a IIS driven web api app to a self hosted app, the reality might not be that easy. First, make sure you have a OWIN startup class initiated. Usually you would have one if you are using OWIN. And you’d also need a Program.cs file which is standard one for a console application as now your web app would be  a self-hosted application. So go ahead and add these two files to your web project and if you need reference, both of them are shown here.
  2. Now, although you have a startup.cs and Program.cs initiated already, you still haven’t converted your web app project to a console application. To do so, please go over your project properties and  on the Applications section, select Output Type as Console Application and set the Program class as the Startup object. Now, if you forget to add Program.cs as startup object which you’d only be able to do after you have created that file with a static void Main(string[] args) initiated on it, as long as it is named Program.cs the project would invoke it automatically.
  3. If you are using anything from Microsoft.Owin.Host.SystemWeb library, please make sure that these wont work anymore for you. For example if you are using code segments like the following to resolve file path to your traditional web app deployment and subsidiary folders like App_data, it wont work anymore for you.
     string path = System.Web.Hosting.HostingEnvironment.MapPath(@"~/App_Data/EmailTemplates/");
    

    So for things like these you might have to resort yourself to solutions that resolves deployment path in a console application.

  4. Now, there are some other pranks too. If you are familiar with Asp.net Identity and use it for your default identity provider, you’re in for a treat. Usually when you develop a expirable token generation paradigm for email and passwords, you’d need to use a IDataProtectionProvider which is usually the MachineKeyDataProtectionProvider under the namespace of Microsoft.Owin.Host.SystemWeb.DataProtection which you’d probably ditch because you would be using owin self host/Katana now. So, expect a null there where you try something like app.GetDataProtectionProvider() where app is your IAppBuilder. I ran into this myself and thanks to Katana being open source, you just pick up the MachineKeyDataProtectionProovider class from here.Just make sure you use it as  IDataProtectionProvider like the following:
    using System;
    using System.Web.Security;
    using Microsoft.Owin.Security.DataProtection;
    
    namespace TaskCat.Lib.DataProtection
    {
        using DataProtectionProviderDelegate = Func<string[], Tuple<Func<byte[], byte[]>, Func<byte[], byte[]>>>;
        using DataProtectionTuple = Tuple<Func<byte[], byte[]>, Func<byte[], byte[]>>;
    
        ///
    <summary>
        /// Used to provide the data protection services that are derived from the MachineKey API. It is the best choice of
        /// data protection when you application is hosted by ASP.NET and all servers in the farm are running with the same Machine Key values.
        /// </summary>
    
        internal class MachineKeyDataProtectionProvider: IDataProtectionProvider
        {
            ///
    <summary>
            /// Returns a new instance of IDataProtection for the provider.
            /// </summary>
    
            /// <param name="purposes">Additional entropy used to ensure protected data may only be unprotected for the correct purposes.</param>
            /// <returns>An instance of a data protection service</returns>
            public virtual MachineKeyDataProtector Create(params string[] purposes)
            {
                return new MachineKeyDataProtector(purposes);
            }
    
            public virtual DataProtectionProviderDelegate ToOwinFunction()
            {
                return purposes =>
                {
                    MachineKeyDataProtector dataProtecter = Create(purposes);
                    return new DataProtectionTuple(dataProtecter.Protect, dataProtecter.Unprotect);
                };
            }
    
            IDataProtector IDataProtectionProvider.Create(params string[] purposes)
            {
                return this.Create(purposes);
            }
        }
    }
    

    And you’d need to do the same with MachineKeyDataProtector class and use it as a IDataProtector like the following:

    using System.Web.Security;
    using Microsoft.Owin.Security.DataProtection;
    
    namespace TaskCat.Lib.DataProtection
    {
        internal class MachineKeyDataProtector: IDataProtector
        {
            private readonly string[] _purposes;
    
            public MachineKeyDataProtector(params string[] purposes)
            {
                _purposes = purposes;
            }
    
            public virtual byte[] Protect(byte[] userData)
            {
                return MachineKey.Protect(userData, _purposes);
            }
    
            public virtual byte[] Unprotect(byte[] protectedData)
            {
                return MachineKey.Unprotect(protectedData, _purposes);
            }
        }
    }
    
    

    Then you can use it as your replacement of your old one from Microsoft.Owin.Host.SystemWeb.DataProtection. This is not really needed if you don’t use Asp.net Identity but of to good use if you land in the same problem.Now, hopefully your current web api app would be self hosted just fine if you try something like the following:

    // Start OWIN host
    using (WebApp.Start<Startup>(url: baseAddress))
    {
      Console.WriteLine("Press ENTER to stop the server and close app...");
      Console.ReadLine();
    }
    

Now, lets try salvaging that Visual Studio project as much as we can.

Project changes when it comes to Visual Studio

  1. Now, you can use the converted project but from my personal experience it would be easier for you if you just click the new ServiceFabric project button now and create a new Stateless WebApi project. You can port/reuse most of your code and as the boilerplates are already written, you don’t really have to rewrite that.
  2. If you want to keep the old project intact, you can of course reference the old project in the new stateless web api service project and use your old startup class to hook up the OwinCommunicationListener instead of the one that it comes with. But beware, nuget might make it a nightmare for you if dependencies are not met/mismatched.

Hope you guys have fun with Azure Service Fabric and if you really want to look for more how you can do you own stateless web api from scratch take a look here. If I find time, I’d write one too.

Windows Phone For Newbies – Geofencing in Windows Phone 8.1 WinRT

Geofencing was one of the newest features Windows Phone 8.1 provided and for the developers it indeed was cool. Previously people did it their way but it’s always better to have an api in our hand. So, before digging into that i strongly suggest you guys to read my previous 2 articles if you haven’t done it already.

1. The Complete Reference of Map Control in Windows Phone 8.1 RT

2. Tracking Location in Windows Phone 8.1

These two would get you started pretty fast and you’d probably want that. If you already read my aforementioned 2 articles, I expect you know how to use a map control in Windows Phone 8.1. We’re going to need that later.

Geofencing:

Geofencing is a mechanism where you put a virtual boundary around a specific point of interest and you want to interact with the user when you arrive or leave that specific boundary. You seriously can do pretty amazing stuff with it actually.

Initial Work:

Now, let’s take a more practical approach to this, every time I go into my room, Im supposed to check my mails. But I usually forget that. How cool it would be if my phone reminds me every time I get into my room and tells me to check my mail?  😉

So, let’s go ahead and create a new Windows Phone app and name it GeoFencingTest. Now, lets see how to do that actually.

So, first of all go over your Package.appxmanifest file and go to the Capabilities Tab and select the Location capability.

Geofencing 1

Now, when you are done with that, let’s move on and start cracking. Before going deep lets get a basic UI running. And it looks like following:

GeoFencing - 2

And on the XAML part it looks like the following:

    &lt;Grid&gt;
        &lt;Grid.RowDefinitions&gt;
            &lt;RowDefinition Height=&quot;Auto&quot;&gt;&lt;/RowDefinition&gt;
            &lt;RowDefinition Height=&quot;4*&quot;&gt;&lt;/RowDefinition&gt;
            &lt;RowDefinition Height=&quot;8*&quot;&gt;&lt;/RowDefinition&gt;
        &lt;/Grid.RowDefinitions&gt;
        

        &lt;Grid Grid.Row=&quot;0&quot;&gt;
            &lt;TextBlock Text=&quot;GeoFencing Sample&quot; FontSize=&quot;25&quot; Margin=&quot;10&quot;/&gt;
        &lt;/Grid&gt;

        &lt;Grid Grid.Row=&quot;1&quot;&gt;
            &lt;Grid.ColumnDefinitions&gt;
                &lt;ColumnDefinition&gt;&lt;/ColumnDefinition&gt;
                &lt;ColumnDefinition&gt;&lt;/ColumnDefinition&gt;
            &lt;/Grid.ColumnDefinitions&gt;

            &lt;StackPanel Margin=&quot;10&quot;&gt;
                &lt;TextBlock Text=&quot;Longitude&quot; Margin=&quot;0,0,0,15&quot;  FontSize=&quot;20&quot;&gt;&lt;/TextBlock&gt;
                &lt;TextBlock Text=&quot;Latitude&quot;  Margin=&quot;0,0,0,15&quot; FontSize=&quot;20&quot;&gt;&lt;/TextBlock&gt;
                &lt;TextBlock Text=&quot;Altitude&quot;  Margin=&quot;0,0,0,15&quot; FontSize=&quot;20&quot;&gt;&lt;/TextBlock&gt;
            &lt;/StackPanel&gt;

            &lt;StackPanel Grid.Column=&quot;1&quot; Margin=&quot;10&quot;&gt;
                &lt;TextBlock Name=&quot;Longitude&quot; Margin=&quot;0,0,0,15&quot;  FontSize=&quot;20&quot;&gt;&lt;/TextBlock&gt;
                &lt;TextBlock Name=&quot;Latitude&quot;  Margin=&quot;0,0,0,15&quot; FontSize=&quot;20&quot;&gt;&lt;/TextBlock&gt;
                &lt;TextBlock Name=&quot;Altitude&quot;  Margin=&quot;0,0,0,15&quot; FontSize=&quot;20&quot;&gt;&lt;/TextBlock&gt;
            &lt;/StackPanel&gt;

        &lt;/Grid&gt;

        &lt;Grid Grid.Row=&quot;2&quot; Margin=&quot;10&quot;&gt;
            &lt;Grid.ColumnDefinitions&gt;
                &lt;ColumnDefinition Width=&quot;4*&quot;&gt;&lt;/ColumnDefinition&gt;
                &lt;ColumnDefinition Width=&quot;7*&quot;&gt;&lt;/ColumnDefinition&gt;
            &lt;/Grid.ColumnDefinitions&gt;
            &lt;Grid.RowDefinitions&gt;
                &lt;RowDefinition&gt;&lt;/RowDefinition&gt;
                &lt;RowDefinition&gt;&lt;/RowDefinition&gt;
                &lt;RowDefinition&gt;&lt;/RowDefinition&gt;
            &lt;/Grid.RowDefinitions&gt;

            &lt;CheckBox Content=&quot;Single Use&quot;&gt;&lt;/CheckBox&gt;
            &lt;TextBlock Grid.Row=&quot;1&quot; FontSize=&quot;20&quot; Text=&quot;Dwell Time&quot; Margin=&quot;0,8,0,0&quot;/&gt;
            &lt;TextBox Grid.Column=&quot;1&quot; Name=&quot;DwellTime&quot;  Grid.Row=&quot;1&quot;/&gt;

            &lt;Button x:Name=&quot;CreateGeoFencingButton&quot; Grid.Row=&quot;2&quot; Grid.ColumnSpan=&quot;2&quot; Content=&quot;Create GeoFence&quot; HorizontalAlignment=&quot;Stretch&quot; VerticalAlignment=&quot;Bottom&quot;/&gt;
            
        &lt;/Grid&gt;

    &lt;/Grid&gt;

Before going in deep on what is for what let’s get some basic stuff done. Let’s get our current location first.

Since you guys have seen in the last articles, it’s fairly easy to do so. All I’m doing here is get the current location using Geolocator and put the Latitude, Longitude and Altitude into the textblocks defined in the MainPage.xaml.

    public sealed partial class MainPage : Page
    {
        Geolocator geolocator = new Geolocator();
        CancellationTokenSource CancellationTokenSrc = new CancellationTokenSource();

        public MainPage()
        {
            this.InitializeComponent();

            this.NavigationCacheMode = NavigationCacheMode.Required;
        }

        protected override void OnNavigatedTo(NavigationEventArgs e)
        {

            InitializeLocation();

        }

        async private void InitializeLocation()
        {
            try
            {

                geolocator = new Geolocator();

                CancellationTokenSrc = new CancellationTokenSource();
                CancellationToken token = CancellationTokenSrc.Token;

                var position = await geolocator.GetGeopositionAsync(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(30)).AsTask(token);

                Longitude.Text = position.Coordinate.Point.Position.Longitude.ToString();
                Latitude.Text = position.Coordinate.Point.Position.Latitude.ToString();
                Altitude.Text = position.Coordinate.Point.Position.Altitude.ToString();
            }

            catch (Exception)
            {
                if (geolocator.LocationStatus == PositionStatus.Disabled)
                {
                    ShowMessage(&quot;Location Services are turned off&quot;);
                }

            }
            finally
            {
                CancellationTokenSrc = null;
            }

        }

        private async void ShowMessage(string message)
        {
            MessageDialog dialog = new MessageDialog(message);
            await dialog.ShowAsync();
        }
    }

The only thing that is interesting here is having a CancellationToken generated from a CancellationTokenSource. Usually these are used if you really want control over your task after it’s being fired, the rest of the procedure should look familiar as it’s a basic Geolocator usage to find current geolocation.

Setting up a GeoFence:

Now comes the setting up the actual GeoFence part and even this is fairly simple. The method looks like the following:

        Geofence geofence = null;
        int DefaultDwellTimeInSeconds = 0;
        private void SetupGeofence()
        {
            //Set up a unique key for the geofence
            string GeofenceKey = &quot;My Home Geofence&quot;;
            BasicGeoposition GeoFenceRootPosition = GetMyHomeLocation();

            double Georadius = 500;

            // the geocircle is the circular region that defines the geofence
            Geocircle geocircle = new Geocircle(GeoFenceRootPosition, Georadius);

            bool singleUse = (bool)SingleUse.IsChecked;

            //Selecting a subset of the events we need to interact with the geofence
            MonitoredGeofenceStates GeoFenceStates = 0;

            GeoFenceStates |= MonitoredGeofenceStates.Entered;
            GeoFenceStates |= MonitoredGeofenceStates.Exited;

            // setting up how long you need to be in geofence for enter event to fire
            TimeSpan dwellTime;

            dwellTime = DwellTime.Text!=&quot;&quot;? ParseTimeSpan(DwellTime.Text) : TimeSpan.FromSeconds(DefaultDwellTimeInSeconds);

            // setting up how long the geofence should be active
            TimeSpan GeoFenceDuration;
            GeoFenceDuration = TimeSpan.FromDays(10);

            // setting up the start time of the geofence
            DateTimeOffset GeoFenceStartTime = DateTimeOffset.Now;

            geofence = new Geofence(GeofenceKey, geocircle, GeoFenceStates, singleUse, dwellTime, GeoFenceStartTime, GeoFenceDuration);

            //Add the geofence to the GeofenceMonitor
            GeofenceMonitor.Current.Geofences.Add(geofence);

        }

        private TimeSpan ParseTimeSpan(string DwellTimeInSeconds)
        {
            return TimeSpan.FromSeconds(Double.Parse(DwellTimeInSeconds));

        }

        private static BasicGeoposition GetMyHomeLocation()
        {
            BasicGeoposition GeoFenceRootPosition;
            GeoFenceRootPosition.Latitude = 23.742766;
            GeoFenceRootPosition.Longitude = 90.417566;
            GeoFenceRootPosition.Altitude = 0.0;
            return GeoFenceRootPosition;
        }

Now, there’s quiet a lot to catch up.

The first one is definitely the GeofenceKey. Put a GeofenceKey in position just to identify a specific Geofence. I put s generic string to identify my home geofence.

Then I called up GetMyHomeLocation() method to set the geofence center to the location of my room. This will identify the center of our desired Geofence.

And the next thing to define is a radius that will define the radius of our circular geofence. We are using Geocircle object to define our geofence here, thus we are using a center and a radius in meters to defined our geofence. You can definitely see the next thing we did is define our geocircle. Actually the Geofence takes a Geoshape object to define it’s radius and currently only Geocircle is supported.

            string GeofenceKey = &quot;My Home Geofence&quot;;
            BasicGeoposition GeoFenceRootPosition = GetMyHomeLocation();

            double Georadius = 500;

            // the geocircle is the circular region that defines the geofence
            Geocircle geocircle = new Geocircle(GeoFenceRootPosition, Georadius);

Now, we have put a checkbox in our XAML named SingleUse. This is supposed to define whether our geofence would be used for only one time or not. You can set up temporary one time used geofence using this.


bool singleUse = (bool)SingleUse.IsChecked;

Now, the next thing we did is we defined how our desired geofence would be treated. Which events are we considering with this geofence? Should this be fired only when we get into that geofence or should this will only be considered when we exit the geofence, or even we remove the geofence? We can actually select all or a subset of that group. So we used Logical OR operator to add up all the cases we want our geofence to react upon.


            MonitoredGeofenceStates GeoFenceStates = 0;

            GeoFenceStates |= MonitoredGeofenceStates.Entered;
            GeoFenceStates |= MonitoredGeofenceStates.Exited;

Let’s move on into the time domain now. Now we need to figure how long we need to stay in our favourite geofence to trigger up it’s desired events. I put a textbox in front to define that in seconds, if that is defined, we fire out events from our geofence in that number of seconds and if not we fire out events in our default dwell time defined by 0 seconds.

            // setting up how long you need to be in geofence for enter event to fire
            TimeSpan dwellTime;

            dwellTime = DwellTime.Text!=&quot;&quot;? ParseTimeSpan(DwellTime.Text) : TimeSpan.FromSeconds(DefaultDwellTimeInSeconds);

You can even define how long a geofence should be up and running. Like I want my rooms geofence to be running for next 10 days.


            // setting up how long the geofence should be active
            TimeSpan GeoFenceDuration;
            GeoFenceDuration = TimeSpan.FromDays(10);

Even with that thing defined, one question remains, even though it will run for 10 days, when does the geofence gets activated so it can start counting to that defined 10 days. Answer is right below:


// setting up the start time of the geofence
            DateTimeOffset GeoFenceStartTime = DateTimeOffset.Now;

Well, the only thing left to do is create our desired geofence and add it to GeofenceMonitor.


            geofence = new Geofence(GeofenceKey, geocircle, GeoFenceStates, singleUse, dwellTime, GeoFenceStartTime, GeoFenceDuration);
            
            //Add the geofence to the GeofenceMonitor
            GeofenceMonitor.Current.Geofences.Add(geofence);

Now, that’s how you create your own geofence and add it to the GeofenceMonitor. Now you might wonder that whether you’d really have to define all that parameters to create a geofence. Not actually, the littlest you can do is define an Id and a Geocircle to start your geofence. 😉 The rest of the values would take the default assumption.

Handling GeoFence Notifications on Foreground:

We have all our background work to create a geofence but we wont be able to do a single thing unless we test our geofence. Now, we have already defined that our geofence would trigger events after the aforementioned dwellTime and will trigger events if I enter or leave it. So, I definitely have to attach a event handler at a certain place. Don’t I?

Before that, let’s create the method stub for the click event handler for CreateGeoFencingButton and start SetupGeofence() method inside it. If you don’t know how to do that, select the CreateGeoFencingButton in XAML, go to properties, select events tab and double click on the click textbox.


private void CreateGeoFencingButton_Click(object sender, RoutedEventArgs e)
{
SetupGeofence();
}

Then let’s add the handler in the end of SetupGeofence()


GeofenceMonitor.Current.GeofenceStateChanged += Current_GeofenceStateChanged;

Now, let’s say we want to show a message every time I enter and exit my room’s geofence. The handler might look like the following:

        private async void Current_GeofenceStateChanged(GeofenceMonitor sender, object args)
        {
            var Reports = sender.ReadReports();

            await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =&gt;
            {
                foreach (GeofenceStateChangeReport report in Reports)
                {
                    GeofenceState state = report.NewState;

                    Geofence geofence = report.Geofence;

                    if (state == GeofenceState.Entered)
                    {
                        ShowMessage(&quot;You are pretty close to your room&quot;);

                    }
                    else if (state == GeofenceState.Exited)
                    {

                        ShowMessage(&quot;You have left pretty far away from your room&quot;);
                    }
                }
            });
        }

There you go! That’s how you set up your geofence in an app running in the foreground.

Setting up GeoFence on the background:

Now, this is where things get a little bit tricky. There’s a number of steps involved in this.

  1. Create a Windows Runtime Component Project and add it to your solution.
  2. Change Package.appxmanifest to declare the BackgroundTask that has been created.
  3. Register the BackgroundTask
  4. Handle Geofence Notifications from Background Task

Create a Windows Runtime Component Project:

Usually, background tasks are windows runtime component so you have to add one to your solution. Please go over to your solution and add a Windows Runtime Project.

GeoFencing - 3

GeoFencing - 4

After the new project is created go over class1.cs and rename it to your liking . I renamed it to BackgroundTask and added IBackgroundTask interface to the class and implemented it. I didn’t write anything afterwards. Let’s just keep it like that for a while and move to our GeofenceTest Project. Go over references and   add the newly created GeofenceTask.BackgroundGeofenceTask project to it. So our Wp8.1 project now has reference to the GeofenceTask.BackgroundGeofenceTask project.

Change Package.appxmanifest to declare the BackgroundTask:

Now let’s scoot over to our Package.appxmanifest file and go over Declarations tab. From Available Declarations combobox select BackgroundTask and select loction under the Properties list.

GeoFencing - 5

Now on the Entry point field please put the full qualified assembly name of your newly created background task. Don’t get confused. It’s fairly easy. Let’s have a look at BackgroundGeofenceTask class first.

namespace GeofenceTask
{
    public sealed class BackgroundGeofenceTask : IBackgroundTask
    {
        public void Run(IBackgroundTaskInstance taskInstance)
        {
            throw new NotImplementedException();
        }
    }
}

Now put the name as (namespace.classname) format so that leaves us with GeofenceTask.BackgroundGeofenceTask

GeoFencing - 6

Register the BackgroundTask:

Now comes the part where you register the BackgroundTask. We need to register the background task to the app so it knows it to invoke it on time.  The registering method looks like following:

private async Task RegisterBackgroundTask()
        {
            BackgroundAccessStatus backgroundAccessStatus = await BackgroundExecutionManager.RequestAccessAsync();

            var geofenceTaskBuilder = new BackgroundTaskBuilder
            {
                Name = &quot;My Home Geofence&quot;,
                TaskEntryPoint = &quot;GeofenceTask.BackgroundGeofenceTask&quot;
            };

            var trigger = new LocationTrigger(LocationTriggerType.Geofence);
            geofenceTaskBuilder.SetTrigger(trigger);

            var geofenceTask = geofenceTaskBuilder.Register();
            geofenceTask.Completed += GeofenceTask_Completed;

            switch (backgroundAccessStatus)
            {
                case BackgroundAccessStatus.Unspecified:
                case BackgroundAccessStatus.Denied:
                    ShowMessage(&quot;This application is denied of the background task &quot;);
                    break;
            }

        }

Let’s see what we have in here now. The very first one is get the BackgroundAccessStatus. That helps us to know whether our application is capable of accessing BackgroundTasks. Now, let’s move up and use BackgroundTaskBuilder to create the background task we intend to create. Now, if you look at the TaskEntryPoint we provided there, you’d now realize why we created our Background Task project before and added it to the manifest. Because we are using the same entry point name here. I put a name to identify the background task. This is needed as if there is another background task with the same name you would be thrown an exception. If you want to know whether there is another BackgroundTask with the same name you can iterate through BackgroundTaskRegistration.AllTasks and then make sure whether there is one with the same name or not. You can use a method like the following to do so:

        public static bool IsTaskRegistered(string TaskName)
        {
            var Registered = false;
            var entry = BackgroundTaskRegistration.AllTasks.FirstOrDefault(keyval =&gt; keyval.Value.Name == TaskName);

            if (entry.Value != null)
                Registered = true;

            return Registered;
        }

I haven’t used that in this solution but feel free to definitely use this. You can even unregister to make sure that your Background Task is unregistered to open up places for others

        public static void Unregister(string TaskName)
        {
            var entry = BackgroundTaskRegistration.AllTasks.FirstOrDefault(keyval =&gt; keyval.Value.Name == TaskName);

            if (entry.Value != null)
                entry.Value.Unregister(true);
        }

The next thing that comes off is defining the LocationTrigger for the BackgroundTaskBuilder object. So, we defined a new LocationTrigger of type LocationTriggerType.Geofence and used SetTrigger method to set it up. Then we used BackgroundTaskBuilder object to register the task and we instantiated a GeofenceTask_Completed event handler for that too.

At the bottom you’d see a switch with backgroundAccessStatus and putting a MessageDialog when it’s denied or unspecified.

Handle Geofence Notifications from Background Task:

So, lets move to our Windows Runtime Component which actually is the Background Task project and implement the IBackgroundTask interface. A method named

public void Run(IBackgroundTaskInstance taskInstance)

would pop up, we’d put our regular geofence event handling in there like there in the Foreground example. Now it looks like following when we are handling it from background

    public sealed class BackgroundGeofenceTask : IBackgroundTask
    {
        public void Run(IBackgroundTaskInstance taskInstance)
        {
            var Reports = GeofenceMonitor.Current.ReadReports();
            var SelectedReport =
                Reports.FirstOrDefault(report =&gt; (report.Geofence.Id == &quot;My Home Geofence&quot;) &amp;&amp; (report.NewState == GeofenceState.Entered || report.NewState == GeofenceState.Exited));

            if (SelectedReport==null)
            {
                return;
            }

            //We are creating a toast this time as this is running in Background
            var ToastContent = ToastNotificationManager.GetTemplateContent(ToastTemplateType.ToastText02);

            var TextNodes = ToastContent.GetElementsByTagName(&quot;text&quot;);

            if (SelectedReport.NewState == GeofenceState.Entered)
            {
                TextNodes[0].AppendChild(ToastContent.CreateTextNode(&quot;You are pretty close to your room&quot;));
                TextNodes[1].AppendChild(ToastContent.CreateTextNode(SelectedReport.Geofence.Id));
            }
            else if(SelectedReport.NewState == GeofenceState.Exited)
            {
                TextNodes[0].AppendChild(ToastContent.CreateTextNode(&quot;You are pretty close to your room&quot;));
                TextNodes[1].AppendChild(ToastContent.CreateTextNode(SelectedReport.Geofence.Id));
            }

            var settings = ApplicationData.Current.LocalSettings;

            if (settings.Values.ContainsKey(&quot;Status&quot;))
            {
                settings.Values[&quot;Status&quot;] = SelectedReport.NewState;
            }
            else
            {
                settings.Values.Add(new KeyValuePair&lt;string, object&gt;(&quot;Status&quot;, SelectedReport.NewState.ToString()));
            }

            var Toast = new ToastNotification(ToastContent);
            var ToastNotifier = ToastNotificationManager.CreateToastNotifier();
            ToastNotifier.Show(Toast);

        }
    }

Now, this looks a wee bit different from the first example. The first change that is noticable is that we are now selecting the reports based on our geofence id and the report’s new state. As we are in background now, we are using our geofence id to get the proper geofence from the whole list. And as now we are in background we are using a Toast Notification instead of a MessageDialog to show our notification.

Handling it from the App Side:

Now you guys might get confused what to do with the foreground code we made earlier. Do we need Current_GeofenceStateChanged event handler anymore. Now here we might have to be a bit careful. Now if we want our app to react different when it is on foreground and make some UI changes it needed then we have to use GeofenceTask_Completed event rather than Current_GeofenceStateChanged. And there’s another thing to be added. We get our GeofenceMonitor Reports using  GeofenceMonitor.Current.ReadReports() and this step can only be done once for every change. So if your background reads it first then, your foreground app would not be able to read it. So, we have to save it somewhere when the background reads it so our foreground app can read it from there.

So we are using ApplicationData.Current.LocalSettings to save our state in the Background Task. If you look closely you’d find the following snippet:


            var settings = ApplicationData.Current.LocalSettings;

            if (settings.Values.ContainsKey(&quot;Status&quot;))
            {
                settings.Values[&quot;Status&quot;] = SelectedReport.NewState;
            }
            else
            {
                settings.Values.Add(new KeyValuePair&lt;string, object&gt;(&quot;Status&quot;, SelectedReport.NewState.ToString()));
            }

So, in our app side GeofenceTask_Completed event handler we’d read the status and fill up a new textblock.

Thus we added a new set of textblocks in the MainPage.xaml


&lt;TextBlock Grid.Row=&quot;2&quot; Text=&quot;Status&quot; FontSize=&quot;25&quot;/&gt;
&lt;TextBlock Grid.Row=&quot;2&quot; Grid.Column=&quot;1&quot; Name=&quot;Status&quot;  FontSize=&quot;25&quot;/&gt;

Now, our GeofenceTask_Completed event handler looks like the following:

        private async void GeofenceTask_Completed(BackgroundTaskRegistration sender, BackgroundTaskCompletedEventArgs args)
        {
            if (sender != null)
            {
                // Update the UI with progress reported by the background task
                await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =&gt;
                {
                    try
                    {
                        // If the background task threw an exception, display the exception
                        args.CheckResult();

                        var settings = ApplicationData.Current.LocalSettings;

                        // get status
                        if (settings.Values.ContainsKey(&quot;Status&quot;))
                        {
                            Status.Text = settings.Values[&quot;Status&quot;].ToString();
                        }

                    }
                    catch (Exception ex)
                    {
                        ShowMessage(ex.Message);
                    }
                });
            }

        }

Now, if you look closely you will see all we did here is get the “Status” object from our ApplicationData.Current.LocalSettings and posted it on the TextBlock named Status. 🙂

Now, all we have left to do is do a little change in UI and add an extra button to set up the Geofence background task. Now it looks like the following:

GeoFencing - 8

I created the method stub for the new buttons click event too and it looks like the following:

        private async void CreateBackgroundServiceButton_Click(object sender, RoutedEventArgs e)
        {
            if (IsTaskRegistered(&quot;My Home Geofence&quot;))
            {
                Unregister(&quot;My Home Geofence&quot;);
            }
            await RegisterBackgroundTask();
        }

Testing the App on the Emulator:

You can definitely test the app on the emulator. Like always click the debug button with a emulator selected as the device and when it opens go to additional tools button and open up the location tab. Select a close location to your geofence, I did select a close one to my room and the status changed itself to “Entered”. It was that easy! You can even try the foreground example in the same way!

GeoFencing - 7

The source code is here.

Stay frosty! Hope this would help.