Systematically Tackling Performance Issues in Your Code

“I just got this report from this monitoring service and the new page takes 20 seconds to load!”

Almost all of us in software have had someone complain that a web page or part of a program takes too long. The information thrown at us is not always clear, and someone wants it fixed. And they want it fixed now. Resist the temptation to create a knee-jerk solution and remember to tackle this problem systematically.

Verify The Current State

First, get your hands on the performance report they’re talking about. Continue reading

Tips for Good Entity Framework Code First Models

One of the things in programming that you should get right the first time is code first models. Refactoring a database is way more work than normal code, and there will be cases where it’s so much work that it won’t be worth it.

So here’s a few tips from my recent (sometimes painful) refactoring work.

  1. Don’t think about how your model will look as a table, too much. Continue reading

My Entity Framework Code First Checklist

How many times have we all worked on some technology and then wanted to use it again on a different project but couldn’t figure out how to get started? We’ve all done it, and Entity Framework with Code First is definitely one of those technologies that is hard to get started.

…unless you have a checklist. That’s why I’m writing this. It’s my checklist. It helps me start new projects and hopefully it helps you, too. Continue reading

Using Entity Framework 6 with an existing database

DungeonMart’s starter data set is designed for SQL Server, so it’s time to leave DocumentDB behind and start the SQL work. First, I need to create the SQL database and load the seed data. The schema files that I’m using for the initial load can be found here: https://github.com/qanwi1970/dnd-srd-35/tree/master/V1. If you want to follow along, just run them against a new database and you’re good to go.

Since I already have a database, I’m going to use Entity Framework’s database first model to get talking to it quickly. Continue reading

Creating Automated Integration Tests for a DocumentDB Repository

You can unit test all your layers, but when you get to the repository it becomes an exercise in mocking that provides little real benefit. That’s why we have to give up on the unit test idea for repositories and embrace integration tests. There’s nothing wrong with integration tests, as I explained a while back. So let’s review the criteria of effective automated tests and see if this is worth while.

  1. Automated – Yes, my tests will be standard NUnit tests that can be run just about anywhere by just about any automated process. Continue reading

Refactoring My Repository Methods to be Asynchronous

Now I’ve written a DocumentDB based repository for Equipment data in DungeonMart, but I didn’t try to make it perfect, I just tried to make it work. Here it is, if you’re catching up: Part 1 and Part 2.

My next step in getting it closer to perfect is to make all the methods aynchronous. Face it, we’re going out to the cloud to get or write data and we shouldn’t make the whole service wait while that happens. DocumentDB gives you Async methods at least for writing, and we can figure out how to make the read methods async. If we were in Entity Framework, we would already have async methods for both reading and writing.
Continue reading

Creating a DocumentDB Repository Layer (Part 2 – Writing)

“Last time, on Bill DeLude dot com…”

Hmm, it kind of loses its effect without a good narrator voice. Does someone have Morgan Freeman’s number? Because he could say that last time I talked about creating a data repository class using DocumentDB as the data store and covered the setup of the class and the read methods. This time, I’m going to go over the write methods.

But first a disclaimer before we get started. Not everything in this class is complete. As with normal programming, I like to take a highly iterative approach. First, I’ll make it work. Then I’ll improve some part of it. And then I’ll improve some other part, and so on. There’s two things conspicuously missing from today’s code. First, the write methods, as they are offered up, are synchronous. Second, it doesn’t allow for the Equipment type to change. I’ll address both of these in a later post.

First, I’m going to work on adding Equipment to the collection. Here’s the implementation of the AddEquipment method:

public Equipment AddEquipment(Equipment equipment)
{
    dynamic doc = Client.CreateDocumentAsync(CollectionLink, equipment)
        .Result.Resource;
    Equipment result = doc;
    return result;
}

Adding a document to a collection in DocumentDB is pretty straight-forward. All you really have to do is call CreateDocumentAsync with the link to the collection and the document you’re adding.

I made it a little more complicated because I like to return the added document back to the caller, so they can have any changes such as the id or a timestamp. To accomplish that, I need to get the Resource of the CreateDocumentAsync result as a dynamic type. Then I put that into an Equipment type and return the equipment.

Now that we can add documents, let’s make sure we can update them. Updating in DocumentDB is a two step process. You first have to get a document link by reading the collection, and then you replace the document with your changed version. Gone are the days of “UPDATE Equipment SET …” because DocumentDB, as with most NoSQL databases, just doesn’t work that way. Here’s the code for updating:

public Equipment UpdateEquipment(string id, Equipment equipment)
{
    var doc = Client.CreateDocumentQuery(CollectionLink)
        .AsEnumerable().First(d => d.Id == id);
    dynamic updatedDoc = Client.ReplaceDocumentAsync(doc.SelfLink,
        equipment).Result.Resource;
    Equipment result = updatedDoc;
    return result;
}

Notice that when I get the document this time, I’m not specifying a type to return. By omitting the return type, I get it as a Document and not as an Equipment. I want to do that because Equipment doesn’t have the self link I need to do the replace on the next line. Once you have the document, replacing it is pretty simple. Just call ReplaceDocumentAsync with the document’s self link and the new document. Returning the updated equipment requires some extra work, but it’s just like the Create.

Finally, we need to delete documents from the collection. This works a lot like updating: get a self link, then call a delete method. It looks like this:

 public void DeleteEquipment(string id)
{
    var doc = Client.CreateDocumentQuery(CollectionLink)
        .AsEnumerable().First(d => d.Id == id);
    Client.DeleteDocumentAsync(doc.SelfLink);
}

And that’s it. Now you have a full blown Equipment repository using DocumentDB as the data store.

The code that was used at the time of this writing is in my github at this commit.

Creating a DocumentDB Repository Layer (Part 1 – Reading)

Even with NoSQL, having a repository layer is a good idea. In the world of DocumentDB, writing this layer is really simple. First, we’ll start with an interface, of course. In your application, this interface might already exist.

public interface IEquipmentRepository
{
    IEnumerable<Equipment> GetEquipments();
    Equipment GetEquipmentById(string id);
    Equipment AddEquipment(Equipment equipment);
    Equipment UpdateEquipment(string id, Equipment equipment);
    void DeleteEquipment(string id);
}

As, you can see, this is a pretty straight-forward interface with CRUD (Create-Read-Update-Delete) operations. Notice that the identifier is a string, not an integer like you might be used to seeing. DocumentDB, like other document databases, likes the identifier to be a GUID. Don’t try to make it the .NET GUID type. Just go with string, and then the database will handle it just like SQL does with integer identifiers.

If you’re following along in your own code, you’ll need the Equipment model. Here’s a snapshot of what it looked like when this post was written: Equipment.cs.

The next thing would be to implement the repository, but before you do that you’ll need a couple of extension methods to make your life easier. You can find them in my previous post on getting existing databases and collections. Do that, and then come back. I’ll be right here. You can also look at the source for the whole project, using the link at the bottom.

You’re going to need two fields in your repository: the DocumentClient and the Document Link. This is how I’m getting them at the time I write this. It isn’t optimal, but I have plans for more posts about making that better and those topics are beyond the scope of this post. So here it is:

private static readonly DocumentClient Client =
    new DocumentClient(new Uri(<Your Url>), <Your Key>);

private static readonly string DocumentLink =
    Client.GetOrCreateDocumentCollectionAsync("dmart", "equipment")
    .Result.DocumentsLink;

Now that we have those, let’s implement the first method, GetEquipments. There’s not much to it, but there’s a lot to talk about:

public IEnumerable<Equipment> GetEquipments()
{
    return Client.CreateDocumentQuery<Equipment>(DocumentLink)
        .AsEnumerable();
}

This method uses the DocumentClient we created as a field. It calls the CreateDocumentQuery<T> method to query the document collection referenced by the DocumentLink (our other field).

Notice that the method returns an IEnumerable<T> and not a List<T>. What this means is that when the method returns, we haven’t actually queried the collection. We’re putting that off until later to allow the caller to filter on the IEnumerable. This makes the usage of our get list method more flexible, and possibly saves us from unnecessarily passing a huge collection across the wire.

However, in some cases there will be a way of filtering that is so common we’re just going to provide the users of our class a built in method for that. The seminal example would be wanting to get an Equipment by it’s identifier. For that, we’ll implement the next method.

public Equipment GetEquipmentById(string id)
{
    return Client.CreateDocumentQuery<Equipment>(DocumentLink)
        .AsEnumerable().First(d => d.id == id);
}

It looks a lot like the last method, except that it filters and returns a single Equipment object. Notice that I’m using First instead of FirstOrDefault, which will throw an exception if the Equipment with that guid doesn’t exist. Since a guid is a really hard key to fake, I don’t expect that will happen much and an exception is probably appropriate. I might regret that decision later.

That’s it for reading from your DocumentDB collection. In my next post, I’ll go over the remaining write methods.

The code used for this post lives here:
https://github.com/qanwi1970/dungeon-mart/tree/3ad6081b36b5fc804b0c6dce03daa1a2699edd87

Continue with the writing methods in Part 2.

Update: The code when I published was missing the .AsEnumerable() in the GetEquipmentById method. Sorry about that. By the time I found and fixed it, the code had moved on and would just be confusing to point to, now.