Developers procrastinating their way to better code

In The surprising habits of original thinkers Adam Grant describes the sweet spot between precrastinators and procrastinators for optimal original thought.

In short, completing your tasks far ahead of time or at the very last minute leads to less original thought.

Think of this in terms of a developer getting their tasks done.

Typically we're pulling tickets off job queue. If you're a developer then you've probably worked with GitHub Issues or Jira or some similar system. Tickets are usually prioritized from most important to least important.

If we've never seen the ticket or problem statement that we've just pulled to work on then our original thought process for solving this is the equivalent of an extreme procrastinator. We're learning about the problem at the same time as we're solving it. This leads to less original solutions and probably the first solution we think of instead of the best solution.

One way to mitigate this is to have grooming sessions. We meet as a team and go through the backlog of tickets and in addition to prioritizing them the ticket creator will explain the ticket in more detail so that we can estimate the effort in completing this. More details are also added to the ticket if needed. The team sometimes discusses possible approaches to solving this. Most important, in my opinion, is that this problem has been dropped into your subconscious and whether you like it or not your brain is noodling on a solution in the background.

Something that I feel we don't do often enough is "kick tickets down the road." In other words, there are tickets that we should procrastinate on that we don't.

Some tickets are critical and need to be done immediately. Major security vulnerabilities fall in this category.

If the ticket is not urgent and can be punted and you or the team are uncomfortable with the current array of possible solutions then do it. Kick that ticket down the road. And do it intentionally and without shame. Add a comment on the ticket stating exactly that:

The team discussed this issue on DD-MMM-YYYY and we were unable to come up with an idea for addressing this in way that we wanted so we're putting this back in the queue to mull on.

My Ideal Full Stack JavaScript Team Environment

Here are some of the tools and practices that I'm currently in favor of when working in a team environment on a JavaScript Full Stack project. In no particular order.

Pull Requests

All code should have a second set of eyes on it at some point. This serves the following purposes:

1. Catch bugs or potential bugs.
2. Teach both the reviewer and reviewee.
3. Keeps the team members up-to-date with what other teams members are working on.
4. Introduces best practices, new features/techniques and new libraries.

PRs should almost always be assigned to someone. Others can be @mentioned. Almost anyone (senior enough) should be able to signed off with a Looks Good To Me (LGTM) in the comments and merge.

The developer submitting the PR should let the reviewer know the urgency of the PR.

1. Urgent. The submitter immediately accepts the PR. The review is done post-facto as a learning and review process. Can also catch bugs that are fixed in the next PR.
2. Normal: Someone is expected to review this within an hour.
3. Low: Review within the next day.

Linting

An automated linting tool should be part of the commit process. Locking in lint errors to a maximum value will allow the gradual introduction of linting to existing projects.

My default (and currently the mostly popular) linting tool for JavaScript projects is ESLint.

Developers should be encouraged to have a lint helper plugin in their editors so they rarely/never see lint errors on commit.

The team should have an evolving .eslintrc file that's shared across all projects. Some projects might have specific override .eslintrc rules if they are catching up with the standard or have exceptions.

Keep the master .eslintrc in a dot-files folder that's a sibling of all the other projects and reference it from there.

ES6

ES6 is here. There are very few reasons not to start using this. Babel is an excellent transpiler that will allow developers to start using those features.

Use code reviews to help teach other team members ES6.

If you're not using ES6 you'll start leaking developers to other teams and companies. More on this below...

Hot Loading

Most projects should have tooling setup so that they can quickly iterate on small changes to code that they are working on and see the results.

This rapidly accelerates both learning and development.

If, for example, you're working on a React/Node project then, in my opinion, you'd have the Webpack Hot Module Replacement transpiling your changes in memory and making them almost instantly available in the browser.

As developers we're always on a learning curve. We can never be expected to know everything. Using a fast responsive hot loader we can quickly try different techniques and see the results. This will speed our learning of how that corner of the world works.

Front End Framework

Most projects should be using a modern front-end framework for the UI component. The most popular over the last several years has been AngularJS. The current rising start is ReactJS.

I'm in favor of ReactJS. It looks like AngularJS and EmberJS are converging on what ReactJS has introduced.

Latest Versions

We should strive, as much as possible, to be using the latest versions of everything. It's perfectly acceptable to hold off on the brightest and shiniest until it's stable. That should be a conscious decision and not the default.

Npm has a great command npm outdated --depth 0 that will give us a summary of how our project looks compared to what's available. Use that command with the -g option to check your globally installed tooling.

The 3 main reasons usually given for keeping up with the latest version (1. Features 2. Performance 3. Bug Fixes) are completely irrelevant. The reason you want to stay current is to keep your developers happy and engaged. Nobody wants to work on yesterday's technology.

Keeping a buzz in the team about moving to the next version will keep everyone up-to-date on what's happening and engaged with the team. The grass-is-greener syndrome with other teams or companies won't exist on the technology front because you'll be using the latest. In fact I've successfully used this as a recruiting tool for developers who are stuck in other companies or teams on old technology.

Related:

Your tooling should automatically keep your project's dependencies up-to-date with what would be installed when someone initially installs the dependencies on the project. I use the npm-update-outdated module as part of my pre-commit hook to update my dependencies before the tests are run so I know that my tests have run against what the CI server will use.

Project Onboarding

If someone asks you to work on a project then you should be able to do the following pretty quickly.

1. git clone
2. npm install
2. npm test (should run clean)
4. npm start

And have a server up and running that the browser can hit and interact with.

There are lots of complexities that might stop something that simple from working.

The project's readme should have bullet proof (as good as possible) instructions on how to address that.

For example, most developers have Docker. If you have a dependency on a MongoDB then npm start or npm install should run Docker and pull down a MongoDB container and get it up-and-running.

A new-to-the-project developer who experiences problems with an out-of-date README.md and who then updates the README.md should be given the rest of the day off as a reward. (That might be a bit extreme. This almost never happens so it might also be okay.)

Testing

All projects should have tests. These should run during a pre-commit hook. Ideally these are run implicitly through the Code Coverage tool.

When bugs are fixed a test should usually be written demonstrating the bug before it's fixed. This test will prevent a regression on this bug later.

Tests are difficult to write. Do lots of cross-education with other team members during PRs with tests. Get to know and love all the tooling around tests. When you're using a tool for the first time ask for help and opinion in the PR.

Some of the tools I use for testing:

• Mocha
• Jasmine
• Istanbul
• Sinon
• Rewire
• Proxyquire

Code Coverage

Code coverage such as Istanbul can be automated as part of a pre-commit hook. It will help make sure that critical sections or sections that you're about to change have sanity tests. It has awesome "graphics"

Code Coverage Minimum Levels

It's easy for a developer to commit new code that doesn't have test coverage. In my enthusiasm to get my new code out there I often forget to write tests for it. Using a tool like Istanbul we can prevent code coverage from dropping from the previous level in a pre-commit hook. This will remind us that we need to add tests.

Update 11/2/15

Isomorphism

I didn't used to think that isomorphic JavaScript was that important. I believe that it is now becoming important.

Least of all you should never have duplicate code that's duplicated for the sake of using on both server and client. There is almost always a way to provide the same code block to both client and server to provide a DRY and isomorphic code-base.

Parsing text for a Credit Card Number

A while back I wrote some code that parses text and looks for credit card numbers embedded in the text and then optionally masks all or some of the credit card in the text. This code is open source and available on GitHub: Credit Card Parsing

The main use case for this code was to remove credit card numbers that customers would email in to customer support when they were trying to buy something. The Payment Cards Industry (PCI) standards require the credit card data be stored under more secure conditions than most default standard storage configurations.

If customers' support tickets are being stored in a regular unencrypted database and they are sending you their credit card numbers then you are not PCI compliant. You have two choices. Make the support ticket database PCI compliant or remove the credit card numbers from the text that you're storing in this database.

The code in this library provides a mechanism to find and then optionally mask the credit card numbers in any block of text.

While I was developing this code I learned something interesting. 0.6% of all UUIDs/GUIDs have a valid credit card number in them. When I say valid I mean that it passes the Luhn Algorithm. I discovered this because our support emails had a lot of UUIDs that were part of URLs and other keys that the customers needed to communicate to us.

The first time I saw this false positive I thought that it was a one in a million chance. Not wanting to rely on that (bad) luck I used Monte Carlo Simulation to determine the probability of this happening and it came out at 0.6%. I had to adjust the credit card finding algorithm to ensure that what it thought was a credit card was not part of a UUID.

Development of this library was an unusual experience in that unit tests were immediately useful and protective. Each time a defect came back from production (i.e. a missed credit card or a false positive) I added that block of text* to a unit test and confirmed that it failed. I then fixed the code to ensure that the unit test passed and every time I did that it broke one of the other unit tests. My adherence to strictly unit test everything I wrote in this project ensured that it never had any regression bugs.

* If it was a missed credit card then that credit card would be replaced by a fake one before it was inserted into the unit test. I used Graham King's Credit Card Number Generator to create these fake credit cards.

Ship It Day #2

On Friday July 12, 2013 our team, held our second Ship It day with the judging taking place the following Monday, July 7/15.

At the previous ship it day we focused all the prizes on the demonstrations that were given. In both instances we asked a panel of judges to submit a score, based on their own criteria, from 0 to 10 for each team/demo.

Initially the judges wanted guidance on how to score the submissions. Several attributes were suggested but I resisted this because I felt that each judge already has a different viewpoint of what they like to see in a demo and what is important to them. The selection of a wide range of judge types was more important than giving them attributes to score on that might not make sense to each individual. The judges were Business Analysts, User Experience (UX) Engineers, Product Managers, and Senior Software Engineers.

At this Ship It Day we changed the focus to emphasize the importance on creating something that was actually shippable at the end of the exercise. Small prizes have been established around the demo but the biggest prizes are around who can ship in the week immediately after Ship It Day and then the prize size decreases for week two and week three. Given that we are in the middle of week one nothing has been given out yet but there are indications that at least one of the teams will ship in the two weeks immediately following Ship It Day. I'll update this blog post with more details in around three weeks.

In total we had fourteen self-organized teams at this event which is up six (+75%) from the eight we had at the last event. As usual we provided lunch for the team and snacks and drinks throughout the day and at the end of the day we had adult beverages for everyone.

ShipIt Day at Go Daddy

I manage a team of developers in the Presence and Commerce division of Go Daddy. We're responsible for providing small businesses with their Digital Identity. Our two main products are Website Builder and Quick Shopping Cart. Last Friday, May 10, 2013 we had our first ShipIt Day.
ShipIt Days were created by Atlassian. In their words:

Every quarter, we give employees the chance to work on anything that relates to our products, and deliver it during ShipIt Day, our 24-hour hackathon. Been wanting to build that plugin, redesign that interface, or completely rethink that feature that’s been bugging you? You’ve got 24 hours...go!

Our first ShipIt Day pretty much followed the format described by that statement.

We picked a Friday to do it because Fridays are usually less busy than other days because we try and avoid them for deployments. The teams self-organized which included the skill sets of Quality Assurance, Developers, DBAs and Business Analysts. We found a room where everyone could come together and work. Tip: When you do this bring power strips, duct tape and extension cords to make sure everyone has power throughout the day. Also make sure that the wireless is working well and has the bandwidth needed.

During the day we made sure that the team was sugared up on sodas and snacks and brought in lunch and towards the end of the day we brought in some adult beverages when the brain numbing complexity of the coding was done. This allowed everyone to relax and celebrate their work and cruise into the weekend.

We delayed the judging and demos until first thing on Monday morning to allow weekend travelers to get home.

Each team had six minutes on Monday morning to show what they'd created and the Product Managers and other non-participating managers joined in the judging. As score collector I asked the judges to use whatever criteria they through appropriate but to roll it all up into a single score from zero to ten and give that to me. Decimal places were allowed and one of the judges used five criteria and sent me scores to two decimal places.

Based on the generally high scores given out by all the judges, the quality and usefulness of the work done was exceptionally high. I cannot list the features because we will probably ship all of them in one of the two products already mentioned. However, once they're shipped I'll come back here and talk about them some more.

All of the members of the top three teams each received a prize to recognize their innovation. It was a tough call because the scores were close together as all the features were awesome.
Tips for a successful ShipIt Day (not saying we did all of these but in future ShipIt Days we hope to cover them all):

• Make sure you have power, wireless and a comfortable place for everyone to work.
• Make sure the team understands what they're doing. (i.e. a feature to enhance the product that they're working on but it doesn't have to come from the backlog.)
• Let the team know the ShipIt Day date well ahead of time so they can start forming their teams and ideas around what they'll work on and cancel that vacation to Hawaii so they can be there.
• Make sure they have drinks, snacks, and lunch during the day and of course an adult beverage at the end of the day.
• Get a budget for prizes, who doesn't like to be recognized?

I'm very lucky to be working with such an awesome team of developers. They are constantly going above-and-beyond in their efforts to make world class products and it's a privilege to be part of this team.

Codility Equi Task

Codility is a site that tests coders. Their demo problem task is called Equi. You can read the full description of the problem and solutions in a number of different languages on their blog post about it: Solutions for task Equi

I was somewhat surprise at the complexity of the code required in many languages. Using C#'s LINQ syntax makes it obvious and simple to write:

static int equi(int[] A)
{
    for (int i = 0; i < A.Length; i++)
    {
        if (A.Take(i).Sum() == A.Skip(i + 1).Sum())
        {
            return i;
        }
    }
    return -1;
}

It's also not difficult to modify that function to return all the equilibrium indexes: 

static IEnumerable<int> equi2(int[] A)
{
    for (int i = 0; i < A.Length; i++)
    {
        if (A.Take(i).Sum() == A.Skip(i+1).Sum())
        {
            yield return i;
        }
    }
}

Note that this solution although simple and elegant to write is not performant because the full array get summed in each loop. i.e. the time complexity of this solution is O(n^2). For a time linear solution you would increase and decrease a left and right sum variable as you moved through the index and compare those values.

Overloading methods in TypeScript

At yesterday's presentation at Desert Code Camp I felt that I gave an inadequate explanation on how you would implement an overloaded method in TypeScript. Hopefully this code snippet will address that:

interface Thing {
    a: number;
    b: string;
    foo(x: string): string;
    foo(n: number): number;
}

function myfunction(myParam: Thing) {
    var stringResult: string = myParam.foo(myParam.b);
    var numberResult: number = myParam.foo(myParam.a);

    console.log(stringResult);
    console.log(numberResult);
}


var myObj = {
    a: 16,
    b: "My String",
    foo: (x: any) => {
        if (x && typeof x === 'string') {
            return x.length.toString();
        } else if (x && typeof x === 'number') {
            return x * x;
        } else {
            if (x) {
                throw { message: "null parameter is unsupported" };
            } else {
                throw { message: "unsupported type: " + typeof x };
            }
        }
    }
}

myfunction(myObj);

Note that in myfunction() you would not be able to call myParam.foo() with any other type of parameter than string or number because the compiler will complain about that. So even though the implementation checks for another type in the final else you cannot pass in another type (e.g. object) because TypeScript won't let you.

Toastmasters Timer in TypeScript

I recently wrote a Toastmasters Timer in TypeScript for two reasons:

1. The other timers that I found out on the web didn't quite do what I wanted them to do and
2. I needed a small project to learn TypeScript

All of the timers I found followed the same approximate format as this one created by Stan Birdwell (Central Toastmasters 2277-31) and Michael K. Heney (Goddard Toastmasters 3496-36):

I was looking for something that would should the time in a large font on the page and change the color of a large portion of the page at each stage. I needed this to be able to constantly monitor the time and color from a distance while I am pacing around the room practicing a speech. I didn't want to have to lean forward to see the time. This is the final result: Toastmasters Timer

Here is the TypeScript source code for that timer (the compiled JS, HTML and CSS can be pulled from the page):
/// <reference path="jquery.d.ts" />

class SpeechType {
    constructor (public name: string, public greenTime: string, public yellowTime: string, public redTime: string, public id: string) {
    }
}

class TSTimer {
    timerToken: number;
    speeches: SpeechType[];
    started: bool;
    startTime: Date;
    stopTime: Date;
    green: number;
    yellow: number;
    red: number;

    constructor (speeches: SpeechType[]) {
        this.started = false;
        this.speeches = speeches;

        $.each(this.speeches, (indexInArray: number, valueOfElement: SpeechType) => {
            var newButton = $('<span>')
                .attr('id', valueOfElement.id)
                .addClass('speech-type')
                .html(valueOfElement.name);
            newButton.click( (event) => {
                this.activateSpeech($(event.target).attr('id'));
            });
            newButton.appendTo('#buttons');
        });
       
        $(window).resize( () => {
            this.resizeTime();
        });
        // call on initialization in case the browser starts off narrow
        this.resizeTime();

        $('#btnReset').click( () => {
            this.resetButton();
        });

        $('#btnStart').click(() => {
            this.startButton();
        });

    }

    resetButton() {
        this.stop();
        $('#trafficlight').text('0:00');
        $('#body').css('background-color', '#EFEEEF');
        this.startTime = null;
    }

    startButton() {
        if (this.started) {
            this.stop();
        } else {
            this.start();
        }
    };

    resizeTime() {
        var width = $(window).width();
        var x: number = Math.floor((width < 900) ? (width / 900) * 28 : 28);
        $('#trafficlight').css('font-size', x + 'em');
    }

    setElementText(elapsedSeconds: number) {
        $('#trafficlight').text(this.formatTime(elapsedSeconds));
        if (elapsedSeconds >= this.red) {
            $('#body').css('background-color', '#FF4040');
        } else if (elapsedSeconds >= this.yellow) {
            $('#body').css('background-color', '#FCDC3B');
        } else if (elapsedSeconds >= this.green) {
            $('#body').css('background-color', '#A7DA7E');
        }
    }

    timerEvent() {
        if (!this.startTime) {
            this.startTime = new Date();
        }
        var timeNow = new Date();
        var elapsedSeconds: number = this.timeDiffInSeconds(this.startTime, timeNow);
        this.setElementText(elapsedSeconds);
    }

    // Returns the difference in seconds between
    timeDiffInSeconds(earlyTime: Date, lateTime: Date): number {
        var diff: number = lateTime.getTime() - earlyTime.getTime();
        return Math.floor(diff / 1000);
    }

    formatTime(elapsedSeconds: number) {
        var minutes: number = Math.floor(elapsedSeconds / 60);
        var seconds: number = elapsedSeconds % 60;
        return minutes + ":" + ((seconds < 10) ? "0" + seconds.toString() : seconds.toString());
    }

    start() {
        $('#btnStart').val('Stop');
        this.started = true;
        if (this.startTime) {
            // i.e. no reset since the last time it was started so adjust the start time
            // to reflect the time that's elapsed since it was stopped.
            var newStartTime = new Date().getTime() - (this.stopTime.getTime() - this.startTime.getTime());
            this.startTime.setTime(newStartTime);
        }
        this.green = this.getSecondsFromTextBox('#green-light');
        this.yellow = this.getSecondsFromTextBox('#yellow-light');
        this.red = this.getSecondsFromTextBox('#red-light');
        this.timerToken = setInterval(() => this.timerEvent(), 1000);
    }

    stop() {
        $('#btnStart').val('Start');
        this.started = false;
        this.stopTime = new Date();
        clearTimeout(this.timerToken);
    }
   

    getSecondsFromTextBox(id: string): number {
        var greenLight: string = $(id).val();
        return parseInt(greenLight.split(':')[0]) * 60 + parseInt(greenLight.split(':')[1]);
    }

    setDefault() {
        this.activateSpeech('st-standard');
    }

    activateSpeech(speechId: string) {
        $.each(this.speeches, function (indexInArray: number, valueOfElement: SpeechType) {
            if (valueOfElement.id === speechId) {
                $('#green-light').val(valueOfElement.greenTime);
                $('#yellow-light').val(valueOfElement.yellowTime);
                $('#red-light').val(valueOfElement.redTime);
            }
        });
        $('.active-speech').removeClass('active-speech');
        $('#' + speechId).addClass('active-speech');
    }
}


$(document).ready(function () {
    var speeches = [];
    speeches.push(new SpeechType("Table&nbsp;Topics", "1:00", "1:30", "2:00", "st-table-topics"));
    speeches.push(new SpeechType("Evaluation", "2:00", "2:30", "3:00", "st-evaluation"));
    speeches.push(new SpeechType("Icebreaker", "4:00", "5:00", "6:00", "st-icebreaker"));
    speeches.push(new SpeechType("Standard", "5:00", "6:00", "7:00", "st-standard"));
    speeches.push(new SpeechType("Advanced", "8:00", "9:00", "10:00", "st-advanced"));
    speeches.push(new SpeechType("Test", "0:02", "0:04", "0:06", "st-test"));
    var timer = new TSTimer(speeches);

    timer.setDefault();
});

TypeScript template doesn't work in FireFox

If you're using Visual Studio 2012 and you've added the TypeScript for Microsoft Visual Studio 2012 extension then you will be able to jump start a TypeScript project using the template. If you click File > New > Project and look under the Visual C# templates you'll see a template called HTML Application with TypeScript. Selecting this template will generate a new project with a TypeScript file with the following contents:

class Greeter {
    element: HTMLElement;
    span: HTMLElement;
    timerToken: number;
   
    constructor (element: HTMLElement) {
        this.element = element;
        this.element.innerText += "The time is: ";
        this.span = document.createElement('span');
        this.element.appendChild(this.span);
        this.span.innerText = new Date().toUTCString();
    }

    start() {
        this.timerToken = setInterval(() =>
          this.span.innerText = new Date().toUTCString(), 500);
    }

    stop() {
        clearTimeout(this.timerToken);
    }
}

window.onload = () => {
    var el = document.getElementById('content');
    var greeter = new Greeter(el);
    greeter.start();
};

The problem with this code is the use of the .innerText property which is called .textContent in FireFox.

Here is a replacement to the above template that will work in FireFox and other browsers to get you moving forward again.

Tested on:
FireFox 16.0.2
Safari for Windows 5.1.7
IE9
Chrome 23.0.1271.64 m
Opera 12.10

class Greeter {
    element: HTMLElement;
    span: HTMLElement;
    timerToken: number;
    hasInnerText: bool;
   
    constructor (element: HTMLElement) {
        this.element = element;
        // Firefox uses textContent
        this.hasInnerText = !this.element.textContent;
        this.appendElementText(this.element, "The time is: ");
        this.span = document.createElement('span');
        this.element.appendChild(this.span);
        this.setElementText(this.span, new Date().toUTCString());
    }

    setElementText(elem: HTMLElement, value: string) {
        if (this.hasInnerText) {
            elem.innerText = value;
        } else {
            elem.textContent = value;
        }
    }

    appendElementText(elem: HTMLElement, value: string) {
        if (this.hasInnerText) {
            elem.innerText += value;
        } else {
            elem.textContent += value;
        }
    }

    start() {
        this.timerToken = setInterval(() =>
          this.setElementText(this.span, new Date().toUTCString()), 500);
    }

    stop() {
        clearTimeout(this.timerToken);
    }   
}

window.onload = () => {
    var el = document.getElementById('content');
    var greeter = new Greeter(el);
    greeter.start();
};

 

 

TypeScript

I'm going to be giving a presentation called TypeScript, what's all the fuss? at the Fall 2012 Desert Code Camp on November 17, 2012. I'll be adding a list of resources and my slides to this blog post.

TypeScript Resources:

• Official TypeScript web site
  
  • Presentation by TypeScript creator Anders Hejlsberg, downloads and tools.
• Brian Harry's real world story of moving to TypeScript
  
  • How a team at Microsoft migrated 80,000 lines of JavaScript to TypeScript and found 13 unknown bugs.
• Nicholas C. Zakas' thoughts on TypeScript
  
  • Good blog post covering the basic syntax and pros and cons by a well respected JavaScript guru and author of several JavaScript books.
• Other blog posts about TypeScript you may find of use:
  
  • Is TypeScript gradually typed? by Jeremy Siek
  • Microsoft TypeScript: the JavaScript we need, or a solution looking for a problem? by Peter Bright
  • Why does TypeScript have to be the answer to anything? by Scott Hanselman

Here's an example of method overloading in TypeScript which I didn't cover adequately in this presentation.

Amazon SES and Easy DKIM

I've been using Amazon's Simple Email Service (SES) for at about a year now and it's been great. Failures are very rare and the API and libraries provided are excellent. With SES you build up trust which increases your volume quota (per 24 hour period) and your send rate (per second). My only complaint is that my send rate is not as high as I want it to be.

Today I received an email saying the I can easily DKIM-Sign my emails with Easy DKIM in the SES control panel (aka AWS Management Console). This was exciting because I'd never heard of DKIM before and it sounded so easy to do.

It turned out to be pathetically easy. In the AWS Management Console you simply select your email address and under the DKIM tab you click the "Generate DKIM Settings" button.

This generates 3 CNAME records that you have to update your DNS records with. I use Go Daddy so I launched the DNS Manager for that domain and under the CNAME section I clicked the Quick Add button and copy/pasted the 3 keys and values from Amazon. Both Go Daddy and Amazon warned that it could take up to 48 to 72 hours to complete, however, less than 2 hours later I received an email from Amazon stating that it was complete and I tested it and it worked.

The emails sent through SES on behalf of that domain (to my gmail account) now have 2 new headers:

DomainKey-Status: good

I'm guessing that GMail added this header to signal that it validated the DomainKey that Amazon had created.

DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; t=12345;
    s=fa; d=fake.com;
    h=From:Reply-To:To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Date:Message-ID;
    bh=fake
    b=fake

This is what the actual signature looks like. I've replaced the values to protect the innocent. The one and two letter keys stand for:

v = version
a = algorithm (signing algorithm)
q = query (default query method)
c = canonicalization algorithm for header and body
t = timestamp
s = selector
d = domain (signing domain)
h = header (a list of the signed header fields)
bh = body hash
b = body (the actual digital signature of the contents, both header and body)

If you're already using SES then this is a no-brainer. Total time is around 15 minutes at the most. The first 10 minutes is initial setup in Amazon and then with your registrar. Once you get the confirmation back from Amazon (about 2 hours later) that it's been setup then take the other 5 minutes to go back into your Amazon account and enable DKIM by clicking a link. The first 2 times I clicked the link it didn't work so keep clicking it until it does.

If you want to test it then send yourself an email through SES before you enable DKIM and then another afterwards and compare the headers.

 

Collection was modified; enumeration operation may not execute.

This .Net error typically occurs when the underlying collection is modified during enumeration. For example if you remove items from the collection while enumerating the collection.

It can also happen when you're accessing a collection while adding to the collection. See if you can see the bug in the following code which is in the Menus accessor in a singleton object.

if (_Menus == null)
{
    lock (lockObj)
    {
        if (_Menus == null)
        {
            _Menus = new List<MenuItem>();
            _Menus.Add(new MenuItem
            {
                Link = "/mylink",
                AnchorText = "MyText",
                TitleText = "My Title"
            });
            // ... add more items
        }
    }
}
return _Menus;

Note above that we are correctly doing the double null check. Also, not visible in the code we have marked the _Menus item as volatile to prevent the compiler from removing the seemingly redundant double null check.

Here is the fixed code:

if (_Menus == null)
{
    lock (lockObj)
    {
        if (_Menus == null)
        {
            List<MenuItem> tempMenus = new List<MenuItem>();
            tempMenus.Add(new MenuItem
            {
                Link = "/mylink",
                AnchorText = "MyText",
                TitleText = "My Title"
            });
            // ... add more items
           _Menus = tempMenus;
        }
    }
}
return _Menus;

The solution, as demonstrated, is to create a locally scoped temporary collector and build that before assigning it to the _Menus variable. That way all other requests for this object will hold on the lock until this has completed building and then won't attempt to build it because it already exists as determined by the second null check.
 

 

Creating a composite key in your container

If you need to create a composite key (for a dictionary for example) where the natural key could come from multiple sources a standard technique is to prepend the source to the natural key to ensure that it is unique. However, this can still cause duplicate keys so to solve this problem you should always add a delimiter between any concatenations that you do.

The problem is easily illustrated by looking at how files are managed in folders. Typically the composite key is the fully directory path and file name which identifies the file you are looking for. The delimiter is the backslash.

Say you had 2 files: bat.txt and mybat.txt

Let's say that they were in 2 directories:
c:\temp\somy and c:\temp\so
i.e.
c:\temp\somy\bat.txt
and
c:\temp\so\mybat.txt

Without those backslashes we would have:
c:tempsomybat.txt
and
c:tempsomybat.txt
giving us the same key.

So in code you may have:

public void AddItem(string source, string itemName,
  object data, Dictionary<string, object> myDictionary)
{
     string compositeKey = source + "+" + itemName;
     myDictionary.Add(compositeKey, data);
}
 

Optimizing CSS in ASP.NET MVC

 The CSS files in a web application don't change very often. For this reason they are usually cached by the browser to improve performance. These are the steps that I take to optimize CSS files from my web apps. Although I'm using ASP.NET MVC this can be applied to any web app written in any language on any platform.

1. Combine all CSS files into a single file.
2. Replace tokens.
3. Minify the combined CSS file.
4. Cache the CSS file on the server.
5. Name the CSS file with a timestamp.
6. Compress the output to the browser.
7. Set the browser cache date to a far future date.

1. Combine all CSS files

To make a project more manageable you might want to maintain multiple CSS files. I setup a pattern so that I can easily combine the files without changing the code. For example, precede each CSS file with a 2 digit number that dictates the order in which they should be combined:

01_main.css
02_forums.css
03_footers.css

The first time your CSS file is requested your code will read all the CSS files from the CSS folder and combine them in alphabetical order.

Complications: Sometimes you might want another css file added in there based on browser. For example, you might have another CSS file you only want combined with the "numbered" css files if the browser is IE6 or IE7. In cases such as that you would create several different combined files on the server and serve up the appropriate one at runtime.

At the end of this step you have an in-memory string representing the combined CSS files.

2. Replace tokens

This is an optional step but you might have image URLs in your CSS that reside on different subdomains depending on your environment. For example, in my dev and test environments the images are in a path off the dev and test domains. However, in production, the images are on one or more sub-domains in order to parallelize the image downloads in the browser.

If your configuration follows this pattern then you might want to put tokens in the CSS files which are replaced at runtime depending on the environment.

3. Minify the combined CSS file

Run the combined CSS file through a minifier to make it as small as possible. This file is only going to be read and used by a browser so pretty formatting and extra whitespace is no longer needed.

If you're developing on the .NET platform then you can use the YUI Compressor for .NET to minify your CSS. Once you've included the library in your project it will require a single line of code:

string minifiedCSS = CssCompressor.Compress(combinedCSS.ToString());

4. Cache the CSS file on the server

Once you've built and minified the CSS file (steps 1 and 2) you will probably want to cache this on the server so you don't have to do this again. How you cache it is up to you. ASP.NET has a cache object you can put it in or you can use a static variable, or singleton etc.

5. Name the CSS file with a timestamp

In the <head> section of your HTML you need to reference your CSS file. I do this by referencing a controller and action with a timestamped filename as the single parameter. For example, the served up HTML will look something like this:

<head>
...
<link href="/site/css/20110418095157.css" type="text/css" rel="stylesheet" />
...
</head>

In my .master file for the site or _Layout file for my razor views I have the following code:

<link href="<%:MySingleton.Instance.GetCssFileName() %>" type="text/css"
rel="stylesheet" />

The first time that the static MySingleton.Instance.GetCssFileName() function is called it pulls all of the relevant CSS files and finds the one with the most recent date. Using that date it constructs the filename using the following .NET format pattern: yyyyMMddHHmmss. This computed filename is also cached so that it is only ever created once.

 

6. Compress the output to the browser

 

I do this by telling IIS to compress all output to the browser. How to enable compression on IIS6 and IIS7. If you don't want to enable compression on IIS then you can add a filter (attribute) to your action.

 

7. Set the browser cache date to a far future date

 

In the MVC Action that returns the CSS file set the HTTP headers to be far in the future. You never have to expire this CSS file because the name will change which will force a fresh request of the CSS files after one of them has been modified. Here is the type of code you would add to the start of your Action method to set the HTTP headers appropriately.

 

TimeSpan duration = TimeSpan.FromSeconds(15552000);
Response.Cache.SetCacheability(HttpCacheability.Public);
Response.Cache.SetExpires(DateTime.Now.Add(duration));
Response.Cache.SetMaxAge(duration);

 

 

 

ASP.NET MVC3 Razor Notes - Partial Views

What's the difference between Html.RenderPartial() and Html.Partial()?

Html.RenderPartial() is an HTML helper method that was introduced in MVC1 and writes directly into the response object.

Html.Partial() is an HTML helper method that was introduced in MVC2 and like the other HTML helper methods returns a string and does not write into the response object.

Both helpers are used with partial views (sometimes still called controls) to implement the DRY principal of shared View code across multiple pages.

In the MVC WebForms View Engine views have the extension .aspx and partial views .ascx. This makes it easy to distinguish between views and partial views. In the MVC Razor View Engine both views and partial views have the extension of .cshtml. In order to easily distinguish between a view and a partial view in your project I suggest that you precede the partial views name with an underscore.

ASP.NET MVC3 Razor Notes - view data

The WebViewPage in an abstract class that inherits from the WebViewPage<TModel> generic abstract class.

Two properties of the WebViewPage<TModel> generic abstract class are:

1. Model (dynamic)
2. ViewData (key/value collection)

Among other properties, the WebViewPage adds:

1. ViewBag (dynamic)

The ViewBag property is another way to get to the ViewData data.

For example:

If in the controller you did the following:

ViewData["website"] = "guyellisrocks.com";
ViewData["somenumbers"] = Enumerable.Range(0, 10);

Then in the view you could access that data through the ViewBag:

<div>
    @ViewBag.website
</div>
@foreach (int i in ViewBag.somenumbers)
{
<div>
    @i
</div>
}

and this would emit to the web page:

guyellisrocks.com

0

1

2

3

4

5

6

7

8

9

 

ASP.NET MVC3 Razor Notes - switching between text and code

Switching between text and code in Razor:

If you're in a code block and want to emit text you can use the <text> tag:

@{
   if(condition == true) {
      <text>this is true text</text>
   } else {
      <text>this is false text</text>
   }
}

The <text> tag will not be included in the generated HTML.

If you're in a text section and want to explicitly invoke code that could be confused as text you can wrap the code in parenthesis:

<img src="@(filename).jpg" />

If you had not put the () parens in above it would have tried to evaluate the .jpg as a property on the filename object.

 

Blue moons this century

I've been doing some work on calendars and I needed to generate a list of full moons in a given year. This got me wondering when the next blue moon will be and how many blue moons there will be this century. The next blue moon will be on 31 August 2012 at 1:57:45pm UTC and there will be 41 blue moons this century. Here is the code that I used to generate the list of dates:  

DateGenerator dateGenerator = new DateGenerator();
int[] years = Enumerable.Range(2000, 100).ToArray();
HashSet<int> hashSet = new HashSet<int>();
foreach (int year in years)
{
    List<DateTime> fullMoons = dateGenerator.FullMoons(year, 0);
    foreach (DateTime date in fullMoons)
    {
        int yearMonth = date.Year * 100 + date.Month;
        if (hashSet.Contains(yearMonth))
        {
            Console.WriteLine(date.ToString("dd-MMM-yyyy HH:mm:ss"));
        }
        hashSet.Add(yearMonth);
    }
}
Console.ReadLine();

Here is a full list of blue moons for this century. Times are UTC

30-Nov-2001 20:51:10
31-Jul-2004 18:06:05
30-Jun-2007 13:50:25
31-Dec-2009 19:14:20
31-Aug-2012 13:57:45
31-Jul-2015 10:45:46
31-Jan-2018 13:27:46
31-Mar-2018 12:37:46
31-Oct-2020 14:51:29
31-Aug-2023 01:37:05
31-May-2026 08:46:46
31-Dec-2028 16:49:45
30-Sep-2031 18:58:02
31-Jul-2034 05:56:35
31-Jan-2037 14:05:42
31-Mar-2037 09:54:43
31-Oct-2039 22:36:54
31-Aug-2042 02:05:14
30-May-2045 17:53:13
31-Jan-2048 00:16:32
30-Sep-2050 17:32:47
30-Jul-2053 17:08:32
31-Mar-2056 10:26:26
31-Oct-2058 12:54:58
30-Aug-2061 22:20:53
30-May-2064 10:37:03
31-Dec-2066 14:42:26
30-Mar-2067 20:11:05
30-Sep-2069 18:12:11
31-May-2072 22:19:43
30-Apr-2075 18:40:43
31-Oct-2077 10:39:00
31-Jul-2080 19:16:42
31-May-2083 09:43:39
31-Dec-2085 00:00:58
30-Sep-2088 15:28:47
30-Jul-2091 12:03:20
31-Jan-2094 12:40:11
30-Apr-2094 13:58:47
31-Oct-2096 11:20:22
30-Aug-2099 17:59:15

 

Speed of Contains() on string list and hash set in C#

Although I was 99.9% certain that a Contains() on a HashSet would outperform the same on a List I needed to see it happen before I could sleep again.

Here is the code that I used to test it.

int iterateNum = 100000000;
string longString = "this is a very very very long string that is in the list";
string searchString = "this is the search string";

List<string> stringList = new List<string>();
for (int i = 0; i < iterateNum; i++)
{
    stringList.Add(longString);
}
stringList.Add(searchString);

Stopwatch stopwatch = Stopwatch.StartNew();
bool contains = stringList.Contains(searchString);
Console.WriteLine("contains on list took: " + stopwatch.Elapsed.ToString());

HashSet<string> hashSet = new HashSet<string>();
for (int i = 0; i < iterateNum; i++)
{
    hashSet.Add(longString);
}
hashSet.Add(searchString);
stopwatch.Reset(); // should be stopwatch.Restart();
contains = hashSet.Contains(searchString);
Console.WriteLine("contains on hashset took: " + stopwatch.Elapsed.ToString());


And here are the results:

contains on list took: 00:00:01.2830928
contains on hashset took: 00:00:00

i.e. Was not possible to measure how long the hash set took.

In the code I'm adding the string to find as the last string in the list to make sure that the entire list is searched before it finds the string.

Edit: As per Bill's suggestion in the first comment I changed the code to populate a smaller and more realistic list and then exercised the Contains() method multiple times to get an "average" of its performance. I also replaced the Reset() method on the Stopwatch object with Restart(). Here is the new code:

int iterateNum = 4000;
int testNum = 100000;
string longString = "this is a very very very long string that is in the list";
string searchString = "this is the search string";

List<string> stringList = new List<string>();
for (int i = 0; i < iterateNum; i++)
{
    stringList.Add(longString);
}
stringList.Add(searchString);

Stopwatch stopwatch = Stopwatch.StartNew();
for (int i = 0; i < testNum; i++)
{
    bool contains = stringList.Contains(searchString);
}
Console.WriteLine("contains on list took: " + stopwatch.Elapsed.ToString());

HashSet<string> hashSet = new HashSet<string>();
for (int i = 0; i < iterateNum; i++)
{
    hashSet.Add(longString);
}
hashSet.Add(searchString);

stopwatch.Restart(); // .NET 4 method on Stopwatch
for (int i = 0; i < testNum; i++)
{
    bool contains = hashSet.Contains(searchString);
}
Console.WriteLine("contains on hashset took: " + stopwatch.Elapsed.ToString());

And the results are:

contains on list took: 00:00:04.6986760
contains on hashset took: 00:00:00.0056837
 

 

Nested Parallel.ForEach()

I recently wrote some code that was using Parallel.ForEach() and in the function called during the Parallel.ForEach() I nested another call to Parallel.ForEach() to process a second array. This then got me thinking. If the first array that you are processing is larger than the number of cores on the machine that you're running it on then there's no advantage to nesting a second Parallel.ForEach(). You might as well just loop in that function because we'll already have all the processors saturated. So I put together a test to see the difference between Parallel.ForEach() with a regular loop and Parallel.ForEach() with a nested Parallel.ForEach()

static void ParallelMainNestedTest()
{
    int[] lowerRange = Enumerable.Range(0, 10).ToArray();
    int[] upperRange = Enumerable.Range(1000, 1010).ToArray();

    Stopwatch timer = Stopwatch.StartNew();
    Parallel.ForEach(lowerRange, lowerValue =>
        DoItemNotNested(lowerValue, upperRange));
    Console.WriteLine("Not Nested: {0}", timer.Elapsed);

    timer.Restart();
    Parallel.ForEach(lowerRange, lowerValue =>
        DoItemNested1(lowerValue, upperRange));
    Console.WriteLine("Nested: {0}", timer.Elapsed);

    Console.ReadKey();
}

static void DoItemNotNested(int lowerValue, int[] upperRange)
{
    foreach (int upperValue in upperRange)
    {
        Thread.Sleep(10);
    }
}

static void DoItemNested1(int lowerValue, int[] upperRange)
{
    Parallel.ForEach(upperRange, upperValue =>
        DoItemNested2(lowerValue, upperValue));
}
static void DoItemNested2(int lowerValue, int upperValue)
{
    Thread.Sleep(10);
} 

The results of this test showed:

Not Nested: 00:00:14.6155628
Nested: 00:00:09.0907248

Now the reason that I put a sleep in the inner loop of the two tests is because this test code is designed to simulate network access (a REST call to a site) and I wanted to create a test where there would be idle wait time to determine which strategy is better. I appears, as you can see, that a pair of nested Parallel.ForEach() calls is most efficient in this scenario.

What if there was no latency in the inner loop and it was all down to the speed of the processor?

My suspicion is that the nested Parallel.ForEach() would be slower because of the extra overhead in managing the extra threads. To test this I replaced the Thread.Sleep(10) call with the following:

static Random r = new Random(Guid.NewGuid().GetHashCode());
private static void DoSomething(int lowerValue, int upperValue)
{
    for (int i = 0; i < 5000; i++)
    {
        int rand = r.Next(lowerValue, upperValue);
        rand = rand * rand;
    }
}

The result:

Not Nested: 00:00:05.6169188
Nested: 00:00:06.1500767

Confirmed: The nested Parallel.ForEach() shows a degradation in performance because of the extra overhead in managing all those extra threads.

Conclusion: Used nested Parallel.ForEach()'s when you know that the inner loop will be waiting on external resources (database, disk, network etc.). Use regular looping in the inner loop if you are doing a math or algorithmic intensive calculation that relies on the CPU alone.