Node Foundation Membership Election

Mikeal Rogers recently posted Nominations for the 2016 Election for individual representation on the Node Foundation board.

I'm putting myself forward and answering the question "Why would you like to represent the individual membership of the foundation?"

Why?

I think that Node.js and the ecosystem around it plays a critical role in our current technology stack. This is going to become more important in the future.

I would like to be able to help shape the success of this platform. One way to do that is to understand what the membership wants and needs to get out of what we have today. More importantly what do they expect from the future. As part of the bridge between membership and the board I will be working to ensure that members' opinions are represented at the leadership level.

Qualifications

Leadership: In the past I've served as a director of engineering for a large IT company managing a team of 35+ managers and developers.

Technical: I have a few open source projects that I manage. I also contribute to open source projects I can improve through pull-requests. My day-to-day work is mostly writing JavaScript against the Node.js platform and system design and architecture.

Questions

I've intentionally kept this brief. I'll answer any questions. If you ask them as comments to this post then we can keep them in one place. Anyone else reading this will be informed.

Docker and GLIBC_2.14 not found

Hit a problem today when I was using Docker and Docker-compose to run a Node.js app in a CentOS container. I had done an "npm install" locally and then the docker-compose command had setup the container and copied everything over from my Ubuntu workstation.

Problem was that I was using a couple of Node.js modules that used native code. So the Ubuntu compile of those modules was getting copied to the CentOS container and when CentOS was trying to run them it was giving me an error about "GLIBC_2.14 not found" because it hadn't compiled those modules.

I feel that a good practise for almost any Docker setup is to have a .dockerignore file which excludes the "node_modules" folder.

This is what my .dockerignore file looks like:

node_modules/

Consuming a JSON REST result in C# .NET

Yesterday I wrote about an Simple API server for IP Address Information Lookup that I had created in Node.js. This server produces a JSON result that looks like this for IP address 8.8.8.8:

{
   "city":{
      "geoname_id":5375480,
      "names":{
         "en":"Mountain View",
         ...
      }
   },
   "continent":{
      "code":"NA",
      "geoname_id":6255149,
      "names":{
         "de":"Nordamerika",
         "en":"North America",
         ...
      }
   },
   "country":{
      "geoname_id":6252001,
      "iso_code":"US",
      "names":{
         "en":"United States",
         ...
      }
   },
   "location":{
      "latitude":37.386,
      "longitude":-122.0838,
      "metro_code":807,
      "time_zone":"America/Los_Angeles"
   },
   "postal":{
      "code":"94040"
   },
   "registered_country":{
      "geoname_id":6252001,
      "iso_code":"US",
      "names":{
         "en":"United States",
         ...
      }
   },
   "subdivisions":[
      {
         "geoname_id":5332921,
         "iso_code":"CA",
         "names":{
            "en":"California",
            ...
         }
      }
   ]
}

I wanted to consume this end-point in a C# application. A quick search for solutions on how to consume a JSON result from a REST service showed some very complicated ways of doing this. This is a simple solution that I came up with:

// A simple C# class to get the details of an IP address
public class IpApi
{
    public IPInfo GetIPInfo(string ip)
    {
        // Create the URL for the REST endpoint
        string url = "http://localhost:3000/ip/" + ip;
        // Create a .Net WebClient object
        WebClient wc = new WebClient();
        // Get the string result from the REST endpoint
        string json = wc.DownloadString(url);
        // Use Newtonsoft JsonConvert class and its static
        // DeserializeObject method to parse the string into
        // a .Net object
        return JsonConvert.DeserializeObject<IPInfo>(json);
    }
}
// The following classes create the JSON object graph hierarchy
// in static classes. It's a bit tedious looking at the JSON
// object and typing these out. I'd imagine that if you're 
// doing this a lot you could write a tool to generate these
// classes using the JSON as input. Or perhaps there's already
// a tool that does this?
public class Names
{
    // I didn't need the pt-BR and zh-CN key/values from the
    // names so I didn't bother looking up how JsonCovert maps
    // those. 
    public string de, en, fr, ja, ru;
}
public class City
{
    public Names names;
    public int geoname_id;
}
public class Continent
{
    public string code;
    public Names names;
    public int geoname_id;
}
public class Country
{
    public string iso_code;
    public Names names;
    public int geoname_id;
}
public class Location
{
    public decimal latitude;
    public decimal longitude;
    public int metro_code;
    public string time_zone;
}
public class Postal
{
    public string code;
}

public class IPInfo
{
    public City city;
    public Continent continent;
    public Country country;
    public Location location;
    public Postal postal;
    public Country registered_country;
    public Country[] subdivisions;
}

Simple API server for IP Address Information Lookup

I find that I often have the need, in a web application, to lookup information about an IP address that a request comes from.

To serve this purpose I created IPAPI (GitHub), a simple server providing a single API endpoint that returns information about an IP address.

(You should try the server just because the name is a palindrome. Imagine the awe and respect you'll get when you say "I ran a Palindrome API server today.")

The server runs on Node.js and uses the GeoLite2 database from MaxMind.

You can install and run the server with:

git clone git@github.com:guyellis/ipapi.git
npm install
node index.js

or

npm install ipapi
node node_modules/ipapi/index.js

The server will download and unzip the IP database if it's not found locally.

Using the server is as simple as having your client call http://<your-domain/ip/<x.x.x.x> and getting back JSON with information about the IP.

There are plenty of improvements that can made to this simple server. Pull Requests gladly accepted.

Node.js Workshop with HTTP Status Check

I've been using HTTP Status Check for Node.js Workshops to help coach developers wanting to learn or get better at Node.js and JavaScript development.

HTTP Status Check is a Node.js utility that takes a list of URLs and checks that their statuses and other properties are what you would expect. As a web developer we usually have a number of domains, URLs and sites that we need to "keep-an-eye" on and this utility will quickly check the health of all our sites and report it back to us.

The benefits of using this project for a workshop are:

• Most Node.js development is around web development. As a web developer you need to know and understand the HTTP protocol. This utility is all about that.
• It's small and easy to understand.
• It's something that you can (and should) use each day to keep an eye on your web properties. This means that you'll directly benefit from any changes you make to it.
• Which brings us to changes. It's easy and quick to make changes in your fork (branch) of the project if you need it to do more than what it currently does.
• In addition to learning Node.js/JavaScript you get to learn how to use Git, GitHub, Npm, how to contribute to Open Source and how to build your reputation and resume on GitHub.

To get the most out of a workshop you need your laptop to be in a state that allows you to dive in and start learning Node.js and JavaScript. This means that you need some basic applications installed and accounts setup and configured. Before attending, do the following. If any of the steps are unclear then post a comment below and I'll clarify.

("repo" is short for repository or code-repository. The code that you run is in a repo and you will pull it down to your machine. More on that to come...)

1. Setup a GitHub account and log into it: https://github.com/join
2. Fork the HTTP Status Check repo. (Forking is the process of making a copy of the code-base in your GitHub account that you have complete control over to do with as you please.)
1. You must be signed into GitHub. Go to the repo: https://github.com/guyellis/http-status-check
2. If you're signed in then you will see some buttons towards the top right of the page: Watch, Star, Fork. Click on the Fork Button.
3. The repo will now be forked (copied) to your account and you will be redirected to it, something like: https://github.com/<your-account-name>/http-status-check
4. Congratulations, you've successfully forked your first GitHub project and taken your first step to contributing to open source.
3. Install Git on your computer. You now need to get that repo (the code-base) from your fork on GitHub to a directory on your computer. This is done through a process called cloning which as the name suggests creates a clone of your fork on your local machine. Before we can do this though we need to install Git:
1. Use the official Git download page to find and install the right Git Client for your OS: http://git-scm.com/downloads
2. There is also a link on that page that will take you to popular Git GUI clients.
3. Once that's done come back here.
4. Open a command window and type git and hit enter. You should be presented with a list of git commands. This confirms that git has been successfully installed.
4. Clone the repo.
1. Create a directory where you want to keep your source code. You don't need to create a directory for the HTTP Status Check project, just one that will hold your projects. For example /source/ or /code/ or /myrepos/ or something like that.
2. Now open a command window in that directory or open a command window and change to that directory.
3. Clone the repo by executing this command after you have replaced your-github-account-name with your GitHub account name. (This is where you forked the repo to in the Fork step above.):
   git clone https://github.com/<your-github-account-name>/http-status-check.git
• Some notes about this:
• You can find the correct link to use on your GitHub page by looking at the forked repo and on the right hand side you'll see something that says HTTPS clone URL. Right below that is a text box that you can copy the link from.
• If you want to use SSH then you can click the SSH link below that box to switch the link to the correct SSH link.
4. If this is successful then you'll see something that looks like this. The numbers that you see will be different:
   
   Cloning into 'http-status-check'...
   remote: Counting objects: 395, done.
   remote: Total 395 (delta 0), reused 0 (delta 0)
   Receiving objects: 100% (395/395), 56.90 KiB | 0 bytes/s, done.
   Resolving deltas: 100% (224/224), done.
   Checking connectivity... done.
5. A directory would have also been created for you called http-status-check.
5. Code Editor. You need a code editor to edit the code that you've just cloned. If you don't already have one installed then try one of these which are available on almost all Operating Systems:
1. WebStorm: 30 day trial and then $49 to buy and $29/year renewal after that (for non-commercial use).
2. Sublime Text: Free evaluation and then a $70 single payment for continued use.
6. Install Node.js and Npm. I deliberately put this step at the end. I want you to be able to immediately run something after you've installed Node.js and see it in action which is why you got the code setup first.
1. Go to the Node.js site and install it for your Operating System. This will also install Npm which you'll need later.
2. Once installed open a command window and change directory to the HTTP Status Check directory and run node.
3. You should see a > command prompt. This confirms it was installed. Hit Ctrl+C twice to exit.
7. Run Npm. 
1. In the same command window in the http-status-check directory run npm install. (You can also run npm i which is a shorter version of this.)
2. You should now see all the dependencies being downloaded and added to the project. Here is an example of some of what you'll see in the console window. Some of the numbers might be different:
   
   debug@2.0.0 node_modules/debug
   └── ms@0.6.2
   
   chai@1.9.1 node_modules/chai
   ├── assertion-error@1.0.0
   └── deep-eql@0.1.3 (type-detect@0.1.1)
   
   lodash@2.4.1 node_modules/lodash
8. Run HTTP Status Check.
1. Now type node index.js and hit enter and the HTTP Status Check utility will run. You should see something like this:
   
   _ Google (http://google.com) testing disabled.
   _ HTTP Status Check on Guy's Blog (http://www.guyellisrocks.com/2014/06/http-status-check.html) working as expected.
   _ Missing URL example (http://www.guyellisrocks.com/2014/06/will-this-get-written.html) working as expected.
   _ LinkSilk WWW (http://www.linksilk.com) working as expected.
   _ LinkSilk (http://linksilk.com) working as expected.
   A total of 5 URIs were tested.
   Failure count:  0
   Success count:  4
   Disable count:  1
9. Done! You've successfully forked, cloned, installed and run your first Node.js application. Now you're ready to start making changes to it to customize how it works. We'll cover code changes in the workshop. In the mean time you can change the list of web sites that it checks to check your sites and immediately provide you with some value:
1. Copy the samplesites.js file (in the root of the project) to a file called checksites.js. (checksites.js will take precedence over samplesites.js if it is present.)
2. Edit the checksites.js file and replace the sample web sites with your own websites and rules. The file is heavily commented (lines starting with // are comments) to guide you to what changes you can and should make.
3. Now run node index.js again and you'll see your sites being checked.
10. Workshop and Questions. If you weren't able to get to this point then post comments to this blog post. You should also bring your questions to the workshop. If you're not able to edit your checksites.js file and run it against your sites before the workshop then you'll miss out on making changes to the code and learning about Node.js and JavaScript.

Bonus Tasks

1. Run the tests.
1. In a console window in the http-status-check directory run npm test.
2. All the tests should pass.
2. Run code coverage.
1. In a console window in the http-status-check directory run npm run istanbul.
2. The tests will run (as above) and code coverage will be calculated. The output should end with something that looks a little bit like this:
   
   Statements   : 100% ( 170/170 )
   Branches     : 88.73% ( 63/71 )
   Functions    : 100% ( 23/23 )
   Lines        : 100% ( 169/169 )
3. Now open the coverage/lcov-report/index.html file that was created as a result of this and look at how the tests cover the solution.
3. Learn more about how Git and GitHub work. Install Ungit and use its graphical interface in the browser to visual understand the repo's structure and work with the repo.
4. New features in HTTP Status Check.
1. Is there a bug that needs to be fixed or feature that you think should be added to HTTP Status Check? Then add it as an issue: HTTP Status Check Issues. (Even if you intend to work on this you should add it to the issues first and then assign it to yourself.)
2. Want to work on existing issues in HTTP Status Check. Then find them in the same place: HTTP Status Check Issues

HTTP Status Check

I recently created a Node.js utility that will cycle through a list of URLs and check their HTTP statuses:

Npm: http-status-check

GitHub: http-status-check

This utility came out of a need to keep daily tabs on a number of URLs and the statuses that they were returning.

I plan on expanding it with other input and output adapters. Obvious input adapters would be databases. What else can you think of? Obvious output adapters would be email and any other type of messaging system.

Pull requests are welcome.

Update 4/July/2014

Added an excludedHeaders option. This is a list of headers that you want the check to fail on if they are present in the response from the server.

At first blush this may seem like a strange check to make. The common use case is the X-Powered-By header. This header allows the server to advertise the technology that is powering it. As a security concern, when possible, I'll remove this from sites that I publish. I feel that telling malicious attack bots what you're running on will help them exploit any vulnerabilities your stack might have.

Update 5/July/2014

Added an expectedText option. This is text that we expect to be present in the body of the response from the server. By default it is case insensitive but you can change that by supplying an object instead of a string.

Using WebSockets when your Reverse Proxy doesn't allow it

As developers we don't always get to choose where our software runs. We often face economic or other restrictions based on infrastructure that already exists.

I recently moved a Node.js application from Linux server to a Windows Server 2008R2. Crazy right? It's working surprisingly well in the Windows environment. IIS 7.5 already owned port 80 so I had to setup a site on IIS, bind the domain to that site and use it as a reverse proxy to the Node app which was running on an arbitrary port.

In this case it happened to be IIS in any other case it might be Ngnix, Apache or any other server or reverse proxy that is between your Node.js application and the web. The problem that I faced is that this version of IIS does not support WebSockets so it looked like I couldn't use that and had to allow socket.io to fall back to long polling for this application.

There is, however, a solution, and it's rather simple.

Your site's facade, let's call it mdomain.com, is running on port 80 on IIS which is configured to run as a reverse proxy passing all traffic through to port (say) 4444 where your Node application is running.

When a client (a browser) connects to your site you provide it with the usual payload of HTML, CSS and JavaScript and in that you also provide it with the port number or sufficient information for the WebSocket part of the client to make a direct connection to your Node.js server and bypass the reverse proxy completely.

Using this little trick our site can remain on the default port going through the reverse proxy and all our WebSocket traffic can run over the application specific port.

IIS Reverse Proxy to multiple sites running under a single Node.js instance

Subtitled: How to setup Node.js to handle multiple sites and to run on Windows while IIS still occupies port 80

I did this on a Window 7 machine which was running IIS 7.5

 

Setting up your local box for testing

In your HOSTS file add some sites to test:
127.0.0.1     node1.com
127.0.0.1     node2.com
127.0.0.1     node3.com

IIS

Create a new web site in IIS:
Open IIS.
Right click on Sites.
"Add Web Site..."
Site Name: MyReverseProxy
Physical Path: C:\inetpub\wwwroot (can be anything as we're not going to use it)
Host name: node1.com
Now click on Bindings under Actions on RHS
Click Add.
Host name: node2.com
Click Add.
Host name: node3.com
Close.
You now have 3 host names running on port 80 on IIS and your HOSTS file will direct all local requests to this web site.

Click on MyReverseProxy on left panel. If the URL Rewrite widget is not under IIS then you might need to install it and might also need to add Application Request Routing (ARR). I can't remember if it came installed by default on IIS 7.5 or if I added it.

Open the URL Rewrite widget and click on Add Rule(s)...
Select a Reverse Proxy rule and click OK.
Server name: localhost:3000/ (or where the Node server is running.)
If you're using relative URLs in the Node.js code then you don't have to rewrite the domain names of the links in the HTTP responses. If you are using absolute names then you're in trouble if you're handling multiple domains as you won't know at this stage which one it's for. The only solution I have for this is to use relative URLs and DO NOT check the option to "Rewrite the domain names..."

Now start your Node.js application on port 3000 and try and access the sites node1.com, node2.com, and node3.com and they should all return the contents of your site running on port 3000.

We're not done yet. If, in Node.js/Express, you take a look at the req.headers.host value you will see that it reads "localhost:3000" and has not passed through the value you were expecting which was one of node1.com/node2.com/node3.com

To get this to work you need to go to this folder with an Administrator command prompt:
C:\Windows\System32\inetsrv
and run this command:
appcmd.exe set config -section:system.webServer/proxy /preserveHostHeader:"True" /commit:apphost

What this command will do is open this file:
C:\Windows\System32\inetsrv\config\applicationHost.config
and find the section called <system.webServer> and change this:
        <proxy enabled="true" />
 to this:
        <proxy enabled="true" preserveHostHeader="true" />
Recycle the app pool on your reverse proxy site in IIS and try and access it again. The host "head" property will be correctly set.

Now the final "thing" you want to do is to be able to handle multiple sites from your Node.js/Express application.

Here is some basic middleware in app/server.js that will switch between sites:

app.use(function(req, res, next) {
    console.log('Domain is: ' + req.headers.host);
    switch(req.headers.host) {
        case 'node1.com':
            break;
        case 'node2.com':
            break;
        case 'node3.com':
            break;
        default:
            console.log('Unknown domain: ' + req.headers.host);
            break;
    }
    next();
});

You would then setup routes that appropriately handled routes for each host that you were expecting. Where routes are ambiguous you would put a switch statement like above in there to arbitrate among domain functionality.

You could also put a node-proxy in front of these sites to switch between different node apps but then you're defeating the purpose of IIS which can already do that and more efficiently so I can't see the need for a node-proxy.

Some of the answers from this Stackoverflow question might help readers of this topic.