Ad Management services versus doing it yourself

Should you use an advertising management solution such as DFP Small Business or do it yourself?

(DFP stands for DoubleClick For Publishers. DoubleClick was bought by Google in March 2008 for $3.1 billion.)

If someone approaches me and wants to advertise on one of my sites my usual response is to tell them to use Google's AdWords program and target my site or pages on my site with their adverts. In my opinion, Google does a great job and they only take 32%. I don't have to get involved at all.

Sometimes it is worth my while to split that 32% with an advertiser and I'll add their advert to a site for a given time period.

The advantages of doing it yourself are advantages for the the advertiser, not for you the publisher:

• If you host the advert's image and text on your site then it is almost impossible for Ad Blockers to block the ads because they are integrated images.
• Your advertiser gets a one-way follow link to their site from your page.

The disadvantage is that you have to create an embedded solution to allow advertisers to upload their adverts to your site (the sophisticated solution) or you have to do it manually. Doing this yourself requires work on your part either way and you will have to measure the Return On Investment and if that extra revenue over AdSense is worth it.

SQL Injection Attack from 82.45.20.100

While perusing the application logs this morning I noticed a number of SQL Injection attempts. The site that was being attacked uses MVC with automatic Form values to object binding so the exceptions were being thrown early on in the controller when the framework was trying to bind the attack string to, for example, a boolean.

Here are some of the attack strings the system logged:

 

%20AND%201=1
%20AND%201=2
' AND '1'='1
' AND '1'='2
' AND 1=1/*
' AND 1=2/*
AND 1=1/*
AND 1=2/*

The source of the attack:

IP: 82.45.20.100
City: Bristol
State: England
Country: United Kingdom
ISP: Virgin Media Limited.

The attack bot wandered around the site and tried many different entry points as you'd expect. Some of the attack points on the site identified it as an attack through the pattern being requested and stopped processing .An initial inspection of the database shows no damage.

One of the tricky things about logging SQL Injection attacks is that if you're logging them to a database then your logging process might facilitate the attack when the initial attack failed. I have 2 suggestions for this this:

1. Ensure that any code you use to insert or update to the DB is appropriately parameterized and injection proof.

2. Become a SQL Injection expert and try and break your fortress with injection attempts.

I'm a little bit surprised that SQL Injection attempts are still going on. I would have thought that any site worth taking over would have dealt with this by now. It's been over a decade since this became a well known security problem.

 

 

Kintiskton LLC IP Ranges

There's a company called Kintiskton LLC who either own or are owned by Mark Manager and they provide a trademark protection service. They have a spider that crawls the web attempting to identify their customers' copyright material posted on sites other than their customers'. In principal I don't have a problem with this because I agree that copyright should be respected.

There's a good write-up about them here: http://endellion.me.uk/info/Kintiskton.html

The problem is that their spider aggressively spiders sites without respecting the robots.txt file. It hits the site hard and fast and ignores the crawl-delay directive and exclude directives. Ignoring the excludes directive is understandable (but not tolerable) as rouge web sites that are violating copyright could "hide" their content from respectful spiders by adding an exclude directive in the robots.txt file for that part of the site. This spider, however, also ignores the crawl-delay and is also not very well written as it generates a fair number of errors in the log files making it easy to see.

If you want to exclude this spider from your site you can exclude these IP ranges: 

65.208.151.112 - 65.208.151.119
63.110.158.48 - 63.110.158.55
65.200.47.0 - 65.200.47.7
65.208.189.24 - 65.208.189.31
65.208.185.96 - 65.208.185.103
65.211.195.16 - 65.211.195.23
5.208.151.112 - 5.208.151.119 (probably a mistake - see Zap's comment below)

If you discover another range that they are using please post as a reply to this blog post and I'll add it to the above list.

 

Google Maps slows down on Saturday mornings

I use Google Maps a lot to investigate a place I'm going to or to get direction there. I've noticed that on Saturday mornings Google Maps often fails but when it doesn't it just runs real slow and I get this message:

Still loading... Slow? Use the troubleshooting guide or basic HTML.

My theory is that everyone is using Google Maps to find directions before or while going out on a Saturday and this is overloading their maps servers. When are you most likely going to need directions? When you go somewhere you haven't been to in a while or ever. That's not going to happen during the week when you're going to and from work, that's going to happen on the weekend when you have the time to adventure off the beaten path. So my guess is that the requests to the mapping service peaks over the weekend.

AdWords AdSense Arbitrage

 I've heard about people doing AdWords/AdSense Arbitrage but I question if it's even possible.

First of all, what is arbitrage? The classic definition of arbitrage has someone, usually a trader in the stock markets, buy and sell a financial instrument at exactly the same time such that there is zero risk and an instant profit. Essentially you are the middle-man who has a buyer and seller lined up and you pass the item being sold from one to the other and pull in the difference.

With AdWords/AdSense arbitrage it's a little different because it is by no means risk free and it does not take place at the same time. The idea is that you buy traffic using AdWords and then you sell it on to another site using AdSense and the rate that you buy it at is lower than the rate that you sell it at.

Let's look at the math involved.

Google keeps 32% of the revenue earned from a click on an advert on your site (assumes AdSense-for-Content). So if you pay $1 through AdWords to bring a visitor to your site you need to earn $1.47 from AdSense from that visitor in order to break-even. So far this is not impossible but there's a lot of competition out there and you have to assume that your visitors are looking for the same type of item so advertising rates should be similar. It also assumes that 100% of visitors arriving through your AdWords campaigns click on an AdSense advert.

Now let's imagine that only 10% of your visitors that arrive from your AdWords campaigns click on one of your AdSense adverts. At our example rate of $1/visitor you have spent $10 to get someone to click on your AdSense advert and you need to make sure that you earn $14.71 from that click to break-even. That's a huge jump from a $1 Adwords campaign to an AdSense advert.

This table shows you how much you have to earn per AdSense click in order to break-even based on the click-through-rate based on $1/AdWords click:

CTR	AdSense
1	147.06
5	29.41
10	14.71
20	7.35
50	2.94
75	1.96
100	1.47

This is why I think that AdWords/AdSense Arbitrage is almost impossible. 

Time taken for Google to de-index 301 redirect pages

I changed a bunch of the URL's on a site that's in the Alexa to 100,000 - i.e. a moderately busy site that the Google bot visits every day. The old URL's now do a 301 redirected to the new structure. At the time that I did the switch over and every day for the next 60 days I took a snapshot of how many URL's where indexed by Google in each section. I used the following search command in Google:
inurl:/old/folder/pattern site:mysite.com
I entered the number of URL's into a spreadsheet for each of 3 folder patterns. Each pattern started off with 13, 18, and 87 URL's in Google's index. The objective of the exercise was to see how long it would take Google to de-index these pages. Here is a chart of the results:

The folder pattern with 87 URL's is shown against the right axis and the other two against the left.
Expectations:
My expectation was that as soon as Google found the new URL's (it found almost all of them within 5 days) that it would rapidly de-index the old URL's. Remember that I'm telling Google that this is a permanent (301) not temporary (302) redirect.
Actual results:

1. It took around 55 days to naturally de-index all the pages. Much longer than I was expecting.
2. The de-indexing for the 2 smaller collections of pages was linear.
3. The de-indexing for the larger collection of folders was sudden and this happened after 48 days.

There are other techniques for de-indexing pages from Google. For example, Google's Webmaster Tools has a place for you to enter the URL's you want to remove and you can also add the pattern to your robots.txt file which might have de-indexed them faster. The objective of this exercise was not to rapidly de-index those pages but to see how Google naturally de-indexed them over time when given a 301 redirect directive.
My surprise is how long it took to do that.
I'm not going to show a chart of the indexing of the new URL's because it's exactly as you would expect with the line rising rapidly up to the previous values. As I mentioned, 93% of the new links had been indexed within 5 days of them appear on the site and 100% had been indexed by day 13.

Loading a web page in a browser

Tony Gentilcore has just written a good post on How a web page loads and why blocking needs to take place when scripts and CSS load. I'm hoping that he's going to follow this up with a post on the SPDY protocol which looks pretty interesting for a faster web.

Google say that SPDY is an experiment with protocols for the web. Its goal is to reduce the latency of web pages.

At the time of writing this the third draft of the specification for SPDY could be found here.

One of the advantages that SPDY provides is that the resources needed by the page (JavaScript, images, CSS etc.) can be sent to the client in a compressed header and requested by the client before the client parses the HTML.

There's no indication if Google are going to push this specification yet because it's still experimental. One of the great advantages about owning an increasingly popular browser (Chrome) and a couple of popular web sites is that you can define the transport layer when your users are using your software end-to-end.

SEO Top Negative Ranking Factors

Great search engine ranking factors article recently published SEOmoz.

Point #2 in the Top 5 Negative Ranking Factors is "Link Acquisition from Known Link Brokers/Sellers." I still find this item hard to believe although I've heard this type of statement before.

I remember attending an SEO talk where an "expert" said that links from "bad" sites to your site could harm your SEO. I asked the presenter if she was able to provide any evidence of this and she said that she could not. To date, I have not seen any evidence of this, and as I said I find it hard to believe and this is why:

A competitor could publish links to your site from "bad" sites and/or buy links to your site on a link broker's site to degrade your SEO performance and allow him/her to rise above you in search results. Are you telling me that the engineers that work on the search engine algorithms have not thought of that? These are clever guys, believe me, they've thought of this.

In my opinion, link building from "bad" sites or buying from link brokers will not have a negative impact on your SEO, it will only have a negative impact on your time and a neutral impact on your SEO. i.e. you're wasting your time by doing it as it will have no impact.

Named parameters in a URL

Bill Brown was showing me his NLP links in Pinboard when I noticed that he was typing named parameters in the pinboard URL. I'm intimately familiar with named parameters which have been around for a long time in computer languages and have recently been introduced into C# and am a big fan of them. However, I have never seen them used (or thought of using them) in a URL before, here is an example:

http://pinboard.in/u:bbrown/t:python/t:nlp

In this URL we're saying find user (key=u, value=bbrown) named bbrown and all items that he's tagged (key=t) with both python and nlp. If we swapped the tags around the query produces the same results:

http://pinboard.in/u:bbrown/t:nlp/t:python

All is not perfect, however, in Utopia. If you swap the user param to after the tag param it won't work. This link takes you to a broken page:

http://pinboard.in/t:nlp/t:python/u:bbrown

This is fairly easy to fix and I'm guessing that the guys at pinboard.in don't think that many people will be hacking their url's and so didn't put the time into making them work in any order like true named parameters would.

It's still great to such innovation in the use of the url without using the query param which would have achieved the same result. Out of curiosity I tried:

http://pinboard.in/?u=bbrown&t=nlp

but that doesn't translate for them.

Google AdSense Best Day Of Week

I've just been doing some Google AdSense analysis for some web sites that have moderate traffic. The owner said that I can publish this bit of information from her site. She's interested to hear if other AdSense publishers have had similar results.

The objective was to find out which day of the week the Cost Per Click (CPC) was the highest. So we ran the averages across a number of pages and the average CPC by day of week ranked the days in order from worst to best as follows:

1. Monday
2. Sunday
3. Thursday
4. Friday
5. Wednesday
6. Tuesday
7. Saturday

Before running the analysis she was pretty certain that Tuesday was the best day of the week and was surprised to see that Saturday was in fact the best.

Rocket Surgery Made Easy by Steve Krug

I just finished reading Rocket Surgery Made Easy by Steve Krug and recommend you read it.

It's sub-titled The Do-It-Yourself Guide to Finding and Fixing Usability Problems.

It's a great read. He writes well, has appropriate sarcasm and wit at the right places, and delivers everything you need to know about usability testing. I bet that even usability professionals will find information in there that's of use even though it's not targeted at them. He also gets extra points for quoting Douglas Adams of Hitchhiker's Guide fame: "I love deadlines, I love the whooshing noise they make as they go by."

Right at the beginning of the book he gives you a link to an online video of someone taking a usability test and him guiding them through the test. This would really suck if you didn't have internet access close to you when you were reading the book so watch the usability video as soon as you get the book and don't even wait to get to that part.

One of the cool things that I learned (and I have no idea why I didn't think this would exist) is that you can outsource your usability testing and there are sites on the web that will do it for you. This is a no-brainer if you don't have the resources to set this up yourself and it sounds like a reasonable price to pay for the information that you'll get out of it. Just point the candidate at your site (or your beta site) and let them at it.

I thought he'd listed several online usability sites but while paging back through the book I can only find www.usertesting.com.

 

Building Quality Links to your Site

Great blog post by Google Webmaster Central Blog about building quality links to your site.

This paragraph sums up why you shouldn't engage in buying links or link exchange programs:

It's important to clarify that any legitimate link building strategy is a long-term effort. There are those who advocate for short-lived, often spammy methods, but these are not advisable if you care for your site's reputation. Buying PageRank-passing links or randomly exchanging linksare the worst ways of attempting to gather links and they're likely to have no positive impact on your site's performance over time. If your site's visibility in the Google index is important to you it's best to avoid them.

From the Buying PageRank link:

Buying or selling links that pass PageRank is in violation of Google's webmaster guidelines and can negatively impact a site's ranking in search results.

There is no point or reason to be trying to game Google's Page Rank system. It's going to be a ton of effort and it's not going to work. Just don't waste your time with it.

Google home page has images

I noticed this morning that Google has take a leaf from the Bing book and added images as a background to their home page. I think this is great as the images are always fascinating and high quality. However, the one thing that irks me with both Google and Bing home page images is that there doesn't appear to be an easy way to find out where the picture was taken. I would love to know where this was taken, but it may always be a mystery to me unless someone posts a comment here letting me know the source.

Google TV and Advertising Commercials

I was pretty excited when I watched the Google TV announcement last week.

Today I read a couple of commentaries about it and they went on about the features etc. but nobody mentioned the advertising. This surprised me as my initial guess is the reason that Google have entered this market is to gain more surface area for their advertising machine. In my opinion this is great news for all of us.

Everybody complains about the commercial breaks and if you ask someone with a Tivo why they like it they almost always tell you it's because they can fast forward through the commercials. If we don't have commercials then there's not going to be anyone to pay for the movies that we watch. That means smaller budgets for the studios and lower quality for us. We have to have adverts or we will end up moving to a subscription based viewing platform such as Netflix and that might kill off competition for great series and movies and lower the quality as well.

The perfect compromise is show commercials that you want to see. This, I believe, can be achieved with Google TV. By registering yourself and your interests on your device you make it easy for Google (or whatever content provider) to target you with relevant adverts. If I'm in the market to buy a car then I can change my profile settings to reflect that. I can give them my location, price range and model types and dealers in my area can then show ads to me during the breaks. The local dealers aren't going to pay much for that ad space because it will only be delivered to a few people like myself who are in the market for their product. National campaigns probably hit deaf ears 95% of the time. Targeted campaigns should be hitting willing-to-buy consumers 100% of the time. In reality this will of course be lower because there will also be car enthusiasts etc. who will be tuned into that type of ad because of their fanatical nature.

Once you have bought your car and are no longer in the market for one you can change your profile and pick something else that interests you. The question "how do we get consumers to change and/or set their profiles?" then arises. Again, this is easy. Constantly remind them that if they want to see adverts about products that interest them then they should update their profiles.

 

Google to help kill IE6

I got an email from Google this morning which in part said:

"...over the course of 2010, we will be phasing out support for Microsoft Internet Explorer 6.0..."

This is great news. I have a couple of sites that I work on that just don't work in IE6 and it's the bane of my life. I really don't want to be wasting investing time in getting IE6 to work when I could be adding new features and improving performance. 

With Google behind this I am hoping that this will accelerate users upgrading from IE6.

Also related are the IE6 Update and the IE6 No More sites.

XMarks Bookmark Synchronization

Last week I installed XMarks Bookmark Synchronizer and I am impressed. I use 4 browsers (sometimes 6 if you include IE6 and IE7) and keeping my bookmarks in sync on all the browsers and across all machines is a real problem. I regularly use about 5 different machines so with my 4 regular browsers that makes 20 browsers that I would like to keep in-sync. XMarks makes this a breeze.

XMarks supports IE8 (probably IE6 and IE7 but haven't installed on these so don't know), Firefox, and Chrome. I don't believe that it supports Opera or at least I haven't found something that will work with Opera and XMarks yet.

When you install it do so on your browser that has your definitive or most comprehensive set of bookmarks and when walking through the wizard select the option to synchronize your bookmarks. On subsequent browsers you want to select the option to blow away your local bookmarks and use the ones on the server.

After that, any book mark you add, delete or modify on any browser will then be synchronized with all other browsers and across multiple machines.

Very impressed with this free utility.

NTLM Active Directory Integration in Firefox

At work we use Active Directory to authenticate our internal websites. This is great if you're using Internet Explorer because it will pass through to the application without requiring further authentication. However, on Firefox you're required to make some changes to get this to work.

Open Firefox and go to about:config

Filter on "auth"

Set all booleans to true

Set network.automatic-ntlm-auth.trusted-uris to a comma separated list of domains that you want AD to do pass through authentication on.

That last instruction never used to work for me. I used to put in the full domain name but for some reason it didn't like it. I've now learned that you only need to put in the trailing part of the domain (TLD/Top Level Domain) and it will authenticate all domains and sub-domains for you.

Say you work for IBM and your internal domains follow a pattern of something.internal.ibm - all you need to do is add ".internal.ibm" to the trusted-uris setting and it will work for you. In fact, all you need to add is ".ibm".

Another thing I often need to do is connect to a site by IP. This will also work by dropping in the last octet of the IP into this list. e.g. ".164" (Obviously you could cover all IP addresses by dropping in all possible 256 octets.)

Here is how to generate all the octets using PowerShell:

$numbers = 0..255 | %{".{0}" -f $_}
$octets = [string]::join(",", $numbers)
$octets

The proxy server is refusing connections in Firefox

If you're a developer and you're using Firefox and you suddenly get this message:

The proxy server is refusing connections

It could be because you closed Fiddler but still have Firefox set to send traffic to Fiddler. Just click on the Fiddler link in your status bar and switch it off or restart Fiddler.

Changing Graffiti to use Google Analytics Async

Google have just released an asynchronous version of their Google Analytics site tracking code. The main advantage of this is that you can have the code start executing and collecting stats as soon as the page has started loading (because you can put it higher up in the HTML) and it won't interfere with your load times because it's asynchronous.
This blog is running on Graffiti CMS and I was pleasantly surprised at how easy it was to change to the asynchronous version of analytics.
When logged in to your Graffiti CMS site as an admin click on the Control Panel link and then on Site Options. From Site Options click on the Settings box.

At the bottom of the "Your Site Options" page you will see two text areas. If you are already using Google Analytics then you will probably have your code in the first box labeled Web Statistics. Putting your original analytics code in this box is good because this puts the analytics code at the bottom of your page just before the </body> tag.
However, now that we have an asynchronous version of analytics we should put the new code in the second box, the Header box, because this will put the analytics JavaScript in the <head> tag and it will load earlier and also ensure that your page loads faster. (Positioning it in the header won't make your page load faster but the fact that it's asynchronous will.) This is what these two boxes will look like when you've finished editing.

Before you delete your old analytics code from the Web Statistics text area you need to save your site's UA code. This is the only part of your old code that you'll need.
Then you will want to paste the following into the Header part of this page:

<script type="text/javascript">
  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-XXXXX-X']);
  _gaq.push(['_trackPageview']);

  (function() {
    var ga = document.createElement('script');
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    ga.setAttribute('async', 'true');
    document.documentElement.firstChild.appendChild(ga);
  })();
</script>

and replace the UA-XXXXX-X part with your value.
It really is as easy as that. I suddenly have new found respect for Graffiti CMS.

Age Analyzer

As part of the text classification work that I do I follow the uClassify blog who have just release a new web site called Age Analyzer. which guesses the age of the author of a blog. I plugged in the URL of this site and it guessed me in the 26-35 bracket. This would be flattering if you were to look at me but I find it insulting that my writing is considered this immature. It should have put me in the 36-50 bracket.

My friend Rob Manderson's eloquent prose has aged his Ultramaroon Rises Again blog in the 65-100 range (I'm sure you're not that old Rob). Dan Esparza's Esta Nublado puts him at a very young 18-25, and Bill Brown's New Clarion puts him in the 65-100 bracket while his bblog looks a little more accurate at 26-35.