OSCON Introduction to Clojure

The Introduction to Clojure page on the OSCON site.

Neal Ford (ThoughtWorks)
9:00am Tuesday, 07/23/2013
bit.ly/clojureinsideout

My notes from the Introduction to Clojure session.

Clojure is Functional, Dynamic, Hosted, Open Source, has an Atomic Succession Model, Lisp

The most enjoyable part of this presentation was that I was back in Lisp land again. Although I only spent 6 to 12 months working in Lisp I love the radically different structure and syntax of this language.

Data:
EDN Extensible Data Notation
data is immutable
nil - nil, null, nothing
booleans - true or false
strings - double quotes, can span multiple lines,, include \t \r \n
char - same as JVM
integer - same
double - same

Two syntaxes for naming things
symbols - e.g. variable names and namespaces, should map to something
keywords - like enumeration values, must start with :

Collections
lists - a sequence of values, zero or more elements within () like lisp
vectors - in [] like an array, supports random access
maps - key/value associations, looks like JSON, can have a vector as key, keys are unique
sets - {} collection of unique values

Clojure syntax is edn + language syntax
println("Hello World") becomes (println "Hello, world")

Operators: (+ 1 2 3 4 5)

Infix
int x = 40 - (5 + 10 * 2) becomes:
(def x (- 40 (* 2 (+ 5 10))))

Defining functions
(defn greet     <- name
"some text"  <-
[your-name]
(str "Hello, " your-name))

(defn larger [x y]
    (if (> x y) x y ))

Anonymous Functions
(map (fn [x] (* x 2)) (range 10))
i.e. pass functions as parameters
result (0 2 4 ... 18)
syntactic sugar for above:
(map #(* % 2) (range 10))

Namespaces
(ns com.example.foo)
- maps to JVM as you'd expect
Can wrap requires:
(:require...
Can add namespace meta data
Use :exclude to override (e.g. override + for vector math)
Don't use :use

Platform Interop
java Math.PI
Clojure sugar: Math/PI

java dot notation
person.getAddress().getZipCode() becomes:
(.. person getAddress getZipCode)

Clojure is a homoiconic language. i.e. it's a language that's represented by its data structure. Languages that you might have heard of that exhibit homoiconicity:
Curl
Julia
Lisp
Scheme (Lisp dialect)
Clojure (Lisp dialect)
Racket (Lisp dialect)
Mathematica
Prolog
SNOBOL
Tcl
XSLT

Functional Programming
- Easier to reason about - higher level of abstraction
- Easier to test
- Easier to compose
- Essential at scale

Persistent Data Structures
Composite values - immutable
'Change' is function of value to value
Collection maintains performance guarantees

uses Bit-partitioned hash tries

Constructing Values
Looping via recursion (recur)
prefer higher-order library fns when appropriate

Recursive Loops
No mutable locals in Clojure
No tail recursion optimization in the JVM
recur op does constant-space recursive loping
Rebinds and jumps to nearest loop or function frame

Loop alternatives
(loop
; reduce with adder fn
(reduce
;apply data constructor fn
(apply
;map into empty (or not) structure
(into
;get lucky
(zipmap

Benefits of Abstraction
Better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures - Alan J. Perlis
All Clojure collections have (seq available.

Sequences
Abstraction of traditional Lisp lists
(seq coll)
(first seq)
(rest seq)

In Lisp the first and rest functions used to be called car and cdr.

Lazy Seqs
Evaluated at execution
Very similar to LINQ in C#, same type of syntax

Operations on sequences:
(drop
(take 9 (cycle
(interleave
; splits collection into param partitions
(partition 3
; create key values
(map vector
; use like String.Join():
(apply str (interpose \, "asdf"))
-> "a,s,d,f"
(reduce + (range 100)) -> 4950

(for
is a macro and not an imperative loop

Seq Cheat Sheet
http://clojure.org/cheatsheet

Vectors
(def v [42 :rabbit [1 2 3]])
(v 1) -> :rabbit
(peek v) -> [1 2 3]
(pop v)
(subvec v 1)
(contains? v 0) -> true
(contains? v 42) -> false ; last param is index

Maps
(def m {:a 1 :b 2 :c 3})
(m :b) -> 2 ;also (:b m) will work
(keys m) -> (:a :b :c)
(assoc m :d 4 :c 42) ->
(dissoc m :d)
(merge-with

Nested Structures
(def jdoe {:name "John Does", :address {:zip 27705, ...}})
(get-in
(assoc-in
(update-in

Sets
(use clojure.set)
(def colors #["red" "green" "blue})
(disj
(difference
(intersection
(union

Addition
(def v [1 2 3]) ;vector
(def l '(1 2 3)) ;list
(conj v 42) -> [1 2 3 42]
(conj l 42 -> '(42 1 2 3)

(into v [99 :bottles])
(into l [99 :bottles])

Destruction
Pervasive Destructuring
DSL for binding names
Works with abstract structure
Available wherever names are made
sequential (vector) map (associative)

Special Forms
(def symbol init?)
(if test then else?)
;do returns the last value executed:
(do exprs*)
(quote form
(fn name? [params*] exprs*)

(let [binding
(loop
(recur
(throw
(try expr* catch-clause* finally-clause?]

Macros
Thread First ->
(-> 
Do everything from inside out - i.e. first statement is inside statement

Thread Last ->>
(->>

Boids
github.com/relevance/boids
Algorithm in Clojure
Cross compiled to ClojureScript

Example: perfect #

Break exercises
clojure.org - install clojure
leiningen.org - build tool
github.com/functional-koans/clojure-koans -> for beginners
projecteuler.net -> if you know the syntax of clojure

Second part of session

Leiningen
run a REPL etc.
"Maven meets Ant (withou the pain)"
RubyGems/Bundler/Rake

Compojure
Lightweight web framework - most popular
MVC based
github.com/abedra/shouter - example compojure app

Traditional OO
objects are mutable
encapsulate change & info
polymorphism lives inside object
extend via inheritance
interfaces are optional

Clojure
expose immutable data
encapsulate change (not data types)
polymorphism a la carte
interfaces are mandatory
extend by composition

defrecord
(defrecord Person [fname lname address])
(defrecord Address...
(def stu (Person. "Stu" "Halloway" (Address...

defrecord Details:
Type fields can be primitives
Value-based equality & hash
in-line methods defs can inline
keyword field looks can inline
protocols make interfaces

Protocols
(defprotocol AProtocol
     "A doc string for AProtocol abstraction"
      (bar [a b] "bar docs")
      (baz [a] "baz docs"))
named set of generic functions
polymorphic on type of first argument
no implementation
define fns in the same namespaces as protocols

Extending Protocols
(extend-type
Like extension methods in .Net but more extensible
(extend-protocol

ClojureScript extension
; extends JS array
 (extend-type array ISequable

Reify
(let [x 42 r (reify AProtocol ; implement 0 or more protocols or interfaces

deftype
use defrecord for information
use deftype

Concurrency - done at the same time
Parallelism - the execution of items in parallel

Values
immutable
maybe lazy
cacheable forever
can be arbitrarily large
share structure

What can be a value?
42
{:first-name "Stu"...}
anything...

References
refer to values or other references
permit atomic, functional succession
model time and identity
compatible with a wide variety of update parameters

API
(def counter (atom 0))
(swap! counter + 10)

Shared Abilities
@some-ref
(deref some-ref)
(add-watch...

Atoms
(def a (atom 0))
(swap! a inc)
=> 1
(compare-and-set! a 0 42)
=> false
(compare-and-set! a 1 7)
=> true

! - mutates
? - returns true or false

Software Transactional Memory (STM)
refs can change only within a transaction
provides the ACI in ACID (Atomic Consistent Isolated)
(defn transfer
   [from to amount]
   (dosync
     (alter from - amount)
     (alter to + amount)))

(alter from - 1)
=> IllegalStateException No transaction running

STM details
uses locks, latches internally to avoid churn
deadlock detection and barging
no read tracking
only Haskell and Clojure have transactional memory
readers never impede writers
nobody impedes readers

Pending references
work not done yet

Future
(def result (future...
(defef result 1000 or-else) ; timeout
@result deref result) ; wait without timeout

Delay
(def result (delay dont-need-yet))

Promise
(def result (promise)) ; no-org constructor

Java API
all work in Clojure
thin wrapper over Java API

Summary
Clojure is a 21st century Lisp
popular with framework designers
most advanced language constructs
most interoperable with underlying platform
powerful abstractions
active community
"a programming language beamed back from the near future" -Stuart Halloway

OSCON 6 Minute Apps

The 6 Minute Apps! Build Your First Modern Web App page on OSCON.

James Ward (Typesafe)
1:30pm Monday, 07/22/2013

OSCON Introduction to Scala

The official Introduction to Scala page on the OSCON site.

Dianne Marsh (Netflix), Bruce Eckel (Mindview, LLC)
9:00am Monday, 07/22/2013

Great presentation by Bruce and Dianne. They covered a lot of material in an easy to digest manner. After each nugget of information they had a great set of exercises that stretched the mind well beyond what they had explained. i.e. explain a concept and the accompanying syntax and then ask you to apply it to a typical real world problem.

(They call the nuggets atoms because it's the smallest amount of information you can explain about a corner of Scala without it being overwhelming and still being complete. I believe that this is the format that their book has followed.)

This was the first time that I've ever seen Scala and my immediate impression was that this is a language that is building on Java to play catchup with C#. Due to the lack of features being added to Java over the years C# had moved ahead with a great set of new features and to address this on the JVM Scala took up the challenge. Remember that I've spent all of 3 hours on Scala so this could be completely wrong.

 My notes on what I've picked up so far.

All objects are either a val or var. Vals are called const or constant in most other languages and are immutable. A var is variable that can be changed. Val and var declaration take data types as optional parameters otherwise the type is inferred.

val myNumber:Int = 1
val myFloat:Double = 1.1
val myBool:Boolean = false
val myWord:String = "A string"
val myLines:String = """Triple quotes let you have quotes" and
multiple lines under a another trifecta of quotes"""

Use semicolons to put multiple expressions on the same line:
var age=42; val color="blue"

Unit is the same as void in most C based languages or Nothing in Visual Basic.

The last line of code executed inside curly braces will be the value assigned to the variable, for example:

val area = {
     val pi = 3.14
     val radius = 5
     pi * pi * radius
}

Comments are the same as Java and other modern C languages: // and /* comment */

If statements are the same as Java:
if(x) {
}
if(!x) { // if x is not true
}

A difference with Scala is that the result of an if (and else) statement evaluation can be assigned to a val or var.

e.g.
val result = {
    if(99 > 100) { 4 }
    else { 42 }
}

In Java/C# this would be:
int result = 99 > 100 ? 4 : 42;

Boolean && and || are as you'd expect.

...Unfinished...

O'Reilly Open Source Convention (OSCON) 2013

This blog post will be an index into my notes from the sessions that I attended at OSCON 2013 from July 22 to July 26.

Monday July 22, 2013

• Morning Session: Introduction to Scala
• Afternoon Session: 6 Minute Apps! Build Your First Modern Web App

Tuesday July 23, 2013

•  Morning Session: Introduction to Clojure
• Afternoon Session: Improve Your Team With Improv

Wednesday July 24, 2013

• GitHub Power Tools
• 10 Reasons You'll Love Dart
• JavaScript Framework Face Off
• Server-Side Push: Comet, Web Sockets, and Server-Sent Events
• Getting Hadoop, Hive and HBase up and running in 15 minutes
• Translated Strings and Foreign Language Support in JavaScript Web Apps

Thursday July 25, 2013

• Getting Started with 3D Programming in Three.js
• Deal with multiple types of input in your HTML5/JavaScript code
• Rebooting the Team
• Adventures in Node.js
• New Rules For JavaScript
• Developing full stack JavaScript applications

Friday July 26, 2013

• OSCON Town Hall
• The Google Open Source Update
• Clientside MVC: A Journey

Ship It Day #2

On Friday July 12, 2013 our team, held our second Ship It day with the judging taking place the following Monday, July 7/15.

At the previous ship it day we focused all the prizes on the demonstrations that were given. In both instances we asked a panel of judges to submit a score, based on their own criteria, from 0 to 10 for each team/demo.

Initially the judges wanted guidance on how to score the submissions. Several attributes were suggested but I resisted this because I felt that each judge already has a different viewpoint of what they like to see in a demo and what is important to them. The selection of a wide range of judge types was more important than giving them attributes to score on that might not make sense to each individual. The judges were Business Analysts, User Experience (UX) Engineers, Product Managers, and Senior Software Engineers.

At this Ship It Day we changed the focus to emphasize the importance on creating something that was actually shippable at the end of the exercise. Small prizes have been established around the demo but the biggest prizes are around who can ship in the week immediately after Ship It Day and then the prize size decreases for week two and week three. Given that we are in the middle of week one nothing has been given out yet but there are indications that at least one of the teams will ship in the two weeks immediately following Ship It Day. I'll update this blog post with more details in around three weeks.

In total we had fourteen self-organized teams at this event which is up six (+75%) from the eight we had at the last event. As usual we provided lunch for the team and snacks and drinks throughout the day and at the end of the day we had adult beverages for everyone.

TypeScript as a better JSLint

TypeScript, if you haven't heard of it, is Yet Another JavaScript Transpiler (YAJST). Their marketing pitch:

TypeScript is a language for application - scale JavaScript development. TypeScript is a typed superset of JavaScript that compiles to plain JavaScript. Any browser. Any host. Any OS. Open Source.

Apart from using the language for its added benefits you can also use the TypeScript Compiler (tsc.exe) as a very powerful and better form of JSLint that can catch "compile time errors" in a language that isn't compiled, in this case JavaScript.

A compiled language like Java, C, C++, or C# benefit from a first round of implicit unit tests which is the compilation of the code. i.e. before deploying the code it's been validated by the compiler and we know that syntactically it is "clean." Of course it may still be very buggy.

JavaScript doesn't have this benefit. We can run unit tests against the JS we write and if not syntactically correct we can catch some of those syntactic errors. However, if we don't have exhaustive, comprehensive and high quality unit tests we won't catch everything. We also can't test for intent.

Here's a trivial JavaScript example that works and is valid but may be a bug:

function getDivisor() {
    return "5";
}

function myTest() {
    console.log(10 / getDivisor());
}

If you run this through the TypeScript Compiler then you'll get this error:

TypeScriptTest.ts(3,17): error TS2112: The right-hand side of an arithmetic operation must be of type 'any', 'number' or an enum type.

JavaScript will execute this code correctly. It coerces the "5" in a string into an int and everything works as expected. However, TypeScript knows that the only type that can be returned by the getDivisor() function is string and by using implicit typing it knows that you can't (or shouldn't) try and divide by a string.

We can fix this by changing getDivisor() to look like this:

function getDivisor() {
    return parseInt("5");
}

Even if you don't use any of the features of TypeScript, your code quality can improve by inserting TypeScript as part of your build process. I'd suggest that you add this immediately after any other code in your system has been compiled and just before you run any unit tests.

As a note, TypeScript will not accept a .js file as an input, it will only accept .ts files. As such you will need to copy your .js files to a temporary location, renamed them to .ts files, run tsc.exe against them and capture the output.

Scott Guthrie Event 2013

On 16 May 2013, Scott Guthrie along with Damian Edwards and Joshua Twist came the Valley of the Sun for Scott Gu's tenth annual technical presentation.

I was lucky enough to be on-stage at 4pm for a ten minute presentation. The event took place at the Scottsdale Center for the Performing Arts, 7380 E. 2nd Street, Scottsdale, AZ. There were around 800 attendees but I estimate that there were probably 400 left by that time.

Below is the content of my speech.

Good afternoon ladies and gentlemen. My name is Guy Ellis. I’m the director for software development for the presence and commerce team at Go Daddy. This is my sixth year as an employee at Go Daddy. I started off at Go Daddy as a .Net developer working with the internal tools team. I then moved to our SEO product called Search Engine Visibility which I managed before moving onto the Presence and Commerce team which is responsible for creating our Website Builder and Quick Shopping Cart products.

Over the years I’ve been to almost all the Scott Guthrie events and like this one they’ve all been fantastic and I’d like to thank Scott Cate and everyone else involved for making this available to us at no cost.

I’d like to share with you how I explain SSL to the layperson, or as I sometimes say "SSL for your grandmother."

As developers you most likely know how SSL works but how do you explain to a non-geek that Secure Sockets Layer uses asymmetric public key encryption to prevent a man-in-the-middle attack? How do you explain to them how their username, password and financial data that’s traveling over the ether when they’re accessing their bank account cannot be used if it’s intercepted. They see the green address bar in the browser and the S at the end of HTTP and they know they want it but don’t understand how it works.

To help us understand this I need to take you back in time. A very long time ago. Before the internet. Before television. Before phones where invented. I was a very young man. I lived in a small village on the banks of river. One day I was walking next to the river and on the other side I saw the most beautiful woman I’d ever seen. I couldn’t help myself, I waved and shouted across to her. She took one look at me and headed off. You may notice that men still try and use this technique today except they are usually hanging out of a car window but the impact is generally the same.

I had to get to know this woman so I rushed back to the village and asked around and found out that she lived in the village on the other side of the river. Unfortunately, back then I was afraid of water and unable to take the ferry across the river to meet her. My only choice was to write her a letter and win her over with my romantic prose.

I spent all night writing a letter and in the morning I gave it to the ferryman to deliver to her. I watched from the dock as he paddled across the river disappeared into a house. About an hour later a young man, about the same age as me, came out of the house and went off to her house with the letter and delivered it to her house. She opened the letter and read it and appeared very excited by it but then hugged and kissed the man who had delivered it.

I hired an investigator to find out what had happened and soon it became apparent that this other young man was the ferryman’s son and he had taken my letter and rewritten it from himself and was using my carefully crafted masterpiece to win her heart for himself.

I knew I had to work out a way to get my letters to her without the possibility of interception. I got myself a box, put my next letter in it, put a padlock on it and sent it across the river. I included instructions for her to attach her padlock and send it back to me. Once I received the box back I removed my padlock and then sent it back to her knowing full well that she was the only one that could open the box. My plan was infallible, or so I thought. I later discovered that the ferryman’s son had attached his padlock to the box and stolen my letter again.

I then recruited the help of a man who referred to himself as Mr. Authority. I sent the box with padlock and letter and it came back with an old rusty padlock on it in addition to mine. Mr. Authority took a look at the other padlock and shook his head. This isn’t her lock he said. I rejected the box and the next time it came back it had a beautiful pink padlock that smelled of roses. Mr. Authority confirmed that this was her lock and I removed mine and she finally started receiving my letters.

Several letters later she took the ferry across the river and we met and fell in love and she is now my wife and that, ladies and gentlemen, is how SSL works.

This is the view from the stag when the lights are not switch on. When the lights are on you can't see anybody or any of the seating. It's just a blazing glow.

Photos courtesy of Richard Kimbrough Photography.