This posting on the Java core library mailing list proposes to replace the current quicksort with a new, 2-pivot quicksort.

Ordinary quicksort in Haskell looks something like this:

quicksort [] = []
quicksort (pivot:rest) =
   quicksort [x| x ← rest, x ≤ pivot] ++
   [pivot] ++
   quicksort [x| x ← rest, x > pivot]

Note the similarity of this to the notation for ‘invariants’ used in the email:

[ <= p | >= p ]

Similarly, for the 2-pivot invariants (I have corrected a typo in this):

[ < P1 | P1 <= & <= P2 | > P2 ]

We can derive the code:

quicksort2 [] = []
quicksort2 (only:[]) = [only]
quicksort2 (first:second:rest) =
   quicksort2 [x|x ← rest, x < p1] ++
   [p1] ++
   quicksort2 [x|x ← rest, p1 ≤ x && x ≤ p2] ++
   [p2] ++
   quicksort2 [x|x ← rest, p2 < x]
  where (p1,p2) = if first ≤ second then (first,second) else (second,first)

Which does, in fact, run faster than the ordinary 1-pivot quicksort.

n-pivot quicksort

Of course, this being Haskell we want to take things to their logical conclusions, and allow any number of pivots to be used. My formulation follows:

quicksortN _ [] = []
quicksortN _ (x:[]) = [x]
quicksortN n xs
    | n > length xs = quicksortN ((n `div` 2) `max` 1) xs
    | otherwise = part True pivots
    where
      (takexs,dropxs) = splitAt n xs
      pivots = quicksortN 1 takexs
      part False (p2:[])      = quicksortN n [x|x ← dropxs, p2 < x]
      part False (p1:p2:rest) = quicksortN n [x|x ← dropxs, p1 < x && x ≤ p2] ++ [p2] ++ part False (p2:rest)
      part True  (p1:[])      = quicksortN n [x|x ← dropxs, x ≤ p1] ++ [p1] ++ quicksortN n [x|x ← dropxs, p1 < x] -- remember me? :)
      part True  (p1:p2:rest) = quicksortN n [x|x ← dropxs, x ≤ p1] ++ [p1] ++ part False (p1:p2:rest)

Now there are only 3 things that I can see that are left ‘tweakable’;

• how to reduce ‘n’ when it is greater than length of the list
  • dividing by 2 here
• how to sort the pivots
  • using a recursive call to the 1-pivot version of itself here
• whether or not to reduce the number of pivots as sorting continues
  • ‘no’ chosen here

I have no idea whether these are sensible defaults

Which n should I choose?

Finally, some code for the timings. You’ll need timeit to run it.

main = do
 gen ← getStdGen
 let n = 10^6
 let r = take n $ randoms gen :: [Int]
 print $ sum r -- force list
 timeIt . putStrLn . ("quicksort = "++) . show . sum $ quicksort r
 timeIt . putStrLn . ("quicksort2 = "++) . show . sum $ quicksort2 r
 mapM_ (\x → timeIt . putStrLn . (("quicksortN "++ show x++" = ")++) . show . sum $ quicksortN x r)
    [1..20]

Sample output:

$ ./sort
858124546
quicksort = 858124546
CPU time:   4.60s
quicksort2 = 858124546
CPU time:   3.66s
quicksortN 1 = 858124546
CPU time:   5.19s
quicksortN 2 = 858124546
CPU time:   3.77s
quicksortN 3 = 858124546
CPU time:   3.28s
quicksortN 4 = 858124546
CPU time:   3.27s
quicksortN 5 = 858124546
CPU time:   3.06s
quicksortN 6 = 858124546
CPU time:   3.04s
quicksortN 7 = 858124546
CPU time:   3.04s
quicksortN 8 = 858124546
CPU time:   3.07s
quicksortN 9 = 858124546
CPU time:   3.10s
quicksortN 10 = 858124546
CPU time:   3.14s
quicksortN 11 = 858124546
CPU time:   3.22s
quicksortN 12 = 858124546
CPU time:   3.31s
quicksortN 13 = 858124546
CPU time:   3.30s
quicksortN 14 = 858124546
CPU time:   3.37s
quicksortN 15 = 858124546
CPU time:   3.54s
quicksortN 16 = 858124546
CPU time:   3.41s
quicksortN 17 = 858124546
CPU time:   3.52s
quicksortN 18 = 858124546
CPU time:   3.65s
quicksortN 19 = 858124546
CPU time:   3.63s
quicksortN 20 = 858124546
CPU time:   3.77s

As expected, generalized quicksort performs slower than the equivalent hard-coded quicksorts for specific values of n, but higher values of n than 1 or 2 can perform better.

I didn’t like the fact that in the old code there were places where explicit try points needed to be included. It seems that these arose because the ‘obsolete’ syntax was tacked-on to the EBNF for the normal syntax, creating much overlap. Since I merged the syntaxes together, there are no explicit try points needed (there are some implicit ones, I believe, such as in optional). This makes the code both faster and easier to understand.

module Text.Email.Validation (isValid)
where
 
import Text.Parsec
import Text.Parsec.Char
import Data.Char (chr)
 
isValid :: String -> Bool
isValid x = 	either (const False) (const True) (valid x)
 
simply = (>> return ())
-- simply converts a parser returning something to a parser returning nothing
 
valid :: String -> Either ParseError ()
valid = parse addrSpec ""
 
addrSpec = localPart >> char '@' >> domain >> eof
 
localPart = dottedAtoms
domain = dottedAtoms <|> domainLiteral 
 
dottedAtoms = simply $ (optional cfws >> (atom <|> quotedString) >> optional cfws)
	`sepBy1` (char '.')
atom = simply $ many1 atomText
atomText = simply $ alphaNum <|> oneOf "!#$%&'*+-/=?^_`{|}~"
 
domainLiteral =  between (optional cfws >> char '[') (char ']' >> optional cfws) $
	many (optional fws >> domainText) >> optional fws
domainText = ranges [[33..90],[94..126]] <|> obsNoWsCtl
 
quotedString = between (char '"') (char '"') $
	many (optional fws >> quotedContent) >> optional fws
quotedContent = quotedText <|> quotedPair
quotedText = ranges [[33],[35..91],[93..126]] <|> obsNoWsCtl
quotedPair = char '\\' >> (vchar <|> wsp <|> lf <|> cr <|> obsNoWsCtl <|> nullChar)
 
cfws = simply $ many (comment <|> fws)
fws = (many1 wsp >> optional (crlf >> many1 wsp))
	<|> (many1 (crlf >> many1 wsp) >> return ())
 
comment = simply $ between (char '(') (char ')') $
	many (commentContent <|> fws)
commentContent = commentText <|> quotedPair <|> comment
commentText = ranges [[33..39],[42..91],[93..126]] <|> obsNoWsCtl
 
nullChar = simply $ char '\0'
wsp = simply $ oneOf " \t"
cr = simply $ char '\r'
lf = simply $ char '\n'
crlf = simply $ cr >> lf
vchar = ranges [[0x21..0x7e]]
obsNoWsCtl = ranges [[1..8],[11,12],[14..31],[127]]
ranges = simply . oneOf . map chr . concat

This now passes all of Dominic Sayer’s tests that it is meant to—the domain validation used in Dominic Sayer’s tests is more strict than RFC5322 specifies. Expect this to change!

For those who’d like to know, email addresses that now parse that didn’t before include the often-used (‘|’ is merely to indicate the end of whitespace):

I.                        |
 am.                  |
 a.      |
 nice.|
 guy@(yeah)you.com

In recent times there have been several calls for websites to properly validate email addresses. Invariably, the compiled regex from Perl’s RFC822 is pasted up as The Way To Do It. The problem with this is (as the source code from the Perl module notes) is that email addresses cannot be validated by a simple regex (due to requiring parenthesis-matching). The Perl code addresses this by first stripping out all comments and then parsing via regex.

With this in mind, I thought that implementing the Addr-Spec specification from RFC 5322 (only released less than 6 months ago) might be a good test of the Haskell library Parsec. So, without further ado I went ahead and translated the EBNF from RFC 5322 directly into Parsec.

The mapping is something like this:

juxtaposition

>>

/

<|>

*

many

1*

many1

[]

optional

Here is the result:

import Text.Parsec
import Text.Parsec.Char
import Data.Char (chr)
 
isValid :: String -> Bool
isValid x = let result = valid x in
	either (const False) (const True) result
 
valid :: String -> Either ParseError ()
valid x = parse addrSpec "" x
 
ignore x = x >> return ()
 
addrSpec = localPart >> char '@' >> domain >> eof
 
localPart = dotAtom <|> quotedString <|> obsLocalPart <?> "local part"
domain = dotAtom <|> domainLiteral <|> obsDomain <?> "domain"
 
domainLiteral = optional cfws >> char '[' >>
		many ( optional fws >> dtext) >>
		optional fws  >> char ']' >> optional cfws
		<?> "domain literal"
 
ranges = oneOf . map chr . concat
vchar = ranges [[0x21..0x7E]] -- from Backus-Naur RFC
dtext = ranges [[33..90],[94..126]] <|> obsDtext
qtext = ranges [[33],[35..91],[93..126]] <|> obsQtext
atext = alphaNum <|> oneOf "!#$%&'*+-/=?^_`{|}~"
ctext = ranges [[33..39],[42..91],[93..126]] <|> obsCtext
wsp = char ' '
	<|> char '\t'
	<?> "space or tab"
 
cr = char '\r' <?> "carriage return"
lf = char '\n' <?> "line feed"
crlf = cr >> lf <?> "CRLF line ending"
 
-- # modification: added try
cfws = try (many1 (optional fws >> comment) >> optional fws) <|> ignore fws
-- # modification from RFC: adding try because of overlap
fws = try (optional (many wsp >> crlf) >> many1 wsp)
	<|> many1 wsp
	<|> obsFws
 
-- # modification: added try
comment = between (char '(') (char ')') (many (try (optional fws >> ccontent)) >> optional fws)
	<?> "comment"
ccontent = ignore ctext
	<|> ignore quotedPair
	<|> comment
 
atom = optional cfws >> many1 atext >> optional cfws
dotAtomText = many1 atext >> many (char '.' >> many1 atext)
dotAtom = optional cfws >> dotAtomText >> optional cfws
 
-- # other change from RFC -- merge prefix
quotedPair = char '\\' >> ((vchar <|> wsp) <|> obsQp)
qcontent = qtext <|> quotedPair
quotedString = optional cfws >>	char '\"' >> many (optional fws >> qcontent) >>
	optional fws >>	char '\"' >> optional cfws
	<?> "quoted string"
 
-- # Obsolete syntax
obsNoWsCtl = ranges [[1..8],[11..12],[14..31],[127]]
obsCtext = obsNoWsCtl
obsDtext = obsNoWsCtl <|> quotedPair
obsQtext = obsNoWsCtl
-- # change: see above
obsQp = (char (chr 0) <|> obsNoWsCtl <|> lf <|> cr)
obsLocalPart = word >> many (char '.' >> word) >> return ()
obsDomain = atom >> many (char '.' >> atom) >> return ()
obsFws = many1 wsp >> many (crlf >> many1 wsp) >> return []
word = atom <|> quotedString

A note before I continue: since Parsec by default does no backtracking (in order to remain efficient), there are a couple of places (two that I’ve found so far) where the original EBNF needs to be changed slightly. I have noted these in the source above. It is possible there are a couple more places that need fixing, but I haven’t run this against a large test suite yet to find them. (They are most likely to be in the ‘obsolete syntax’ section.)

And of course, some demonstrations (keep in mind that there is an extra level of escaping operating here… where relevant I’ve included the unescaped email address in a comment):

isValid "porges@example.com" == True
isValid "porges@@example.com" == False
isValid "\"porges@\"@example.com" == True -- # "porges@"@porg.es
isValid "\"por(g)es@\"@example.com" == True -- # "por(g)es@"@porg.es
isValid "porges(comment)@example.com" == True
isValid "porges(comme(nests)nt)@example.com" == True
isValid "porges(comme(nests)nt())@example.com" == True
isValid "porges(()comme(nests)nt())@example.com" == True
isValid "()porges(()comme(nests)nt())@example.com" == True
isValid "((lol)porges(()comme(nests)nt())@example.com" == False
isValid "((lol))porges(()comme(nests)nt())@example.com" == True
isValid "(lol))porges(()comme(nests)nt())@example.com" == False
isValid "((lol))porges(()comme(nests)nt())@example.com" == True
isValid "\"s\\\0\"@example.com" == True
-- # "s\NUL"@example.com, where NUL is actually the
-- # null character! Yep, can't strlen() on email addresses...

I managed to find this post on email addresses by Phil Haack which has the following tests:

[("\"Abc\\@def\"@example.com",True),
("\"Fred Bloggs\"@example.com",True),
("\"Joe\\Blow\"@example.com",True),
("\"Abc@def\"@example.com",True),
("customer/department=shipping@example.com",True),
("$A12345@example.com",True),
("!def!xyz%abc@example.com",True),
("_somename@example.com",True),
("NotAnEmail",False),
("@NotAnEmail",False),
("\"test\\\\blah\"@example.com",True),
("\"test\\blah\"@example.com",True),
-- # Phil gets false for this, which I think is wrong
-- # (Dominic Sayers notes the same at the end of the comment thread)
("\"test\\\rblah\"@example.com",True),
("\"test\rblah\"@example.com",False),
("\"test\\\"blah\"@example.com",True),
("\"test\"blah\"@example.com",False),
("customer/department@example.com",True),
("$A12345@example.com",True),
("!def!xyz%abc@example.com",True),
("_Yosemite.Sam@example.com",True),
("~@example.com",True),
(".wooly@example.com",False),
("wo..oly@example.com",False),
("pootietang.@example.com",False),
(".@example.com",False),
("\"Austin@Powers\"@example.com",True),
("Ima.Fool@example.com",True),
("\"Ima.Fool\"@example.com",False),
("\"Ima Fool\"@example.com",False),
("Ima Fool@example.com",False)]

Next job is to test it against this batch…

Update: Yep, fails in several areas with the obsolete syntax. I’ve fixed one above. (Note that I’m not concerned with the failures in the domain part of the address, as the RFC5322 EBNF for this is more liberal than the tests require.)

Might have to refactor the syntax… there is a large overlap with the obsolete syntax. (Or just use ‘try’, but that’s not so efficient.)

How did we all miss that before?

—Peaker

The answer (of course!) is that while we might have, they didn’t. For example, Edward Kmett’s category-extras package has had Control.Monad.Indexed available for use in Haskell since early last year, while the concept for its implementation in Haskell has been around since at least 2005 (see the paper “Monadic Augment and Generalised Short Cut Fusion”). Needless to say, Oleg has been there too.

For me, this prompted a vaguely-related revelation. Say that we have a Category class:

class Category (⤳) where
	id ∷ a ⤳ a
	(∘) ∷ (b ⤳ c) → (a ⤳ b) → (a ⤳ c)

I had recently been wondering how categories other than (→), such as graphs (and even the trivial category) fit into this model. Now that I have figured it out, the answer is simple; it is a matter of arity. Graphs and other datastructure-related categories don’t have the extra type arguments (unless you’re working in a dependently-typed language), and so can’t fit into this. What we want to do is fake the extra arguments, by having some extra ones that we can just ignore (the name, FF, is short for FakeFake, i.e. two fake arguments).

newtype FF f a b = FF { unFF :: f }

Now we can define the trivial category!

class Category (FF ()) where
	id = FF ()
	f ∘ g = FF ()

Viewed in this light, it is obvious why (as Dan notes in the original post) Monoids give rise to Monads and Categories give rise to ParametrizedMonads, as we have:

class Monoid m ⇒ Category (FF m) where
	id = FF mempty
	(FF f) ∘ (FF g) = FF (mappend f g)

Next, we want to write implementations for all our old Monads in the new PMonad class. We might do it like this (with a new newtype for the different arguments of this datatype, RealFakeFakeReal):

newtype RFFR f x y a = RFFR { unRFFR ∷ (f a) }
 
class PMonad m where
	(>>=) ∷ m s1 s2 a → (a → m s2 s3 b) → m s1 s3 b
	return ∷ a → m s s a
 
instance (Old.Monad m) ⇒ PMonad (RFFR m) where
	(RFFR x) >>= f = RFFR (x Old.>>= unRFFR ∘ f)
	return x = RFFR (Old.return x)

There is a big issue that this highlights. It is hard to work with types of different arities in Haskell. (This is noted in the ‘quantified contexts’ proposal.)

As an example, since we now know that every Monoid is a Category, we might think to drop Monoids and just use Categories everywhere (I should note that this is actually a silly idea !). Unfortunately, we can’t just do this as we’ll have to use FF wrappers/unwrappers everywhere:

x = unFF $ FF [1,2,3] ∘ FF [4,5,6]

I have not yet found a workaround for this. Help would be nice

f x y z = ...

… and you want:

f x y z w = ...

First of all you should check the contexts where f is used. For example, if f is quite often used as an argument to map (fmap, <$>):

elsewhere xs = map (f a b) xs

If you naïvely add w to the end of the argument list you’ll end up with this instead:

elsewhere xs = map (\x-> f a b x c) xs

On the other hand, if you take a look at the usage of f you can decide that because f is often used to map over things, you can insert the additional parameter before the last, and end up with nicer code:

elsewhere xs = map (f a b c) xs

You may even discover that the existing f is used in a lot of lambdas that could be avoided by rearranging its arguments!

data Exp = Lit Int -- literal value
	| Call String Exp -- calling a function
	| Bin BinOp Exp Exp -- binary operations
	| Param -- the parameter
	deriving (Show,Eq)
 
data Function = Func String Exp -- a function has a name and expression
data BinOp = Add | Subtract | Multiply | Divide deriving(Show,Eq)
 
-- an instance to allow us to write nicer definitions
instance Num Exp where
	x * y = Bin Multiply x y
	x - y = Bin Subtract x y
	x + y = Bin Add x y
	fromInteger x = Lit (fromInteger x)
 
outFunc (Func name it) = name ++ -- output function name
	":\npush ebp\nmov ebp,esp\n" ++ -- save old stack frame
	output it ++ -- output function
	"pop eax\npop ebp\n" ++ -- restore old stack frame
	"ret 4\n\n" -- remove argument from stack and return
 
output (Bin op l r) = outBin op l r -- binary operations
output (Lit x) = "push dword " ++ show x ++ "\n" -- push literal
output (Param) = "push dword [ebp+8]\n" -- push parameter
output (Call name with) = output with ++
	"call " ++ name ++
	"\npush eax\n" -- put return value onto stack
 
outBin Add = outBinGen "add"
outBin Subtract = outBinGen "sub"
outBin Multiply = outBinGen "imul"
outBin Divide = outBinGen "idiv"
 
outBinGen op l r = output l ++ output r ++
	"pop ebx\npop eax\n" ++ op ++ " eax,ebx\npush eax\n"

And an example of using it:

main = putStr $ concatMap outFunc [double,quad]
double = Func "double" (Param * 2)
quad = Func "quad" (Call "double" (2*Param))

This example outputs:

double:
	push ebp
	mov ebp,esp
	push dword [ebp+8]
	push dword 2
	pop ebx
	pop eax
	imul eax,ebx
	push eax
	pop eax
	pop ebp
	ret 4
 
quad:
	push ebp
	mov ebp,esp
	push dword 2
	push dword [ebp+8]
	pop ebx
	pop eax
	imul eax,ebx
	push eax
	call double
	push eax
	pop eax
	pop ebp
	ret 4

Not the most efficient code in the world The calling convention is somewhat ad-hoc; the callee must remove its arguments from the stack and returns its value in eax.

If we have a sorted list of numbers, how can we efficiently get a sorted list of the sums of every pair of numbers in that list?

My initial reply is also there. I decided that, since I have a moment of spare time I should implement this. I’m semi-reprinting it here, but you might want to read it anyway.

Algorithm

The idea is that if you have a sorted list , you can write out the sums as a matrix where :

I’ve left out the lower half of the matrix as it is clearly symmetric ().

You can see that clearly and (since if then ), so we only have to examine the “top-left corners” of the matrix. As an example, we would first examine :

Then accept it since it is the only option, and move on to the next top-left corner, which again is a single choice:

Then we get two options:

And so on:

I think you get the picture

One of the points of this post is that I horribly misimplemented this and have spent many wasted hours today trying to figure out why it wasn’t working. Instead of the last sequence of diagrams, my algorithm was doing the following:

That is, every time a corner was eliminated it was adding both neighbours to the list of corners even if there was a corner further left on the row below it. As such, the following naïve algorithm was beating it by a factor of 4:

sortedSumSortedList xs = sort $ concat [[x+y|y<-drop nx xs]|x<-xs|nx<-[0..]]

Implementation

A preliminary: you’ll need the following imports for the code below.

import Data.Array.Unboxed
import Control.Monad
import qualified Data.IntMap as IntMap
import Test.QuickCheck
import Data.List
import Data.Time.Clock

First of all we need the sums of the list with itself:

sumLists :: [Int] -> [Int] -> UArray (Int,Int) Int
sumLists xs ys = array ((0,0),(length xs-1,length ys-1)) $ sumLists' xs ys 0
	where
	sumLists' _ [] _ = []
	sumLists' xs' (y:ys') ny = [((nx,ny),x+y)|x<-drop ny xs'|nx<-[ny..]] ++ sumLists' xs ys' (ny+1)

This does basically the same thing as the code above (concat [[x+y|y<-drop nx xs]|x<-xs|nx<-[0..]]) except that we want an array rather than a list so that we have constant-time addressing. In Haskell an array is created by specifying the boundaries of the array and then the data for the array. Note that the bottom of the array is left empty as in the diagrams; this is fine, we can leave some data unspecified as long as we never access it.

And here's the workhorse. I've commented it more thoroughly so you should be able to understand what's happening.

orderedSumsOfOrderedList sums corners row
	| IntMap.null corners = []
	| otherwise = replicate (length minCorners) minimumValue ++ orderedSumsOfOrderedList sums corners' row'
	where
	-- minCorners = all corners with the same (lowest) value
	-- allCorners = all other corners currently being considered (as a IntMap)
	(minCorners,allCorners) = IntMap.deleteFindMin corners
	-- minimumValue = the value in the lowest corner(s)
	minimumValue = sums ! (head minCorners)
 
	-- row is incremented if any of the neighbours has moved down a row
	row' = if any (\(_,_,b)-> b) neighbours then row+1 else row
	-- update the corners to reflect all neighbours of the just-removed corners
	corners' = foldr (\(key,value)-> IntMap.insertWith (++) key [value]) allCorners $ map (\(x,y,_) -> (x,y)) neighbours
 
	-- a list of values, their positions in the array, and whether to increment row
	neighbours = concatMap (\p-> goRight p ++ goDown p) minCorners
		where
			-- try to get a neighbour further right, within bounds of array
			goRight (x,y) =	ifM (x < maxX)                       (sums ! (x+1,y), (x+1,y),False)
			-- try to get a neighbour further down, within bounds of array and not on the same row as another one
			-- (if successful we should update row)
			goDown  (x,y) = ifM (y < x && y < maxY && y+1 > row) (sums ! (x,y+1), (x,y+1),True)
			maxX = (fst$snd$bounds$sums)
			maxY = (snd$snd$bounds$sums)

To efficiently choose the lowest number each time, we keep track of them in something like a binary tree. In Haskell the provided implementation is Data.Map (or Data.IntMap which is specialized for integer keys).

The rest of what this code does is basically:

1. Choose the lowest corner(s) from the tree of available corners.
2. Find the neighbours of these corner(s), ensuring they are valid (in this case this means checking bounds and checking that no two are on the same row—it suffices to ensure that no downwards-searching neighbours are added which are on a row less than the maximum row reached so far), and insert these into the tree.
3. Output the lowest corner(s) and recurse with the new tree and row number.

ifM is a nice little function I've had cause to use several times, and it looks like this:

ifM :: (MonadPlus m) => Bool -> a -> m a
ifM pred res = if pred
		then return res
		else mzero

And here's the wrapper function (osool = ‘ordered sums of ordered list’), we take in a list of Ints and return a list of Ints. Note that we do not check the precondition (that the input list is sorted).

osool :: [Int] -> [Int]
osool [] = []
osool xs = orderedSumsOfOrderedList sums corners row 
	where
	sums = sumLists xs xs
	corners = IntMap.singleton (sums ! (0,0)) [(0,0)]
	row = 0

sums is the sums of the list, and corners is the quick-lookup structure (in this case IntMap) which allows us to find the minimum fast. We initialize it with the value of the top-left corner of the matrix, and its coördinates. Finally, row is a value indicating which row we are up to. As shown above, we don't want to add any new corners which are less than this value when eliminating a square and looking for a neighbouring square in the downwards direction.

And finally, we can test this with the wonderful QuickCheck:

checkMe = do
	time "osool" $ quickCheck (\xs -> let sxs = sort xs in length (osool sxs) == num sxs)
	time "checkSumLists" $ quickCheck (\xs -> let sxs = sort xs in length (checkSumLists sxs) == num sxs)
 
	time "osool" $ longCheck (\xs -> let sxs = sort xs in length (osool sxs) == num sxs)
	time "checkSumLists" $ longCheck (\xs -> let sxs = sort xs in length (checkSumLists sxs) == num sxs)
 
	time "areEqual" $ quickCheck (\xs -> let sxs = sort xs in checkSumLists sxs == osool sxs)
	where
	longCheck p = check (defaultConfig { configSize = const 1000, configMaxTest = 500}) p
	checkSumLists xs = sort $ concat [[x+y|y<-drop nx xs]|x<-xs|nx<-[0..]]
	num xs = sum [1..length xs]
	time s d = do
		putStrLn $ s ++ ":"
		t1 <- getCurrentTime
		d
		t2 <- getCurrentTime
		putStrLn $ show $ t2 `diffUTCTime` t1

Results

$ ./test
osool:
OK, passed 100 tests.
0.024591s
checkSumLists:
OK, passed 100 tests.
0.014857s
osool:
OK, passed 500 tests.
15.448749s
checkSumLists:
OK, passed 500 tests.
82.871982s
areEqual:
OK, passed 100 tests.
0.007773s

As you can see, the wins over the naïve version are substantial.

class Wrapper<T>
{
    T value;
    public Wrapper(T theObject)
    {
        value = theObject;
    }
    // Some other methods...
}

Now, since this is a just a wrapper around some type T, we would like to implement some simple interfaces around this object. For example, if T is comparable, we would like the wrapper class to implement IComparable<Wrapper<T>>. However this is not possible with C#. In Haskell we would have something like this:

instance (Disposable a) => Disposable (Wrapper a) where ...

This says “if a is a type that is an instance of Disposable, then the type Wrapper a is also an instance of Disposable”. In C#, I’d like to see something like this:

class Wrapper<T>
    : IComparable<Wrapper<T>>
        when T : IComparable<T> //this is the syntax extension
{
    //...
}

And then in any methods that the interface specifies (in this case int Compare(Wrapper<T> other)), it is assumed that for any object of type T you have access to all the methods that T has when it implements IComparable<T>.

At the moment the best you can do is to always implement IComparable<Wrapper<T>> and just throw a runtime exception when T doesn’t implement IComparable<T>, which isn’t very nice.

A simple idea, but it would add great power to C#.

After some (very quick) research I haven’t found anything that suggests anyone else has attempted to get this into C#, but there is JavaGI (see section 2.4 for the equivalent of what this post talks about) where the authors have extended Java to deal with what they term ‘generalized interfaces’—a concept which alters Java (rather radically) to make interface implementations similar to Haskell’s typeclasses.

main = (putStr . ap (++) show) "main = (putStr . ap (++) show) "

Edit: I have found it somewhere else Here, to be exact.

Bignum support

In Scheme and Common Lisp, by default you can’t overflow an integer…

Prelude> fac n = product [2..n]
Prelude> fac 100
933262154439441526816992388562667004907159682643816214685929638952175999932299156089414639761565182862536979208272237582
51185210916864000000000000000000000000

In Common Lisp, you can force your code to use fixed-size numbers (fixnums) for efficiency…

Prelude> let fac n = product [2..n] :: Int
Prelude> fac 17
-288522240

Ruby and Python, by default, treat language-integers as logical-integers. Java, C, C++, and Perl don’t.

Prelude> :t 15
15 :: (Num t) => t

Optional type declarations

[Common Lisp] allows, but does not require, type declarations.

Prelude> let doubleApply f x = f (f x)
Prelude> :t doubleApply
doubleApply :: (t -> t) -> t -> t

The compiler can also use type declarations to perform compile-time typechecking. (Sadly, it isn’t required to do this.)

Prelude> doubleApply 3 doubleApply
 
<interactive>:1:12:
    No instance for (Num (((t -> t) -> t -> t) -> (t -> t) -> t -> t))
      arising from the literal `3' at <interactive>:1:12

Tail recursion

Proper (unbounded) tail-recursion (Scheme.)

Prelude> let loop = do { putStr "."; loop }
Prelude> loop
......................................
......................................
[etc]
......................................
..........................Interrupted.

(But see the Haskell wiki. Laziness introduces a small trick here.)

Functions which depend upon the type of multiple arguments

Methods with multiple-dispatch (Common Lisp.)

(This doesn’t exactly match conceptually because Haskell doesn’t have nominal overloading.)

multi Nothing Nothing = 0
multi (Just x) (Just y) = x + y
multi Nothing (Just y) = y
multi (Just x) Nothing = x

Fast implementations

Some Lisp implementations are fast.

Haskell #7 on the Great Language Shootout (in terms of speed), and getting faster by the day as the backend of GHC is rewritten.

Syntactic Simplicity

Syntactic simplicity (Scheme.)

The Haskell CFG.

Or perhaps Python is another incarnation of syntactic simplicity.

The Layout Rule

First of all we have to have an idea of what the numbers look like. We want to simulate this in XSL:

data Nat = Zero | Succ Nat

Using XSD we can have something like this:

<complexType name="Nat">
    <choice>
        <element name="Zero"/>
        <element name="Succ" type="Nat"/>
    </choice>
</complexType>

(Note that this is not tested, and probably wrong!)

However, since I don’t have an XSLT processor on hand that can deal with XSD types, I had to drop this course of action (although I still used the introduced type for the data). The type gives data like this:

<Succ><Succ><Zero/></Succ></Succ>

…in this case standing for the number two. Now the idea of addition on these numbers is given by a function:

add Zero y = y
add (Succ x) y = Succ (add x y)

Since this function involves pattern-matching it is an ideal candidate for implementation in XSLT, and it looks something like this:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" 
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 
	<!-- apply the add function to two numbers -->
	<xsl:template name="add" match="add">
		<xsl:apply-templates mode="adder" select="*[1]">
			<xsl:with-param name="rest" select="*[2]"/>
		</xsl:apply-templates>
	</xsl:template>
 
	<!-- add Succ(x) y = Succ(add x y) -->
	<xsl:template name="addSucc" mode="adder" match="Succ">
		<xsl:param name="rest"/>
		<Succ>
			<xsl:apply-templates mode="adder">
				<xsl:with-param name="rest" select="$rest"/>
			</xsl:apply-templates>
		</Succ>
	</xsl:template>
 
	<!-- add Zero y = y -->
	<xsl:template name="addZero" mode="adder" match="Zero">
		<xsl:param name="rest"/>
		<xsl:copy-of select="$rest"/>
	</xsl:template>
 
</xsl:stylesheet>

The first template matches any occurances of <add/>, and calls the real ‘add’ function on them. It passes the second number as a parameter, while the first is what the function actually recurses on—exactly as in the Haskell example. The rest proceeds similarly.

You can test it yourself by using the W3C online XSLT processor, which uses my processor and some sample input data. It should work for any similar data that you can throw at it as well. In this case it is adding 2 + 2 and coming up with 4, as we expect

Now, a functor is—according to higher sources—something which takes any type to a type . Let’s see how we can accomplish that with our functor, using Haskell code:

data MyFunctor a = Construct a

This is quite simply a type constructor for our functor for any type . For example, we can use it on the Integer type as Construct Integer, which will give us the type MyFunctor Integer.

That’s half the definition of a functor. The other half is that it also converts any function to a function . This means for any function, it converts that function to work on stuff “inside” the functor.

In terms of our Haskell code, this means that we have a function that acts on another function to modify it. This function is usually called “fmap” (for “functor map” or something similar). This function has the type:

fmap :: (a -> b) -> (MyFunctor a -> MyFunctor b)

It takes any function to . Now we can define the function itself. This is the part which will be the most unique for our functor, and we can define it how we like. For our functor, this isn’t going to be very interesting:

fmap f (MyFunctor a) = MyFunctor (f a)

We unwrap the object from our MyFunctor, apply the function , then wrap it back up.

Now there are two laws that must satisfy in order for this to be a “real” functor. In Haskell these are:

(fmap f) . (fmap g) == fmap (f . g)
id . fmap == fmap . id == fmap

These can’t be enforced from “within” Haskell, but you must keep them in mind when writing your functor. If you’d like, you can check that these laws hold for our MyFunctor defined above.

Now we have our entire functor:

data MyFunctor a = Construct a
fmap f (MyFunctor a) = MyFunctor (f a)

…and in fact, Haskell has a built-in class for functors:

class Functor f where
  fmap :: (a -> b) -> (f a -> f b)

We can declare our MyFunctor to be a Functor like this:

instance Functor MyFunctor where
  fmap f (MyFunctor a) = MyFunctor (f a)

…which still doesn’t seem very interesting.

An interesting functor is . This is defined as:

data List a = Empty | Cons a (List a)
 
instance Functor List where
  fmap f (Cons first rest) = Cons (f first) (fmap f rest)
  fmap f Empty = Empty

…wherein the functor takes a type to a list of , and is the familiar map list function, only expressed in a functor context.

A Comparison

Firstly, I’ll show a short comparison between some Java code and some code from a language that doesn’t have NullPointerExceptions, but does have something that allows you to accomplish anything you might want to do with null pointers.

This Java code:

AnObject thing = someMethod();

is equivalent to the following Haskell:

let thing = Maybe AnObject

That is, every Java object reference is possibly null. What this means is that (using the type notation mentioned in earlier posts) every Java reference has the type:

(Every object reference for an object is either to ‘null’ (AKA ‘unit’ or ‘singleton’, or the object itself.)

… and when you’re dealing with so many objects—Java is, after all, an object-oriented language, so (almost) everything is an object—this simple little “+1″ gets lost easily.

Why so bad?

This wouldn’t seem to be such a big deal, but when contrasted against Java’s penchant for making things (sometimes painfully) explicit—think public static void main (string argh!)—a small, implicit item like this is easily overlooked. Conversely, one might say that the verbosity of Java increases the cognitive load of understanding its code; thus helping the chances of a small mistake like this sneaking through.

Either way, I think I can say that this is objectively A Bad Thing. C# has gone some of the way to a ‘correct’ solution in its introduction of possibly-null primitives, which are explicitly marked with a ? (such as int?), but unfortunately this hasn’t been ‘back-ported’ to the rest of the language. If one were able to force between possibly-null and definitely-not-null references in C#—perhaps by the analogous object? vs. object, this would help to reduce the number of errors ‘hidden’ by the syntax.

data Surreal = Zero | Plus Surreal | Minus Surreal

… which also seems very similar to the usual construction for natural numbers:

data Nat = Zero | Succ Nat

(I noted this in a comment on the original post.)

This got me thinking about how this construction (the sign-expanded version) mapped onto the original construction of the surreal numbers, which uses a left and a right set:

data Surreal = Surreal [Surr] [Surr]

These types translate to the following (in order of appearance):

This last type is suggestive for the natural numbers as well, and of course there is a analogous construction for the naturals:

…which is a set-based construction for the naturals. (Note that in the above formulas multisets are used instead of true sets, but I don’t think this makes a difference in this context.)

Of course this raises the question of what is possible with three sets (or a 3-enum (3-num?)). More on this in another post.

If we take integration to be the logical inverse of differentiation (which is defined in the article as ‘removal of an element from a type’, then we have integration as the addition of an element to a type.

For example, take the type of an array of 3 elements of the same type:

F[T] = T3

If we are to add an element to this array, we want to integrate it. This gives us:

F[T] = T4 ÷ 4

Clearly this is not what we expected. If we want to add an element to a 3-element array, we are expecting a 4-element array. However, if we look again at the above result, we can reinterpret it as telling us something: the mere integration of the type is not supplying the expected result; and if we interpret division as ‘without’ (just as addition is ‘or’ and multiplication is ‘and’, then we can understand the result obtained.

What this is in fact telling us is that we need to provide another parameter (an enum in this case), which will give us the required result. In other words (if we do it backwards…:

F[T] = T4 -- wanted
F'[T] = 4.T3 -- needed

In order to obtain a 4-element array, we need a 3-element array and a 4-enum. This makes sense, because if we have a 3-element array:

x y z

…and we wish to add an element ‘w’ to it, we need to know where to insert it. The number of insertion points is:

(1) x (2) y (3) z (4)

…and thus we need to supply a 4-enum to specify exactly where we want the insertion to take place. Once we have done this, we obtain our 4-element array with no ‘missing’ parameters.

comp [] _ = False
comp _ [] = True
comp (_s) (_:ys) = comp xs ys

foldr getCounts (0,0,0) $ map classify $ concat [[20..30], [490..500], [8120..8130]]

In order to understand what is happening here, one needs to start from the RHS of the expression and then work backwards—much the same as in this snippet of APL:

(∼R∈R∘.×R)/R←1↓⍳R

(See the Wikipedia article on APL for what is happening here.)

In both cases, the code is operating upon a common parameter which is modified by each function in turn. Similarly, there seems to be a rather high-level isomorphism between these kinds of functional code and the concept of Unix pipes:

sort file | uniq -c | sort -n

…with the difference being that in this case, the common parameter is being passed from the left instead of from the right.

Now, in Haskell I can define a new “pipe” operator (| is already taken):

(~>) :: a -> (a->b) -> b
(~>) x f = f x

Which just has the reverse signature to the $ operator, and:

foldr getCounts (0,0,0) $ map classify $ concat [[20..30], [490..500], [8120..8130]]

…becomes…

[[20..30], [490..500], [8120..8130]] ~> concat ~> map classify ~> foldr getCounts (0,0,0)

…which makes it a bit easier to read, at least in my (used-to-Linux) eyes.