// Write the first character:
Console.WriteLine(s[0]);
 
// Reverse the string:
var backwards = s.ToArray();
Array.Reverse(backwards);
Console.WriteLine(new string(backwards));
 
// List the characters, separated by commas:
Console.WriteLine(string.Join(", ", s.ToArray()));

All of these are potential bugs.

Just set the following and you’ll get some nice output:

string s = "𠂔";

�
��
�,�

There are other problems as well:

string A = "𝔄"; // MATHEMATICAL FRAKTUR CAPITAL A, if you can't see it
Console.WriteLine(char.IsUpper(A[0])); // False
Console.WriteLine(char.IsUpper(A[1])); // False
Console.WriteLine(char.IsUpper(A,0)); // True - this is the correct way

Why is this?

Console.WriteLine(char.GetUnicodeCategory(A[0])); // Surrogate
Console.WriteLine(char.GetUnicodeCategory(A[1])); // Surrogate
Console.WriteLine(char.GetUnicodeCategory(A,0)); // UppercaseLetter

Ah.

The UCS-2 Hangover

These are all symptoms of ‘migrated’ libraries, originally developed for UCS-2 (in the early days of Unicode), and now touted as UTF-16.

Whereas UCS-2 was a fixed-width, 16-bit format, UTF-16 is a variable-width format which uses pairs of ‘surrogate’ 16-bit units to support characters that UCS-2 does not. These are the characters outside the Basic Multilingual Plane (BMP), sometimes termed the ‘astral characters’.

Libraries with char as a 16-bit quantity are living a lie. They live in a delusional world of fixed-width characters, where they let you believe you can index, substring, and play around at will, whereas:

• You cannot select arbitrary substrings — you might get half a character included.
• You cannot index into the string arbitrarily — you might select half a character.
• You cannot treat a string as a bunch of isolated characters — the surrogate pairs will get broken up.

So?

The general response seems to be “it doesn’t matter, most of the stuff I deal with is in the BMP, so I don’t care”.

A simple example of the problem with this: if any of these ‘orphan surrogates’ gets into an XML document, you have an application-stopping bug on your hands. (Just ask LINQPad, which refuses to run any of the examples in the first block of code. Luckily it has good exception handling.) A quick Google search shows that many developers are already encountering this problem.

All of the latest CJK extensions (B, C, D) also live in the Supplementary Ideographic Plane (SIP, the second non-BMP plane). I haven’t really been able to find out just how many of these characters are actually used in any real sense, but you’re going to encounter them eventually (and recent additions to Unicode might start turning up).

There’s a hole in your abstraction

The problem with UCS-2–flavoured UTF-16 is that it hands all the work of handling surrogate characters correctly over to the developer.

Underlining the third character in a string sounds like a simple task. But, if someone slips in a non-BMP character, then…

foreach (var s in new[] {"0123456789", "〇一二三四五六七八九", "𝟘123456789"})
{
	Console.WriteLine(s.Substring(0,2) + "<span style='border:2px #FF4500 solid'>" + s[2] + "</span>" + s.Substring(3));
}

0123456789
〇一二三四五六七八九
𝟘123456789

…and now the developer has to write their own linear-time indexing functions to actually find the character they need. This is not a good abstraction.

From here

Once you give up the idea that you can safely treat UTF-16 as a fixed-width format, it really doesn’t have anything going for it. (Perhaps this is why the idea is so firmly embedded — elements of cognitive dissonance.)

Here’s a quick (mainly biased) diagram:

A Comparison

Firstly, I’ll show a short comparison between some Java code and some code from a language that doesn’t have NullPointerExceptions, but does have something that allows you to accomplish anything you might want to do with null pointers.

This Java code:

AnObject thing = someMethod();

is equivalent to the following Haskell:

let thing = Maybe AnObject

That is, every Java object reference is possibly null. What this means is that (using the type notation mentioned in earlier posts) every Java reference has the type:

(Every object reference for an object is either to ‘null’ (AKA ‘unit’ or ‘singleton’, or the object itself.)

… and when you’re dealing with so many objects—Java is, after all, an object-oriented language, so (almost) everything is an object—this simple little “+1″ gets lost easily.

Why so bad?

This wouldn’t seem to be such a big deal, but when contrasted against Java’s penchant for making things (sometimes painfully) explicit—think public static void main (string argh!)—a small, implicit item like this is easily overlooked. Conversely, one might say that the verbosity of Java increases the cognitive load of understanding its code; thus helping the chances of a small mistake like this sneaking through.

Either way, I think I can say that this is objectively A Bad Thing. C# has gone some of the way to a ‘correct’ solution in its introduction of possibly-null primitives, which are explicitly marked with a ? (such as int?), but unfortunately this hasn’t been ‘back-ported’ to the rest of the language. If one were able to force between possibly-null and definitely-not-null references in C#—perhaps by the analogous object? vs. object, this would help to reduce the number of errors ‘hidden’ by the syntax.

Intrigued at what I first thought was a joke, I downloaded one of the proudly-presented documents on the front page: Understanding the BPMN-XPDL-BPEL value chain. I chose it because of the three as-yet-unknown acronyms in the title… and I wasn’t disappointed.

I present you with the following gem, lifted from one paragraph of the article:

Several BPM engines are able to run XPDL natively, which allows run-time modification and process migration to be readily supported. Where these processes focus on broader-scope collaboration among people, they can remain within XPDL/BPMN. Where pieces are decomposed into system-to-system interactions, these can be translated to BPEL for transmission to an EAI-oriented BPM engine. These are three very different and very compatible roles. But that’s the nature of the value chain—BPEL and XPDL are entirely different things for entirely different purposes.

I suppose the warning sign should have been their slogan:

Process Thought Leadership™

Don’t use this method any more! I have a much easier way to do it in a more recent post.

But for posterity…

First install resolvconf and pdnsd: sudo apt-get install pdnsd resolvconf. Resolvconf handles the messy interaction between different programs fighting over the /etc/resolv.conf file, and pdnsd provides a persistent DNS daemon, which will cache the DNS lookups on your machine in order to make browsing faster.

Next, you just need to edit pdnsd’s configuration file, /etc/pdnsd.conf as root. Edit it to look like this:

global {
        perm_cache=512;
        cache_dir="/var/cache/pdnsd";
        max_ttl=604800;
        run_as="pdnsd";
        paranoid=on;
#       next setting allows ppp/ip-up update the name servers -- ABa / 20040213
        status_ctl=on;
        server_port=53;
        server_ip="127.0.0.1";
}

#Edit these to be your own servers if wished:
server {
        label = "OpenDNS";
        ip=208.67.222.222,208.67.220.220;
        proxy_only=on;
        timeout=10;
}

# if you installed resolvconf, and status_ctl=on
server {
    label="resolvconf";
}

source {
        ttl=86400;
        owner="localhost.";
#       serve_aliases=on;
        file="/etc/hosts";
}

And finally, restart pdnsd with sudo /etc/init.d/pdnsd restart to use the new settings. In the example above, I've used OpenDNS's servers, but you can change these to whatever you want. As far as actual results go:

~$ dig example.com

; <<>> DiG 9.3.2 <<>> example.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17447
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;example.com. IN A

;; ANSWER SECTION:
example.com. 172800 IN A 192.0.34.166

;; Query time: 513 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Oct 5 15:24:19 2006
;; MSG SIZE rcvd: 45

$ dig example.com

; <<>> DiG 9.3.2 <<>> example.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19338
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;example.com. IN A

;; ANSWER SECTION:
example.com. 172799 IN A 192.0.34.166

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Oct 5 15:24:20 2006
;; MSG SIZE rcvd: 45


Not only does Google search fail to recognize that “it’s” is a word, it also ignores the quote marks, searching for the phrase as individual words.