Emerging Structure or Intelligent Design

Esther Dyson is one of the thought giants of the Internet. She thinks on such grand scales and puts them in such clear terms that it's a joy to read her stuff and hear her talk.

Her post about MetaWeb's Freebase is no different. I posted about Freebase earlier and I offhandedly mused that it was counter intuitive to the philosophy of Web 2.0 whereby we are creating an emerging structure of tagged/microformated/syndicated/user-centric/open content.

Esther has written a far more eloquent post about it. Although the creators of MetaWeb are her friends and she is decidedly for their approach she takes the time to put their efforts in context and frame the debate of 'Emerging Structure' (i.e. structure will emerge when most sites provide structured content and algorithms sort it out) vs. 'Intelligent Design' (where the structure comes from a given database architecture).

She writes:

"This all reflects a fundamental if still incoherent debate. There's one school of thought that says that if you just collect enough data and throw enough algorithms at it, the inherent structure - and the understanding of that structure - will emerge. After all, that's what happens with human beings, though it takes a decade or more. (And in some people, the process even continues into old age.) The recent explosion of tagging is taken as evidence of this: With their tags, users are creating implicit relationships among online objects, and indeed, complex webs of relationships are emerging, with nodes, clusters and other rich structures. But the relationships themselves are poorly defined, other than strong or weak - and possibly, links made by my friends or by trusted authorities, vs. links created by anyone.

By contrast, the opposing point of view says we have to hand-design the relationships and structures - like the complex database schema about cars."

Like everything she writes, it is well worth a read.a