The CIA is sponsoring anthropologists and thus there is a lot of discussion about whether that is acceptable.

How to combine tags with facets

Information architects (like me) tend to say we should combine different methods of increasing findability. In this post I'll explain one way of combining user generated metadata with editorial metadata, and more specifically, tags with facets. I'm not gonna do definitions and such (ok, I heard that collective sigh of relief!). The big advantage of adding metadata in the form of facets is that we know how to make an easy to use interface for facets. Check Flamenco. We could try adding ontology-like metadata, but we don't know yet how to create good interfaces with those. Let's look at your standard tagcloud page: If we added a few facets to this it could look something like this (quick and nasty mockup): Notice that the facets I came up with are relatively techie, reflecting delicious' userbase. If we dig deeper into the data, we'll probably find more facets like event, time and such. Other tagclouds might need different facets. Also notice that this new arrangement makes it much easier for users to find something if they're looking for a specific item. It makes the tagcloud more browsable. And something I just noticed myself: we can bring up less popular items in less popular facets to the homepage. The big difference in browsability (arg!) happens when a user moves into the tagcloud. When looking at a tag, they can refine their search by the various other facets (as is standard in faceted browsing systems). Check epinions.com for a nice example - if I had all day I'd mock it up. This makes it easy for a user to refine a search in a really large tagcloud. How do we scale the assigning of tags to facets? Simple: editors create facets (it takes some consideration to create a useful and valid facet), and users assign tags to facets. We can let users assign tags because facets are mutually exclusive, that is, tags can only logically be classified in 1 facet, not in multiple ones. So there should be little confusion as to what facet a tag should belong to (apart from the expected tags with more than 1 meaning). The mutual exclusivity comes from the editorial selection of facets: we choose our facets so that they are (at least pretty much) mutually exclusive. I expect a small number of facets (between 5 and 10) to be quite useful. Asking users to assign tags to facets shouldn't be too hard. You can do it when they add a tag, or when they're browsing a tag that hasn't been assigned yet. The question can look something like this: Again, because of the mutual exclusivity requirement for facets, disagreement on what facet a tag belongs to should be limited. So I would just let the system assume that each tag belongs to one facet. So what do we know so far?

• We can let editors come up with a few useful facets, and let users assign tags to facets. This should work because of the mutual exclusivity requirement of facets: the hard work (creating facets) is done by editors, the easier work (assigning facets) is distributed (it is easy but a lot MORE work) amongst our users. This should work, and should scale.
• Once we have facets for some of our tags, browsing is made a lot easier for our users. The browsing interface is well known and tested, and should just work.
• This is good because we have now found a way to combine the strenghts of editors (ie., spend a whole afternoon considering what a certain facet should be) with the strength of user generated metadata (ie., you get lots of it if the decision is easy).

And what don't we know?

• Will users indeed find it easy to classify tags in facets without much disagreement? (They should if we create our facets following the exclusivity requirement.)

Comments?

Joi Ito's Web: Technorati.jp beta launched

I'm spending a week in Belgium, at my moms house, lazing about. French fries, mmm. With mayo. Slowly gearing up to work on my global IA research.

Exactly 1 year ago me and Jay started the videoblogging mailing list. In the past year, it has become an amazing community. 100s of videoblogs have grown up around it. Happy birthday.

IA as a competitive advantage

Wired News: The Beeb Shall Inherit the Earth: "America's entertainment industry is committing slow, spectacular suicide, while one of Europe's biggest broadcasters -- the BBC -- is rushing headlong to the future, embracing innovation rather than fighting it."

The BBC is indeed quietly getting waay ahead of most other media companies.

I often talk about IA as a competitive advantage: the BBC gets that. (I shouldn't define IA this broadly, but hey). They have consistent URI's for every program ever made (working on it at least), they have API's in the new BBC Backstage, they embrace RSS and let users play with their data. In other words: they architect their information with an eye towards the future. Long term metadata. Good URI's. Open to users.

What will happen (and this is where the "IA as a competitive advantage" comes in), is that its competition (CNN, ...) will wake up one day and see the advantage they have built. The CEO will proclaim: "Let's build the same". They'll try to find vendors. The CEO will assume she can buy this.

But the way the beeb is architecting its information is as much a cultural achievement as a technical one. It takes dedication, love and years of time to make a company open up its information, even if it's in the statutes that the company is there to serve the public, as in the beebs' case. IA is hard work.

The big advantage the BBC has had is that it's attitude towards information hasn't been lead by vendor pitches, but by passionate and talented people for years. You can't just buy that, or expect to catch up with that in a year or two. That's IA as a competitive advantage.

Akshaya, a project in India that I visited and blogged about extensively, wins the ARS Electronica PRIX for digital communities (via Dina).

Language Log: Pass the hat.: Google searches with accents. Mmm.

ITworld.com - Filing system taxonomy blues: "All those pieces of paper. They all need to be filed somewhere. But where? Is this piece of paper best filed under 'insurance', or 'house' or 'Acme Insurances Inc.' or 'bills'? So many options, so many incorrect taxonomies to choose from. So much desire to find the one that is perfect. I should know better of course. There is no perfect taxonomy for a filing system." (via Jon Udell)

SiteLines - Ideas About Web Searching: May 2005 Archives: "In a March 29 2005 paper, Ira Machefsky and John Fernandez of search engine Accoona, clue us in to some key differences between searchers in China and the US. The authors compared search terms used in the US and Chinese versions of Accoona. Whereas US searchers focus on news, gossip, and entertainment, Chinese searches show a strong focus on business information, particularly manufacturing."

I've been interested in finding out differences in search habits in different cultures. Any other pointers?

The RSS wars

Here's what might happen: Apple, Google, Yahoo et al will start expanding RSS (with namespaces) or creating their own. Their tools will use these namespaces, so developers will start supporting them as well (to get their stuff into iTunes and such). So brace for the RSS wars.

I finally approved a bunch of recent comments (the comment spam means no comments go through unapproved these days), and my Enterprise search still a technology conversation post has a lot of good ones. Check it and leave yours - I promise to moderate faster!

Poorbuthappy guide to India | Holy men, immodium and technology. I'm slowly starting writing at my India discussion site again.

Anthropology at work: "Brenda runs through the mundane minutiae of her daily life: how her three sons like her to iron their T-shirts and tracksuits, but hate it when she gets them mixed up - so she has created a labelling system to tell the identical, perfectly pressed T-shirts apart."

antropologi.info: "according to dance anthropologist Judith Lynne Hanna, there may be as many dance languages as humanity's 6,000-plus verbal languages."

The Collier Classification System Of Very Small Objects. A kind of art project. All Caps Titles With Words Like Everything or Very are Cool.

The Application of Weblike Design to Data: Designing Data for Reuse: "Every episode uniquely identifiable and addressable, forever!"

Simple yet long term useful metadata is the best metadata. I am often surprised by how ambitious some people are when trying to do enterprise-wide metadata: "We'll collect dozens of fields of metadata for each piece of content!" Yeah right. The BBC didn't make that mistake: simplicity first, yet great ambition at the same time. Brilliant.

Drupal is a great CMS but it was always kinda hard to follow what was going on with it. So finally, a Drupal newsletter! With an unfortunate name though. Drupal Drops

apophenia: categorization negatively affects memory: "I do a little mental happy dance every time I del.icio.us a link and I say "YES! I *get* to add as many tags as I want and no one can stop me, nyahhh nyahhh!" (really, I do)." (in the comments)

I use Bloglines, and I read about 300 feeds. I wish it would make it easier to remove feeds from my list, since there are quite a lot I don't follow anymore.

I find myself writing series on this blog fairly often: related articles that can be read together. The problem is that blogging software doesn't provide an easy way to create navigation to make sure the individual posts of a series hang together. I have been using numbered bulleted lists of links at the beginning of a post, as in my Akshaya project series, or the series about global IA.

I wish Wordpress (my blog software) would provide some way of indicating a series and then automatically generate navigation for them. In general, I wish blogging software would let me bring much more structure in my blog entries, when needed.

Apart from the fact that it doesn't work on Firefox, yet, Yahoo's new PhotoMailis brilliant. Let the email wars begin! I think Yahoo might seduce me back to their service, after years of neglect.

By the way, if you hadn't noticed, Yahoo is totally the new hip company with cool new products. They've been hiring IA's like crazy, too.

If you are moving in NYC and need someone to help you carry that heavy sofa and stuff upstairs/downstairs, Greg at 917 257 23 17 is experienced, very helpful and has super reasonable rates. He helped me move and I can heartily recommend him.

The Dewey Decimal people are trying to figure out how to classify graphic novels (ie. comic books). Discussion here. The kinds of questions they're trying to answer are:

- should graphic novels go in the 700s (arts) or the 800s (literature)? (Answer: like comic books, 700s)

- Should they be lumped together with comic books? (Answer: yes - "separating graphic novels would be difficult for classifiers to do consistently")

- How do you distinguish between comics, graphic novels, ...? Answer: we can't. ("We have tentatively decided to treat everything from single-frame caricatures to three-frame newspaper comic strips to comic books to graphic novels all in the same way. Although this is a broad range of material, we have found no good places to break the continuum so as to separate the material usefully into different categories.")

- How do we subdivide? Answer: by country of writer/artist. This makes sense, because in the world of comics, styles kind of follow geographical boundaries (the Belgian/French school, the US school, the Japanese school). How long this will last I'm not sure about. Meanwhile, the Dewey editors are considering subarranging by country of original publication, rather than country of artist or writer. The rationale is that artists and writers of different nationalities may collaborate on the same work, and a single artist or writer may contribute to works originally published in different countries, but the artists and writers will aim for the style of the country in which the work is to be published. Mmm...

Meanwhile, on the international front, Discussions are underway on a new Arabic translation of DDC 22. Right now, they are considering an additional optional arrangement in the famously biased 200 Religion schedule. This proposal will be discussed at the ALA Annual Conference in June 2005, and at the IFLA Conference in August 2005.

Broadband Wi-Fi Spreads in Latin America

Oh, that Guru Meditation Error!

Enterprise search still a technology conversation

In short, best bets (where an editor can select the top results for certain search queries) is seen by many information professionals as about the cheapest and best way to improve your search engine, but the enterprise search industry doesn't have much of a clue. Many enterprise search products don't explicitly support this. More generalized, most companies seem to think of search as a technology problem, whereas most of the consultants and experts understand the importance of adding people to the mix. In 2003, I started an article at Onlamp like this: "A useful search engine is more than a search algorithm. This article explains how to create a search query analysis tool, a best bets feature, and a basic controlled vocabulary." The idea was to write for the techies who are building the tools about what we, information architects, think are the things missing from most search engines. Onlamp is O'Reilly's publication for open source hackers, and I was on a mission to spread the word about IA (also to other groups, like designers). My point was: there are easy things you can add to your search engine that let humans add value to it, like best bets, or a search log analysis tool. It's not rocket science - if I could write a techie how-to article, the search vendors should be able to figure this out. Last week, at the 2005 Enterprise search summit, I did a little unscientific survey with the vendors about best bets. I asked them if they had such a functionality in their product (I had to explain it to most), and what they called it. The results were in line with my overall impression of enterprise search. Most of the products work like this:

1. Spider content and rank
2. Auto-generate and auto-populate taxonomies to add value to search

Notice the absence of humans in that process. The control panels of the products tend to contain a section with sysadmin-like functionality, and some analytics (most allow you to see what search queries people have been using). Most of them assume that the person using it has been trained to use this tool. There is surprisingly little functionality aimed at the person whose job it might be to tune the engine with best bets and such. The people I spoke with who actually do that job, use things like Perl scripts or open source software to analyse search queries. (For example, I was told Googlebox doesn't handle logging multilingual search queries (it searches fine), so one person used Webalizer instead.) When I asked the best bets question ("does your product do best bets, defined as ..."), even after explaining the functionality, I got surprisingly many blank stares. Best whats? Why would you want to do that? Some products have best bets, but the closest a lot of them could come was to say you could create rules to improve the result of certain documents. That's like saying, sure, you can do HTML with Word. In theory perhaps, but it's not really useful. Here is an incomplete list of products that do best bets, and what they call it. This is an unscientific and uncomplete survey, which may have mistakes in it. Don't use it to judge a particular product, use it to get a sense of the field.

• Autonomy: you can kinda do them through rules.
• BA-insight: no best bets Yes, through SharePoint.
• FAST: yes (although I have doubts here).
• IBM: yes, they're called Quick Links.
• ISYS: not really.
• Mondosoft: yes, they're called Top Hits.
• Open Text: it's coming up in their next release.
• SER Solutions: no.
• Verity: yes, calls them Sponsored Links.
• Vivisimo: yes, kind of.

I didn't have time to ask the other vendors - feel free to add in the comments. By the way, to work well with users, best bets should appear in-line with the other search results, not separate from them. If I was to do a more complete survey, I'd add that in as a criteria, together with an easy to use admin interface, CV functionality and an easy to use search analysis tool that includes analysis of suddenly popular queries.

This is probably the event that we'll look back at when we think "when did Google loose it's sexyness again?"

Yahoo is the new darling of the technorati (the people, not the company). The circle is round.

I just got to Belgium (for 2 weeks, with a workshop in Edinburgh thrown in). Long flight, crazy jetlag. And lots of paperwork to get done.

Happy birthday Lou!

local6.com - News - Ohio Wal-Mart Caters To Amish: "The store has an expanded parking lot that includes 37 hitching posts for horse-drawn carriages.

Also, the store is stocked with blocks of ice instead of crushed ice and fabrics for clothes to be made at home."

NowPublic.com is a community site to develop citizen news stories.

An Argentinian reference to my IA talk in Madrid.

JD Lassica, the guy behind Ourmedia, is releasing a mini version of his new book at Darknet

The Garamond Agency: "The Garamond Agency represents authors of non-fiction exclusively. Our clients are academics, scholars, journalists, business people, and writers whose books make important ideas accessible to a wide audience of readers."

Linux radio show - LugRadio: the first Lugradio podcast has an audio review of my book. I'm posting this before hearing it, so I don't know if it's positive or negative. Whatever, I got the Japanese translation (just out) the other day, so I'm happy. Anything in Japanese is cool.

Today, everyone's a librarian. A vlog about categories: "We had a huge debate in our family wether we should establish certain categories."

Joho the Blog: Miscellaneous innovation: "To be miscellaneous is to be placed next to things unlike you. Isn't that pretty much a condition for innovation?"

I am continually intrigued with the "everything else" category. (Check it on eBay!)

One of the few blogs about immigration in the US. I spoke with Mario, he's a good guy. Recommended if you need a laywer.

The French worry about Google writing the planet's history. Yes, search is political.

Y! Music Engine: Yahoo is trying to replace Windows Media player, iTunes and the lot. This is big news in the fucked up mediaplayer landscape. Here's the inside story (you need a Yahoo 360 account to read it)

An article in the Guardian about MySpace...: "It's really nice to be called a Social Software expert again - particularly after slightly losing my identity inside the monstrous belly of the BBC. I have felt a bit disconnected from the social software community over the last year or so in that while I feel I still have as much to say, I never really get the chance to express it in public. And that makes you invisible and eventually redundant. I'm going to have to try and spend a little more time engaging in the debates of the day from now on, I think."

I know a lot of IA's who work in small and large companies and have a lot of expertise to share, but don't. I want to encourage them. Write a blog. Or a book. The blog is easier, and it gets you almost as much recognition as a book does, so start with that. It's more fun too. But if you always wanted to write a book, that's cool too. If I've learnt one thing from my time online, it's that conversation is what it's all about. I'll shut up now.

The Disney Muppets: why that just isn't funny. I don't think Disney can bring the muppets back. Something about large corporations - the values of the leaders trickle down, and if those values are cold, hard cash, creativity suffers. I watch a lot of Disney movies (the cartoon ones), including the recent ones. They had a few good ones in the 90s (Alladin, The little mermaid), but mostly they've been dissapointing (The lion king was the best of the bad ones), and the latest 5 years they haven't done 1 decent one (I don't consider the recent brilliant Pixar movies (Toy Story, Finding Nemo) Disney movies. Disney just did the distribution.). Disney will probably fuck up the Muppets, and kill millions of cherised childhood memories along the way. Ah.

Maria is writing in Spanish about New York. Check it out.

Folksonomies: "There only need be a small connecting layer between folksonomies, classic AI and classic IA for the end result to be very, very compelling on many levels." Exactly right.

You remember those Spanish innovative UX guys I spoke about. Javier Canada let me know they are beta-testing La Coctelera, a new free blogging service with a pretty innovative interface. It's light on features, but in that good way that makes it feel easy and sufficient. It's Ruby on Rails and Ajax and all those buzzwords (who cares?), but one of the big, visible innovations is their really easy navigation: to change something, click on the area of the screen that corresponds with it:

Another review in English.

Photoethnography.com: Fieldnotes: My interview fieldkit. And: "I have read of people who go to really dirt-poor parts of the globe and pack a Canon battery-powered 4x6 dye-sub printer, so they can make prints on the spot and give them to the people they photograph as a thank-you gift. In many countries, less blase about photography than our pampered industrial societies, these would become prized family heirlooms."

Photoethnography.com: Fieldnotes: My interview fieldkit: doing foto ethnography on the road.

Waw, in March I served 1,049,751 pages on poorbuthappy.com. That's a million! I wonder how much of th upsurge in traffic the recent months is due to spammers.

Stupid fucking academic burocrats. In order to speak to a researcher to help her with her dissertation, I have to sign and mail or fax a disclosure form to them. I can't just email an "I agree". I am moving and don't have access to a printer. What century is this?