vanderwal.net

The Data Journalism Handbook is Available

The Data Journalism Handbook is finally available online and soon as the book Data Journalism Handbook - from Amazon or The Data Journalism Handbook - from O’Reilly, which is quite exciting. Why you ask?

In the October of 2010 the Guardian in the UK posted a Data Journalism How To Guide that was fantastic. This was a great resource not only for data journalists, but for anybody who has interest in finding, gathering, assessing, and doing something with the data that is shared found in the world around us. These skill are not the sort of thing that many of us grew up with nor learned in school, nor are taught in most schools today (that is another giant problem). This tutorial taught me a few things that have been of great benefit and filled in gaps I had in my tool bag that was still mostly rusty and built using the tool set I picked up in the mid-90s in grad school in public policy analysis.

In the Fall of 2011 at MozFest in London many data journalist and others of like mind got together to share their knowledge. Out of this gathering was the realization and starting point for the handbook. Journalists are not typically those who have the deep data skills, but if they can learn (not a huge mound to climb) and have it made sensible and relatively easy in bite sized chunk the journalists will be better off.

All of us can benefit from this book in our own hands. Getting to the basics of how gather and think through data and the questions around it, all the way through how to graphically display that data is incredibly beneficial. I know many people who have contributed to this Handbook and think the world of their contributions. Skimming through the version that is one the web I can quickly see this is going to be an essential reference for all, not just journalists, nor bloggers, but for everybody. This could and likely should be the book used in classes in high schools and university information essentials taught first semester first year.

January 2011 Books Read

My monthly list of books read is something I have had in mind for a long time. I was inspired by Matt Webb's book list which he was doing for a while years back. Not only is the sharing out with others helpful, but it also helps me finish reading a book.

Books read January 2001 with short summaries.

Shibumi: A Novel by Traviathan

A really good thriller set in Japan and Europe. Not only was the story good, but the details and a good cultural view of Japan during World War II. This book caught and held my attention early and I really enjoyed it.

Halting State by Charles Stross

This thriller set slightly in the future where MMORPGs start intertwingling with life. A bank robbery occurs in the game which starts the whole story rolling. The interplay and storyline between virtual games and physical life interwoven with its pervasive digital layers we depend on today is really well done.

Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers by Alexander Osterwalder and Yves Pigneur

Business Model Generation is a surprise gem in that I had heard very good things about it and a quick skim of it in a bookstore convinced me to pick it up. But, the design, layout, and thoughtful thinking of how it steps through the model for understanding and thinking through business models is nothing short of stellar. The stuffy, staid, and often broken world of business models got tipped on its ear through design and understanding that makes walking through creation of a business model a sane process, but also leads to rethinking existing models for whole organizations or parts. It is a great way to look to see where software and services can have a positive impact when mapping out an organizations model.

Smart Things: Ubiquitous Computing User Experience Design by Mike Kuniavsky

Smart Things is a fantastic walk through design considerations and methods for information interfaces for and streams from physical products. This book is very well thought out, well written and augmented with examples and very well produced. Not only is this a great book for designers, but for people working through ideation, iterations, and innovations for improving information use in, from, and with the world of things around us.

Good Night Sweet Six Apart

Odd how news of business transactions hits. Today I saw Six Apart was bought by VideoEgg (yes, Six Apart that makes TypePad and MovableType). This news made me a bit sad and nostalgic.

Movable Type is Old Skool

As the young people used to say (they were still young back then) Movable Type (MT) is OG. MT was far from the first blogging platform, as Blogger and others like Grey Matter (this influenced the blogging platform under this blog a lot). MT was started by the couple Ben and Mena Trott out of their house in the dust of the test boom bursting. It was started as a blogging platform and build to be a well thought through platform for others to use. By 2002 it had been launched user tested and was the first underpinnings of Boxes and Arrows Magazine (as deployed by Jay Allen).

Those were good days and MT was a solid product. MT turned in to Six Apart and offered a hosted platform, TypePad, which my Personal InfoCloud blog is hosted on. The became the first real blogging platform company and it grew into a solid company.

The Show Goes On

The announcements around the purchase state MT and TypePad will continue to be supported. The big change is a new owner, a new company name (SAY Media), and possibly a new direction. I've had a lot of friends work at Six Apart or work with them.

The days of sitting up late talking blogging platforms in the early days at SXSWi and how all of this was more than just something the "broken one's" did. We have seen WordPress surface out of another platform and thrive as have many other personal sharing social platforms that have not only changed the web, but have also largely changed the world for the better. MT and other blogging platforms gave nearly everybody a printing press with a wider reach around the globe than any personal printing press ever had. As well the readers or people with affinity to the ideas shared, or even trying to get a new or different view.

I look forward to my new TypePad overlords and hope they are as inspiring and gracious as the kind, smart, gentle, and caring folks at Six Apart always have been. I also have my own hand rolled blogging platform that sits under vanderwal.net, as well as my vanderwal on Tumblr blog, and all those other social platforms I share things within and on. Be well Six Apart as you move from a blogging platform company with a name on the door, to our collective blogged memories.

Social Design for the Enterprise Workshop in Washington, DC Area

I am finally bringing workshop to my home base, the Washington, DC area. I am putting on a my “Social Design for the Enterprise” half-day workshop on the afternoon of July 17th at Viget Labs (register from this prior link).

Yes, it is a Friday in the Summer in Washington, DC area. This is the filter to sort out who really wants to improve what they offer and how successful they want their products and solutions to be.

Past Attendees have Said...

“A few hours and a few hundred dollar saved us tens of thousands, if not well into six figures dollars of value through improving our understanding” (Global insurance company intranet director)

From an in-house workshop…
“We are only an hour in, can we stop? We need to get many more people here to hear this as we have been on the wrong path as an organization” (National consumer service provider)

“Can you let us know when you give this again as we need our [big consulting firm] here, they need to hear that this is the path and focus we need” (Fortune 100 company senior manager for collaboration platforms)

“In the last 15 minutes what you walked us through helped us understand a problem we have had for 2 years and a provided manner to think about it in a way we can finally move forward and solve it” (CEO social tool product company)

Is the Workshop Only for Designers?

No, the workshop is aimed at a broad audience. The focus of the workshop gets beyond the tools’ features and functionality to provide understanding of the other elements that make a giant difference in adoption, use, and value derived by people using and the system owners.

The workshop is for user experience designers (information architects, interaction designers, social interaction designers, etc.), developers, product managers, buyers, implementers, and those with social tools running already running.

Not Only for Enterprise

This workshop with address problems for designing social tools for much better adoption in the enterprise (in-house use in business, government, & non-profit), but web facing social tools.

The Workshop will Address…

Designing for social comfort requires understanding how people interact in a non-mediated environment and what realities that we know from that understanding must we include in our design and development for use and adoption of our digital social tools if we want optimal adoption and use.

• Tools do not need to be constrained by accepting the 1-9-90 myth.
• Understanding the social build order and how to use that to identify gaps that need design solutions
• Social comfort as a key component
• Matrix of Perception to better understanding who the use types are and how deeply the use the tool so to build to their needs and delivering much greater value for them, which leads to improved use and adoption
• Using the for elements for enterprise social tool success (as well as web facing) to better understand where and how to focus understanding gaps and needs for improvement.
• Ways user experience design can be implemented to increase adoption, use, and value
• How social design needs are different from Web 2.0 and what Web 2.0 could improve with this understanding

More info...

For more information and registration to to Viget Lab's Social Design for the Enterprise page.

I look forward to seeing you there.

Catching Up On Personal InfoCloud Blog Posts

Things here are a little quiet as I have been in writing mode as well as pitching new work. I have been blogging work related items over at Personal InfoCloud, but I am likely only going to be posting summaries of those pieces here from now on, rather than the full posts. I am doing this to concentrate work related posts, particularly on a platform that has commenting available. I am still running my own blogging tool here at vanderwal.net I wrote in 2001 and turned off the comments in 2006 after growing tired of dealing comment spam.

The following are recently posted over at Personal InfoCloud

SharePoint 2007: Gateway Drug to Enterprise Social Tools

SharePoint 2007: Gateway Drug to Enterprise Social Tools focusses on the myriad of discussions I have had with clients of mine, potential clients, and others from organizations sharing their views and frustrations with Microsoft SharePoint as a means to bring solid social software into the workplace. This post has been brewing for about two years and is now finally posted.

Optimizing Tagging UI for People & Search

Optimizing Tagging UI for People and Search focuses on the lessons learned and usability research myself and others have done on the various input interfaces for tagging, particularly tagging with using multi-term tags (tags with more than one word). The popular tools have inhibited adoption of tagging with poor tagging interaction design and poor patterns for humans entering tags that make sense to themselves as humans.

LinkedIn: Social Interaction Design Lessons Learned (not to follow)

I have a two part post on LinkedIn's social interaction design. LinkedIn: Social Interaction Design Lessons Learned (not to follow) - 1 of 2 looks at what LinkedIn has done well in the past and had built on top. Many people have expressed the new social interactions on LinkedIn have decreased the value of the service for them.

The second part, LinkedIn: Social Interaction Design Lessons Learned (not to follow) - 2 of 2 looks at the social interaction that has been added to LinkedIn in the last 18 months or so and what lessons have we as users of the service who pay attention to social interaction design have learned. This piece also list ways forward from what is in place currently.

Optimizing Tagging UI for People & Search

Overview/Intro

One of my areas of focus is around social tools in the workplace (enterprise 2.0) is social bookmarking. Sadly, is does not have the reach it should as it and wiki (most enterprise focused wikis have collective voice pages (blogs) included now & enterprise blog tools have collaborative document pages (wikis). I focus a lot of my attention these days on what happens inside the organization’s firewall, as that is where their is incredible untapped potential for these tools to make a huge difference.

One of the things I see on a regular basis is tagging interfaces on a wide variety of social tools, not just in social bookmarking. This is good, but also problematic as it leads to a need for a central tagging repository (more on this in a later piece). It is good as emergent and connective tag terms can be used to link items across tools and services, but that requires consistency and identity (identity is a must for tagging on any platform and it is left out of many tagging instances. This greatly decreases the value of tagging - this is also for another piece). There are differences across tools and services, which leads to problems of use and adoption within tools is tagging user interface (UI).

Multi-term Tag Intro

The multi-term tag is one of the more helpful elements in tagging as it provides the capability to use related terms. These multi-term tags provide depth to understanding when keeping the related tag terms together. But the interfaces for doing this are more complex and confusing than they should be for human, as well as machine consumption.

In the instance illustrated to the tag is comprised or two related terms: social and network. When the tool references the tag, it is looking at both parts as a tag set, which has a distinct meaning. The individual terms can be easily used for searches seeking either of those terms, but knowing the composition of the set, it is relatively easy for the service to offer up "social network" when a person seeks just social or network in a search query.

One common hindrance with social bookmarking adoption is those familiar with it and fans of it for enterprise use point to Delicious, which has a couple huge drawbacks. The compound multi-term tag or disconnected multi-term tags is a deep drawback for most regular potential users (the second is lack of privacy for shared group items). Delicious breaks a basic construct in user focussed design: Tools should embrace human methods of interaction and not humans embracing tech constraints. Delicious is quite popular with those of us malleable in our approach to adopt a technology where we adapt our approach, but that percentage of potential people using the tools is quite thin as a percentage of the population.. Testing this concept takes very little time to prove.

So, what are the options? Glad you asked. But, first a quick additional excursion into why this matters.

Conceptual Models Missing in Social Tool Adoption

One common hinderance for social tool adoption is most people intended to use the tools are missing the conceptual model for what these tools do, the value they offer, and how to personally benefit from these values. There are even change costs involved in moving from a tool that may not work for someone to something that has potential for drastically improved value. The "what it does", "what value it has", and "what situations" are high enough hurdles to cross, but they can be done with some ease by people who have deep knowledge of how to bridge these conceptual model gaps.

What the tools must not do is increase hurdles for adoption by introducing foreign conceptual models into the understanding process. The Delicious model of multi-term tagging adds a very large conceptual barrier for many & it become problematic for even considering adoption. Optimally, Delicious should not be used alone as a means to introduce social bookmarking or tagging.

We must remove the barriers to entry to these powerful offerings as much as we can as designers and developers. We know the value, we know the future, but we need to extend this. It must be done now, as later is too late and these tools will be written off as just as complex and cumbersome as their predecessors.

If you are a buyer of these tools and services, this is you guideline for the minimum of what you should accept. There is much you should not accept. On this front, you need to push back. It is your money you are spending on the products, implementation, and people helping encourage adoption. Not pushing back on what is not acceptable will greatly hinder adoption and increase the costs for more people to ease the change and adoption processes. Both of these costs should not be acceptable to you.

Multi-term Tag UI Options

Compound Terms

I am starting with what we know to be problematic for broad adoption for input. But, compound terms also create problems for search as well as click retrieval. There are two UI interaction patterns that happen with compound multi-term tags. The first is the terms are mashed together as a compound single word, as shown in this example from Delicious.

The problem here is the mashing the string of terms "architecture is politics" into one compound term "architectureispolitics". Outside of Germanic languages this is problematic and the compound term makes a quick scan of the terms by a person far more difficult. But it also complicates search as the terms need to be broken down to even have LIKE SQL search options work optimally. The biggest problem is for humans, as this is not natural in most language contexts. A look at misunderstood URLs makes the point easier to understand (Top Ten Worst URLs)

The second is an emergent model for compound multi-term tags is using a term delimiter. These delimiters are often underlines ( _ ), dots ( . ), or hyphens ( - ). A multi-term tag such as "enterprise search" becomes "enterprise.search", "enterprise_search" and "enterprise-search".

While these help visually they are less than optimal for reading. But, algorithmically this initially looks to be a simple solution, but it becomes more problematic. Some tools and services try to normalize the terms to identify similar and relevant items, which requires a little bit of work. The terms can be separated at their delimiters and used as properly separated terms, but since the systems are compound term centric more often than not the terms are compressed and have similar problems to the other approach.

Another reason this is problematic is term delimiters can often have semantic relevance for tribal differentiation. This first surface terms when talking to social computing researchers using Delicious a few years ago. They pointed out that social.network, social_network, and social-network had quite different communities using the tags and often did not agree on underlying foundations for what the term meant. The people in the various communities self identified and stuck to their tribes use of the term differentiated by delimiter.

The discovery that these variations were not fungible was an eye opener and quickly had me looking at other similar situations. I found this was not a one-off situation, but one with a fair amount of occurrence. When removing the delimiters between the terms the technologies removed the capability of understanding human variance and tribes. This method also breaks recommendation systems badly as well as hindering the capability of augmenting serendipity.

So how do these tribes identify without these markers? Often they use additional tags to identity. The social computing researchers add "social computing", marketing types add "marketing", etc. The tools then use their filtering by co-occurrence of tags to surface relevant information (yes, the ability to use co-occurrence is another tool essential). This additional tag addition help improve the service on the whole with disambiguation.

Disconnected Multi-term Tags

The use of distinct and disconnected term tags is often the intent for space delimited sites like Delicious, but the emergent approach of mashing terms together out of need surfaced. Delicious did not intend to create mashed terms or delimited terms, Joshua Schachter created a great tool and the community adapted it to their needs. Tagging services are not new, as they have been around for more than two decades already, but how they are built, used, and platforms are quite different now. The common web interface for tagging has been single terms as tags with many tags applied to an object. What made folksonomy different from previous tagging was the inclusion of identity and a collective (not collaborative) voice that intelligent semantics can be applied to.

The downside of disconnected terms in tagging is certainty of relevance between the terms, which leads to ambiguity. This discussion has been going on for more than a decade and builds upon semantic understanding in natural language processing. Did the tagger intend for a relationship between social & network or not. Tags out of the context of natural language constructs provide difficulties without some other construct for sense making around them. Additionally, the computational power needed to parse and pair potential relevant pairings is somethings that becomes prohibitive at scale.

Quoted Multi-term Tags

One of the methods that surfaced early in tagging interfaces was the quoted multi-term tags. This takes becomes #&039;research "social network" blog' so that the terms social network are bound together in the tool as one tag. The biggest problem is still on the human input side of things as this is yet again not a natural language construct. Systematically the downside is these break along single terms with quotes in many of the systems that have employed this method.

What begins with a simple helpful prompt...:

Still often can end up breaking as follows (from SlideShare):

Comma Delimited Tags

Non-space delimiters between tags allows for multi-term tags to exist and with relative ease. Well, that is relative ease for those writing Western European languages that commonly use commas as a string separator. This method allows the system to grasp there are multi-term tags and the humans can input the information in a format that may be natural for them. Using natural language constructs helps provide the ability ease of adoption. It also helps provide a solid base for building a synonym repository in and/or around the tagging tools.

While this is not optimal for all people because of variance in language constructs globally, it is a method that works well for a quasi-homogeneous population of people tagging. This also takes out much of the ambiguity computationally for information retrieval, which lowers computational resources needed for discernment.

Text Box Per Tag

Lastly, the option for input is the text box per tag. This allows for multi-term tags in one text box. Using the tab button on the keyboard after entering a tag the person using this interface will jump down to the next empty text box and have the ability to input a term. I first started seeing this a few years ago in tagging interfaces tools developed in Central Europe and Asia. The Yahoo! Bookmarks 2 UI adopted this in a slightly different implementation than I had seen before, but works much the same (it is shown here).

There are many variations of this type of interface surfacing and are having rather good adoption rates with people unfamiliar to tagging. This approach tied to facets has been deployed in Knowledge Plaza by Whatever s/a and works wonderfully.

All of the benefits of comma delimited multi-term tag interfaces apply, but with the added benefit of having this interface work internationally. International usage not only helps build synonym resources but eases language translation as well, which is particularly helpful for capturing international variance on business or emergent terms.

Summary

This content has come from more than four years of research and discussions with people using tools, both inside enterprise and using consumer web tools. As enterprise moves more quickly toward more cost effective tools for capturing and connecting information, they are aware of not only the value of social tools, but tools that get out the way and allow humans to capture, share, and interact in a manner that is as natural as possible with the tools getting smart, not humans having to adopt technology patterns.

This is a syndicated version of the same post at Optimizing Tagging UI for People & Search :: Personal InfoCloud that has moderated comments available.

Tale of Two Tunnels: Web 2.0 and Enterprise 2.0

Yesterday I made a few comments in Twitter that prompted a fair amount of questions and requests for more information. The quips I made were about the differences between Web 2.0 (yes, an ambiguous term) and Enterprise 2.0 (equally ambiguous term both for the definition of enterprise and the 2.0 bit). My comments were in response to Bruce Stewart's comment The whole "Enterprise 2.0" schtick is wearing thin, unless you've been monitoring real results. Otherwise you're just pumping technology.. In part I agree, but I am really seeing things still are really early in the emergence cycle and there is still much need for understanding of the social tools and the need for them, as well as how they fit in. There are many that are selling the tools as technologies with great promise. We have seen the magic pill continually pitched and bought through out the history of business tools. (For those new to the game or only been paying attention for the last 15 years, a huge hint, THERE IS NO MAGIC PILL).

Tale of 2 Tunnels

One comment I made yesterday is, "the difference between Web 2.0 and Enterprise 2.0 is like the difference building a tunnel through rock and tunnel under water".

That this is getting at is Web 2.0 takes work to build to get through the earth, but once built it can suffer from imperfections and still work well. The tunnel can crack and crumble a little, but still get used with diminished capacity. We can look at Facebook, which has a rather poor interface and still gets used. Twitter is another example of a Web 2.0 solution that has its structural deficiencies and outages, but it still used as well as still loved (their Fail Whale is on a t-shirt now and a badge of pride worn by loyal users).

The Enterprise 2.0 tunnel is built under water. This takes more engineering understanding, but it also requires more fault testing and assurances. A crack or crumbling of a tool inside an organization is not seen kindly and raises doubts around the viability of the tool. The shear volume of users inside an organization using these tools is orders of magnitude less than in the open consumer web world, but faults are more deadly.

The other important factor is perceived fear of the environment. Fewer people (by pure numbers - as the percentages are likely the same, more on this later) are fearful of tunnels through land, they may not have full faith in them, but they know that they will likely make it safely on all of their journeys. The tunnels under water have greater fears as one little crack can cause flooding and drowning quickly. Fears of use of social tools inside an organization is often quite similar, there may be many that are not fearful, but if you spend time talking to people in organizations not using tools (it is the majority at this point) they are fearful of open sharing as that could lead to trouble. People are not comfortable with the concept as they are foreign to it as they are lacking the conceptual models to let them think through it.

Enterprise 2.0 is not Web 2.0

Another statement yesterday that garnered a lot of feedback was, "Web 2.0 does not work well in enterprise, but the approaches and understandings of Web 2.0 modified for enterprise work really well." The web is not enterprise or smaller organizations for that matter. The open consumer web has different scale and needs than inside organizations and through their firewalls. A small percentage of people using the web can get an account on a tool have have appear to be wildly successful correctly claiming 70 million or 100 million people are or have used their tool. But, even 100 million people is a small percentage of people using the web. Looking at real usage and needs for those tools the numbers are really smaller. Most darlings of the Web 2.0 phase have fewer than 10 million users, which is about 5% of the open consumer web users in the United States. On the web a start-up is seen as successful with 500,000 users after a year or two and is likely to have the capability to be self sufficient at that level too. Granted there are many players in the same market niches on the web and the overall usage for link sharing and recommending for Digg, Mixx, or Reddit is much higher across the sum of these tools than in just one of these tools (obviously).

These percentages of adoption and use inside organizations can make executives nervous that their money is not reaching as many employees as they wish. The percentages that can be similar to the web's percentages of high single digit adoption rates to the teens is seen as something that really needs more thinking and consideration.

Enterprise 2.0 is more than just tools (see my Enterprise Social Tools: Components for Success for better understanding) as it also includes interface/interaction design for ease of use, sociality, and encouragement of use. The two biggest factors that are needed inside an organization that can receive less attention on the web are the sociality and encouragement of use.

Understanding sociality is incredibly important inside an organization as people are used to working in groups (often vertical in their hierarchy) that have been dictated to them for use. When the walls are broken down and people are self-finding others with similar interests and working horizontally and diagonally connecting and sharing with others and consuming the collective flows of information their comfortable walls of understanding are gone. A presentation in Copenhagen at Reboot on Freely Seeping Through the Walls of the Garden focussed just on this issue. This fear inside the enterprise is real. Much of the fear is driven by lacking conceptual models and understanding the value they will derive from using the tools and services. People need to know who the other people are that they are sharing with and what their motivations are (to some degree) before they have comfort in sharing themselves.

Encouraging use is also central to increased adoption inside organizations. Many organizations initial believe that Web 2.0 tools will take off and have great adoption inside an organization. But, this is not a "build it and they will come" scenario, even for the younger workers who are believed to love these tools and services and will not stay in a company that does not have them. The reality is the tools need selling their use, value derived from them, the conceptual models around what they do, and easing fears. Adoption rates grow far beyond the teen percentages in organizations that take time guiding people about the use of the tools and services. Those organizations that take the opportunity to continually sell the value and use for these tools they have in place get much higher adoption and continued engagement with the tools than those who do nothing and see what happens.

Gaps in Enterprise Tools

The last related statement was around the gaps in current and traditional enterprise tools. At the fantastic Jive Enterprise UI Summit in Aspen a few weeks ago there was a lot of discussion about enterprise tools, their UI, and ease of use for employees by the incredible collection of people at the event. One of the things that was shown was a killer path of use through a wide encompassing enterprise toolset that was well designed and presented by SAP's Dan Rosenberg who has done an incredible job of putting user experience and thinking through the needed workflows and uses of enterprise tools at the forefront of enterprise software planning. Given the excellent design and incredible amount of user experience thought that went into the tools behind the SAP toolset in the scenario (one of the best I have seen - functioning or blue sky demoed) there are still gaps. Part of this is identifying of gaps comes from traditional business thinking around formal processes and the tools ensure process adherence. But, the reality is the tools are quite often inflexible (I am not talking about SAP tools, but traditional enterprise tools in general), the cost of time and effort is beyond the gain for individuals to document and annotate all decisions and steps along the way. The hurdles to capture information and share it are often too large for capturing one to 10 quick sentences of information that can be retained for one's own benefit or shared with other where it is relevant.

There is another gap in business around the collective intelligence that is needed, which can lead to collaboration. Most businesses and their tools focus on collaboration and set groups, but at the same time wonder why they do not know what their company knows and knowledge is not all being captured. First there is a difference between collective and collaborative activities and the tools and design around and for those different activities is more than a nuance of semantics it is a huge barrier to capturing, sharing, and learning from information that leads to knowledge if it is not understood well. Enterprise has gone through its phases of knowledge management tools, from forms for capturing information, forums for sharing, and up to enterprise content management systems (ECM) that encompass document management, content management, knowledge management, and information harvesting. But, the gaps still exist.

These existing gaps are around conversations not being captured (the walls of the halls have no memory (well today they do not)) and increasingly the ubiquitous communication channel in organizations, e-mail, is being worked around. Quick decisions are not being documented as it is not enough for a document or worth completing a form. As the iterative processes of development, design, and solution engineering are happening at quicker and smaller increments the intelligence behind the decisions is not being captured or shared. This is largely because of the tools.

As has always been the case large enterprise systems are worked around through the use of smaller and more nimble solutions that augment the existing tools. Even in Dan's incredible demo I saw gaps for these tools. The quick tools that can fill these gaps are blogs, wikis, social bookmarking, tagging, Twitter type sharing, Veodia type video sharing, instant messaging, etc. There are many avenues to quickly capture information and understanding and share it. These tools get out of the way and allow what is in someone's head to get digitized and later structured by the individual themselves or other people whom have had the information shared with them in a community space. This turns into flows through streams that can be put into many contexts and needs as well as reused as needed.

Another point Dan stated at the Enterprise UI Summit that is dead on, is organizations are moving out of the vertical structures and moving to the horizontal. This is having a profound effect on the next generation of business tools and processes. This is also an area for Enterprise 2.0 tools as they easily open up the horizontal and diagonal prospects and tie into it the capability for easily understanding who these newly found people are in an organization through looking at their profiles, which eases their fears around sharing and unfamiliar environments as well as their related tasks.

[Comments are open and moderated at Tale of Two Tunnels: Web 2.0 and Enterprise 2.0 :: Personal InfoCloud]

"Building the social web" Full-day Workshop in Copenhagen on June 30th

Through the wonderful cosponsoring of FatDUX I am going to be putting on a full-day workshop Building the Social Web on June 30th in Copenhagen, Denmark (the event is actually in Osterbro). This is the Monday following Reboot, where I will be presenting.

I am excited about the workshop as it will be including much of my work from the past nine months on setting social foundations for successful services, both on the web and inside organizations on the intranet. The workshop will help those who are considering, planning, or already working on social sites to improve the success of the services by providing frameworks that help evaluating and guiding the social interactions on the services.

Space is limited for this workshop to 15 seats and after its announcement yesterday there are only 10 seats left as of this moment.

Explaining the Granular Social Network

This post on Granular Social Networks has been years in the making and is a follow-up to one I previously made in January 2005 on Granular Social Networks as a concept I had been presenting and talking about for quite some time at that point. In the past few years it has floated in and out of my presentations, but is quite often mentioned when the problems of much of the current social networking ideology comes up. Most of the social networking tools and services assume we are broadline friends with people we connect to, even when we are just "contacts" or other less than "friend" labels. The interest we have in others (and others in us) is rarely 100 percent and even rarer is that this 100 percent interest and appreciation is equal in both directions (I have yet to run across this in any pairing of people, but I am open to the option that it exists somewhere).

Social Tools Need to Embrace Granularity

What we have is partial likes in others and their interests and offerings. Our social tools have yet to grasp this and the few that do have only taken small steps to get there (I am rather impressed with Jaiku and their granular listening capability for their feed aggregation, which should be the starting point for all feed aggregators). Part of grasping the problem is a lack of quickly understanding the complexity, which leads to deconstructing and getting to two variables: 1) people (their identities online and their personas on various services) and 2) interests. These two elements and their combinations can (hopefully) be seen in the quick annotated video of one of my slides I have been using in presentations and workshops lately.

Showing Granular Social Network

Granular Social Network from Thomas Vander Wal on Vimeo.

The Granular Social Network begins with one person, lets take the self, and the various interest we have. In the example I am using just five elements of interest (work, music, movies, food, and biking). These are interest we have and share information about that we create or find. This sharing may be on one service or across many services and digital environments. The interests are taken as a whole as they make up our interests (most of us have more interests than five and we have various degrees of interest, but I am leaving that out for the sake of simplicity).

Connections with Others

Our digital social lives contain our interests, but as it is social it contains other people who are our contacts (friends is presumptive and gets in the way of understanding). These contacts have and share some interests in common with us. But, rarely do the share all of the same interest, let alone share the same perspective on these interests.

Mapping Interests with Contacts

But, we see when we map the interests across just six contacts that this lack of fully compatible interests makes things a wee bit more complicated than just a simple broadline friend. Even Facebook and their touted social graph does not come close to grasping this granularity as it is still a clumsy tool for sharing, finding, claiming, and capturing this granularity. If we think about trying a new service that we enjoy around music we can not easily group and capture then try to identify the people we are connected to on that new service from a service like Facebook, but using another service focussed on that interest area it could be a little easier.

When we start mapping our own interest back to the interest that other have quickly see that it is even more complicated. We may not have the same reciprocal interest in the same thing or same perception or context as the people we connect to. I illustrate with the first contact in yellow that we have interest in what they share about work or their interest in work, even though they are not stating or sharing that information publicly or even in selective social means. We may e-mail, chat in IM or talk face to face about work and would like to work with them in some manner. We want to follow what they share and share with them in a closer manner and that is what this visual relationship intends to mean. As we move across the connections we see that the reciprocal relationships are not always consistent. We do not always want to listen to all those who are sharing things, with use or the social collective in a service or even across services.

Focus On One Interest

Taking the complexity and noise out of the visualization the focus is placed on just music. We can easily see that there are four of our six contacts that have interest in music and are sharing their interest out. But, for various reasons we only have interests in what two of the four contacts share out. This relationship is not capturing what interest our contacts have in what we are sharing, it only captures what they share out.

Moving Social Connections Forward

Grasping this as a relatively simple representation of Granular Social Networks allows for us to begin to think about the social tools we are building. They need to start accounting for our granular interests. The Facebook groups as well as listserves and other group lists need to grasp the nature of individuals interests and provide the means to explicitly or implicitly start to understand and use these as filter options over time. When we are discussing portable social networks this understanding has be understood and the move toward embracing this understanding taken forward and enabled in the tools we build. The portable social network as well as social graph begin to have a really good value when the who is tied with what and why of interest. We are not there yet and I have rarely seen or heard these elements mentioned in the discussions.

One area of social tools where I see this value beginning to surface in through tagging for individuals to start to state (personally I see this as a private or closed declaration that only the person tagging see with the option of sharing with the person being tagged, or at least have this capability) the reasons for interest. But, when I look at tools like Last.fm I am not seeing this really taking off and I hear people talking about not fully understanding tagging as as it sometimes narrows the interest too narrowly. It is all an area for exploration and growth in understanding, but digital social tools, for them to have more value for following and filtering the flows in more manageable ways need to more in grasping this more granular understanding of social interaction between people in a digital space.

YouTube New Interface and Social Interaction Design Santiy Check

YouTube has released a new design for the site and its individual video pages. This gets shared in Google Operating System :: User Inferface Updates at YouTube and TechCrunch :: YouTube Updates Layout, Now with Tabs and Statistics. While the new design looks nice and clean, it has one design bug that is horribly annoying it has mixed interaction design metaphors for its tabs or buttons.

Broken Interaction Design on Buttons or Tabs

As the image shows the Share, Favorite, Playlists, and Flag buttons or tabs all have similar design treatment, but they do not have the same actions when you click on them. Three of the items (Share, Playlists, and Flag) all act as tabs that open up a larger area below them to provide more options and information. But, the Favorites acts like a button that when clicked it marks the item as a favorite.

This is incredibly poor interaction design as all the items should act in the same manner. If the items do not have the same action properties they really should not look the same and be in the same action space. Favorites should be a check box or a binary interface for on and off. That interaction patter more closely matches the Rate section and seems like it should have been there rather than showing a lack of understanding interaction design basics and confusing people using the site/service.

Social Sites Seem to Share a Lack of Interaction Understanding

This should have been a no brainer observation for a design manager or somebody with a design sanity check. YouTube is far from the the only site/service doing this. Nearly all of the services are not grasping the basics or are broadly applying design patterns to all user scenarios when they really do not fit all scenarios and user types (nearly every service I talk to know exactly the use type a person fits into but never takes this into account in optimization of design patterns that match that use need). Facebook really falls into this hole badly and never seems to grasp they are really making a mess of things the more features and functionality they are bringing into their service without accounting for the design needs in the interface.

My seemingly favorite site to nit pick is LinkedIn which I use a lot and has been a favorite, but their social interaction additions and interactive interfaces really need much better sanity checks and testing before they go into production (even into the beta interface). LinkedIn is really trying to move forward and they are moving in the right direction, but they really need better design thinking with their new features and functionality. Their new design is ready to handle some of the new features, but the features need a lot more refining. The new design shows they have a really good grasp that the interface needs to be a flexible foundation to be used as a framework for including new features, which could benefit from treating them as options for personalization. LinkedIn has pulled back many of the social features and seems to be rethinking them and refining them, but they really need some good sanity checks before rolling them out again.

Social Interaction in Enterprise Tools

The befuddled interaction understanding is not germane to commercial or consumer public social web sites, but it also plagues tools aimed at the enterprise. This is not overly surprising as many of the social enterprise (enterprise 2.0) tools and services are copying the public web tools and services to a large degree. This is a good thing, as it puts the focus on ease of use, which has been horribly missing in business focussed tools for far too long. But, the down side for enterprise focussed tools is they are not for the public web they are for business users, who most often do not have familiarity with the conventions on the public web and they have a large cognitive gap in understanding what the tools do and their value. There is less time for playing and testing in most business people's worklife. This means the tools need to get things right up front with clear understanding of the use needs of the people they are building for in business. This seems to be lacking in many tools as there is much copying of poor design that really needs to be tested thoroughly before launching. Business focussed tools are not hitting the same people as are on the web, which will work through poor design and functionality to see what things do. It is also important to consider that there are a wide variety of types of people using these tools with varying needs and varying interaction understandings (this will be another blog post, actually a series of posts that relate to things I have been including in workshops the last six months and presenting the last couple).

[Comments are available and moderated as usual at: YouTube New Interface and Social Interaction Design Santiy Check :: Personal InfoCloud]

Denning and Yaholkovsky on Real Collaboration

The latest edition of the Communications of the ACM (Volume 51, Issue 4 - April 2008) includes an article on Getting to "we", which starts off by pointing out the misuse and mis-understanding of the term collaboration as well as the over use of the practice of collaboration when it is not proper for the need. The authors Peter Denning and Peter Yaholkovsky break down the tools needed for various knowledge needs into four categories: 1) Information sharing; 2) Coordination; 3) Cooperation; and Collaboration. The authors define collaboration as:

Collaboration generally means working together synergistically. If your work requires support and agreement of others before you can take action, you are collaborating.

The article continues on to point out that collaboration is often not the first choice of tools we should reach for, as gathering information, understanding, and working through options is really needed in order to get to the stages of agreement. Their article digs deeply into the resolving "messy problems" through proper collaboration methods. To note, the wiki - the usual darling of collaboration - is included in their "cooperation" examples and not Collaboration. Most of the tools many businesses consider in collaboration tools are in the lowest level, which is "information sharing". But, workflow managment falls into the coordination bucket.

This is one of the better breakdowns of tool sets I have seen. The groupings make a lot of sense and their framing of collaboration to take care of the messiest problems is rather good, but most of the tools and services that are considered to be collaborations tools do not even come close to that description or to the capabilities required.

[Comments are open at Denning and Yaholkovsky on Real Collaboration :: Personal InfoCloud]

6th Internet Identity Workshop Coming Up

The other event that I am finding to be fantastic is also in the Bay Area the week of May 12th is the 6th Internet Identity Workshop. This is the event for people working around identity related issues (any social application or service) that are now the core of nearly all products on the web and intranet. I have found that those who attend this event really grasp the meaning and deep impact of identity along with the needed tools and services around identity. It is really rare that I find somebody talking or writing about identity related issues in a smart manner that has not been part of one of the past IIW events.

As the discussion around the social graph has become hot identity (and the issue of privacy) has come to the forefront even more. Most services are not dealing with identity in an intelligent manner that is recognizable by a huge majority of people who are using these digital services. Much of the mangled discussion around social graph is missing solid understanding from a digital identity perspective and the use and reuse of statements of relationship that do not transcend various services.

Discussions around persona (not the IA persona variety) and identity abound and the need for services that grasp these differences are worked through. The need for better understanding the incredible value the role of identity in tagging services has also been discussed here, which is something many services do not grasp and are doing a dis-service to the people who want to tag items in their own perspective and context to ease their own refinding of the object (Twine really needs a much better understanding of tagging as their automated tagging is incredibly poor and missing many tangents for understanding that need to be applied for full and proper understanding of the objects in their service).

I am really hoping to get to part of the IIW event this time around my workshop in Las Vegas to continue with the great identity conversations from the past IIW events.

Understanding Taxonomy and Folksonmy Together

I deeply appreciate Joshua Porter's link to from his Taxonomies and Tags blog post. This is a discussion I have quite regularly as to the relation and it is in my presentations and workshops and much of my tagging (and social web) training, consulting, and advising focusses on getting smart on understanding the value and downfalls of folksonomy tagging (as well as traditional tagging - remember tagging has been around in commercial products since at least the 1980s). The following is my response in the comments to Josh' post...

Response to Taxonomy and Tags

Josh, thanks for the link. If the world of language were only this simple that this worked consistently. The folksonomy is a killer resource, but it lacks structure, which it crucial to disambiguating terms. There are algorithmic ways of getting close to this end, but they are insanely processor intensive (think days or weeks to churn out this structure). Working from a simple flat taxonomy or faceted system structure can be enabled for a folksonomy to adhere to.

This approach can help augment tags to objects, but it is not great at finding objects by tags as Apple would surface thousands of results and they would need to be narrowed greatly to find what one is seeking.

There was an insanely brilliant tool, RawSugar [(now gone thanks to venture capitalists pulling the plug on a one of a kind product that would be killer in the enterprise market)], that married taxonomy and folksonomy to help derive disambiguation (take appleseed as a tag, to you mean Johnny Appleseed, appleseed as it relates to gardening/farming, cooking, or the anime movie. The folksonomy can help decipher this through co-occurrence of terms, but a smart interface and system is needed to do this. Fortunately the type of system that is needed to do this is something we have, it is a taxonomy. Using a taxonomy will save processor time, and human time through creating an efficient structure.

Recently I have been approached by a small number of companies who implemented social bookmarking tools to develop a folksonomy and found the folksonomy was [initially] far more helpful than they had ever imagined and out paced their taxonomy-based tools by leaps and bounds (mostly because they did not have time or resources to implement an exhaustive taxonomy (I have yet to find an organization that has an exhaustive and emergent taxonomy)). The organizations either let their taxonomist go or did not replace them when they left as they seemed to think they did not need them with the folksonomy running. All was well and good for a while, but as the folksonomy grew the ability to find specific items decreased (it still worked fantastically for people refinding information they had personally tagged). These companies asked, "what tools they would need to start clearing this up?" The answer a person who understands information structure for ease of finding, which is often a taxonomist, and a tool that can aid in information structure, which is often a taxonomy tool.

The folksonomy does many things that are difficult and very costly to do in taxonomies. But taxonomies do things that folksonomies are rather poor at doing. Both need each other.

Complexity Increases as Folksonomies Grow

I am continually finding organizations are thinking the social bookmarking tools and folksonomy are going to be simple and a cure all, but it is much more complicated than that. The social bookmarking tools will really sing for a while, but then things need help and most of the tools out there are not to the point of providing that assistance yet. There are whole toolsets missing for monitoring and analyzing the collective folksonomy. There is also a need for a really good disambiguation tool and approach (particularly now that RawSugar is gone as a viable approach).

Folksonomy Provides 70 Percent More Terms Than Taxonomy

While at the WWW Conference in Banff for the Tagging and Metadata for Social Information Organization Workshop and was chatting with Jennifer Trant about folksonomies validating and identifying gaps in taxonomy. She pointed out that at least 70% of the tags terms people submitted in Steve Museum were not in the taxonomy after cleaning-up the contributions for misspellings and errant terms. The formal paper indicates (linked to in her blog post on the research more steve ... tagger prototype preliminary analysis) the percentage may even be higher, but 70% is a comfortable and conservative number.

Is 70% New Terms from Folksonomy Tagging Normal?

In my discussion with enterprise organizations and other clients that are looking to evaluate their existing tagging services, have been finding 30 percent to nearly 70 percent of the terms used in tagging are not in their taxonomy. One chat with a firm who had just completed updating their taxonomy (second round) for their intranet found the social bookmarking tool on their intranet turned up nearly 45 percent new or unaccounted for terms. This firm knew they were not capturing all possibilities with their taxonomy update, but did not realize their was that large of a gap. In building their taxonomy they had harvested the search terms and had used tools that analyzed all the content on their intranet and offered the terms up. What they found in the folksonomy were common synonyms that were not used in search nor were in their content. They found vernacular, terms that were not official for their organization (sometimes competitors trademarked brand names), emergent terms, and some misunderstandings of what documents were.

In other informal talks these stories are not uncommon. It is not that the taxonomies are poorly done, but vast resources are needed to capture all the variants in traditional ways. A line needs to be drawn somewhere.

Comfort in Not Finding Information

The difference in the taxonomy or other formal categorization structure and what people actually call things (as expressed in bookmarking the item to make it easy to refind the item) is normally above 30 percent. But, what organization is comfortable with that level of inefficiency at the low end? What about 70 percent of an organizations information, documents, and media not being easily found by how people think of it?

I have yet to find any organization, be it enterprise or non-profit that is comfortable with that type of inefficiency on their intranet or internet. The good part is the cost is relatively low for capturing what people actually call things by using a social bookmarking tool or other folksonomy related tool. The analysis and making use of what is found in a folksonomy is the same cost of as building a taxonomy, but a large part of the resource intensive work is done in the folksonomy through data capture. The skills needed to build understanding from a folksonomy will lean a little more on the analytical and quantitative skills side than the traditional taxonomy development. This is due to the volume of information supplied can be orders of magnitude higher than the volume of research using traditional methods.

It is Finally IT and Design in Enterprise (and Small Business)

My recent trip to Northern California to speak at the UIE Web App Summit and meetings in the Bay Area triggered some good ideas. One thread of discovery is Enterprise, as well as small and medium sized business, is looking at not only technology for solutions to their needs, but design.

IT Traditions

Traditionally, the CIO or VP IT (and related upper management roles) have focussed on buying technology "solutions" to their information problems. Rarely have the solutions fixed the problems as there is often a "problem with the users" of the systems. We see the technology get blamed, the implementation team get blamed (many do not grasp the solution but only how to install the tools, as that is the type of service that is purchased), and then the "users need more training".

Breaking the Cycle of Blame and Disappointment

This cycle of blame and disappointment in technology is breaking around a few important realizations in the IT world.

Technology is not a Cure All

First, the technology is always over sold in capability and most often needs extensive modification to get working in any environment (the cost of a well implemented system is usually about the same as a built from scratch solution - but who has the resources to do that). Most CIOs and technology managers are not trusting IT sales people or marketing pitches. The common starting point is from the, "your tool can not do what you state" and then discussions can move from there. Occasionally, the tools actually can do what is promised.

Many, decision makers now want to test the product with real people in real situations. Solution providers that are good, understand this and will assist with setting up a demonstration. To help truly assess the product the technical staff in the organization is included in the set-up of the product.

People and Information Needs

Second, the problems are finally being identified in terms of people and information needs. This is a great starting place as it focusses on the problems and the wide variety of personal information workflows that are used efficiently by people. We know that technology solutions that mirror and augment existing workflows are easily adopted and often used successfully. This mirroring workflow also allows for lower training costs (occasionally there is no training needed).

Design with People in Mind

Third, design of the interaction and interface must focus on people and their needs. This is the most promising understanding as it revolves around people and their needs. Design is incredibly important in the success of the tools. Design is not just if it looks pretty (that does help), but how a person is walked through the steps easily and how the tools is easy to interact with for successful outcomes. The lack of good design is largely what has crippled most business tools as most have focussed on appealing to the inner geek of the IT manager. Many IT managers have finally realized that their interface and interaction preferences are not remotely representative of 95 percent of the people who need to or should be using the tools.

It is increasingly understood that designing the interaction and interface is very important. The design task must be done with the focus on the needs of real people who will be using the product. Design is not sprinkling some Web 2.0 magic dust of rounded corners, gradients, and fading yellow highlights, but a much deeper understanding that ease of use and breaking processes into easy steps is essential.

Smile to Many Faces

This understanding that buying a technology solutions is more than buying code to solve a problem, but a step in bringing usable tools in to help people work efficiently with information. This last week I talk to many people in Enterprise and smaller businesses that were the technical managers that were trying to get smarter on design and how they should approach digital information problems. I also heard the decision managers stating they needed better interfaces so the people using the tools could, well use the tools. The technology managers were also coming to grips that their preferences for interfaces did not work with most of the people who need the tools to work.

Technology Companies Go Directly to the Users

I have also been seeing the technology tool makers sitting with their actual people using their tools to drastically improve their tools for ease of use. One President of a technology tool maker explained it as, ":I am tired of getting the blame for making poor tools and losing contracts because the technology decision makers are not connected with the real needs of the people they are buying the tools for." This president was talking to three or four users on problems some of his indirect clients were having with a tool they really needed to work well for them. This guy knows the tech managers traditionally have not bought with the people needing to use the tools in mind and is working to create a great product for those people with wants and needs. He also knows how to sell to the technology managers to get their products in the door, but knows designing for the people using the product is how he stays in the company.