vanderwal.net

Understanding Taxonomy and Folksonmy Together

I deeply appreciate Joshua Porter's link to from his Taxonomies and Tags blog post. This is a discussion I have quite regularly as to the relation and it is in my presentations and workshops and much of my tagging (and social web) training, consulting, and advising focusses on getting smart on understanding the value and downfalls of folksonomy tagging (as well as traditional tagging - remember tagging has been around in commercial products since at least the 1980s). The following is my response in the comments to Josh' post...

Response to Taxonomy and Tags

Josh, thanks for the link. If the world of language were only this simple that this worked consistently. The folksonomy is a killer resource, but it lacks structure, which it crucial to disambiguating terms. There are algorithmic ways of getting close to this end, but they are insanely processor intensive (think days or weeks to churn out this structure). Working from a simple flat taxonomy or faceted system structure can be enabled for a folksonomy to adhere to.

This approach can help augment tags to objects, but it is not great at finding objects by tags as Apple would surface thousands of results and they would need to be narrowed greatly to find what one is seeking.

There was an insanely brilliant tool, RawSugar [(now gone thanks to venture capitalists pulling the plug on a one of a kind product that would be killer in the enterprise market)], that married taxonomy and folksonomy to help derive disambiguation (take appleseed as a tag, to you mean Johnny Appleseed, appleseed as it relates to gardening/farming, cooking, or the anime movie. The folksonomy can help decipher this through co-occurrence of terms, but a smart interface and system is needed to do this. Fortunately the type of system that is needed to do this is something we have, it is a taxonomy. Using a taxonomy will save processor time, and human time through creating an efficient structure.

Recently I have been approached by a small number of companies who implemented social bookmarking tools to develop a folksonomy and found the folksonomy was [initially] far more helpful than they had ever imagined and out paced their taxonomy-based tools by leaps and bounds (mostly because they did not have time or resources to implement an exhaustive taxonomy (I have yet to find an organization that has an exhaustive and emergent taxonomy)). The organizations either let their taxonomist go or did not replace them when they left as they seemed to think they did not need them with the folksonomy running. All was well and good for a while, but as the folksonomy grew the ability to find specific items decreased (it still worked fantastically for people refinding information they had personally tagged). These companies asked, "what tools they would need to start clearing this up?" The answer a person who understands information structure for ease of finding, which is often a taxonomist, and a tool that can aid in information structure, which is often a taxonomy tool.

The folksonomy does many things that are difficult and very costly to do in taxonomies. But taxonomies do things that folksonomies are rather poor at doing. Both need each other.

Complexity Increases as Folksonomies Grow

I am continually finding organizations are thinking the social bookmarking tools and folksonomy are going to be simple and a cure all, but it is much more complicated than that. The social bookmarking tools will really sing for a while, but then things need help and most of the tools out there are not to the point of providing that assistance yet. There are whole toolsets missing for monitoring and analyzing the collective folksonomy. There is also a need for a really good disambiguation tool and approach (particularly now that RawSugar is gone as a viable approach).

The Social Enterprise

I am just back from Enterprise 2.0 Conference held in Boston, where I presented Bottom-up All The Way Down: How Tags Help Businesses Organize (thanks to Stowe Boyd for the tantalizing session title), which was liveblog captured by Sandy Kemsley as "Enterprise 2.0: Thomas Vander Wal". I did not catch all of the conference due to some Boston business meetings and connecting with friends and meeting digi-friends whose work I really enjoy face-to-face. The sessions I made it to were good and enlightening and as always the hallway conversations were worth their weight in gold.

Ms. Perceptions and Fear Inside the Corporate Walls

Having not been at true business focussed conference in years (until the past few weeks) I was amazed with how much has changed and how much has stayed the same. I was impressed with the interest and adoption around the social enterprise tools (blogs, wikis, social bookmarking/folksonomy, etc.). But, the misperceptions (Miss Perceptions) are still around and have grown-up (Ms. Perception) and are now being documented by Forrester and others as being fact, but the questions are seemingly not being asked properly. Around the current social web tools (blogs, wikis, social bookmarking, favoriting, shared rating, open (and partially open collaboration) I have been finding little digital divide across the ages. Initially there is a gap when tools get introduced in the corporate environment. But this age gap very quickly disappears if the incredible value of the tools is made clear for peoples worklife, information workflow, and collaboration, as well as simple instructions (30 second to 3 minute videos) and simply written clear guidelines that outline acceptable use of these tools.

I have been working with technology and its adoption in corporations since the late 80s. The misperception that older people do not get technology, are foreign to the tools, and they will not ever get the technical tools has not changed. It is true that nearly all newer technologies come into the corporation by those just out of school and have relied on these tools in university to work intelligently to get their degree. But, those whom are older do see the value in the tools once they have exposure and see the value to their worklife (getting their job done), particularly if the tools are relatively simple to use and can be adopted with simple instruction (if it needs a 10 to 200 page manual and more than 15 minutes of training to start using the product effectively adoption will be low). Toby Redshaw of Motorola stated on a panel that he found in Motorola (4600 blogs and wikis and 2600 people using social bookmarking) "people of all ages adopt these tools if they understand the value connected to their work". Personally, I have seen this has always been the case in the last 20 years as this is how we got e-mail, messaging, Blackberries, web pages, word processing, digital collaboration tools (the last few rounds and the current ones), etc. in the doors of small to large organizations. I have worked in and with technically forward organizations and ones that are traditionally thought of as slow adopters and found adoption is based on value to work and ease of use and rarely based on age.

This lack of understanding around value added and (as Toby Redshaw reinforced) "competitive advantage" derived from the social tools available today for use in the enterprise is driven by fear. It is a fear of control that is lost from the top-down. But, the advantage to the company from having this information shared and easy found and used for collaboration to improve knowledge, understanding, and efficiency can not be dismissed and needs to be embraced. The competitive advantage is what is gained today, but next month or next quarter it could mean just staying even.

Getting Beyond Fear

But, what really is important is the communication and social enterprise tools are okay and add value, but the fear is overplayed, as a percentage rarely occurs, and handling the scary stuff it relatively easy to handle.

Tagging and Social Bookmarking in Enterprise

In the halls I had many conversations around tagging ranging from old school tagging being painful because the experts needed to tag things (meaning they were not doing the job as expert they were hired to do and their terms were not widely understood) all the way to the social bookmarking tools are not scaling and able to keep up with the complexity, nor need to disambiguate the terms used. But, I was really impressed with the number of organizations that have deployed some social bookmarking effort (officially or under somebody's desk) and found value (often great value).

Toby Redshaw: I though folksonomy was going to be some Bob Dillon touchy-feely hippy taxonomy thing, but it has off the chart value far and above any thing we had expected.

My presentation had 80 to 90 percent of the people there using social bookmarking tools in some manner in their organization or worklife. The non-verbal feed back as I was presenting showed interest in how to make better sense of what was being tagged, how to use it better in their business, how to integrate with their taxonomy, and how to work with the information as the tools scale. The answers to these are longer than the hour I have, they are more complex because it all depends on the tools, how they are set-up and designed, how they are used, and the structures of information inside and outside their organization.

Stitching Conversation Threads Fractured Across Channels

Communicating is simple. Well it is simple at its core of one person talking with another person face-to-face. When we communicate and add technology into the mix (phone, video-chat, text message, etc.) it becomes more difficult. Technology becomes noise in the pure flow of communication.

Now With More Complexity

But, what we have today is even more complex and difficult as we are often holding conversation across many of these technologies. The communication streams (the back and forth communication between two or more people) are now often not contained in on communication channel (channel is the flavor or medium used to communicate, such as AIM, SMS, Twitter, e-mail, mobile phone, etc.).

We are seeing our communications move across channels, which can be good as this is fluid and keeping with our digital presence. More often than not we are seeing our communication streams fracture across channels. This fracturing becomes really apparent when we are trying to reconstruct our communication stream. I am finding this fracturing and attempting to stitch the stream back together becoming more and more common as for those who are moving into and across many applications and devices with their own messaging systems.

The communication streams fracture as we pick-up an idea or need from Twitter, then direct respond in Twitter that moves it to SMS, the SMS text message is responded back to in regular SMS outside of Twitter, a few volleys back and forth in SMS text, then one person leaves a voicemail, it is responded to in an e-mail, there are two responses back and forth in e-mail, an hour later both people are on Skype and chat there, in Skype chat they decide to meet in person.

Why Do We Want to Stitch the Communication Stream Together?

When they meet there is a little confusion over there being no written overview and guide. Both parties are sure they talked about it, but have different understandings of what was agreed upon. Having the communication fractured across channels makes reconstruction of the conversation problematic today. The conversation needs to be stitched back together using time stamps to reconstruct everything [the misunderstanding revolved around recommendations as one person understands that to mean a written document and the other it does not mean that].

Increasingly the reality of our personal and professional lives is this cross channel communication stream. Some want to limit the problem by keeping to just one channel through the process. While this is well intentioned it does not meet reality of today. Increasingly, the informal networking leads to meaningful conversations, but the conversations drifts across channels and mediums. Pushing a natural flow, as it currently stands, does not seem to be the best solution in the long run.

Why Does Conversation Drift Across Channels?

There are a few reasons conversations drift across channels and mediums. One reason is presence as when two people notice proximity on a channel they will use that channel to communicate. When a person is seen as present, by availability or recently posting a message in the service, it can be a prompt to communicate. Many times when the conversation starts in a presence channel it will move to another channel or medium. This shift can be driven by personal preference or putting the conversation in a medium or channel that is more conducive for the conversation style between people involved. Some people have a preferred medium for all their conversations, such as text messaging (SMS), e-mail, voice on phone, video chat, IM, etc.. While other people have a preferred medium for certain types of conversation, like quick and short questions on SMS, long single responses in e-mail, and extended conversations in IM. Some people prefer to keep their short messages in the channel where they begin, such as conversations that start in Facebook may stay there. While other people do not pay attention to message or conversation length and prefer conversations in one channel over others.

Solving the Fractured Communication Across Channels

Since there are more than a few reasons for the fractured communications to occur it is something that needs resolution. One solution is making all conversations open and use public APIs for the tools to pull the conversations together. This may be the quickest means to get to capturing and stitching the conversation thread back together today. While viable there are many conversations in our lives that we do not want public for one reason or many.

Another solution is to try to keep your conversations in channels that we can capture for our own use (optimally this should be easily sharable with the person we had the conversation with, while still remaining private). This may be where we should be heading in the near future. Tools like Twitter have become a bridge between web and SMS, which allows us to capture SMS conversations in an interface that can be easily pointed to and stitched back together with other parts of a conversation. E-mail is relatively easy to thread, if done in a web interface and/or with some tagging to pull pieces in from across different e-mail addresses. Skype chat also allows for SMS interactions and allows for them to be captured, searched, and pulled back together. IM conversations can easily be saved out and often each item is time stamped for easy stitching. VoIP conversations are often easily recorded (we are asking permission first, right?) and can be transcribed by hand accurately or be transcribed relatively accurately via speech-to-text tools. Voice-mail can now be captured and threaded using speech-to-text services or even is pushed as an attachment into e-mail in services as (and similar to) JConnect.

Who Will Make This Effortless?

There are three types of service that are or should be building this stitching together the fractured communications across channels into one threaded stream. I see tools that are already stitching out public (or partially public) lifestreams into one flow as one player in this pre-emergent market (Facebook, Jaiku, etc.). The other public player would be telecoms (or network provider) companies providing this as a service as they currently are providing some of these services, but as their markets get lost to VoIP, e-mail, on-line community messaging, Second Life, etc., they need to provide a service that keeps them viable (regulation is not a viable solution in the long run). Lastly, for those that do not trust or want their conversation streams in others hands the personally controlled application will become a solutions, it seems that Skype could be on its way to providing this.

Is There Demand Yet?

I am regularly fielding questions along these lines from enterprise as they are trying to deal with these issues for employees who have lost or can not put their hands on vital customer conversations or essential bits of information that can make the difference in delivering what their customers expect from them. Many have been using Cisco networking solutions that have some of these capabilities, but still not providing a catch all. I am getting queries from various telecom companies as they see reflections of where they would like to be providing tools in a Come to Me Web or facilitating bits of the Personal InfoCloud. I am getting requests from many professionals that want this type of solution for their lives. I am also getting queries from many who are considering building these tools, or pieces of them.

Some of us need these solutions now. Nearly all of us will need these solutions in the very near future.

Full-Day Social Web & Folksonomy Workshop at d.construct

Tickets for the d.construct Workshops go on sale June 14, 2007. Buying a ticket for a full-day workshop provides free access to the full d.construct conference on September 7th. I am presenting the following workshop on September 6th...

Building the Social Web with Tagging / Folksonomies

Folksonomy Provides 70 Percent More Terms Than Taxonomy

While at the WWW Conference in Banff for the Tagging and Metadata for Social Information Organization Workshop and was chatting with Jennifer Trant about folksonomies validating and identifying gaps in taxonomy. She pointed out that at least 70% of the tags terms people submitted in Steve Museum were not in the taxonomy after cleaning-up the contributions for misspellings and errant terms. The formal paper indicates (linked to in her blog post on the research more steve ... tagger prototype preliminary analysis) the percentage may even be higher, but 70% is a comfortable and conservative number.

Is 70% New Terms from Folksonomy Tagging Normal?

In my discussion with enterprise organizations and other clients that are looking to evaluate their existing tagging services, have been finding 30 percent to nearly 70 percent of the terms used in tagging are not in their taxonomy. One chat with a firm who had just completed updating their taxonomy (second round) for their intranet found the social bookmarking tool on their intranet turned up nearly 45 percent new or unaccounted for terms. This firm knew they were not capturing all possibilities with their taxonomy update, but did not realize their was that large of a gap. In building their taxonomy they had harvested the search terms and had used tools that analyzed all the content on their intranet and offered the terms up. What they found in the folksonomy were common synonyms that were not used in search nor were in their content. They found vernacular, terms that were not official for their organization (sometimes competitors trademarked brand names), emergent terms, and some misunderstandings of what documents were.

In other informal talks these stories are not uncommon. It is not that the taxonomies are poorly done, but vast resources are needed to capture all the variants in traditional ways. A line needs to be drawn somewhere.

Comfort in Not Finding Information

The difference in the taxonomy or other formal categorization structure and what people actually call things (as expressed in bookmarking the item to make it easy to refind the item) is normally above 30 percent. But, what organization is comfortable with that level of inefficiency at the low end? What about 70 percent of an organizations information, documents, and media not being easily found by how people think of it?

I have yet to find any organization, be it enterprise or non-profit that is comfortable with that type of inefficiency on their intranet or internet. The good part is the cost is relatively low for capturing what people actually call things by using a social bookmarking tool or other folksonomy related tool. The analysis and making use of what is found in a folksonomy is the same cost of as building a taxonomy, but a large part of the resource intensive work is done in the folksonomy through data capture. The skills needed to build understanding from a folksonomy will lean a little more on the analytical and quantitative skills side than the traditional taxonomy development. This is due to the volume of information supplied can be orders of magnitude higher than the volume of research using traditional methods.

Folksonomy Book In Progress

Let me start with an announcement. I have not had any answer for continual question after I present on tagging and folksonomy (I also get the question after the Come to Me Web and Personal InfoCloud presentations), which is "where is your book?" Well I finally have an answer, I have signed with O'Reilly to write a book, initially titled Understanding Folksonomy (this may change) and it may also be a wee bit different from your normal O'Reilly book (we will see how it goes).

I am insanely excited to be writing for O'Reilly as I have a large collection of their books from over the years - from the programming PHP, Perl, Python, and Ruby to developer guides on JavaScript, XHTML, XML, & CSS to the Polar Bear book on information architecture, Information Dashboard Design, and Designing Interfaces: Patterns for Effective Interaction Design along with so many more.

Narrowing the Subject

One of the things that took a little more time than I realized it would take is narrowing the book down. I have been keeping a running outline of tagging and folksonomy related subjects that have been in my presentations and workshop as well as questions and answers from these sessions. The outline includes the deep knowledge that has some from client work on the subject (every single client has a different twist and set of constraints. Many of the questions have answers and for some the answers are still being worked out, but the parameters and guidelines are known to get to viable solutions.

Well, when I submitted the outline I was faced with the knowledge that I had submitted a framework for a 800 to 1,000 page book. Huh? I started doing the math based on page size, word counts, bullets in the outline, and projects words per bullet and those with knowledge were right. So, I have narrowed it down to an outline that should be about 300 pages (maybe 250 and maybe a few more than 300).

What Is In This Smaller Book?

I am using my tagging and folksonomy presentations as my base, as those have been iterated and well tested (now presented some version of it well over 50 times). While I have over 300 design patterns captured from tagging service sites (including related elements) only a few will likely be used and walked through. I am including the understanding from a cognitive perspective and the lessening of technology pain that tagging services can provide to people who use them. There will also be a focus on business uses for intranet and web.

When Will This Be Done

Given that I have a rather busy Fall with client work, workshops, and presentations I set a goal to finish the writing by the end of Summer. It sounds nuts and it really feels like grad school all over again, but that is my reality. I have most of the words in my head and have been speaking them. Now I need to write them (in a less dense form than I blog).

Your Questions and Feedback

If you have questions and things you would like covered please e-mail me (contact in the header nav above). I will likely be setting up a blog to share and post questions (this will happen in a couple weeks). I am also looking for sites, organizations, and people that would like to be included in the case studies and interviews (not all will end up in the book, but those that are done will end up tied to the book in some manner). Please submit suggestions for this section if you have any.

From the HQ Office

It has been a good stretch of travel, mostly for work/professional life, but also took a trip to Florida for family holiday. At the moment I am back in the office working on proposals for upcoming projects of various lengths for clients and working through the process of writing, which involves dead trees at the final stages.

Web 2.0 Expo and SF Bay Area

I am soon off again to the San Francisco Bay Area to speak at the O'Reilly Web 2.0 Expo and have business meetings around the Bay Area (please ping via e-mail if you would like to meet-up during this time).

WWW2007 Workshop on Tagging and Metadata

Early next month I am off to Banff to keynote the WWW2007 Workshop on Tagging and Metadata for Social Information Organization. I am not sure how long this trip will be as I will have some pressing work around this time.

Social Software Summit

Lastly, I should note there will be a Social Software Summit that will run at the same time as the ASIS&T IA Summit next Spring (Spring 2008) in Miami. The Social Software Summit is still in its early stages of planning (the idea, dates, location, and interest have been launched). The dates are April 10-11 2008. I have a role in the planning and preparation for this event, along with some other incredible people.

On SXSW Tag You're It Panel

I am a panelist on the Tag You're It Panel at South by SouthWest in Austin, Texas. Others on the panel are the ever fantastic: Heath Row (moderator), George Oates, and Ben Brown. The panel information:

Tag You're It on Saturday 10 March 2007 at 2:00 pm to 3:00 pm. The panel will be looking at what people are actually doing inside social tagging systems and where things have gone in the past couple years with tagging. We will get beyond the notion that tagging is cool by providing examples of how people are really using the tools in innovative and useful ways.

Stop by and say hello.

Ma.Del Tagging Bookmarklet

Out of frustration with Yahoo MyWeb no longer surfaces results in Yahoo Search, which made Yahoo Search much better for me than any other web search. Now that this is no long functioning and there is no response as to if or when it will return I am back to Google and Microsoft Live Search (the relevancy is better for me on many things). But, this change also removes the value of MyWeb and this has me looking back to Ma.gnolia (also am a huge fan of Raw Sugar and their facets, but that is another longer post).

New Tagging Combo Bookmarklet

When I became a fan of MyWeb I used some glue to make Del.icio.us and MyWeb Combo Bookmarklet.

So now I have done the same for Ma.gnolia and del.icio.us with the Ma.Del Marklet (drag to bookmark bar in FireFox and Safari only). This was built using the Ma.gnolia bookmarklet Ma.rker Mini as its base.

Cuban Clocks and Music Long Tail Discovery

The last two trips to San Francisco I have heard a latin version of Coldplay's Clocks on KFOG and it really intrigued me. This last trip I was in the car for four songs and one of them was Coldplay's Clocks by the Cuban All Stars. I have been trying to track this track down since first hearing, but am not having great luck. This continually happens when I listen to KFOG, which is about the only regular radio station I will listen to (I much prefer XM Radio for is lack of advertising and blathering idiots spouting off while playing overplayed songs that have little merit.

What I like about this version of Clocks by the Cuban All Stars (I have seen the dashboard metadata list it as Ibrahim Ferrer, but it has not been described as such by the DJs on KFOG). This is where my music recommendations break. But, some digging on the KFOG website points me to Rhythms Del Mundo as the source (but their Flash site seems horribly broken in all browsers as none of the links work). I have found the album on iTunes, but only a partial listing and none of the physical music store options have this in stock as it is not mainstream enough (how I miss Tower).

This all seems like far more work that should be needed. But, not if one has even slightly long tail musical interests. I had a wonderful discussion along these lines wish Cory from IODA about this and the lack of really good long tail discovery systems.

I use Last.fm to discover new things from friend#039;s lists, but the Last.fm neighbor recommendations seem to only work on more mainstream interests (Pandora really falls off on the long tail for me). Now if KFOG put their play list in KFOG, it would help greatly and I would add them to my friend list (or I could move back home to the San Francisco Bay Area).

Stikkit Adds an API

Stikkit has finally added an API for Stikkit. This makes me quite happy. Stikkit has great ease of information entry and it is perfect for adding annotations to web-based information.

Stikkit is My In-line Web Triage

I have been using Stikkit, from the bookmarklet, as my in-line web information triage. If I find an event or something I want to come back to latter (other than to read and bookmark) I pop that information into Stikkit. Most often it is to remind me of deadlines, events, company information, etc. I open the Stikkit bookmarklet and add the information. The date information I add is dumped into my Stikkit calendar, names and addresses are put into the Stikkit address book, and I can tag them for context for easier retrival.

Now with the addition of the API Stikkit is now easy to retrieve a vCard, ical, or other standard data format I can drop into my tools I normally aggregate similar information. I do not need to refer back to Stikkit to copy and paste (or worse mis-type) into my work apps.

I can also publish information from my preferred central data stores to Stikkit so I have web access to events, to dos, names and addresses, etc. From Stikkit I can then share the information with only those I want to share that information with.

Stikkit is growing to be a nice piece for microcontent tracking in my Personal InfoCloud.

It is Finally IT and Design in Enterprise (and Small Business)

My recent trip to Northern California to speak at the UIE Web App Summit and meetings in the Bay Area triggered some good ideas. One thread of discovery is Enterprise, as well as small and medium sized business, is looking at not only technology for solutions to their needs, but design.

IT Traditions

Traditionally, the CIO or VP IT (and related upper management roles) have focussed on buying technology "solutions" to their information problems. Rarely have the solutions fixed the problems as there is often a "problem with the users" of the systems. We see the technology get blamed, the implementation team get blamed (many do not grasp the solution but only how to install the tools, as that is the type of service that is purchased), and then the "users need more training".

Breaking the Cycle of Blame and Disappointment

This cycle of blame and disappointment in technology is breaking around a few important realizations in the IT world.

Technology is not a Cure All

First, the technology is always over sold in capability and most often needs extensive modification to get working in any environment (the cost of a well implemented system is usually about the same as a built from scratch solution - but who has the resources to do that). Most CIOs and technology managers are not trusting IT sales people or marketing pitches. The common starting point is from the, "your tool can not do what you state" and then discussions can move from there. Occasionally, the tools actually can do what is promised.

Many, decision makers now want to test the product with real people in real situations. Solution providers that are good, understand this and will assist with setting up a demonstration. To help truly assess the product the technical staff in the organization is included in the set-up of the product.

People and Information Needs

Second, the problems are finally being identified in terms of people and information needs. This is a great starting place as it focusses on the problems and the wide variety of personal information workflows that are used efficiently by people. We know that technology solutions that mirror and augment existing workflows are easily adopted and often used successfully. This mirroring workflow also allows for lower training costs (occasionally there is no training needed).

Design with People in Mind

Third, design of the interaction and interface must focus on people and their needs. This is the most promising understanding as it revolves around people and their needs. Design is incredibly important in the success of the tools. Design is not just if it looks pretty (that does help), but how a person is walked through the steps easily and how the tools is easy to interact with for successful outcomes. The lack of good design is largely what has crippled most business tools as most have focussed on appealing to the inner geek of the IT manager. Many IT managers have finally realized that their interface and interaction preferences are not remotely representative of 95 percent of the people who need to or should be using the tools.

It is increasingly understood that designing the interaction and interface is very important. The design task must be done with the focus on the needs of real people who will be using the product. Design is not sprinkling some Web 2.0 magic dust of rounded corners, gradients, and fading yellow highlights, but a much deeper understanding that ease of use and breaking processes into easy steps is essential.

Smile to Many Faces

This understanding that buying a technology solutions is more than buying code to solve a problem, but a step in bringing usable tools in to help people work efficiently with information. This last week I talk to many people in Enterprise and smaller businesses that were the technical managers that were trying to get smarter on design and how they should approach digital information problems. I also heard the decision managers stating they needed better interfaces so the people using the tools could, well use the tools. The technology managers were also coming to grips that their preferences for interfaces did not work with most of the people who need the tools to work.

Technology Companies Go Directly to the Users

I have also been seeing the technology tool makers sitting with their actual people using their tools to drastically improve their tools for ease of use. One President of a technology tool maker explained it as, ":I am tired of getting the blame for making poor tools and losing contracts because the technology decision makers are not connected with the real needs of the people they are buying the tools for." This president was talking to three or four users on problems some of his indirect clients were having with a tool they really needed to work well for them. This guy knows the tech managers traditionally have not bought with the people needing to use the tools in mind and is working to create a great product for those people with wants and needs. He also knows how to sell to the technology managers to get their products in the door, but knows designing for the people using the product is how he stays in the company.

Digging Out of Digital Limbo

Things are mostly back to normal here. The new MacBook Pro has been wonderful (it is a bit odd to mostly focus on it as a tool and not the center of adoration as I normally do for Apple products) and even better getting my digital life flowing again. I found some gaps in e-mail had been backed-up in an odd place and they were restored.

The gap between sending my PowerBook out for data recovery (not possible as the drive was toast) and getting the MacBook Pro in my hands, even until the time of my first full back-up was one of trying to balance new info and old. Fearful of a "perfect storm" I was pushing my digital life out into corners of the internet. I have been pulling things back in to my laptop and re-building notes from paper I jot quick bits on.

Moving Forward

I have since picked up a Maxtor OneTouch III Turbo with one terrabyte of capacity set to RAID1 across its two drives (translates to mirrored 500MB drives). This seems like an insane amount of space, but knowing I can have more than one version of a back-up (that is part of what knocked me out) and can back-up my other external drive info (where my 70GB of music is stored) provides a nice peace of mind. I am also integrating my Amazon S3 into the equation as well as my .Mac storage. I am using Apple Backup and Retrospect Express to currently do my back-ups. I want a book able image for use on my Intel Mac, which it seems to have the capability of providing. I am still looking at other options. I have been a big fan of Carbon Copy Cloner, but it does not provide a bootable image capability on Intel Macs.

This New Year

I mostly have my feet under myself this new year, oh yes, Happy New Year! Work is taking off full bore and some of the bumps from last year seem to have been ironed out and lessons learned. I had planned to do a year end book wrap-up and a first year in business as InfoCloud Solutions, Inc., but the lack of a computer pushed those ideas off the table. I do not make New Years resolutions, as I try to make needed adjustments as they are needed and try not to put them off to some arbitrary date. I will be providing one or two most posts later this week, when the pile subsides and things go out the door.

Until then, enjoy Macworld and do your backups.

Ghosts of Technology Past, Present, and Future

The past two days have brought back many memories that have reminded me of the advances in technology as well as the reliance on technology.

Ghost of Rich Web Past

I watched a walk through of a dynamic prototype yesterday that echoed this I was doing in 1999 and 2000. Well, not exactly doing as the then heavy JavaScript would blow up browsers. The DHTML and web interfaces that helped the person using the site to have a better experience quite often caused the browser to lock-up, close with no warning, or lock-up the machine. This was less than 100kb of JavaScript, but many machines more than two years old at that time and with browsers older than a year or two old did not have the power. The processing power was not there, the RAM was not there, the graphics cards were not powerful, and the browsers in need of optimizing.

The demonstration yesterday showed concepts that were nearly the exact concept from my past, but with a really nice interface (one that was not even possible in 1999 or 2000). I was ecstatic with the interface and the excellent job done on the prototype. I realized once again of the technical advances that make rich web interfaces of "Web 2.0" (for lack of a better term) possible. I have seen little new in the world of Ajax or rich interfaces that was not attempted in 2000 or 2001, but now they are viable as many people's machines can now drive this beauties.

I am also reminded of the past technologies as that is what I am running today. All I have at my beck and call is two 667MHz machines. One is an Apple TiBook (with 1 GB of RAM) and one is a Windows machine (killer graphics card with 256MB video RAM and 500MB memory). Both have problems with Amazon and Twitter with their rich interfaces. The sites are really slow and eat many of the relatively few resources I have at my disposal. My browsers are not blowing up, but it feels like they could.

Ghost of Technology Present

The past year or two I have been using my laptop as my outboard memory. More and more I am learning to trust my devices to remind me and keep track of complex projects across many contexts. Once things are in a system I trust they are mostly out of my head.

This experience came to a big bump two days ago when my hard drive crashed. The iterative back-ups were corrupted or faulty (mostly due to a permission issue that would alter me in the middle of the night). The full back-up was delayed as I do not travel with an external drive to do my regular back-ups. My regularly scheduled back-ups seem to trigger when I am on travel. I am now about 2.5 months out from my last good full back-up. I found an e-mail back-up that functioned from about 3 weeks after that last full backup. Ironically, I was in the midst of cleaning up my e-mail for back-up, which is the first step to my major back-up, when the failure happened.

I have a lot of business work that is sitting in the middle of that pile. I also have a lot of new contacts and tasks in the middle of that period. I have my client work saved out, but agreements and new pitches are in the mire of limbo.

Many people are trying to sync and back-up their lives on a regular basis, but the technology is still faulty. So many people have faulty syncing, no matter what technologies they are using. Most people have more than two devices in their life (work and home computer, smart phone, PDA, mobile phone with syncable address book and calendar, iPod, and other assorted options) and the syncing still works best (often passably) between two devices. Now when we start including web services things get really messy as people try to work on-line and off-line across their devices. The technology has not caught up as most devices are marketed and built to solve a problem between two devices and area of information need. The solutions are short sighted.

Ghost of the Technosocial Future

Last week I attended the University of North Carolina Social Software Symposium (UNC SSS) and while much of the conversation was around social software (including tagging/folksonomy) the discussion of technology use crept in. The topic of digital identity was around the edges. The topic of trust, both in people and technology was in the air. These are very important concepts (technology use, digital identity, and trusted technology and trusted people). There is an intersection of the technosocial where people communicate with their devices and through their devices. The technology layer must be understood as to the impact is has on communication. Communication mediated by any technology requires an understanding of how much of the pure signal of communication is lost and warped (it can be modified in a positive manner too when there are disabilities involved).

Our digital communications are improving when we understand the limitations and the capabilities of the technologies involved (be it a web browser of many varied options or mobile phone, etc.). Learning the capabilities of these trusted devices and understanding that they know us and they hold our lives together for us and protect our stuff from peering eyes of others. These trusted devices communicate and share with other trusted devices as well as our trusted services and the people in our lives we trust.

Seeing OpenID in action and work well gave me hope we are getting close on some of these fronts (more on this in another post). Seeing some of the great brains thinking and talking about social software was quite refreshing as well. The ability to build solid systems that augment our lives and bring those near in thought just one click away is here. It is even better than before with the potential for easier interaction, collaboration, and honing of ideas at our doorstep. The ability to build an interface across data sets (stuff I was working on in 1999 that shortened the 3 months to get data on your desk to minutes, even after running analytics and working with a GIS interface) can be done in hours where getting access to the wide variety of information took weeks and months in the past. Getting access to data in our devices to provide location information with those we trust (those we did not trust have had this info for some time and now we can take that back) enables many new services to work on our behalf while protecting our wishes for whom we would like the information shared with. Having trusted devices working together helps heal the fractures in our data losses, while keeping it safe from those we do not wish to have access. The secure transmission of our data between our trusted devices and securely shared with those we trust is quickly arriving.

I am hoping the next time I have a fatal hard drive crash it is not noticeable and the data loss is self-healed by pulling things back together from resources I have trust (well placed trust that is verifiable - hopefully). This is the Personal InfoCloud and its dealing with a Local InfoCloud all securely built with trusted components.

Let Me Count the 24 Ways

It is that wonderful time of the year for 24 ways, the wonderful 24 gifts from one web developer to the rest of us. I deeply enjoyed them last year and am looking forward to the remainder of the gems.