vanderwal.net

Weeknote - 3 September 2022

You are asking, “Where are you? Are you okay? Are you still blogging?”

In TikTok parlance, “Great questions. Let me tell you.” First, this standard TikTok pattern is one I find really interesting. It fills in he politeness / nicety gap that has become common in the last decade or two, where people jump into answering questions. This nod to thanking the person asking encourages questions and puts people at ease who asked a question (speaking up is often not something most people are comfortable with). But, the pattern has been used so much and is just a common / required custom, it starts to come off as forced or canned, much like required legal disclaimers. None-the-less, it is a good practice.

Well this was a long “week” (parts of this were in an end of March weeknote that needed finishing, so now edited and updated). Things on the work front got incredibly busy and hectic. I’m going to treat this “weeknote” as a catch-up of things that have held my attention over the past year.

I’m hoping to get back to posting regular weeknotes and blogging. My other blog Personal Infocloud has been quite for a long time, but been waiting for about 2 years for SquareSpace to fix a defect that impacted styling and showing full posts. I have a lot of older content I’ve long used in presentations and workshops, that I’m working to turn into videos of some of the pieces of them that are clearer for understanding in video / animated form. I’m also back working on the 70 plus set of social / complexity lenses I’ve been working on for around 14 years with that label, but around 20 years all together going back to the Model of Attraction (still a foundation for a lot of thinking and framing).

With my son off to college, I may have a little more time to write and share. I’m also looking at a digital garden model (see the last section) and as of recently Massive Wiki for a collaborative or commons approach of moving the Lenses forward (well outward).

Note Taking

I have been deep into cataloging, reading, and using the heck out of Obsidian since trying it out in June of 2020 and going all in at the end of July 2020 and it is now second nature. But, this built on my 10 or so years of taking markdown notes in a directory, which I had 8 to 10 year of text notes in that same directory (which were bulk renamed to markdown). My approach and use of aliases and front matter have changed how I do things, but more on that in the Productivity section below. Many of my issues in a quick test of Roam proved to save me from that path and set of problems, Notion not being mine and not a standard file format so I can reuse the notes easily has stopped being used, I use DevonThink but its backlinking and attempt at other Obsidian functionality was clumsy in my source archive (and I just pull in my directory that Obsidian sits on top of so search is relevant with resources saved), and with Obsidian now having iOS capability I’m really using it a lot on the go. I have a seriously strong preference for having the notes be separate from a system that wrangles and provides organizing for and around them. Having used nvAlt for nearly 10 years when it broke badly and wouldn’t open, all of the 2,500 or so markdown (and text) notes that sat under the app in a directory (and linked with file metadata tags). Putting Obsidian in the same notes directory and crating that directory as a vault things just continued on, but now with far more functionality.

One irony is my use of Obsidian, and in particular my daily notes (Daily Dump), has me posting and sharing here less. It is ironic as I write in the Daily Dump as if I am writing to others, but the notes are just to myself (for now - this may change if I can sort out how to keep some of the reading, learning, observations, etc. separate from work or formative observations. The Daily Dump was partly intended to capture things that could be shared back out in a weeknote. Things like the Personal Operating System, which I found insanely insightful (read below), things from Sentiers and The Near Future Lab (particularly around Generalists, which I find quite similar to a bumping into a brightline for polymaths, but also bumps up against Jane McConnell’s book The Gig Mindset Advantage (more on these later as well).

With Obsidian having tags (used to aggregate related things, as a hook metaphor I’ve used for 18 years or more) and the backlinks to use as bridges to move to related materials and ideas much in the way any hypertext environment functions. I used VooDoo Pad on my Macs for roughly 15 years, but it not easily working across iPad and phone to easily read, edit, or add to the corpus had it shift out of my main workflow. I also use Drafts for quick input from mobile and sometimes iPad.

Obsidian has been amazing with its pace and quality of development over the last 2 years. The iPad version is pretty solid, but I’m usually in reach of my iPad so I’m not leaning on it all that much at the moment. This past week there were large changes to the insider build for 0.16 (it was reworking some underpinnings to improve many of community built plug-ins, themes, and templates), but it was the first time the updates broke things in my workflow rather badly. Normally updates cause no problems, but only offer benefits and improvements, with occasional bumps that are resolved in 5 to 10 minutes. But, with the bump this week, I still love it for thinking through writing and note capturing and interlinking. There isn’t anything out there that is close to it.

Read

I have been reading a lot, with a good portion coming through me trusty RSS feed reader, NetNewswire, which echoes my vanderwal.net links page.

Newsletters

While I am not a huge fan of newletters (mostly the part that they arrive in email and not RSS, but the ones that also have RSS feeds are the ones I have been sticking to). Many of these arrive on Sunday, but I really wish it were Friday night or early Saturday morning so I have Saturday and Sunday mornings to get through them and follow the links and devour what is there, but Sunday mornings many arrive and I spend the week going through them.

The one that I am a huge fan of from a general purpose is Patrick Tanguay’s own newsletter Sentiers, which I find to be a real gem. I lost track of Patrick for a while after his Alpine review stopped publishing. I would love to see his Sentiers grow to be a bit more as I find it to be such a good offering. I have been reading it regularly for about a year or a bit more and Patrick has been popping up on podcasts I follow (Near Future Lab (now mostly moved to a Discord for and The Informed Life - more on these later).

Jorge Arango’s newsletter, Informa(c)tion, is an information and organization focussed gem that arrives every other Sunday. There are always good pieces and the links are gems.

Others I really enjoy and tend to link to things that open more browser tabs are: The Marganilian by Maria Popova; Curtis McHale’s PKM newsletter; and Monocle Weekend Edition newsletter (there are many times of late were the newsletter is a bit off target, but the balance for me mostly entertainment).

Books

The Gig Mindset Advantage has been a gem, mostly as it is very familiar as it is pretty much my natural (unintended) MO (modus operendi).

The Map of Knowledge, by Violet Moller has become one of my favorite books. It quickly turned into a slow meditative read as it broke some of my prior understanding of the world of knowledge and creation of advanced math, sciences, and philosophy. This refactoring of my understanding was around the realization that most of the “great books” and works that are the foundations are just very tiny slivers of knowledge that made it through an insanely fragile process of keeping paper copies. A book would need to be hand-written to create a copy and that copy on paper would only last 50 to 80 years before it would heavily decay. I knew well that most of what Western Europe used as fodder in the Renaissance for advancement were works from ancient Greece that had been kept alive through the great libraries and education systems in Persia, Near East, Middle East, Arabian regions, North Africa, and Moorish efforts in Spain. The realization that were may not be looking at the best thinking from the “classics”, but just those that made it through time.

While I had a decent understanding of the vast contributions advancing math, sciences, medicine, and philosophy that are the foundations of much of western thinking, I didn’t know much of the who, when, and where. The Map of Knowledge goes into these areas with a very good level of understanding. The book also does a great job laying out the cycle of life for advanced learning and libraries in each of the regions and progressions through time. One of the common cycles that causes the downfall in many regions was gap in the civilization between those with advanced knowledge and learning and those in power, as well as regular poeple. That chasm between the advanced and those not caused a lot of friction, most often leading to the destruction of libraries and institutions. Many of the civilizations never returned to anything close to the advancements. But, the libraries and learning institutions dispersed and found new benefactors and locations to continue moving forward.

Violet Moller has certain given me a good foundation to learn more.

Gillian Tett’s Anthro-Vision was a book, well a chapter in that book, I’ve been waiting for for many years. The chapter on “Financial Crisis” where Tett had been researching financial markets using her background as an anthropologist for a new role at The Financial Times. Tett followed the paths of understanding in the way a good ethnographer / anthropologist does looking to understand the quiet, and seemingly foundational, areas that seem to be out of focus. This area was that of credit swaps in financial loan markets, which were what caused the 2007 housing market collapse and in 2008 at the massive meltdown of the financial markets. The model for building understanding is one that should be common, but sadly isn’t. The remainder of the book is quite good as well.

Productivity

The biggest thing around productivity is my use of Obsidian continues, as mentioned above. In a couple chats recently I have found other have brought up the backlinking / crosslinking as the most valuable feature. For those of us who have been using Macs for a while we find it reminiscent of not only wikis and their power, but in particular VooDoo Pad, which was light weight and everything was easily interlinked and backlinked and search was incredibly good. VooDoo Pad ran locally on your Mac (eventually it also could sync and run on iOS devices, but it needed a special application to run it). The genius bit about Obsidian is it is just markdown notes with an app that acts as an over watcher to connect and index things, but leaves the markdown notes fully usable by any other app or service that can use Markdown.

Having been taking notes in one directory (and its sub-directories) for some 20 years the ability to always get to my notes and use them is highly valuable. I have run through numerous other apps (particularly cloud based) that just die or go away as they are no longer popular or the owners have the wonderfully tragic combination of being ignorant and arrogant. I can pick-up any of my notes in markdown that have backlinks and they also function in Drafts or other programs. The principle of Small Apps Loosely Joined still has resonance and deep value.

Along with Obsidian and the backlinked notes, I have also been keeping a keen eye on Digital Gardening (Maggie Appleton explains this really well and had links to others also diving deeply). At some point I also stumbled upon Software for your second brain - The Stack Overflow Podcast with Alexander Obenauer talking about his quest for creation of a “personal operating system”, which he shares out in his Lab Notes. Much of Alexander’s quest became refocussed on Obsidian as it was doing a lot of what he needed and was trying to frame out so to build it. He has crated extensions to Obsidian to close some of his perceived gaps, but the underlying principle is data portability and a concept incredibly close to the Small Apps Loosely Joined.

Rebuilding My Note Taking and Management System and Model

The past many weeks I have been digging into a better note taking and management method, while also embracing what I have and my core underlying principles. A continual genre in YouTube I watch is around productivity, particularly around personal knowledge management methods and tools. A couple years back I ran into Zettelkasten Method, that comes from Niklas Luhmann, which focuses on his prolific reading and his card catalogue and related note taking system. Then a few months back I heard Jorge Arango’s interview with Beck Tench it drew Zettelkasten back into focus. The interview with Beck focussed on Tinderbox, which I love, but I also want mobile access to my notes from phone and tablet.

Early Exploration

I have been using Notion a little bit, but my only use the last few months is as an interstitial capture for YouTube and some other rich media. [I like Notion and it seems like a modern take on Podio and has a similar downfall of not sorting out an adaptive data structure for interoperability and consistency.] But, the communities that are interested in Notion became obsessed with Roam Research, so I looked at Roam. Roam and Notion are two vastly different approaches, which can complement each other but in to way replace each other. But, each has a similar faults, no API, no standard export for structured information, and fully cloud based. That is too many common failure points wrapped into one product (Notion is working on and API, which is really good). Roam bugged me most because it relies on an outline format but has no clue about OPML exporting, but worse has no good export model. The cloud based, which requires being connected and online is a model I really don’t like as, particularly if their isn’t a local sync nor standard data format model. What I really like about Roam is its block focussed format, that is akin to purple numbers model of small chunks that are addressable and reusable.

In this time of looking what a next generation of quick note taking would look like, but long used tool, NValt failed spectacularly, in that it would not find my directory where my 1,200+ notes were stored, nor could I add new notes. Fortunately all of my notes are in plain markdown text files, so all I was missing was my tagging of the files in NValt (Brett Terpstra who created NValt has been working on a new tool that can replace NValt but has been taking forever to show up and my need became immediate). This is one of the common reasons for owning my own notes and having them locally and not using somebody else’s model and framework. But, also using the [small apps loosely joined] model where many tools pointing at well formatted / structured data / information can function to their best ability and can use their strengths without breaking anything with the information / data.

Seriously Looking at Note Taking and Management Tools

I started looking at about five or six different note taking tools. I was building out a rough attribute model of tools to help see what each offered or didn’t. I am needing to write this up, but it started with watching Mike and Matty’s, Notion vs Roam vs Obsidian vs Remnote - How to best fit note taking app for you and using their criteria as a base, then building on it. Obsidian and Remnote were already on my list, but also included Zettelnote, Zettlr, and a couple that extended Tidlywiki for a Zettelkasten type model. I also included OmniOutliner as that has been (and will be) my core outlining tool that interplays well with OPML and I can back and forth with good mind mapping tools that also output and import OPML data standard. I also included DevonThink Pro as it is my long used (since 2005) note and information storage and smart search tool (it already was indexing my notes directories) that there is no chance I’m going to give up, but also knew it didn’t have the core functionality I was seeking, wiki-style back linking.

I did a quick test or Roam and ruled it out as it broke rules I try not to break, and it broke many of them (biggest one is know now you are going to exit before you enter anything and a lack of any structure nor API made it a giant risk I’ve been burned by too many times, but the developers have a lot of arrogance about their approach that far too often leads to disasters - sometimes the kindest, smartest, and solid planning people end up with disasters that I feel very badly about but arrogance and ignorant I don’t).

Zettlr and Remnote were next. But the setup took a bit more of me managing and building things and I know when I lose focus those may not be best choices for myself (my past self 15 years ago or more would have loved it and done well with it, but those days are not now).

Obsidian Ticks the Right Boxes and Adapts to My Existing Model

Obsidian is where I put some time. I pointed its “Vault” to my notes directory (and sub-directory) where I had my 1,200 markdown notes already (some of them were .txt extensions, which I did bulk extension swap on) and it could read everything perfectly. One of my first tests was adding backlinks to some of my social lenses and social scaling notes, which worked really well by making related elements connected. I started capturing my notes about what I was doing in Obsidian and the ease of not only connecting things with backlinks, but having the ability to set empty node wiki links (many notes with the same link to a note / page that doesn’t exist yet, but have the same link to it) and then being able to use backlink following from that non-existent notes link list of things pointing to it was insanely valuable.

I have quite a few book list and book note pages already and I started linking them and linking authors and making author pages. I also found I was wanting note page templates for simple book pages in a Zettelkasten model, a book notes template, author / creator template, and a few others. I created these from existing structured notes I’ve used for years and put the outlines in TextExpander using a simple input line or two to label all of the headers with author name or other name.

I started typing out my notes and highlights from books I’ve read and annotated over the years and after the first three or so books I was deeply hooked.

The Use Where Obsidian Showed I was Hooked

Where I knew I was sold was this last weekend I went back to one of Matt Webb’s blog posts on Small Groups that is dense and has links out to great resources. I captured my initial notes on Matt’s post, and annotated relating to his sections. But, I also quickly dug through the linked materials and created and filled out structured note pages for those as well. The James Mullholland post on Small Groups was fantastic and it spidered out to more related resources, so I followed those and took notes. All of this was cross-linked and back-linked and fleshed out small group notes that I have been building as part of social scaling I’ve been writing on and presenting (talks and workshops) for years. The small group size they focus on is roughly team size, but not a team. Both of these are cooperative social models, which scale from teams, groups (small to large groups with similar social interaction models, but the dynamics shift quite a bit around 75 people and break fully about 300 to 500 people), community (everybody inside a firewall or inside an walled off construct), and network (inside and outside a firewall - so for business it is customers, contractors, consultants, vendors, etc. where there needs to be a safe model for sharing information with shared goals as different roles with their purpose come together for back and forth exchange) - more can be found in my related write-up 5 Core Insights for Community Platforms Today.

This note taking and contextualizing and cross linking to rip through and gut a series of related and interrelated pieces has been something I’ve long looked for and wanted. Many dog years ago in college I took reading notes on note cards with citations and context. When writing a paper / essay I would assemble the note cards in an order that could tell a story. Then I would build an outline in WordStar and type in the quotes. Then I would write the narrative and wrapper. Obsidian is starting to get at that, but ripping through a resource to pull out highlights, quotes, annotations, and notes is utterly fantastic. It gives me a solid resource to easily pull together ideas and supporting information.

Other Obsidian Capabilities

Obsidian can show two note pages at once so to easily copy book citation information from the structured book note file into the book note page. The multiple notes in panels also works well for copying quotes to quote pages and cross linking.

Using Obsidian and Still Working from Mobile and Tablet

The mobile use essential had been broken for a bit after Dropbox stopped supporting softlinks in Mac and requiring that to be native in Dropbox and doing the softlink from the Mac to Dropbox. I moved the directory to Dropbox, which leaves a copy locally usable should something happen to Dropbox and added a softlink for local backups. I pointed DevonThink to this directory to index and I was back running. Now I can use Drafts to take a quick note from my iOS devices and push it to the notes directory (later go back and fix the file name) and I have good inbound notes and can use backlinks (which I test later). This method also works for share sheet to Drafts from Overcast or YouTube and having the link to the media and the notes all pulled in.

Happiness with notes has been missing for a while, perhaps happiness has returned.

Resources

• Mike and Matty
• Curtis McHale
• Effective Remote Work
• Mariana’s Study Corner

The Sidebar is the New Back Bar

Back in the days when I had time to hangout with friends for drinks, many places had a “back bar” that was more quiet and private.

I haven’t been in the habit of posting here as much as I used to, but I still am putting things into Pinboard and somethings I flag with “linkfodder”, which is my relatively unique tag to pull out favorites. I use the API from Pinboard to pull the linkfodder tagged items into the sidebar here. Occasionally I annotate them and that is brought here as well. This sidebar mini blog feels sort of like the back bar, which is a little more quiet and calm and you can see some things of interest to me (if that is of interest to you) and track them down.

Diffen for Comparisons

I stumbled across Diffen the other day thanks to Macdrifter’s Diffen mention. I have found it a great service for comparing things. But keep thinking of all the things I really want it to compare, but since it has two fields that allow you to compare any two things you can do it. I may keep the Diffen compares Holland vs Netherlands close at hand as it is a pet peeve and explaining it isn’t quite as easy as pointing to the link (particularly when people say, “well everybody calls it Holland”, the answer is everybody doesn’t and understanding what is correct is easy).

Happy New Year - 2016 Edition

Happy New Year!

I’m believing that 2016 will be a good year, possibly a quite good year. After 7 years of bumpy and 2015 off to a rough start on the health front it stayed rather calm.

I don’t make resolutions for the new year. It is a practice that always delayed good timing of starting new habits and efforts when they were better fits. The, “oh, I start doing this on on New Years” always seemed a bit odd when the moment something strikes you is a perfectly good moment to start down the path to improvement or something new.

This blog has been quiet for a while, far too long in fact. Last year when I was sick it disrupted a good stretch of posting on a nearly daily basis. I really would like to get back to that. I was planning to start back writing over the past couple weeks, but the schedule was a bit filled and chaotic.

Digging Through Digital History

This past year I did a long stretch working as expert witness on a social software case. The case was booted right before trial and decided for the defense (the side I was working with). In doing this I spent a lot of time digging back through the last 5 to 10 years of social software, web, enterprise information management, tagging / folksonomy, and communication. Having this blog at my disposal and my Personal InfoCloud blog were a great help as my practice of knocking out ideas, no matter how rough, proved a great asset. But, it also proved a bit problematic, as a lot of things I liked to were gone from the web. Great ideas of others that sparked something in me were toast. They were not even in the Ineternet Archive’s Wayback Machine. Fortunately, I have a personal archive of things in my DevonThink Pro repository on my laptop that I’ve been tucking thing of potential future interest into since 2005. I have over 50,000 objects tucked away in it and it takes up between 20GB to 30GB on my hard drive.

I have a much larger post brewing on this, which I need to write and I’ve promised quite a few others I would write. The big problem in all of this is there is a lot of good, if not great, thinking gone from the web. It is gone because domains names were not kept, a site changed and dropped old content, blogging platforms disappeared (or weren’t kept up), or people lost interest and just let everything go. The great thinking from the 90s that the web was going to be a repository for all human thinking with great search and archival properties, is pretty much B.S. The web is fragile and not so great at archiving for long stretches. I found HTML is good for capturing content, but PDF proved the best long term (10 years = long term) digital archive for search in DevonThink. The worst has been site with a lot of JavaScript not saved into PDF, but saved as a website. JavaScript is an utter disaster for archiving (I have a quite a few things I tucked away as recently as 18 months ago that are unreadable thanks to JavaScript (older practices and modifications which may be deemed security issues or other changes of mind have functional JavaScript stop working). The practice of putting everything on the web, which can mean putting application front ends and other contrivances up only are making the web far more fragile.

The best is still straight up HMTL and CSS and enhancing from there with JavaScript. The other recent disaster, which is JavaScript related, is infinite scroll and breaking distinct URLs and pages. Infinite scroll is great for its intended use, which is stringing crappy content in one long string so advertisers see many page views. It is manufacturing a false understanding that the content is valued and read. Infinite scroll has little value to a the person reading, other than if the rare case good content is strung together (most sites using infinite scroll do it because the content is rather poor and need to have some means of telling advertisers that they have good readership). For archival purposes most often capturing just the one page you care about gets 2 to 5 others along with it. Linking back to the content you care about many times will not get you back to the distinct article or page because that page doesn’t actually live anywhere. I can’t wait for this dim witted practice to end. The past 3 years or so of thinking I had an article / page of good content I could point to cleanly and archive cleanly was a fallacy if I was trying to archive in the playland of infinite scroll cruft.

Back to Writing Out Loud

This past year of trying to dig out the relatively recent past of 5 to 10 years with some attempts to go back farther reinforced the good (that may be putting it lightly) practice of writing out loud. In the past few years I have been writing a lot still. But, much of this writing has been in notes on my local machines, my own shared repositories that are available to my other devices, or in the past couple years Slack teams. I don’t tend to write short tight pieces as I tend to fill in the traces back to foundations for what I’m thinking and why. A few of the Slack teams (what Slack calls the top level cluster of people in their service) get some of these dumps. I may drop in a thousand or three words in a day across one to four teams, as I am either conveying information or working through an idea or explanation and writing my way through it (writing is more helpful than talking my way through it as I know somebody is taking notes I can refer back to).

A lot of the things I have dropped in not so public channels, nor easily findable again for my self (Slack is brilliantly searchable in a team, but not across teams). When I am thinking about it I will pull these brain dumps into my own notes system that is searchable. If they are well formed I mark them as blogfodder (with a tag as such or in a large outline of other like material) to do something with later. This “do something with later” hasn’t quite materialized as of yet.

Posting these writing out loud efforts in my blogs, and likely also into my Medium area as it has more constant eyes on it than my blogs these days. I tend to syndicate out finished pieces into LinkedIn as well, but LinkedIn isn’t quite the space for thinking out loud as it isn’t the thinking space that Medium or blogs have been and it doesn’t seem to be shifting that way.

Not only have my own resources been really helpful, but in digging through expert witness work I was finding blogs to be great sources for really good thinking (that is where really good thinking was done, this isn’t exactly the case now, unless you consider adding an infinitely redundant cat photo to a blog being really good thinking). A lot of things I find valuable still today are on blogs and people thinking out loud. I really enjoy David Weinberger, Jeremy Keith, and the return of Matt Webb to blogging. There are many others I read regularly (see my links page for more).

30 Days Had November

Well the attempt to blog every day in November didn’t get as much wind in its sails as I had hoped it would have. November had 30 days, but I had far fewer posts than 30. I didn’t get a new habit set. But, I am not yet deterred as I am keeping this attempt going.

One of the things I’ve thought about doing is doing a weekly link dump of items I found of interest through out the week. These are some of my favorite week end reads when I curl up with the laptop (or iPad), a cup of coffee and a blanket. I have long been a fan of Om’s weekend delivery of his 5 (or 7) things to read this weekend as they are nearly always high quality, in depth, and decent (meaning on the longer side) length. I’ve also grown to head toward Josh Ginter of “The Newsprint” weekly “The Sunday Edition” link dump of the week. Recently Michael Sippy has been back sharing on the web (just like the days of yore) and his Filtered Week series on Medium is an annotated link list gem.

One of the reasons I have been considering this (and actually tucking links away for posting) is the past few weeks I realized my blog’s link list on the side of this blog that was posted from Delicious is now dead and gone. The link roll that was a staple of Delicious from 2004 or so was nuked with many other helpful features and functionality that kept Delicious as one part of my workflow. I moved most of my social bookmarking to Pinboard a few years back, but I still fed Delicious and then Delicious fed Pinboard. Well, that was the case until the new owners of Delicious (sadly Delicious gets passed around the IT community like a hot potato with each new owner thinking they are going to update it and improve it, yet lack a rat’s clue of the basics nor how to keep (now kept) the workings that underpin many parts of the web running) fumbled things and Pinboard stopped automatically ingesting Delicious. I really missed the change at Pinboard (was heads down on a project when that happened), but now all my social bookmarking is happening there.

The lack of a link sidebar is the push to share things that I find as gems from the week. I could repoint the Delicious javascript to Pinboard, but Pinboard also captures all my starred / favorited items (I don’t use starring in Twitter as a favorite, but more of a hook for things to hold on to and / or come back to) and things favorited in Instagram, and imports from other services. Pinboard, because I use the paid archive service is an endpoint that serves also as a place to do full text search of the items bookmarked there. Until I can sort out a good filter or means of tagging in my workflow, I am keeping the link list off the sidebar. But, in its place I am likely going to do a weekly link list.

The Data Journalism Handbook is Available

The Data Journalism Handbook is finally available online and soon as the book Data Journalism Handbook - from Amazon or The Data Journalism Handbook - from O’Reilly, which is quite exciting. Why you ask?

In the October of 2010 the Guardian in the UK posted a Data Journalism How To Guide that was fantastic. This was a great resource not only for data journalists, but for anybody who has interest in finding, gathering, assessing, and doing something with the data that is shared found in the world around us. These skill are not the sort of thing that many of us grew up with nor learned in school, nor are taught in most schools today (that is another giant problem). This tutorial taught me a few things that have been of great benefit and filled in gaps I had in my tool bag that was still mostly rusty and built using the tool set I picked up in the mid-90s in grad school in public policy analysis.

In the Fall of 2011 at MozFest in London many data journalist and others of like mind got together to share their knowledge. Out of this gathering was the realization and starting point for the handbook. Journalists are not typically those who have the deep data skills, but if they can learn (not a huge mound to climb) and have it made sensible and relatively easy in bite sized chunk the journalists will be better off.

All of us can benefit from this book in our own hands. Getting to the basics of how gather and think through data and the questions around it, all the way through how to graphically display that data is incredibly beneficial. I know many people who have contributed to this Handbook and think the world of their contributions. Skimming through the version that is one the web I can quickly see this is going to be an essential reference for all, not just journalists, nor bloggers, but for everybody. This could and likely should be the book used in classes in high schools and university information essentials taught first semester first year.

Late to Realizing Ovi Maps Does Exactly What I Wish

I been a big fan of Nokia's mapping solution built into its smart phones, Ovi Maps as it provides the best mobile turn by turn directions I've seen on any mobile device. But, this is largely because Nokia owns Navteq, which has long been the leader for on board mapping and driving solutions.

That FINALLY! Moment Reached

While I have been incredibly impressed with the Ovi mapping on my Nokia E72 device and often use the Ovi resources on the web, I hit that finally, somebody got this right moment with Ovi over the weekend. While, many web mapping solutions allow you to save favorites on the web getting those to sync to your mobile device, with your directions has been left out of most of these solutions (I have been complaining to friends at Google, Yahoo, and elsewhere for many years that this is a no-duh next step). Well, it seems Ovi figured this out quite a while back. (I noticed Google Mobile Maps provided this at the end of 2009, but have never been able to get it to work, even on my supported Symbian device.)

The simplicity and ease with with Nokia's Ovi pulls this off is rather stunning. With this aha moment, I feel like I was the last one to see this and sort it out, but in chats with other mobile maps and navigation users, they have been pained waiting for exactly this functionality, as most people it seems will get a location link and add it to their desktop maps (particularly for travel) but that does them little good as they don't take their desktop or open laptop into the car with them, they take their mobile. Understanding context of use is incredibly valuable.

Now may be a good time to check your device's capability, although iPhone does not seem to have this functionality supported by Google maps (surprised?).

Closing Delicious? Lessons to be Learned

There was a kerfuffle a couple weeks back around Delicious when the social bookmarking service Delicious was marked for end of life by Yahoo, which caused a rather large number I know to go rather nuts. Yahoo, has made the claim that they are not shutting the service down, which only seems like a stall tactic, but perhaps they may actually sell it (many accounts from former Yahoo and Delicious teams have pointed out the difficulties in that, as it was ported to Yahoo’s own services and with their own peculiarities).

Redundancy

Never the less, this brings-up an important point: Redundancy. One lesson I learned many years ago related to the web (heck, related to any thing digital) is it will fail at some point. Cloud based services are not immune and the network connection to those services is often even more problematic. But, one of the tenants of the Personal InfoCloud is it is where you keep your information across trusted services and devices so you have continual and easy access to that information. Part of ensuring that continual access is ensuring redundancy and backing up. Optimally the redundancy or back-up is a usable service that permits ease of continuing use if one resource is not reachable (those sunny days where there's not a cloud to be seen). Performing regular back-ups of your blog posts and other places you post information is valuable. Another option is a central aggregation point (these are long dreamt of and yet to be really implemented well, this is a long brewing interest with many potential resources and conversations).

With regard to Delicious I’ve used redundant services and manually or automatically fed them. I was doing this with Ma.gnol.ia as it was (in part) my redundant social bookmarking service, but I also really liked a lot of its features and functionality (there were great social interaction design elements that were deployed there that were quite brilliant and made the service a real gem). I also used Diigo for a short while, but too many things there drove me crazy and continually broke. A few months back I started using Pinboard, as the private reincarnation of Ma.gnol.ia shut down. I have also used ZooTool, which has more of a visual design community (the community that self-aggregates to a service is an important characteristic to take into account after the viability of the service).

Pinboard has been a real gem as it uses the commonly implemented Delicious API (version 1) as its core API, which means most tools and services built on top of Delicious can be relatively easily ported over with just a change to the URL for source. This was similar for Ma.gnol.ia and other services. But, Pinboard also will continually pull in Delicious postings, so works very well for redundancy sake.

There are some things I quite like about Pinboard (some things I don’t and will get to them) such as the easy integration from Instapaper (anything you star in Instapaper gets sucked into your Pinboard). Pinboard has a rather good mobile web interface (something I loved about Ma.gnol.ia too). Pinboard was started by co-founders of Delicious and so has solid depth of understanding. Pinboard is also a pay service (based on an incremental one time fee and full archive of pages bookmarked (saves a copy of pages), which is great for its longevity as it has some sort of business model (I don’t have faith in the “underpants - something - profit” model) and it works brilliantly for keeping out spammer (another pain point for me with Diigo).

My biggest nit with Pinboard is the space delimited tag terms, which means multi-word tag terms (San Francisco, recent discovery, etc.) are not possible (use of non-alphabetic word delimiters (like underscores, hyphens, and dots) are a really problematic for clarity, easy aggregation with out scripting to disambiguate and assemble relevant related terms, and lack of mainstream user understanding). The lack of easily seeing who is following my shared items, so to find others to potentially follow is something from Delicious I miss.

For now I am still feeding Delicious as my primary source, which is naturally pulled into Pinboard with no extra effort (as it should be with many things), but I'm already looking for a redundancy for Pinboard given the questionable state of Delicious.

The Value of Delicious

Another thing that surfaced with the Delicious end of life (non-official) announcement from Yahoo was the incredible value it has across the web. Not only do people use it and deeply rely on it for storing, contextualizing links/bookmarks with tags and annotations, refinding their own aggregation, and sharing this out easily for others, but use Delicious in a wide variety of different ways. People use Delicious to surface relevant information of interest related to their affinities or work needs, as it is easy to get a feed for not only a person, a tag, but also a person and tag pairing. The immediate responses that sounded serious alarm with news of Delicious demise were those that had built valuable services on top of Delicious. There were many stories about well known publications and services not only programmatically aggregating potentially relevant and tangential information for research in ad hoc and relatively real time, but also sharing out of links for others. Some use Delicious to easily build “related information” resources for their web publications and offerings. One example is emoted by Marshall Kirkpatrick of ReadWriteWeb wonderfully describing their reliance on Delicious

It was clear very quickly that Yahoo is sitting on a real backbone of many things on the web, not the toy product some in Yahoo management seemed to think it was. The value of Delicious to Yahoo seemingly diminished greatly after they themselves were no longer in the search marketplace. Silently confirmed hunches that Delicious was used as fodder to greatly influence search algorithms for highly potential synonyms and related web content that is stored by explicit interest (a much higher value than inferred interest) made Delicious a quite valued property while it ran its own search property.

For ease of finding me (should you wish) on Pinboard I am http://pinboard.in/u:vanderwal

----

Good relevant posts from others:

• Why You Should have a Website - Jeremy Keith
• We can save Delicious, but probably not in the way you think
• Delicious in Purgatory :: TechCrunch

As If Had Read

The idea of a tag "As If Had Read" started as a riff off of riffs with David Weinberger at Reboot 2008 regarding the "to read" tag that is prevalent in many social bookmarking sites. But, the "as if had read" is not as tongue-in-cheek at the moment, but is a moment of ah ha!

I have been using DevonThink on my Mac for 5 or more years. It is a document, note, web page, and general content catch all that is easily searched. But, it also pulls out relevance to other items that it sees as relevant. The connections it makes are often quite impressive.

My Info Churning Patterns

I have promised for quite a few years that I would write-up how I work through my inbound content. This process changes a lot, but it is back to a settled state again (mostly). Going back 10 years or more I would go through my links page and check all of the links on it (it was 75 to 100 links at that point) to see if there was something new or of interest.

But, that changed to using a feedreader (I used and am back to using Net News Wire on Mac as it has the features I love and it is fast and I can skim 4x to 5x the content I can in Google Reader (interface and design matters)) to pull in 400 or more RSS feeds that I would triage. I would skim the new (bold) titles and skim the content in the reader, if it was of potential interest I open the link into a browser tab in the background and just churn through the skimming of the 1,000 to 1,400 new items each night. Then I would open the browser to read the tabs. At this stage I actually read the content and if part way through it I don't think it has current or future value I close the tab. But, in about 90 minutes I could triage through 1,200 to 1,400 new RSS feed items, get 30 to 70 potential items of value open in tabs in a browser, and get this down to a usual 5 to 12 items of current or future value. Yes, in 90 minutes (keeping focus to sort the out the chaff is essential). But, from this point I would blog or at least put these items into Delicious and/or Ma.gnolia or Yahoo MyWeb 2.0 (this service was insanely amazing and was years ahead of its time and I will write-up its value).

The volume and tools have changed over time. Today the same number of feeds (approximately 400) turn out 500 to 800 new items each day. I now post less to Delicious and opt for DevonThink for 25 to 40 items each day. I stopped using DevonThink (DT) and opted for Yojimbo and then Together.app as they had tagging and I could add my context (I found my own context had more value than DevonThink's contextual relevance engine). But, when DevonThink added tagging it became an optimal service and I added my archives from Together and now use DT a lot.

Relevance of As if Had Read

But, one of the things I have been finding is I can not only search within the content of items in DT, but I can quickly aggregate related items by tag (work projects, long writing projects, etc.). But, its incredible value is how it has changed my information triage and process. I am now taking those 30 to 40 tabs and doing a more in depth read, but only rarely reading the full content, unless it is current value is high or the content is compelling. I am acting on the content more quickly and putting it into DT. When I need to recall information I use the search to find content and then pull related content closer. I not only have the item I was seeking, but have other related content that adds depth and breath to a subject. My own personal recall of the content is enough to start a search that will find what I was seeking with relative ease. But, were I did a deeper skim read in the past I will now do a deeper read of the prime focus. My augmented recall with the brilliance of DevonThink works just as well as if I had read the content deeply the first time.

Social Design for the Enterprise Workshop in Washington, DC Area

I am finally bringing workshop to my home base, the Washington, DC area. I am putting on a my “Social Design for the Enterprise” half-day workshop on the afternoon of July 17th at Viget Labs (register from this prior link).

Yes, it is a Friday in the Summer in Washington, DC area. This is the filter to sort out who really wants to improve what they offer and how successful they want their products and solutions to be.

Past Attendees have Said...

“A few hours and a few hundred dollar saved us tens of thousands, if not well into six figures dollars of value through improving our understanding” (Global insurance company intranet director)

From an in-house workshop…
“We are only an hour in, can we stop? We need to get many more people here to hear this as we have been on the wrong path as an organization” (National consumer service provider)

“Can you let us know when you give this again as we need our [big consulting firm] here, they need to hear that this is the path and focus we need” (Fortune 100 company senior manager for collaboration platforms)

“In the last 15 minutes what you walked us through helped us understand a problem we have had for 2 years and a provided manner to think about it in a way we can finally move forward and solve it” (CEO social tool product company)

Is the Workshop Only for Designers?

No, the workshop is aimed at a broad audience. The focus of the workshop gets beyond the tools’ features and functionality to provide understanding of the other elements that make a giant difference in adoption, use, and value derived by people using and the system owners.

The workshop is for user experience designers (information architects, interaction designers, social interaction designers, etc.), developers, product managers, buyers, implementers, and those with social tools running already running.

Not Only for Enterprise

This workshop with address problems for designing social tools for much better adoption in the enterprise (in-house use in business, government, & non-profit), but web facing social tools.

The Workshop will Address…

Designing for social comfort requires understanding how people interact in a non-mediated environment and what realities that we know from that understanding must we include in our design and development for use and adoption of our digital social tools if we want optimal adoption and use.

• Tools do not need to be constrained by accepting the 1-9-90 myth.
• Understanding the social build order and how to use that to identify gaps that need design solutions
• Social comfort as a key component
• Matrix of Perception to better understanding who the use types are and how deeply the use the tool so to build to their needs and delivering much greater value for them, which leads to improved use and adoption
• Using the for elements for enterprise social tool success (as well as web facing) to better understand where and how to focus understanding gaps and needs for improvement.
• Ways user experience design can be implemented to increase adoption, use, and value
• How social design needs are different from Web 2.0 and what Web 2.0 could improve with this understanding

More info...

For more information and registration to to Viget Lab's Social Design for the Enterprise page.

I look forward to seeing you there.

Tale of Two Tunnels: Web 2.0 and Enterprise 2.0

Yesterday I made a few comments in Twitter that prompted a fair amount of questions and requests for more information. The quips I made were about the differences between Web 2.0 (yes, an ambiguous term) and Enterprise 2.0 (equally ambiguous term both for the definition of enterprise and the 2.0 bit). My comments were in response to Bruce Stewart's comment The whole "Enterprise 2.0" schtick is wearing thin, unless you've been monitoring real results. Otherwise you're just pumping technology.. In part I agree, but I am really seeing things still are really early in the emergence cycle and there is still much need for understanding of the social tools and the need for them, as well as how they fit in. There are many that are selling the tools as technologies with great promise. We have seen the magic pill continually pitched and bought through out the history of business tools. (For those new to the game or only been paying attention for the last 15 years, a huge hint, THERE IS NO MAGIC PILL).

Tale of 2 Tunnels

One comment I made yesterday is, "the difference between Web 2.0 and Enterprise 2.0 is like the difference building a tunnel through rock and tunnel under water".

That this is getting at is Web 2.0 takes work to build to get through the earth, but once built it can suffer from imperfections and still work well. The tunnel can crack and crumble a little, but still get used with diminished capacity. We can look at Facebook, which has a rather poor interface and still gets used. Twitter is another example of a Web 2.0 solution that has its structural deficiencies and outages, but it still used as well as still loved (their Fail Whale is on a t-shirt now and a badge of pride worn by loyal users).

The Enterprise 2.0 tunnel is built under water. This takes more engineering understanding, but it also requires more fault testing and assurances. A crack or crumbling of a tool inside an organization is not seen kindly and raises doubts around the viability of the tool. The shear volume of users inside an organization using these tools is orders of magnitude less than in the open consumer web world, but faults are more deadly.

The other important factor is perceived fear of the environment. Fewer people (by pure numbers - as the percentages are likely the same, more on this later) are fearful of tunnels through land, they may not have full faith in them, but they know that they will likely make it safely on all of their journeys. The tunnels under water have greater fears as one little crack can cause flooding and drowning quickly. Fears of use of social tools inside an organization is often quite similar, there may be many that are not fearful, but if you spend time talking to people in organizations not using tools (it is the majority at this point) they are fearful of open sharing as that could lead to trouble. People are not comfortable with the concept as they are foreign to it as they are lacking the conceptual models to let them think through it.

Enterprise 2.0 is not Web 2.0

Another statement yesterday that garnered a lot of feedback was, "Web 2.0 does not work well in enterprise, but the approaches and understandings of Web 2.0 modified for enterprise work really well." The web is not enterprise or smaller organizations for that matter. The open consumer web has different scale and needs than inside organizations and through their firewalls. A small percentage of people using the web can get an account on a tool have have appear to be wildly successful correctly claiming 70 million or 100 million people are or have used their tool. But, even 100 million people is a small percentage of people using the web. Looking at real usage and needs for those tools the numbers are really smaller. Most darlings of the Web 2.0 phase have fewer than 10 million users, which is about 5% of the open consumer web users in the United States. On the web a start-up is seen as successful with 500,000 users after a year or two and is likely to have the capability to be self sufficient at that level too. Granted there are many players in the same market niches on the web and the overall usage for link sharing and recommending for Digg, Mixx, or Reddit is much higher across the sum of these tools than in just one of these tools (obviously).

These percentages of adoption and use inside organizations can make executives nervous that their money is not reaching as many employees as they wish. The percentages that can be similar to the web's percentages of high single digit adoption rates to the teens is seen as something that really needs more thinking and consideration.

Enterprise 2.0 is more than just tools (see my Enterprise Social Tools: Components for Success for better understanding) as it also includes interface/interaction design for ease of use, sociality, and encouragement of use. The two biggest factors that are needed inside an organization that can receive less attention on the web are the sociality and encouragement of use.

Understanding sociality is incredibly important inside an organization as people are used to working in groups (often vertical in their hierarchy) that have been dictated to them for use. When the walls are broken down and people are self-finding others with similar interests and working horizontally and diagonally connecting and sharing with others and consuming the collective flows of information their comfortable walls of understanding are gone. A presentation in Copenhagen at Reboot on Freely Seeping Through the Walls of the Garden focussed just on this issue. This fear inside the enterprise is real. Much of the fear is driven by lacking conceptual models and understanding the value they will derive from using the tools and services. People need to know who the other people are that they are sharing with and what their motivations are (to some degree) before they have comfort in sharing themselves.

Encouraging use is also central to increased adoption inside organizations. Many organizations initial believe that Web 2.0 tools will take off and have great adoption inside an organization. But, this is not a "build it and they will come" scenario, even for the younger workers who are believed to love these tools and services and will not stay in a company that does not have them. The reality is the tools need selling their use, value derived from them, the conceptual models around what they do, and easing fears. Adoption rates grow far beyond the teen percentages in organizations that take time guiding people about the use of the tools and services. Those organizations that take the opportunity to continually sell the value and use for these tools they have in place get much higher adoption and continued engagement with the tools than those who do nothing and see what happens.

Gaps in Enterprise Tools

The last related statement was around the gaps in current and traditional enterprise tools. At the fantastic Jive Enterprise UI Summit in Aspen a few weeks ago there was a lot of discussion about enterprise tools, their UI, and ease of use for employees by the incredible collection of people at the event. One of the things that was shown was a killer path of use through a wide encompassing enterprise toolset that was well designed and presented by SAP's Dan Rosenberg who has done an incredible job of putting user experience and thinking through the needed workflows and uses of enterprise tools at the forefront of enterprise software planning. Given the excellent design and incredible amount of user experience thought that went into the tools behind the SAP toolset in the scenario (one of the best I have seen - functioning or blue sky demoed) there are still gaps. Part of this is identifying of gaps comes from traditional business thinking around formal processes and the tools ensure process adherence. But, the reality is the tools are quite often inflexible (I am not talking about SAP tools, but traditional enterprise tools in general), the cost of time and effort is beyond the gain for individuals to document and annotate all decisions and steps along the way. The hurdles to capture information and share it are often too large for capturing one to 10 quick sentences of information that can be retained for one's own benefit or shared with other where it is relevant.

There is another gap in business around the collective intelligence that is needed, which can lead to collaboration. Most businesses and their tools focus on collaboration and set groups, but at the same time wonder why they do not know what their company knows and knowledge is not all being captured. First there is a difference between collective and collaborative activities and the tools and design around and for those different activities is more than a nuance of semantics it is a huge barrier to capturing, sharing, and learning from information that leads to knowledge if it is not understood well. Enterprise has gone through its phases of knowledge management tools, from forms for capturing information, forums for sharing, and up to enterprise content management systems (ECM) that encompass document management, content management, knowledge management, and information harvesting. But, the gaps still exist.

These existing gaps are around conversations not being captured (the walls of the halls have no memory (well today they do not)) and increasingly the ubiquitous communication channel in organizations, e-mail, is being worked around. Quick decisions are not being documented as it is not enough for a document or worth completing a form. As the iterative processes of development, design, and solution engineering are happening at quicker and smaller increments the intelligence behind the decisions is not being captured or shared. This is largely because of the tools.

As has always been the case large enterprise systems are worked around through the use of smaller and more nimble solutions that augment the existing tools. Even in Dan's incredible demo I saw gaps for these tools. The quick tools that can fill these gaps are blogs, wikis, social bookmarking, tagging, Twitter type sharing, Veodia type video sharing, instant messaging, etc. There are many avenues to quickly capture information and understanding and share it. These tools get out of the way and allow what is in someone's head to get digitized and later structured by the individual themselves or other people whom have had the information shared with them in a community space. This turns into flows through streams that can be put into many contexts and needs as well as reused as needed.

Another point Dan stated at the Enterprise UI Summit that is dead on, is organizations are moving out of the vertical structures and moving to the horizontal. This is having a profound effect on the next generation of business tools and processes. This is also an area for Enterprise 2.0 tools as they easily open up the horizontal and diagonal prospects and tie into it the capability for easily understanding who these newly found people are in an organization through looking at their profiles, which eases their fears around sharing and unfamiliar environments as well as their related tasks.

[Comments are open and moderated at Tale of Two Tunnels: Web 2.0 and Enterprise 2.0 :: Personal InfoCloud]

"Building the social web" Full-day Workshop in Copenhagen on June 30th

Through the wonderful cosponsoring of FatDUX I am going to be putting on a full-day workshop Building the Social Web on June 30th in Copenhagen, Denmark (the event is actually in Osterbro). This is the Monday following Reboot, where I will be presenting.

I am excited about the workshop as it will be including much of my work from the past nine months on setting social foundations for successful services, both on the web and inside organizations on the intranet. The workshop will help those who are considering, planning, or already working on social sites to improve the success of the services by providing frameworks that help evaluating and guiding the social interactions on the services.

Space is limited for this workshop to 15 seats and after its announcement yesterday there are only 10 seats left as of this moment.

Lasting Value of Techmeme?

Tristan Louis has a post looking at Is Techmeme Myopic, which is a good look at the lasting value that Techmeme surfaces. This mirrors my use of Techmeme, which is mostly to have a glance at what is being discussed each day. Techmeme has a very temporal value for me as it is a zeitgeist tool that tracks news and memes as they flow and ebb through the technology news realm of the web.

Personally, I like Techmeme as it aggregates the news and conversations, which makes it really easy to skip the bits I do not care about (this is much of what flows through it), but I can focus on the few things that do resonate with me. I can also see a view of who is providing content and I can select sources of information whose viewpoint I value over perspectives I personally place less respect.

Enterprise 2.0 Boston - After Noah: What to do After the Flood (of Information)

I am looking forward to being at the Enterprise 2.0 Conference in Boston from June 10 to June 12, 2008. I am going to be presenting on June 10, 2008 at 1pm on After Noah: Making Sense of the Flood (of Information). This presentation looks at what to expect with social bookmarking tools inside an organization as they scale and mature. It also looks at how to manage the growth as well as encourage the growth.

Last year at the same Enterprise 2.0 conference I presented on Bottom-up Tagging (the presentation is found at Slideshare, Bottom-up All the Way Down: How Tags Help Businesses Organize, which has had over 8,800 viewing on Slideshare), which was more of a foundation presentation, but many in the audience were already running social bookmarking services in-house or trying them in some manner. This year my presentation is for those with an understanding of what social bookmarking and folksonomy are and are looking for what to expect and how to manage what is happening or will be coming along. I will be covering how to manage heavy growth as well as how to increase adoption so there is heavy usage to manage.

I look forward to seeing you there. Please say hello, if you get a chance.