Notes on seeking wisdom and crafting software

Paper - The Wisdom Hierarchy

We consume an average of 34GB information every day at a rate of 240 words/minute (as per a 2009 report1). It is imperative to understand what information means. What happens to the information we consume? How does it relate to Knowledge and Wisdom? It turns out they are connected and possibly form a hierarchy. We’ll review a paper that dives deep into the taxonomy of Wisdom.

Rowley, Jennifer. “The wisdom hierarchy: representations of the DIKW hierarchy.” Journal of information science 33, no. 2 (2007): 163-180.

Our goal is to understand the difference between data and wisdom. Use the definition to examine if we can relate, classify and evaluate them.

Motivation

Data, Information, Knowledge and Wisdom (DIKW) hierarchy is a central model of information management, information systems and knowledge management. The paper attempts to analyze various articulations of this hierarchy, its meaning and contribution.

Note that Wisdom is the pinnacle of DIKW hierarchy. We will try to learn if there’s a path to That.

Definitions

There are two sources of literature related to our subject. Information Philosophy investigates the conceptual nature and basic principles of information. Knowledge Management deals with the nature of knowledge and knowledge processes. The paper reviews critical literature and expounds on the key aspects presented by them. While we definitely recommend a read, the summary below only highlights key takeaways.

We need to focus on a few things as we internalize the hierarchy (or layers). Can a higher layer be explained from lower layer? What is the transformation process involved to move from layer to another? Is a higher layer transformation or composition of previous layer with additional ingredients?

  • Data is a set of signals, or facts which represent an observation
    • Unorganized, unprocessed and without meaning
    • Note that definitions of Data tell what data is not
  • Information is organized and structured data
    • Makes data meaningful, valuable, relevant and useful
    • Requires interpretation of contents of data within some context
    • Notion of meaning is subjective. Same data can lead to different information and thus can be of varying significance to different individuals
    • How to convert data to information? Classify, Rearrange, Sort, Aggregate, Calculate or Select
  • Knowledge is justified personal belief (Plato?) that increases an individual’s capacity to take effective action
    • Conveys understanding, experience, accumulated learning and expertise in the context of a problem or activity
    • Knowledge is formed from information or data via either a) conversion, or b) enhancement with additional ingredients
    • How to convert information to knowledge? Through belief structuring and formation of justified true beliefs about the world. Use prior knowledge to make sense of received information. Link them with existing knowledge by firmly characterized or vague associations. Internalize new insights from these relationships.
    • Knowledge increases scope of information by including perception skills, training, common sense and experience (additional ingredients).
    • Ranges from know how (tacit) to know what (explicit)
      • Tacit knowledge is embedded in an individual mind through experience and jobs based on personal belief, perspective and values
      • Explicit knowledge is codified in books, reports or papers etc. Designed for sharing
  • Wisdom is the highest level of abstraction
    • Accumulated knowledge that enables application of concepts from one domain to new situations or problems
    • Provides the ability to see beyond the obvious (vision foresight)
    • Based on ethical judgement related to individual’s belief system

Discussion

The paper raises interesting points regarding the relationships and transformation between data, information and knowledge.

Relationships

Understanding is not a separate level in the hierarchy. Moving from data to information requires understanding relations. Information to knowledge requires understanding patterns. And knowledge to wisdom requires understanding principles. Nature of understanding grows subtle as we move from data to wisdom.

Data, Information and Knowledge can be defined in terms of one another. Knowledge can be formed from both information and data through transformation or composition.

Transformations

Data is foundational and raw. Data to information requires a process of interpretation and adding meaning. Simply adding more data to existing data in an objective or quantifiable sense may not lead to information unless it is processed.

All data that is collected, whether in information systems, or in our minds implicitly has some structure as soon as it is collected. Applying an explicit schema to raw data adds certain amount of context and structure. However, whether the schematized data is meaningful to an individual or an organization depends on the alignment between data structure and the cognitive schema of the individual.

Thus, all that can be held in information systems is data, not the meaning or information in that sense2.

Relation between information and knowledge is not concrete. If information requires giving meaning to data, it implies that understanding is also necessary in this transformation. Thus, understanding is not a differentiator between the transformation of data to information and information to knowledge.

We defined explicit knowledge to be codified in books, documents, or systems. Given that knowledge is a function of prior understanding, experience and learning; how is explicit knowledge any different than information? It always takes an individual to create the relevant links to their prior knowledge in order to create new insights.

Variables

The paper defines a few variables and plots their variance as we move across the wisdom hierarchy.

  • Meaning and value grows as we move from data to wisdom
  • Human input is essential navigation from data to wisdom. Computer input reduces
  • Algorithmicity and programmability reduce as we move from data to wisdom
  • Order, structure and human agency grows as we move from data to wisdom

Can we conclude that there is a distinct division between data, information and knowledge? Or are they part of a single continuum with different levels of meaning, structure and actionability occurring on a set of facts? In terms of representation, the paper maps data to transaction processing systems. Management information systems deal with information. Decision support systems handle knowledge. And wisdom is the realm of Expert systems.

Wisdom remains an elusive concept. Some authors argue that wisdom is not about what is known rather it indicates how the known is held and put to use. It may also be defined at a meta level as the approach or attitude towards beliefs, values, knowledge, information, abilities and skills. The paper proposes upending the hierarchy in terms of abundance. Data is infinitely available and perceived whereas wisdom is rare. There is a link between knowledge and wisdom in that sense. Perhaps wisdom is indicated by the judgment, actions, dispositions and attitude of an individual living the knowledge.

Footnotes

  1. Based on 2009 data, an American consumes 34GB of data/day. Our throughput to consume information (240 words/min) changes slowly year over year, however the time spent on consumption increases steadily. Paper measures artificial form of consumption. Our interactions with environment has a much higher bandwidth (high fidelity channel) and thus we consume about 135GB/day from experiences.

    Bohn, Roger E., and James E. Short. How Much Information?: 2009 Report on American Consumers. Global Information Industry Center, University of California, San Diego, 2009.

  2. It is worth calling out that we have since developed ways to encode meaning and semantics, e.g. as vectors. They are widely used for inference in semantic search e.g. matching documents for a query based on meaning and understanding instead of words. Yet the definition of semantics is not the same as significance. Even if the most relevant documents are fetched by a search engine for my query, the importance of the results varies from individual to individual. May be collaborative filtering or similar personalization could help, but we’re definitely not there.