What Data-Driven Thinking Has Shaped My Thinking About Building Well

Wiki Article

AI Is Only As Good As The Culture It's And Is A Part Of
The discussion about artificial intelligence in the business world has a problem which isn't technical. The technological capabilities of current AI and machine learning technologies are remarkable, growing at a rate that renders the majority of forecasts of the place they'll be in eighteen months obsolete even before they have passed. The issue is the gap between the what AI can do in well-controlled conditions - in a appropriately-funded research lab, with uncluttered data, with precise problem description, and engineers who are capable of testing the system until it does what it is supposed to do - and what it can actually deliver when it is implemented inside real organizations with real cultures with real political and organisational dynamics and real people who have an established view of the validity of a brand new system as something worth engaging with or something to navigate around in order to maintain the appearance of compliance. I've been developing with technology for machine learning long before this current flurry of AI enthusiasm paved the way and commonplace for companies to claim fluency in the space. When I founded 1Touch an AI-driven platform, AI-driven matchmaking and recommendation systems were not something we were able to add to make the product more appealing to investors. They were at the heart structure of the product's architecture. They were the method by which the platform created value, as well as the feature that had to function reliably and at high-quality for the company to remain viable. This means I've got direct, experienced what happens when you try to construct something that is truly intelligent to a product and an organisation simultaneously and the thing I keep returning to, across every context in where I've encountered this difficulty, is the technology itself is rarely the factor that limits your success. The main factor that limits the possibilities is almost all the time its culture.
What I mean by that is specific and concrete, not abstract. AI systems require data to perform - clear, consistent structured data that conveys the phenomenon that the system is trying to analyze and make predictions about. Data-driven organizations with a strong culture produce that kind of data in the natural course from their operations. They have clearly defined and consistently implemented definitions of what they are monitoring and why. They have reached an agreement on the way data is recorded, collected, and stored. They have accountability frameworks that place data quality as an explicit accountability, rather than a vague motives. Organizations that do not have strong data culture produce something that technically looks like data. It's there in systems and is accessible for query, and it is used for charting - but is so ambiguous in its definition, and therefore variable in quality, and so full of irregularities in the structure and unmapped exceptions that any AI device built on over it will enhance and reflect the confusion instead of getting a true signals from it. Organizations in that class often do not even know what they are doing until they are well into the process of implementing an AI implementation and its outputs do not meet the vendor's promises, at which point it is tempting to blame the technology. But in reality, the problem lies in the cultural and operational infrastructure which the technology was based on.

Another dimension of culture that will determine AI outcomes is openness within the organisation - - the degree to which members of the organisation are willing to let systems inform or change how they work instead of viewing it as the threat to their own professional competence, their authority in the institution or even their job security. This is a socio-cultural and leadership problem as opposed to a technical one that is a problem that starts at the highest level. If leaders in the top ranks engage with AI outputs only when they are satisfied - accepting the results that affirm what they believed before and disadvantaging those that do and do not, this behaviour sends the impression to everyone who watches that the firm's pledge to data-driven decision-making is conditional rather than true, and this can spread throughout the organisation more quickly than any program of training or change management initiative can reverse. If senior leaders exhibit authentic, consistent engagement AI outputs, and demonstrate the reluctance to alter their choices when evidence suggests they should, the collective capability to utilize AI effectively will improve dramatically and is able to be done so quickly.

This isn't an abstract idea of how organizations should be conducted in the context of theory. It is a description of the pattern I've seen unfold in numerous organizations with significant finances, real strategic dedication to AI adoption, and leader teams who were passionate about the potential of AI technology. The pattern is consistent enough that I have decided to consider data governance practices as the primary diagnostic question when evaluating an organisation's AI capability. Before I ask what the current technology stack is, before I ask about what specific application cases the organisation is considering, I ask about the governance of data. How does the organization define its most important metrics? Who's responsible if information quality is not good enough? If two areas have conflicting data concerning the same business reality, and how can these conflicts be resolved? The answers to these questions provide more information about the likelihood of AI success than any of the discussions about algorithms, platforms or timelines for implementation.

I believe that the organizations that will reap the most lasting value from AI over the next decade will not be those that embrace the latest technology first, or the ones that will invest significantly in AI infrastructure and human resources in the near-term. They are the ones who are able to establish the social and operational base to use the technology to its fullest extent - the data governance methods that produce solid inputs, the decision making frameworks that offer evidence to influence outcomes and the management behaviours which signal to all people in the organisation that the commitment to a data-driven approach is a fact rather than an arbitrary. The technology itself will be increasingly affordable and accessible. The right culture to use it well will remain scarce, because it requires sustained dedication and effort from leaders over time instead of the simple decision of a strategic leader or an investment in technology. The scarcity of it is where the significant competitive advantage will be and is an benefit that, once cultivated is able to grow in a way which only technological advantages will. See the James Deller for site info including what building companies sharpened my thinking on culture about teams.



It's The Data Infrastructure Problem Nobody Wants To Talk About
Every company I've worked closely with over the past one and a half years - whether as a founder, an investor or operational advisor I have been told, at some point in the relationship, that information plays a major role in the way they make decisions. Some of them have truly believed this in a way that will be evident in the way the company actually runs. Most of them believe they're saying this, but what they're discussing is an aspiration, not being a reality in operation - an image of the organization they're trying to create as opposed to the reality they're currently living. The gap between truly data-driven decision-making and the performance of data-driven decisions - the meticulous maintenance of the appearance on the outside of an evidence-based decision-making without the infrastructure that would make it an actual reality - is among many of the most significant gaps found in modern business. It's also one of the areas that remain unaddressed due to the infrastructure issues that cause it is genuinely unglamorous to discuss, challenging to prove to stakeholders outside of the company, and enormously difficult to determine the best way to address it in comparison to the more visible commercial and strategic activities that demand the same attention of leaders and organisational resources.
When organizations talk about data strategy, they typically tend to discuss what capabilities they'd like to develop on top of your data - the tools for analysis, machine-learning applications with real-time operational dashboards which provide the kinds of a predictive information that sounds truly compelling in the form of a presentation for board members or an investor update. What they talk about less often and with a lesser amount of energy and enthusiasm, are the core infrastructure that decides if all of those capabilities are actually working according to the specifications: the data management frameworks that give distinct and consistent definitions of what's being measured and the reasons for it; the collection and storage techniques that assess the quality and comparability of data which is being stored; quality assurance processes that identify the errors and correct them prior to they are propagated through the system and corrupt the outputs that everybody is relying on; the organisational structures and accountability processes that make data quality an ongoing and explicit responsibility instead of relying on everyone's vague and unenforceable goals. The plumbing, also known as. It is not glamorous. It's not easy to photograph to be used in an annual report. It does not produce outputs that could be showcased in a compelling presentation. And, in my experience in a vast number of companies in different areas and at various stages in their development, considerably worse than the organization believes it is.

The issue continues to grow over time by becoming harder and costlier to fix. An organization that has been operating using inconsistent or unclear terminology for data across different areas for three years now has three years in historical data which cannot be easily compared or aggregated, not because the information doesn't exist, however because the same language has been used to denote different terms in different parts of the organisation, and those differences are built into the data itself rather than being apparent on the surface. An organisation whose data quality assurance has been the responsibility of a minor responsibility instead of being a fully resourced and dedicated function is a victim of data whose accuracy is varying in ways that are not documented, and thus cannot be easily accounted when the data is used to determine the outcome. A company that allows multiple operational processes to accumulate overlapping or partial conflicting records on the same products, customers and transactions has a data landscape that's real difficult to address without causing a significant disruption in operation to create a risk.

The reason this issue is present throughout a variety of companies that are really smart about strategy and genuinely dedicated to a data-driven approach to business is because addressing it requires an ongoing commitment to work that produces no visible results in the short term that processes for resource allocation in organisations are designed to reward. Analytics platforms that are new produce tangible outputs, such as dashboards that can be displayed as well as reports that are shared with the board and also insights that can be translated into press releases on digital transformation. A data governance programme produces invisible infrastructure - cleaner underlying definitions that are more consistent with the collection process as well as more reliable inputs to system that was already in existence. It is the first to present in a budget argument since you are able to show people what they will gain. The second needs someone with sufficient organisational credibility and a willingness to argue you believe that this infrastructure initiative will eventually result in better outcomes for every capabilities that are built on top it. It's compelling in the abstract but it can be difficult to make in a competitive environment with initiatives whose benefits will be more tangible and clear.

I've made the case in various organizational contexts and watched it work or fail based on evident reasons, that I can have an extremely clear understanding of what determines whether an organization finally tackles its data infrastructure problem or is able to continue delaying it. It is generally a leader - a specific individual with enough organisational credibility with a deep comprehension of why the infrastructure is crucial, and enough determination to continue making this argument till it is an absolute priority, rather than simply a part of the list of things everyone recognizes as important yet never get to the top. The leader must be willing to accept the immediate cost of the infrastructure investment - the duration, the disruption to existing processes, or the absence evidence of output immediately measurable - in the confidence that the ability it will create will justify the expense by several times. What that requires, ultimately is a culture which long-term infrastructure investment is highly valued and recognized at the high-level of leadership, not only listed in strategy documents but is then systematically relegated to the back burner when the quarterly discussion on resource allocation takes place. Making that change is, in itself, a long-term commitment. It is, however, in my view, one the most rewarding investments an organization who is serious about a data-driven operations can make.}