Central Source of Truth
Choosing the right path to creating a central source of truth
Liran Cohen
January 16, 2025
Recently I had an interesting conversation with a VP Product of a successful startup. We discussed data challenges from his point of view and one of the issues he brought up was the lack of clarity on what the data really means.
“Recently I found out that one of our key metrics was calculated differently than what we thought it would. It definitely affected our decisions."
Data is at the core of each business, and business logic is what turns data into meaningful information.
But information is only useful if it’s accessible, easy to understand, trusted, and has vast coverage of the possible questions people can ask.
Information systems are powerful when they meet four key criteria:
Anyone should be able to easily access data, including non technical business personas. People should be able to know which information is available to them and what it means.
While trust in data is a well known and covered topic with a lot of tooling around data quality, business logic quality is often overlooked. We need to make sure that our business logic is well structured and governed - to reduce duplications and increase trust.
A long-known challenge with data is that the business moves faster than data teams - which creates a gap between the business needs and the available data. An excellent data system is flexible enough to enable users to consume information in a flexible way, even if not previously well defined, yet still keep the process governed and trusted. We believe that is now possible, with the rise of AI and LLMs.
As organizations grow, so does the complexity of their data needs. A great information system is not just designed for today but is built with scalability and evolution in mind. It should be able to integrate new data sources, adapt to changing business processes, and incorporate emerging technologies seamlessly. This ensures that the system remains relevant and valuable over time.
To date, most data teams still implement their business logic in BI tools
BI tools are designed with analysts and BI developers in mind. Adding business logic is simple and intuitive, allowing data teams to build dashboards quickly and efficiently. By connecting raw data or pre-aggregated tables, users can start visualizing insights almost instantly, streamlining the creation process.
BI tools are both context-aware and aggregate-aware. Measures defined within these tools can be effortlessly sliced, filtered, and rolled up in real time, enabling users to explore data dynamically and uncover insights from multiple perspectives.
While BI tools provide flexibility, using them as the primary home for business logic introduces significant challenges:
BI tools often operate in isolation, lacking a centralized source of truth for business logic. This leads to inconsistencies across dashboards and data products, creating confusion and eroding trust. Without alignment, data teams waste valuable time duplicating efforts and reconciling discrepancies instead of driving impactful insights.
Business definitions embedded within BI tools are often siloed and inaccessible to non-technical stakeholders. This creates dependencies between business and data teams, increasing the volume of ad-hoc requests and delaying time-to-insight. Without transparent and shared definitions, collaboration suffers, and decision-making slows down.
Housing business logic within a specific BI tool creates a dependency on that tool’s ecosystem. This poses a major problem when teams need to apply the same business logic across diverse use cases, such as embedded analytics, operational workflows, or AI applications. Vendor lock-in limits flexibility and makes migrating to another platform difficult and costly, as the logic is deeply fragmented and tightly integrated into the current tool.
Business definitions materialized as SQL code is a common practice in the Shift-Left modern paradigm led by tools like dbt.
SQL allows business logic to be written as code, making it version-controlled, testable, and deployable. This ensures consistency, reduces errors, and enables a structured approach to managing changes. By treating business logic like software, data teams can collaborate effectively and maintain high standards of quality and reliability.
SQL is the universal language of data, widely understood and used by data teams across industries. Its ubiquity and familiarity make it a natural choice for building data solutions. Tools like dbt further enhance this experience by enabling clear project organization, modularity, and the ability to trace data lineage easily, helping teams understand how data flows and transforms within their systems.
Business definitions embedded in SQL code are often hidden from non-technical stakeholders. This creates dependencies between business and data teams, leading to a constant stream of ad-hoc requests. The resulting bottlenecks delay time-to-insight and reduce the overall efficiency of decision-making processes.
Pre-aggregated business definitions in SQL limit the flexibility of data exploration. Stakeholders are constrained by the predefined aggregations and lack the ability to slice and dice data dynamically. This hampers their ability to explore deeper insights or answer questions that fall outside the original scope of the SQL logic.
Short-term needs often lead to the rapid creation of materialized metrics and tables. However, without proper review and maintenance, these assets can accumulate unchecked, creating bloated pipelines. This not only increases storage and compute costs but also adds complexity to the system, making it harder to manage and optimize over time.
Universal semantic layers introduce a new hybrid approach for creating and accessing business logic, and creating a central source of truth.
Creating an independent central source of truth
Great Semantic Layers enable non technical users access business logic as information and understand the semantics of how it was defined.
Semantic Layers apply a structured data modeling framework, and the best ones also also apply strong governance features to make sure business definitions are consistent and not duplicated - resulting in increased trust and clarity with data
In the era of AI, new opportunities for cross-team collaboration arise. Business definitions can be made on the fly with extreme high flexibility, while still being governed by the data team via the Semantic Layer governance features. This allows endless coverage over the needed information, when the business needs it.
Universal Semantic Layers often have a data modeling framework built in - which allows adding more data and business logic as the company grows, while keeping the logic clean and organized via its governance features.
Semantic Layer from scratch might become a huge deal. Moving all the previously defined business logic out of the existing tools can be time consuming and tedious. However, the sooner done, the less painful it is.
If you’re reading this, you are probably evaluating Universal Semantic Layers for your company. If you have a growing demand for data accessibility from your business stakeholders and you find it hard to keep up, or business definitions are spread across many tools, resulting in inconsistencies and trust issues - choosing a Universal Semantic Layer is a smart choice.
Book a demo to learn more how Lynk makes the implementation effort seamless via the Discovery Engine and how it enables data accessibility and trust, while reducing manual work and freeing up data teams to work the important stuff - generating business value
Automate data workflows with consistency, clarity and trust, to enable AI and business users succeed with data