Bring business knowledge
into context.

The aim is an organisation-wide knowledge structure that connects different domains: strategy and decisions, organisation and expertise, customers and contracts, products and services, processes and systems, and projects and their outcomes. These resources have different owners and protection requirements. Making them discoverable together does not mean giving everyone access to everything.

A knowledge catalogue first describes which resources exist, where they are held, who owns them and under what conditions they may be used. A shared vocabulary organises domain terms, synonyms and categories. Stable identifiers connect the same product, process or project across systems. Versions, validity and provenance show which statement applied at a particular time.

Authoritative business systems can remain in place. Depending on the need, knowledge is retrieved through approved interfaces or indexed as a controlled, refreshable derivative. A central point of access does not require a complete copy of all company data. Experience from conversations and practical work becomes durably reusable only when it is deliberately documented, interpreted and approved.

In this article, Deep Knowledge describes this architectural principle: content is connected to meaning, relationships, ownership and provenance. The term denotes neither a particular product nor a standardised quality level. A knowledge base remains a maintained selection and organisation of available knowledge; a language model does not automatically make it complete or correct.

Retrieval-Augmented Generation, or RAG, supplies a language model with relevant, authorised sources as context at runtime. Retrieval is distinct from changing model weights through training or fine-tuning; the approaches can complement each other. Answers still require verification. Their value depends on retrieving the right resources and preserving the conditions that give them meaning.

The core of a hybrid architecture is the separation of authoritative sources, knowledge organisation, retrieval and model processing. In this reference architecture, the catalogue, prepared content and search indexes operate within a controlled environment. Originals remain in their designated business systems or repositories. Depending on the task, the model layer can use a local service or a cloud service approved contractually and technically. Every connection needs defined update rules, data location and ownership.

Hybrid does not mean automatic fallback. If a local answer is not good enough, that does not justify sending confidential documents to an external provider without further checks. Before a request is processed, the system verifies identity and access rights, classifies the data, minimises the context and makes a binding decision about the permitted processing route. If the necessary rights or approval are missing, that route is blocked. If reliable sources are missing, asking for clarification or explaining why an answer cannot be given is more useful than switching models.

The local route also needs to be examined from end to end: OCR, embeddings, graph extraction and reranking can themselves call models or external services. Telemetry, error logs and caches can carry information outside the environment. For every processing step, the architecture must therefore specify where it runs, which data it receives and which network destinations are allowed.

This is distinct from hybrid retrieval: combining full-text search, vectors and, where useful, a graph. That search strategy can run entirely locally. Conversely, a locally hosted model can access an external dataset. The two decisions should be planned independently.

The following table describes responsibilities and data locations, rather than a shopping list. Processing nodes can run separately or share infrastructure for an initial pilot. Capacity and separation depend on data volume, concurrency, availability and protection requirements.

An organisation-wide knowledge base needs four separate answers: What is the business subject? How is the content structured? Which format carries it? Which rules govern its use? A contract is a business subject, PDF is a file format and “confidential” is a classification. None of these attributes alone determines the right processing method.

Structured data has defined fields, such as customer, order or financial records. Semi-structured data carries recognisable keys or envelopes, such as JSON, XML, event logs or email metadata. Unstructured text conveys meaning mainly through free language. Multimodal content adds the media dimension: images, audio, video or technical models. These are not four mutually exclusive boxes. An email combines fields, free text and attachments; a PDF can contain prose, tables and scans; CAD geometry can be highly structured.

Field-level types also need to be preserved: numbers, dates, Boolean values, identifiers and enumerations serve different purposes. A product code is usually an identifier, not a number for arithmetic. Currencies, units, time zones and relationships between records carry meaning. Converting everything into prose can lose important validation and analysis capabilities.

Data input starts with a catalogued, approved resource and an appropriate connection. Structured data retains keys and field types; documents retain structure and source locations; media may gain OCR, transcripts or descriptions as labelled derivatives. Docling can convert different document formats into a shared intermediate representation. That working representation supplements the original rather than replacing it.

Every resource needs stable identifiers, provenance, domain ownership, version, modification time, validity and access rules. Source locations may require a page, section, record key or timestamp. Import time and business validity are different concepts. Source permissions must survive through retrieval and output.

Data cleansing is more than removing empty lines. It produces consistent working data that can be checked. This includes character encoding, language, date formats, units, duplicates and the mapping of alternative spellings. But “Art.-Nr. 0042” and “Artikel 42” should only become the same entity under an established domain rule. Leading zeros, version suffixes, negations and exceptions can be decisive.

Originals remain unchanged during preparation; retention and deletion rules continue to apply. Every transformation produces a traceable derivative linked to a version. A checksum can identify an identical document; similar documents require more care. Two almost identical service instructions may apply to different product versions. Merging them without checking would destroy knowledge rather than clean it.

KNIME Analytics Platform can provide a visual layer for these processing steps: reading data, reshaping fields, applying rules, joining datasets and checking results. Custom Python steps or specialised parsers can extend a workflow. A visible sequence makes domain review easier. It does not, by itself, make a process correct or lawful.

A useful process includes quarantine for problematic inputs. Missing permissions, damaged files or doubtful OCR results should not simply pass into the knowledge base. Sampling and automated validation rules complement one another. Working directories and temporary tables can contain sensitive content; KNIME processing should therefore not be equated with “RAM only”.

Chunking divides large documents into units that can be retrieved selectively and processed as context. Sections that are too small separate rules from their exceptions; sections that are too large introduce irrelevant information into an answer and consume more context. There is no universally best size. Document structure, questions, the embedding model, tokenizer and downstream processing determine a useful split.

A good chunk carries its provenance: document ID, version, heading path, source location, validity and access rule. Tables should remain readable with their column headings and units. A list of prerequisites should not be separated from the action to which it applies. In longer chapters, a small search result can refer to a larger parent section; context assembly then determines how much is needed.

Overlap can preserve continuity between two windows, but it also produces duplicate content. This increases index and context size and can cause repetition when results are assembled. The demo deliberately counts words to make the split visible. In a production pipeline, size limits must be checked with the actual tokenizer in use.

Once the content is split, embeddings and search indexes are created. Exact product codes benefit from full-text search or structured filters; semantically similar phrasing benefits from vector search. Results can then be reranked. Access filters apply to every candidate and again before the context is passed on. A relevance score is not a probability that something is true. Text passages also provide source references for derived knowledge.

A relational database organises data into tables with rows and typed columns. Primary keys identify records; foreign keys and junction tables represent relationships. Queries connect tables through JOINs. This fits uses such as transactions, master data, orders and consistent reporting. Relationships are explicitly part of the relational model.

A graph database makes entities and connections direct elements of the data model. In a property graph, nodes and typed, directed edges carry properties. Queries follow those connections, for example from a system to dependent processes, responsible teams and supporting documents. This is particularly useful to explore when paths and changing connections are themselves the subject of the question.

The following example represents the same entities and relationships in both forms. Changing the representation does not change the business answer. The choice depends on the required queries, rules and operating constraints, not on an assumption that graph databases are inherently newer or faster.

An organisation-wide knowledge graph connects domains that repositories often keep apart: people and responsibilities, processes and systems, projects and products, customers and contracts, and policies and documents. Its value lies in cross-domain questions: which processes depend on a system? Who owns them? Which rules and decisions matter to an initiative? The following overview presents possible relationship types as a simplified organising model, not a complete company dataset.

This makes knowledge not just searchable but explicitly related. Changes can be traced through known dependencies for review; new employees can find context and responsible people; several teams can refer to the same terminology and evidence. The graph still contains only documented relationships. A missing edge does not prove there is no connection in the business.

Modelling begins with a small set of stable domain types and identifiers. An organisational unit, person, process, system, project, product and document represent different entities. “owns”, “uses”, “applies to” and “supports” are different relationships. A shared vocabulary and domain-owned mappings prevent matching names from being treated as proof of identity.

Model-assisted extraction initially produces proposals. Each derived statement retains its source, passage, validity, extraction version and review status. Conflicts and unreviewed connections remain visible. Domain owners decide, under defined rules, which statements enter the approved knowledge base.

Graph-assisted RAG can add connected context to text retrieval. A question might examine the surroundings of a project or follow a chain from process to system to policy. Permissions and business validity determine which relationships and sources may enter the answer. A larger graph alone does not produce better answers; suitable modelling, maintenance and evaluation do.

Graph traversal must not cross access boundaries. A neighbouring node, a relationship or a generated summary can also contain protected information. The access rights of underlying sources must therefore constrain every expansion of the search path. Derived relationships should be reviewed, corrected or removed when sources change or are deleted.

Knowledge search retrieves information; RAG uses it to produce an answer. An agent can also select tools, assess their results and initiate further steps. The practical value emerges when a recurring task spans several searches and applications: less manual copying, a traceable process and a prepared handover to the responsible person.

Agents can use this knowledge structure across departments: assemble relevant rules and earlier decisions for a project, identify dependencies of a system change for review, or prepare a research and documentation draft. They connect authorised sources, request missing information and retain evidence. A shared knowledge foundation helps avoid every application creating its own conflicting view of the business. These diagrams illustrate possible approaches; they do not access real systems.

A rule-based workflow is often sufficient for a well-defined process. Agent decisions are useful where the next research or tool step depends on intermediate results. The permitted actions still remain predefined.

Local storage is a location decision, not a data protection certification. A protected knowledge system accounts for rights during import, at every derivation step, during retrieval and at output. Data protection, copyright and information security answer different questions. They should be planned together without being confused with one another.

The following selection illustrates possible components. It is neither a pre-validated combination nor an exhaustive list. Adapters, versions, operational requirements and security capabilities must be verified in the chosen configuration. For sensitive data in particular, it matters whether a capability is actually available in the edition being used.

A single “open source” label would be too broad for this kind of stack. Application code, extensions, model weights and hosted services can carry different terms. Freely available model weights are not automatically unrestricted open-source software; they may be subject to their own terms of use. Suitability for the task, licensing and the operating model should therefore be assessed separately.

Even a tool known for local use may include cloud features. A strictly local route requires verified configuration, network rules and control of every service involved. A product name is no substitute for verifying the actual data flow.

Maintainability is part of the selection: before production use, assess ongoing maintenance, security updates, available adapters and a possible migration path. This applies to graph databases, model servers and user interfaces alike. A combination of tools should be treated as a viable architecture only once its specific integration and operation have been verified.

A credible pilot starts with real domain questions and the source passages expected to support the answers. These should include straightforward searches, questions spanning several documents, outdated versions, conflicting statements and deliberately unanswerable questions. Evaluation distinguishes whether the right source was found, whether it supports the answer and whether the system recognises what it does not know.

Alongside answer quality, compare the workflow with the existing approach: time to an evidenced answer, clarification requests, manual transfers and domain-review effort. For agents, add correctly completed tasks, observed approval requirements, duplicate actions and cost per completed task. A faster draft is an improvement only if reviewing it does not create more work. Agree objectives and measurement methods with the business team before the pilot.

Security also needs testing: can someone without permission still learn something through search, the graph or a summary? Does a deleted document stay unavailable? Can a malicious source document trigger an external action? These checks belong in an approval process alongside answer quality, latency and cost per useful request.

If parsers, chunking, the embedding model or the extraction schema change, affected derivatives may need to be regenerated. The pipeline should record which version built each index. A rollback requires consistent data and configuration, not just the previous application version. Monitoring must not create an uncontrolled second knowledge store containing complete prompts and answers.

A good starting point is a clearly bounded collection with known owners. First, establish reliable data quality, access rights and basic search. Add graph relationships where the domain questions require them. Then assess a cloud route using approved contexts and specific quality or operational requirements. This allows the system to grow from a verifiable core, rather than from the longest possible list of tools.

Beyond individual answers, assess the knowledge foundation itself: which priority domains are covered? Which resources lack owners, current versions or permission information? How quickly are changes reflected? How often do terms or relationships remain unresolved? These measures reveal maintenance needs; they do not prove that all company knowledge has been captured.