Enterprise KM and AI: RAG, Model Choice, Department Costs

Why does enterprise knowledge management still fail to find answers after adopting AI?

"We have already uploaded all our documents to AI, but employees still don't use it."

This is the most common complaint after an enterprise adopts AI for knowledge management. The problem is not that the model is not smart enough; it is that the engineering infrastructure has not kept up.

A typical failure path

1. Upload internal documents to ChatGPT or Claude
2. Ask employees to "go ask the AI"
3. Employees get wrong answers (because the model does not have the latest documents)
4. Employees give up and go back to Google or ask a colleague

The root cause of this problem is: public AI tools do not know your company's private knowledge. Building an enterprise KM system that is actually useful requires a RAG (Retrieval-Augmented Generation) architecture.

The correct architecture for enterprise KM

Employee asks a question
  → Search the vector database (company document embeddings)
  → Retrieve the relevant passages
  → Send them together with the question to the LLM
  → Get a grounded answer (with links to the original documents)

This architecture involves several key choices:

Embedding model: cheap and sufficient

Converting documents into vectors (embedding) does not require a high-end model. text-embedding-3-small can deliver high-quality semantic search:

from openai import OpenAI

client = OpenAI(
    base_url="https://bazaarlink.ai/api/v1",
    api_key="sk-bl-YOUR_KEY",
)

def embed_document(text: str) -> list[float]:
    response = client.embeddings.create(
        model="openai/text-embedding-3-small",
        input=text,
    )
    return response.data[0].embedding

def embed_query(question: str) -> list[float]:
    return embed_document(question)

Answer generation: choose the model based on question complexity

KM questions vary widely in complexity:

def answer_question(question: str, context_chunks: list[str]) -> str:
    context = "\n---\n".join(context_chunks)

    # Use Claude Sonnet for complex questions that require reasoning
    # Use Gemini Flash for simple queries to save cost
    model = "anthropic/claude-sonnet-4-6"

    return client.chat.completions.create(
        model=model,
        messages=[
            {
                "role": "system",
                "content": f"Answer employee questions based on the following company documents. If the documents contain no relevant information, state that clearly.\n\nDocument content:\n{context}"
            },
            {"role": "user", "content": question}
        ],
    ).choices[0].message.content

The four core document types in enterprise KM

Each type calls for a slightly different embedding strategy: product knowledge suits finely divided chunks, while contracts suit chunking by clause.

Cost allocation for multi-department KM

If a KM system is shared across departments, you need to know how much API cost each department consumes.

BazaarLink's department (Team) feature gives each department its own key and budget:

Enterprise KM system
  └─ Sales dept KM key: $50/month (customer data lookup)
  └─ Engineering dept KM key: $80/month (technical document lookup)
  └─ HR KM key: $30/month (HR policy lookup)
  └─ Legal KM key: $40/month (contract clause lookup)

At month end, export the report. The IT department can then calculate the exact AI usage cost of each department, providing a basis for next year's AI budget.

The key to making KM actually get used

Getting the technical architecture right is only the first step. The key to getting employees to actually use KM is trustworthy answers:

1. Cite sources: every answer must indicate which document and which page it comes from
2. Admit ignorance: when there is no basis for an answer, clearly say "no relevant documents found"
3. Timeliness: re-embed documents immediately after they are updated, to avoid outdated information

# A good KM answer format
{
    "answer": "According to page 12 of the Employee Handbook 2026, the calculation of annual leave is...",
    "sources": [
        {"doc": "Employee Handbook 2026", "page": 12, "chunk": "..."},
    ],
    "confidence": "high",  # Clear document support
}

Conclusion

The key to successfully adopting AI for enterprise KM does not lie in "choosing the strongest model," but in:

1. A correct RAG architecture: letting the AI know your private knowledge
2. Model routing: use cheap models for embedding, and choose answer models based on complexity
3. Cost visibility: the KM consumption of each department is clearly attributed
4. Trustworthy output: cite sources and admit when you do not know

BazaarLink's unified API endpoint lets you call both embedding models and generation models within the same system, without managing multiple API keys. A single bill at month end clearly shows how much each department spent.

TWD billing · Taiwan invoices · leading AI models · OpenAI-compatible API