Founder & Software Architect Delivered · In Production (Private)

AI Factory

Private AI Platform — Chat with the Company's Documents

Open WebUIKeycloak / SSOOpenLDAPvLLMSlurmpgvectorPresidio

Diagramas

Private Chat Flow

PRIVATE CHAT FLOW — AI FACTORYUsersigns in via SSOof the directoryFederated IdentitySSO federating LDAPaccounts by automationPrivate Chatover the documentsof the company itselfInference — single endpointIn-house fleetinstitution GPU · vLLM under SlurmExternal providersserverless · same cataloguefallback · retry · circuit breakerClient data — never leaves the buildingdocuments · identity · model — all on the company's infrastructurePII removed before the vector · encrypted quarantine · admin with no access to chat contentprivacy is not policy — it is architecture: the design of the system, not its promisereal delivery in production · client not named · no public repo

Project Overview

A private AI platform built inside the infrastructure of a technology and innovation centre: the team talks to its own documents, in Portuguese, under the identity the company already has — and nothing leaves the building. It is not a chat that ships your document to someone else's API: the content, the identity, and the model stay in-house. It is already delivered and in production, used by real client accounts in their own directory. The thesis running through it all is simple to say and hard to do: in corporate AI, architecture matters more than the choice of model — privacy here is not policy, it is architecture.

Architecture & Implementation

Four architectural decisions sustain the product. (1) Privacy by architecture: sensitive data is removed before a document becomes a vector, whatever fails that review goes to encrypted quarantine and never reaches the index, and not even the administrator reads chat content through the product — auditing records who did what, never the conversation text. (2) Self-hosted inference fleet: models run on the institution's GPU, served locally via vLLM under Slurm, and serverless providers join the same catalogue behind a single endpoint — dozens of models in one selector, with risk curation (those that occupy whole nodes are restricted to admins). (3) Federated identity: login uses the directory the company already has (SSO federating LDAP), with accounts provisioned by automation — no parallel sign-up and no new password. (4) Cost and latency under control: routing between the in-house fleet and external providers, with fallback, retry, and circuit breaker, keeps the bill predictable and the response stable when a provider degrades. Documents use hybrid retrieval (vector + lexical search) in Portuguese, with vectors in the application's own database, and operations run on FreeBSD with a private network, isolated jails, and automated deployment.