AI Factory
Private AI Platform — Chat with the Company's Documents
Diagramas
Private Chat Flow
Project Overview
A private AI platform built inside the infrastructure of a technology and innovation centre: the team talks to its own documents, in Portuguese, under the identity the company already has — and nothing leaves the building. It is not a chat that ships your document to someone else's API: the content, the identity, and the model stay in-house. It is already delivered and in production, used by real client accounts in their own directory. The thesis running through it all is simple to say and hard to do: in corporate AI, architecture matters more than the choice of model — privacy here is not policy, it is architecture.
Architecture & Implementation
Four architectural decisions sustain the product. (1) Privacy by architecture: sensitive data is removed before a document becomes a vector, whatever fails that review goes to encrypted quarantine and never reaches the index, and not even the administrator reads chat content through the product — auditing records who did what, never the conversation text. (2) Self-hosted inference fleet: models run on the institution's GPU, served locally via vLLM under Slurm, and serverless providers join the same catalogue behind a single endpoint — dozens of models in one selector, with risk curation (those that occupy whole nodes are restricted to admins). (3) Federated identity: login uses the directory the company already has (SSO federating LDAP), with accounts provisioned by automation — no parallel sign-up and no new password. (4) Cost and latency under control: routing between the in-house fleet and external providers, with fallback, retry, and circuit breaker, keeps the bill predictable and the response stable when a provider degrades. Documents use hybrid retrieval (vector + lexical search) in Portuguese, with vectors in the application's own database, and operations run on FreeBSD with a private network, isolated jails, and automated deployment.