We've had a lot of developers reach out asking the same question: can we build on top of Nexus Pen? The short answer is yes — and we're getting ready to open the door.
This post is a first look at the Nexus Pen API: what it will let you do, the architecture behind it, what we've learned building the backend, and where we're headed. If you're a developer interested in AI hardware integrations, custom education tooling, or just curious about how a pen can have an API — this one's for you.
Why an API?
When we built the Nexus Pen backend, we made a deliberate architectural choice: the pen is a hardware interface, and the intelligence lives in the cloud. The ESP32 firmware handles audio I/O, BLE, display, and buttons. Everything else — AI processing, user context, conversation history, mode logic — runs on our FastAPI backend.
That separation wasn't just a technical convenience. It means the pen is fundamentally extensible. New capabilities don't require firmware updates. New modes don't require new hardware. A developer who connects to our API can change what Donna says, how she responds, and what she knows — without touching a single line of C++.
What the API Will Expose
The initial API release will focus on three core surfaces:
- Custom Modes — Create your own conversation modes with custom system prompts, response styles, and topic constraints. A medical school could build a "Clinical Reasoning" mode. A law school could build a "Case Analysis" mode. A language school could build a "French Immersion" mode that refuses to respond in English.
- Session Webhooks — Receive real-time events when a user starts a session, asks a question, or ends a conversation. Build integrations with your LMS, CRM, or custom analytics dashboards.
- Conversation History — Read and write to a user's conversation history with proper OAuth scoping. Build review tools, export utilities, or AI tutoring dashboards that surface patterns in what a student has been asking.
The Custom Mode System
Custom modes are the most powerful developer feature we're building. At the core, a mode is a JSON object with a few key fields:
- system_prompt — The instruction set Donna operates under for this mode
- name and description — Displayed in the app's mode selector
- constraints — Optional topic filters, response length limits, language restrictions
- voice_profile — Coming later: custom TTS voice settings
Pro plan users already get five preset modes — Answer, School, Research, Creative, and Language. The API will let Pro and Enterprise accounts create unlimited custom modes and distribute them to users in their organization. A school admin could push a "Chemistry Lab Safety" mode to every student's pen before a lab session. A corporate team could distribute a "Sales Pitch Coach" mode to their field reps.
What We've Learned Building the Backend
Our backend is FastAPI on Python, currently at v4.12. Building it taught us a lot about the constraints of real-time AI in a hardware context. A few things that might be useful for developers building on top of it:
Latency is everything. The pen's speaker starts playing audio within about 800ms of a question being asked. Users are holding a pen waiting for a response — not staring at a loading spinner on a screen. Every millisecond of backend latency is felt physically. We use parallel TTS generation (splitting responses into sentences and streaming them) to keep perceived latency under 1 second for most queries.
Audio format matters more than you'd think. We deliver audio to the pen as 24kHz u-law G.711 over BLE. The choice of codec, sample rate, and chunking strategy has a massive impact on audio quality and battery life. Developers building custom audio responses via the API will need to respect these constraints.
Context windows accumulate fast. Each pen session can run for hours. Managing conversation context — what to keep, what to summarize, what to discard — is an active area of our backend development. The API will expose controls for context management so developers can tune this for their use case.
What Developers Can Build
Here are some of the most interesting use cases we've seen proposed by developers who've reached out:
- A medical school study tool that quizzes students on anatomy using the Socratic method
- A corporate training platform that delivers onboarding content through Donna's voice
- A language immersion app that creates a full-conversation environment with vocabulary tracking
- An accessibility tool that reads documents aloud and answers questions about them
- A coding tutor mode that explains algorithms and reviews pseudocode
- A debate prep tool that takes the opposing position on any argument a student makes
Getting Early Access
We're opening a waitlist for developer API access now. Early access partners will get direct support from our team, the ability to influence the API design, and priority placement in our upcoming developer showcase.
If you're a developer, educator, or company building in the EdTech or AI hardware space and you want early access, reach out through our contact page. We're especially interested in developers building tools for higher education, corporate training, and accessibility.
The pen is just the beginning. The real leverage is in what gets built on top of it.