We've had a lot of developers reach out asking the same question: can we build on top of Nexus Pen? The short answer is yes, and we're getting ready to open the door.
This post is an early look at the Nexus Pen API: what it will let you do, the architecture behind it, what we've learned building the backend, and where we're headed. If you're a developer interested in AI hardware integrations, custom education tooling, or just curious about how a pen can have an API, this one's for you.
Why an API?
When we built the Nexus Pen backend, we made a deliberate architectural choice: the pen is a hardware interface, and the intelligence lives in the cloud. The ESP32 firmware handles audio I/O, wireless connectivity, display, and buttons. Everything else, AI processing, user context, conversation history, mode logic, runs on our FastAPI backend.
That separation wasn't just a technical convenience. It means the pen is fundamentally extensible. New capabilities don't require firmware updates. New modes don't require new hardware. A developer who connects to our API can change what Donna says, how she responds, and what she knows, without touching a single line of C++.
What the API Will Expose
The initial API release will focus on three core surfaces:
- Custom Modes: Create your own conversation modes with custom system prompts, response styles, and topic constraints. A medical-prep company could build a "Clinical Reasoning" mode. A bar-exam coach could build a "Case Analysis" mode. A language tutor could build a "French Immersion" mode that refuses to respond in English.
- Session Webhooks: Receive real-time events when a user starts a session, asks a question, or ends a conversation. Build integrations with your CRM, your training platform, or custom analytics dashboards.
- Conversation History: Read and write to a user's conversation history with proper OAuth scoping. Build study tools, export utilities, or AI tutoring dashboards that surface patterns in what a learner has been asking.
The Custom Mode System
Custom modes are the core developer feature we're building. At the core, a mode is a JSON object with a few key fields:
- system_prompt: The instruction set Donna operates under for this mode
- name and description, Displayed in the app's mode selector
- constraints: Optional topic filters, response length limits, language restrictions
- voice_profile: Coming later: custom TTS voice settings
Nexus Pen has five preset modes: Answer, School, Research, Creative, and Language, all five in the Plus and Family plans. The API will let Family plan accounts build custom modes of their own, such as a "Chemistry Lab Safety" mode for a lab session or a "Sales Pitch Coach" mode for practicing a pitch.
What We've Learned Building the Backend
Our backend is FastAPI on Python, currently at v4.12. Building it taught us a lot about the constraints of real-time AI in a hardware context. A few things that might be useful for developers building on top of it:
Latency is everything. The pen's speaker starts playing audio within about 800ms of a question being asked. Users are holding a pen waiting for a response, not staring at a loading spinner on a screen. Every millisecond of backend latency is felt physically. We use parallel TTS generation (splitting responses into sentences and streaming them) to keep perceived latency under 1 second for most queries.
Audio format matters more than you'd think. We deliver audio to the pen as 16kHz u-law G.711 over the wireless link. The choice of codec, sample rate, and chunking strategy has a massive impact on audio quality and battery life. Developers building custom audio responses via the API will need to respect these constraints.
Context windows accumulate fast. Each pen session can run for hours. Managing conversation context, what to keep, what to summarize, what to discard, is an active area of our backend development. The API will expose controls for context management so developers can tune this for their use case.
What Developers Can Build
Here are some use cases that developers who've reached out have proposed:
- A medical study tool that quizzes you on anatomy using the Socratic method
- A corporate training platform that delivers onboarding content through Donna's voice
- A language immersion app that creates a full-conversation environment with vocabulary tracking
- An accessibility tool that reads documents aloud and answers questions about them
- A coding tutor mode that explains algorithms and checks pseudocode
- A debate prep tool that takes the opposing position on any argument a learner makes
Getting Early Access
We're opening a waitlist for developer API access now. Early access partners will get direct support from us, the ability to influence the API design, and priority placement in our upcoming developer showcase.
If you're a developer or company building in the education technology or AI hardware space and you want early access, email us at NexuspenLLC@gmail.com. We're especially interested in developers building tools for higher education, corporate training, and accessibility. For background on the hardware itself, see how Nexus Pen works or our AI pen guide.
The pen is just the beginning. The Founders Edition is coming soon. Get notified at nexuspen.org.