Authoritative project documentation

ApiaryLens Master Architecture and Design Plan

Document Status

Status: Living master plan

Last updated: 2026-07-15

This is the authoritative entry point for ApiaryLens product architecture and design. It records the current system shape, accepted decisions, active direction, open questions, decision process, and links to detailed source documents.

This document does not replace ADRs. ADRs explain why durable decisions were made; this plan is updated after an ADR is accepted so it always describes the assembled current architecture. Imported documents under docs/source-documents/ preserve project history and are not authoritative when they conflict with this plan or an accepted ADR.

Purpose and Scope

ApiaryLens is an open-source, self-hosted apiary intelligence and hive management platform. It must serve a new beekeeper with one hive, families and mentors, bee clubs, extension offices, researchers, and commercial operations without requiring different products or a cloud service.

This plan governs:

• The open-source product and self-hosted deployment
• Offline PWA behavior and synchronization
• APIs, data, identity, authorization, media, background work, and integrations
• Public project, application, and developer properties
• Product versioning, release artifacts, updates, migrations, recovery, and support
• Future optional hosted-service architecture
• Community galleries and registries
• Architecture research, ADRs, documentation, and repository boundaries

Decision Authority

When documents disagree, use this order:

1. Non-negotiable principles in AGENTS.md
2. Accepted ADRs under docs/adr/
3. This master plan
4. Focused architecture, security, deployment, and research documents
5. Roadmaps and task plans
6. Imported source documents and historical handoff material

An accepted ADR that changes the architecture must update this master plan in the same change. A proposed ADR documents an option under review and does not override the current plan.

Product and Architecture Principles

• Open source first: the core platform and self-hosted path remain open.
• Self-hosted first: core operation requires no cloud service or third-party account.
• Offline-first PWA: field workflows remain usable without connectivity and sync later.
• Privacy-first: no telemetry or data egress by default; publishing and external providers are opt-in.
• Secure by exposure: device-only use may omit an application password, but every LAN, tunnel, VPN, or internet-reachable deployment requires authentication and encrypted transport; internet-facing access requires normal publicly trusted HTTPS.
• AI-assisted, not AI-required: AI can be disabled without reducing core hive-management capability.
• SaaS-capable later: an optional hosted offering may use the same product, but self-hosting remains first-class.
• Portable data: users can export and move their data and media.
• Scale without a rewrite: organizations, permissions, storage, and workflows must support one hive through commercial operations.
• Outdoor usability: mobile interaction, touch targets, contrast, speed, and intermittent connectivity are primary constraints.
• Accessible: interfaces and published sites should target WCAG 2.1 AA or better.
• Portable brand assets: official graphics are committed, versioned build inputs; no private creative studio is required to build or run ApiaryLens.
• Near-free family cloud: the project must pursue an always-available, synchronized family profile with no license cost and zero or predictably minimal hosting cost, without promising that a third-party free tier will remain free.
• Easy to start and operate: a family or hobbyist must not need database, container, DNS, TLS, identity-provider, or cloud-billing expertise to begin.
• Safe to keep current: every supported profile has a traceable, guided, backup-first update and tested recovery path that preserves offline work.
• Tiered without editions: personal, family, and organization footprints use the same core product, contracts, and portable data formats.
• Documented before implementation: meaningful architecture decisions are reviewed and recorded before dependent code is built.

Current Project State

The product architecture and MVP contract are accepted and application scaffolding is now authorized. The repository is transitioning from foundation documentation to implementation; the release is not complete until the code, deployments, and UAT gates described by this plan pass.

Accepted decisions:

• ADR 0001: Core Monorepo with Separate Properties and Operations
• ADR 0002: Domain Strategy
• ADR 0003: Open Source and Self-Hosted First
• ADR 0004: Lucidchart Diagram Standard
• ADR 0005: Activate the Initial Repository Portfolio
• ADR 0006: Cloudflare Hosting for Public Frontends
• ADR 0007: Deployment Profile Priority
• ADR 0008: MVP Application Platform
• ADR 0009: Data, Media, and Offline Synchronization
• ADR 0010: Built-in Identity, Sessions, and Authorization
• ADR 0011: Scout Bee and Deployment Execution
• ADR 0012: Public Frontend Implementation Convention
• ADR 0013: Keyless Release Signing

Accepted product-scope decision:

• ApiaryLens MVP Definition and UAT Contract

The open-source, self-hosted, offline-first, privacy-first, and optional-AI constraints are mandatory through AGENTS.md and accepted ADR 0003. The public product uses Apache License 2.0 and DCO 1.1 contribution sign-off.

System Context

The system serves beekeepers, families, mentors, clubs, staff, and researchers through an offline-capable PWA. The PWA may operate device-locally for personal use or synchronize through the backend API. The backend coordinates relational data, media storage, background work, and optional provider adapters for weather, bloom, sensors, storage, identity, and AI.

The editable source is filed in the dedicated ApiaryLens Lucid folder and cataloged in docs/diagrams/README.md.

The PWA communicates with the API when online but is not merely an online view of server state. Its IndexedDB replica, outbox, synchronization protocol, explicit conflict behavior, and media retry lifecycle are accepted in ADR 0009 and detailed in Offline Synchronization Protocol.

Supported Operating Models

The MVP is designed first for family and hobbyist beekeepers. ApiaryLens uses capability tiers rather than separate product editions:

Delivery experiences include a public demo, a later installable device-only personal PWA, completed MVP Scout Bee deployment, Docker Compose, a Cloudflare-native family cloud, provider-neutral VM deployment, later cloud templates, and an optional future managed service. Docker Compose is the first complete server target on personally controlled hardware. Cloudflare is the first cloud profile target; Compose on an ordinary Linux VM is the second. All consume the same core product and portable data contracts.

See Installation and Deployment Experience.

Repository Architecture

Repository boundaries follow ownership, visibility, deployment, release cadence, and contribution model—not domain ownership alone.

The main apiarylens repository remains authoritative for product behavior, contracts, self-hosted deployment, and releases. Domain repositories consume versioned artifacts or generated documentation; they do not copy or fork the product implementation. Repository visibility and deployment visibility are separate: the .org, .app, and .dev sites are public properties deployed from private repositories. The core open-source repository is the only public repository in the initial portfolio. It also carries its own public community-health files and issue templates because GitHub does not inherit them from a private .github repository.

Future production SaaS infrastructure may require a private apiarylens-cloud or apiarylens-infrastructure repository. The private apiarylens-ops repository must not become an unstructured substitute for production infrastructure.

See Repository Strategy and ADR 0001. The activation timing and initial scaffold are accepted in ADR 0005.

Domain Architecture

Each domain keeps the same meaning as the project grows. See ADR 0002 for the accepted decision.

Public Frontend Hosting

All official apiarylens.org, apiarylens.app, and apiarylens.dev frontends use Cloudflare. New projects target Workers Static Assets with custom domains, managed TLS, caching, security headers, and reviewed preview/production workflows. The current apiarylens.com redirect to .org is also implemented on Cloudflare.

This accepted frontend-hosting boundary does not by itself select the frontend framework or backend services. ADR 0007 separately makes a Cloudflare-native family backend the first cloud target while preserving portable self-hosted, Compose-on-VM, demo, and future managed backends. A self-hoster requires no Cloudflare account.

See ADR 0006 and Task 009.

Diagram and Flowchart Standard

Lucidchart is the editable source of truth for all new or substantially revised ApiaryLens architecture diagrams, data diagrams, and flowcharts. All documents must live in a dedicated Lucid folder named ApiaryLens and be cataloged in docs/diagrams/README.md.

Public documentation must include an accessible SVG or PNG export plus explanatory Markdown so understanding the open-source architecture does not require a Lucid account. Existing Mermaid material is legacy migration input rather than the future authoring standard.

The dedicated folder was created on 2026-07-15. The connected Lucid MCP creates and retrieves documents; the official Lucid REST API performs folder placement and deterministic export where the connector has no matching operation. See ADR 0004.

Application Architecture

The core product uses the accepted TypeScript/pnpm platform in ADR 0008. Runtime composition is limited to adapter boundaries so the same contracts and behavior run on Cloudflare and the portable server.

Web PWA

Responsibilities:

• Installable, responsive application shell
• Supported experience on iPhone, iPad, laptop, and desktop browsers
• Researched device-local personal mode without a required backend account
• Offline read and write workflows
• Local persistence, mutation queue, and synchronization state
• Fast inspection capture, photos, QR entry, and field-safe interaction
• Simple workflows for new beekeepers and efficient workflows for larger operations
• Clear indication of offline, pending, conflicted, and synchronized state
• Same family record available across authorized devices after synchronization

The PWA uses React, TypeScript, Vite, a service worker, and Dexie over IndexedDB. Browser storage is a working replica and outbox, not the only durable family copy. The UI exposes local durability and sync state, and the server/export paths provide verified backup. PWA first remains accepted; native Apple packaging is post-MVP. Map rendering and a chart library are added only when an accepted P0 screen needs them and after accessibility/license review.

iPhone App Store Client

An iPhone application users can download from the Apple App Store is a committed later roadmap direction. It connects to a user's chosen compatible ApiaryLens deployment, including self-hosted servers, the near-free family cloud reference, or a future managed ApiaryLens service.

The iPhone client must reuse the public authentication, API, synchronization, media, and data-portability contracts. It must not require an Apple-only backend, iCloud, or the future ApiaryLens SaaS. Deployment connection and onboarding should support a plain server URL plus a safe guided mechanism such as a QR code or connection file, subject to security research.

The PWA remains the first implementation and proves the mobile workflows. Research and an ADR will decide whether the App Store client uses Capacitor, another thin wrapper, or native code. The decision must address offline storage, background synchronization, camera and media access, notifications, TLS and self-hosted server trust, accessibility, App Store privacy disclosures, signing, updates, and long-term maintenance.

The Apple developer account, signing identities, certificates, and release automation are private maintainer operations. They are not requirements for self-hosting or building the open-source server.

Backend API

Responsibilities:

• Versioned REST API and OpenAPI contract
• Authentication, authorization, organizations, memberships, and roles
• Domain operations for apiaries, hives, inspections, queens, equipment, media, weather, bloom, and harvests
• Synchronization endpoints with idempotency and explicit conflict behavior
• Export, portability, and administrative operations
• Provider abstractions rather than hard-coded SaaS dependencies

Hono is the accepted API framework. Shared Zod schemas drive request validation and OpenAPI 3.1. The API runs in Cloudflare Workers and Node 24; runtime and storage types do not enter domain packages.

Background Worker

The MVP does not require a separately deployed background worker. Future responsibilities may include:

• Weather and bloom synchronization
• Media processing
• Export generation
• Notifications and scheduled work
• Optional AI analysis
• Retryable integration tasks

Client-side thumbnail creation and request-driven export cover MVP work. A durable queue and worker require a later ADR when weather ingestion, video, notifications, sensor processing, or optional AI establishes measured delivery requirements.

Data Store

The accepted MVP relational engines are D1 for Cloudflare and Node's built-in SQLite for Compose. They consume shared migration SQL, repositories, domain rules, and conformance tests through a small asynchronous SQL port. PostgreSQL is a future organization-scale adapter only when measurements justify it.

The accepted MVP domain concepts are:

• Organization, User, Membership, Session, Invitation, Recovery Code, and Role
• Apiary, Hive, Queen, and Equipment Box
• Inspection, Mite Count, Health Observation, Treatment, Feeding, Harvest, and Follow-up Item
• Media Asset, Audit Event, Change Log, and Idempotency Record

The detailed model and Lucid export are in MVP Data Model.

Media Storage

Private R2 objects are used on Cloudflare and a private filesystem volume is used in Compose. Relational metadata carries authorization, hashes, dimensions, capture time, and upload state. Client-side thumbnails reduce MVP processing cost. Media is included with relational data in verified backup, restore, export, and profile migration.

Media is core product data, not an optional attachment subsystem.

Offline and Synchronization Architecture

Offline behavior is a first-order architecture concern and is accepted in ADR 0009. The implemented protocol defines:

• Which records and media are available locally
• Local identifiers and server identifiers
• Mutation ordering, idempotency, retries, and deduplication
• Conflict detection, automatic merge rules, and human resolution
• Deletes, tombstones, retention, and multi-device behavior
• Schema and application upgrades with pending local work
• Authentication expiry while offline
• Storage limits and partial synchronization for large commercial datasets
• Observability and user-visible synchronization status without telemetry egress

Stable UUIDs, base versions, operation IDs, idempotency records, an organization-scoped change log, opaque cursors, tombstones, and visible conflicts provide the synchronization contract. Material care data never uses silent last-writer-wins. See Offline Synchronization Protocol.

Identity, Authorization, and Sharing

Accepted direction from ADR 0010:

• Password-optional operation only for a genuinely device-only profile that is not reachable over a LAN, VPN, tunnel, or public interface
• Lightweight built-in ApiaryLens accounts and sessions as the default for family and small server deployments
• Organization and membership model from the beginning
• Optional standards-based OIDC federation for organization deployments; a separate identity provider is not required for family use
• MVP roles Owner, Beekeeper, and Viewer, with later roles expressed through the same capability model
• Explicit capabilities for every protected data, media, member, recovery, and export action

Scout Bee and server validation must prevent no-auth operation on a non-loopback interface. Networked credentials and sessions require encrypted transport. Internet-facing profiles require publicly trusted HTTPS, secure first-owner bootstrap, generated secrets, throttling, session protection, and safe recovery. The PWA uses secure same-origin opaque cookie sessions; future native clients require an OAuth Authorization Code with PKCE design using an external user-agent.

PBKDF2-HMAC-SHA-256 with per-password salts, a domain-separated server pepper, keyed and rotating sessions, one-time recovery codes, atomic first-owner bootstrap, throttling, origin/CSRF defenses, and organization-scoped authorization are release controls. Optional OIDC, passkeys, public links, and native-client authorization are later extensions.

See Authentication, Authorization, and Sharing.

Privacy and Security Architecture

Apiary locations, hive health, media, production, and user identity can be sensitive. The architecture must provide:

• No analytics, telemetry, or data egress by default
• Explicit consent for external providers and public sharing
• Least-privilege access and organization isolation
• Secure secret handling and documented configuration
• Encryption in transit and appropriate protection at rest
• Auditability for sensitive administrative and sharing operations
• Backup, restore, export, deletion, and retention behavior
• A threat model before public sharing, plugins, or SaaS operation
• Private vulnerability reporting rather than public disclosure
• Secure engineering and release controls including secret, dependency, static, and container scanning; SBOMs; signed artifacts and images; checksums; and provenance

The canonical control areas, required artifacts, and initial risk register are in Security Architecture. Applicable requirements from OWASP ASVS 5.0 form the initial web-application verification baseline; the detailed mapping is produced after the stack and trust boundaries are selected.

Security-sensitive operational details and undisclosed vulnerabilities do not belong in public documentation, but public product trust boundaries and security decisions do.

Public and Private Operational Boundary

The open-source product architecture is completely separate from the maintainer's private governance, identity, vault, project-management, and hosted-infrastructure systems. A user can install, authenticate to, operate, back up, and upgrade ApiaryLens without access to any private maintainer service or account.

The public architecture defines portable secret inputs and safe operator options, such as environment variables, Docker secrets, mounted secret files, and optional external secret-manager adapters. It never exposes or depends on private vault names, tenant details, internal authentication brokers, or private automation.

Maintainer-specific hosted-service infrastructure, credentials, vault placement, naming rules, and governance belong in the private apiarylens-ops repository or a future private infrastructure repository. Only portable interfaces and behavior that affect users belong in this master plan.

Weather, Bloom, Sensors, and External Providers

Weather and bloom data should be stored historically and related to hive events, not merely displayed from a live provider. External integrations must use provider interfaces, allow disabled/manual operation, document licensing and attribution, and avoid making an external account mandatory.

Future integrations may include weather stations, hive scales, temperature and humidity sensors, MQTT, LoRaWAN, Home Assistant, and other adapters. Each requires its own trust, offline, retention, and compatibility analysis.

See Weather and Bloom.

Media and Optional AI

Photos and videos belong to the user and must remain usable without AI. AI features:

• Are disabled by default
• Use provider abstractions, including a disabled mode and possible local models
• Require explicit opt-in before data leaves the installation
• Present possible observations, not definitive diagnoses
• Preserve human review and confirmation
• Record provider, model, input provenance, output, review state, and relevant version information
• Degrade cleanly when unavailable

See Media and AI.

Brand, Graphics, and Media Asset Production

Approved logos, icons, illustrations, marketing graphics, screenshots, and other official media are versioned product assets. Their public source of truth is the ApiaryLens repository that owns and publishes them, initially assets/ plus the public guidance under docs/brand/.

The maintainer may use Hybrid-Solutions-Cloud/studio-foundry as an optional private production environment for official pre-rendered graphics and media. Studio Foundry's research-first workflow, human candidate review, safety checks, budget gates, provenance, and gated publishing are reusable patterns.

Studio Foundry is not an ApiaryLens runtime service, build dependency, contributor requirement, or public source of truth. Private prompts, credentials, endpoints, tenant information, and infrastructure remain private. Approved outputs are human reviewed, safely attributed, committed to ApiaryLens, and remain usable if the studio is unavailable.

Real user hive photos, locations, health data, records, or media must never be sent to a maintainer studio or external generation provider as part of branding. Any future user-facing generative-media feature requires explicit opt-in, privacy and retention analysis, and its own ADR.

See ApiaryLens Brand and Asset System.

Community Galleries and Registries

ApiaryLens may eventually support reusable community assets such as inspection and workflow templates, report templates, regional bloom datasets, equipment profiles, provider or sensor adapters, integrations, and plugins.

This is an architectural consideration, not a commitment to a particular feature, repository, centralized service, or marketplace. The core product must work without any central registry. Relevant research spikes, designs, and ADRs must explicitly record their gallery or registry impact.

The canonical requirements, trust considerations, and repository extraction criteria are in Community Galleries and Registries.

Deployment Architecture

Deployment priority is defined separately for personally controlled hardware and cloud environments.

For a complete server on a laptop, desktop, mini-PC, home server, supported NAS, or local VM, Docker Compose is the first supported target with the intended operator experience:

docker compose up -d

That command is a goal, not a complete deployment design. Before release, the self-hosted path must cover:

• Versioned container images and pinned dependencies
• Configuration and secrets
• Database migrations and rollback expectations
• Persistent database and media volumes
• Health checks and startup ordering
• TLS and reverse-proxy guidance
• Backup, restore, export, and disaster recovery
• Upgrades and compatibility policy
• Resource requirements and supported platforms
• Logs and local operational diagnostics without default telemetry
• Air-gapped or restricted-network considerations where practical

Docker Compose is the portable server foundation, not the complete experience for a non-technical family. Scout Bee provides a React guide and Go loopback executor that can install or update a profile or emit a secret-free, versioned deployment plan.

For cloud deployment, the ranked targets are:

1. A Cloudflare-native family profile, expected to evaluate Workers Static Assets, Workers, D1, R2, and related services
2. Docker Compose on an ordinary provider-neutral Linux VM
3. Later provider-specific managed-container or infrastructure templates
4. A future optional managed ApiaryLens service

The Cloudflare runtime, data, media, authentication, backup, and migration design is accepted in ADRs 0008 through 0011. Current free allowances remain dated capacity inputs and implementation acceptance gates, never permanent promises.

Do not make a provider's free tier a core backend architecture requirement or promise of permanent free hosting. Publish provider-neutral server artifacts and maintain the Compose path while implementing and validating the ranked Cloudflare family profile. Azure, AWS, GCP, and other VM targets may run the same Compose artifacts; additional convenience templates follow only when justified. This does not change the accepted Cloudflare hosting decision for official public frontends. See Installation and Deployment Experience and Cloud Free-Tier Deployment Spike.

The reference family workload, dated allowances, capacity calculation, and gates are published in Application Platform and Storage Research. Measured implementation results are appended before the profile is declared release-ready.

Kubernetes, Helm, provider-specific managed-container templates, and SaaS infrastructure are later deployment tracks and do not replace Compose. See Deployment Strategy.

Testing and Validation Strategy

ApiaryLens must be tested across local, virtualized, hosted, and real-device environments without making a maintainer's private accounts part of the product. The public test matrix includes Linux and Windows installation, provider-neutral cloud deployment, required Cloudflare public-frontend deployment, optional Cloudflare-native and Azure-style backend profiles, current desktop browsers, and real iPhone and iPad PWA journeys.

Release validation covers offline work, synchronization conflicts, server outages, media retries, installation, upgrades, rollback, backup and restore, quota exhaustion, resource use, and dated family-cost measurements. Exact maintainer accounts, credentials, and private infrastructure remain outside the public repo.

See Deployment, PWA, and Cost Test Strategy.

Research, Design, and ADR Process

Architecture work follows this sequence:

1. Record the question or requirement.
2. Run a research spike when meaningful uncertainty exists.
3. Document design options and a recommendation.
4. Propose an ADR for a durable decision.
5. Obtain human review and acceptance.
6. Update this master plan and focused documentation.
7. Create scoped implementation tasks.
8. Implement and verify.

The authoritative workflow chart will be maintained in Lucid and cataloged in docs/diagrams/README.md.

A research spike is required when a decision depends on uncertain framework behavior, offline feasibility, security properties, data migration, performance, licensing, interoperability, or operational complexity. A spike is not required when constraints and consequences are already clear, as with the accepted domain assignment.

Research belongs under docs/research/. Durable decisions belong under docs/adr/. Detailed current-state designs belong under docs/architecture/, docs/security/, or docs/deployment/. Implementation work belongs under tasks/.

Every relevant design or ADR must address:

• Effect on self-hosting and required external services
• Offline behavior and synchronization
• Privacy, security, and data portability
• Scale from small to commercial installations
• Optional hosted-service implications without making SaaS required
• Repository and source-of-truth ownership
• Gallery or registry impact when reusable/community assets are involved
• Alternatives, consequences, migration, and rollback where applicable

Application Scaffolding Gate

The architecture gate is satisfied on 2026-07-15 by ADRs 0003 and 0008 through 0011, the two dated research spikes, the detailed data/sync/Scout designs, the versioning and update lifecycle, the accepted MVP contract, and the initial Lucid set. Application dependencies and infrastructure may now be introduced.

The following remain implementation and release gates rather than architecture blockers: measured Worker/Compose capacity, local-network TLS instructions, deployment-plan schema tests, threat-model/ASVS verification, supply-chain artifacts, public-site production proof, supported-device results, profile-to-profile restore proof, and complete UAT evidence. A failed gate changes or blocks the affected profile; it never silently weakens security, portability, or offline data.

Documentation and Source-of-Truth Map

Future apiarylens.org and apiarylens.dev sites may render or publish material from these sources. They must not create divergent copies of core technical contracts or architecture decisions.

Roadmap Relationship

The architecture supports phased delivery without treating later features as MVP requirements. The current phases cover foundation, core hive records, health and production, weather and bloom intelligence, sharing, optional AI, native wrappers, sensors and integrations, and commercial/research scale.

The authoritative delivery sequence is the ApiaryLens Roadmap, with delivery gates in the Execution Plan. Architecture decisions may change how a phase is implemented, but changes to product sequencing belong in those roadmap documents.

Maintenance Rule

Review this master plan whenever:

• An ADR is accepted, superseded, or deprecated
• A repository or public property is activated
• A technology or deployment target is selected
• An installation tier, Scout Bee behavior, or local-storage assumption changes
• A release version, contract compatibility, migration, update, rollback, support, or artifact-promotion policy changes
• Family cloud cost targets, device support, or provider reference profiles change
• A source-of-truth boundary changes
• Diagram tooling, Lucid folder structure, or export rules change
• Public/private operational boundaries or portable secret inputs change
• A new external provider, plugin mechanism, gallery, or registry is designed
• The brand-production workflow, asset source of truth, licensing, or provenance changes
• The roadmap introduces a new architectural capability
• Implementation reveals that an assumption in this plan is false

The plan should describe the system that is currently intended, clearly labeling accepted decisions, active direction, and unresolved questions.