mirror of
https://github.com/Mail-0/Zero.git
synced 2026-06-29 23:37:05 +00:00
5b7edd2717
# Implement Database Sharding for Email Storage This PR implements database sharding to improve scalability and performance for email storage. The system now distributes email data across multiple shards instead of storing all data in a single database instance. ## Type of Change - ✨ New feature (non-breaking change which adds functionality) - ⚡ Performance improvement ## Areas Affected - [x] Email Integration (Gmail, IMAP, etc.) - [x] Data Storage/Management - [x] API Endpoints ## Description This PR introduces a sharding mechanism for email storage to handle large volumes of data more efficiently: 1. Created a new `ShardRegistry` Durable Object to track and manage shards for each connection 2. Implemented logic to distribute threads across multiple shards based on size limits (9GB per shard) 3. Modified thread operations to work across shards, including: - Thread retrieval with shard fallback - Aggregation of results from multiple shards - Sequential processing for paginated results 4. Moved notification handling from a separate component to the AI sidebar 5. Added shard count display in the user interface 6. Refactored server utilities to support the new sharding architecture 7. Updated API endpoints to work with the sharded database structure The implementation ensures that as email volume grows, the system can scale horizontally by adding new shards rather than being limited by a single database's capacity. ## Testing Done - [x] Manual testing performed - [x] Cross-browser testing (if UI changes) ## Checklist - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] All tests pass locally ## Additional Notes This change significantly improves the system's ability to handle large email volumes by distributing data across multiple database shards. The UI now displays the number of shards in use, providing transparency about the underlying storage architecture. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Introduced sharding support for improved scalability and performance. * Added display of shard count in the user interface. * Enhanced sidebar to handle real-time updates for mail, labels, and state. * **Bug Fixes** * Improved cache invalidation and data refresh for more accurate mail and label updates. * **Refactor** * Centralized thread and label operations for better maintainability. * Simplified and streamlined backend logic for thread and label management. * **Chores** * Updated configuration to support new sharding infrastructure. * Removed unused notification provider component and related logic. * **Documentation** * UI now reflects the number of database shards in relevant menus. <!-- end of auto-generated comment: release notes by coderabbit.ai -->