Files
Adam 18314cd088 Enable spam email processing and improve label management workflow (#1948)
# Improved Email Labeling System with DEV_PROXY Support

## Description

This PR enhances the email labeling workflow with a more sophisticated approach to label management. It replaces the previous labeling system with a new implementation that better handles existing labels and user-defined topics.

Key improvements:
- Added DEV_PROXY environment variable to support local development
- Implemented a more robust label suggestion system that prioritizes existing account labels
- Added ability to create missing labels when appropriate
- Modified thread workflow to reload inbox after syncing
- Enabled processing of messages marked as spam (commented out spam filtering)
- Added a test:cron script for local testing of scheduled handlers

## Type of Change

- [x]  New feature (non-breaking change which adds functionality)
- [x] 🐛 Bug fix (non-breaking change which fixes an issue)
- [x]  Performance improvement

## Areas Affected

- [x] Email Integration (Gmail, IMAP, etc.)
- [x] Development Workflow

## Testing Done

- [x] Manual testing performed

## Checklist

- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] I have updated the documentation

## Additional Notes

The new labeling system now follows a three-step process:
1. Retrieves existing user account labels
2. Gets user-defined topics for potential new labels
3. Intelligently suggests and applies labels, prioritizing existing ones

The DEV_PROXY environment variable allows for easier local development by redirecting notification requests through a local proxy when configured.

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

* **New Features**
  * Improved label suggestion and synchronization, now incorporating user topics and existing labels for more accurate email organization.
  * Added spam detection to prevent intent analysis on spam-tagged messages.
  * Enhanced workflow steps for label management, including new steps for user topic retrieval and label suggestion generation.

* **Bug Fixes**
  * Messages labeled as spam are now properly excluded from certain processing steps.

* **Chores**
  * Updated environment variable defaults to enable workflows in local and staging environments.
  * Added a new script for testing scheduled tasks via a local endpoint.
  * Disabled the "seed-style" CLI command.

* **Other Improvements**
  * Inbox folder now reloads automatically after thread updates.
  * Improved logging for thread processing and label synchronization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-07 14:23:01 -07:00
..

Scripts

This folder contains utility scripts for the Zero email application. These scripts are designed to help with development, testing, and maintenance tasks that are not part of the main application flow.

Overview

The scripts system in Zero is built using cmd-ts, a TypeScript library for building type-safe command-line applications. This provides a structured way to create, organize, and run utility scripts with proper command-line argument handling, help text, and more.

How to Run Scripts

Scripts can be run using the scripts command from the project root:

# Run a script from the project root
pnpm scripts <script-name> [options]

# Example: Run the seed-style script
pnpm scripts seed-style

This command is defined in the root package.json and executes the script runner in the mail app:

"scripts": "dotenv -- pnpm run --cwd apps/mail --silent --elide-lines=0 scripts"

Available Scripts

seed-style

Seeds the writing style matrix for a given connection with sample emails of different styles. This is useful for testing and developing the writing style features of the application.

Usage:

# Interactive mode (will prompt for options)
pnpm scripts seed-style

# With command-line options
pnpm scripts seed-style seed --connection-id <id> --style <style> --size <number> [--reset]
# Or reset the style matrix
pnpm scripts seed-style reset --connection-id <id>

Options:

  • --connection-id, -c: The connection ID to seed the style matrix for
  • --style, -s: The style to use (professional, persuasive, genz, concise, friendly)
  • --size, -n: Number of emails to seed (default: 10)
  • --reset, -r: Reset the style matrix before seeding

Subcommands:

  • seed: Seeds the style matrix with sample emails
  • reset: Resets the style matrix for a connection

How to Add New Scripts

To add a new script to the system:

  1. Create a new script file in the apps/mail/scripts directory or a subdirectory
  2. Export a command object using the cmd-ts library
  3. Register the command in apps/mail/scripts/run.ts

Step 1: Create a new script file

Create a new TypeScript file for your script. For example, apps/mail/scripts/my-script.ts:

import { command, option, string as stringType } from 'cmd-ts';

export const myScriptCommand = command({
  name: 'my-script',
  description: 'Description of what my script does',
  args: {
    // Define command-line arguments
    param1: option({
      type: stringType,
      long: 'param1',
      short: 'p',
      description: 'Description of param1',
    }),
  },
  handler: async (inputs) => {
    // Script implementation
    console.log(`Running my script with param1: ${inputs.param1}`);
    // Do something useful here
  },
});

Step 2: Register the command

Update apps/mail/scripts/run.ts to include your new command:

import { seedStyleCommand } from '@zero/mail/scripts/seed-style/seeder';
import { myScriptCommand } from '@zero/mail/scripts/my-script';
import { subcommands, run } from 'cmd-ts';

const app = subcommands({
  name: 'scripts',
  cmds: {
    'seed-style': seedStyleCommand,
    'my-script': myScriptCommand, // Add your new command here
  },
});

await run(app, process.argv.slice(2));
process.exit(0);

Step 3: Run your script

You can now run your script using:

pnpm scripts my-script --param1 value

Best Practices

When creating scripts:

  1. Use cmd-ts features: Take advantage of the cmd-ts library for argument parsing, validation, and help text
  2. Interactive mode: Consider supporting both interactive mode (using prompts) and command-line options
  3. Error handling: Implement proper error handling and provide useful error messages
  4. Documentation: Document your script's purpose, usage, and options in this README
  5. Modularity: Break complex scripts into smaller, reusable functions
  6. Testing: Consider adding tests for critical script functionality

Dependencies

The scripts system uses several key dependencies:

  • cmd-ts: Command-line parsing and execution
  • @inquirer/prompts: Interactive command-line prompts
  • p-all: Run promises in parallel with limited concurrency
  • p-retry: Retry failed promises