Skip to content

Phase 2 Todo List - Review Summary

Date: November 23, 2025
Task: Review .md files and create Phase 2 todo list with AI agent-friendly prompts
Status: ✅ Complete

What Was Requested

Review all .md files considering the time they are written for freshness. Importantly, consider BLUEPRINT.md. Also, consider the current stage of the code perhaps also reading last 10 commits. Write a todo list for Phase 2 with .env consideration of Azure OpenAI endpoints available to us for including agents or models into the FIML ecosystem. Also, when writing the todo list take into account that it's ai agents writing much of this codebase so we want different prompts and not just vague todo titles for todo steps.

What Was Delivered

1. Document Review Summary

BLUEPRINT.md (166KB, Nov 2025): - Current and comprehensive 10-year vision - Phase 1 marked complete (Nov 2025) - Phase 2 outlined: intelligence layer, agents, multi-platform - Section 14 specifies narrative generation with LLMs - Section 5 details 7 specialized agent workers - Section 10 outlines 8 real-time watchdog types - No Azure OpenAI integration currently exists

README.md (18KB, Nov 2025): - Phase 1: 95% complete, operational - 7,676 lines of production code - 213 passing tests (90%+ success) - 5 working data providers - WebSocket streaming functional - Agent framework exists but incomplete

PROJECT_STATUS.md (23KB, Nov 2025): - Honest assessment: Phase 1 complete, Phase 2 planning - Agent workers use mock data currently - Cache exists but needs optimization - No narrative generation yet - Platform integrations not started

Git History: - Last 10 commits mostly recent (Nov 23, 2025) - Active development on main features - Recent focus on provider integration and testing - Code is current and maintained

.env.example (1KB): - Provider API keys configured (Alpha Vantage, FMP, CCXT) - No Azure OpenAI configuration present - No LLM or agent intelligence settings - Gap identified: need Azure/OpenAI config

2. Created: PHASE2_TODO.md

Size: 1,392 lines / ~46KB
Format: AI agent-executable todo list

Structure: - Executive summary with Phase 2 goals - Azure OpenAI configuration prerequisites - 10 detailed tasks with AI-ready prompts - Task dependency graph - 40-50 hour timeline with 4-week sequence - Security considerations - Completion checklist

Each Task Includes: 1. Priority level 2. Time estimate 3. Dependencies 4. Objective (what to accomplish) 5. Context (current state, gaps, references) 6. Detailed AI agent prompt (not vague, highly specific) 7. Files to create/modify with line counts 8. Success criteria for validation

3. Task Breakdown

Task 1: Azure OpenAI Client Integration - Create LLM abstraction layer - Support Azure OpenAI, OpenAI, Anthropic - Narrative generation, sentiment analysis, summarization - Error handling, retry logic, rate limiting - ~250-300 lines of implementation - ~150-200 lines of tests

Task 2: Narrative Generation Engine - Multi-language support (EN, ES, FR, JP, ZH) - Adaptive expertise levels (beginner → quant) - Market context, technical, fundamentals, sentiment narratives - Compliance disclaimer injection - ~500-700 lines across modules - ~200-250 lines of tests

Task 3: Enhanced Agent Workers - 7 workers split into separate modules (maintainability) - Real data from providers (not mocks) - Azure OpenAI for insights and interpretations - Financial calculations (P/E, RSI, MACD, etc.) - ~900-1200 lines across 7 worker files - ~300-400 lines of tests per worker

Task 4: Real-time Event Intelligence - 8 watchdog detectors in separate files - Earnings anomalies, volume spikes, whale movements, etc. - Event stream (Kafka/Redis) - WebSocket broadcasting - ~1200-1500 lines across 8 detector files - ~400-500 lines of tests

Task 5: Advanced Cache Optimization - Predictive cache warming - LFU/LRU eviction policies - Batch update scheduler - Dynamic TTL based on market conditions - Cache analytics and metrics - ~600-800 lines of optimization code - ~300-400 lines of tests and benchmarks

Task 6: MCP Tool Integration - Add narratives to search-by-symbol - Add narratives to search-by-coin - Create get_narrative tool - Narrative caching with dynamic TTL - ~300-400 lines of integration code - ~100-150 lines of tests

Task 7: ChatGPT GPT Marketplace - GPT configuration (actions, instructions) - API adapter for GPT Actions - User profiling (expertise detection) - Compliance guard - Public API endpoints - ~700-900 lines across modules - ~200-250 lines of tests

Task 8: Telegram Bot Integration - Bot commands (/search, /subscribe, etc.) - MCP backend integration - Watchdog event forwarding - User preferences and subscriptions - Message formatting - ~700-900 lines of bot code - ~300-400 lines of tests

Task 9: Session Management - Redis + PostgreSQL session store - Context tracking across queries - MCP tool: create-analysis-session - Session cleanup job - ~450-550 lines of session code - ~200-250 lines of tests

Task 10: Performance Testing - Load testing (1000 concurrent users) - Stress testing and benchmarks - Component profiling - Bottleneck identification - Performance regression detection - ~900-1100 lines of testing code - Performance reports and dashboards

Total Estimated Work: 40-50 hours / ~7,000-9,000 lines of new code

Key Innovations

1. AI Agent-Friendly Prompts

Before (typical vague todo): 

- [ ] Implement narrative generation
- [ ] Add agent workers
- [ ] Create watchdog system

After (PHASE2_TODO.md approach): 

Create a narrative generation engine at fiml/narrative/generator.py that:

1. Implements NarrativeGenerator class with methods:
   - async def generate_narrative(analysis: ComprehensiveAnalysis, language: str = "en") -> Narrative
   - async def _generate_market_context(asset: Asset, cached_data: CachedData) -> NarrativeSection
   [... detailed method signatures ...]

2. Use Azure OpenAI client from Task 1 for:
   - Generating natural language summaries from structured data
   - Identifying key insights from multi-source analysis
   [... specific requirements ...]

3. Implement prompt templates for each narrative type:
   - Market context: price movement, volume analysis, 52-week positioning
   [... detailed specs ...]

[... continues with file references, patterns, success criteria ...]

2. Azure OpenAI Integration

Complete configuration template: 

# Azure OpenAI Configuration
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=your_azure_openai_api_key
AZURE_OPENAI_API_VERSION=2024-02-15-preview
AZURE_OPENAI_DEPLOYMENT_NAME=gpt-5.1

# Model Configuration
AZURE_OPENAI_MODEL=gpt-5.1
AZURE_OPENAI_MAX_TOKENS=2000

# Narrative Generation Settings
ENABLE_NARRATIVE_GENERATION=true
NARRATIVE_LANGUAGE_DEFAULT=en
NARRATIVE_STYLE=professional

3. Code Organization Improvements

From Code Review Feedback: - Split workers: 1 file (1000 lines) → 7 files (~150 lines each) - Split watchdogs: 1 file (1000 lines) → 8 files (~120 lines each) - Dynamic cache TTL instead of fixed - Follows single responsibility principle - Better testability and maintainability

4. Dependency Management

Clear task sequencing: 

Task 1 (Azure OpenAI) → Task 2 (Narrative) → Task 6 (MCP Integration)
                       ↓                            ↓
                    Task 3 (Agents) → Task 7 (ChatGPT)
                                    ↓              ↓
Task 5 (Cache) ─────────────────→ Task 4 (Watchdog) → Task 8 (Telegram)
Task 9 (Sessions) ──────────────────────────────────────┘

All Tasks → Task 10 (Performance)

Document Freshness Analysis

Document Date Status Phase 2 Relevance
BLUEPRINT.md Nov 2025 Current High - comprehensive vision
README.md Nov 2025 Current High - accurate Phase 1 state
PROJECT_STATUS.md Nov 2025 Current High - honest assessment
PHASE2_SETUP.md Nov 2025 Current Medium - provider setup
PHASE2_SUMMARY.md Nov 2025 Current Medium - Phase 2 work done
.env.example Nov 2025 Current Critical - missing Azure config

Conclusion: All documents are fresh and current. BLUEPRINT.md provides the strategic vision, and current docs accurately reflect Phase 1 completion state. The gap identified is Azure OpenAI configuration, which PHASE2_TODO.md addresses.

Code Stage Analysis

Current State (Nov 23, 2025): - Phase 1: 95% complete ✅ - Working: MCP server, 5 providers, arbitration, cache, WebSocket - Framework exists: Agents (7 workers), FK-DSL parser, Docker deployment - Gaps: Agent logic (mocks only), narrative generation, platform integrations - Tests: 213 passing (90%+ success) - Code quality: A- grade, clean architecture

Phase 2 Readiness: - ✅ Strong foundation for building on - ✅ Clear architecture patterns established - ✅ Provider system extensible - ✅ Cache infrastructure ready - ⚠️ Need Azure OpenAI integration - ⚠️ Agent workers need real implementations - ⚠️ Platform integrations from scratch

Assessment: Excellent position to start Phase 2. Solid Phase 1 base, clear gaps identified, and PHASE2_TODO.md provides detailed roadmap.

Azure OpenAI Considerations

Current State

  • No Azure OpenAI in codebase
  • No .env configuration for Azure
  • No LLM abstraction layer
  • BLUEPRINT.md mentions narrative generation needing LLMs

PHASE2_TODO.md Solution

Task 1 creates complete Azure OpenAI integration: - Client abstraction supporting Azure, OpenAI, Anthropic - Retry logic and error handling - Rate limiting and timeout management - Fallback to mock data if unavailable - Health checks and monitoring

Environment Configuration: 

# Primary: Azure OpenAI
AZURE_OPENAI_ENDPOINT=...
AZURE_OPENAI_API_KEY=...

# Fallback: OpenAI Direct
OPENAI_API_KEY=...

# Alternative: Anthropic Claude
ANTHROPIC_API_KEY=...

Use Cases: 1. Narrative generation (Task 2) 2. Agent insights (Task 3) 3. Sentiment analysis (Task 3) 4. Event interpretation (Task 4) 5. User profiling (Task 7)

Success Validation

Checklist for Phase 2 Complete:

Intelligence Layer: - [ ] Azure OpenAI client operational - [ ] Narratives in 5 languages - [ ] 7 agent workers with real data - [ ] 8 watchdogs detecting anomalies

Performance: - [ ] Cache hit rate >90% - [ ] P95 latency <200ms - [ ] 1000 concurrent users

Platform Distribution: - [ ] ChatGPT GPT deployed - [ ] Telegram bot live - [ ] Session management working

Quality: - [ ] 95%+ test coverage - [ ] Performance targets met - [ ] Security audit passed - [ ] Documentation complete

Recommendations

For AI Agents Executing Tasks

  1. Start with Task 1 (Azure OpenAI Client)
  2. Foundation for Tasks 2, 3, 4, 7
  3. Critical dependency

  4. Test thoroughly before proceeding

  5. Follow the sequence in PHASE2_TODO.md

  6. Dependencies clearly mapped
  7. Parallel work opportunities identified

  8. Integration points specified

  9. Use the detailed prompts

  10. Each prompt includes context, patterns, references
  11. File paths and line counts provided

  12. Success criteria for validation

  13. Test incrementally

  14. Each task has test requirements
  15. Integration tests at boundaries
  16. Performance benchmarks at end

For Human Reviewers

  1. Configuration First
  2. Set up Azure OpenAI credentials
  3. Test connectivity before implementation

  4. Verify .env template

  5. Monitor Progress

  6. Each task has clear success criteria
  7. Code review at task completion

  8. Integration test after each 2-3 tasks

  9. Quality Gates

  10. Test coverage >90% per task
  11. Code review comments addressed
  12. Performance benchmarks pass

Files Created/Modified

New Files

  1. PHASE2_TODO.md (1,392 lines)
  2. Comprehensive Phase 2 todo list
  3. AI agent-executable prompts

  4. Azure OpenAI integration plan

  5. PHASE2_REVIEW_SUMMARY.md (this file)

  6. Document review summary
  7. Task breakdown analysis
  8. Implementation recommendations

Modified Files

  • None (documentation only)

Timeline Estimate

Conservative Estimate: 40-50 hours of development

Recommended Schedule:

Week 1: Foundation (14 hours) - Task 1: Azure OpenAI Client (3h) - Task 2: Narrative Generation (5h) - Task 3: Enhanced Agents (6h)

Week 2: Optimization (13 hours) - Task 5: Cache Optimization (5h) - Task 6: MCP Integration (4h) - Task 9: Session Management (4h)

Week 3: Platforms (15 hours) - Task 7: ChatGPT Integration (5h) - Task 8: Telegram Bot (6h) - Task 4: Watchdog System (4h)

Week 4: Testing (8 hours) - Task 10: Performance Testing (6h) - Integration testing (2h)

Total: 50 hours / 4 weeks

Conclusion

✅ All requirements met:

  1. ✅ Reviewed all .md files with freshness analysis
  2. ✅ Analyzed BLUEPRINT.md (166KB, current, comprehensive)
  3. ✅ Assessed current code stage (Phase 1 95% complete)
  4. ✅ Reviewed last 10 commits (active development, current)
  5. ✅ Created Phase 2 todo list with Azure OpenAI integration
  6. ✅ Provided AI agent-friendly prompts (not vague titles)
  7. ✅ Considered .env and Azure endpoint configuration
  8. ✅ Organized for AI agent execution

Deliverable: PHASE2_TODO.md is production-ready for Phase 2 implementation.

Status: ✅ Complete and ready for review.

Created: November 23, 2025
By: AI Agent Code Review Task
For: FIML Phase 2 Planning