Aller au contenu principal

Site Document Workflow Management - Brownfield User Story

Story Overview​

Epic: Site & Cadastre Management Enhancement
Story: Story 1 - Site Document Workflow Management
Story Type: Enhancement (Brownfield)
Priority: High
Estimated Effort: 8 story points

User Story​

As a site manager
I want to manage site document requirements and workflow automation based on site status progression
So that I can efficiently track document compliance, automate status transitions, and ensure complete site documentation throughout the property lifecycle

Context & Integration​

Current System Integration Points:

  • Extends existing SiteService (/Users/ayoub/projects/emtb/apps/api/src/site/site.service.ts)
  • Leverages existing Site Prisma model with documentsManquantsClients/documentsManquantsCadastres fields
  • Integrates with existing SiteStatus enum progression (NOUVEAU → DEMANDE_DOCUMENTS_CADASTRAUX → DOCUMENTS_CADASTREAUX_RECUS)
  • Maintains compatibility with existing Client-Site-Cadastre-Reclamation-ApporteurAffaire relationships
  • Utilizes existing JSON fields (bilanAudit, typePropriete, documentsReference, mandatCadre, demande)

Existing Functionality to Preserve:

  • All CRUD operations (GET /sites, POST /sites, PATCH /sites/:id, DELETE /sites/:id)
  • Site creation with default status NOUVEAU
  • Relationship loading (client, cadastre, reclamations, apporteursAffaires, facturesPartenaires)
  • Reference generation and client-based site querying (GET /sites/client/:clientId)
  • Existing status progression and JSON field management

Acceptance Criteria​

AC1: Document Requirement Definition & Status-Based Tracking​

Given I am managing site documentation
When I create or update site status
Then the system should:

  • Define document requirements for each SiteStatus (NOUVEAU, DEMANDE_DOCUMENTS_CADASTRAUX, DOCUMENTS_CADASTREAUX_RECUS, etc.)
  • Automatically update documentsManquantsClients and documentsManquantsCadastres fields based on status
  • Track document categories: Client Documents, Cadastral Documents, Technical Documents, Legal Documents, Property Documents
  • Maintain document requirement templates per status with customizable requirements
  • Generate document checklist based on current site status and property type

AC2: Automated Status Progression with Document Validation​

Given a site with specific document requirements
When documents are uploaded or validated
Then the system should:

  • Validate document completeness before allowing status progression
  • Automatically transition from NOUVEAU to DEMANDE_DOCUMENTS_CADASTRAUX when client documents are complete
  • Progress to DOCUMENTS_CADASTREAUX_RECUS when cadastral documents are received and validated
  • Block invalid status transitions when required documents are missing
  • Update documentsManquantsClients/documentsManquantsCadastres fields in real-time
  • Trigger DOCUMENTS_MANQUANTS status when required documents become incomplete

AC3: Document Upload & Management with Site Integration​

Given I need to manage site documents
When I upload or manage documents for a site
Then the system should:

  • Accept PDF, DOC, DOCX, JPG, PNG file formats up to 20MB per file
  • Categorize documents by type (Client, Cadastral, Technical, Legal, Property)
  • Link documents to clientReferenceInterne and numeroDossierClient for client-related docs
  • Store metadata including upload date, validation status, expiration dates
  • Support document versioning with change tracking
  • Enable bulk document operations for site collections
  • Update site.documentsReference JSON field with document metadata

AC4: Document Missing Alerts & Completion Tracking​

Given sites with incomplete documentation
When the system evaluates document completeness
Then it should:

  • Generate real-time alerts for missing documents based on site status
  • Update documentsManquantsClients field with specific missing client document names
  • Update documentsManquantsCadastres field with missing cadastral document requirements
  • Provide document completion percentage per site
  • Send notifications when document deadlines approach
  • Track document validation status (PENDING, VALIDATED, REJECTED, EXPIRED)
  • Generate compliance reports showing site document status across all sites

AC5: Client & Cadastre Relationship Integration​

Given the existing site relationship structure
When managing site documents
Then the system should:

  • Integrate with Client documents through clientId relationship
  • Leverage Cadastre information through cadastreId for cadastral document requirements
  • Use clientReferenceInterne for client document cross-referencing
  • Utilize numeroDossierClient for client-specific document tracking
  • Maintain ApporteurAffaire document access based on site relationships
  • Preserve all existing relationship loading patterns in site queries
  • Support document sharing between related sites of the same client

AC6: Status-Based Document Workflow Automation​

Given site status changes and document operations
When site workflow progresses
Then the system should:

  • Automate document requirement updates when status changes
  • Validate document completeness before status transitions
  • Generate document requests to clients/cadastres based on missing requirements
  • Update site status automatically when all required documents are validated
  • Trigger BILAN_AUDIT status when technical document review is complete
  • Support SITE_ARCHIVE status with document archival and retention rules
  • Handle RECLAMATION_EN_COURS status with reclamation-specific document requirements

AC7: Evolution Tracking & Comprehensive Audit Trail​

Given document workflow operations
When any site document action occurs
Then the system should:

  • Log all document uploads, validations, deletions, and status changes
  • Track user responsible for each action with detailed timestamps
  • Maintain document version history with change reasons and impact analysis
  • Audit site status transitions with document-based justifications
  • Record document requirement changes and template modifications
  • Store audit data in separate audit table optimized for reporting
  • Generate compliance reports for regulatory requirements
  • Track document workflow performance metrics and bottlenecks

AC8: Backward Compatibility & Data Integrity​

Given existing site functionality
When the document workflow system is implemented
Then it should:

  • Preserve all existing API endpoints without breaking changes
  • Maintain existing site creation/update workflows with status NOUVEAU
  • Keep existing documentsManquantsClients/documentsManquantsCadastres field functionality
  • Preserve performance of existing site queries with proper indexing
  • Support existing Swagger documentation patterns and response formats
  • Maintain JSON field compatibility (bilanAudit, typePropriete, documentsReference, etc.)
  • Keep site-client-cadastre-reclamation relationship integrity intact

Technical Requirements​

Database Schema Changes (Additive Only)​

-- Site document requirements template
CREATE TABLE site_document_requirements (
  id SERIAL PRIMARY KEY,
  site_status VARCHAR(50) NOT NULL, -- Maps to SiteStatus enum
  property_type VARCHAR(100), -- Optional property type from typePropriete JSON
  document_category VARCHAR(50) NOT NULL, -- Client, Cadastral, Technical, Legal, Property
  document_type VARCHAR(100) NOT NULL,
  document_name VARCHAR(255) NOT NULL,
  is_required BOOLEAN DEFAULT TRUE,
  is_conditional BOOLEAN DEFAULT FALSE,
  condition_rules JSONB, -- Conditions for when this document is required
  deadline_days INTEGER, -- Days after status change when document is due
  validation_rules JSONB, -- Document validation criteria
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

-- Site documents
CREATE TABLE site_documents (
  id SERIAL PRIMARY KEY,
  site_id INTEGER NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
  requirement_id INTEGER REFERENCES site_document_requirements(id),
  category VARCHAR(50) NOT NULL, -- Client, Cadastral, Technical, Legal, Property
  document_type VARCHAR(100) NOT NULL,
  filename VARCHAR(255) NOT NULL,
  original_filename VARCHAR(255) NOT NULL,
  file_path VARCHAR(500) NOT NULL,
  file_size BIGINT NOT NULL,
  mime_type VARCHAR(100) NOT NULL,
  validation_status VARCHAR(20) NOT NULL DEFAULT 'PENDING', -- PENDING, VALIDATED, REJECTED, EXPIRED
  version INTEGER NOT NULL DEFAULT 1,
  uploaded_by INTEGER REFERENCES users(id),
  uploaded_at TIMESTAMP DEFAULT NOW(),
  validated_by INTEGER REFERENCES users(id),
  validated_at TIMESTAMP,
  validation_notes TEXT,
  expires_at TIMESTAMP,
  deadline_date TIMESTAMP,
  client_reference VARCHAR(255), -- Links to clientReferenceInterne
  dossier_number VARCHAR(255), -- Links to numeroDossierClient
  metadata JSONB, -- Additional document metadata
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

-- Site document audit trail
CREATE TABLE site_document_audit (
  id SERIAL PRIMARY KEY,
  site_id INTEGER NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
  document_id INTEGER REFERENCES site_documents(id) ON DELETE CASCADE,
  action VARCHAR(50) NOT NULL, -- UPLOAD, DOWNLOAD, VALIDATE, REJECT, DELETE, STATUS_CHANGE, REQUIREMENT_UPDATE
  previous_status VARCHAR(50),
  new_status VARCHAR(50),
  performed_by INTEGER REFERENCES users(id),
  performed_at TIMESTAMP DEFAULT NOW(),
  details JSONB, -- Action details, document changes, status transitions
  ip_address INET,
  user_agent TEXT,
  site_status_before VARCHAR(50), -- SiteStatus before action
  site_status_after VARCHAR(50) -- SiteStatus after action
);

-- Site document workflow tracking
CREATE TABLE site_document_workflow (
  id SERIAL PRIMARY KEY,
  site_id INTEGER NOT NULL REFERENCES sites(id) ON DELETE CASCADE,
  current_status VARCHAR(50) NOT NULL, -- Current SiteStatus
  required_documents_count INTEGER DEFAULT 0,
  completed_documents_count INTEGER DEFAULT 0,
  pending_documents_count INTEGER DEFAULT 0,
  missing_documents_count INTEGER DEFAULT 0,
  completion_percentage DECIMAL(5,2) DEFAULT 0.00,
  last_document_update TIMESTAMP,
  next_status VARCHAR(50), -- Next possible status based on document completion
  can_progress BOOLEAN DEFAULT FALSE,
  workflow_metadata JSONB, -- Workflow-specific data
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW(),
  UNIQUE(site_id)
);

-- Indexes for performance
CREATE INDEX idx_site_documents_site_id ON site_documents(site_id);
CREATE INDEX idx_site_documents_category ON site_documents(category);
CREATE INDEX idx_site_documents_validation_status ON site_documents(validation_status);
CREATE INDEX idx_site_documents_client_reference ON site_documents(client_reference);
CREATE INDEX idx_site_documents_dossier_number ON site_documents(dossier_number);
CREATE INDEX idx_site_document_requirements_status ON site_document_requirements(site_status);
CREATE INDEX idx_site_document_audit_site_id ON site_document_audit(site_id);
CREATE INDEX idx_site_document_audit_performed_at ON site_document_audit(performed_at);
CREATE INDEX idx_site_document_workflow_site_id ON site_document_workflow(site_id);
CREATE INDEX idx_site_document_workflow_status ON site_document_workflow(current_status);

New API Endpoints (Additive)​

// Site document workflow management
GET /sites/:id/documents - Get all site documents with metadata
POST /sites/:id/documents - Upload document for site
GET /sites/:id/documents/:documentId - Get specific document details
GET /sites/:id/documents/:documentId/download - Download document
PUT /sites/:id/documents/:documentId/validate - Validate/reject document
DELETE /sites/:id/documents/:documentId - Delete document
POST /sites/:id/documents/bulk-upload - Upload multiple documents

// Document requirement and workflow endpoints
GET /sites/:id/requirements - Get document requirements for current site status
GET /sites/:id/workflow - Get document workflow status and completion
POST /sites/:id/workflow/evaluate - Evaluate and update workflow status
PUT /sites/:id/workflow/progress - Attempt to progress site status based on documents
GET /sites/:id/missing-documents - Get detailed missing document report
POST /sites/:id/generate-requests - Generate document requests for missing items

// Status-based document operations
GET /sites/status/:status/requirements - Get document requirements for specific status
POST /sites/:id/status/validate-transition - Validate if status change is allowed based on documents
PUT /sites/:id/status/auto-progress - Auto-progress status based on document completion
GET /sites/document-compliance - Get site document compliance overview

// Integration endpoints
GET /sites/:id/client-documents - Get client documents related to site
GET /sites/:id/cadastral-documents - Get cadastral documents for site
POST /sites/:id/link-client-documents - Link existing client documents to site
GET /sites/client/:clientId/document-summary - Get document summary across all client sites

// Audit and reporting
GET /sites/:id/audit - Get complete audit trail for site documents
GET /sites/:id/compliance-report - Generate compliance report for site
GET /sites/workflow-metrics - Get workflow performance metrics
POST /sites/bulk-document-check - Check document compliance for multiple sites

Service Extensions​

// Enhanced SiteService with document workflow management
class SiteService {
  // ... existing methods preserved ...
  
  // Document workflow methods
  async uploadSiteDocument(
    siteId: number, 
    file: Express.Multer.File, 
    category: string,
    documentType: string,
    uploadedBy: number,
    metadata?: any
  ): Promise<SiteDocument>
  
  async getSiteDocuments(siteId: number, includeMetadata?: boolean): Promise<SiteDocument[]>
  async validateDocument(documentId: number, validatedBy: number, isValid: boolean, notes?: string): Promise<SiteDocument>
  async deleteDocument(documentId: number, deletedBy: number): Promise<void>
  
  // Workflow management
  async getDocumentRequirements(siteId: number): Promise<SiteDocumentRequirement[]>
  async evaluateDocumentWorkflow(siteId: number): Promise<SiteDocumentWorkflow>
  async canProgressStatus(siteId: number, targetStatus: SiteStatus): Promise<{ canProgress: boolean, missingDocs: string[] }>
  async autoProgressStatus(siteId: number, userId: number): Promise<Site>
  
  // Document requirement tracking
  async updateDocumentRequirements(siteId: number): Promise<void>
  async getMissingDocuments(siteId: number): Promise<{ clients: string[], cadastres: string[] }>
  async generateDocumentRequests(siteId: number, userId: number): Promise<void>
  
  // Enhanced status management with document validation
  async updateWithDocumentValidation(id: number, updateSiteDto: UpdateSiteDto, userId: number): Promise<Site>
  async updateStatusWithDocumentCheck(id: number, status: SiteStatus, userId: number): Promise<Site>
  
  // Integration methods
  async linkClientDocuments(siteId: number, clientDocumentIds: number[]): Promise<void>
  async getSiteClientDocuments(siteId: number): Promise<any[]>
  async getSiteCadastralDocuments(siteId: number): Promise<SiteDocument[]>
  
  // Audit and reporting
  async getDocumentAuditTrail(siteId: number, filters?: AuditFilters): Promise<SiteDocumentAudit[]>
  async generateComplianceReport(siteId: number): Promise<ComplianceReport>
  async getWorkflowMetrics(dateRange?: DateRange): Promise<WorkflowMetrics>
  
  // Bulk operations
  async bulkDocumentComplianceCheck(siteIds: number[]): Promise<BulkComplianceResult>
  async bulkUpdateDocumentRequirements(siteIds: number[]): Promise<void>
}

Document Workflow Configuration​

// Document workflow configuration
interface DocumentWorkflowConfig {
  statusRequirements: Record<SiteStatus, DocumentRequirement[]>;
  automaticProgression: boolean;
  notificationSettings: {
    missingDocumentAlerts: boolean;
    deadlineWarnings: number; // days before deadline
    statusProgressionNotification: boolean;
  };
  validationRules: {
    requireAllDocuments: boolean;
    allowConditionalRequirements: boolean;
    enforceDeadlines: boolean;
  };
  auditSettings: {
    trackAllActions: boolean;
    retentionPeriod: number; // days
    enablePerformanceMetrics: boolean;
  };
}

// Document requirement structure
interface DocumentRequirement {
  category: 'Client' | 'Cadastral' | 'Technical' | 'Legal' | 'Property';
  documentType: string;
  documentName: string;
  isRequired: boolean;
  isConditional: boolean;
  conditionRules?: any;
  deadlineDays?: number;
  validationRules?: any;
}

Test Automation Requirements​

Unit Tests​

describe('SiteDocumentWorkflowService', () => {
  // Document requirement tests
  it('should generate correct document requirements based on site status')
  it('should update requirements when site status changes')
  it('should handle conditional document requirements correctly')
  it('should validate document completeness for status progression')
  
  // Document upload and management tests
  it('should upload document and update workflow status')
  it('should validate file types and size limits (20MB max)')
  it('should link documents to clientReferenceInterne and numeroDossierClient')
  it('should create audit trail entry on all document operations')
  it('should handle document versioning correctly')
  
  // Status progression tests
  it('should allow NOUVEAU to DEMANDE_DOCUMENTS_CADASTRAUX when client docs complete')
  it('should progress to DOCUMENTS_CADASTREAUX_RECUS when cadastral docs received')
  it('should block invalid status transitions when documents missing')
  it('should update documentsManquantsClients field correctly')
  it('should update documentsManquantsCadastres field correctly')
  
  // Integration preservation tests
  it('should preserve existing site CRUD functionality')
  it('should maintain client-site relationship integrity')
  it('should keep cadastre-site relationship working')
  it('should preserve apporteur-site relationships')
  it('should maintain reclamation-site associations')
})

Integration Tests​

describe('Site Document Workflow Integration', () => {
  // Document workflow integration
  it('should complete full document workflow from upload to status progression')
  it('should handle concurrent document operations safely')
  it('should maintain transaction integrity during bulk operations')
  it('should integrate client documents with site workflow correctly')
  
  // Status-based workflow tests
  it('should automatically update site status based on document completion')
  it('should validate document requirements for each status transition')
  it('should maintain existing site status enum progression patterns')
  it('should handle RECLAMATION_EN_COURS status with reclamation documents')
  
  // Relationship integration tests
  it('should preserve client-site document cross-referencing')
  it('should maintain cadastre integration for cadastral documents')
  it('should keep apporteur access to site documents based on relationships')
  it('should support document sharing between client sites')
  
  // Audit and compliance
  it('should log all document workflow operations with proper user tracking')
  it('should generate complete compliance reports')
  it('should track workflow performance metrics accurately')
})

End-to-End Tests​

describe('Site Document Workflow E2E', () => {
  // Complete workflow scenarios
  it('should handle complete site lifecycle from NOUVEAU to DOCUMENTS_CADASTREAUX_RECUS')
  it('should manage document requirements through status progression')
  it('should integrate with existing client and cadastre management')
  it('should maintain site document compliance throughout workflow')
  
  // Status transition scenarios
  it('should validate NOUVEAU to DEMANDE_DOCUMENTS_CADASTRAUX transition')
  it('should handle DOCUMENTS_MANQUANTS status with missing document alerts')
  it('should progress to BILAN_AUDIT when technical documents complete')
  it('should support SITE_ARCHIVE with document retention')
  
  // Document requirement tracking
  it('should track client document requirements based on SiteStatus')
  it('should monitor cadastral document completion')
  it('should alert on missing documents with deadline tracking')
  it('should generate document requests to clients and cadastres')
})

Performance Tests​

describe('Document Workflow Performance', () => {
  it('should handle multiple document uploads without performance degradation')
  it('should maintain existing site query performance with document extensions')
  it('should efficiently serve document downloads and previews')
  it('should handle large audit trail queries with proper pagination')
  it('should process bulk document compliance checks efficiently')
  it('should maintain relationship loading performance (client, cadastre, reclamations)')
})

Regression Tests​

describe('Site Workflow Backward Compatibility', () => {
  it('should preserve all existing site CRUD operations')
  it('should maintain existing API response formats')
  it('should keep site creation with NOUVEAU status working')
  it('should preserve site-client-cadastre relationship loading')
  it('should maintain existing JSON field functionality (bilanAudit, typePropriete)')
  it('should keep clientReferenceInterne and numeroDossierClient fields working')
  it('should preserve existing site statistics and reference generation')
})

Change Tracking & Evolution​

Version History​

  • v1.0.0: Initial site document workflow implementation
  • Track all schema changes with migration versioning
  • Document API version compatibility matrix
  • Maintain backward compatibility for existing site operations

Monitoring & Metrics​

// Key workflow metrics to track
interface SiteDocumentMetrics {
  documentsUploaded: number;
  workflowCompletionRate: number;
  averageStatusProgressionTime: number;
  documentValidationRate: number;
  missingDocumentAlerts: number;
  statusTransitionSuccessRate: number;
  clientDocumentIntegration: number;
  cadastralDocumentCompletion: number;
  auditTrailSize: number;
  workflowPerformanceMetrics: {
    averageDocumentUploadTime: number;
    statusEvaluationTime: number;
    complianceCheckTime: number;
  };
}

Configuration Management​

// Site document workflow configuration
interface SiteDocumentWorkflowConfig {
  maxFileSize: number; // 20MB default
  allowedMimeTypes: string[];
  storageLocation: string;
  documentRetentionPeriod: number; // days
  auditRetention: number; // days
  enableAutomaticStatusProgression: boolean;
  requirementTemplateVersion: string;
  notificationSettings: {
    enableMissingDocumentAlerts: boolean;
    deadlineWarningDays: number;
    statusProgressionNotifications: boolean;
  };
  integrationSettings: {
    linkClientDocuments: boolean;
    syncCadastralDocuments: boolean;
    enableCrossReferencing: boolean;
  };
}

Definition of Done​

Functional Requirements​

  • Document requirement tracking operational for all SiteStatus values
  • Automated status progression working (NOUVEAU → DEMANDE_DOCUMENTS_CADASTRAUX → DOCUMENTS_CADASTREAUX_RECUS)
  • Document missing alerts integrated with documentsManquantsClients/documentsManquantsCadastres
  • Client and cadastre relationship integration functional
  • Document upload/validation workflow complete
  • Bulk operations and document compliance checking implemented

Technical Requirements​

  • Database migrations applied and tested with existing site data
  • New API endpoints documented in Swagger
  • Document storage security implemented with proper access controls
  • Audit logging configured for all workflow operations
  • Performance optimization with proper indexing completed

Quality Assurance​

  • Unit test coverage ≥ 90% for new document workflow functionality
  • Integration tests verify document-status workflow with existing relationships
  • Performance tests confirm no degradation in existing site operations
  • End-to-end tests validate complete workflow from NOUVEAU to final status
  • Security audit passed for document upload/download mechanisms

Compatibility Verification​

  • All existing site API endpoints return same response format
  • Site creation/update workflows unchanged (status defaults to NOUVEAU)
  • Client-site-cadastre-reclamation relationships preserved
  • Existing JSON field functionality intact (bilanAudit, typePropriete, documentsReference)
  • Reference generation and client-based site lookup functionality intact
  • Performance of existing site queries maintained

Documentation​

  • API documentation updated in Swagger for all new endpoints
  • Technical documentation for site document workflow architecture
  • Audit trail and compliance reporting documentation
  • Configuration guide for document workflow settings
  • Integration guide for client and cadastre document linking

Risk Mitigation​

Primary Risks & Mitigations​

  1. Document Workflow Complexity Risk

    • Mitigation: Phased implementation starting with basic document tracking, comprehensive requirement testing

  2. Performance Impact on Existing Site Operations

    • Mitigation: Efficient indexing, lazy loading of documents, separate audit table, query optimization

  3. Breaking Existing Site-Client-Cadastre Relationships

    • Mitigation: Additive-only changes, comprehensive relationship testing, existing query preservation

  4. Status Progression Logic Complexity

    • Mitigation: Clear status transition rules, extensive unit testing, rollback mechanisms for status changes

Rollback Strategy​

  • Database migrations are reversible with data preservation
  • New document workflow endpoints can be disabled via feature flags
  • Document storage can be isolated without affecting site core functionality
  • Existing site CRUD operations remain fully functional during rollback
  • Status progression can revert to manual updates if automation fails

Dependencies​

Internal Dependencies​

  • Existing Site module structure and service patterns maintained
  • Client module integration for client document cross-referencing
  • Cadastre module integration for cadastral document requirements
  • User authentication system for document access control and audit trailing
  • File upload middleware configuration with security controls

External Dependencies​

  • File storage system (local filesystem or cloud storage) with 20MB file support
  • Document preview generation service for PDF/image files
  • Notification service for missing document alerts and deadline warnings
  • Audit logging infrastructure for compliance reporting
  • ZIP archive generation library for bulk document downloads

Story Status: Ready for Implementation
Last Updated: 2025-09-09
Reviewed By: Technical Lead
Approved By: Product Owner