Data Policy

1. Data Sources and Licensing

1.1 Primary Data Sources

All market data is sourced directly from licensed exchanges and authorized data vendors:

1.2 Supplementary Data Sources

• Tiingo: Backup pricing data and US market coverage
• Financial Modeling Prep: Fundamental data and ratios
• Exchange Websites: Corporate announcements and regulatory filings
• Bloomberg/Reuters: News and research (where licensed)

2. Data Quality and Validation

2.1 Quality Framework

2.2 Quality Metrics

We maintain the following quality standards:

• Completeness: ≥99.5% of expected records received daily
• Accuracy: ≥99.9% price accuracy vs. exchange official data
• Timeliness: T+1 delivery by SLA for each exchange
• Consistency: No gaps in trading day coverage
• Corporate Actions: ≥95% same-day capture of ex-dates

2.3 Data Validation Checks

• Price Validation: OHLC relationship checks (O,C within H-L range)
• Volume Validation: Non-negative values, reasonable thresholds
• Date Validation: Trading calendar alignment, no future dates
• Corporate Actions: Ex-date validation, ratio reasonableness
• Cross-Reference: Multi-source validation for critical securities

3. Data Lineage and Metadata

3.1 Lineage Tracking

Every data point includes comprehensive lineage metadata:

Standard metadata fields included with every response:

3.2 Version Control

• Dataset Versions: Semantic versioning (major.minor.patch)
• Schema Evolution: Backward compatibility maintained
• Correction Tracking: Full audit trail for data corrections
• Historical Snapshots: Point-in-time data reconstruction

3.3 Correction and Restatement Policy

When data corrections are received:

• Original data is preserved with correction flag
• Corrected data is published with new as_of timestamp
• Correction notifications sent to affected API consumers
• Historical queries can access both original and corrected values

4. Symbol Mapping and Identifiers

4.1 Universal Identifiers

We maintain cross-reference mappings for all instruments:

• ISIN: International Securities Identification Number
• FIGI: Financial Instrument Global Identifier
• LEI: Legal Entity Identifier (for issuers)
• Local Ticker: Exchange-specific symbol
• Bloomberg Ticker: Bloomberg terminal symbol
• Reuters RIC: Reuters Instrument Code

4.2 Corporate Action Handling

Symbol changes and corporate actions are tracked with full history:

• Stock splits and bonus issues with adjustment factors
• Ticker symbol changes with effective dates
• Mergers and acquisitions with successor instruments
• Delistings with final trading dates

5. Data Retention and Archival

5.1 Retention Periods

5.2 Backup and Disaster Recovery

• Real-time Replication: Multi-region database replication
• Daily Backups: Encrypted backups to object storage
• Archive Storage: AWS Glacier for long-term retention
• Recovery Testing: Monthly disaster recovery drills

6. Privacy and Confidentiality

6.1 Market Data Privacy

• No personally identifiable information in market data
• Trading data is aggregated and anonymized
• Client usage patterns kept confidential
• Access controls prevent unauthorized data exposure

6.2 Client Data Protection

• API usage logs encrypted at rest and in transit
• Personal information handled per Privacy Policy
• No sharing of client-specific usage with third parties
• Secure deletion of data upon account termination

7. Compliance and Regulatory Alignment

7.1 Exchange Compliance

We maintain compliance with all exchange requirements:

• Monthly usage reporting to data vendors
• Professional vs. non-professional user classification
• Redistribution monitoring and control
• Attribution requirement enforcement

7.2 Regulatory Standards

• MiFID II: Transparency and reporting requirements
• GDPR: Data protection for EU clients
• SOX: Financial reporting controls
• ISO 27001: Information security management

8. Data Access and API Standards

8.1 API Design Principles

• RESTful Design: Standard HTTP methods and status codes
• Consistent Schemas: Standardized response formats
• Pagination: Cursor-based pagination for large datasets
• Filtering: Flexible query parameters
• Rate Limiting: Fair usage enforcement

8.2 Response Format Standards

{
  "data": {
    "symbol": "QNBK.QSE",
    "date": "2025-09-27",
    "open": 18.50,
    "high": 18.75,
    "low": 18.40,
    "close": 18.65,
    "volume": 125000
  },
  "metadata": {
    "source_exchange": "QSE",
    "license_id": "QSE-2024-BASIC",
    "dataset_version": "1.0",
    "as_of": "2025-09-28T06:00:00Z",
    "attribution": "Data provided by Qatar Stock Exchange"
  }
}
                

9. Data Usage Monitoring

9.1 Usage Tracking

We monitor all data access for compliance and optimization:

• Real-time API call tracking
• Data volume and frequency monitoring
• Geographic access pattern analysis
• License compliance verification

9.2 Automated Alerts

• Usage limit approaching notifications
• Unusual access pattern detection
• Data quality threshold breaches
• License violation warnings

10. Future Data Expansion

10.1 Planned Additions

• Real-time streaming data (pending licenses)
• Options and derivatives data
• Alternative data sources (ESG, sentiment)
• Fixed income and Islamic finance products

10.2 Technology Roadmap

• GraphQL API endpoints for flexible queries
• WebSocket streaming for real-time data
• Machine learning data quality scoring
• Blockchain-based data integrity verification