work on market-data-gateway

docs/data-services/README.md

@@ -0,0 +1,43 @@
+# Data Services
+
+Data services manage data storage, processing, and discovery across the trading platform, providing structured access to market data, features, and metadata.
+
+## Services
+
+### Data Catalog
+- **Purpose**: Data asset management and discovery
+- **Key Functions**:
+  - Data asset discovery and search capabilities
+  - Metadata management and governance
+  - Data lineage tracking
+  - Schema registry and versioning
+  - Data quality monitoring
+
+### Data Processor
+- **Purpose**: Data transformation and processing pipelines
+- **Key Functions**:
+  - ETL/ELT pipeline orchestration
+  - Data cleaning and normalization
+  - Batch and stream processing
+  - Data validation and quality checks
+
+### Feature Store
+- **Purpose**: ML feature management and serving
+- **Key Functions**:
+  - Online and offline feature storage
+  - Feature computation and serving
+  - Feature statistics and monitoring
+  - Feature lineage and versioning
+  - Real-time feature retrieval for ML models
+
+### Market Data Gateway
+- **Purpose**: Market data storage and historical access
+- **Key Functions**:
+  - Historical market data storage
+  - Data archival and retention policies
+  - Query optimization for time-series data
+  - Data backup and recovery
+
+## Architecture
+
+Data services create a unified data layer that enables efficient data discovery, processing, and consumption across the platform. They ensure data quality, consistency, and accessibility for both operational and analytical workloads.