stock-bot/database/postgres/providers
2025-06-14 09:17:48 -04:00
..
01-ib-simple.sql work on ib and cleanup 2025-06-14 09:17:48 -04:00
01-ib.sql work on postgress / will prob remove and work on ib exchanges and symbols 2025-06-13 19:59:35 -04:00
README.md work on postgress / will prob remove and work on ib exchanges and symbols 2025-06-13 19:59:35 -04:00

Interactive Brokers Database Setup

This directory contains the PostgreSQL schema setup for Interactive Brokers data.

Quick Setup

1. Create the Schema and Tables

# Run the SQL schema setup
bun run db:setup

# Or manually with psql:
psql -U postgres -d stock_bot -f database/postgres/providers/01-ib.sql

2. Populate with Exchange Data

# Populate exchanges from ib-exchanges.json
bun run db:populate-ib

# Or run the complete setup (schema + data):
bun run db:setup-ib

What Gets Created

📊 Schema: ib_data

  • exchanges - All IB trading exchanges with metadata
  • asset_types - Types of financial instruments (Stocks, Options, etc.)
  • exchange_assets - Many-to-many mapping of exchanges to asset types
  • securities - Individual tradeable instruments
  • market_data - Real-time and historical price data
  • data_fetch_jobs - Queue for data collection tasks

🔍 Views

  • exchanges_with_assets - Exchanges with their supported asset types
  • latest_market_data - Most recent market data per security
  • securities_full_view - Securities with full exchange and asset type info

Functions

  • get_or_create_exchange() - Utility to insert/update exchanges
  • add_assets_to_exchange() - Parse and add asset types to exchanges

Database Structure

-- Example queries you can run after setup:

-- View all exchanges with their supported assets
SELECT * FROM ib_data.exchanges_with_assets LIMIT 10;

-- Count exchanges by region
SELECT region, COUNT(*) 
FROM ib_data.exchanges 
GROUP BY region 
ORDER BY COUNT(*) DESC;

-- Find exchanges that support stocks
SELECT e.exchange_code, e.exchange_name, e.country
FROM ib_data.exchanges e
JOIN ib_data.exchange_assets ea ON e.id = ea.exchange_id
JOIN ib_data.asset_types at ON ea.asset_type_id = at.id
WHERE at.code = 'Stocks'
ORDER BY e.exchange_code;

Environment Variables

Set these in your .env file for the populate script:

DB_HOST=localhost
DB_PORT=5432
DB_NAME=stock_bot
DB_USER=postgres
DB_PASSWORD=your_password

Integration with Your Code

The schema is designed to work with your existing ib.tasks.ts file:

// Your fetchSession() function can now store data like:
import { query } from '@stock-bot/postgres-client';

// Create a fetch job
await query(`
  INSERT INTO ib_data.data_fetch_jobs (job_type, status, metadata)
  VALUES ('SYMBOL_SUMMARY', 'PENDING', $1)
`, [{ url: 'https://...', proxy: '...' }]);

// Store exchange data
const exchangeId = await query(`
  SELECT ib_data.get_or_create_exchange($1, $2, $3, $4, $5)
`, ['NASDAQ', 'NASDAQ Global Select Market', 'United States', 'Americas', 'US']);

Next Steps

  1. Run the setup scripts
  2. 🔧 Update your IB tasks to use the database
  3. 📊 Start collecting market data
  4. 🚀 Build your trading strategies on top of this data layer!