{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Forecast Revisions with TimeDB\n",
"\n",
"This notebook demonstrates how to work with overlapping forecast revisions in TimeDB.\n",
"\n",
"#### What you'll learn:\n",
"1. **Creating overlapping forecasts** - Multiple forecasts for the same series with different known_time\n",
"2. **Reading all forecast revisions** - Querying all versions of forecasts with a single query\n",
"3. **Visualizing forecast revisions** - Plotting overlapping forecasts to see how they evolve\n",
"\n",
"**Key Concepts:**\n",
"- `known_time`: When the forecast was made (batch_start_time)\n",
"- `valid_time`: The time period the forecast predicts\n",
"- Overlapping forecasts: Multiple forecasts made at different times predicting the same future periods\n",
"- Series identification: Each series has a unique combination of name, unit, and labels\n",
"\n",
"## Part 1: Setup and Create Database Schema\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Creating database schema...\n",
"✓ Schema created successfully\n"
]
}
],
"source": [
"# Import timedb SDK\n",
"import timedb as td\n",
"\n",
"# Import pandas for data manipulation and matplotlib for plotting\n",
"import pandas as pd\n",
"import numpy as np\n",
"from datetime import datetime, timedelta, timezone\n",
"import matplotlib.pyplot as plt\n",
"\n",
"td.delete()\n",
"td.create()\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✓ Got/created forecast series (series_id: a161b2fd-029d-49fa-ac45-55dbdc8d0b83)\n",
" name=\"forecast\", unit=\"MW\", labels={\"type\": \"power_forecast\", \"model\": \"sinus_with_error\"}\n",
"✓ Got/created actual series (series_id: feafad38-e0a8-43a5-84cf-79e2204180ff)\n",
" name=\"actual\", unit=\"MW\", labels={\"type\": \"power_actual\"}\n"
]
}
],
"source": [
"\n",
"# Create series for forecast and actual values\n",
"# Use different names to distinguish between forecast and actual\n",
"# get_or_create_series returns existing series if it already exists with same name/unit/labels\n",
"forecast_series_id = td.get_or_create_series(\n",
" name='forecast',\n",
" unit='MW',\n",
" labels={'type': 'power_forecast', 'model': 'sinus_with_error'},\n",
" description='Forecasted power values with overlapping revisions'\n",
")\n",
"\n",
"actual_series_id = td.get_or_create_series(\n",
" name='actual',\n",
" unit='MW',\n",
" labels={'type': 'power_actual'},\n",
" description='Actual power values (ground truth)'\n",
")\n",
"\n",
"print(f'✓ Got/created forecast series (series_id: {forecast_series_id})')\n",
"print(f' name=\"forecast\", unit=\"MW\", labels={{\"type\": \"power_forecast\", \"model\": \"sinus_with_error\"}}')\n",
"print(f'✓ Got/created actual series (series_id: {actual_series_id})')\n",
"print(f' name=\"actual\", unit=\"MW\", labels={{\"type\": \"power_actual\"}}')\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Part 2: Create 4 Forecasts with Shifting Valid Times\n",
"\n",
"We'll create 4 forecasts in a loop, where each forecast's `valid_time` range starts at its `known_time`. The `batch_start_time` will default to the current time when the script runs. All forecasts:\n",
"- Use the same series_id (with name='forecast', unit='MW', labels={'type': 'power_forecast', 'model': 'sinus_with_error'})\n",
"- Have a 3-day horizon with hourly resolution (72 hours)\n",
"- Have `known_time` set 1 day apart\n",
"- Each forecast's `valid_time` starts at its `known_time`, so later forecasts are shifted forward in time\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Creating 4 forecasts...\n",
"Forecast horizon: 72 hours (3 days)\n",
"Each forecast's valid_time starts at its known_time\n",
"\n",
"Data values inserted successfully.\n",
"✓ Forecast 1/4 created\n",
" known_time: 2025-01-01 00:00:00+00:00\n",
" valid_time range: 2025-01-01 00:00:00+00:00 to 2025-01-03 23:00:00+00:00\n",
" Batch ID: 5f89e0a7-2c19-4d6a-832c-b26483697419\n",
"\n",
"Data values inserted successfully.\n",
"✓ Forecast 2/4 created\n",
" known_time: 2025-01-02 00:00:00+00:00\n",
" valid_time range: 2025-01-02 00:00:00+00:00 to 2025-01-04 23:00:00+00:00\n",
" Batch ID: ffe5a911-bed5-4728-bfcd-e87a2534b9fe\n",
"\n",
"Data values inserted successfully.\n",
"✓ Forecast 3/4 created\n",
" known_time: 2025-01-03 00:00:00+00:00\n",
" valid_time range: 2025-01-03 00:00:00+00:00 to 2025-01-05 23:00:00+00:00\n",
" Batch ID: 6c28f629-2601-41ec-9548-b48e6d053869\n",
"\n",
"Data values inserted successfully.\n",
"✓ Forecast 4/4 created\n",
" known_time: 2025-01-04 00:00:00+00:00\n",
" valid_time range: 2025-01-04 00:00:00+00:00 to 2025-01-06 23:00:00+00:00\n",
" Batch ID: cc292874-cf7b-4ce2-9f15-05522ae42052\n",
"\n",
"✓ All 4 forecasts created successfully!\n",
"Each forecast uses the same series_id: a161b2fd-029d-49fa-ac45-55dbdc8d0b83\n",
"Series metadata: name='forecast', unit='MW', labels={'type': 'power_forecast', 'model': 'sinus_with_error'}\n",
"\n",
"============================================================\n",
"Inserting actual values (true sinus function)...\n",
"============================================================\n",
"Data values inserted successfully.\n",
"✓ Actual values inserted successfully!\n",
" Series ID: feafad38-e0a8-43a5-84cf-79e2204180ff\n",
" Valid time range: 2025-01-01 00:00:00+00:00 to 2025-01-06 23:00:00+00:00\n",
" Number of data points: 144\n",
" Batch ID: 01d5c25c-eb59-44ff-b12a-8d04265c52a8\n",
"\n",
"Forecast series: name='forecast', unit='MW', labels={'type': 'power_forecast', 'model': 'sinus_with_error'}\n",
"Actual series: name='actual', unit='MW', labels={'type': 'power_actual'}\n",
"Each forecast's valid_time starts at its known_time, shifted forward by 1 day for each subsequent forecast\n"
]
}
],
"source": [
"# Base time for first forecast\n",
"base_valid_time = datetime(2025, 1, 1, 0, 0, tzinfo=timezone.utc)\n",
"\n",
"# Forecast horizon: 3 days with hourly resolution\n",
"forecast_horizon_hours = 72\n",
"\n",
"# Number of forecasts to create\n",
"num_forecasts = 4\n",
"\n",
"# Store forecast metadata\n",
"forecast_metadata = []\n",
"\n",
"print(f\"Creating {num_forecasts} forecasts...\")\n",
"print(f\"Forecast horizon: {forecast_horizon_hours} hours (3 days)\")\n",
"print(f\"Each forecast's valid_time starts at its known_time\")\n",
"print()\n",
"\n",
"series_ids_map = None\n",
"\n",
"for i in range(num_forecasts):\n",
" # Each forecast's known_time is 1 day apart, starting from base_valid_time\n",
" known_time = base_valid_time + timedelta(days=i)\n",
" \n",
" # Each forecast's valid_time range starts at its known_time\n",
" # Generate valid_time range (3 days, hourly) starting from known_time\n",
" valid_times = [known_time + timedelta(hours=j) for j in range(forecast_horizon_hours)]\n",
" \n",
" # Generate forecast values: sinus function base + random walk error\n",
" # This makes forecasts look realistic - following a pattern with small forecast errors\n",
" \n",
" # Base sinus function parameters\n",
" base_power = 100.0 # Mean power level\n",
" amplitude = 30.0 # Amplitude of sinus function (daily pattern)\n",
" period_hours = 24 # Period of sinus (24 hours for daily pattern)\n",
" \n",
" # Generate sinus function for the base pattern\n",
" # Use hours from start of valid_time range as x-axis\n",
" hours_from_start = np.arange(forecast_horizon_hours)\n",
" sinus_base = base_power + amplitude * np.sin(2 * np.pi * hours_from_start / period_hours)\n",
" \n",
" # Add random walk error - small relative to sinus amplitude\n",
" # Error represents forecast uncertainty that accumulates over time\n",
" np.random.seed(42 + i) # Different seed for each forecast for reproducibility\n",
" error_std = amplitude * 0.03 # Error std is 1% of sinus amplitude (small relative to pattern)\n",
" error_steps = np.random.normal(0, error_std, forecast_horizon_hours)\n",
" error_walk = np.cumsum(error_steps)\n",
" \n",
" # Combine sinus base with error walk\n",
" forecast_values = sinus_base + error_walk\n",
" forecast_values = np.round(forecast_values, 2).tolist()\n",
" \n",
" # Create DataFrame (column name 'forecast' matches series name)\n",
" df_forecast = pd.DataFrame({\n",
" \"valid_time\": valid_times,\n",
" \"forecast\": forecast_values\n",
" })\n",
" \n",
" # Insert forecast - batch_start_time defaults to current time\n",
" # known_time is the start of this forecast's valid_time range\n",
" if i == 0:\n",
" # First forecast: use explicit series_id\n",
" result = td.insert_batch(\n",
" df=df_forecast,\n",
" known_time=known_time,\n",
" series_ids={\"forecast\": forecast_series_id}\n",
" )\n",
" series_ids_map = result.series_ids\n",
" else:\n",
" # Subsequent forecasts: reuse the same series_id\n",
" result = td.insert_batch(\n",
" df=df_forecast,\n",
" known_time=known_time,\n",
" series_ids=series_ids_map\n",
" )\n",
" series_ids_map = result.series_ids\n",
"\n",
" forecast_metadata.append({\n",
" \"forecast_num\": i + 1,\n",
" \"known_time\": known_time,\n",
" \"valid_time_start\": valid_times[0],\n",
" \"valid_time_end\": valid_times[-1],\n",
" \"batch_id\": result.batch_id,\n",
" \"series_id\": forecast_series_id\n",
" })\n",
" \n",
" print(f\"✓ Forecast {i+1}/{num_forecasts} created\")\n",
" print(f\" known_time: {known_time}\")\n",
" print(f\" valid_time range: {valid_times[0]} to {valid_times[-1]}\")\n",
" print(f\" Batch ID: {result.batch_id}\")\n",
" print()\n",
"\n",
"print(f\"✓ All {num_forecasts} forecasts created successfully!\")\n",
"print(f\"Each forecast uses the same series_id: {forecast_series_id}\")\n",
"print(f\"Series metadata: name='forecast', unit='MW', labels={{'type': 'power_forecast', 'model': 'sinus_with_error'}}\")\n",
"\n",
"# Now insert the true sinus function as \"actual\" (flat time series without revisions)\n",
"print(\"\\n\" + \"=\" * 60)\n",
"print(\"Inserting actual values (true sinus function)...\")\n",
"print(\"=\" * 60)\n",
"\n",
"# Calculate the overall valid_time range covering all forecasts\n",
"# We'll insert actual values for the entire range\n",
"start_valid = base_valid_time\n",
"end_valid = base_valid_time + timedelta(days=num_forecasts - 1) + timedelta(hours=forecast_horizon_hours)\n",
"\n",
"# Generate all valid_times for the actual series\n",
"actual_valid_times = []\n",
"current_time = start_valid\n",
"while current_time < end_valid:\n",
" actual_valid_times.append(current_time)\n",
" current_time += timedelta(hours=1)\n",
"\n",
"# Generate actual values using the true sinus function (no error)\n",
"base_power = 100.0\n",
"amplitude = 30.0\n",
"period_hours = 24\n",
"\n",
"hours_from_start = np.arange(len(actual_valid_times))\n",
"actual_values = base_power + amplitude * np.sin(2 * np.pi * hours_from_start / period_hours)\n",
"actual_values = np.round(actual_values, 2).tolist()\n",
"\n",
"# Create DataFrame for actual values (column name 'actual' matches series name)\n",
"df_actual = pd.DataFrame({\n",
" \"valid_time\": actual_valid_times,\n",
" \"actual\": actual_values\n",
"})\n",
"\n",
"# Insert actual values with known_time set to the last valid_time\n",
"# This represents when the actual values become known (after all data is collected)\n",
"actual_known_time = actual_valid_times[-1]\n",
"result_actual = td.insert_batch(\n",
" df=df_actual,\n",
" known_time=actual_known_time,\n",
" series_ids={\"actual\": actual_series_id}\n",
")\n",
"\n",
"print(f\"✓ Actual values inserted successfully!\")\n",
"print(f\" Series ID: {actual_series_id}\")\n",
"print(f\" Valid time range: {actual_valid_times[0]} to {actual_valid_times[-1]}\")\n",
"print(f\" Number of data points: {len(actual_valid_times)}\")\n",
"print(f\" Batch ID: {result_actual.batch_id}\")\n",
"print(f\"\\nForecast series: name='forecast', unit='MW', labels={{'type': 'power_forecast', 'model': 'sinus_with_error'}}\")\n",
"print(f\"Actual series: name='actual', unit='MW', labels={{'type': 'power_actual'}}\")\n",
"print(f\"Each forecast's valid_time starts at its known_time, shifted forward by 1 day for each subsequent forecast\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Part 3: Read All Forecast Revisions\n",
"\n",
"Now we'll read back all forecast revisions using `td.read_values_flat()`.\n",
"This returns the latest forecast for each valid_time (most recent based on known_time) in a single query.\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✓ Read 144 data points\n",
"DataFrame shape: (144, 2)\n",
"Index: valid_time\n",
"Columns: ['forecast', 'actual']\n",
"\n",
"This shows the latest forecast values for each valid_time.\n",
"\n",
"First few rows:\n"
]
},
{
"data": {
"application/vnd.microsoft.datawrangler.viewer.v0+json": {
"columns": [
{
"name": "valid_time",
"rawType": "datetime64[ns, UTC]",
"type": "unknown"
},
{
"name": "forecast",
"rawType": "float64",
"type": "float"
},
{
"name": "actual",
"rawType": "float64",
"type": "float"
}
],
"ref": "f611c66a-1be7-4c8e-bdf4-b8792b52614f",
"rows": [
[
"2025-01-01 00:00:00+00:00",
"100.45",
"100.0"
],
[
"2025-01-01 01:00:00+00:00",
"108.09",
"107.76"
],
[
"2025-01-01 02:00:00+00:00",
"115.91",
"115.0"
],
[
"2025-01-01 03:00:00+00:00",
"123.49",
"121.21"
],
[
"2025-01-01 04:00:00+00:00",
"128.05",
"125.98"
],
[
"2025-01-01 05:00:00+00:00",
"130.83",
"128.98"
],
[
"2025-01-01 06:00:00+00:00",
"133.28",
"130.0"
],
[
"2025-01-01 07:00:00+00:00",
"132.94",
"128.98"
],
[
"2025-01-01 08:00:00+00:00",
"129.53",
"125.98"
],
[
"2025-01-01 09:00:00+00:00",
"125.25",
"121.21"
]
],
"shape": {
"columns": 2,
"rows": 10
}
},
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" name \n",
" forecast \n",
" actual \n",
" \n",
" \n",
" valid_time \n",
" \n",
" \n",
" \n",
" \n",
" 2025-01-01 00:00:00+00:00 \n",
" 100.45 \n",
" 100.00 \n",
" \n",
" \n",
" 2025-01-01 01:00:00+00:00 \n",
" 108.09 \n",
" 107.76 \n",
" \n",
" \n",
" 2025-01-01 02:00:00+00:00 \n",
" 115.91 \n",
" 115.00 \n",
" \n",
" \n",
" 2025-01-01 03:00:00+00:00 \n",
" 123.49 \n",
" 121.21 \n",
" \n",
" \n",
" 2025-01-01 04:00:00+00:00 \n",
" 128.05 \n",
" 125.98 \n",
" \n",
" \n",
" 2025-01-01 05:00:00+00:00 \n",
" 130.83 \n",
" 128.98 \n",
" \n",
" \n",
" 2025-01-01 06:00:00+00:00 \n",
" 133.28 \n",
" 130.00 \n",
" \n",
" \n",
" 2025-01-01 07:00:00+00:00 \n",
" 132.94 \n",
" 128.98 \n",
" \n",
" \n",
" 2025-01-01 08:00:00+00:00 \n",
" 129.53 \n",
" 125.98 \n",
" \n",
" \n",
" 2025-01-01 09:00:00+00:00 \n",
" 125.25 \n",
" 121.21 \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✓ Latest forecast and actual values plotted successfully!\n"
]
}
],
"source": [
"# Plot the latest forecast and actual values (flat mode)\n",
"plt.figure(figsize=(14, 7))\n",
"\n",
"# Plot forecast and actual\n",
"plt.plot(df_flat.index, df_flat['forecast'], marker='o', markersize=2, linewidth=1.5, \n",
" label='Latest Forecast', alpha=0.7, color='blue')\n",
"plt.plot(df_flat.index, df_flat['actual'], marker='s', markersize=2, linewidth=2, \n",
" label='Actual (True Sinus)', alpha=0.9, color='red')\n",
"\n",
"plt.xlabel('Valid Time', fontsize=12)\n",
"plt.ylabel('Power [MW]', fontsize=12)\n",
"plt.title('Latest Forecast vs Actual Values (Flat Mode)', fontsize=14, fontweight='bold')\n",
"plt.grid(True, alpha=0.3)\n",
"plt.legend()\n",
"plt.xticks(rotation=45)\n",
"plt.tight_layout()\n",
"plt.show()\n",
"\n",
"print(\"✓ Latest forecast and actual values plotted successfully!\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Part 4: Read All Forecast Revisions (Overlapping Mode)\n",
"\n",
"Now let's read all forecast revisions using `td.read_values_overlapping()`.\n",
"This returns all forecasts with both `known_time` and `valid_time`, showing how\n",
"forecasts evolve over time. This is useful for analyzing forecast revisions and backtesting.\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Reading all forecast revisions (overlapping mode)...\n",
"============================================================\n",
"✓ Read 432 data points\n",
"\n",
"DataFrame shape: (432, 2)\n",
"\n",
"Index levels: ['known_time', 'valid_time']\n",
"\n",
"Columns: ['forecast', 'actual']\n",
"\n",
"This shows all forecast revisions, including all known_time values.\n",
"\n",
"First few rows:\n"
]
},
{
"data": {
"application/vnd.microsoft.datawrangler.viewer.v0+json": {
"columns": [
{
"name": "('known_time', 'valid_time')",
"rawType": "object",
"type": "unknown"
},
{
"name": "forecast",
"rawType": "float64",
"type": "float"
},
{
"name": "actual",
"rawType": "float64",
"type": "float"
}
],
"ref": "059acbb4-8760-4fdb-b577-b4e547d953cc",
"rows": [
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 00:00:00+0000', tz='UTC'))",
"100.45",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 01:00:00+0000', tz='UTC'))",
"108.09",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 02:00:00+0000', tz='UTC'))",
"115.91",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 03:00:00+0000', tz='UTC'))",
"123.49",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 04:00:00+0000', tz='UTC'))",
"128.05",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 05:00:00+0000', tz='UTC'))",
"130.83",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 06:00:00+0000', tz='UTC'))",
"133.28",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 07:00:00+0000', tz='UTC'))",
"132.94",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 08:00:00+0000', tz='UTC'))",
"129.53",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 09:00:00+0000', tz='UTC'))",
"125.25",
null
]
],
"shape": {
"columns": 2,
"rows": 10
}
},
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" name \n",
" forecast \n",
" actual \n",
" \n",
" \n",
" known_time \n",
" valid_time \n",
" \n",
" \n",
" \n",
" \n",
" 2025-01-01 00:00:00+00:00 \n",
" 2025-01-01 00:00:00+00:00 \n",
" 100.45 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 01:00:00+00:00 \n",
" 108.09 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 02:00:00+00:00 \n",
" 115.91 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 03:00:00+00:00 \n",
" 123.49 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 04:00:00+00:00 \n",
" 128.05 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 05:00:00+00:00 \n",
" 130.83 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 06:00:00+00:00 \n",
" 133.28 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 07:00:00+00:00 \n",
" 132.94 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 08:00:00+00:00 \n",
" 129.53 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 09:00:00+00:00 \n",
" 125.25 \n",
" NaN \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✓ All forecast revisions and actual values plotted successfully!\n"
]
}
],
"source": [
"# Plot all overlapping forecasts and actual values\n",
"plt.figure(figsize=(16, 9))\n",
"\n",
"# Separate forecast and actual data\n",
"df_forecast_overlapping = df_overlapping[['forecast']].dropna()\n",
"df_actual_overlapping = df_overlapping[['actual']].dropna()\n",
"\n",
"# Get unique known_times from the forecast series only (exclude actual's known_time)\n",
"# Each known_time in the forecast series represents a different forecast revision\n",
"unique_known_times = df_forecast_overlapping.index.get_level_values(\"known_time\").unique()\n",
"unique_known_times = sorted(unique_known_times)\n",
"\n",
"print(f\"Plotting {len(unique_known_times)} forecast revisions...\")\n",
"\n",
"# Plot each forecast revision with dashed blue lines\n",
"for idx, known_time in enumerate(unique_known_times):\n",
" # Get all values for this known_time\n",
" forecast_data = df_forecast_overlapping.xs(known_time, level=\"known_time\")\n",
" \n",
" # Extract valid_time and values\n",
" valid_times_plot = forecast_data.index\n",
" values = forecast_data['forecast'].values\n",
" \n",
" # Plot this forecast revision with dashed blue line\n",
" label = f\"Forecast {idx+1} (known_time: {known_time.strftime('%Y-%m-%d %H:%M')})\"\n",
" plt.plot(valid_times_plot, values, linewidth=1.5, linestyle='--', \n",
" label=label, color='blue', alpha=0.5)\n",
"\n",
"# Plot the latest forecast from flat data (read_values_flat) with solid blue line\n",
"plt.plot(df_flat.index, df_flat['forecast'], linewidth=2, linestyle='-', \n",
" label='Latest Forecast (flat)', color='blue', alpha=0.9, zorder=5)\n",
"\n",
"# Plot actual values\n",
"# Actual values will have the same known_time (when they were inserted)\n",
"# Get unique valid_times and actual values (they should be the same across known_times)\n",
"actual_data = df_actual_overlapping.reset_index()\n",
"# Group by valid_time and take the first value (they should all be the same)\n",
"actual_data = actual_data.groupby('valid_time')['actual'].first().sort_index()\n",
"\n",
"plt.plot(actual_data.index, actual_data.values, \n",
" linewidth=2.5, label='Actual (True Sinus)', alpha=0.9, color='red', zorder=10)\n",
"\n",
"# Formatting\n",
"plt.xlabel('Valid Time (forecasted period)', fontsize=12)\n",
"plt.ylabel('Power [MW]', fontsize=12)\n",
"plt.title('All Overlapping Forecast Revisions vs Actual Values', fontsize=14, fontweight='bold')\n",
"plt.grid(True, alpha=0.3)\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc='upper left', fontsize=9)\n",
"plt.xticks(rotation=45)\n",
"plt.tight_layout()\n",
"plt.show()\n",
"\n",
"print(\"✓ All forecast revisions and actual values plotted successfully!\")\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✓ DataFrame with (known_time, valid_time) index and 'forecast' column\n",
"\n",
"DataFrame shape: (432, 2)\n",
"\n",
"Index levels: ['known_time', 'valid_time']\n",
"\n",
"Column: ['forecast', 'actual']\n",
"\n",
"First few rows:\n"
]
},
{
"data": {
"application/vnd.microsoft.datawrangler.viewer.v0+json": {
"columns": [
{
"name": "('known_time', 'valid_time')",
"rawType": "object",
"type": "unknown"
},
{
"name": "forecast",
"rawType": "float64",
"type": "float"
},
{
"name": "actual",
"rawType": "float64",
"type": "float"
}
],
"ref": "9e9d4bcb-0c64-4989-a7fc-e3c7ff6457e4",
"rows": [
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 00:00:00+0000', tz='UTC'))",
"100.45",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 01:00:00+0000', tz='UTC'))",
"108.09",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 02:00:00+0000', tz='UTC'))",
"115.91",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 03:00:00+0000', tz='UTC'))",
"123.49",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 04:00:00+0000', tz='UTC'))",
"128.05",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 05:00:00+0000', tz='UTC'))",
"130.83",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 06:00:00+0000', tz='UTC'))",
"133.28",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 07:00:00+0000', tz='UTC'))",
"132.94",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 08:00:00+0000', tz='UTC'))",
"129.53",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 09:00:00+0000', tz='UTC'))",
"125.25",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 10:00:00+0000', tz='UTC'))",
"118.62",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 11:00:00+0000', tz='UTC'))",
"110.96",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 12:00:00+0000', tz='UTC'))",
"103.41",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 13:00:00+0000', tz='UTC'))",
"93.93",
null
],
[
"(Timestamp('2025-01-01 00:00:00+0000', tz='UTC'), Timestamp('2025-01-01 14:00:00+0000', tz='UTC'))",
"85.14",
null
]
],
"shape": {
"columns": 2,
"rows": 15
}
},
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" name \n",
" forecast \n",
" actual \n",
" \n",
" \n",
" known_time \n",
" valid_time \n",
" \n",
" \n",
" \n",
" \n",
" 2025-01-01 00:00:00+00:00 \n",
" 2025-01-01 00:00:00+00:00 \n",
" 100.45 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 01:00:00+00:00 \n",
" 108.09 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 02:00:00+00:00 \n",
" 115.91 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 03:00:00+00:00 \n",
" 123.49 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 04:00:00+00:00 \n",
" 128.05 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 05:00:00+00:00 \n",
" 130.83 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 06:00:00+00:00 \n",
" 133.28 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 07:00:00+00:00 \n",
" 132.94 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 08:00:00+00:00 \n",
" 129.53 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 09:00:00+00:00 \n",
" 125.25 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 10:00:00+00:00 \n",
" 118.62 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 11:00:00+00:00 \n",
" 110.96 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 12:00:00+00:00 \n",
" 103.41 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 13:00:00+00:00 \n",
" 93.93 \n",
" NaN \n",
" \n",
" \n",
" 2025-01-01 14:00:00+00:00 \n",
" 85.14 \n",
" NaN \n",
" \n",
" \n",
"
\n",
"