---
name: econ-scratchpad
description: General data analysis playground for exploring curiosities of all kinds — creating notebook-style pages from data that lives anywhere on the workspace or is pulled from the internet. Covers the pre-rendered JSON pattern, NotebookTemplate component, and the overall workflow for organizing data and building visualization pages.
compatibility: Created for Zo Computer
metadata:
author: rob.zo.computer
---
# Econ Scratchpad
This skill documents the **web/scratch site** (`file 'web/scratch'`) — a general data analysis playground for exploring curiosities of any kind through notebook-style pages combining prose, data visualizations, and analysis.
**When to use this skill**: When you're asked to explore or add to the web/scratch site, create new analysis notebooks, or work with data analysis where data comes from anywhere on the workspace or the internet.
---
## Quick Reference
The site lives at `file 'web/scratch'` and consists of:
| Type | Location | Purpose |
|------|----------|---------|
| Pages | `src/pages/` | React components for each notebook |
| Data | `data/` | JSON files (pre-rendered), raw CSVs, and source data |
| Server | `server.ts` | Hono backend with API routes |
| Home | `src/pages/Home.tsx` | Links to all notebooks |
**Access**: The site runs at `file ''web/scratch''` and is viewable in the Zo app iframe.
---
## Site Philosophy
Unlike `Skills/personal-data` (structured personal datasets with standardized schemas), this site is an **ad-hoc exploration space** for analyzing *any data you're curious about*. Data can come from anywhere on your workspace or be pulled from the internet, organized, then analyzed.
Typical explorations include:
- Understanding macroeconomic trends (Fed Funds Rate, inflation)
- Exploring demographic shifts (fertility rates, age-based sentiment)
- Testing hypotheses (e.g., the "vibecession" thesis)
- Visualizing complex interactions (AI compute vs energy grid)
- Analyzing public datasets (government, research, industry)
- Any random curiosity or question worth investigating
Notebooks here are **less structured** than rc-data. Each one explores a specific question or curiosity without a standardized schema or cross-dataset joining patterns.
**Key difference from personal-data**: This is NOT for personal data (health, activity, communication, financial). For those topics, use the personal-data skill. Here, data sources are diverse (public APIs, downloaded CSVs, scraped web data, spreadsheet exports, etc.) — bring in what you need and explore it.
---
## Current Notebooks
| Path | Description | Data Source |
|------|-------------|-------------|
| `/econ` | Economic Events Dashboard | FRED data |
| `/compounding` | "Should the Kids YOLO?" | YOLO spreadsheet model |
| `/snap-wic` | SNAP & WIC Programs | Federal data |
| `/fertility-rates` | US Fertility Trends | CDC / public data |
| `/fed-funds` | Historical Fed Funds Rate | FRED |
| `/consumer-economics` | Consumer Economics Metrics | BLS, surveys |
| `/vibecession` | The Vibecession Thesis | Multiple sources |
| `/ai-performance` | AI Compute: Power vs Output | Industry estimates |
---
## How Notebooks Work
### Data Pattern: Pre-rendered JSON
**Best practice**: Pre-render data to JSON files and import them at build time.
```
web/scratch/
├── src/
│ └── pages/
│ └── MyNotebook.tsx ← Imports JSON
└── data/
└── my_notebook.json ← Generated by script
```
**Why pre-render?**
- **Fast**: Pages load instantly, no database/API calls
- **Static**: Site can be published without backend
- **Versioned**: JSON snapshots in git show data evolution
- **Decoupled**: Data extraction separate from visualization
**Current state**: Not all existing notebooks follow this pattern. Some fetch data via API (`/api/*` routes in `server.ts`). **Going forward, prefer pre-rendered JSON.**
### Creating a Notebook
#### 1. Acquire Data
Data can come from anywhere on your workspace or be pulled from the internet:
- **Download from public sources**: Government APIs (FRED, BLS, EIA, Census), research databases, Kaggle, GitHub
- **Existing workspace files**: Data already in `data/`, raw files in other projects, spreadsheets anywhere in workspace
- **Scrape from web**: Save web pages as HTML or CSV for extraction
- **Manual export**: Download data from tools/platforms as CSV, Excel, JSON
Place raw data in `data/raw/` or organize it anywhere convenient. Work with what you have.
#### 2. Write a Data Processing Script
Create a Python script to transform raw data into clean JSON:
```python
#!/usr/bin/env python3
import json
import csv
from pathlib import Path
# Input
INPUT_CSV = Path(__file__).parent.parent / "data" / "raw" / "source.csv"
OUTPUT_JSON = Path(__file__).parent.parent / "data" / "my_notebook.json"
# Read and transform
data = []
with open(INPUT_CSV) as f:
reader = csv.DictReader(f)
for row in reader:
data.append({
"date": row["date"],
"value": float(row["value"]),
})
OUTPUT_JSON.write_text(json.dumps(data, indent=2))
print(f"Wrote {OUTPUT_JSON}")
```
Run: `python3 /home/workspace/web/scratch/scripts/my_notebook.py`
#### 3. Create the Page Component
Create `src/pages/MyNotebook.tsx`:
```tsx
import React from "react";
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
import {
type ChartConfig,
ChartContainer,
ChartTooltip,
ChartTooltipContent,
} from "@/components/ui/chart";
import { LineChart, Line, XAxis, YAxis, CartesianGrid } from "recharts";
import data from "@/data/my_notebook.json";
const chartConfig = {
value: { label: "Value", color: "var(--chart-1)" },
} satisfies ChartConfig;
export default function MyNotebook() {
return (
{/* Back link */}
← Back to home
{/* Title and context */}
My Notebook Title
Subtitle explaining the notebook's purpose
{/* Prose explanation */}
Explain what we're looking at, why it matters, what patterns emerge...
{/* Charts */}
Chart Title} />
);
}
```
#### 4. Add the Route
In `src/App.tsx`:
```tsx
import MyNotebook from "./pages/MyNotebook";
// In Routes:
} />
```
#### 5. Update Home
Add to `src/pages/Home.tsx`:
```tsx
My Notebook Title
```
---
## Notebook Template (Recommended)
A reusable `NotebookTemplate` component exists at `src/components/NotebookTemplate.tsx`. Going forward, all new notebooks should use it for consistent UX.
```tsx
import React from "react";
import NotebookTemplate from "@/components/NotebookTemplate";
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
import data from "@/data/my_notebook.json";
export default function MyNotebook() {
return (
{/* Explanation */}
Explain what we're looking at, why it matters, what patterns emerge...
{/* Data visualizations */}
Section Title
{/* Chart component */}
{/* Key observations */}
Key Takeaways
Observation one
Observation two
);
}
```
**Why the template?**
- Consistent navigation (back button)
- Uniform spacing and typography
- Clear sections: context → data → takeaways → sources
- Easy for readers to scan and understand
The component handles the overall layout, back button, header, and footer automatically. Just pass your content as children.
---
## Chart Guidelines
Charts use **ShadCN/UI components** (built on Recharts). Key rules:
1. **Always use ChartContainer + ChartConfig**
2. **Color variables**: `var(--chart-1)` through `var(--chart-5)` — never wrap in `hsl()`
3. **Use theme-aware colors**: `var(--color-value)` references the ChartConfig color
4. **Tooltips**: Use `ChartTooltipContent` for consistent styling
Example color reference:
```tsx
const chartConfig = {
value1: { label: "Series 1", color: "var(--chart-1)" },
value2: { label: "Series 2", color: "var(--chart-2)" },
value3: { label: "Series 3", color: "var(--chart-3)" },
} satisfies ChartConfig;
// In chart:
```
---
## API Routes (Legacy Pattern)
Some existing pages fetch data via API routes in `server.ts` instead of pre-rendered JSON. Example:
```ts
// server.ts
app.get("/api/fed-funds-rate", (c) => {
const data = Bun.file("data/fed_funds_rate.json");
return c.json(JSON.parse(await data.text()));
});
```
**Prefer pre-rendering new data**. API routes add runtime complexity and require the server. Only use API routes when:
- Data changes frequently and needs live updates
- You need server-side transformation on every request
---
## Data Directory Structure
```
data/
├── fed_funds_rate.json # Pre-rendered JSON (FED)
├── disposable_income_sentiment.json # BLS data
├── expenditure_shares.json # Consumer spending
├── ai-performance.json # AI compute metrics
├── vibecession/ # Vibecession notebook data
│ ├── sentiment_recessions.json
│ ├── baumol_prices.json
│ ├── housing_affordability.json
│ └── ...
└── raw/ # Source data
├── bls/
├── eia/
├── fred/
└── surveys/
```
**Convention**: When a notebook has multiple related JSON files, create a subdirectory (e.g., `data/vibecession/`).
---
## Adding Data Sources
### FRED (Federal Reserve Economic Data)
1. Download series as CSV from https://fred.stlouisfed.org/
2. Convert to JSON with a Python script
3. Import in the notebook component
```python
import pandas as pd
df = pd.read_csv("CPIAUCSL.csv")
data = [{"date": r["DATE"], "value": r["CPIAUCSL"]} for _, r in df.iterrows()]
```
### CSV from Any Source
```python
import csv
with open("source.csv") as f:
reader = csv.DictReader(f)
data = [dict(row) for row in reader]
```
### Excel Files
```python
import pandas as pd
df = pd.read_excel("source.xlsx", sheet_name="Sheet1")
```
---
## Updating Existing Notebooks
To add data to an existing notebook:
1. **Add the JSON** to `data/` or `data/{notebook-name}/`
2. **Import** in the page component:
```tsx
import newData from "@/data/new_data.json";
```
3. **Add type definitions** for TypeScript
4. **Create a new chart** or section using the data
---
## Updating Documentation
When you add significant components or workflows to the site, update the main README.md:
1. Add new entries to the **Project Notes** section for notable features
2. Document any architectural decisions or patterns
3. Record important takeaways from debugging sessions
The README.md should serve as an accurate, comprehensive piece of documentation for the project.
---
## Site Architecture
### Tech Stack
- **Runtime**: Bun (not Node.js)
- **Backend**: Hono web framework
- **Frontend**: React + Vite
- **Routing**: React Router DOM
- **Styling**: Tailwind CSS 4
- **Charts**: Recharts + ShadCN/UI components
- **Build**: Vite bundling
### File Structure
```
web/scratch/
├── server.ts # Hono server + Vite middleware
├── src/
│ ├── App.tsx # React Router setup
│ ├── main.tsx # React entry point
│ ├── components/ # Shared components + shadcn/ui
│ ├── pages/ # Notebook pages
│ └── data/ # JSON files (symlinked to ../../data)
├── data/ # Data files (JSON, CSV, raw/)
├── backend-lib/ # Helpers for Zo API, etc.
└── zosite.json # Zo deployment config
```
### Development Notes
- **Never restart the server manually** — Zo manages it
- **Hot reload**: Edit files and they auto-reload
- **Port**: Site runs on local port (see `zosite.json`)
- **Access**: View in Zo app iframe at `file ''web/scratch''`
---
## Common Patterns
### Time Series Charts
```tsx
import { LineChart, Line, XAxis, YAxis, CartesianGrid } from "recharts";
v.split("-")[0]} />
`${v}%`} />
```
### Comparison Charts
```tsx
import { BarChart, Bar, XAxis, YAxis, CartesianGrid } from "recharts";
const chartConfig = {
value1: { label: "Series A", color: "var(--chart-1)" },
value2: { label: "Series B", color: "var(--chart-2)" },
} satisfies ChartConfig;
```
---
## Keeping This Skill Updated
**This skill should evolve as the site grows.** If you:
- Add a new notebook → document its data source and key insights
- Discover a useful pattern → add to Common Patterns
- Update the data workflow → revise the Creating a Notebook section
- Encounter a gotcha → note it in a relevant section
The goal: someone coming in fresh can understand how to work with this site, add data, and create new notebooks.
---
## Related Skills
- `Skills/personal-data` — Structured personal data analysis (different pattern)
- `Skills/markdown-to-public-site` — Publishing notebooks publicly
The econ scratchpad site can be published using the publish_site tool when ready to share openly.