Claude Code for data science: A first look

Quickly standardize a messy data science project
Author

Sjoerd de Haan

Published

April 21, 2025

First Look at Claude Code: When AI Meets Command Line Development

Today I took Claude Code for its first spin on a real data science project, and I’m impressed. As someone who lives in the terminal and swears by Neovim, I was curious to try Claude Code, as it is a terminal tool!

The Project: Urban Sounds Classification

My test case was an audio machine learning project called “urban-sounds” – a typical data science mess with notebooks scattered across the repo, hardcoded paths everywhere, and model configurations duplicated in multiple places. You know the drill. I did not touch this project for months and I knew it was in a bad state.

# In notebook 1
datadir = Path("../samples")
model = "laion/larger_clap_general"

# In notebook 2  
dataset_path = Path("samples")
MODEL_LARGE = "laion/larger_clap_general"

# In notebook 3
data_folder = Path("../../samples")
# ... you get the idea

The goal was simple: standardize data loading, centralize configuration, and apply some basic software engineering hygiene to make the project maintainable.

Claude Code in Action

What struck me immediately was how Claude Code just gets it. I didn’t need to explain software engineering principles or best practices – it automatically suggested creating a central config.py file, consolidating data directories, and replacing hardcoded paths with constants.

Here’s what Claude Code proposed and executed:

1. Central Configuration

# config.py - One source of truth
PROJECT_ROOT = Path(__file__).parent

# Data directories
DATA_DIR = PROJECT_ROOT / "data"
SAMPLES_DIR = DATA_DIR / "samples"

# Model configurations
MODELS = {
    "clap_music_speech": "laion/larger_clap_music_and_speech",
    "clap_general": "laion/larger_clap_general",
    "ast": "MIT/ast-finetuned-audioset-10-10-0.4593"
}

# Device auto-detection
DEVICE = "cuda:0" if torch.cuda.is_available() else "cpu"

2. Systematic Refactoring

Claude Code methodically went through every notebook and QMD file, replacing: - Path("../samples") → config.SAMPLES_DIR - "laion/larger_clap_general" → config.MODELS["clap_general"] - Hardcoded label dictionaries → config.URBAN_SOUNDS_LABELS

It skipped however the qmd files, that contain Python blocks. I had to tell it to go over those files.

3. Data Organization

data/
├── samples/     # Audio files (moved from root)
├── archives/    # Compressed datasets  
├── processed/   # Transformed data
└── cache/       # HuggingFace cache

What Impressed Me

Command Line Native: Claude Code operates in the terminal where I’m already working. No context switching to a web interface or IDE extension. It reads files, runs bash commands, and edits notebooks directly.

Engineering Mindset: I told claude to apply software engineering principles and it did: - Single source of truth for configuration - Environment-aware device detection
- Verification scripts to test the migration - Use of system tools like rg to search for files that contain python code blocks

Jupyter Notebook Fluency: It understood the structure of Jupyter notebooks and could edit individual cells precisely, maintaining the interactive nature while improving the code quality.

Project Understanding: After analyzing the codebase, it created comprehensive documentation in CLAUDE.md that perfectly captured the project’s purpose and workflow.

4. Automated Testing & Verification

Here’s where Claude Code really shines – it doesn’t just make changes and hope for the best. It automatically created a test script to verify the migration:

# test_migration.py - Auto-generated verification
def test_config():
    """Test that all configuration constants are accessible"""
    assert config.SAMPLES_DIR.exists()
    assert len(config.MODELS) > 0
    assert config.DEVICE in ["cpu", "cuda:0", "cuda:1"]
    # ... more assertions

Then it ran the test… and it failed! 

python test_migration.py
# ModuleNotFoundError: No module named 'torch'

The AI had forgotten about my conda environment setup. I pointed out: “Before running a python script, use micromamba activate urban-sounds or micromamba run -n urban-sounds python.”

Claude Code immediately adapted:

micromamba run -n urban-sounds python test_migration.py
# ✨ Migration verification complete!

This interaction perfectly demonstrates how Claude Code learns from corrections and respects your specific development environment.

Why This Impressed Me

Command Line Native: Claude Code operates in the terminal where I’m already working. No context switching to a web interface or IDE extension. It reads files, runs bash commands, and edits notebooks directly.

Engineering Mindset: I told claude to apply software engineering principles and it did: - Single source of truth for configuration - Environment-aware device detection
- Verification scripts to test the migration

Jupyter Notebook Fluency: It understood the structure of Jupyter notebooks and could edit individual cells precisely, maintaining the interactive nature while improving the code quality.

Project Understanding: After analyzing the codebase, it created comprehensive documentation in CLAUDE.md that perfectly captured the project’s purpose and workflow.

Perfect Fit for Terminal Workflow

As a Neovim user, I live in the terminal. Claude Code integrates seamlessly:

# Start Claude Code in project directory
claude-code

# It analyzes, suggests, implements
> /init  # Creates project documentation
> Please consolidate data loading...  # Natural language requests

The tool respects my environment setup, understanding that I need to run Python scripts with micromamba run -n urban-sounds python script.py rather than assuming a global Python installation.

The Results (and Some Rough Edges)

In about 30 minutes, Claude Code transformed a typical “research code” project into something that follows software engineering best practices:

✅ Single source of truth for all paths and configuration
✅ No more relative path hell across notebooks
✅ Standardized model and dataset loading
✅ Environment-aware configuration
✅ Comprehensive documentation
✅ Automated testing and verification

The migration was quite good. Files were moved and code updated.

However, it wasn’t perfect. Claude Code had missed all qmd files in the migration. Several Quarto documents still contained hardcoded paths and weren’t using the new configuration constants. This highlights an important limitation: while Claude Code is excellent at systematic refactoring, it can still miss edge cases or less common file types.

The lesson? Always do a final manual check, especially in projects with diverse file formats. But even with these oversights, the bulk of the work was completed correctly, saving hours of manual refactoring.

Costs

Running The agentic workflows is not cheap. The whole exercise cost $2.40, which is a lot of money compared to running a single prompt.

Refactoring the qmd files costed another $1.18. It’s not cheap.

Bottom Line

Claude Code feels like having a senior software engineer pair programming with you in the terminal. It doesn’t just write code – it thinks about architecture, maintainability, and best practices. For data scientists who want to apply proper software engineering to their projects without leaving their terminal workflow, this is a game-changer.

The fact that it seamlessly integrates with command line development and understands the nuances of both data science workflows and software engineering principles makes it incredibly valuable for anyone working at the intersection of research and production code.

Would I use it again? Absolutely. This is exactly the kind of tool that helps bridge the gap between experimental notebooks and maintainable software – something the data science community desperately needs.

Looking to transform your team’s R&D workflow with powerful command line and AI tooling? I help organizations build efficient, maintainable data science pipelines that bridge the gap between research and production. Whether you’re struggling with notebook chaos, inconsistent environments, or scaling ML experiments, I can guide your team toward engineering practices that actually work in research settings.

Interested in learning more? Let’s talk about how modern AI-assisted development can supercharge your team’s productivity.