LangChain / Production

Save & Load Prompt & LLM Config in LangChain

Serialize prompts and LLM configurations to JSON for clean separation between development and production. No hardcoded configs. No API keys in code.

📅 May 2026 ⏱️ 6 min read 👤 Trenzy Vibes
Python LangChain OpenAI JSON Production DevOps

The Problem

Hardcoding prompts and LLM configs in your Python files is fine for prototyping. But in production, you need:

💡 The Solution

Serialize prompts with LangChain's dumps()/loads() and LLM configs with Pydantic's model_dump(). Store as JSON files. Load at runtime. Clean separation between config and code.

Architecture

Developer / CI Pipeline
📝 Create Prompt
→
⚙️ Configure LLM
→
💾 dumps() + model_dump()
📄 prompt_config.json
→
📄 llm_config.json
Production app loads these files — no code changes needed
Production App
📂 loads()
→
🔧 ChatOpenAI(**config)
→
⚡ Running Chain

Step 1: Saving (Developer / CI)

Import dependencies and load environment variables:

python 
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.load import dumps, loads
from langchain_core.output_parsers import StrOutputParser
from dotenv import load_dotenv
import json

load_dotenv()

1a. Create and Save the Prompt

Use LangChain's dumps() with pretty=True for readable JSON:

python 
# Create your prompt
prompt = ChatPromptTemplate.from_messages([
    ("human", "Explain the concept of {topic} in simple terms.")
])

# Use LangChain's dedicated dumper (pretty=True adds indentation)
dumped_prompt_config = dumps(prompt, pretty=True)

# Write to file
with open("prompt_config.json", "w") as f:
    f.write(dumped_prompt_config)

1b. Create and Save the LLM Config

Use Pydantic's model_dump() — exclude the API key for security:

python 
# Create LLM with production settings
llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0.2,  # lower = more deterministic, higher = more creative
    max_tokens=200,
    verbose=True  # debug complex outputs
)

# Use Pydantic's model_dump, explicitly EXCLUDING the API key
llm_config_dict = llm.model_dump(exclude={"openai_api_key", "openai_api_base"})

# Save using standard json.dump
with open("llm_config.json", "w") as f:
    json.dump(llm_config_dict, f, indent=2)

⚠️ Security Note

Always exclude openai_api_key and openai_api_base from the dumped config. The production app will fetch the API key from environment variables (.env file), keeping secrets out of version control.

Step 2: Loading (Production App)

Your production application reads the JSON files and reconstructs everything:

2a. Load the Prompt

python 
# Load Prompt
with open("prompt_config.json", "r") as f:
    prompt_json_string = f.read()

loaded_prompt_config = loads(prompt_json_string)

2b. Load the LLM Config

python 
# Load LLM
with open("llm_config.json", "r") as f:
    llm_config_dict = json.load(f)

# Reconstruct LLM by passing the dict into the constructor.
# Because we excluded the API key, ChatOpenAI will automatically 
# fetch it from your .env file, creating a perfectly healthy sync client!
loaded_llm_config = ChatOpenAI(**llm_config_dict)

2c. Rebuild and Execute the Chain

python 
# Instantiate Output Parser
parser = StrOutputParser()

# Rebuild the chain
loaded_chain = loaded_prompt_config | loaded_llm_config | parser

# Execute the chain
response = loaded_chain.invoke({
    "topic": "How can I build models for doctors?"
})

print(response)

✅ Why This Pattern Rocks

• Config-driven: Change prompts without redeploying code
• Secure: API keys never touch version control
• Portable: Same config works across dev/staging/prod
• Team-friendly: Non-devs can edit JSON configs
• Reproducible: Exact same chain every time you load

Generated JSON Files

Here's what your saved files look like:

prompt_config.json 
{
  "lc": 1,
  "type": "constructor",
  "id": ["langchain", "prompts", "chat", "ChatPromptTemplate"],
  "kwargs": {
    "messages": [
      {
        "lc": 1,
        "type": "constructor",
        "id": ["langchain", "prompts", "chat", "HumanMessagePromptTemplate"],
        "kwargs": {
          "prompt": {
            "lc": 1,
            "type": "constructor",
            "id": ["langchain", "prompts", "prompt", "PromptTemplate"],
            "kwargs": {
              "template": "Explain the concept of {topic} in simple terms.",
              "input_variables": ["topic"]
            }
          }
        }
      }
    ]
  }
}
llm_config.json 
{
  "model": "gpt-4o-mini",
  "temperature": 0.2,
  "max_tokens": 200,
  "verbose": true,
  "model_name": "gpt-4o-mini",
  "n": 1,
  "streaming": false,
  "max_retries": 2,
  "timeout": null,
  "max_retries": 2,
  "tags": [],
  "metadata": {}
}

Dependencies

bash 
pip install langchain langchain-openai python-dotenv

Next Steps

Take this pattern further:

  1. Store configs in Git for version-controlled prompt evolution
  2. Use environment-specific folders: configs/dev/, configs/prod/
  3. Add validation with Pydantic models before loading
  4. Build a config UI for non-technical team members
  5. Combine with the RAG Pipeline for full document Q&A systems

🚀 Need Help Deploying to Production?

I help teams productionize LangChain apps with proper config management, CI/CD, and monitoring.

Let's Talk