# Visualizing Latency Comparisons Between LLM APIs: OpenRouter vs Bedrock

Large Language Models (LLMs) are now integral to modern software applications, powering tasks such as summarization, code generation, and technical explanations. When evaluating multiple LLM APIs, **latency**, **response quality**, and **consistency** are critical. Today, I share a detailed analysis of latency comparison between **OpenRouter** and **Bedrock**, along with methodology, visualization, and insights.

## **Experiment Overview**

The primary objective of this experiment was to measure and compare **response latency** for OpenRouter and Bedrock across multiple prompts. The experiment was designed to capture not only the speed of each API but also its consistency across repeated queries.

### **Prompts Used**

Three representative prompts were chosen for the comparison:

1. Explain Kubernetes in simple terms for a beginner.
    
2. Write a Python function to reverse a linked list.
    
3. Summarize the book *Atomic Habits* in three sentences.
    

Each prompt was sent **five times** to each API to generate multiple latency measurements for statistical analysis.

## **Data Collection Methodology**

The latency comparison was conducted using Python, with the following approach:

1. **OpenRouter API Calls:**
    
    * Sent HTTP POST requests to the OpenRouter API with the prompt, specifying the `gpt-oss-20b` model.
        
    * Measured **start and end timestamps** to calculate latency.
        
    * Extracted the text response from the API JSON payload.
        
2. **AWS Bedrock API Calls:**
    
    * Used the `boto3` client to invoke the Bedrock model `openai.gpt-oss-20b-1`.
        
    * Sent the prompt in the OpenAI-style chat format.
        
    * Measured latency from request initiation to response.
        
    * Extracted the returned text from the API payload.
        
3. **Data Storage:**
    
    * Each query stored the following fields: prompt, repeat number, OpenRouter response, OpenRouter latency, Bedrock response, and Bedrock latency.
        
    * All results were saved into a CSV file (`llm_comparison.csv`) for analysis and visualization.
        

This setup ensured a **repeatable and reliable** dataset for performance analysis and comparison.

Here is a **condensed snippet** showing the main idea of the comparison script:

```python
for prompt in prompts:
    for i in range(REPEATS):
        or_text, or_time = call_openrouter(prompt)
        print(f"OpenRouter [{i+1}/{REPEATS}] Latency: {or_time:.2f}s")
        br_text, br_time = call_bedrock(prompt)
        print(f"Bedrock   [{i+1}/{REPEATS}] Latency: {br_time:.2f}s")
        data_rows.append({
            "prompt": prompt,
            "repeat": i+1,
            "openrouter_response": or_text,
            "openrouter_latency": or_time,
            "bedrock_response": br_text,
            "bedrock_latency": br_time
        })
```

This allowed me to build a **structured dataset** with both responses and latencies for each prompt and repeat.

## **Latency Analysis**

Using the CSV data, we conducted both statistical and visual analysis to compare the APIs.

### **OpenRouter Latency**

* Minimum Latency: 2.32 seconds
    
* Maximum Latency: 7.28 seconds
    
* Average Latency: Approximately 4.60 seconds
    
* Observation: OpenRouter exhibited **higher variability**, particularly for repeated technical explanation prompts.
    

### **Bedrock Latency**

* Minimum Latency: 2.00 seconds
    
* Maximum Latency: 3.24 seconds
    
* Average Latency: Approximately 3.05 seconds
    
* Observation: Bedrock was consistently faster and **more stable** across repeats and prompt types.
    

### **Prompt-Specific Patterns**

* **Kubernetes Explanation:** Bedrock consistently responded under 3 seconds, while OpenRouter spiked to over 7 seconds in one repeat.
    
* **Python Code Reversal:** Both APIs performed similarly in early repeats, but Bedrock remained slightly faster.
    
* **Book Summarization:** Bedrock maintained both speed and stability, whereas OpenRouter showed variability in later repeats.
    

## **Visualization Approach**

To better understand latency differences, the following visualizations were created:

1. **Boxplot:** Shows overall latency distribution for each API, highlighting median, quartiles, and outliers.
    
2. **Lineplot Per Prompt:** Displays latency across repeats for each prompt, revealing **consistency and spikes**.
    

These visualizations make trends immediately clear, allowing developers to make informed choices between APIs.

## **Python Script for Plotting**

```python
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

df = pd.read_csv("llm_latency_comparison.csv")

print(df[['openrouter_latency', 'bedrock_latency']].describe())

plt.figure(figsize=(12,6))
sns.boxplot(data=df[['openrouter_latency', 'bedrock_latency']])
plt.title("Latency Comparison: OpenRouter vs Bedrock")
plt.ylabel("Latency (seconds)")
plt.show()

plt.figure(figsize=(14,6))
for prompt in df['prompt'].unique():
    prompt_data = df[df['prompt'] == prompt]
    sns.lineplot(x='repeat', y='openrouter_latency', data=prompt_data, label=f'OpenRouter: {prompt}', marker='o')
    sns.lineplot(x='repeat', y='bedrock_latency', data=prompt_data, label=f'Bedrock: {prompt}', marker='o')
plt.title("Latency Trends Per Prompt Repeat")
plt.xlabel("Repeat Number")
plt.ylabel("Latency (seconds)")
plt.legend(bbox_to_anchor=(1.05, 1), loc='upper left')
plt.show()
```

We obtained the following from the visualtion:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1764164561012/6904465f-f831-400b-9041-a0f87fea18c3.png align="center")

## **Insights From the Data**

1. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1764164571884/f1159935-a5a8-4318-b10f-981e0c0bddda.png align="center")
    
2. **OpenRouter Latency Observations:**
    
    * Minimum Latency: 2.32 seconds
        
    * Maximum Latency: 7.28 seconds
        
    * Average Latency: Approximately 4.60 seconds
        
    * Variability: Significant across different prompts and repeats, indicating inconsistent performance under certain queries.
        
3. **Bedrock Latency Observations:**
    
    * Minimum Latency: 2.00 seconds
        
    * Maximum Latency: 3.24 seconds
        
    * Average Latency: Approximately 3.05 seconds
        
    * Variability: Much lower than OpenRouter, indicating more consistent performance.
        
4. **Prompt-Specific Trends:**
    
    * For **Kubernetes explanation prompts**, OpenRouter latency increased up to 7.28 seconds in the fourth repeat, while Bedrock remained under 3 seconds.
        
    * For **code generation prompts**, both APIs performed similarly in early repeats, but Bedrock consistently had faster responses.
        
    * For **book summarization**, Bedrock was faster and more stable, with lower standard deviation.
        

## **Takeaways**

1. **Consistency Matters:** Bedrock is more predictable, making it preferable for real-time applications.
    
2. **Measure Repeats:** Single API calls can be misleading; repeated measurements reveal stability.
    
3. **Latency vs. Prompt Complexity:** Certain prompts can trigger spikes in OpenRouter latency, which developers should consider for production workloads.
    
4. **Data-Driven Decision Making:** Structured data collection enables informed API selection.
    

## **Conclusion**

This experiment shows that **Bedrock provides lower and more consistent latency** across prompts and repeated queries compared to OpenRouter. Collecting and visualizing latency not only reveals performance differences but also helps developers make informed choices about which API to integrate for production systems.

By sharing both the **data collection and visualization workflow**, I hope to provide a **practical template** for evaluating LLM APIs for real-world projects.