Making HTTP Better on ESP32

Question

HTTP on ESP32 Isn't Hard. The Unnecessary Work Around It Is.

Making an HTTP request from an ESP32 is relatively straightforward.

Connect to Wi-Fi, create an HTTP client, send a request, receive a response.

The real challenge usually appears later.

As an embedded application grows, HTTP communication starts accumulating responsibilities:

• Authentication tokens
• Custom headers
• Query parameters and dynamic paths
• JSON request bodies
• JSON response parsing
• Temporary buffers
• Memory allocations
• Repeated connections
• Error handling and status validation

None of these things are particularly difficult on their own.

The problem is the amount of unnecessary work surrounding a simple HTTP request.

On a server or desktop application, that overhead might not matter much. On an ESP32 running continuously alongside sensors, Wi-Fi, display drivers, Bluetooth, and other tasks, memory usage and repeated allocations can become much more important.

This article explores a question I started asking while building ESP32 applications:

How much work does an ESP32 actually need to do to make an HTTP request?

---

A Simple Request Can Turn Into a Long Workflow

Consider a common situation.

An API returns the following JSON:

{
  "temperature": 24.8,
  "humidity": 68,
  "device": {
    "id": "esp32-office",
    "location": "office"
  },
  "metadata": {
    "version": "1.4",
    "updatedAt": 1725500000
  }
}

But your ESP32 only needs this:

temperature
device.id

A traditional workflow might look like this:

HTTP Request
     ↓
Receive the complete response
     ↓
Store the response in memory
     ↓
Create a JSON document
     ↓
Parse the entire JSON
     ↓
Navigate through the structure
     ↓
Extract two values
     ↓
Discard temporary data

This approach is completely valid.

In fact, a full JSON parser is often the right solution when your application genuinely needs to work with an entire JSON document dynamically.

But there is an important question:

If the application already knows exactly which values it needs, do we need to allocate and process everything else?

That question became the starting point for a different approach.

---

The Real Cost Isn't HTTP

HTTP itself is not necessarily the problem.

The surrounding workflow often is.

A request can involve:

Create Client
     ↓
Configure Headers
     ↓
Build Authentication
     ↓
Format URL & Query Parameters
     ↓
Build JSON Request Body
     ↓
Open Connection
     ↓
Send Request
     ↓
Receive Response
     ↓
Store Response
     ↓
Allocate JSON Document
     ↓
Parse JSON
     ↓
Extract Values
     ↓
Destroy Temporary Objects

Repeat that process continuously, and things become more interesting.

For an ESP32 application that runs for hours, days, or even months, developers often start thinking about questions that are less important in traditional applications:

• How often am I allocating memory on the heap?
• Am I storing data I don't actually need?
• Can I reuse this connection across requests?
• Can authentication configuration persist without repeated boilerplate?
• Can I avoid repeatedly rebuilding headers and URLs?
• Can I extract only the fields I care about directly into variables?
• Will a missing JSON key crash my device or leave it in an unstable state?

The goal isn't to eliminate every allocation at all costs.

The goal is to avoid allocations and processing that don't provide value to the application.

---

Memory Predictability Matters on Embedded Devices

The ESP32 is a capable microcontroller, but it still operates under resource constraints.

An embedded application might be handling several things simultaneously:

ESP32 Application
│
├── Wi-Fi
├── Sensors
├── Display
├── Bluetooth
├── FreeRTOS Tasks
├── HTTP Communication
└── Application Logic

Every component competes for memory.

This is why memory predictability can be just as important as raw performance.

Repeated dynamic allocations are not automatically bad. However, when an application performs frequent network operations over long periods, reducing unnecessary temporary allocations can help create a more predictable memory profile.

This becomes particularly relevant when dealing with HTTP responses.

---

Do You Really Need the Entire JSON Response?

Let's return to the earlier example.

Suppose the ESP32 only needs:

temperature
device.id

Instead of building an entire JSON structure in memory, another approach is to bind those fields directly to variables.

For example:

float temperature;
char deviceId[32];

client.get("/sensor")
      .getBody("temperature", &temperature)
      .getBody("device.id", deviceId, sizeof(deviceId));

The application tells the HTTP client:

When you find temperature, write it here.

And:

When you find device.id, write it here.

The application doesn't need to manually navigate through a JSON document afterward.

The values go directly to their final destination.

This concept is called response binding, and it became the central idea behind my library, ESP32-HTTP-Client.

Instead of following this workflow:

Response
   ↓
String
   ↓
JSON Document
   ↓
Extract Values

The workflow becomes:

Response Stream
   ↓
Find Required Fields
   ↓
Write Directly Into Variables

The complete response body does not need to be stored before processing.

The library processes the response as a stream and binds matched fields directly to the variables provided by the application.

Nested Paths, Arrays, and Missing Keys

Response binding isn't limited to shallow properties. It supports deep dot notation and numeric array indexing:

char city[32];
int secondReading;

client.get("/data")
      .getBody("sensors.0.readings.1", &secondReading)
      .getBody("device.location.city", city, sizeof(city));

And critically for embedded reliability: if a key is missing or the schema changes, the target variable is simply left untouched. The library does not crash, throw exceptions, or leave memory in an inconsistent state.

---

Reducing Work Means More Than Reducing Memory

Memory was one motivation behind this approach.

But reducing unnecessary work can also improve code simplicity.

Consider the amount of code developers often write around a simple request using the standard ESP32 libraries (HTTPClient + ArduinoJson):

// The traditional approach: ~15-20 lines of procedural boilerplate
HTTPClient http;
http.begin("https://api.example.com/report");
http.addHeader("Authorization", "Bearer my-jwt-token");
http.addHeader("X-Device-ID", "ESP32-001");

int code = http.GET();
if (code == HTTP_CODE_OK) {
    // 1. Read complete payload into RAM (heap allocation #1)
    String payload = http.getString();

    // 2. Allocate JSON document (heap allocation #2)
    DynamicJsonDocument doc(2048);
    DeserializationError err = deserializeJson(doc, payload);

    if (!err) {
        // 3. Navigate structure and copy values manually
        int userId = doc["userId"];
        float temperature = doc["sensor"]["temp"];
        char city[32];
        strncpy(city, doc["address"]["city"] | "", sizeof(city));
    }
}
http.end();

Notice everything that happens here:

1. Authentication and headers are rebuilt for every single request.
2. Intermediate buffers (payload String and doc JSON tree) are allocated on the heap to store data the application will never use.
3. Manual parsing and navigation are written out imperatively.
4. Cleanup and disposal repeat on every iteration.

A good abstraction should reduce repetition without hiding important control from the developer.

With ESP32-HTTP-Client, those responsibilities are divided where they actually belong:

• Authentication and headers belong to the client configuration and are declared once (client.bearer(...), client.setHeader(...)), persisting automatically across requests.
• Buffers and deserialization are bypassed entirely—the response stream binds matched values straight into your destination variables.

So when the goal is simply to retrieve a few values from an API, the application code expresses only its actual intent:

int userId;
float temperature;
char city[32];

// Client already has auth & headers; request simply fetches and binds:
client.get("/report")
      .getBody("userId", &userId)
      .getBody("sensor.temp", &temperature)
      .getBody("address.city", city, sizeof(city));

The code describes the application's actual intent:

Make a request and give me these values.

And if a specific request needs query parameters, a custom timeout, or needs to capture a response header, they fold into the exact same fluent chain without extra boilerplate:

String token;

client.get("/report")
      .query("period", "latest")
      .timeout(5000)
      .getHeader("Authorization", &token)
      .getBody("userId", &userId)
      .getBody("sensor.temp", &temperature)
      .getBody("address.city", city, sizeof(city));

That is the philosophy behind the fluent API used by ESP32-HTTP-Client.

Fluent Payloads and Full REST Support

This fluent design isn't limited to GET requests. REST communication is two-way.

Building JSON request bodies manually with snprintf or String concatenations frequently introduces subtle formatting bugs and heap churn. With ESP32-HTTP-Client, sending structured data is just as straightforward:

int newId;

// POST request with fluent JSON body construction
client.post("/devices")
      .body("name", "ESP32-Office")
      .body("interval", 30)
      .body("active", true)
      .getBody("id", &newId);

The client provides first-class support for all standard REST methods: get(), post(), put() (with update() as an alias), patch(), and del().

Path and Query Parameters Without String Concatenation

Dynamic URLs in Arduino sketches are typically built with repetitive string operations:

// Fragile and causes heap fragmentation:
String url = "/devices/" + String(deviceId) + "/readings?limit=" + String(limit);

ESP32-HTTP-Client solves this at the builder level with .path() and .query():

client.get("/devices/{id}/readings")
      .path("id", deviceId)
      .query("limit", 20)
      .query("filter", "active")
      .getBody("summary.count", &count);

Placeholders are replaced and query parameters are appended smoothly, keeping the code readable and avoiding intermediate String objects on the heap.

---

Beyond Primitives: Struct <-> JSON Mapping

Binding individual primitive variables works well for small endpoints. But in production firmware, you often work with structured data models.

Traditionally, handling a data model on an ESP32 involves:

1. Allocating a DynamicJsonDocument.
2. Parsing the incoming JSON.
3. Manually copying each field into a C++ struct.
4. Freeing the document.
5. Repeating the reverse process whenever you need to serialize the struct to send it back.

ESP32-HTTP-Client addresses this with zero-allocation bidirectional struct mapping via the REST_JSON_MAP macro:

struct DeviceConfig {
    float telemetryInterval = 60.0;
    bool enabled = true;
    char mode[16] = "normal";

    REST_JSON_MAP(
        REST_FIELD(telemetryInterval),
        REST_FIELD(enabled),
        REST_FIELD(mode)
    )
};

Once declared, the struct can be populated directly from an API response:

DeviceConfig config;

// The network stream binds directly into the struct fields
client.get("/device/config").getBody(&config);

And sending a struct to an API is just as clean:

client.post("/device/config").body(config);

No intermediate JSON documents to size or manage. Serialization and deserialization happen directly between your native C++ structures and the network stream.

---

Authentication Shouldn't Become Repeated Boilerplate

Performance and memory are only part of the problem.

Authentication is another common source of repeated HTTP boilerplate.

Many IoT APIs require authentication using:

• Bearer tokens
• JWT tokens
• HTTP Basic Authentication
• API Keys
• Custom headers

Without a reusable abstraction, authentication logic can easily start appearing before every request.

For example:

Request A
 ├── Add Authorization Header
 └── Send Request

Request B
 ├── Add Authorization Header
 └── Send Request

Request C
 ├── Add Authorization Header
 └── Send Request

But authentication usually belongs to the client configuration, not to every individual request.

A reusable client can configure authentication once:

client.bearer("your-jwt-token");

Then continue making requests:

client.get("/profile");

client.get("/devices");

client.get("/telemetry");

The authentication header persists across subsequent requests made by that client instance.

The same idea applies to Basic Authentication:

client.basic("username", "password");

And API keys:

client.apiKey("X-API-Key", "your-api-key");

The client also supports persistent custom headers:

client.setHeader("X-Device-ID", "ESP32-001");

These configurations remain associated with the client and are applied to subsequent requests.

Extracting Response Headers

What about headers returned by the server? Often, authentication tokens, rate limits, or server timestamps are sent in HTTP response headers rather than the JSON body.

Instead of writing custom header collection routines, you can extract response headers inside the same fluent chain:

String refreshedToken;
int rateLimit;

client.get("/telemetry")
      .getHeader("Authorization", &refreshedToken)
      .getHeader("X-RateLimit-Remaining", &rateLimit);

Header lookups are case-insensitive and can be captured into standard C types or Arduino Strings.

This may seem like a small improvement.

But in embedded applications, reducing repeated code also reduces the chances of inconsistent configuration between requests.

---

Reuse the Client, Not Just the Code

Another overlooked source of unnecessary work is connection management.

Imagine an application that does this repeatedly:

Create HTTP Client
      ↓
Connect
      ↓
Make Request
      ↓
Close Connection
      ↓
Destroy Client

Then starts again a few seconds later.

For applications making frequent requests to the same server, connection reuse can significantly reduce overhead.

ESP32-HTTP-Client enables HTTP Keep-Alive and reuses the underlying connection when possible, allowing subsequent requests to avoid unnecessarily rebuilding the connection.

The intended usage looks more like this:

ESP32HTTPClient client("https://api.example.com");

void loop() {
    float temperature;

    client.get("/sensor")
          .getBody("temperature", &temperature);

    delay(5000);
}

The client instance is reused.

That doesn't mean every application should keep a connection alive forever.

There are trade-offs.

An active HTTPS connection can retain TLS buffers in memory. According to the library's documented measurements, calling client.end() after a burst of requests can release resources when the device will remain idle for a long period.

So the better question isn't:

Should I always keep the connection alive?

It is:

Does connection reuse make sense for how frequently my device communicates?

That distinction matters.

---

Measuring the Difference

Performance claims without context are not particularly useful.

So I tested the approach using 100 consecutive HTTP GET requests with JSON responses on a real ESP32 device.

The benchmark compared:

Standard Approach
HTTPClient + ArduinoJson

vs.

ESP32-HTTP-Client
Streaming + Direct Response Binding

The Wi-Fi connection was already established before the benchmark started, and both approaches used their default configurations.

The documented benchmark results were:

Metric	HTTPClient + ArduinoJson	ESP32-HTTP-Client
Heap allocation per request	~58.2 KB	~15 bytes
Average RAM footprint	34.2%	24.3%
Minimum free heap	114.3 KB	128.6 KB
Average execution time	~750 ms	~59 ms

The difference comes from several architectural choices, including connection reuse and streaming JSON processing rather than storing the complete response and parsing it afterward.

However, these numbers should be interpreted correctly.

They represent this specific benchmark configuration.

They do not mean every HTTP request on every ESP32 application will be twelve times faster.

Network conditions, server behavior, payload size, TLS configuration, connection reuse, and application logic can all affect the result.

Benchmarking should always be reproducible and specific to the problem being solved.

The full benchmark setup and source information are documented with the project.

---

A More Practical Example

Let's consider a realistic IoT scenario.

An ESP32 communicates with a remote API.

It needs to:

1. Authenticate using a Bearer token.
2. Identify the device using a custom header.
3. Send telemetry periodically.
4. Receive configuration from the server.
5. Extract only the configuration values it needs.

The workflow looks like this:

               ESP32     
                 │
      Read Sensor Data (Temp & Humidity)
                 │
                 ▼
       Send Telemetry (POST /telemetry)
                 │
                 ▼
          Authenticated API
                 │
                 ▼
   Receive Configuration (GET /device/config)
                 │
                 ▼
     Extract Required Fields Directly

The client configuration happens once during initialization:

#include <WiFi.h>
#include "ESP32HTTPClient.h"

ESP32HTTPClient client("https://api.example.com");

void setup() {
    Serial.begin(115200);
    // Connect to Wi-Fi...

    // Configure persistent auth and headers once:
    client.bearer("your-jwt-token");
    client.setHeader("X-Device-ID", "ESP32-001");
}

Then, in the periodic loop, the application performs both the telemetry upload and configuration check without intermediate boilerplate:

void loop() {
    // 1. Read sensor data
    float currentTemp = 24.8;
    float currentHumidity = 65.0;

    // 2. Send telemetry via POST with a fluent JSON payload
    client.post("/telemetry")
          .body("temperature", currentTemp)
          .body("humidity", currentHumidity);

    if (client.isSuccess()) {
        Serial.println("Telemetry sent successfully.");
    }

    // 3. Retrieve server configuration via GET with direct response binding
    float interval = 30.0;
    bool enabled = true;

    client.get("/device/config")
          .getBody("telemetryInterval", &interval)
          .getBody("enabled", &enabled);

    if (client.isSuccess()) {
        Serial.printf("Config updated: interval=%.1fs, enabled=%d\n", interval, enabled);
    } else {
        Serial.printf("Request error [%d]: %s\n", 
                      client.getStatusCode(), 
                      client.getErrorMessage().c_str());
    }

    delay(interval * 1000);
}

The application does not need to manually:

1. Format a JSON string with snprintf or String additions.
2. Retrieve a complete JSON response string.
3. Allocate a JSON document on the heap.
4. Deserialize the document.
5. Navigate through the JSON structure.

Instead, it declares what it sends and what it expects.

That is a much closer representation of what the application actually needs.

---

But Should You Always Avoid a Full JSON Parser?

No.

This is important.

A full JSON library remains extremely useful.

There are situations where you genuinely need:

• Dynamic JSON structures
• Unknown fields
• Complex transformations
• Entire arrays
• Full objects
• Runtime exploration of a JSON document

In those situations, storing and manipulating the entire document may be the correct solution.

The goal is not:

Never use ArduinoJson.

The goal is:

Don't process more data than your application needs.

If your application requires only three predictable values from a large API response, direct response binding can be a simpler and more memory-conscious alternative.

If your application needs to inspect and manipulate an entire dynamic JSON document, a full JSON parser is probably the better tool.

Choosing the right abstraction depends on the problem.

---

When Does This Approach Actually Matter?

If your ESP32 sends one small request every few hours, HTTP optimization may not be your biggest concern.

But the approach becomes more interesting when your application:

• Runs continuously.
• Makes frequent requests.
• Communicates with the same server repeatedly.
• Has limited memory available.
• Uses multiple peripherals (display, BLE, Wi-Fi, sensors).
• Processes JSON responses.
• Needs predictable memory usage without heap fragmentation.
• Uses authentication across multiple API endpoints.

For these scenarios, reducing unnecessary allocations and repeated configuration can make the networking layer simpler and potentially more efficient.

---

The Idea Behind ESP32-HTTP-Client

I built ESP32-HTTP-Client around a simple question:

Can HTTP communication on an ESP32 be expressed closer to what the application actually needs?

Instead of:

Get everything
     ↓
Store everything
     ↓
Parse everything
     ↓
Use a small part

The approach becomes:

Request what you need
     ↓
Extract what you need
     ↓
Store it where you need it

The library combines several ideas:

• Direct JSON response binding: Maps response values straight into primitive variables or native C++ structs (REST_JSON_MAP).
• Streaming response processing: Parses data on the fly directly from the network stream with zero full-payload buffering.
• Full REST suite with fluent payloads: Clean GET, POST, PUT, PATCH, and DELETE without manual string formatting.
• Safe URL builders: Dynamic path parameters (.path()) and query strings (.query()) without dynamic String concatenation.
• Response header extraction: Binds headers directly via .getHeader() in the same call.
• Persistent authentication: Reusable Bearer tokens, Basic Auth, API keys, and custom headers.
• HTTP Keep-Alive and connection reuse: Drastically reduces handshake latency across sequential requests.
• Built-in reliability: Missing response fields never crash the device, and status checking (isSuccess(), callbacks) is baked in.

The objective was never simply to create another HTTP wrapper.

It was to question how much unnecessary work an ESP32 performs between:

"I need this data"

and:

"Here is the value."

---

Final Thoughts

Making HTTP requests on an ESP32 isn't difficult.

The challenge is everything that tends to accumulate around those requests as an application grows.

Authentication.

Headers.

Temporary buffers.

JSON parsing.

Repeated connections.

Repeated boilerplate.

For many projects, the traditional approach is perfectly fine.

But embedded development often benefits from asking a different question:

What work is actually necessary?

If the ESP32 already knows which values it needs from an API, processing and storing an entire response may not always be necessary.

If authentication remains the same across multiple requests, rebuilding the headers every time may not be necessary.

If the device communicates repeatedly with the same server, reconnecting from scratch may not always be necessary.

Small decisions like these can change how an embedded application behaves over time.

That is the idea behind ESP32-HTTP-Client:

HTTP isn't necessarily the expensive part. Sometimes, it's all the unnecessary work around it.

---

Explore the Project

ESP32-HTTP-Client is an open-source Arduino library designed for ESP32 applications that need a lightweight and fluent approach to REST communication.

You can explore the documentation, examples, benchmarks, and source code here:

ESP32-HTTP-Client Documentation