SSL/TLS Failure in urequests on ESP32 with ESP-IDF 5.4

[jsa91]

Hi,

I'm using an ESP32-WROOM32 module to fetch and display electricity prices on an ILI9341 display.
The prices are fetched from:

https://www.elprisetjustnu.se/api/v1/prices/2025/10-09_SE3.json

This setup has been working flawlessly for a long time, but after pulling the latest MicroPython codebase, fetching data from the API now fails due to SSL/TLS errors in urequests.

for url, day in zip(self.get_url(), self.days):
    print(f"Fetching JSON from: {url}")
    response = urequests.get(url)
    print("Response received")

Using a manual socket test, it seems the error occurs at the line:

s = ctx.wrap_socket(s)

Debugging with Raw socket and ssl.

def test(self, url=b"https://www.elprisetjustnu.se/api/v1/prices/2025/10-09_SE3.json", addr_family=0, use_stream=True):
    # Split the given URL into components
    proto, _, host, path = url.split(b"/", 3)
    assert proto == b"https:"

    # Lookup the server address
    ai = socket.getaddrinfo(host, 443, addr_family, socket.SOCK_STREAM)
    print("Address infos:", ai)

    # Select the first address
    ai = ai[0]

    # Create and connect the socket
    s = socket.socket(ai[0], ai[1], ai[2])
    addr = ai[-1]
    print("Connect address:", addr)
    s.connect(addr)

    # Upgrade to TLS
    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    ctx.check_hostname = False
    ctx.verify_mode = ssl.CERT_NONE
    s = ctx.wrap_socket(s)
    print(s)

    # Send request and read response
    request = b"GET /%s HTTP/1.0\r\nHost: %s\r\n\r\n" % (path, host)
    if use_stream:
        s.write(request)
        print(s.read(4096))
    else:
        s.send(request)
        print(s.recv(4096))

    # Close socket
    s.close()

The failures are:

• Runtime freeze/hang (timeout is not working).
• Occasional crash with the following traceback:

Traceback (most recent call last):
  File "main.py", line 100, in <module>
  File "main.py", line 38, in main
  File "app/price.py", line 212, in test
  File "ssl.py", line 1, in wrap_socket
OSError: 16
MicroPython v1.27.0-preview.275.g6729493531.dirty on 2025-10-09; 
Generic ESP32 module with ESP32
Type "help()" for more information.

The failure seems related to the SSL/TLS handshake or certificate verification layer.

[shariltumin]

This's what I did:

1. Connect to Wifi
2. Use proper DNS. I used '1.1.1.1' (or, '8.8.8.8', '9.9.9.9' ...)
3. Use tls directly

import socket
import tls  # use tls directly
import network
from time import sleep

from wifi import key # (SSID, PWD)
ssid, pwd = key  # give correct cred for your wifi

# Set up WiFi
sta_if = network.WLAN(network.STA_IF)
sta_if.active(True)
sta_if.connect(ssid, pwd)
while not sta_if.isconnected():
    pass

NS = '1.1.1.1'
ip, nm, gw, _ = sta_if.ifconfig()
sta_if.ifconfig((ip, nm, gw, NS))
sleep(5)
print(sta_if.ifconfig()) # check proper DNS set

SRV = "www.elprisetjustnu.se"

ctx = tls.SSLContext(tls.PROTOCOL_TLS_CLIENT)
ctx.verify_mode = tls.CERT_NONE

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
addr = socket.getaddrinfo(SRV, 443)[0][-1]
s.connect(addr)

sx = ctx.wrap_socket(s, server_side=False)

# Send request and read response
path = "api/v1/prices/2025/10-09_SE3.json"
request = b"GET /%s HTTP/1.0\r\nHost: %s\r\n\r\n" % (path, SRV)
sx.write(request)
print(s.read(4096))

# close socket
sx.close()
s.close()

Then, I ran the above script, (test.py):

$ mpr mount . run test.py
Local directory . is mounted at /remote
('192.168.4.85', '255.255.252.0', '192.168.4.1', '1.1.1.1')
b'\x17\x03\x03\x01a\x00\x00\x00\x00\x00\x00\x00\x01\xce\xfb\xf0\x1f\x13t\xe9M\x05$\x124~x\xbe\xcb\xfc\x93\xc1\x05K\xd2)\xe9i[x\xa9\x9f\xc7\xa6\x0c\x1cX\x97\xfb\x1f\\\xab\x81\x88s\x0e\xa4\x11y\xf7\xdd\xee\xf8\xdf\xc1:vc\x9e\xbb\x18\xbd\xff\x03\x0........

The mpr is my short name for mpremote.

If you are interested to see more examples, please take a look at https://github.com/shariltumin/ssl-tls-examples-micropython

[jsa91]

Thank you for your assistance @shariltumin. I have looked at the examples, but unfortunately, I’m still experiencing issues with ctx.wrap_socket.

server = "www.elprisetjustnu.se"
print(server)

ctx = tls.SSLContext(tls.PROTOCOL_TLS_CLIENT)
ctx.verify_mode = tls.CERT_NONE

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
addr = socket.getaddrinfo(server, 443)[0][-1]
s.connect(addr)
print(addr)

pytime.sleep(5)
gc.collect()

sx = ctx.wrap_socket(s, server_side=False)

www.elprisetjustnu.se
('188.114.96.1', 443)
Traceback (most recent call last):
  File "main.py", line 99, in <module>
  File "main.py", line 38, in main
  File "app/price.py", line 111, in get_prices
OSError: 16
MicroPython v1.27.0-preview.307.gddd6ca75be.dirty on 2025-10-10; Generic ESP32 module with ESP32
Type "help()" for more information.

Compiled with ESP-IDF 5.4

I have a lot of traffic tagged with; Connection established and close attempt by responder seen (but no reply from originator)

Could it be that I’m also affected by ##8940 ?

[shariltumin]

Can you run a minimum test script right after a power-on reset? I ran you script and get answer back.

MicroPython v1.27.0-preview.262.g35d07df445 on 2025-10-02; Generic ESP32 module with ESP32
Type "help()" for more information.
>>> 
Remount local directory ./ at /remote
>>> import test_1
www.elprisetjustnu.se
('188.114.96.1', 443)
b'\x17\x03\x03\x01a\x00\x00\x00\x00\x00\x00\x00\x01;~cK\xbd\xc3\x15()Z4o\xa8\xee
...
...

It works. I try with a firmware downloded from micropython.org

[jsa91]

You are perfectly right, thank you! See solution below.

[jsa91]

It wasn’t related to urequests, but rather an issue with ESP-IDF 5.4, which is addressed in #16650.