In the clients we restore the session using the fn ClientBuilder::homeserver_url to restore the previous homeserver URL. However, this modifies the HomeserverConfig the client is built with, so if we had used HomeserverConfig::ServerNameOrHomeserverUrl or HomeserverConfig::ServerName when we logged in we'll now use HomeserverConfig::HomeserverUrl and that will override the existing homeserver config, so when we try to reset the server info we'll try to fetch the server name for that, which is no longer there.
We'd need to use ClientBuilder::server_name_or_homeserver_url instead, or a variant that allows passing both values. But then, this forces a server info fetch operation when we build the Client and if this fails, it won't be built, so no more offline mode in any implementing clients.
So what we really need is some way to allow that request to fail when restoring an existing session, while also making sure we can retrieve the existing server info from the StateStore. Which isn't accessible before the Client is created.