The code to determine the endpoint prefix to use when constructing link relation hrefs is:
const determineEndpoint = (req) => {
  const xStacEndpoint = req.get('X-STAC-Endpoint')
  if (xStacEndpoint) return xStacEndpoint
  if (process.env['STAC_API_URL']) return process.env['STAC_API_URL']
  const rootPath = process.env['STAC_API_ROOTPATH'] || ''
  if (req.get('X-Forwarded-Proto') && req.get('X-Forwarded-Host')) {
    return `${req.get('X-Forwarded-Proto')}://${req.get('X-Forwarded-Host')}${rootPath}`
  }
  return req.event && req.event.requestContext && req.event.requestContext.stage
    ? `${req.get('X-Forwarded-Proto')}://${req.get('Host')}/${req.event.requestContext.stage}`
    : `${req.get('X-Forwarded-Proto')}://${req.get('Host')}${rootPath}`
}
In most cases, one of X-STAC-Endpoint (for proxys) or STAC_API_URL (when an explicit custom domain name is used) are set.
It seems that X-Forwarded-Proto and possibly X-Forwarded-Host are not set by the API Gateway REST API integration. I doubt this is a new behavior since AWS usually keeps things like that very consistent, so it may never have worked. In this particular case, X-Forwarded-Proto is undefined, so the urls come out starting with undefined://