Package fails due to USPTO endpoint update

[Musaefendic]

Hi,

Thanks for providing this package. I was able to use it successfully in February, but it hasn’t worked for the past few days. It seems that USPTO may have changed something on their side, which now causes the package to fail. I couldn’t pinpoint which endpoint needs to be updated.

I used the code from the README for the minimal reproducible example:

Could anyone help identify what might be causing the issue or suggest a fix?

Thanks!

MRE

# Import the model classes you need
from patent_client import Inpadoc, Assignment, USApplication

# Fetch US Applications
app = USApplication.objects.get('15710770')

---------------------------------------------------------------------------
ConnectError                              Traceback (most recent call last)
File .venv\Lib\site-packages\httpx\_transports\default.py:72, in map_httpcore_exceptions()
     71 try:
---> 72     yield
     73 except Exception as exc:

File .venv\Lib\site-packages\httpx\_transports\default.py:236, in HTTPTransport.handle_request(self, request)
    235 with map_httpcore_exceptions():
--> 236     resp = self._pool.handle_request(req)
    238 assert isinstance(resp.stream, typing.Iterable)

File .venv\Lib\site-packages\httpcore\_sync\connection_pool.py:256, in ConnectionPool.handle_request(self, request)
    255     self._close_connections(closing)
--> 256     raise exc from None
    258 # Return the response. Note that in this case we still have to manage
    259 # the point at which the response is closed.

File .venv\Lib\site-packages\httpcore\_sync\connection_pool.py:236, in ConnectionPool.handle_request(self, request)
    234 try:
    235     # Send the request on the assigned connection.
--> 236     response = connection.handle_request(
    237         pool_request.request
    238     )
    239 except ConnectionNotAvailable:
    240     # In some cases a connection may initially be available to
    241     # handle a request, but then become unavailable.
    242     #
    243     # In this case we clear the connection and try again.

File .venv\Lib\site-packages\httpcore\_sync\connection.py:101, in HTTPConnection.handle_request(self, request)
    100     self._connect_failed = True
--> 101     raise exc
    103 return self._connection.handle_request(request)

File .venv\Lib\site-packages\httpcore\_sync\connection.py:78, in HTTPConnection.handle_request(self, request)
     77 if self._connection is None:
---> 78     stream = self._connect(request)
     80     ssl_object = stream.get_extra_info("ssl_object")

File .venv\Lib\site-packages\httpcore\_sync\connection.py:124, in HTTPConnection._connect(self, request)
    123 with Trace("connect_tcp", logger, request, kwargs) as trace:
--> 124     stream = self._network_backend.connect_tcp(**kwargs)
    125     trace.return_value = stream

File .venv\Lib\site-packages\httpcore\_backends\sync.py:207, in SyncBackend.connect_tcp(self, host, port, timeout, local_address, socket_options)
    202 exc_map: ExceptionMapping = {
    203     socket.timeout: ConnectTimeout,
    204     OSError: ConnectError,
    205 }
--> 207 with map_exceptions(exc_map):
    208     sock = socket.create_connection(
    209         address,
    210         timeout,
    211         source_address=source_address,
    212     )

File C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2544.0_x64__qbz5n2kfra8p0\Lib\contextlib.py:158, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    157 try:
--> 158     self.gen.throw(value)
    159 except StopIteration as exc:
    160     # Suppress StopIteration *unless* it's the same exception that
    161     # was passed to throw().  This prevents a StopIteration
    162     # raised inside the "with" statement from being suppressed.

File .venv\Lib\site-packages\httpcore\_exceptions.py:14, in map_exceptions(map)
     13     if isinstance(exc, from_exc):
---> 14         raise to_exc(exc) from exc
     15 raise

ConnectError: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

ConnectError                              Traceback (most recent call last)
Cell In[264], line 3
      1 from patent_client import Inpadoc, Assignment, USApplication
----> 3 app = USApplication.objects.get('15710770')

File .venv\Lib\site-packages\patent_client\util\manager.py:180, in Manager.get(self, *args, **kwargs)
    178 """If the critera results in a single record, return it, else raise an exception"""
    179 mger = self.filter(*args, **kwargs)
--> 180 length = len(mger)
    181 if length > 1:
    182     raise ValueError("More than one document found!")

File .venv\Lib\site-packages\patent_client\util\manager.py:165, in Manager.__len__(self)
    164 def __len__(self) -> int:
--> 165     return self.count()

File .venv\Lib\site-packages\patent_client\_sync\uspto\peds\manager.py:59, in USApplicationManager.count(self)
     57 def count(self):
     58     api = PatentExaminationDataSystemApi()
---> 59     max_length = (api.create_query(**self.get_query_params())).num_found
     60     return min(max_length, self.config.limit) if self.config.limit else max_length

File .venv\Lib\site-packages\patent_client\_sync\uspto\peds\api.py:113, in PatentExaminationDataSystemApi.create_query(self, query, query_fields, default_field, facet, return_fields, filter_query, minimum_match, sort, start, rows)
    111 url = "https://ped.uspto.gov/api/queries"
    112 logger.debug(f"POST {url}\n{params}")
--> 113 response = client.post(
    114     url,
    115     json=params,
    116     headers={"Accept": "application/json"},
    117 )
    118 self.check_response(response)
    119 return PedsPage.model_validate(response.json())

File .venv\Lib\site-packages\httpx\_client.py:1157, in Client.post(self, url, content, data, files, json, params, headers, cookies, auth, follow_redirects, timeout, extensions)
   1136 def post(
   1137     self,
   1138     url: URL | str,
   (...)   1150     extensions: RequestExtensions | None = None,
   1151 ) -> Response:
   1152     """
   1153     Send a `POST` request.
   1154 
   1155     **Parameters**: See `httpx.request`.
   1156     """
-> 1157     return self.request(
   1158         "POST",
   1159         url,
   1160         content=content,
   1161         data=data,
   1162         files=files,
   1163         json=json,
   1164         params=params,
   1165         headers=headers,
   1166         cookies=cookies,
   1167         auth=auth,
   1168         follow_redirects=follow_redirects,
   1169         timeout=timeout,
   1170         extensions=extensions,
   1171     )

File .venv\Lib\site-packages\httpx\_client.py:837, in Client.request(self, method, url, content, data, files, json, params, headers, cookies, auth, follow_redirects, timeout, extensions)
    822     warnings.warn(message, DeprecationWarning)
    824 request = self.build_request(
    825     method=method,
    826     url=url,
   (...)    835     extensions=extensions,
    836 )
--> 837 return self.send(request, auth=auth, follow_redirects=follow_redirects)

File .venv\Lib\site-packages\httpx\_client.py:926, in Client.send(self, request, stream, auth, follow_redirects)
    922 self._set_timeout(request)
    924 auth = self._build_request_auth(request, auth)
--> 926 response = self._send_handling_auth(
    927     request,
    928     auth=auth,
    929     follow_redirects=follow_redirects,
    930     history=[],
    931 )
    932 try:
    933     if not stream:

File .venv\Lib\site-packages\httpx\_client.py:954, in Client._send_handling_auth(self, request, auth, follow_redirects, history)
    951 request = next(auth_flow)
    953 while True:
--> 954     response = self._send_handling_redirects(
    955         request,
    956         follow_redirects=follow_redirects,
    957         history=history,
    958     )
    959     try:
    960         try:

File .venv\Lib\site-packages\httpx\_client.py:991, in Client._send_handling_redirects(self, request, follow_redirects, history)
    988 for hook in self._event_hooks["request"]:
    989     hook(request)
--> 991 response = self._send_single_request(request)
    992 try:
    993     for hook in self._event_hooks["response"]:

File .venv\Lib\site-packages\httpx\_client.py:1027, in Client._send_single_request(self, request)
   1022     raise RuntimeError(
   1023         "Attempted to send an async request with a sync Client instance."
   1024     )
   1026 with request_context(request=request):
-> 1027     response = transport.handle_request(request)
   1029 assert isinstance(response.stream, SyncByteStream)
   1031 response.request = request

File .venv\Lib\site-packages\hishel\_sync\_transports.py:201, in CacheTransport.handle_request(self, request)
    193             full_response.extensions["cache_metadata"] = metadata  # type: ignore[index]
    194         return Response(
    195             status_code=full_response.status,
    196             headers=full_response.headers,
    197             stream=CacheStream(fake_stream(full_response.content)),
    198             extensions=full_response.extensions,
    199         )
--> 201 response = self._transport.handle_request(request)
    202 assert isinstance(response.stream, tp.Iterable)
    203 httpcore_response = httpcore.Response(
    204     status=response.status_code,
    205     headers=response.headers.raw,
    206     content=CacheStream(response.stream),
    207     extensions=response.extensions,
    208 )

File .venv\Lib\site-packages\httpx\_transports\default.py:235, in HTTPTransport.handle_request(self, request)
    221 assert isinstance(request.stream, SyncByteStream)
    223 req = httpcore.Request(
    224     method=request.method,
    225     url=httpcore.URL(
   (...)    233     extensions=request.extensions,
    234 )
--> 235 with map_httpcore_exceptions():
    236     resp = self._pool.handle_request(req)
    238 assert isinstance(resp.stream, typing.Iterable)

File C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2544.0_x64__qbz5n2kfra8p0\Lib\contextlib.py:158, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    156     value = typ()
    157 try:
--> 158     self.gen.throw(value)
    159 except StopIteration as exc:
    160     # Suppress StopIteration *unless* it's the same exception that
    161     # was passed to throw().  This prevents a StopIteration
    162     # raised inside the "with" statement from being suppressed.
    163     return exc is not value

File .venv\Lib\site-packages\httpx\_transports\default.py:89, in map_httpcore_exceptions()
     86     raise
     88 message = str(exc)
---> 89 raise mapped_exc(message) from exc

ConnectError: [Errno 11001] getaddrinfo failed

[mustberuss]

I noticed things stopped working on April 1st. I think it might just be globally changing /dirsearch-public/ to /api/ in the URLs in patent_client/_async/uspto/public_search/api.py

example:
https://ppubs.uspto.gov/dirsearch-public/searches/counts becomes
https://ppubs.uspto.gov/api/searches/counts

[mrcoliva]

I noticed things stopped working on April 1st. I think it might just be globally changing /dirsearch-public/ to /api/ in the URLs in patent_client/_async/uspto/public_search/api.py

example: https://ppubs.uspto.gov/dirsearch-public/searches/counts becomes https://ppubs.uspto.gov/api/searches/counts

This has worked for me for /counts, but e.g. not for retrieving documents with
https://ppubs.uspto.gov/{dirsearch-public or api}/internal/patents/{bib.guid}/highlight"

Does anyone know how these url changed or where to find documentation about this API?

[mustberuss]

It isn't a true API, it's what's backing the advanced search https://ppubs.uspto.gov/pubwebapp/ I used chrome's inspect to try to figure out what they changed.

Could you try removing the /internal? It looks like it should be https://ppubs.uspto.gov/api/patents/{bib.guid}/highlight

[riemannzeta]

It looks like unasync.py wasn't run also. The _sync version of the public_search/api.py still has the old urls. If you're running in a Jupyter Notebook, for example, you might try manually updating the urls from _async to _sync and/or running unasync.py!

[Musaefendic]

It isn't a true API, it's what's backing the advanced search https://ppubs.uspto.gov/pubwebapp/ I used chrome's inspect to try to figure out what they changed.

Could you try removing the /internal? It looks like it should be https://ppubs.uspto.gov/api/patents/{bib.guid}/highlight

Thanks for the hints @mustberuss

I was able to figure out the changes needed for downloading documents, only for this scope:

• from patent_client import PublicSearchDocument.

It was necessary to update the base URL in patent_client/_sync/uspto/public_search/api.py:

• Before: https://ppubs.uspto.gov/dirsearch-public
• Now: https://ppubs.uspto.gov/api

Also, the URL pattern for retrieving documents changed:

• Before: https://ppubs.uspto.gov/dirsearch-public/internal/patents/{bib.guid}/highlight
• Now: https://ppubs.uspto.gov/api/patents/highlight/{bib.guid}

Note:

• they removed /internal,
• switched from dirsearch-public to api,
• reordered the path: highlight now comes before {bib.guid}.

After these updates, document retrieval works again:

from patent_client import PublicSearchDocument

uspto_number = '20250058346'
doc = PublicSearchDocument.objects.get(uspto_number)

doc.claims
# Output: 
# [
#   Claim(number=1, depends_on=[], text="1. A system for coating an item,..."),
#   ...
# ]

[Musaefendic]

It looks like @parkerhancock already pushed an update (v5.0.19, early April, see this commit)

• that corrected the URLs, but only for the _async functions.
• the _sync still use the old endpoints and haven't been updated yet.

[firmai]

Looking forward to the changes and updated pypi package.

[firmai]

It looks like @parkerhancock already pushed an update (v5.0.19, early April, see this commit)

* that corrected the URLs, but only for the `_async` functions.

* the `_sync` still use the old endpoints and haven't been updated yet.

@Musaefendic are you able to push a commit?

[parkerhancock]

Appreciate the report and stack trace. USPTO shut down the legacy PEDS host on March 14, 2025 (https://www.uspto.gov/system-status/20250212-patent-examination-data-system-peds-retirement), which is why those requests now fail to resolve. We already have Open Data Portal (ODP) coverage in the client, so I’m preparing a PR to route the APIs through ODP by default and remove the deprecated endpoints. I’ll update this issue again once that work is ready for review.

[parkerhancock]

Appreciate the report and stack trace. USPTO shut down the legacy PEDS host on March 14, 2025 (https://www.uspto.gov/system-status/20250212-patent-examination-data-system-peds-retirement), which is why those requests no longer resolve. We already ship an Open Data Portal (ODP) implementation in the client, so I’m preparing a PR to route the primary USApplication APIs through ODP and remove the deprecated endpoints. I’ll update this issue again once that work is ready for review.