v1.22.0: Yda Hext

[Xe]

Someone has to make an effort at reconciliation if these conflicts are ever going to end.

In this release, we finally fix the odd number of CPU cores bug, pave the way for lighter weight challenges, make Anubis more adaptable, and more.

Big ticket items

Proof of React challenge

A new "proof of React" has been added. It runs a simple app in React that has several chained hooks. It is much more lightweight than the proof of work check.

Smaller features

• The segments function was added for splitting a path into its slash-separated segments.
• Added possibility to disable HTTP keep-alive to support backends not properly handling it.
• When issuing a challenge, Anubis stores information about that challenge into the store. That stored information is later used to validate challenge responses. This works around nondeterminism in bot rules. (#917)
• One of the biggest sources of lag in Firefox has been eliminated: the use of WebCrypto. Now whenever Anubis detects the client is using Firefox (or Pale Moon), it will swap over to a pure-JS implementation of SHA-256 for speed.
• Proof of work solving has had a complete overhaul and rethink based on feedback from browser engine developers, frontend experts, and overall performance profiling.
• Optimize the performance of the pure-JS Anubis solver.
• Web Workers are stored as dedicated JavaScript files in static/js/workers/*.mjs.
• Pave the way for non-SHA256 solver methods and eventually one that uses WebAssembly (or WebAssembly code compiled to JS for those that disable WebAssembly).
• Legacy JavaScript code has been eliminated.
• When parsing Open Graph tags, add any URLs found in the responses to a temporary "allow cache" so that social preview images work.
• The hard dependency on WebCrypto has been removed, allowing a proof of work challenge to work over plain (unencrypted) HTTP.
• The Anubis version number is put in the footer of every page.
• Add a default block rule for Huawei Cloud.
• Add a default block rule for Alibaba Cloud.
• Added support to use Traefik forwardAuth middleware.
• Add X-Request-URI support so that Subrequest Authentication has path support.

Fixes

Odd numbers of CPU cores are properly supported

Some phones have an odd number of CPU cores. This caused interesting issues. This was fixed by using Math.trunc to convert the number of CPU cores back into an integer.

Smaller fixes

• A standard library HTTP server log message about HTTP pipelining not working has been filtered out of Anubis' logs. There is no action that can be taken about it.
• Added a missing link to the Caddy installation environment in the installation documentation.
• Downstream consumers can change the default log/slog#Logger instance that Anubis uses by setting opts.Logger to your slog instance of choice (#864).
• The Thoth client is now public in the repo instead of being an internal package.
• Custom-AsyncHttpClient's default User-Agent has an increased weight by default (#852).
• Add option for replacing the default explanation text with a custom one (#747)
• The contact email in the LibreJS header has been changed.
• Firefox for Android support has been fixed by embedding the challenge ID into the pass-challenge route. This also fixes some inconsistent issues with other mobile browsers.
• The default favicon pattern in data/common/keep-internet-working.yaml has been updated to permit requests for png/gif/jpg/svg files as well as ico.
• The --cookie-prefix flag has been fixed so that it is fully respected.
• The default patterns in data/common/keep-internet-working.yaml have been updated to appropriately escape the '.' character in the regular expression patterns.
• Add optional restrictions for JWT based on the value of a header (#697)
• The word "hack" has been removed from the translation strings for Anubis due to incidents involving people misunderstanding that word and sending particularly horrible things to the project lead over email.
• Bump AI-robots.txt to version 1.39
• Inject adversarial input to break AI coding assistants.
• Add better logging when using Subrequest Authentication.

Security-relevant changes

• Add a server-side check for the meta-refresh challenge that makes sure clients have waited for at least 95% of the time that they should.

Fix potential double-spend for challenges

Anubis operates by issuing a challenge and having the client present a solution for that challenge. Challenges are identified by a unique UUID, which is stored in the database.

The problem is that a challenge could potentially be used twice by a dedicated attacker making a targeted attack against Anubis. Challenge records did not have a "spent" or "used" field. In total, a dedicated attacker could solve a challenge once and reuse that solution across multiple sessions in order to mint additional tokens.

This was fixed by adding a "spent" field to challenges in the data store. When a challenge is solved, that "spent" field gets set to true. If a future attempt to solve this challenge is observed, it gets rejected.

With the advent of store based challenge issuance in #749, this means that these challenge IDs are only good for 30 minutes. Websites using the most recent version of Anubis have limited exposure to this problem.

Websites using older versions of Anubis have a much more increased exposure to this problem and are encouraged to keep this software updated as often and as frequently as possible.

Thanks to @taviso for reporting this issue.

Breaking changes

• The "slow" frontend solver has been removed in order to reduce maintenance burden. Any existing uses of it will still work, but issue a warning upon startup asking administrators to upgrade to the "fast" frontend solver.
• The legacy JSON based policy file example has been removed and all documentation for how to write a policy file in JSON has been deleted. JSON based policy files will still work, but YAML is the superior option for Anubis configuration.

New Locales

• Lithuanian #972
• Vietnamese #926

What's Changed

• build(deps): bump on-headers and compression in /docs by @dependabot[bot] in build(deps): bump on-headers and compression in /docs #910
• chore: expose thoth in lib by @Xe in chore: expose thoth in lib #911
• test(lib): add a test for the X-Forwarded-For middleware by @Xe in test(lib): add a test for the X-Forwarded-For middleware #912
• build(deps): bump brace-expansion from 1.1.11 to 1.1.12 in /docs by @dependabot[bot] in build(deps): bump brace-expansion from 1.1.11 to 1.1.12 in /docs #909
• test: add automated Pale Moon tests by @Xe in test: add automated Pale Moon tests #903
• feat(expressions): add segments function to break path into segments by @Xe in feat(expressions): add segments function to break path into segments #916
• feat(default-rules): add weight to Custom-AsyncHttpClient by @Xe in feat(default-rules): add weight to Custom-AsyncHttpClient #914
• build(deps): bump the github-actions group with 2 updates by @dependabot[bot] in build(deps): bump the github-actions group with 2 updates #929
• fix(anubis): store the challenge method in the store by @Xe in fix(anubis): store the challenge method in the store #924
• build(deps): bump the gomod group in build(deps): bump the gomod group #931
• Update is.json by @sveinki in Update is.json #935
• fix: polish Turkish translations by @bitigchi in fix: polish Turkish translations #897
• fix(lib): add the ability to set a custom slog Logger by @Xe in fix(lib): add the ability to set a custom slog Logger #915
• fix: allow social preview images by @Xe in fix: allow social preview images #934
• refactor(web): redo proof of work web worker logic by @Xe in refactor(web): redo proof of work web worker logic #941
• Add HackLab.TO to known instances by @lillian-b in Add HackLab.TO to known instances #936
• fix(web/sha256-browserjs): fix function name by @Xe in fix(web/sha256-browserjs): fix function name #943
• Add swedish local by @axellse in Add swedish local #913
• docs: remove JSON examples from policy file docs by @Xe in docs: remove JSON examples from policy file docs #945
• fix(internal): silence unsolicited response log lines by @Xe in fix(internal): silence unsolicited response log lines #950
• fix(web): embed challenge ID in pass-challenge invocations by @Xe in fix(web): embed challenge ID in pass-challenge invocations #944
• build(deps): bump the github-actions group with 2 updates by @dependabot[bot] in build(deps): bump the github-actions group with 2 updates #952
• Revert "build(deps): bump the github-actions group with 2 updates" by @JasonLovesDoggo in Revert "build(deps): bump the github-actions group with 2 updates" #962
• Fix capitalisation in bokmål and nynorsk translations by @turtlegarden in Fix capitalisation in bokmål and nynorsk translations #959
• Added Dutch translation by @SecularSteve in Added Dutch translation #937
• fix(localization): Improve Czech language translation by @Medvidek77 in fix(localization): Improve Czech language translation #895
• feat(checker): allow png/gif/jpg/jpeg/svg favicons as well as ico by @arcayr in feat(checker): allow png/gif/jpg/jpeg/svg favicons as well as ico #961
• default pattern fixes by @arcayr in default pattern fixes #963
• feat: support HTTP redirect for forward authentication middleware in Traefik by @phoval in feat: support HTTP redirect for forward authentication middleware in Traefik #368
• Update known-instances.md: add lab.civicrm.org by @mlutfy in Update known-instances.md: add lab.civicrm.org #971
• feat(lib): Add optional restrictions for JWT based on a specific header value by @Earl0fPudding in feat(lib): Add optional restrictions for JWT based on a specific header value #697
• add Lithuanian locale by @rimas-kudelis in add Lithuanian locale #972
• fix(locales): remove the word "hack" from the description of Anubis by @Xe in fix(locales): remove the word "hack" from the description of Anubis #973
• feat(web): Add option for customizable explanation text by @Earl0fPudding in feat(web): Add option for customizable explanation text #747
• Bump ai.robots.txt to v1.39 by @Dryusdan in Bump ai.robots.txt to v1.39 #982
• feat(blog): add short funding update post by @Xe in feat(blog): add short funding update post #994
• fix(lib): ensure issued challenges don't get double-spent by @Xe in fix(lib): ensure issued challenges don't get double-spent #1003
• fix(default-config): block Huawei Cloud by @Xe in fix(default-config): block Huawei Cloud #1004
• fix(default-config): also block alibaba cloud by @Xe in fix(default-config): also block alibaba cloud #1005
• Update installation.mdx to include a link to the Caddy docs by @juliankrieger in Update installation.mdx to include a link to the Caddy docs #993
• s/Wordpress/WordPress in docs by @bradp in s/Wordpress/WordPress in docs #1020
• docs: fix "stored" typo in CHANGELOG.md by @Krinkle in docs: fix "stored" typo in CHANGELOG.md #1008
• lib/checker: Implement X-Original-URI support by @samip5 in lib/checker: Implement X-Original-URI support #1015
• Add XOriginal to allowed words by @JasonLovesDoggo in Add XOriginal to allowed words #1031
• build(deps-dev): bump the npm group across 1 directory with 3 updates by @dependabot[bot] in build(deps-dev): bump the npm group across 1 directory with 3 updates #1032
• fix(worker): constrain nonce value to be a whole integer by @Xe in fix(worker): constrain nonce value to be a whole integer #1045
• fix: middleware traefik redirect url by @phoval in fix: middleware traefik redirect url #1040
• docs(blog): add post about the odd CPU core count bug by @Xe in docs(blog): add post about the odd CPU core count bug #1058
• docs: Fix broken link by @phuzion in docs: Fix broken link #1059
• Allow to disable keep-alive for the targets not supporting it properly by @samm-git in Allow to disable keep-alive for the targets not supporting it properly #1049
• Alienbob: add Slackware URLs that are now protected by Anubis by @alienbob in Alienbob: add Slackware URLs that are now protected by Anubis #1051
• docs: document client IP headers and interop with cloudflare by @TinyServal in docs: document client IP headers and interop with cloudflare #1034
• Update nginx.mdx - needs port_in_redirect off setting by @own3mall in Update nginx.mdx - needs port_in_redirect off setting #1018
• internal/log: Implement logging of HOST when using subrequest auth by @samip5 in internal/log: Implement logging of HOST when using subrequest auth #1027
• feat: add 'proof of React' challenge by @Xe in feat: add 'proof of React' challenge #1038
• add Lithuanian locale by @rimas-kudelis in add Lithuanian locale #998
• chore: introduce issue templates by @Xe in chore: introduce issue templates #939
• feat(localization): Add Vietnamese translation by @ninfia in feat(localization): Add Vietnamese translation #926
• fix(challenge/metarefresh): ensure that clients have waited long enough by @Xe in fix(challenge/metarefresh): ensure that clients have waited long enough #1068
• ci: fix tests by @Xe in ci: fix tests #1069
• chore: break AI agents in this code tree by @Xe in chore: break AI agents in this code tree #1065
• Updates to lt.json by @rimas-kudelis in Updates to lt.json #1075
• test: ensure FORCED_LANGUAGE works by @Xe in test: ensure FORCED_LANGUAGE works #1083
• feat: glob matching for redirect domains by @Xe in feat: glob matching for redirect domains #1084

New Contributors

• @bitigchi made their first contribution in fix: polish Turkish translations #897
• @lillian-b made their first contribution in Add HackLab.TO to known instances #936
• @axellse made their first contribution in Add swedish local #913
• @SecularSteve made their first contribution in Added Dutch translation #937
• @Medvidek77 made their first contribution in fix(localization): Improve Czech language translation #895
• @arcayr made their first contribution in feat(checker): allow png/gif/jpg/jpeg/svg favicons as well as ico #961
• @phoval made their first contribution in feat: support HTTP redirect for forward authentication middleware in Traefik #368
• @mlutfy made their first contribution in Update known-instances.md: add lab.civicrm.org #971
• @rimas-kudelis made their first contribution in add Lithuanian locale #972
• @juliankrieger made their first contribution in Update installation.mdx to include a link to the Caddy docs #993
• @bradp made their first contribution in s/Wordpress/WordPress in docs #1020
• @Krinkle made their first contribution in docs: fix "stored" typo in CHANGELOG.md #1008
• @samip5 made their first contribution in lib/checker: Implement X-Original-URI support #1015
• @phuzion made their first contribution in docs: Fix broken link #1059
• @samm-git made their first contribution in Allow to disable keep-alive for the targets not supporting it properly #1049
• @alienbob made their first contribution in Alienbob: add Slackware URLs that are now protected by Anubis #1051
• @TinyServal made their first contribution in docs: document client IP headers and interop with cloudflare #1034
• @own3mall made their first contribution in Update nginx.mdx - needs port_in_redirect off setting #1018
• @ninfia made their first contribution in feat(localization): Add Vietnamese translation #926

Full Changelog: v1.21.3...v1.22.0

---

This discussion was created from the release v1.22.0: Yda Hext.

[gucci-on-fleek]

A new "proof of React" has been added. It runs a simple app in React that has several chained hooks. It is much more lightweight than the proof of work check.

This link is broken in the Discussion post; I think that it should be https://anubis.techaro.lol/docs/admin/configuration/challenges/preact.

[Xe]

Yep. It's fixed in the release. I'll fix it here when I wake up tomorrow. Sorry! I copied from the change log file.

[zc-devs]

Thank you for the app in general and for Traefik ForwardAuth middleware support in particular 🥇