--html-location option appears not to work with -unit-id package schemes

[mpickering]

As reported by @mpilgrem in matrix.

Hi! Cabal gurus. A question: from GHC 9.8.1, GHC puts the Haddock documentation for its boot packages in folders named after the id or key field in the *.conf file - eg base-4.19.2.0-943b - not after the package name-version. Does Cabal (the tool) deal with that? If so, how? Stack assumes all packages have documentation in $pkg-$version folders, and consequently links between GHC boot packages in Haddock documentation no longer work. I can't see a way through.

In Stack's case, it uses Setup.hs haddock with --html-location=../$pkg-$version/ (link) relying on:

--html-location=url
Specify a template for the location of HTML documentation for prerequisite packages. The substitutions (see listing) are applied to the template to obtain a location for each package, which will be used by hyperlinks in the generated documentation. For example, the following command generates links pointing at Hackage pages:

runhaskell Setup.hs haddock --html-location='http://hackage.haskell.org/packages/archive/$pkg/latest/doc/html'

Here the argument is quoted to prevent substitution by the shell. If this option is omitted, the location for each package is obtained using the package tool (e.g. ghc-pkg).

So, when it builds (say) Haddock documentation files for aeson, the links to the Haddock documentation files for base assume they will be in (eg) ../base-4.19.2.0/ and the links to the documentation files for (say) scientific assume they will be in (eg) ../scientific-0.3.8.0/.

I can't see that there is a way to tell Cabal's haddock command that the Haddock documentation files for GHC boot libraries are in folders which are the id or key value but the documentation files for other libraries are in folders which are named after the package name-version. Further there is no Cabal substitution for use with the --html-location template that is any more specific than a package name-version.
(edited)

Cabal (the library), of course, only builds Haddock documentation files for a package. However, I believe that Cabal (the tool) can build Haddock documentation files for an entire project, so I am assuming that Cabal (the tool) could have encountered the same problem with the post-GHC 9.6.7 change to how the GHC project now provides Haddock files for its boot packages.

However, maybe Cabal (the tool) does not do what Stack does for projects. For projects, Stack moves all of the Haddock documentation files for project packages and all of their dependencies (including GHC boot packages) under a single parent folder, so that all that documentation files can be used together as a single 'unit'. (edited)

I think that, post-GHC 9.6.7, the GHC project provides the Hackage project with special versions of the Haddock documentation files for GHC boot packages that do not use name-version-hash links between them. See:

https://downloads.haskell.org/ghc/9.8.4/docs/users_guide/9.8.3-notes.html#haddock
https://gitlab.haskell.org/ghc/ghc/-/issues/24086

but that is not what GHC ships with.

https://hackage.haskell.org/package/base-4.19.0.0/docs/GHC-Exts.html#v:listThreads-35- has a "Source" link to https://hackage.haskell.org/package/ghc-prim-0.11.0-997e/docs/src/GHC.Prim.html#listThreads%23 which doesn't exist, because it contains part of the ABI hash....

[mpickering]

cc @mpilgrem

[mpickering]

Experiment 1

If I generate docs using ./Setup haddock (without any --html-location argument) then the urls nearly point to the right place..

file:///home/matt/test-proj/dist/doc/xch2rr2qs6bd5pqqjwb0wp9mk6zs944q-ghc-9.12.2-doc/share/doc/ghc/html/libraries/base-4.21.0.0-539a/Prelude.html#t:IO

Experiment 2:

I tried to generate docs using ./Setup haddock --html-location=../$pkgid and the "pkgid" is does not include the full unitid of a package (just the pkg-version name).

Experiment 3:

What does cabal haddock do?

The links by default here are the same as experiment 1. No html-location is passed to ./Setup haddock, ghc-pkg is used to determine the paths to the documentation.

@mpilgrem Is there a reason you move the documentation into base-4.11-<HASH> rather than base-4.11? Then you could continue to use your previous scheme.

Cabal doesn't do the same as stack, the documentation links by default locally are queried directly by ghc-pkg, so the links point into where the documentation is installed.

[mpilgrem]

@mpickering, on your question:

Is there a reason you move the documentation into base-4.11-<HASH> rather than base-4.11?

It is the other way around, as follows:

• Stack uses Cabal (the library) to build Haddock documentation for each project package and its dependencies (other than, the pre-built GHC boot packages) with the --html-location=../$pkg-$version/ convention for links between packages
• GHC 9.6.7 and earlier come with Haddock documentation for each GHC boot package following the same $pkg-$version convention for links between boot packages
• So, when Stack moved all the Haddock documentation (for project packages and their dependencies, including boot packages) into $pkg-$version directories under a .stack-work\install\ ... \doc root, it all worked together as a single 'unit' - like it does on Hackage
• Post-GHC 9.6.7, GHC comes with Haddock documentation for each GHC boot package following a $pkg-$version-<HASH> convention for links between those boot packages. This means Stack can't integrate GHC-supplied Haddock documentation for boot packages with that for all other packages

[mpickering]

@mpilgrem Please forgive me but I am not following one of the steps.

You say in one line.

• When Stack moved all the Haddock documentation (for project packages and their dependencies, including boot packages) into $pkg-$version directories.

and then you say

• Post-GHC 9.6.7, GHC comes with Haddock documentation for each GHC boot package following a $pkg-$version- convention for links between those boot packages.

So I am imagining that you end up with a folder which contains folders like....

base-4.19.0
containers-0.11
etc

but it seems that you imply after copying you get the following?

base-4.19.0-asdasd
containers-0.1-asdasds
etc

But, the point I don't understand is why you copy into base-4.19.0-asdas rather than base-4.19.0 folder.

Is the issue that when you move the documentation then the links in the base package documentation no longer work?

[mpilgrem]

@mpickering, to clarify:

With, say, GHC 9.6.7, Stack copies HTML files:

• from: ... \x86_64-windows\ghc-9.6.7\doc\html\libraries\base-4.18.3.0 (provided by GHC)
• to: ... \testGHC9067\.stack-work\install\ ... \doc\base-4.18.3.0
• and from: ... \x86_64-windows\ghc-9.6.7\doc\html\libraries\ghc-prim-0.10.0 (provided by GHC)
• to: ... \testGHC9067\.stack-work\install\ ... \doc\ghc-prim-0.10.0
• to join up with, say, ... \testGHC9067\.stack-work\install\ ... \doc\acme-missiles-0.3

and all is well with the world. All the HTML files understand where to find each other.

With, say, GHC 9.8.4, Stack copies files:

• from: ... \x86_64-windows\ghc-9.8.4\doc\html\libraries\base-4.19.2.0-943b (provided by GHC)
• to: ... \testGHC9084\.stack-work\install\ ... \doc\base-4.19.2.0
• and from: ... \x86_64-windows\ghc-9.8.4\doc\html\libraries\ghc-prim-0.11.0-4de7 (provided by GHC)
• to: ... \testGHC9084\.stack-work\install\ ... \doc\ghc-prim-0.11.0
• to join up with, say, ... \testGHC9084\.stack-work\install\ ... \doc\acme-missiles-0.3

Now, the HTML files for acme-missiles know where to find those for base and ghc-prim, but base cannot find ghc-prim and vice versa. (The base HTML files can find themselves.)

[mpickering]

Right I see the issue you are talking about now.

• The links in base documentation assume that the documentation for ghc-internal etc lives in ../ghc-internal-<VER>-<HASH>.
• If you copy these folders like stack does, the links from base to ghc-internal break.
• If you copy the folders verbatim, then acme-missiles will fail to find the documentation for base.

I will have a think about what to do.