Improve linking

[mohd-akram]

Add libz dependency for curl rather than getting it through libarchive and fix overlinking on macOS where --as-needed isn't available.

[bapt]

I don t have a macos to test but I beleive this should break the build when built with an external libcurl? this is support was added for building on macos

[bapt]

@kevemueller maybe you can help?

[mohd-akram]

I don t have a macos to test but I beleive this should break the build when built with an external libcurl? this is support was added for building on macos

It should be fine. I had added the additional libraries in #2442, but they're only needed when using the static internal build of libcurl, so I just adjusted them in this PR. I tried now with ./configure --with-libcurl and it worked as expected:

src/pkg:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
	/opt/local/lib/libarchive.13.dylib (compatibility version 21.0.0, current version 21.9.0)
	/opt/local/libexec/openssl3/lib/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/libexec/openssl3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/lib/libcurl.4.dylib (compatibility version 13.0.0, current version 13.0.0)
	/usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)

Without this patch, we get overlinking when using external libcurl:

src/pkg:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
	/opt/local/lib/libarchive.13.dylib (compatibility version 21.0.0, current version 21.9.0)
	/opt/local/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.8)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.3.1)
	/opt/local/lib/liblzma.5.dylib (compatibility version 14.0.0, current version 14.1.0)
	/opt/local/libexec/openssl3/lib/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/libexec/openssl3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/lib/libcurl.4.dylib (compatibility version 13.0.0, current version 13.0.0)
	/usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 3423.0.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1226.0.0)
	/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1351.101.1)

[kevemueller]

Hi @bapt,
in general I would not touch anythinc macOS specific until the CI to build on macOS is restored.

External libcurl was added by you upon my request, as the bundled libcurl never compiled under macOS, it used a pre-generated configuration header, instead of running configure and obtaining proper configuration values for use with macOS.

macOS comes with libcurl and so far we always linked against the system provided libcurl on macOS. I cannot recall anybody having issues with that.
The patch adds the macOS frameworks in all cases, although they are not needed when you build against system libcurl. The least I would ask for the frameworks and the other lib juggling to follow automake variables that are derived from the automake invocation. It should be dead obvious which version of libcurl, libarchive, libz is being used. Btw. we always use bundled libarchive as macOS as the system libarchive is missing headers.

It is also unclear to me if @mohd-akram uses macOS system clang or a recent (brew) clang. We used a recent clang for good reason (e.g. all the sanitizers are only working with brew clang).

All-in-all, I am happy to see macOS improvements, but I would start with restoring the CI build. System clang (and automake fixes for it) as well as bundled libcurl can be all made alternatives in the CI build and automatically tested.

Unfortunately I continue not to have time to delve into this, so even more happy to see somebody else interested in this as well.

Cheers,

Keve

[mohd-akram]

External libcurl was added by you upon my request, as the bundled libcurl never compiled under macOS, it used a pre-generated configuration header, instead of running configure and obtaining proper configuration values for use with macOS.

The bundled libcurl works as of #2442, but it requires memrchr. That can be fixed with a trivial ifdef here:

pkg/external/libcurl/curl_config.h

Lines 457 to 458 in 367b63e

	/* Define to 1 if you have the memrchr function or macro. */
	#define HAVE_MEMRCHR 1

Alternatively, it can be undef'd in all cases to use curl's version on all systems. I didn't make any change there since that's a generated file, and I don't know how patches/tweaks are managed in this repo. I have my own wrapper for string.h in my path, so I was able to build regardless.

The patch adds the macOS frameworks in all cases, although they are not needed when you build against system libcurl. The least I would ask for the frameworks and the other lib juggling to follow automake variables that are derived from the automake invocation. It should be dead obvious which version of libcurl, libarchive, libz is being used. Btw. we always use bundled libarchive as macOS as the system libarchive is missing headers.

Actually this patch fixes #2442 which added macOS frameworks in all cases. With this PR, they are only added when the static build of curl is used. It doesn't seem possible to use the system libcurl from what I can tell. pkg-config is used to get the flags and that's not shipped with macOS.