Skip to content

Conversation

@overbalance
Copy link
Contributor

@overbalance overbalance commented Sep 10, 2025

What this does

Hoists all build devDependencies to the root package.json.

  • 14 fewer packages in node_modules (1.13% reduction)
  • 878 fewer in dependency tree (11.39% reduction)
  • 304 fewer packages added by npm ci (10.44% reduction)

Key changes

Test fixes

  • Updated karma configs to load plugins from root after dependency hoisting
  • Fixed webpack process polyfill path for ESM compatibility
  • Updated expect import syntax for socket.io tests
  • Updated AWS SDK mock response Content-Length headers

Dependencies

  • All build devDependencies moved to root package.json
  • Removed unused jQuery dependencies

Configuration

  • Converted root .mocharc.yml to .mocharc.json
  • Fixed axios import syntax in express example (namespace import → default import)

Package-specific changes

@opentelemetry/instrumentation-socket.io

  • Changed import * as expect to import expect (expect v29 uses default export)

@opentelemetry/instrumentation-dns

@opentelemetry/instrumentation-aws-sdk

  • Updated Bedrock mock response Content-Length headers

Browser packages (propagator-aws-xray, propagator-instana, instrumentation-user-interaction, instrumentation-long-task)

  • Updated karma.conf.js to load plugins from root package.json

@overbalance overbalance force-pushed the overbalance/hoist-all-shared-deps branch from 3056842 to 804aa71 Compare October 13, 2025 16:19
@JamieDanielson
Copy link
Member

@trentm If Node 18 support is needed then there's more work to do. I missed this requirement when I started this work. 😕 How do you want to proceed? Try to roll back dependency versions until it's happy?

🤔 We say we support Active and Maintenance LTS versions of Node, and Node 18 is officially end of life. So it might be fine in Contrib to drop support for 18. That's a bigger discussion though (cc @open-telemetry/javascript-maintainers ) and we'll need to update docs too.

@trentm
Copy link
Contributor

trentm commented Oct 15, 2025

If Node 18 support is needed then there's more work to do. I missed this requirement when I started this work. 😕 How do you want to proceed? Try to roll back dependency versions until it's happy?

❯ npm ci
npm warn EBADENGINE Unsupported engine {

@overbalance I don't think those EBADENGINE warnings are blocking for this PR. We already have a number of those warnings in the current state. Unfortunately an EBADENGINE warning will matter for some deps, but not for some deps that are just for testing. For example this one:

npm warn EBADENGINE Unsupported engine {
npm warn EBADENGINE   package: '@azure/[email protected]',
npm warn EBADENGINE   required: { node: '>=20.0.0' },
npm warn EBADENGINE   current: { node: 'v18.20.8', npm: '10.8.2' }
npm warn EBADENGINE }

That dep is used by tedious@17:

% npm ls @azure/core-auth
[email protected] /Users/trentm/src/opentelemetry-js-contrib
└─┬ @opentelemetry/[email protected] -> ./packages/instrumentation-tedious
  └─┬ [email protected]
    ├─┬ @azure/[email protected]
    │ ├── @azure/[email protected]
    │ ├─┬ @azure/[email protected]
    │ │ └── @azure/[email protected] deduped
    │ └─┬ @azure/[email protected]
    │   └── @azure/[email protected] deduped
    └─┬ @azure/[email protected]
      ├─┬ @azure-rest/[email protected]
      │ └── @azure/[email protected] deduped
      ├── @azure/[email protected] deduped
      └─┬ @azure/[email protected]
        └── @azure/[email protected] deduped

which is a test dep. We intentionally install a recent-ish major version of tedious so that unit tests (i.e. npm test) for an instrumentation (instrumentation-tedious in this case) tests with a relatively modern version of the target package. We then rely on our test-all-versions tests to handle testing against those versions of tedious that work with older Node.js versions (e.g. Node.js v18)

@overbalance overbalance force-pushed the overbalance/hoist-all-shared-deps branch from 46672a0 to 6ac6dbd Compare October 15, 2025 17:24
@overbalance overbalance changed the title build: hoist all devDeps to root refactor: hoist all devDeps to root Oct 15, 2025
@overbalance
Copy link
Contributor Author

@trentm Those warnings are blocking CI from what I see. What do you recommend?

@trentm
Copy link
Contributor

trentm commented Oct 17, 2025

Those warnings are blocking CI from what I see.

I don't know what is going on. I can npm ci this feature branch locally.

This CI step failed with a network error: https://github.com/open-telemetry/opentelemetry-js-contrib/actions/runs/18537223726/job/53057847496?pr=3032

...
npm WARN EBADENGINE }
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '[email protected]',
npm WARN EBADENGINE   required: { node: '20 || >=22' },
npm WARN EBADENGINE   current: { node: 'v18.19.0', npm: '10.2.3' }
npm WARN EBADENGINE }
npm ERR! code ECONNRESET
npm ERR! network aborted
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network 
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly.  See: 'npm help config'

npm ERR! A complete log of this run can be found in: /home/runner/.npm/_logs/2025-10-17T23_04_44_742Z-debug-0.log
Error: Process completed with exit code 1.

The other test jobs are stuck in npm ci. I cancelled the run after 20m.
I think the "network aborted" above is just a fluke. The real issue is the stuck npm ci steps.

The lint workflow, which does npm ci --ignore-scripts, worked fine.

Ideas on what this could be:

  • Some install/build/postinstall script in one of the deps is breaking things.
  • Some GitHub Actions or npmjs bug/incident that is causing this to break.
  • Some limit is being hit in GH Actions? THis is total guess.

@overbalance overbalance force-pushed the overbalance/hoist-all-shared-deps branch 3 times, most recently from 99cc06d to 324495b Compare October 21, 2025 20:36
@overbalance overbalance force-pushed the overbalance/hoist-all-shared-deps branch 2 times, most recently from 469663e to 9cb3aef Compare October 29, 2025 15:24
trentm pushed a commit that referenced this pull request Oct 29, 2025
... because we are seeing hangs in `npm ci` on #3032.
@overbalance overbalance force-pushed the overbalance/hoist-all-shared-deps branch 4 times, most recently from 6fce51c to 6e7f06f Compare October 30, 2025 05:12
@overbalance
Copy link
Contributor Author

@trentm I repaired the lockfile and this is finally 🟢. Thanks for your help.

Copy link
Contributor

@trentm trentm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple smaller Qs left.

"peerDependencies": {
"@opentelemetry/api": "^1.3.0"
"@opentelemetry/api": "^1.3.0",
"redis": "^5.6.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the move of this from devDependencies to peerDependencies accidental?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is intentional. I think someone else asked about this in this PR. (I know there's too much noise to see it now.) Leaving Redis in devDeps was causing TAV to fail. Would you like me to reconfirm on a new branch?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think someone else asked

I feel it was likely me that asked that before and that I said I'd try to figure out why it is necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't acceptable to add redis as a peerDep, because it'll result in installing that particular redis when a user attempts to install the instrumentation:

% cd .../packages/instrumentation-redis
% npm pack
...
opentelemetry-instrumentation-redis-0.56.0.tgz
% cp opentelemetry-instrumentation-redis-0.56.0.tgz ~/tmp/asdf/

Then in a play/clean npm package in "~/tmp/asdf":

% npm install ./opentelemetry-instrumentation-redis-0.56.0.tgz

% cat package.json
{
...
  "dependencies": {
    "@opentelemetry/instrumentation-redis": "file:opentelemetry-instrumentation-redis-0.56.0.tgz"
  }
}

% npm ls -a
[email protected] /Users/trentm/tmp/asdf
└─┬ @opentelemetry/[email protected]
  ├── @opentelemetry/[email protected]
  ├─┬ @opentelemetry/[email protected]
  │ ├─┬ @opentelemetry/[email protected]
  │ │ └── @opentelemetry/[email protected] deduped
  │ ├── @opentelemetry/[email protected] deduped
  │ ├─┬ [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ └── [email protected] deduped
  │ │ ├── [email protected]
  │ │ ├── [email protected]
  │ │ └── [email protected]
  │ └─┬ [email protected]
  │   ├─┬ [email protected]
  │   │ └── [email protected]
  │   └── [email protected] deduped
  ├── @opentelemetry/[email protected]
  ├── @opentelemetry/[email protected]
  └─┬ [email protected]
    ├─┬ @redis/[email protected]
    │ └── @redis/[email protected] deduped
    ├─┬ @redis/[email protected]
    │ └── [email protected]
    ├─┬ @redis/[email protected]
    │ └── @redis/[email protected] deduped
    ├─┬ @redis/[email protected]
    │ └── @redis/[email protected] deduped
    └─┬ @redis/[email protected]
      └── @redis/[email protected] deduped

I'll see if I can find the TAV issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll see if I can find the TAV issue.

TAV passes for me locally (on macOS, node v20.19.1) after I apply this change to move the "redis" dep back to "devDependencies", and run npm i to update package-lock.json:

diff --git a/packages/instrumentation-redis/package.json b/packages/instrumentation-redis/package.json
index 068654d0..a8912441 100644
--- a/packages/instrumentation-redis/package.json
+++ b/packages/instrumentation-redis/package.json
@@ -48,15 +48,15 @@
     "access": "public"
   },
   "peerDependencies": {
-    "@opentelemetry/api": "^1.3.0",
-    "redis": "^5.6.0"
+    "@opentelemetry/api": "^1.3.0"
   },
   "devDependencies": {
     "@opentelemetry/api": "^1.3.0",
     "@opentelemetry/context-async-hooks": "^2.0.0",
     "@opentelemetry/contrib-test-utils": "^0.54.0",
     "@opentelemetry/sdk-trace-base": "^2.0.0",
-    "@opentelemetry/sdk-trace-node": "^2.0.0"
+    "@opentelemetry/sdk-trace-node": "^2.0.0",
+    "redis": "^5.6.0"
   },
   "dependencies": {
     "@opentelemetry/instrumentation": "^0.207.0",

Then to run TAV tests I did:

cd packages/instrumentation-redis
npm run test-services:start
npm run test-all-versions:with-services-env

So I'm hoping we can just make that change. Whatever the thing is that you hit is hopefully gone now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is my patch to your feature branch: https://gist.github.com/trentm/c415da5938e4ddf81aea9a7670bd4118

Unless I'm missing something, I don't have perms to push to your branch:

% git push embrace-io overbalance/hoist-all-shared-deps
ERROR: Permission to embrace-io/opentelemetry-js-contrib.git denied to trentm.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

@overbalance overbalance requested a review from trentm November 3, 2025 02:11
@overbalance overbalance force-pushed the overbalance/hoist-all-shared-deps branch from e8586e6 to 221a5ef Compare November 3, 2025 22:19
@overbalance overbalance force-pushed the overbalance/hoist-all-shared-deps branch from 79da445 to 57d2922 Compare November 4, 2025 16:28
@trentm
Copy link
Contributor

trentm commented Nov 5, 2025

Unhelpful npm rant

I want to try to care about reviewing package-lock changes, but ... npm, I don't even. This latest update:

  1. updates a bunch of other packages, because the only reasonable way I know of (and probably what you did) was completely regenerate the package-lock file by deleting and re-running npm install.
  2. Loses all the 'integrity' and 'resolved' fields, e.g.:
-      "resolved": "https://registry.npmjs.org/uuid/-/uuid-9.0.1.tgz",
-      "integrity": "sha512-b+1eJOlsR9K8HJpow9Ok3fiWOWSIcIzXodvv0rQjVoOVNpWMpxf1wZNpt4y9h10odCNrqnYp1OBzRktckBe3sA==",

So basically those fields are just useless noise from npm in lock files. What a waste. The package-lock.json tooling design is ...poor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has the "package-lock has lost all the platform-specific optionalDependencies except the one platform" problem again, described at https://cloud-native.slack.com/archives/C08T7MZTV8W/p1761781178575799?thread_ts=1761762353.429559&cid=C08T7MZTV8W

You'll need to manually re-add them, or do some voodoo process under a full moon something like the following to correctly re-gen the lock file:

rm package-lock.json
rm -rf node_modules
npm i

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

has:sponsor This package or feature has a sponsor that has volunteered to review PRs and respond to questions pkg:auto-configuration-propagators pkg:auto-instrumentations-node pkg:auto-instrumentations-web pkg:host-metrics pkg:id-generator-aws-xray pkg:instrumentation-amqplib pkg:instrumentation-aws-lambda pkg:instrumentation-aws-sdk pkg:instrumentation-bunyan pkg:instrumentation-cassandra-driver pkg:instrumentation-connect pkg:instrumentation-cucumber pkg:instrumentation-dataloader pkg:instrumentation-dns pkg:instrumentation-document-load pkg:instrumentation-express pkg:instrumentation-fastify pkg:instrumentation-fs pkg:instrumentation-generic-pool pkg:instrumentation-graphql pkg:instrumentation-hapi pkg:instrumentation-ioredis pkg:instrumentation-kafkajs pkg:instrumentation-knex pkg:instrumentation-koa pkg:instrumentation-long-task pkg:instrumentation-lru-memoizer pkg:instrumentation-memcached pkg:instrumentation-mongodb pkg:instrumentation-mongoose pkg:instrumentation-mysql pkg:instrumentation-mysql2 pkg:instrumentation-nestjs-core pkg:instrumentation-net pkg:instrumentation-openai pkg:instrumentation-oracledb pkg:instrumentation-pg pkg:instrumentation-pino pkg:instrumentation-redis pkg:instrumentation-restify pkg:instrumentation-router pkg:instrumentation-runtime-node pkg:instrumentation-socket.io pkg:instrumentation-tedious pkg:instrumentation-undici pkg:instrumentation-user-interaction pkg:instrumentation-winston pkg:plugin-react-load pkg:propagation-utils pkg:propagator-aws-xray pkg:propagator-aws-xray-lambda pkg:propagator-instana pkg:propagator-ot-trace pkg:redis-common pkg:resource-detector-alibaba-cloud pkg:resource-detector-aws pkg:resource-detector-azure pkg:resource-detector-container pkg:resource-detector-gcp pkg:resource-detector-github pkg:resource-detector-instana pkg:sampler-aws-xray pkg:sql-common pkg:test-utils pkg-status:unmaintained:autoclose-scheduled pkg-status:unmaintained This package is unmaintained. Only bugfixes may be acceped until a new owner has been found.

Projects

None yet

Development

Successfully merging this pull request may close these issues.