Rbroughan/fix proto input stream hang

[tryangul]

What

Fixes hanging in other pipelines when one parallel pipeline would fail when using PROTO data medium.

How

• The proto reader is blocking
• When one pipeline fails we close all input streams which unblocks the proto reader and allows the pipelines to cancel as expected
• We run the jobs on a dedicated scope so we can better control the cooperative cancelling
• Rearranges things to make them more readable now that we have this extra logic

To Read

1. PipelineRunner — adds ExceptionHandler for propagating input stream closes and capturing failure cause to rethrow

Other

• Moves dispatcher (thread pool) instantiation into DispatcherBeanFactory
• Re-organizes PipelineRunner.kt for legibility
• Removes unnecessary SocketInputStream — closing the upstream channel causes a source exception to take precedence over the destination exception and must be avoided.

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

• Developing Connectors Locally
• Managing Connector Secrets
• On-Demand Live Tests
• On-Demand Regression Tests
• #connector-ci-issues
• #connector-publish-updates
• #connector-build-statuses

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

• /format-fix - Fixes most formatting issues.
• /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
  Example: /update-connector-cdk-version connector=destination-bigquery
• /bump-version - Bumps connector versions.
  • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
  • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
• /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
• /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
• /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
• /poe source example lock - Alias for /poe connector source-example lock.
• /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
• /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

[edgao]

lgtm, had a a few nits + question about exceptions in exception handling but otherwise

[subodh1810]

@tryangul we need to close the aggregationDispatcher otherwise it causes a hang (tested it). I raised a PR for it on top of your branch, take a look.