Creating solo-docker-environment project

[hendrikebbers]

Today, the easiest way to run real integration tests for any SDK, library, or application based on Hiero is by using solo. Since solo depends on Kubernetes and other tooling, the setup on a local machine is quite complex.

With the hiero-solo-action project, we provide an easy way to set up all that within a GitHub Action. The project is already used in several of our SDKs today to run integration tests against a Hiero-based network directly in GitHub (Actions). Running integration tests in CI/CD is a significant step forward, but it doesn't solve the problem on local machines.

We (the hiero-solo-action maintainers) created an MVP for a solution that wraps the full setup of a Hiero-based network in Docker and provides a way to run a network in Docker. The key advantage of this workflow is that it requires only Docker on a local machine to establish a full network. The network can then be used to run any (integration) tests against it. Like with the GitHub Action, a user account is automatically created and can be used as an operator account for any tests.

Today, the MVP is developed in an internal folder of the GitHub Action repo: https://github.com/hiero-ledger/hiero-solo-action/tree/main/local

We propose creating a new repository, "hiero-solo-docker," alongside the "hiero-solo-action" repository, and defining "Hiero Test Infrastructure" as the sub-project that owns both repositories. The current maintainers/committers of the "hiero-solo-action" repository should become maintainers/committers of the sub-project.

Supporters of the project

Hendrik Ebbers (Hashgraph)
Georgi Stoykov (LimeChain)
Milan Wiercx van Rhijn (8Bees)
Jason Fabritz (BugBytes)

Vision of the project

We want to create an excellent developer experience when working with any Hiero based network. While developers can test any SDK, lib, tool or application today against Hedera Testnet we need more general approaches for automation (CI/CD) and offline support. Next to that a concrete network (as good as it is) should not be the only way to test generic Hiero functionallity. The "Hiero Test Infrastructure" will move all those topics forward by using workflows, tools and libraries that developers already know and often use today in their daily work. Writing and testing any code interacting with a Hiero-based network should not be harder than doing the same with a SQL-based database.

[rbarker-dev]

I disagree with the base assumption that setting up solo is quite complex. The documentation is very straight forward and helpful.

Also the solo development team has developed a quick-start command (and are working on improving it). The solo-action maintainers need to be communicating with the solo development team so that there are no conflicts in the existing roadmap for solo.

[nathanklick]

@hendrikebbers The solo quick-start command provides a two step process to deploy a network today. In the next release, we will be reducing it to a single command which handles the KinD setup as well.

This project is a duplication of work and creates a product which competes directly with the Solo project. This proposal has raised concerns within the Solo team as to why we are building new features which will just be copied by a competing Hiero project.

The Solo maintainers are also concerne with this project being affiliated with the official Solo project by use of the solo moniker.

@jeromy-cannon and @nathanklick do not agree with this project proposal and naming of this project.

[hendrikebbers]

The quick-start command installs tools like kind etc. on your machine. I assume most devs are not interested in that and will not try something with that many new tools.

[hendrikebbers]

This project is a duplication of work and creates a product which competes directly with the Solo project

Where and how do you see that? I do not see any use case of the Docker-based version in that I want to use solo and the needed dependencies directly on my machine. On the other hand I see usecases of solo where the Docker-based approach is useless. But seeing how you will do easy testing with solo for a dev that wants to try Hiero as an example is not clear to me.

[rbarker-dev]

As far as having a clean developer experience, having a tool that installs everything I need to operate is perfect, in my opinion. The quick-start does everything we need to get solo running on a local machine, CI runner, etc and it's a hands-off approach that allows a user to immediately use the full solo toolset.

[nathanklick]

@hendrikebbers KinD is simply a helper command and an OCI container. It runs in docker, podman, CRI-O, ContainerD or any other OCI compliant container runtime. We will support KinD as a standalone binary included with Solo in the next release.

In upcoming releases, we will support podman for installing an OCI container runtime if a developer does not have one available.

[bugbytesinc]

I disagree with the base assumption that setting up solo is quite complex. The documentation is very straight forward and helpful.

As a dev who is building an entire language SDK for Hiero, Solo has a lot of potential, but it is indeed quite complex. The documentation is good, but it still assumes you have a working knowledge of all the component services of the Hiero network, know the ins-and-outs of k8s, know what KinD is, and have a bash shell and docker installed and....well there was more. The barrier to entry is not insignificant.

I failed recently to utilize Solo until I reached out to @rbair23 and team, they helped me out with a custom script to get me unstuck, I am forever grateful for that, otherwise I would have abandoned Solo for the 3rd time.

Ways to make this local hiero environment turn-key for projects developing locally are quite welcome. We're not talking about a huge lift here, even if it is some duplicate work. If the quick-start-command is indeed robust enough to be nearly turn-key, wrap it in a Docker file and call it a win. Seriously, the current experience of having to install all these dependencies in a wsl, vm or on metal really is a headwind at the moment. I think this is a great idea.

[jeromy-cannon]

@hendrikebbers , we are currently under development with features that will automatically check for existing dependencies such as Kind, kubectl, Helm, Docker (or Podman). If they are not met, then it will have the ability to install them.

Essentially, the user can run this command in the future:

npx @hashgraph/solo@latest quick-start single deploy

and the only requirement is to have Docker or Podman installed. But, that is assuming their machine meets the RAM/CPU requirements, which we are also in the process of reducing.

[hendrikebbers]

I believe you will lose many non-JS developers if you say

npx @hashgraph/solo@latest quick-start single deploy

is the best way to go.

Question for better understanding: Are all dependencies (Kind, kubectl, Helm, ...) installed in a local (folder) environment or as global tools, including PATH manipulation, etc.

[jeromy-cannon]

first version is installing them into solo home subdirectory. we haven't got the plans documented on what to do about global and PATH manipulations. crawl / walk / run ...

[jeromy-cannon]

Also, when they did surveys (not sure which one), most of our community is familiar and work with NodeJS regularly. It might be possible that a higher percentage is more familiar with NodeJS than they are with Docker containers. I don't think that survey has been ran.

[hendrikebbers]

Suggestion to move forward:

• We do not propose a "Hiero Test Infrastructure" project. My goal was to make maintainer / committer management more easy. I'm fine with having seperated maintainer / committer management for the new repo.
• I'm fine with not using "solo" in the name of the new repo. I assume that is a benefit, but if the solo team dislikes it, I'm 100% fine with using a different name. "hiero-localnet-docker" could be a name. Question: Should we rename "hiero-solo-action" in parallel to "hiero-localnet-action"? If you have better names, please share them ;)

I still believe that the projects should be developed in parallel to solo. Once everything has stable APIs and a version greater than 1.x, we can discuss transferring the projects directly to solo to have a release train where everything uses the same Solo version. Today, I believe no user cares about that, and the proposed way lets us move forward quickly.

As for the GitHub action, this does not mean that we want to exclude the solo team here. It is more the opposite. We want feedback from the team and have already needed several times the help and knowledge of the solo team. I hope that this cooperation continues with the new project / repo.

cc @nathanklick @jeromy-cannon @rbarker-dev

[rbarker-dev]

I still believe that the projects should be developed in parallel to solo.

I still disagree with this. I don't see the benefit of wrapping solo in a docker container and not keeping it version locked with the solo release version.

I think the best place for this project is within the solo repository as the docker file definition can be updated in lock-step and added into release checks.

[rbarker-dev]

Should we rename "hiero-solo-action" in parallel to "hiero-localnet-action"?

Please no 😄 i think that would add confusion as to whether this was using local-node or solo. I think the action is pretty clear in its intent and what it's doing :)