Big Networks

[simonrmaskell]

I would like to have a network with 9000 nodes described in Yawning Titan but it all seems to grind to a halt. My colleague, Alberto, should be able to provide some specific details.

[ChrisMcCarthyDev]

Hi @simonrmaskell! Thank you for your question.

This is a very interesting one and we'd love to know more about:

• By what means are you creating a 9000-node network?
• Is the network fully connected or are there disconnected nodes?
• When you say "grind to a halt", what exactly is grinding to a halt? Is it a low RAM issue, or a CPU utilisation issue? Or just generally that YT doesn't really do anything and just, as you say, grinds to a halt?
• At what stage is it grinding to a halt (NetworkConfig instantiation, NetworkInterface instantiation, GenericNetworkEnv instantiation, env check and reset, agent.learn, or policy evaluation).
• What is the largest network (number of nodes) you've managed to get working?
• Yawning-Titan version.
• System specs: OS, RAM, CPU, Python version.

Once we have more details there are a few things we can do:

• Add in a bunch of additional debug logging in at all stages to capture key class instantiations, function calls etc.
• Perform an initial run of a 9000 node network then:
  • Monitor system vitals.
  • Monitor the debug level logging.
• Scale up in a VM and perform the same test to see if adding more RAM and CPU has any effect.
• Perform incremental testing by increasing the network size by n each time to determine the melting point if one exists.
• If grinding to a halt on a specific step or function. looking at refactoring for performance. Things like assessing data types, collection types, look-up functions etc.
• Potentially
• Run checks to determine whether GPU is being used or not, and if it is, whether it's being fully utilised.

We look forward to receiving more details Alberto and yourself.

Many thanks,

@ChrisMcCarthyDev and the Yawning-Titan dev team! 🙌

[A-acuto]

Hello @ChrisMcCarthyDev, apologies for the late reply to the discussion, I spent some time creating and documenting the answers better.

• we wanted to understand if Yawning Titan can be used in "realistic" network scenarios, therefore imaging a large network (i.e., 5000, 9000 nodes).
• the network would be fully connected, so not separated pieces of smaller networks.
• testing on 5000 nodes on a local machine (for reference Microsoft 10, 16 GB RAM, not fully dedicated, python3.9, Yawning titan version 1.0.1 and previous) network interface was the bottleneck. The network was created using an adjacency matrix and, from what it was possible to check, it could be saved (not using the internal, old, saving network mechanism of Yawning titan but as an npz file, to save space).
• 5000 nodes, was able to initiate the agents and try to train them. Not achieved to completion.

These were the initial tests, I have also tried them on an HPC computer:

• HPC setups: Linux-3.10.0 x86_64 processor, 9.6 Gb per core
• using this setup, 2000 nodes, were able to load and train (for 5000 steps) a PPO agent.
• using the same, 5000 nodes was possible to load but the training was not possible due to memory exceeded (spikes of memory allocations above the core limit).
• allowing more CPUs per task, it was possible to start running the 5000, however, the code seemed to just allocate more memory but not actually use it.

In this case, only CPU power was used, I trained some agents on GPUs and it worked fine in a network of 25 nodes and 100. However, some RL codes ran into memory exceeded issues (e.g. Numpy allocating too many resources) in the 100 nodes cases (DQN) but, I fear this is an issue regarding stable baselines more than Yawning Titan.

We wondered also, if while developing or checking you have tried the functionality of the software on such large networks and if you had to change some parameters or it was beyond your initial scope.

Happy to chat more about these things and about what we have briefly talked during your presentation.
Cheers,
Alberto Acuto