networkx to rustworkx

[MIWdlB]

Hi there, this is my first issue in Mesa so i'd be very happy to hear if theres anything i've missed, or any helpful advice you may have 😄

What's the problem this feature will solve?
Given that we're working with python, any amount of performance improvement is helpul. Networkx is significantly slower than rustworkx to build graphs and run algorithms. NX does have quite a broad range of graph methods, but Mesa doesn't make much use of these, so as far as i can tell we wouldn't be losing out.

Describe the solution you'd like
Convert graph-based classes to internally represent graphs with rustworkx. These include:

• discreet_space.network.Network
• space.NetworkGrid

I expect it should be possible to do this without changing the type signature of any functions. Additionally, for these classes, we can ensure backwards compatability by converting nx graphs using rx.networkx_converter.

The main sticking point is that nodes are handled differently in rustworkx, in that they can't be arbitrary python objects. The workaround for this is to use the node index to access data stored in the PyGraph, when the actual object is required.

Additional context
From what I can see, changes to the visualisation module may be more involved than the changes to space classes. Networkx returns an Artist when drawing edges, wheras the rustworkx mpl_draw method returns a whole figure. I'd appreciate some input from someone more familiar with the visualization submodule.

[tpike3]

@MIWdlB Thank you for these thoughts. Performance, particularly with large graphs is always a challenge.

Right now through Google Summer of Code, @Sahil-Chhoker is reworking the visualization front end. At a surface level my concern would be we use Solara and users can currently use matplolib and altair, but we can expand to plotly. So I would be curious of the impact for at least altair.

However, I will let @Sahil-Chhoker, provide more in-depth comments.

[jackiekazil]

One thing I would like to point out is that -- Mesa doesn't expose networkx functions because they are accessible directly in Mesa. So, we don't know which functions people are using.

But I see the speed argument with this. However, more people are familiar with the networkx.
I want to propose the solution being an override -- allow people to over with rustworkx until there is more adoption? Maybe there is a help doc on how to do this?

@tpike3 with @Sahil-Chhoker's work for the summer being wrapped - how do you feel now?

[Sahil-Chhoker]

Yeah, I think the visualization part is where things might get tricky. With NetworkX, the drawing functions are a bit more modular — for example, draw_networkx_nodes and draw_networkx_edges both work on the same global axes, so nodes and edges can be drawn independently. That makes it easier to extend or customize the visualization pipeline.

From what I can tell, rustworkx’s mpl_draw generates a full figure in one go, which doesn’t give you the same decoupling between nodes and edges. That means some of the flexibility Mesa currently has could be harder to replicate unless there’s a workaround or wrapper to reintroduce that separation.

That’s just my impression though — happy to be corrected if I’ve misunderstood something here.

[jackiekazil]

I wasn't even thinking about that part.
It would need to be experimental for a while or an outside project before it gets adopted into core.

[MIWdlB]

Thanks everyone for your thoughts and comments.

@Sahil-Chhoker your comment made me go look again, turns out rustworkx does implement seperate functions for drawing nodes and edges with mpl see here, so perhaps it wouldn't be too difficult.

I'm not familiar with Solara or altair so I don't know how involved that change would be. Do you have any insight on how graphs are used therer

@jackiekazil I can cerrtainly understand not wanting to break any behaviour for users, so I like the idea of an optional approach. It may well be that the performance benefit is only noticable for a small number of users. I imagine this PR would need some benchmarking so that would tell us more?

[EwoutH]

Thanks for working on this!

I don't think we need full support in both matplotlib and altair, one of them is fine.

A PoC that is A) works and is B) faster would be very usefull.

I don't think the use case is that niche, network analysis and simulation is somewhat niche in ABM because it's slow and complex, maybe this helps, at least on the first front.

[jackiekazil]

@jackiekazil I can cerrtainly understand not wanting to break any behaviour for users, so I like the idea of an optional approach. It may well be that the performance benefit is only noticable for a small number of users. I imagine this PR would need some benchmarking so that would tell us more?

I think seeing a prototype PR (with assumed integration, because that is the easiest way to demostrate) and benchmarking would be good to assess if there is integration or something separate.
But I do like the idea of something thrown together for discussion, then refined when we figure out -- exact benefit and if it makes sense to put into main.

[Sahil-Chhoker]

@Sahil-Chhoker your comment made me go look again, turns out rustworkx does implement seperate functions for drawing nodes and edges with mpl see here, so perhaps it wouldn't be too difficult.

You're right, this simplifies things a lot, we can make the required changes to the NetworkSpaceDrawer's draw_matplotlib function, not sure how we'll be able to determine whether the user is using networkx or rustworkx.

I'm not familiar with Solara or altair so I don't know how involved that change would be. Do you have any insight on how graphs are used therer

No need to worry about altair, the plot made here doesn't depend on any library, rather it turns the data into a network like dataframe and plots it.
Solara just calls the respective draw function from specified backend every frame to generate the space.