Engine optimization PR guidelines

[lawnjelly]

As discussed in core PR meeting a couple of weeks ago, here is a draft of some notes for contributors making optimization PRs.

Feel free to take over and iterate on. 👍

[Ivorforce]

This is a great start!

I think my most general feedback addresses the structure. You're pretty close to what I'd do myself, but there are some small things I'd change (going by what the user would want to / need to read):

• Intro (explain our general mind-set, I think you nailed that already)
• Choosing what to optimize (going into profiling, bottlenecks, exposed functions, and performance problems reported in issues. For me, this includes the "best / worst case" / "happy / sad path" info)
• Optimizing the code (going into the optimization workflow. We can probably do this mostly by adding links to resources, but we should explain the general mindset and tools you have directly here. e.g. benchmarks, low-level vs high-level, etc.
• Making a pull request (e.g. explaining that the thought process and tests should be documented, and that we need to see numbers vs master)
• Detailed / domain aware info (e.g. the gpu stuff)

[clayjohn]

Alright, I updated this with the review comments and to address some of the ordering suggestions from @Ivorforce's review. I also fleshed out the process a bit more.

[lawnjelly]

Optimizing the code (going into the optimization workflow. We can probably do this mostly by adding links to resources, but we should explain the general mindset and tools you have directly here. e.g. benchmarks, low-level vs high-level, etc.

Just to mention the general optimization section in the regular documentation (https://docs.godotengine.org/en/4.5/tutorials/performance/general_optimization.html#performant-design) does contain some guidance on low level / high level optimization and we should probably link to this where possible and try and avoid too much repetition of similar info in the contributor docs.

Although yes this could be done with a sentence reminding to ensure to consider high level optimizations before getting carried away with low level. On the flip side, high level optimizations are usually more prone to regressions, but overall they should be preferred in the long run.

[lawnjelly]

"It can be difficult to predict which code would benefit from optimization without using performance tools." ?

I wouldn't say it is hard, just non-intuitive?

[lawnjelly]

Actually I've realised we should probably add a paragraph or two about good / bad benchmarks, pitfalls, random data, hot / cold cache, and optimizers completely optimizing out the benchmark. Can have a look later if none of you guys get to it.

Update:
Am limited on time so maybe it would be better to go ahead for now as any docs are better than nothing, I can't guarantee I'll be able to do more on this.

[Ivorforce]

I would avoid going too much into detail, since you could write multiple books about optimization practices. But two or three paragraphs is probably a good idea.

[Ivorforce]

The improvements are great. Looking very promising!

[Ivorforce]

Suggested change

	Optimization Guidelines
	Optimization guidelines

[Ivorforce]

Suggested change

	latest master branch and that you use an optimized build of Godot, since many
	latest ``master`` branch and that you use an optimized build of Godot, since many

[Ivorforce]

Suggested change

	Once you have decided what you should optimize. You should start by capturing a
	Once you have decided what you should optimize, you should start by capturing a

[Ivorforce]

Suggested change

	games. Benchmarks can be helpful, but they can also be misleading as the
	performance of an algorithm given synthetic data may be different than the same
	algorithm running on real world data.
	games. Benchmarks can be helpful, but they can also be misleading because
	it is very difficult benchmarks that reflect how performant the code will be in
	an actual game.

[AThousandShips]

Suggested change

	games. Benchmarks can be helpful, but they can also be misleading as the
	performance of an algorithm given synthetic data may be different than the same
	algorithm running on real world data.
	games. Benchmarks can be helpful, but they can also be misleading because
	it is very difficult to create benchmarks that reflect how performant the code will be in
	an actual game.

Rather

[Ivorforce]

Suggested change

	By default Godot builds with optimizations enabled. See `the Godot build docs
	By default, Godot builds with optimizations enabled. See `the Godot build docs

[Ivorforce]

Suggested change

	and compare the results.
	and compare the results. Note that results will fluctuate, so you'll need to make your test project or benchmark intensive enough to isolate the code you're trying to optimize (ideally, go for ~2 seconds of real-life runtime).
	Additionally, you should run the test multiple times, and observe how much the results fluctuate. Fluctuations of up to 10% are common and expected.

[Ivorforce]

I would put this, along with a broad overview of strengths / weaknesses, in a new section called "Tools for optimization". This would include:

• Profilers
• Benchmarks
• Assembly viewer / godbolt (with a link to Agner Fog's resources)
• A milliseconds per frame counter
• Maybe something else I forgot :D

Suggested change

	Instructions on using some common profilers with Godot can be found `here
	<https://docs.godotengine.org/en/stable/engine_details/development/debugging/using_cpp_profilers.html>`_.

[Ivorforce]

I'd rather put this in the "tools" section mentioned above. Assembly is difficult to wrap your head around, and I wouldn't consider it a requirement if the code is provably way faster than before.

Suggested change

- When micro-optimizing, show assembly before / after to confirm how the optimization is working.