Feature #1791
openautoworker end_phase() 145x performance improvements.
0%
Description
Autoworker performance improvements from 1 hr, 24 min, 35 sec to 35 seconds in the end_phase for the given saved game (a 99.3% or 145x improvement).
Please ask design questions and make recommendations.
This approach is more aggressive and less manually directed.
Files
Updated by John Robertson 14 days ago
- File 1791.patch 1791.patch added
[3.3.90.7-dev] seems unstable, especially for the Qt6 client — I will update the patch to future releases as needed.
I will also address reasonable recommendations.
Updated by Marko Lindqvist 14 days ago
Step 1: Remove unrelated (and in many cases plain wrong) changes from the patch. If there are unrelated, but good, changes please open new tickets for them.
There's also some (not many) style issues you may find yourself already before detailed review by rereading doc/CodingStyle and the patch.
Updated by Marko Lindqvist 13 days ago
Attempt to build the patch result in:
1) Number of errors that look like you have forgotten to include packets.def changes (struct packet_worker_task not being what the code expects)
2) Number of "no previous prototype for ..." -errors
Updated by Marko Lindqvist 13 days ago
Has a textual conflict with #1766. Rebase on top of that.
Updated by Marko Lindqvist 13 days ago
- Blocked by Bug #1766: handle_unit_change_activity_real() may keep ACTIVITY_GOTO after clearing goto_tile added
Updated by John Robertson 11 days ago
- File 1761.v2.patch 1761.v2.patch added
patch reduced and updated.
I realize that I am not communicating worker-task-creation between client and server sides. Can you recommend a starting function to use?
Updated by Marko Lindqvist 9 days ago
John Robertson wrote:
Autoworker performance improvements from 1 hr, 24 min, 35 sec to 35 seconds in the end_phase for the given saved game (a 99.3% or 145x improvement).
Can you attach a savegame which you are used for that testing? Sounds very curious if autoworker part takes over an hour. Is the overall turn change time in weeks :-)
Updated by Marko Lindqvist 9 days ago
ecoal = extra_type_by_rule_name(("Coal"));
efallout = extra_type_by_rule_name(("Fallout"));
...
That's completely broken approach. You are assuming that the ruleset has specific extras, even specifically named ones. You should look in the specific properties of the extras you are interested about, and to be prepared for the ruleset to have any number of extras (incl. zero) that match your criteria.
For example, look at how your patch would work with the alien ruleset.
Updated by John Robertson 2 days ago
I started this endeavor with a large map that had over 10K autoworker units that took 18 hours to complete. I found that most of the delay was due to the limited 5 levels of recursive calls finding the "best" task.
Removing the recursion, and choosing the closest task is the crux of the proposed speed improvement.
Additionally, I took the liberty of changing the autoworkers to be more autonomous according to my own personal sentiments. Before, the user needed to direct, by hand the autoworker activities, city by city. Proposed is, for the autoworker to follow the governor and build roads and farms automatically, maximizing food or shields and respecting resources. If my personal sentiments strike accord with other stakeholders, it could be embellished to follow the city's governor more or less aggressively. E.g. no transformations, or cultivations, unless the 'food' setting for the governor is above 20. E.g. no mines when shields is below 5.
Attached is a 1,000 autoworker saved game with the following characteristics:
Unoptimized
3: Workers: 0.639994 sec turn, 0.639993 sec game
3: Workers: 0.491397 sec turn, 1.13139 sec game
Optimized
3: Workers: 0.011371 sec turn, 0.386664 sec game
3: Workers: 0.011157 sec turn, 0.103477 sec game
This is a 50x performance improvement.
Evidently, this is not a linear improvement, affecting larger maps more significantly. I also suspect that, other than AI, users might not be using large numbers of autoworkers. But, even with mostly AI usage, the server performance for large maps will be significantly improved.
Updated by Marko Lindqvist 1 day ago
John Robertson wrote in #note-9:
Before, the user needed to direct, by hand the autoworker activities, city by city.
What do you mean? Autoworkers have always worked by themselves (they are called auto workers, after all). Only recently we've added the worker task concept that user can use to request autoworkers to do specific task, overriding their own task selection methods for the duration of the task.