Project

General

Profile

Actions

Bug #483

closed

generate_packets.py: array-diff behavior for variable-sized arrays not clearly defined

Added by Alina Lenk 7 months ago. Updated 7 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Bootstrap
Target version:
Start date:
04/19/2024
Due date:
% Done:

0%

Estimated time:

Description

When a variable-sized array grows, array-diff doesn't currently have a reasonable, defined behavior for how to treat the newly-added elements.

Three options:
  • Always transmit new elements
  • Diff them relative to empty/zeroed elements
  • Diff them relative to whatever element used to exist in that spot when the array was last that big

The third option is what we're technically doing right now, although not necessarily consistenly so, since no code specifically cares about keeping those extra elements the same.
The first option (always transmitting the new elements) is probably the best option. I don't expect variable-sized arrays to contain many empty elements, but if that does end up being the case (or if we do multi-dimension array-diff), we can still do the second option later.


Files

Actions #2

Updated by Alina Lenk 7 months ago

  • Status changed from In Review to New

Just realized that the special case for diffing a varsize array whose elements themselves also get diff'd (e.g. a multidimensional array) doesn't actually work.

Actions #3

Updated by Alina Lenk 7 months ago

Problem: The receiving side doesn't know the array's old size (since it already overwrote that).
  • Solution 1: As a general rule, don't diff the elements of a varsize array.
    • Upside: The easy way out; least amount of work now; keeps both the generating and generated code simple.
    • Downside: Less efficient if the sub-arrays usually only change a little at a time.
  • Solution 2: Transmit the newly added elements in full, but still in the same format as if they were diff'd.
    • Upside: We can diff the sub-arrays.
    • Downside: Very inefficient whenever new sub-arrays are added.
    • Downside: Adds more bloat to generated code (now three ways to transmit a field type).
  • Solution 3: Make the receiving side remember the old size.
    • Upside: We can diff the existing sub-arrays and plainly transmit the new sub-arrays in full.
    • Upside: Opens the door for future optimizations based on being conscious of the size changing.
    • Downside: Requires code changes that aren't local, and generating variable names that don't collide with anything.

I'll probably go with solution 1 for now. None of the performance aspects actually matter until we actually get in this actual situation; this whole patch is already addressing a case that doesn't currently happen, and I don't want to write too much code that doesn't actually get executed/generated, for the sake of both testing and avoiding bloat.
Solution 3's potential for extra optimizations is all based on varsize arrays with complex elements. So if and when we find ourselves making a varsize array of vectors that we want to fully deep-diff, then it becomes reasonable to do that.

Actions #5

Updated by Alina Lenk 7 months ago

  • Status changed from New to In Review
Actions #6

Updated by Alina Lenk 7 months ago

  • Status changed from In Review to Closed
Actions

Also available in: Atom PDF