Bug #483
closed
generate_packets.py: array-diff behavior for variable-sized arrays not clearly defined
Added by Alina Lenk 7 months ago.
Updated 7 months ago.
Description
When a variable-sized array grows, array-diff doesn't currently have a reasonable, defined behavior for how to treat the newly-added elements.
Three options:
- Always transmit new elements
- Diff them relative to empty/zeroed elements
- Diff them relative to whatever element used to exist in that spot when the array was last that big
The third option is what we're technically doing right now, although not necessarily consistenly so, since no code specifically cares about keeping those extra elements the same.
The first option (always transmitting the new elements) is probably the best option. I don't expect variable-sized arrays to contain many empty elements, but if that does end up being the case (or if we do multi-dimension array-diff), we can still do the second option later.
Files
- Status changed from In Review to New
Just realized that the special case for diffing a varsize array whose elements themselves also get diff'd (e.g. a multidimensional array) doesn't actually work.
Problem: The receiving side doesn't know the array's old size (since it already overwrote that).
- Solution 1: As a general rule, don't diff the elements of a varsize array.
- Upside: The easy way out; least amount of work now; keeps both the generating and generated code simple.
- Downside: Less efficient if the sub-arrays usually only change a little at a time.
- Solution 2: Transmit the newly added elements in full, but still in the same format as if they were diff'd.
- Upside: We can diff the sub-arrays.
- Downside: Very inefficient whenever new sub-arrays are added.
- Downside: Adds more bloat to generated code (now three ways to transmit a field type).
- Solution 3: Make the receiving side remember the old size.
- Upside: We can diff the existing sub-arrays and plainly transmit the new sub-arrays in full.
- Upside: Opens the door for future optimizations based on being conscious of the size changing.
- Downside: Requires code changes that aren't local, and generating variable names that don't collide with anything.
I'll probably go with solution 1 for now. None of the performance aspects actually matter until we actually get in this actual situation; this whole patch is already addressing a case that doesn't currently happen, and I don't want to write too much code that doesn't actually get executed/generated, for the sake of both testing and avoiding bloat.
Solution 3's potential for extra optimizations is all based on varsize arrays with complex elements. So if and when we find ourselves making a varsize array of vectors that we want to fully deep-diff, then it becomes reasonable to do that.
Updated patch: Solution 1, as described above.
- Status changed from New to In Review
- Status changed from In Review to Closed
Also available in: Atom
PDF