You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some old version of MiniPB uses stream decoding exclusively. However due to how Protobuf handles repeated non-repeated fields (i.e. multiple fields with the same ID while the field is non-repeated) this was changed to pre-indexing the whole message and decode in one go.
We might still be able to bring stream decoding back by making the decode states read-write and return a snapshot of it when the decoding is done (by reaching an end-of-message marker either set by the user or naturally occurring).
This could also make implementing #11 easier and in a much cleaner way (instead of committing the states to a list/object, we just yield the state changes the first time we see them. It would be user's responsibility to only use the last one that pops up).
The text was updated successfully, but these errors were encountered:
I am reversing into a very sloppy work around for accomplishing this on a streaming file for GTFS-RT (transit trip updates). Not sure if I have technical skill to help develop this but wanted to say it would open a lot of doors for me and perhaps others on their projects. This library is so extremely useful and light weight already, but being able to use it with 192kb of RAM is still a challenge.
I got a device that has enough RAM to decode in one go. Adafruit Matrix Portal S3 which as 2MB vs 192KB. Unfortunately, the memory on device gets too fragmented to store results so back to square one. (3.5 kb allocations fail while 300+ kb memory is free).
Do you have any tips on best places place garbage collection calls in minipb.py to keep the memory?
Or any decent ways in theory I can evalate and throw out messages that aren't of interest? I know this is likely the same as a streaming feature you mentioned in the issue body and likely not so simple.
Some old version of MiniPB uses stream decoding exclusively. However due to how Protobuf handles repeated non-repeated fields (i.e. multiple fields with the same ID while the field is non-repeated) this was changed to pre-indexing the whole message and decode in one go.
We might still be able to bring stream decoding back by making the decode states read-write and return a snapshot of it when the decoding is done (by reaching an end-of-message marker either set by the user or naturally occurring).
This could also make implementing #11 easier and in a much cleaner way (instead of committing the states to a list/object, we just yield the state changes the first time we see them. It would be user's responsibility to only use the last one that pops up).
The text was updated successfully, but these errors were encountered: