The issue seems to be the wifi stack / protocol groups packets which can cause delays or out of order packets. The AOG application sees the grouped packets as one position update.
It seemed to be related to using a wifi router. One user was able to overcome the issue by making the ESP32 the AP and some changes to the code to avoid grouping packets.
Thanks Gruni. I learn something new about AOG and satellite navigation almost everyday on this forum.