Losing All AOG when Cell Service drops

If someone is interested, I have a solution to this problem. It requires some changes in the autosteer Teensy code (this is not a problem with the Teensy itself).

I am very much interested

Did some testing this morning. Ran AgOpenGPS on my laptop connected to a panda system. I connected to my cell phone hotspot which in my shop gets poor service unless in the window. I tried for an hour disconnecting and reconnecting. Moving the phone in and out of service. I never did get the system to black screen.

I played with the teensy code and put in some Serial.println commands in before reading the Ntrip packet and sending the ntrip packet.

I also put in some Serial.println commands sending the packet size of each Ntrip packet.

What was interesting is there is a blast of packets sent when a poor connection is reestablished. I wonder if it is flooding the F9P receiver. I also wonder if it is a bad packet getting to the F9P disrupting GPS position.

Since I could not get it to fail in my test environment I am not sure. I will do some more testing later.

1 Like

Did more testing tonight to see if I could reproduce the black screen. I did capture it one time. I have not been able to get it to happen when the serial monitor is up showing the teensy stream.

When it did black out the teensy led continued to blink. Also the F9P led continued to blink.

Starts to black screen around the 1:00 minute mark

weā€™d all be interested, do you have a link to github with your changes maybe?

Hi. Iā€™ve watched this thread with interest, because Iā€™ve experienced this also. If this helps anyone, good, if not, then ignore it. Iā€™d been doing some UDP testing trying to find other problems, and decided to try pinging the panda when AOG lost gps. The interesting result was the panda never quit responding the whole time.


Also there never was a break in the panda readings coming to the serial monitor. Which would point to this being strictly an agio issue

1 Like

Interesting. I plan on some more testing this week. I will take this into account and try to debug AgIO when I am testing.

When I was testing earlier I put in some serial print and println statements in the teensy code for receiving and passing on the NTRIP data. When this code was in place I never could get it to fail. I donā€™t know if that was coincidence or not. I will attempt to test this also this week.

When those serial print statements were in the code I watched on the serial monitor and several NTRIP packets would be sent between panda strings when the signal was low then reestablished. Normally a NTRIP packet would get sent every so many panda strings.

More testing required for conclusive results. I still donā€™t understand why the steering button wired into the teensy would not disengage autosteer when this is happening.

I made a simple UDP to serial converter that printed out every pgn it received from agio. And when AOG would blackscreen, every message would stop. Even the ā€œHello from AGIOā€ message. I wanted to do the same listening to the panda, but I donā€™t have the knowledge for that.

Is there any chance that ā€œvoid Autosteer_Loop()ā€ doesnā€™t ever get run when agio communication gets lost? I donā€™t use AIO so canā€™t do much testing there.

Using strsvr for the first time today in poor cell service and it is working great.

OK, I believe I have determined the line of code that is causing the issue. It is in AgIO, NTRIPComm.Designer.cs.

Line 212

IPAddress[] addresslist = Dns.GetHostAddresses(actualIP);

This line of code is attempting to get the IP address of the URL of the NTRIP caster.

When there is an issue with the wifi connection the DNS server cannot be reached and this line of code hangs trying to resolve the URL. I have been able to replicated this behaviour and this line being the culprit for approximately 5 times.

The Microsoft help page for Dns.GetHostAddresses states that if an IP address is entered then it automatically returns the IP address without attempting to connect to the DNS server.

The temporary work around that I am suggesting we try at this point is to enter the IP address of the NTRIP caster into the ā€œEnter Broadcaster URL or IPā€ text box. As we are finished with planting for the spring I wonā€™t be able to test this in the field anytime soon. If you all that have been affected by this would please test and let me know if this indeed works as a work around then I will work on threading the AgIO program so that the NTRIP side of the program runs on a separate thread than the main AgIO program.

Please provide feedback so I know if this resolves the issue.

3 Likes

My broadcaster junps between 2 ipā€™s. Is this normal?

I bench tested using your modification, and I could never get it to black screen(something of a record, considering it would every thirty seconds with deadly regularity before). So it seems that fixed the issue for me at least.
Thanks!

1 Like

I always thought verifying the URL (resolving the IP) would just use that IP then instead of resolving the IP everytime after that but i guess not all domains use static IPs.

I donā€™t know if it is normal but it is definitely possible. If this is the case this workaround may not work. You can try by entering one of the IP addresses. If you canā€™t get RTK then it may be that the IP has changed.

Itā€™s working fine so far. Iā€™m in good cell service today though so I canā€™t comment on the freezing

if you run nslookup in command line and get more than one IP there its simply load balance between the two servers. The DNS server make the client connect to a random server from the answer.

example:

C:\Users\User>nslookup vg.no
Server:  UnKnown
Address:  ***.***.***.***

Non-authoritative answer:
Name:    vg.no
Addresses:  2001:67c:21e0::16
          195.88.55.16
          195.88.54.16

Here is it 195.88.55.16 and 195.88.54.16 who provide the same service. Just pick one.

1 Like

It does do a dns lookup upon every reconnection, perhaps that should only occur during ntrip setup and only use the ip for reconnect.

If it is wrong - do it again in setup, check the dns, save the ip address.

The other option - donā€™t allow names? What would be most prudent?

Ok, the planā€¦

I will have to make an additional setting entry and save the ip and the dns name

But only use the ip in the reconnection, and only update the ip either manually or by dns lookup in ntrip setup. IF you enter only ip, then name and ip will be both addresses. The only way to update the stored IP will be by dns lookup in ntrip setup.

For now, just enter an ip and not a name.

I have code that will put the lookup on another thread. This should allow the rest of AgIO to run normally while the lookup is attempted. Then if the lookup hangs temporarily it is not an issue.

The plan also sounds like a good plan but if a caster service is not on a static IP then it could cause someoneā€™s NTRIP to quit without knowing why.

I think the threaded operation is more robust for the end user.

Iā€™ve had the ip change at least once since we started using AOG a year ago. So it obviously does sometimes.