GPS autosteer over the browser - Proof of concept

Inspired on the AOG software I have created a browser version, you could see it in simulation mode in the url: Browser GPS . It still is in its infancy so (obviously) many things to improve.

Although sharing a lot in terms of gnss, sensors, motor, controlling ic… it follows a slightly different system/sw architecture. This approach uses a raspberry pi 3b+ to get the hard work done, the rpi acts as a server for the browser application, gets the information from WAS, BNO08X, GNSSs … and makes the control over the ibt-2 controller. A rough explanation of the architecture in video: gps autosteer over the browser - YouTube

The advantages:

  • Lower Investment : Cheap tablet (Android, windows, ios, linux…, even your own telephone, anything with a modern browser) would be enough as display, a rpi 3b+ could run the whole controlling bit (no need for additional udp connections between components, nor hubs)
  • Easy to install , no need for wired connection between pc and controller
  • Easy to upgrade/maintain : it avoids segregation of computing components, all calculations have its roots in the rpi, updating/changing it would be enough to upgrade the whole system. Also it could be distributed easily just posting the rpi image, putting that image on a memory card would make it work, as no interference nor interaction with other parts is undertaken. In terms of software, all is done under the same programming language, so it is also easy to follow and maintain.
  • Powerful , the rpi has tons of power to undertake more complex task (more than Arduino, esp32 or teensy 4.1)
  • Easy to expand , the rpi let us enable many shortcuts (either by keyboards, joysticks, buttons…) in a very simple manner, as it is a computer with all the capabilities of an Operating System.
  • Other advantages: Several displays could be connected to the controller at the same time, micro-c availability

As said previously, it has a lot of edges to polish yet, as it is a new approach.

What do you folks think? is it a path worth to explore? Would it be interesting in any of your applications?

19 Likes

Sounds like an excellent path to explore, to me!

I agree!

It would definitely be useful! Worth exploring for sure.

Some day it would be nice to have AOG split into a backend and frontend anyway. Would make it easier to port it to other platforms.

3 Likes

I would really love it, especially for steering ready machines. Only one Box on the Top and a wifi connected Tablet in the Cab!

great job!

Staggeringly good GUI. Also, seperation of the GUI and processing is what AOG desperately needs, along with a more suitable platform.

Brilliant! This is hugely impressive.

I agree 100%. Porting aside, it would make it way easier for those with processing concept ideas to experiment without having to delve into the mire of winforms.

Here is a hardware concept from @GoRoNb
to work with Raspberry pi. It may be useful.

I am glad to see the positive feedback. :partying_face:

To be fair I did not know what to expect, the range was from “what is a different software in this forum” to hundreds of people asking about the next steps and how to participate in its development.

I was planning to develop a bit further until a beta version, however if there is a major conceptual flaw, I believe is better to tackle asap, that is why I decided to post it in its current stage.

From my view, would be nice to arrange a technical discussion about what could be the best road to follow. By technical I mean an “in depth” IT architecture/sw development dialog to evaluate which option is best for the future, and how to address its development.

The base of what I have shown is javascript, node in the raspberry pi, handling the computing and running the server, and of course in the browser with ES6 standards and modular design as core using some standard well-known libraries like bootstrap and threejs. Any of these parts are open to be modified, if required, but obviously there are reasons to that selection of buildings blocks, obviously too long and arid explanation for a simple post.

From that discussion, once the fundations for sw interaction are fully agreed, the hw pcb could be a parallel line of action. I hope that building on the current experiences (ie. the excellent job undertaken on the All-in-one pcb design) could be very productive.

What do you think? Is there anyone interested in joining the sw design discussion?

3 Likes

For me, It’s better to keep one common straight line in development platform.

That’s kind of the crux of the issue. As much as AOG could benefit from a solid architecture and cross-platform compatibility, to do so means establishing and maintaining a complete fork of the project, which requires a ton of commitment, even if there is cross-pollination between the projects. I tried this a bit a couple of years ago with a direct C++/Qt port (it’s not dead, but it’s certainly in deep sleep). There is also the QtOpenGuidance, which is worth a look at how he used a dataflow programming paradigm. My biggest challenge with a C++ port is keeping up with the pace of change and updates in AOG. Eventually AOG may slow as it reaches feature-completeness. But certainly it was too much for me to keep up on. Because the GUI touches nearly all parts of AOG, even changing GUI toolkits while keeping the rest of the code the same is a challenge. It certainly can be done!

That said I’d enjoy talking about possible designs for a good, future architecture. AOG is a great teacher. The code isn’t impossible to understand, and Brian’s been trying lots of different things to great effect. There are certainly other ways taking what AOG teaches us and implementing them. I’m certainly better at talking than coding. :slight_smile: Bike shedding at its finest.

1 Like

I agree 100%.

From my view, a major point is how to enable a team effort in the software maintenance/development, a modular approach that could allow many people to handle the software and share that big workload, and how to manage that team effort.

In my opinion Brian has done and keeps doing a superb job and I would love to be able to help in the development/maintenance anyhow, but, due to the way the sw is structured, I simply have no clue on how to help. Of course, I could do (as some colleagues are doing) some post-modifications from the main version, but this will always be constricted to that, a patch, with the main workload in one brilliant mind.

The main issue that I see with C++ & Qt is that they are 2 languages, and also, these two languages need some adaptation for different OS, and that could be very tricky in terms of maintenance.

The modularity and correct interfacing between the modules could deal with the issue of everything embedded on the GUI.

The nice thing is that AOG is an excellent tool that is fully functional and that allows to have a meaningful though with no time pressures.

In order to maintain a cross platform development, you must keep up with Brian’s fast changes. That is simply not possible without an automatic conversion. You can work all night and day on catching up, and he will restructure the code about the time you upload your modifications. It is not a bad thing, but it is impossible to keep up. You can get close late summer when he is on the tractor. But after harvest, look out. Nice browser option. I built a small android phone version just to see if I could. There is a lot more to do on this.

3 Likes

A bit off-topic here, but here’s a couple of things along the path to portability and eventual modularization that could be done any time and submitted to the dev team and may be likely to be accepted:

  • Convert OpenGL immediate-mode calls to pipeline calls
    • Need not be done all at once. conversion can be made a little bit at a time.
    • Requires simple shader scripts (which I already did in QtAOG
    • Create a small library of helper functions that emulate the immediate mode drawing commands for lines, triangles, and strips(again I did that in QtAOG). This means to create GL buffers of the vertices and bind them to shader variables.
    • model-view matrix must be precalculated in C# and passed to the GL calls and shaders, and any translations or other manipulations of the “camera” must be done in C#. OpenTK has a built-in matrix library to make that very easy.
    • Eventually make the original AOG vector classes used to store triangles such that they can be directly used by OpenGL as buffers, eliminating the need to copy them.
  • Eliminate circular references back to the main form to reduce class coupling and increase modularity
    • replace calls to main form method functions with custom events (pretty sure this is possible in WinForms even without making custom controls
    • either move variables from the main form into the classes where appropriate, or pass variables in to the class methods.
    • I know it’s possible to do all this and retain the same basic code structure as I did it in the C++ port.
1 Like

Back to the original concept, yes it’s a good one! Even if it’s its own separate project that borrows algorithms from AOG and is perhaps compatible with AOG interfaces.

Any thoughts on the most efficient way to connect the back end with the front end? As you move about the field, the front end needs to draw the coverage triangles. I assume you used a scene manager of some kind to do the rendering? So the backend would need a way to update the front end on new coverage triangles. Or do you think doing all the rendering on the backend and just transmitting a frame is the better way to go? I like the ability to drag the view around with your finger, so I assume that means the rendering has to be done in the front end.

From my view, rather than reinventing the wheel, I find more productive to use well proben libraries or even engines to handle the low level drawing.
For instance, I used threejs to cope with the 3d rendering.

2 Likes

Websockets is what i use for bidirectional communication between server and browser.

Regarding code borrowing, as the aog code is in different language, with quite different aproach i find it difficult that borrowing, but I may be mistaken. Obviously the principles could be similar, the implementation may differ a lot though.

yeah I mean adapting the principles and algorithms.