Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support of ArduPilot Copter/Rover Follow mode #7811

Open
rmackay9 opened this issue Sep 13, 2019 · 74 comments
Open

Support of ArduPilot Copter/Rover Follow mode #7811

rmackay9 opened this issue Sep 13, 2019 · 74 comments
Assignees

Comments

@rmackay9
Copy link

ArduPilot Copter and Rover have a Follow flight mode which is implemented on the flight controller. There are pros and cons of implementing the logic in the vehicles vs the ground station but in any case, this enhancement request is to make use of the flight controller based Follow mode for regular "FollowMe" where the vehicle follows the person holding the ground station computer.

To make this work, the ground station must publish it's position at 1hz (or faster if possible) to the vehicle using the GLOBAL_POSITION_INT message. Beyond this it's possible that the GCS user could modify the position of the vehicle etc with one of the existing mavlink parameters but I haven't looked into this much.

FYI we have a somewhat related issue on the AP side here to support QGC's GCS based follow-mode (i.e. the logic is done on the ground station computer) but I think we should support both methods.

Some videos of the mode in action:

@DonLakeFlyer
Copy link
Contributor

Should it publish GLOBAL_POSITION_INT all the time to any ArduPilot vehicle? Or just when the flight mode is follow me?

@DonLakeFlyer DonLakeFlyer self-assigned this Sep 13, 2019
@DonLakeFlyer DonLakeFlyer added this to the Release v3.6 milestone Sep 13, 2019
@rmackay9
Copy link
Author

rmackay9 commented Sep 14, 2019

@DonLakeFlyer, the GCS only needs to send the position when the vehicle is in Follow mode. The vehicle will stop and hold position if it's not getting position updates.

@DonLakeFlyer
Copy link
Contributor

@rmackay9 I assume GLOBAL_POSITION_INT.relative_alt should always be 0?

@DonLakeFlyer
Copy link
Contributor

@rmackay9 Also ok to set GLOBAL_POSITION_INT.hdg = UINT16_MAX or do you need that?

@rmackay9
Copy link
Author

@DonLakeFlyer, for ground vehicles it won't matter but for flying vehicles it can. On the AP side we have a parameter (FOLL_ALT_TYPE) which allows the user to decide if it will use the relative or absolute altitude from the lead vehicle/GCS. By default is uses the relative alt.

I guess it depends upon whether we want the feature to be useful on ski hills or not.

@DonLakeFlyer
Copy link
Contributor

DonLakeFlyer commented Sep 19, 2019

What is a relative alt for a GCS Position? Relative to what? The first ever gcs position gotten? Kind of an odd concept.

@rmackay9
Copy link
Author

rmackay9 commented Sep 19, 2019

@DonLakeFlyer, the relative altitude in the GLOBAL_POSITION_INT message is relative to the senders home. In AP's current implementation we assume that all the vehicles have very similar home altitudes which is probably true in most cases. This is why we allow using the absolute altitude though as well in case this isn't true. In practice though, because GPSs altitudes are often off using absolute altitudes doesn't always work very well.

We also have some logic in how the position offset is calculate that may paper over some altitude issues. In particular if the FOLL_OFS_X/Y/Z values are left at zero then they're initialised when the following vehicle first sees the target. So even if the altitudes were apparently off by something ridiculous like 200m, the follow may still work because it will try and maintain that 200m difference.

Re the heading, sending a heading of UINT16_MAX will require a change on the AP side. Right now the follow fails in some cases at least if it doesn't get a proper heading. for example the offsets can be setup to be relative to the lead vehicle so in these cases at least it will fail. This is not a really difficult change for us to make though. Alternatively I guess you could make the GCS's heading come from it's ground course... or maybe some tablets know which way they're facing?

@DonLakeFlyer
Copy link
Contributor

Ok no problem, I can get you relative alt and heading.

@DonLakeFlyer
Copy link
Contributor

Another issue is that depending on the device you are running on you may not be able to get things like heading and/or velocity information. I could try to calcuate that from locations but I"m not sure that's going to provide quality data good enough for follow. Should I just bail on sending in that case?

@rmackay9
Copy link
Author

@DonLakeFlyer, I'm not sure. we current depend upon all three being sent (pos, vel, heading).. we can make changes on the AP side to handle them not being present of course or we can just say those devices aren't support for now. I'm not sure

@DonLakeFlyer
Copy link
Contributor

I'm going to not send if the data is missing then. In the current GLOBAL_POSITION_INT as speced there is no way to send with missing bits of information.

This case is a reason I like the FOLLOW_TARGET message better for this. It tells the vehicle which pieces of information are available and then the vehicle can decide if it has enough information to work from. As opposed to coding this into the ground station side. In reality that is the major difference between the two messages. Other than that they work exactly the same and achieve the same result. There isn't anything special with regard to how QGC uses FOLLOW_TARGET. The QGC code is exactly the same up to the point where it sends a mavlink message to the vehicle. In PX4 it sends FOLLOW_TARGET, in ArduPilot it sends GLOBAL_POSITION_INT.

There was a lot of random discussion around this on the ArduPilot issue which headed of in a different more complicated direction. But in reality it should be a few minutes of coding to support FOLLOW_TARGET in ArduPilot. The data is the same, just a different message.

Anyway... I have it all coded. It's going to make me a while to get set up to this this though.

@DonLakeFlyer
Copy link
Contributor

@rmackay9 Can I test this using the current Copter stable?

@DonLakeFlyer
Copy link
Contributor

Tested on Copter master and working

@rmackay9
Copy link
Author

Wow, great!

@DonLakeFlyer DonLakeFlyer reopened this Sep 23, 2019
@DonLakeFlyer
Copy link
Contributor

There is still a problem here. Since HOME_POSITION isnt streamed QGC will only get home position if you boot the vehicle after you start QGC,

@DonLakeFlyer
Copy link
Contributor

This last pull should have everything needed now

@Hwurzburg
Copy link

@DonLakeFlyer Don, so to test this with the QGC nightly build, I would open QGC, and try to connect to the telem connection to my Rover (I use a BT based repeater to connect to the phone from the 915 receiver), then boot the Rover, and put it into FOLLOW mode....correct?

@rmackay9
Copy link
Author

@Hwurzburg do you think you could test this?

@DonLakeFlyer
Copy link
Contributor

@Hwurzburg There is a follow parameter you need to enable as well I believe.

@DonLakeFlyer
Copy link
Contributor

@Hwurzburg I just merged in the home position streaming change to QGC. So a new daily build should be available in about 30 minutes.

@DonLakeFlyer
Copy link
Contributor

Don, so to test this with the QGC nightly build, I would open QGC, and try to connect to the telem connection to my Rover (I use a BT based repeater to connect to the phone from the 915 receiver), then boot the Rover, and put it into FOLLOW mode....correct?

Correct. Other than I think it's FOLLOW_ENABLE parameter or something like that. Don't exactly remember from my testing. I tested with Copter and not Rover. Hopefully shouldn't be a problem.

@Hwurzburg
Copy link

Unfortunately, the daily build apparently is no longer being loaded on my Android Phone every day...and cant get it from play store, says its installed(its not anymore on my phone), but then says all my phones are incompatible...says my Razr is incompatible (stable works fine on it)...

@DonLakeFlyer
Copy link
Contributor

DonLakeFlyer commented Sep 27, 2019

Hmm, the Daily store is probably screwed up with the latest requirements Google just imposed. YOu can install it manually from here: https://qgroundcontrol.s3-us-west-2.amazonaws.com/builds/master/QGroundControl.apk

@DonLakeFlyer
Copy link
Contributor

but follow NED with automatic offset capture on mode entry works for boats and rovers well enough...better than nothing

That is just way below my quality bar for something I'd want to support in QGC.

@Hwurzburg
Copy link

Hwurzburg commented Feb 25, 2020

so what is the issue with just putting FOLLOW in the mode list and broadcasting GPS position? no widgets or dialogs?
I dont know what support you would have to provide...

@DonLakeFlyer
Copy link
Contributor

so what is the issue with just putting FOLLOW in the mode list and broadcasting GPS position? no widgets or dialogs?

Because it's way too easy to get a multi-rotor flying straight at your head by doing that. Or crashing into something when it doesn't work like you expected it to. To me it's a basic safety issue.

@DonLakeFlyer
Copy link
Contributor

The key is that I'm not trying to just expose all functionality of a firmware with QGC. I'm trying to provide a usable and safe experience for QGC users. No matter what feature they may be using.

@rmackay9
Copy link
Author

I suspect that QGC will need to ignore the compass and instead heavily filter and "gate" the GPS heading. So for example, maintain a separate heading that is only updated if the GPS ground speed is over some lower limit (like 1m/s). Some extra heuristics or filtering might also be required to ensure that it doesn't flip back and forth too much.

Another approach we could take is to add a new simpler follow behaviour to ArduPilot in addition to our existing two methods (NED offset and relative offset). This new method would simply try to maintain a distance and ignore the vehicle's heading. So it would just try to move to the edge of a circle surrounding the position provided by the GCS.

Perhaps it's not necessary but I'd like to point out that AP's follow mode is pretty awesome. It's level of performance is far better than existing methods of follow and here are a couple of videos demonstrating this.

Quad following a plane: https://www.youtube.com/watch?v=0s1ZBSB0tvM
Boats following each other: https://www.youtube.com/watch?v=ZSzI7XddT4A

The issue that we're facing is not a shortcoming of AP's follow mode but adapting follow mode to deal with poor inputs. i.e. "garbage in garbage out".

@DonLakeFlyer
Copy link
Contributor

I suspect that QGC will need to ignore the compass and instead heavily filter and "gate" the GPS heading. So for example, maintain a separate heading that is only updated if the GPS ground speed is over some lower limit (like 1m/s). Some extra heuristics or filtering might also be required to ensure that it doesn't flip back and forth too much.

Yup. That is one of the things I"m going to try. These other is to get the heading from a device compass. Hopefully it won't be too large of a change and I can get it into a point release soon.

So it would just try to move to the edge of a circle surrounding the position provided by the GCS.

I"m going to go out and test a Phantom today to see what it does. I think this may be how it works.

@DonLakeFlyer
Copy link
Contributor

@rmackay9 Ok, so I did so more digging into this including looking at how other firmwares implement. The simple answer is that the firmware should not depend on the gcs for a heading. It should generate it's own heading in whatever way is appropriate so it understands the quality of the value. This is why the mavlink FOLLOW_TARGET message does not include heading. I am so many hours into trying to support this it's just ridiculous. Especially given the fact that QGC already supports FOLLOW_TARGET! So my current point of view is that ArduPilot should support FOLLOW_TARGET which is part of the mavlink spec. I just can't spend more time fighting with another way to do follow. Since you already have follow vehicle you should be inches away on the firmware side from being able to do this.

@Hwurzburg
Copy link

sounds like the way to do it to me.....we would only have to add the capability of broadcasting a FOLLOW_TARGET message from the vehicle to allow vehicle to vehicle following like now...or use QGC as the "vehicle" to follow...

the follower would pick a SYS_ID to follow, an offset(alt,distance.angle{NED}) which could be the current offset to target, and follow the broadcaster, correct?
like it was moving guided waypoint(but offset to it)

QGC sends this message with its position and the gcs startup home altitude(or no altitude? or current GCS GPS altitude?), not that it absolutely needed, could be fixed above home in the offset setup...

@DonLakeFlyer
Copy link
Contributor

Not sure there is a need to change the vehicle to vehicle follow code since that is all working fine. Only reason would be to support following some other firmware vehicle which has supported FOLLOW_TARGET. But nothing like that exists right now as far as I know.

I'd just layer gcs follow me support on top of that and support both.

The Altitude field in follow target is up to the provider. Which in reality I'm not sure shouldn't be specified in mavlink spec as AMSL home alt. But for QGC it sends AMSL home altitude.

@Hwurzburg
Copy link

sorry for the probably stupid question, but QGC sends its home alt (which is GPS alt right)? or vehicle's its connected to?

@dogmaphobic
Copy link
Contributor

It sends the altitude reported by the device's GPS (same source as the coordinates).

https://github.com/mavlink/qgroundcontrol/blob/master/src/FollowMe/FollowMe.cc#L111

@DonLakeFlyer
Copy link
Contributor

DonLakeFlyer commented Feb 26, 2020

It sends the altitude reported by the device's GPS (same source as the coordinates).

Hmm forgot about that. That altitudes from a device gps is generally crap. I wonder if PX4 Firmware follow for example just disregards it. And uses vehicle home position altitude.

@DonLakeFlyer
Copy link
Contributor

PX4 docs seems to say height above vehicle home position. Which seems to imply the alt in FOLLOW_TARGET is not used. I'll dig some more on that.

@rmackay9
Copy link
Author

OK, thanks for all the efforts on this. I'll add support for the FOLLOW_TARGET unless someone beats me to it. I'll try and get that done in the timely manner.

@DonLakeFlyer
Copy link
Contributor

I need to do a little more digging but I'm considering adding an option to QGC for follow which allows you to specify whether you want to use vehicle home alt, versus gps alt. With likely default to vehicle home alt which is the higher quality value.

@DonLakeFlyer
Copy link
Contributor

@rmackay9 So according to PX4 folks the follow uses the vehicle home alt internally to calc follow height. It does not use the value from the FOLLOW_TARGET which is the gcs altitude. Which make sense since the gcs alt value can be crap. Which means the FOLLOW_TARGET.alt value isn't used for anything.

@lmore377
Copy link

lmore377 commented Apr 6, 2020

I just came across this issue and I want to help. I have a rover vehicle and I have 2 phones with compasses, a tablet without a compass, and my laptop with a gps dongle so if there's anything I can help with just mention/reply to me.

@DonLakeFlyer
Copy link
Contributor

@lmore377 Thanks. Since your own this thread now you'll be updated when this gets picked up again.

@DonLakeFlyer DonLakeFlyer removed this from the Release V4.1 milestone Nov 8, 2020
@DonLakeFlyer
Copy link
Contributor

Waiting on ardupilot support

@drtrigon
Copy link

drtrigon commented Jul 5, 2021

Any news here? Is this support included in ardupilot 4.1.0 release?

@DonLakeFlyer
Copy link
Contributor

As far as I know not supported bny Ardupilot

@rmackay9
Copy link
Author

rmackay9 commented Jul 8, 2021

That's right, not included in Copter/Rover-4.1 I'm afraid.

@drtrigon
Copy link

drtrigon commented Jul 8, 2021

That's a pitty! Thanks for the info!

Does a schedule exists on when this will get implemented?

@drtrigon
Copy link

The sky-viper journey video drone claims to have follow-me mode on ardupilot: https://sky-viper.com/journey/

@rmackay9
Copy link
Author

@drtrigon, yes it does but it doesn't use QGC. Instead it has a custom application that sends the follow-me commands that AP supports. AP has follow mode by the way, the issue is just that it uses different mavlink messages than QGC sends.

Anyway, we have a concrete plan now actually that Rishabh will sort this out over the next few months.

@drtrigon
Copy link

Anyway, we have a concrete plan now actually that Rishabh will sort this out over the next few months.

Perfect, that are very good news!

@DragonOnGit
Copy link

Guys, i have a problem about the follome mode.
You all talk about following the Groundstation. Is there possible that can follow some other things like other vehicles, cars, phones that can provide its own position information?

@lmore377
Copy link

Guys, i have a problem about the follome mode.
You all talk about following the Groundstation. Is there possible that can follow some other things like other vehicles, cars, phones that can provide its own position information?

You can tell ardurover what mavlink ID to follow

https://ardupilot.org/rover/docs/follow-mode.html

@DragonOnGit
Copy link

Guys, i have a problem about the follome mode.
You all talk about following the Groundstation. Is there possible that can follow some other things like other vehicles, cars, phones that can provide its own position information?

You can tell ardurover what mavlink ID to follow

https://ardupilot.org/rover/docs/follow-mode.html

thank you for responding.
I am trying to use pixhawk+QGC to achieve it. And maybe change code for QGC to create a new mode or some button. Just like takeoff and RTL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants