Initial implementation of the Bevy Remote Protocol (Adopted) #14880

mweatherley · 2024-08-22T20:07:15Z

Objective

Adopted from #13563.

The goal is to implement the Bevy Remote Protocol over HTTP/JSON, allowing the ECS to be interacted with remotely.

Solution

At a high level, there are really two separate things that have been undertaken here:

First, RemotePlugin has been created, which has the effect of embedding a JSON-RPC endpoint into a Bevy application.
Second, the Bevy Remote Protocol verbs (excluding POLL) have been implemented as remote methods for that JSON-RPC endpoint under a Bevy-exclusive namespace (e.g. bevy/get, bevy/list, etc.).

To avoid some repetition, here is the crate-level documentation, which explains the request/response structure, built-in-methods, and custom method configuration:

Click to view crate-level docs

//! An implementation of the Bevy Remote Protocol over HTTP and JSON, to allow
//! for remote control of a Bevy app.
//!
//! Adding the [`RemotePlugin`] to your [`App`] causes Bevy to accept
//! connections over HTTP (by default, on port 15702) while your app is running.
//! These *remote clients* can inspect and alter the state of the
//! entity-component system. Clients are expected to `POST` JSON requests to the
//! root URL; see the `client` example for a trivial example of use.
//!
//! The Bevy Remote Protocol is based on the JSON-RPC 2.0 protocol.
//!
//! ## Request objects
//!
//! A typical client request might look like this:
//!
//! ```json
//! {
//!     "method": "bevy/get",
//!     "id": 0,
//!     "params": {
//!         "entity": 4294967298,
//!         "components": [
//!             "bevy_transform::components::transform::Transform"
//!         ]
//!     }
//! }
//! ```
//!
//! The `id` and `method` fields are required. The `param` field may be omitted
//! for certain methods:
//!
//! * `id` is arbitrary JSON data. The server completely ignores its contents,
//!   and the client may use it for any purpose. It will be copied via
//!   serialization and deserialization (so object property order, etc. can't be
//!   relied upon to be identical) and sent back to the client as part of the
//!   response.
//!
//! * `method` is a string that specifies one of the possible [`BrpRequest`]
//!   variants: `bevy/query`, `bevy/get`, `bevy/insert`, etc. It's case-sensitive.
//!
//! * `params` is parameter data specific to the request.
//!
//! For more information, see the documentation for [`BrpRequest`].
//! [`BrpRequest`] is serialized to JSON via `serde`, so [the `serde`
//! documentation] may be useful to clarify the correspondence between the Rust
//! structure and the JSON format.
//!
//! ## Response objects
//!
//! A response from the server to the client might look like this:
//!
//! ```json
//! {
//!     "jsonrpc": "2.0",
//!     "id": 0,
//!     "result": {
//!         "bevy_transform::components::transform::Transform": {
//!             "rotation": { "x": 0.0, "y": 0.0, "z": 0.0, "w": 1.0 },
//!             "scale": { "x": 1.0, "y": 1.0, "z": 1.0 },
//!             "translation": { "x": 0.0, "y": 0.5, "z": 0.0 }
//!         }
//!     }
//! }
//! ```
//!
//! The `id` field will always be present. The `result` field will be present if the
//! request was successful. Otherwise, an `error` field will replace it.
//!
//! * `id` is the arbitrary JSON data that was sent as part of the request. It
//!   will be identical to the `id` data sent during the request, modulo
//!   serialization and deserialization. If there's an error reading the `id` field,
//!   it will be `null`.
//!
//! * `result` will be present if the request succeeded and will contain the response
//!   specific to the request.
//!
//! * `error` will be present if the request failed and will contain an error object
//!   with more information about the cause of failure.
//!
//! ## Error objects
//!
//! An error object might look like this:
//!
//! ```json
//! {
//!     "code": -32602,
//!     "message": "Missing \"entity\" field"
//! }
//! ```
//!
//! The `code` and `message` fields will always be present. There may also be a `data` field.
//!
//! * `code` is an integer representing the kind of an error that happened. Error codes documented
//!   in the [`error_codes`] module.
//!
//! * `message` is a short, one-sentence human-readable description of the error.
//!
//! * `data` is an optional field of arbitrary type containing additional information about the error.
//!
//! ## Built-in methods
//!
//! The Bevy Remote Protocol includes a number of built-in methods for accessing and modifying data
//! in the ECS. Each of these methods uses the `bevy/` prefix, which is a namespace reserved for
//! BRP built-in methods.
//!
//! ### bevy/get
//!
//! Retrieve the values of one or more components from an entity.
//!
//! `params`:
//! - `entity`: The ID of the entity whose components will be fetched.
//! - `components`: An array of fully-qualified type names of components to fetch.
//!
//! `result`: A map associating each type name to its value on the requested entity.
//!
//! ### bevy/query
//!
//! Perform a query over components in the ECS, returning all matching entities and their associated
//! component values.
//!
//! All of the arrays that comprise this request are optional, and when they are not provided, they
//! will be treated as if they were empty.
//!
//! `params`:
//! `params`:
//! - `data`:
//!   - `components` (optional): An array of fully-qualified type names of components to fetch.
//!   - `option` (optional): An array of fully-qualified type names of components to fetch optionally.
//!   - `has` (optional): An array of fully-qualified type names of components whose presence will be
//!      reported as boolean values.
//! - `filter` (optional):
//!   - `with` (optional): An array of fully-qualified type names of components that must be present
//!     on entities in order for them to be included in results.
//!   - `without` (optional): An array of fully-qualified type names of components that must *not* be
//!     present on entities in order for them to be included in results.
//!
//! `result`: An array, each of which is an object containing:
//! - `entity`: The ID of a query-matching entity.
//! - `components`: A map associating each type name from `components`/`option` to its value on the matching
//!   entity if the component is present.
//! - `has`: A map associating each type name from `has` to a boolean value indicating whether or not the
//!   entity has that component. If `has` was empty or omitted, this key will be omitted in the response.
//!
//! ### bevy/spawn
//!
//! Create a new entity with the provided components and return the resulting entity ID.
//!
//! `params`:
//! - `components`: A map associating each component's fully-qualified type name with its value.
//!
//! `result`:
//! - `entity`: The ID of the newly spawned entity.
//!
//! ### bevy/destroy
//!
//! Despawn the entity with the given ID.
//!
//! `params`:
//! - `entity`: The ID of the entity to be despawned.
//!
//! `result`: null.
//!
//! ### bevy/remove
//!
//! Delete one or more components from an entity.
//!
//! `params`:
//! - `entity`: The ID of the entity whose components should be removed.
//! - `components`: An array of fully-qualified type names of components to be removed.
//!
//! `result`: null.
//!
//! ### bevy/insert
//!
//! Insert one or more components into an entity.
//!
//! `params`:
//! - `entity`: The ID of the entity to insert components into.
//! - `components`: A map associating each component's fully-qualified type name with its value.
//!
//! `result`: null.
//!
//! ### bevy/reparent
//!
//! Assign a new parent to one or more entities.
//!
//! `params`:
//! - `entities`: An array of entity IDs of entities that will be made children of the `parent`.
//! - `parent` (optional): The entity ID of the parent to which the child entities will be assigned.
//!   If excluded, the given entities will be removed from their parents.
//!
//! `result`: null.
//!
//! ### bevy/list
//!
//! List all registered components or all components present on an entity.
//!
//! When `params` is not provided, this lists all registered components. If `params` is provided,
//! this lists only those components present on the provided entity.
//!
//! `params` (optional):
//! - `entity`: The ID of the entity whose components will be listed.
//!
//! `result`: An array of fully-qualified type names of components.
//!
//! ## Custom methods
//!
//! In addition to the provided methods, the Bevy Remote Protocol can be extended to include custom
//! methods. This is primarily done during the initialization of [`RemotePlugin`], although the
//! methods may also be extended at runtime using the [`RemoteMethods`] resource.
//!
//! ### Example
//! ```ignore
//! fn main() {
//!     App::new()
//!         .add_plugins(DefaultPlugins)
//!         .add_plugins(
//!             // `default` adds all of the built-in methods, while `with_method` extends them
//!             RemotePlugin::default()
//!                 .with_method("super_user/cool_method".to_owned(), path::to::my::cool::handler)
//!                 // ... more methods can be added by chaining `with_method`
//!         )
//!         .add_systems(
//!             // ... standard application setup
//!         )
//!         .run();
//! }
//! ```
//!
//! The handler is expected to be a system-convertible function which takes optional JSON parameters
//! as input and returns a [`BrpResult`]. This means that it should have a type signature which looks
//! something like this:
//! ```
//! # use serde_json::Value;
//! # use bevy_ecs::prelude::{In, World};
//! # use bevy_remote::BrpResult;
//! fn handler(In(params): In<Option<Value>>, world: &mut World) -> BrpResult {
//!     todo!()
//! }
//! ```
//!
//! Arbitrary system parameters can be used in conjunction with the optional `Value` input. The
//! handler system will always run with exclusive `World` access.
//!
//! [the `serde` documentation]: https://serde.rs/

Message lifecycle

At a high level, the lifecycle of client-server interactions is something like this:

The client sends one or more BrpRequests. The deserialized version of that is just the Rust representation of a JSON-RPC request, and it looks like this:

pub struct BrpRequest {
    /// The action to be performed. Parsing is deferred for the sake of error reporting.
    pub method: Option<Value>,

    /// Arbitrary data that will be returned verbatim to the client as part of
    /// the response.
    pub id: Option<Value>,

    /// The parameters, specific to each method.
    ///
    /// These are passed as the first argument to the method handler.
    /// Sometimes params can be omitted.
    pub params: Option<Value>,
}

These requests are accumulated in a mailbox resource (small lie but close enough).
Each update, the mailbox is drained by a system process_remote_requests, where each request is processed according to its method, which has an associated handler. Each handler is a Bevy system that runs with exclusive world access and returns a result; e.g.:

pub fn process_remote_get_request(In(params): In<Option<Value>>, world: &World) -> BrpResult { // ... }

The result (or an error) is reported back to the client.

Testing

This can be tested by using the server and client examples. The client example is not particularly exhaustive at the moment (it only creates barebones bevy/query requests) but is still informative. Other queries can be made using curl with the server example running.

For example, to make a bevy/list request and list all registered components:

curl -X POST -d '{ "jsonrpc": "2.0", "id": 1, "method": "bevy/list" }' 127.0.0.1:15702 | jq .

Future direction

There were a couple comments on BRP versioning while this was in draft. I agree that BRP versioning is a good idea, but I think that it requires some consensus on a couple fronts:

First of all, what does the version actually mean? Is it a version for the protocol itself or for the bevy/* methods implemented using it? Both?
Where does the version actually live? The most natural place is just where we have "jsonrpc" right now (at least if it's versioning the protocol itself), but this means we're not actually conforming to JSON-RPC any more (so, for example, any client library used to construct JSON-RPC requests would stop working). I'm not really against that, but it's at least a real decision.
What do we actually do when we encounter mismatched versions? Adding handling for this would be actual scope creep instead of just a little add-on in my opinion.

Another thing that would be nice is making the internal structure of the implementation less JSON-specific. Right now, for example, component values that will appear in server responses are quite eagerly converted to JSON Values, which prevents disentangling the handler logic from the communication medium, but it can probably be done in principle and I imagine it would enable more code reuse (e.g. for custom method handlers) in addition to making the internals more readily usable for other formats.

@coreh

This commit implements most of the [Bevy Remote Protocol proposal] by @coreh in the form of an HTTP server that's activated when `RemotePlugin` is added to the `App`. Requests and responses are sent via JSON, using `serde` and the reflection capability as appropriate. The server is implemented using Tokio and `hyper`. All of the [proposal] has been implemented, with the exception of the `POLL` request, as it's not needed for the editor and I'm uncertain as to its design. In addition to the verbs in the proposal, an additional verb has been added, `LIST`. It allows the client to enumerate all components attached to an entity, or, if called without an entity, all components that are registered to the `App`. Two new examples have been added, `server` and `client`. The `server` example spawns a test scene and allows connections. The `client` example offers a simple command-line interface to BRP queries. I considered having `client` be more extensive, but it struck me as out of scope for this PR, which is already rather large. [Bevy Remote Protocol proposal]: https://gist.github.com/coreh/1baf6f255d7e86e4be29874d00137d1d#file-bevy-remote-protocol-md [proposal]: https://gist.github.com/coreh/1baf6f255d7e86e4be29874d00137d1d#file-bevy-remote-protocol-md

to implement the BRP

NthTensor

Thank you for picking this up!

One small thing, I think we want to add protocol versioning. In my opinion, we should start at 1.0.0 and follow semver for all verbs, networking types, and server handlers, just like it was a public rust API. The client should include the protocol version on every request.

This will allow compatibility ranges between the editor and game apps. I would prefer we not move this in lock-step with the bevy release; it would be best if BRP was somewhat more stable than the engine.

Cargo.toml

soqb · 2024-08-25T11:12:35Z

One small thing, I think we want to add protocol versioning. In my opinion, we should start at 1.0.0 and follow semver for all verbs, networking types, and server handlers, just like it was a public rust API. The client should include the protocol version on every request.

i agree protocol versioning is important, but i'm not sure that we should start at 1.0 considering Bevy is not yet 1.0, so i think 0.1.0 is probably a better initial version.

Move to JSON-RPC, structure errors, code quality

… component to the server example

MiniaczQ · 2024-08-30T21:51:13Z

BRP is mostly intended for editor, can we keep this as "experimental" and start versioning when the editor integrates it?
There will be more changes coming and there isn't even a consumer

NthTensor · 2024-08-30T21:52:26Z

can we keep this as "experimental"

yeah thats fine

MiniaczQ · 2024-08-30T21:52:01Z

Cargo.toml

+hyper = { version = "1", features = ["full"] }
+hyper-util = { version = "0.1", features = ["full"] }


Probably scope down to only necessary features

MiniaczQ · 2024-08-30T21:52:38Z

crates/bevy_remote/Cargo.toml

+hyper = { version = "1", features = ["full"] }
+hyper-util = { version = "0.1", features = ["full"] }


MiniaczQ · 2024-08-30T21:57:05Z

crates/bevy_remote/src/builtin_methods.rs

+    /// The *full paths* of the component types that are to be requested
+    /// from the entity.
+    ///
+    /// Note that these strings must consist of the *full* type paths: e.g.
+    /// `bevy_transform::components::transform::Transform`, not just
+    /// `Transform`.
+    pub components: Vec<String>,


Doc is duplicated a lot and the exact method of getting the "full" path is not mentioned
Can we newtype string into ComponentPath that has a new::<C: Component> constructor?

EDIT: this is used A LOT, and the descriptions differ

MiniaczQ · 2024-08-30T22:01:49Z

crates/bevy_remote/src/lib.rs

+            .add_systems(Startup, start_server)
+            .add_systems(Update, process_remote_requests);


Not in scope for this PR, but mixing "editor" logic with game logic will cause trouble in debugging, like during system stepping you won't be able to use BRP, because it's just a "game logic" system.
Is there a proposed solution for this?

Probably to special-case out the systems that make the editor 'embassy' tools like this work. This discussion might be better suited to discord.

ChristopherBiscardi · 2024-08-31T00:52:00Z

examples/remote/client.rs

+
+/// The application entry point.
+#[apply(main!)]
+async fn main(executor: &Executor<'_>) -> AnyhowResult<()> {


Is there a strong reason to have this example be this manual and not use something like ureq (or any other client). I feel like most people looking to make a request against the server aren't going to be looking for "how to use http1::handshake::", etc, and the async isn't really necessary either.

I put the relevant diff up in a PR for consideration: mweatherley#2

benefits would be: smaller example to maintain, more directly applicable to likely end-user use case.

downside would be: adding a dev-dep on ureq (or another client)

pcwalton and others added 19 commits May 28, 2024 19:45

Make the BRP extensible

9885f1a

Merge remote-tracking branch 'origin/main' into brp

4c4dbcc

Allow REPARENT to remove objects from their parent

acc8f37

Reword the doc comment to not imply that HTTP and JSON are the only way

58d4a57

to implement the BRP

Merge remote-tracking branch 'origin/main' into brp

fb56bca

Make handlers systems

d86fe41

Move builtin verbs into their own file

d3806ef

Make the HashMap field in RemoteVerbs private

baff8ba

Merge remote-tracking branch 'origin/main' into brp

a2ca594

Migrate from Tokio to smol

2cdab11

Merge remote-tracking branch 'origin/main' into brp

2b2223b

Merge remote-tracking branch 'origin/main' into brp

a56805a

Comments, move from hardcoded IP to default

38b9979

Restructure RemotePlugin verbs

5ac7893

Let RemotePlugin::with_address accept an Into<IpAddress>

3f87074

Document helper functions

237fd78

Cosmetic changes to verb implementations

9b82fbc

Update code style and error formatting

1abcad1

alice-i-cecile mentioned this pull request Aug 22, 2024

Implement the Bevy Remote Protocol (BRP). #13563

Closed

NthTensor reviewed Aug 23, 2024

View reviewed changes

IQuick143 reviewed Aug 23, 2024

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

mweatherley and others added 2 commits August 23, 2024 08:39

Revert change to volumetric fog example in Cargo.toml

39d2f9c

jsonrpc, code quality

f4868b3

Merge PR #1 from mweatherley/bevy/brp

0239322

Move to JSON-RPC, structure errors, code quality

soqb self-assigned this Aug 27, 2024

soqb self-requested a review August 27, 2024 12:03

soqb removed their assignment Aug 27, 2024

mweatherley added 12 commits August 27, 2024 09:08

Fix batching, add some code comments

aed1967

Merge branch 'main' into brp

2d71527

Fix issues with Reflect -> PartialReflect from merge

90b9dec

Fix doc link

0aa327d

Only serialize data field of error if it's Some

ce8c03c

Top-level documentation of built-in and custom methods

e4402d4

Fix issue with bevy_ecs features

b9eaf5d

Fix typo

013eb76

Make option/has part of bevy/query method actually work; add a marker…

ad67e5c

… component to the server example

Taplo fmt

c77dac3

Trivial doc change

a8535db

Merge branch 'main' into brp

ac96b05

mweatherley marked this pull request as ready for review August 30, 2024 21:43

mweatherley added S-Needs-Review Needs reviewer attention (from anyone!) to move forward and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Aug 30, 2024

MiniaczQ reviewed Aug 30, 2024

View reviewed changes

ChristopherBiscardi reviewed Aug 31, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial implementation of the Bevy Remote Protocol (Adopted) #14880

Initial implementation of the Bevy Remote Protocol (Adopted) #14880

mweatherley commented Aug 22, 2024 •

edited

Loading

NthTensor left a comment •

edited

Loading

soqb commented Aug 25, 2024

MiniaczQ commented Aug 30, 2024

NthTensor commented Aug 30, 2024

MiniaczQ Aug 30, 2024

MiniaczQ Aug 30, 2024

MiniaczQ Aug 30, 2024

MiniaczQ Aug 30, 2024

NthTensor Aug 30, 2024

ChristopherBiscardi Aug 31, 2024 •

edited

Loading

		hyper = { version = "1", features = ["full"] }
		hyper-util = { version = "0.1", features = ["full"] }

		.add_systems(Startup, start_server)
		.add_systems(Update, process_remote_requests);

Initial implementation of the Bevy Remote Protocol (Adopted) #14880

Are you sure you want to change the base?

Initial implementation of the Bevy Remote Protocol (Adopted) #14880

Conversation

mweatherley commented Aug 22, 2024 • edited Loading

Objective

Solution

Message lifecycle

Testing

Future direction

NthTensor left a comment • edited Loading

Choose a reason for hiding this comment

soqb commented Aug 25, 2024

MiniaczQ commented Aug 30, 2024

NthTensor commented Aug 30, 2024

MiniaczQ Aug 30, 2024

Choose a reason for hiding this comment

MiniaczQ Aug 30, 2024

Choose a reason for hiding this comment

MiniaczQ Aug 30, 2024

Choose a reason for hiding this comment

MiniaczQ Aug 30, 2024

Choose a reason for hiding this comment

NthTensor Aug 30, 2024

Choose a reason for hiding this comment

ChristopherBiscardi Aug 31, 2024 • edited Loading

Choose a reason for hiding this comment

mweatherley commented Aug 22, 2024 •

edited

Loading

NthTensor left a comment •

edited

Loading

ChristopherBiscardi Aug 31, 2024 •

edited

Loading