Rerun

Rerun 0.17: defaults and overrides, streaming in notebooks, and better website embeddings The release brings a huge increase in explicit control over your visualizations. You can now use blueprints to both set default component values on a view, and to override components on specific entities in the view. You can do this both from code and in the UI. In the video, we use a default to bulk edit the size of all camera frustums and an override to edit just the front-facing frustum size. With the introduction of blueprint overrides and defaults, Rerun 0.17 gives you direct control over what visualizers are applied to what entities and makes all of this easy to inspect in the UI. Together, these features increase the amount of flexibility and control you have over exactly how your data is visualized in Rerun. Beyond defaults and overrides, this release also comes with: 🔸 Improved notebook and website embedding support 👉 You can now stream data from the notebook cell to the embedded viewer. 👉 There is improved support for having multiple viewers on the same web page. 👉 And you have more more configuration options to control the visibility of the Menu bar, time controls, etc. 🔸 Additional configurability from code, for example: 👉 ImagePlaneDistance(size of the Pinhole frustum visualization) 👉 AxisLength(axis length of the transform visualization) 👉 and all settings on TensorViews 🔸 New examples: 👉 PaddleOCR 👉 Vista, a generative driving world model 👉 Stereo Vision SLAM And much more 🚀🚀🚀 Check out the blog post on overrides and defaults, and the full change log in the links in the comments 👇

nuScenes in Rerun! 🚗🚙🛻 Fridays are for classics. Check out our example showing how to visualize the #nuScenes dataset by Motional in Rerun. The scenes in this dataset encompass data collected from a comprehensive suite of sensors on autonomous vehicles. These include 6 cameras, 1 LIDAR, 5 RADAR, GPS and IMU sensors. Consequently, the dataset provides information about the vehicle's pose, the images captured, the recorded sensor data and the results of object detection at any given moment. Link to the code and the example in the browser in the comments 👇

Streaming visualizations in a notebook ✨ With Rerun's improved notebook integration comes the ability to add to an existing recording within a notebook. This makes it possible to e.g. interactively try different learning rates, losses, and batch sizes, while continuously visualizing the training progress interactively. In this example notebook, we demonstrate this when fitting a simple neural field to a 2D image. The neural field is a simple multilayer perceptron with optional positional input encoding. The image is sampled uniformly, and the network is trained to predict the color given the pixel position. To visualize the progress of the training, we log the loss and regularly densely query the network to retrieve the image encoded in the network weights. Check out the notebook example, a Colab, and our how-to guide in the comments below 👇

Train a model on the #pusht dataset and visualize it in Rerun! 🦾 #LeRobot is a project by Hugging Face 🤗 that aims to provide models, datasets, and tools for real-world robotics in PyTorch. This example shows how to train a model on the pusht dataset and visualize its progress in rerun. Find the full example and a link to the data set in the comments 👇! Remi Cadene

📊 Image Histograms with Kornia in #rust -- and Rerun viz :) 💡An image histogram is a graphical representation of the number of pixels in an image as a function of their intensity, crucial for various image processing tasks because they provide insights into the image's contrast, brightness, and dynamic range. 🔥Checkout the example here: https://lnkd.in/dBBtE-bx What color is gonna win ? 🐉 ⚔️ #kornia #computerVision #rust #opensource #artificialintelligence #houseofdragons

Web embeddings are key to unifying your visualization stack 💡 Our goal at Rerun is to build a visualization tool kit that allows companies to unify their visualization stack. From early prototyping, to running things in production, to training jobs: you should be able to use the same tools. A key unlock for making that happen is the ability to embed visualizations everywhere you might want them. Prototyping in a notebook? Building a custom dashboard? Expose visualizations in your own product? All of that you can do with Rerun. The video is showing embedded Rerun in our docs. With the latest Rerun 0.17 we have made embedded Rerun, and especially Rerun in a notebook much better. Check out an embedded Rerun viewer in our docs, the How-to, and the full 0.17 changelog in the links in the comments 👇

Stereo Slam on Kitti dataset 📷 🚘 📷 This example shows Farhad Dalirani's stereo visual SLAM implementation. Its input is video footage from a stereo camera, and it produces the trajectory of the vehicle and a point cloud of the surrounding environment. You can find the code on the link in the comments 👇. It’s using the latest Rerun 0.17 release.

Data vs styling should be a user decision 💡 A core design tenet we've had at Rerun for a long time is that what should be considered "data" and what should be considered "styling" has to be a user decision. Take something as simple as the color of a detected point. Your detection algorithm might estimate the true color of the detected point, in which case color is obviously data. You may also want to color all detected points blue and all tracked points green to make them easy to visually distinguish, in which case color is clearly styling. This idea has led us to build a system that doesn't distinguish between "data" and "styling". Instead, visualizers have inputs (e.g., positions, colors, labels, radii) which are sourced from either data recordings or blueprints. Any loggable data can also be put in the blueprint, which gives you maximum control to model things in the way that best fits your application. Fundamentally, we believe that this is the only viable approach to unify the visualization stack for robotics and embodied AI. Check out the blog post on overrides and defaults and the full changelog in the links in the comments 👇

✨ A formalized system for sourcing data to visualization ✨ The latest Rerun release 0.17 introduces defaults and overrides. The ability to set both default and override components in the blueprint is part of a new formalized system of how data gets sourced for visualizations. The goal is to make it easier for users to understand how each visualization gets generated precisely and to give full control over that process. For each entity that is visualized in the viewer, you can now see and edit the list of visualizers that run on it, what components they expect, and how each visualizer is sourcing the data for its inputs. For any input component to a visualizer, the final value is resolved in the following order: 1️⃣ Override: the per-entity override (the highest priority) 2️⃣ Store: the value that was logged to the data store (e.g., with the rr.log() API) 3️⃣ Default: the default value for this component type 4️⃣ Fallback: a context-specific fallback value that may depend on the specific visualizer and view type (the lowest priority) Check out the blog post on overrides and defaults and the full changelog in the links in the comments 👇

Rerun 0.17 is out! 🚀 It makes it easier to embed Rerun views in your web-apps, upgrades the notebook experience, and massively expands your control over visualizations from blueprint, Rerun’s language for describing multimodal visualizations from code or UI interactions. The ability to set defaults or override any data from the blueprint, together with the notion that all possible inputs to visualizations are data, is super powerful and a key piece of the multimodal visualization system that we’ve been designing and building for a long time. The video shows overriding the opacity of a segmentation image to hide or reveal the color image below it. We believe teams building robotics and embodied AI products should be able to unify their visualization and data debugging stack and have been going to extreme lengths over the last two years to make that possible. That includes everything from a custom code generation framework, our own in-memory multimodal time series database, a cross-platform renderer, and a new kind of system for describing visualizations and feeding them with streaming data. We have a lot more coming in the next months, but in the meantime, we are excited to hear what everyone in the community does with this last release! Check out the blog post on overrides and defaults, and the full changelog in the links in the comments 👇