This tutorial will help you get started with the nuScenes Package in YonoArc. nuScenes Dataset is a large-scale dataset that enriches with various sensory data, from cameras, Lidar, Radar, IMU, and GPS, in 1000 scenes at different locations. It facilitates the development of new algorithms in the field of autonomous driving.
The tutorial goes through the following:
- Purchasing the nuScenes Dataset as well as the nuScenes Package, in the form of YonoArc blocks, from YonoStore.
- Create a simple YonoArc pipeline to start interacting with the basic components of the package, especially the Dataset Player.
Check this to learn more about YonoStore.
Purchasing Assets from YonoStore
To purchase an asset from YonoStore:
- Navigate to YonoStore from the main view of Yonohub.
- Log in with your Yonohub account.
- Type in the search bar “nuScenes” and then press Enter.
- Choose the nuScenes Dataset v1.0 Full from the list of search results.
- Choose Buyout from the Usage License drop-down menu. Click on Add to Cart.
- After adding the dataset to Cart, it is essential to purchase the corresponding YonoArc blocks to interface with the dataset.
- Back to the search bar results, search for the following nuScenes YonoArc blocks,
- Dataset Player – nuScenes
- Sample Annotations to Eval Boxes – nuScenes
- Eval Boxes Preprocessing – nuScenes
- Predictions Appender – nuScenes
- Predictions Loader – nuScenes
- Tracking Benchmark – nuScenes
- Boxes Frame Transformer – nuScenes
- Lidar Point Cloud Converter – nuScenes
- Draw 3D Boxes – nuScenes
- Boxes to Markers – nuScenes
- Also, you can search for the Rviz YonoArc block which we will use for the 3D visualization.
- NOTE: These are almost all the blocks that will be used during the upcoming tutorials.
- For all of the above, choose Pay As You Go from the Usage License drop-down menu and then click on Add to Cart.
- Go to your cart by clicking on My cart button in the upper right corner of the page.
- Make sure you have all you need. Finally, click on Proceed to checkout button.
- You will be asked for billing information before placing your order.
- After successfully placing your order, you can find your purchased items nuScenes Dataset in YonoStoreDataset Folder, while the nuScenes blocks in your YonoArc blocks.
Next, you have to read the description of all the purchased YonoArc blocks as well as its ports and properties to be ready for the pipeline construction in the next section. You can follow the “Hello YonoArc” tutorial to get familiar with the YonoArc interface.
Create your First Pipeline with the nuScenes Package
In this section, you will be able to construct your first YonoArc pipeline with the nuScenes Package. The aim is to get familiar with the basic component which is the Dataset Player block.
- Click on the YonoArc icon from Yonohub’s Main View.
- Click the + button in the left upper corner, then click the Input section and choose the Dataset Player – nuScenes.
- Click the Dashboard section and choose the Video Viewer block.
- Configure each block by clicking the Settings icon in the upper left corner of the block.
- Dataset Player – nuScenes
- First, you need to insert the path of the dataset. Under the Properties tab, you can browse to the path of the nuScenes Dataset in the Dataset Directory property. Click on Browse -> YonoStoreDatasets -> nuScenesDataset-v1.0-Full. You find three different dataset folders, we deal with the v1.0-mini dataset version. Thus, select v1.0-mini.
- Second, select the dataset version from the Dataset Version property. Click on the drop list, and choose the v1.0-mini version. We will work on different versions of the dataset in the upcoming tutorials.
- The nuScenes dataset contains different raw data collected from different types of sensors. The Dataset Player block gives you the freedom to stream the sensory data of a specific type(s). It is recommended to choose the sensor(s) you work on to increase the maximum publishing rate you can achieve. For the sack of this tutorial, we check the Front Camera Output.
- NOTE: the sensor output means it contains the raw data (images, point cloud), transforms, and intrinsic matrices (for camera sensors).
- Then, choose the appropriate value of the publishing rate you want to stream the data with. Insert the value of 5 in the Publishing Rate property.
- In this Dataset Player, you have two publishing modes. Continuous Mode streams the data continuously with only the ability to pause/reset the streaming through the corresponding buttons or through the control signal port. On the other hand, Step Mode gives you full control of the streaming process. We will discuss this mode further in the upcoming tutorials. Select the Continous option from the Publishing Mode drop list property.
- Video Viewer
- Rename the Title as you like. We rename it to “Front Camera Image“.
- You can change the Quality of the streaming as you want.
- Dataset Player – nuScenes
- Connect the two blocks as shown below. You can connect several blocks by select them and click “Ctrl + E“.
- Launch the pipeline and wait a while till you get all the blocks running.
- Wait for the INFO alert with a description “Dataset has been loaded“, then click the Play button in the Dataset Player Settings to start streaming the data.
- Open the dashboard by clicking on the Dashboard button in the bottom left corner.
- Follow the transition between the scenes through the INFO alert, for example, “Scene: 1/10” which means that we are streaming the first scene out of 10 scenes.
- NOTE: Click on the Help tap in the Settings of any block to understand the functionality of each port/property.
You can follow the below visual tutorial to see the corresponding output of the above sequence of steps.