Dashboards in WanDB

A. The Importance and Benefits of Dashboards

Dashboards in WanDB provide a centralized, visual representation of your machine learning model’s performance. It allows for real-time tracking of multiple aspects of a run, such as loss, accuracy, and other custom metrics.

The benefits of using dashboards include:

  1. Collaboration: Dashboards allow team members to share their results and insights, fostering a collaborative environment.

  2. Visualization: They provide clear visualizations of complex data, which can reveal patterns and insights that might be hard to understand in raw, numeric format.

  3. Real-time Monitoring: Dashboards are updated in real-time, enabling you to immediately identify and rectify any issues during the training process.

B. Your Dashboard

When we create a project, WanDB will automatically create a dashboard for it, you can find it using the left navigation bar icon called Workspace.

You can view it here: WanDB Dashboard

Here’s a screenshot:

Runs

The area between the navigation bar on the left and the Sections on the right is called Runs. This area shows all the completed working run on the model.

Sections

The dashboard is divided into sections. In the screenshot we can see 3 sections: 1. Charts - Display charts related to the model. 2. System - Display charts related to the hardware used to run the model. 3. Hidden Panel - A special section to hide the charts that you find important but don’t want to clutter the other sections.

If we want to add a new section, we can click on the Add Section button at the bottom.

Panels

Inside each sections we have panels. If we look at the Charts section, there are 3 panels or charts inside. Each panel can be customized and edited. If we want to add a new panel, there is an Add Panel button at the top right of the section.

C. Customizing Your Dashboard

You can customize your dashboard to showcase the data that are most relevant to you:

  1. Adding Visualizations: Click on the Add Panel icon in a section to add a visualization. You can choose from a variety of chart types, such as line plots, scatter plots, bar charts, and more.

  2. Rearranging Sections: To rearrange sections in your dashboard, simply drag and drop them into the desired order.

  3. Editing Section Titles: Click on the section title to edit it.

D. Tracking Important Project Information

Tracking important project information on your dashboard is crucial for evaluation and decision-making:

  • Model Performance Metrics: Track key metrics such as accuracy, precision, recall, F1 score, and ROC AUC, among others, depending on your specific use case.

  • System Metrics: You can also track system metrics like GPU usage, CPU usage, memory usage, and more to understand the computational efficiency of your model.

  • Hyperparameters: Track the hyperparameters used in each run to understand their impact on the model’s performance.

  • Loss Graphs: Visualize how your model’s loss changes over time to identify if and when your model is converging.

Remember, what you track will depend on your project’s specific goals and requirements.

Monitoring Models with WanDB

A. Uploading a Model for Monitoring

Once you’ve trained your model, you can upload it to WanDB for monitoring:

  1. Save your model in a format that is convenient for you (e.g., .h5 for Keras models, .pt for PyTorch models).
  2. Use the wandb.save function to save your model:
wandb.save("path_to_your_model")

This creates a new artifact with the contents of your file and links it to your run.

If you run your model inside a Jupyter notebook and did not use wandb.save then WanDB will create an artifact pointing to your Jupyter notebook.

B. Understanding the Metrics and Logs

WanDB allows you to log various types of metrics:

  • System Metrics: These metrics include CPU usage, memory usage, GPU temperature, etc. They can be helpful to monitor the computational efficiency of your model.

  • Model Metrics: These are metrics that indicate the performance of your model, such as loss, accuracy, precision, recall, etc.

  • Custom Metrics: You can also create and log custom metrics based on your specific needs.

To log a metric, use the wandb.log function:

wandb.log({"metric_name": metric_value})

C. Interpreting Graphs and Charts

WanDB provides a variety of graphs and charts to visualize your logged data:

  • Line Plots: These are used primarily to track changes in metrics over time, like loss or accuracy during the training process.

  • Histograms: These can be used to visualize the distribution of a metric or parameter.

  • Scatter Plots: These can be used to explore the relationship between two variables.

  • Bar Charts: These can be useful to compare the performance of different models or runs.

You can mouse over any point on a graph or chart to see the exact values, and you can click on any point to go to the corresponding step in the run.

D. Comparing Different Models

WanDB makes it easy to compare different models or runs:

  1. Go to the project page on the Wandb web application.
  2. Use the checkboxes next to each run to select the runs you want to compare.
  3. Click on the ‘Compare’ button to see a side-by-side comparison of the selected runs.

In the comparison view, you can see a variety of information, including the values of different metrics, the system and model parameters, and more. This feature makes it easy to identify the best-performing model and understand why it performs well.

Let’s view a few examples of WanDB usage:

  • CIFAR10 Model: https://colab.research.google.com/drive/1-aV6DZJwHnN24rQFZwl2i3pSqO9RL6Xj

  • CIFAR10 WanDB: https://wandb.ai/ayamerushia/cifar10-cnn-wandb/overview

  • Text Classification on Huggingface Model : https://colab.research.google.com/drive/1wwZp46jhMubG-qgp2GGOf-A-IhZ3J1RO

  • Text Classification on Huggingface WanDB: https://wandb.ai/ayamerushia/indo-roberta-small-finetune-sentiment-analysis/overview?workspace=user-ayamerushia

Back to top