Build Label Rasters From ROIs

This topic describes how to label features of interest with regions of interest (ROIs), for training mask-based (pixel segmentation) models.

With ENVI Deep Learning, many more ROIs are needed to identify samples of features, compared to traditional supervised classification methods that can produce good results from only a few ROIs. Although it is possible to get sufficient results from labeling only a few features, you will get much more accurate results by investing some time in this process.

See the following sections:

Label Training Rasters
Train the Deep Learning Model
Optional: Use the Build Label Raster From ROIs Tool
- Label Training Rasters
- Build Label Rasters

Label Training Rasters

This section describes how to create label rasters from one or more training rasters and ROIs that represent the features you are interested in. The easiest way to label your training rasters is to use the Deep Learning Labeling Tool. The steps used throughout this topic reflect the use of this tool. You can also use the Build Label Raster from ROI tool to create label rasters. This option is described at the end of this help topic.

The Deep Learning Labeling Tool automatically creates label rasters once you define a set of ROIs for each training raster. You do not have to take any extra steps to build the label rasters.

To get started, select Deep Learning > Deep Learning Labeling Tool in the ENVI Toolbox. You will use this tool to define your output classes and to manage project files, ROIs, and training rasters. You can also start the Deep Learning Labeling Tool from the Deep Learning Guide Map by clicking this button sequence in the first panel: Pixel Segmentation > Train a New Model > Train a Multiclass Model > Label Rasters.

Set Up a Deep Learning Project

Projects help you organize all of the files associated with the labeling process, including training rasters and ROIs. They also keep track of class names and the values you specify for training parameters.

Select File > New Project from the Labeling Tool menu bar. The Create New Labeling Project dialog appears.
From the Project Type drop-down list, select Pixel Segmentation. If you accessed the Labeling Tool from the Guide Map, this option is automatically selected and you cannot change it.
Enter a Project Name.
Click the Browse button next to Project Folder and select an empty folder in which to store project files.
Click OK.

When you create a new project and define your training rasters, ENVI creates subfolders for each training raster that you select as input. Each subfolder will contain the ROIs and label rasters created during the labeling process.

ENVI also creates a file named source_raster.json in each subfolder. This is a simplified version (also called a dehydrated form) of the training raster, where all of its information is condensed down to JSON code. If you move the project folder to a different location, the source_raster.json file in each project subfolder tells ENVI where to find the training raster. This way, you do not have to keep track of file locations yourself.

To restore a previously created project, select File > Open Project in the Labeling Tool menu bar. Navigate to your project folder and select the file deep_learning_labeling.json.

After setting up a project, the next step is to define your output classes.

Define Classes

Use the Class Definitions section of the Labeling Tool to define the features you will label. The feature names determine the class names in the output classification image.

Click the Add button in the lower-left corner of the Class Definitions section.
Enter a Name for the first feature that you will label, then click the button. The feature is added to the Class Definitions list.
Optional: Click in the Color column to change the color of the class.
If you will be extracting multiple features, repeat Steps 1-3 as needed for additional features (classes).
The following figure shows an example of four classes that represent different types of property damage to residential homes (from the ENVI Deep Learning Tutorial: Extract Multiple Features):

Next, you will select the rasters that you want to use for training.

Add Training Rasters

Click the Add button below the Rasters section of the Labeling Tool. The Data Selection dialog appears.
Select one or more training rasters and click OK.
- All training rasters must have the same number of bands. If one image has a different number of bands than the others, you can select that image in the Data Selection dialog and click the Spectral Subset button. Then select the appropriate number of bands.
- The training rasters are listed in the Rasters section.
- To remove all training rasters, click the Select All button , then the Remove Selected button .
Optional: Select a training raster and click the Rename Selected button to provide a name that is more meaningful than its filename. You can only rename one raster at a time.

Now you are ready to label the features in your training rasters by drawing ROIs in them.

Draw ROIs

For each training raster, the "ROI(s)" column shows the fraction of ROIs drawn to the total number of classes. Follow these steps to begin drawing ROIs:

Select a training raster from the list and click the Draw button . The training raster is displayed. ENVI creates a new set of ROIs with the same names and colors as the classes that you defined. The ROIs are listed in the Layer Manager and Data Manager. The Region of Interest (ROI) Tool appears, with the first ROI selected. The following image shows an example.
The cursor changes to a crosshair so that you can begin drawing ROI polygons.
The Labeling Tool synchronizes the names and colors of classes and their associated ROIs. If you change the name or color of a class, the ROI name or color will automatically update (and vice-versa). You can also add or remove ROIs as needed.
Note: Do not use the ROI Tool to change file locations of ROIs, to create or remove ROIs, or to change their colors. Instead, use the Labeling Tool to make these changes so that they can be applied to all of the training rasters.
Optional: Experiment with different contrast stretches using the Stretch Type drop-down list in the ENVI toolbar to ensure that the features of interest are clearly distinguishable from the background. Training images that are saturated, "washed out," or have low contrast will make the labeling process more difficult. The image below shows an example of applying a contrast stretch to an image of uprooted trees after a tornado. Trying to label trees in the left image would be too difficult.
Draw ROIs on as many examples of the feature as possible within the image. See Tips for Labeling Features.
To label a different training raster, select its name in the Labeling Tool and click the Draw button. The "ROI(s)" column in the previous training raster is updated to show the fraction of ROIs drawn to the total number of ROIs defined.
Continue labeling the image in this manner for each ROI/class. Your progress is automatically saved to the current project, even if you close the Labeling Tool.

Optional: Import Existing ROIs

You can import an existing ROI file into your project for one or more training rasters. The ROI names do not have to match the class names that you specify. Follow these steps:

Define classes, as described earlier.
Add training rasters, as described earlier.
In the Labeling Tool, select the training raster for which you want to import an ROI file. Then click the Options button and select Import ROIs.
Select an ROI file (.xml) and click Open. The Match Input ROIs to Class Definitions dialog appears.
Select an ROI name under the Input ROIs column, then select the class name (under the Class Definitions column) that the ROI represents. Or, do this vice-versa. A line connects the ROI with the associated class name.
Repeat with the remaining ROIs and classes, then click OK. The ROIs are imported into the project.

Optional: Import Vectors

You can import vector files into your project for one or more training rasters; for example, if you already have a shapefile of features that you want to use for labeling. Follow these steps:

Define classes, as described earlier.
Add training rasters, as described earlier.
In the Labeling Tool, select the training raster for which you want to import a vector file and click the Draw button to display the training raster.
In the Labeling Tool, click the Options button and select Import Vectors. The Import from Vector dialog appears.
Click the Browse button next to Input Vector and select a vector file. See the Supported Data Types topic in ENVI Help for a list of supported vector formats.
The Attribute Name drop-down list is populated with the attribute columns in the vector file. If it does not have any attributes, the only option will be <none>.
Select an attribute from the Attribute Name drop-down list. The Record Selection dialog appears.
- If you do not make a selection, then all records from the vector file will be imported into one class. In this case, click the class under "Class Definitions" that all records should be associated with. A line is drawn between "Records" and the selected class; for example:
- If you select an option from the Attribute Name drop-down list, the unique attribute values are listed on the left side of the Record Selection dialog; for example:
- Click an attribute name, then click the associated class name under "Class Definitions." A line is drawn between the two. Repeat this step for all of the attributes and classes; for example:
  If the selected attribute has more than 20 unique values, you can only choose one value to associate with a given class.
Click OK in the Record Selection dialog. The vector records are imported into the current project for the selected training raster.

View Project and Labeling Statistics

Click the Options drop-down list and select Show Labeling Statistics to view information about the current project and how much labeling was performed.

The beginning of the report in the Project Statistics dialog provides general information about the project, including:

Project name and location
Number of classes and their names
Number of bands in each training raster (this should be the same for all training rasters)
Number of training rasters

Following this are details about the ROIs used to label each training raster. The purpose of this section is to report how much labeling you have done in each training raster. You can quickly determine if some training rasters do not have enough ROIs or if others have a higher balance of ROIs. The following image shows an example of the ROI statistics for one training raster:

To save the project statistics to a text file, click the Save button in the Project Statistics dialog. Then select a location to save the text file. The default filename is report.txt.

To copy the project statistics to your system's clipboard, click the Copy button in the Project Statistics dialog.

Train the Pixel Segmentation Model

Once you have labeled all of your training rasters with ROIs, the next step is to train the deep learning model. The Labeling Tool provides a simplified way to train models. A full training scenario involves creating label rasters, initializing a TensorFlow model, and setting multiple training parameters. When you train a deep learning model using the Labeling Tool, the steps of creating label rasters and initializing a TensorFlow model are automatically done for you.

Follow these steps:

Click the Train button in the Labeling Tool. The Train Deep Learning Model dialog appears.
Select a Patch Size from the drop-down list provided. Patches are small images given to the model for training. The Patch Size is the number of pixels along one edge of a square patch. The default value is 572, which means the patch size is 572 x 572 pixels. Larger patch sizes result in faster classification, but for training, the patch size must be smaller than the size of the label rasters and also small enough that at least one patch per batch fits into graphics memory. In general, if your label rasters are smaller than the default patch size, choose the largest size that will fit in your rasters. If your graphics card has more than 8 GB of memory, you may want to increase the Patch Size.
Use the Training/Validation Split (%) slider to specify the percentage of data to use for training versus validation.
Enable the Shuffle Rasters option to shuffle the training and validation rasters before splitting them. This ensures that the training is not biased. Disabling this option means that the same images will be used for training and validation with each training run, which will help achieve more repeatable results.
In the Number of Epochs field, enter the number of epochs to run. Training parameters are adjusted at the end of each epoch. The default value is 25. See Epochs and Batches.
The Number of Patches Per Image parameter defines how often patches are seen during training. See Epochs and Batches for more information. The default value is 100, which is a good starting point. Setting it to larger values will increase the training time. A general recommendation for best training results is to increase the Number of Epochs value while keeping the Number of Patches Per Image relatively low.
Set the Augmentation Scale option to Yes to augment the training data with resized (scaled) versions of the data. See Data Augmentation.
Set the Augmentation Rotation option to Yes to augment the training data with rotated versions of the data. See Data Augmentation.
The Solid Distance field pertains to point and polyline labels only. For each class, enter the number of pixels surrounding point or polyline labels that should be considered part of the target feature. See the Solid Distance background discussion for more information. To use the same value for all classes, click the Set all to same value button . In the Set Value dialog, enter the value to use for all classes and click OK.
The Blur Distance field is used in conjunction with Solid Distance. You can optionally blur features of interest that vary in size. Blurring the edges of features and decreasing the blur during training can help the model gradually focus on the feature of interest. In most cases, you can leave this field blank; however, it is available for you to experiment with. See the Blur Distance background discussion for more information. To use the same value for all classes, click the Set all to same value button . In the Set Value dialog, enter the value to use for all classes and click OK.
In the Class Weight field, enter the minimum and maximum weights for having a more even balance of classes (including background) when sampling. Diversity of sampling is weighted by the maximum value at the beginning of training and decreased to the minimum value at the end of training. The useful range for the maximum value is between 1 and 6. A general recommendation is to set the Min to 2 and the Max to 3 when your features of interest are sparse in your training rasters. Otherwise, set them from 0 to 1. See Training Parameters.
In the Loss Weight field, enter a value between 0 and 1.0. A value of 0 is a good starting point and will be fine for most cases. A value of 0 means the model will treat feature and background pixels equally. Increased values will bias the loss function to place more emphasis on correctly identifying feature pixels than identifying background pixels. This is useful when features are sparse or if not all of the features are labeled.
Specify a filename (.h5) and location for the Output Model. This will be the "best" trained model, which is the model from the epoch with the lowest validation loss. By default, the tool will save the best and last model. Most of the time, the best model will perform the best compared to the last model, but not always. Having both outputs lets you choose which model works best for your scenario.
Specify a filename (.h5) and location for the Output Last Model. This will be the trained model from the last epoch.
Click OK. ENVI automatically creates label rasters for training and populates the "Label Raster" column with "OK" for each training raster. Training a model takes a significant amount of time due to the computations involved. Depending on your system and graphics hardware, processing can take several minutes to several hours. A Training Model dialog shows the progress of training, along with the updated validation loss value; for example:
At the same time, a TensorBoard page displays in a new web browser. TensorBoard is a visualization toolkit included with TensorFlow. It reports real-time metrics such as Loss, Accuracy, Precision, and Recall during training. See View Training Metrics for details.

When training is complete, you can pass the trained model to the TensorFlow Object Classification tool.

Optional: Use the Build Label Raster From ROI Tool

If you choose not to use the Deep Learning Labeling Tool to create label rasters, you will need to define and organize your training rasters and ROIs on your own without the benefit of project files that the labeling tool offers. Then use the Build Label Raster from ROI tool to create label rasters. Those steps are described here.

You can also write a script to build a label raster from ROIs using the BuildLabelRasterFromROI task.

Label Training Rasters

Prepare the training rasters.
Open a training raster in ENVI. Then right-click its name in the Layer Manager and select New Region of Interest. The Region of Interest (ROI) tool appears.
Change the ROI Name to the name of the first feature that you want to label; for example, Roads or Buildings. Press the Enter key to accept the name. ROI names will determine the class names in the output classification image.
Optionally change the ROI color.
Draw polygon, polyline, or point ROIs over the features of interest in your training raster. See Tips for Labeling Features.
If you are extracting multiple features, click the Add ROI button to create an ROI for the next feature. Then repeat Steps 3-5 for that feature.
When you are finished drawing ROIs for the training raster, select File > Save As from the ROI Tool menu bar. Select a filename and location for the ROI file (.xml) and click OK. If you have multiple training rasters, you must create separate ROI files for each raster and label each raster separately.