Minimum Noise Fraction Transform

Use MNF Rotation transforms to determine the inherent dimensionality of image data, to segregate noise in the data, and to reduce the computational requirements for subsequent processing (Boardman and Kruse, 1994).

See the following sections:

Background

The MNF transform as modified from Green et al. (1988) and implemented in ENVI, is a linear transformation that consists of the following separate principal components analysis rotations:

You can also use the MNF transform to remove noise from data by performing a forward transform, determining which bands contain the coherent images (by examining the images and eigenvalues), and running an inverse transform using a spectral subset to include only the good bands, or smoothing the noisy bands before the inverse.

You have the option to subset the output of the MNF rotation based on eigenvalues, which means (with hyperspectral data) that you do not have to generate an output cube of hundreds of bands (floating-point) when you only need tens of bands.

ENVI also has the option of applying an MNF transform to individual spectra. This transforms endmember spectra into MNF space for input into mixture tuned matched filtering (MTMF).

Note: You can use the Spectral Hourglass Workflow to guide you step-by-step through the ENVI hourglass processing flow, including MNF transforms, to find and map image spectral endmembers from hyperspectral or multispectral data.

References

Green, A. A., Berman, M., Switzer, P., and Craig, M. D., 1988, A transformation for ordering multispectral data in terms of image quality with implications for noise removal: IEEE Transactions on Geoscience and Remote Sensing, v. 26, no. 1, p. 65-74.

Boardman, J. W., and Kruse, F. A., 1994, Automated spectral analysis: a geological example using AVIRIS data, north Grapevine Mountains, Nevada: in Proceedings, ERIM Tenth Thematic Conference on Geologic Remote Sensing, Environmental Research Institute of Michigan, Ann Arbor, MI, pp. I-407 - I-418.

Statistics Files

Unlike principal components analysis, the forward MNF transform produces two separate statistics files: MNF noise statistics and MNF statistics. While these files appear to be ordinary ENVI statistics files, they contain information unique to MNF and omit data typically found in ENVI statistics files.

During a forward MNF rotation, ENVI computes the following statistics:

The first rotation stores its complete set of covariance statistics in the MNF noise statistics file. However, this file contains only the noise covariance statistics, not the data normally found in ENVI statistics files.

The second rotation stores its eigenvector matrix and eigenvalues in the MNF statistics file. However, the rest of the covariance statistics for the second rotation are not saved because the covariance placeholder in the MNF statistics file is used to store a special “composite” MNF transformation matrix. This matrix describes the net result of both principal components rotations as well as the band-independent scaling introduced by the noise normalization. This non-orthogonal, non-unit length, matrix allows an inverse MNF rotation to be applied in a single step.

The following table summarizes the contents of the statistics files generated by the MNF transform:

Statistic

MNF Noise Statistics File

MNF Statistics File

Mean

<null>

Of the original input bands

Maximum

<null>

<null>

Minimum

<null>

<null>

Standard Deviation

<null>

<null>

Eigenvalues

Of the noise covariance matrix

Of the second principal components rotation

Covariance matrix

Of the noise

Composite transformation matrix

Correlation matrix

Of the noise

Incorrect values. Do not use.

Eigenvectors

Of the noise covariance matrix

Of the second principal components rotation

Apply Forward MNFs to Spectra

Use Apply Forward MNF to Spectra to transform endmember spectra into MNF space for use in mixture tuned matched filtering. Use the MNF statistics from the mixture tuned matched filtering input data file for the transform. You can collect the spectra to transform from plot windows, spectral libraries, ASCII files, ROIs, and statistics files using a dialog similar to the endmember collection dialog.

  1. From the Toolbox, select Transform > MNF Rotation > Apply Forward MNF to Spectra. The Forward MNF Statistics Filename dialog appears.
  2. Select the MNF statistics filename that was calculated for the mixture tuned matched filtering input data file, then click Open. The Forward MNF Convert Spectra dialog appears.
  3. Import spectra into the dialog by selecting Import > file_type from the dialog's menu bar, or by dragging and dropping the spectra from the Spectral Library Plots window to the Forward MNF Convert Spectra dialog.

    4. If you are transforming library spectra into forward MNF space to use as endmembers in mixture tuned matched filtering, the input library spectra must be scaled identically to the image associated with the MNF statistics being used to do the forward transformation.

  4. To delete a spectrum from the dialog, select the spectrum and click Delete Spectrum.
  5. Click Apply. The transformed spectra appear in the Forward MNF Spectra plot window and you can input the spectra into the Mixture Tuned Matched Filtering Endmember Collection dialog.

Apply Inverse MNFs to Spectra

Use the Apply Inverse MNF to Spectra to transform MNF spectra back to the original data space. ENVI uses forward MNF statistics from a data file for the transform. You can collect the spectra to transform from plot windows, spectral libraries, ASCII files, ROIs, and statistics files using a dialog similar to the endmember collection dialog.

  1. From the Toolbox, select Transform > MNF Rotation > Apply Inverse MNF to Spectra. The Forward MNF Statistics Filename dialog appears.
  2. Select the forward MNF statistics filename, then click Open. The Inverse MNF Convert Spectra dialog appears.
  3. Import spectra into the dialog by selecting Import > file_type from the dialog's menu bar, or by dragging and dropping the spectra from the Spectral Library Plots window to the Forward MNF Convert Spectra dialog.
  4. Click Apply. The transformed spectra appear in the Inverse MNF Spectra plot window.

Forward MNF Transform

Use Forward MNF transforms to estimate the noise statistics used in the first rotation (including estimating the noise from the input data), using noise statistics that were calculated previously, or calculating the noise statistics from a dark image associated with the dataset.

You can also write a script to perform forward MNF transforms using the ForwardMNFTransform task.

Use Forward MNF Transform to estimate noise when you do not have a dark current image for your data, which is usually the case. ENVI assumes that each pixel contains both signal and noise, and that adjacent pixels contain the same signal but different noise. A shift difference can be performed on the data by differencing adjacent pixels to the right and above each pixel and averaging the results to obtain the “noise” value to assign to the pixel being processed. The best noise estimate is gathered using the shift-difference statistics from a homogeneous area rather than from the whole image. ENVI allows you to select the subset for statistics extraction.

  1. From the Toolbox, select Transform > MNF Rotation > Forward MNF Transform. The Forward Minimum Noise Fraction dialog appears.
  2. From the Input Raster field, select a raster file and perform optional spatial and spectral subsetting, then click OK.
  3. Use the Shift Difference Spatial Subset fields, to specify a spatial subset or region of interest (ROI) on which to calculate the shift difference. You can then apply the calculated results to the entire file, or to the file subset if you selected one when you selected the input file.

    4. When estimating noise from image data using the shift differencing approach, try to choose a spatial subset that is spectrally uniform. Also, make sure the subset does not contain any background or "NoData" pixels.

    If any band in your dataset has zero variance, you may encounter a “singularity” problem where the covariance matrix cannot be inverted, and you may receive a “Too May Iterations in TQLI” error message.

    For example, if you use a deep, placid lake for your noise subset, most bands will have very little variation in the subset, which is what you want. However, if one band has the same pixel value throughout the entire subset, the band has zero variance and will cause a singularity in the covariance matrix.

    You can find bands that contain zero variance, large outliers, or statistics that are identical to adjacent bands, by generating and reviewing a statistics report on the same shift-difference subset of your input image that you plan to choose for an MNF transform. If you determine that any band has zero variance, you should choose a slightly different or larger subset for your noise estimate. The shift-difference spatial subset must contain more pixels than you have bands in the image you are transforming.

  4. Specify the Output Number of Bands if you do not wish to output all bands.
  5. In the Input Noise Statistics File field, enter a .sta file from previous a Forward MNF to use in this MNF transform.
  6. In the Output Noise Statistics File field, enter an output filename for the noise statistics.
  7. In the Output MNF Statistics File field, enter an output filename for the MNF statistics. The files for the MNF statistics and noise statistics must have different filenames. This file can be used as input for an Inverse MNF Transform.
  8. Enter a filename and location for the Output Raster.
  9. Enable the Preview check box to see a preview of the settings before you click OK to process the data. The preview is calculated only on the area in the view and uses the resolution level at which you are viewing the image. See Preview for details on the results. To preview a different area in your image, pan and zoom to the area of interest and re-enable the Preview option.
  10. Enable the Display result check box to display the output in the view when processing is complete. Otherwise, if the check box is disabled, the result can be loaded from the Data Manager.

  11. To reuse these task settings in future ENVI sessions, save them to a file. Click the down arrow and select Save Parameter Values, then specify the location and filename to save to. Note that some parameter types, such as rasters, vectors, and ROIs, will not be saved with the file. To apply the saved task settings, click the down arrow and select Restore Parameter Values, then select the file where you previously stored your settings.

  12. To run the process in the background, click the down arrow and select Run Task in the Background. If an ENVI Server has been set up on the network, the Run Task on remote ENVI Server name is also available. The ENVI Server Job Console will show the progress of the job and will provide a link to display the result when processing is complete. See ENVI Servers for more information.

  13. Click OK.

Forward MNF Noise Statistics from Dark Data

Use Forward MNF Noise Statistics from Dark Data to estimate noise statistics from a dark image. Some instruments collect a dark image as part of data calibration or collection. The dark image is the equivalent of a photograph with the lens cap on. In electronic instruments, there is still a signal that is actually the noise of the instrument. You can use the dark image to characterize the instrument noise.

  1. From the Toolbox, select Transform > MNF Rotation > Forward MNF Noise Statistics from Dark Data. The MNF Transform Input File dialog appears.
  2. Select an input file and optional spatial and spectral subsetting, and/or masking, then click OK. The Dark Current Input File dialog appears.
  3. Select the dark current file and perform spatial subsetting as required, then click OK. ENVI calculates the noise statistics using the dark current input file. The Forward MNF Transform Parameters dialog appears.
  4. In the Enter Output Noise Stats Filename [.sta] field, enter an output file for the noise statistics.
  5. In the Enter Output MNF Stats Filename [.sta] field, enter an output file for the MNF statistics. Be sure that the MNF and noise statistics files have different names.
  6. Select output to File or Memory.
  7. Select the number of output MNF bands by using one of the following options:
    • To select the number of output bands without examining the eigenvalues, select No from the Select Subset from Eigenvalues toggle button, then set the Number of Output MNF Bands.
    • To select the number of output MNF bands by examining the eigenvalues, use the following steps:
      1. Select Yes from the Select Subset from Eigenvalues toggle button.
      2. Click OK. ENVI calculates the noise statistics and the first rotation. The Select Output MNF Bands dialog appears, with each band listed with its corresponding eigenvalue. Also listed is the cumulative percentage of data variance contained in each MNF band for all bands.
      3. Set the Number of Output MNF Bands. For the best results, and to save disk space, output only those bands with high eigenvalues. Images with eigenvalues close to 1 are mostly noise.
  8. Click OK.

Inverse MNF Transform

Use Inverse MNF Transform to transform the bands from a previous Forward Minimum Noise Fraction to their original data space.

Note: Smooth noise-dominated images (higher band numbers) before performing the inverse transform or eliminate the noisy bands during the inverse transform using spectral subsetting. Smoothing or eliminating noisy bands reduces the noise in the original data space.

You can write a script to perform an inverse MNF transform using the InverseMNFTransform task.

  1. From the Toolbox, select Transform > MNF Rotation > Inverse MNF Transform. The Inverse Minimum Noise Fraction Transform dialog appears.
  2. From the Input Raster field, select a previous Forward Minimum Noise Fraction raster file and perform optional spatial subsetting, then click OK.
  3. Enter the name of the previous Forward Minimum Noise Fraction statistics file in the MNF Statistics File field.
  4. In the Number of Components field, specify the number of forward MNF components to use for the inverse transform. Leave this field blank to use all components.
  5. Enter a filename and location for the Output Raster.
  6. Enable the Preview check box to see a preview of the settings before you click OK to process the data. The preview is calculated only on the area in the view and uses the resolution level at which you are viewing the image. See Preview for details on the results. To preview a different area in your image, pan and zoom to the area of interest and re-enable the Preview option.
  7. Enable the Display result check box to display the output in the view when processing is complete. Otherwise, if the check box is disabled, the result can be loaded from the Data Manager.

  8. To reuse these task settings in future ENVI sessions, save them to a file. Click the down arrow and select Save Parameter Values, then specify the location and filename to save to. Note that some parameter types, such as rasters, vectors, and ROIs, will not be saved with the file. To apply the saved task settings, click the down arrow and select Restore Parameter Values, then select the file where you previously stored your settings.

  9. To run the process in the background, click the down arrow and select Run Task in the Background. If an ENVI Server has been set up on the network, the Run Task on remote ENVI Server name is also available. The ENVI Server Job Console will show the progress of the job and will provide a link to display the result when processing is complete. See ENVI Servers for more information.

  10. Click OK.