ucar.nc2.dataset
Interface EnhanceScaleMissing

All Known Implementing Classes:
CoordinateAxis, CoordinateAxis1D, CoordinateAxis1DTime, CoordinateAxis2D, VariableDS

public interface EnhanceScaleMissing

A Variable decorator that handles missing data, and scale/offset packed data. Specifically, this handles:

if those "standard attributes" are present.

Standard Use

Implementation rules for missing data. Here "missing data" is a general name for invalid/never written/missing values. Use this interface when you dont need to distinguish these variants. See below for a lower-level interface if you do need to distinguish between them.

  1. By default, hasMissing() is true if any of hasInvalidData(), hasFillData() or hasMissingValue() are true (see below). You can modify this behavior in the constuctor or by calling setInvalidDataIsMissing(), setFillDataIsMissing(), or setMissingValueIsMissing().
  2. Test specific values through isMissing(double val). Note that the data is converted and compared as a double.
  3. Data values of float or double NaN are considered missing data and will return true if called with isMissing(). (However hasMissing() will not detect if you are setting NaNs yourself).
  4. if you do not need to distinguish between _FillValue, missing_value and invalid, then set useNaNs = true in the constructor. When the Variable element type is float or double (or is set to double because its packed), then this sets isMissing() data values to NaNs, which makes further comparisions more efficient.

Implementation rules for scale/offset (no missing data)

  1. If scale_factor and/or add_offset variable attributes are present, then this is a "packed" Variable.
  2. The Variable element type is converted in the following way:
  3. external (packed) data is converted to internal (unpacked) data transparently during the read() call.

Implementation rules for missing data with scale/offset

  1. Always converted to a float or double type.
  2. _FillValue and missing_value values are always in the units of the external (packed) data.
  3. If valid_range is the same type as scale_factor (actually the wider of scale_factor and add_offset) and this is wider than the external data, then it will be interpreted as being in the units of the internal (unpacked) data. Otherwise it is in the units of the external (packed) data.
  4. The dataType is set to float if all attributes used are float (scale_factor, add_offset valid_min, valid_max, valid_range, missing_data and _FillValue) otherwise the dataType is set to double.

Low Level Access

The following provide more direct access to missing/invalid data. These are mostly convenience routines for checking the standard attributes. If you set useNaNs = true in the constructor, these routines cannot be used when the data has type float or double.

Implementation rules for valid_range

  1. if valid_range is present, valid_min and valid_max attributes are ignored. Otherwise, the valid_min and/or valid_max is used to construct a valid range. If any of these exist, hasInvalidData() is true.
  2. To test a specific value, call isInvalidData(). Note that the data is converted and compared as a double. Or get the range through getValidMin() and getValidMax().

Implementation rules for _FillData

  1. if the _FillData attribute is present, it must have a scalar value of the same type as the data. In this case, hasFillData() returns true.
  2. Test specific values through isFillValue(). Note that the data is converted and compared as a double.

Implementation rules for missing_value

  1. if the missing_value attribute is present, it must have a scalar or vector value of the same type as the data. In this case, hasMissingValue() returns true.
  2. Test specific values through isMissingValue(). Note that the data is converted and compared as a double.

Strategies for using EnhanceScaleMissing

  1. Low-level: use the is/has InvalidData/FillData/missingValue routines to "roll-your own" tests for the various kinds of missing/invalid data.
  2. Standard: use is/hasMissing() to test for missing data when you dont need to distinguish between the variants. Use the setXXXisMissing() to customize the behavior if needed.
  3. Efficient : If you expect to scan more than once for missing values, and you are not distinguishing between InvalidData/FillData/missingValue, then set useNaNs in the constructor. This sets isMissing() data values to NaNs when reading, and subsequent checks are more efficient.

Author:
caron

Method Summary
 Array convertScaleOffsetMissing(Array data)
          Convert data if hasScaleOffset, using scale and offset.
 double convertScaleOffsetMissing(byte value)
          Convert this byte value to a double, using scale/offset/missing value if applicable
 double convertScaleOffsetMissing(double value)
          Convert this double value using scale/offset/missing value if applicable
 double convertScaleOffsetMissing(int value)
          Convert this int value to a double, using scale/offset/missing value if applicable
 double convertScaleOffsetMissing(long value)
          Convert this long value to a double, using scale/offset/missing value if applicable
 double convertScaleOffsetMissing(short value)
          Convert this short value to a double, using scale/offset/missing value if applicable
 boolean getUseNaNs()
          get whether to use NaNs for missing float/double values (for efficiency)
 double getValidMax()
          return the maximum value in the valid range
 double getValidMin()
          return the minimum value in the valid range
 boolean hasFillValue()
          true if Variable has _FillValue attribute
 boolean hasInvalidData()
          true if Variable has valid_range, valid_min or valid_max attributes
 boolean hasMissing()
          true if Variable has missing data values
 boolean hasMissingValue()
          true if Variable has missing_value attribute
 boolean hasScaleOffset()
          true if Variable data will be converted using scale and offet
 boolean isFillValue(double val)
          return true if val equals the _FillValue
 boolean isInvalidData(double val)
          return true if val is outside the valid range
 boolean isMissing(double val)
          true if val is a missing data value.
 boolean isMissingFast(double val)
          true if val is a missing data value.
 boolean isMissingValue(double val)
          return true if val equals a missing_value
 void setFillValueIsMissing(boolean b)
          set if _FillValue is considered isMissing(); better set in constructor if possible
 void setInvalidDataIsMissing(boolean b)
          set if valid_range is considered isMissing(); better set in constructor if possible
 void setMissingDataIsMissing(boolean b)
          set if missing_data is considered isMissing(); better set in constructor if possible
 void setUseNaNs(boolean useNaNs)
          set whether to use NaNs for missing float/double values, for efficiency
 

Method Detail

hasMissing

boolean hasMissing()
true if Variable has missing data values


isMissing

boolean isMissing(double val)
true if val is a missing data value.

Returns:
true if this value is a misssing value

isMissingFast

boolean isMissingFast(double val)
true if val is a missing data value. if useNaNs is true (default) and the variable is of type float or double, then assume that missing values have been set to NaN, which speeds up the test considerably.

Returns:
true if this value is a misssing value

hasInvalidData

boolean hasInvalidData()
true if Variable has valid_range, valid_min or valid_max attributes


getValidMin

double getValidMin()
return the minimum value in the valid range


getValidMax

double getValidMax()
return the maximum value in the valid range


isInvalidData

boolean isInvalidData(double val)
return true if val is outside the valid range


hasFillValue

boolean hasFillValue()
true if Variable has _FillValue attribute


isFillValue

boolean isFillValue(double val)
return true if val equals the _FillValue


hasMissingValue

boolean hasMissingValue()
true if Variable has missing_value attribute


isMissingValue

boolean isMissingValue(double val)
return true if val equals a missing_value


setFillValueIsMissing

void setFillValueIsMissing(boolean b)
set if _FillValue is considered isMissing(); better set in constructor if possible


setInvalidDataIsMissing

void setInvalidDataIsMissing(boolean b)
set if valid_range is considered isMissing(); better set in constructor if possible


setMissingDataIsMissing

void setMissingDataIsMissing(boolean b)
set if missing_data is considered isMissing(); better set in constructor if possible


setUseNaNs

void setUseNaNs(boolean useNaNs)
set whether to use NaNs for missing float/double values, for efficiency


getUseNaNs

boolean getUseNaNs()
get whether to use NaNs for missing float/double values (for efficiency)


hasScaleOffset

boolean hasScaleOffset()
true if Variable data will be converted using scale and offet


convertScaleOffsetMissing

Array convertScaleOffsetMissing(Array data)
                                throws java.io.IOException
Convert data if hasScaleOffset, using scale and offset. Also if useNaNs = true, return NaN if value is missing data.

Parameters:
data - convert this
Returns:
converted data.
Throws:
java.io.IOException

convertScaleOffsetMissing

double convertScaleOffsetMissing(byte value)
Convert this byte value to a double, using scale/offset/missing value if applicable


convertScaleOffsetMissing

double convertScaleOffsetMissing(short value)
Convert this short value to a double, using scale/offset/missing value if applicable


convertScaleOffsetMissing

double convertScaleOffsetMissing(int value)
Convert this int value to a double, using scale/offset/missing value if applicable


convertScaleOffsetMissing

double convertScaleOffsetMissing(long value)
Convert this long value to a double, using scale/offset/missing value if applicable


convertScaleOffsetMissing

double convertScaleOffsetMissing(double value)
Convert this double value using scale/offset/missing value if applicable