ucar.nc2.dataset
Interface EnhanceScaleMissing
- All Known Implementing Classes:
- CoordinateAxis, CoordinateAxis1D, CoordinateAxis1DTime, CoordinateAxis2D, VariableDS
public interface EnhanceScaleMissing
A Variable decorator that handles missing data, and scale/offset packed data.
Specifically, this handles:
- packed data using scale_factor and add_offset
- invalid/missing data using valid_min, valid_max, valid_range,
missing_value or _FillValue
if those "standard attributes" are present.
Standard Use
Implementation rules for missing data. Here "missing data" is a general
name for invalid/never written/missing values. Use this interface when you dont need to
distinguish these variants. See below for a lower-level interface if you do need to
distinguish between them.
- By default, hasMissing() is true if any of hasInvalidData(), hasFillData() or
hasMissingValue() are true (see below). You can modify this behavior in the constuctor
or by calling setInvalidDataIsMissing(), setFillDataIsMissing(), or setMissingValueIsMissing().
- Test specific values through isMissing(double val). Note that the data is converted and
compared as a double.
- Data values of float or double NaN are considered missing data and will return
true if called with isMissing(). (However hasMissing() will not detect if you are
setting NaNs yourself).
- if you do not need to distinguish between _FillValue, missing_value and invalid, then
set useNaNs = true in the constructor. When the Variable element type is float or double
(or is set to double because its packed), then this sets isMissing() data values to NaNs, which
makes further comparisions more efficient.
Implementation rules for scale/offset (no missing data)
- If scale_factor and/or add_offset variable attributes are present,
then this is a "packed" Variable.
- The Variable element type is converted in the following way:
- The dataType is set to the widest type of 1) the data type 2) the scale_factor attribute type
and 3) the add_offset attribute type. (byte < short < int < float < double ).
- external (packed) data is converted to internal (unpacked) data transparently
during the read() call.
Implementation rules for missing data with scale/offset
- Always converted to a float or double type.
- _FillValue and missing_value values are always in the units of the external
(packed) data.
- If valid_range is the same type as scale_factor (actually the wider of
scale_factor and add_offset) and this is wider than the external data, then it
will be interpreted as being in the units of the internal (unpacked) data.
Otherwise it is in the units of the external (packed) data.
- The dataType is set to float if all attributes used are float (scale_factor, add_offset
valid_min, valid_max, valid_range, missing_data and _FillValue) otherwise the dataType is
set to double.
Low Level Access
The following provide more direct access to missing/invalid data. These are mostly convenience
routines for checking the standard attributes. If you set useNaNs = true in the constructor,
these routines cannot be used when the data has type float or double.
Implementation rules for valid_range
- if valid_range is present, valid_min and valid_max attributes are
ignored. Otherwise, the valid_min and/or valid_max is used to construct
a valid range. If any of these exist, hasInvalidData() is true.
- To test a specific value, call isInvalidData(). Note that the data is converted and
compared as a double. Or get the range through getValidMin() and getValidMax().
Implementation rules for _FillData
- if the _FillData attribute is present, it must have a scalar value of the same
type as the data. In this case, hasFillData() returns true.
- Test specific values through isFillValue(). Note that the data is converted and
compared as a double.
Implementation rules for missing_value
- if the missing_value attribute is present, it must have a scalar or vector
value of the same type as the data. In this case, hasMissingValue() returns true.
- Test specific values through isMissingValue(). Note that the data is converted and
compared as a double.
Strategies for using EnhanceScaleMissing
- Low-level: use the is/has InvalidData/FillData/missingValue routines
to "roll-your own" tests for the various kinds of missing/invalid data.
- Standard: use is/hasMissing() to test for missing data when you dont need to
distinguish between the variants. Use the setXXXisMissing() to customize the behavior
if needed.
- Efficient : If you expect to scan more than once for missing values, and
you are not distinguishing between InvalidData/FillData/missingValue, then
set useNaNs in the constructor. This sets isMissing() data values to NaNs when reading,
and subsequent checks are more efficient.
- Author:
- caron
Method Summary |
Array |
convertScaleOffsetMissing(Array data)
Convert data if hasScaleOffset, using scale and offset. |
double |
convertScaleOffsetMissing(byte value)
Convert this byte value to a double, using scale/offset/missing value if applicable |
double |
convertScaleOffsetMissing(double value)
Convert this double value using scale/offset/missing value if applicable |
double |
convertScaleOffsetMissing(int value)
Convert this int value to a double, using scale/offset/missing value if applicable |
double |
convertScaleOffsetMissing(long value)
Convert this long value to a double, using scale/offset/missing value if applicable |
double |
convertScaleOffsetMissing(short value)
Convert this short value to a double, using scale/offset/missing value if applicable |
boolean |
getUseNaNs()
get whether to use NaNs for missing float/double values (for efficiency) |
double |
getValidMax()
return the maximum value in the valid range |
double |
getValidMin()
return the minimum value in the valid range |
boolean |
hasFillValue()
true if Variable has _FillValue attribute |
boolean |
hasInvalidData()
true if Variable has valid_range, valid_min or valid_max attributes |
boolean |
hasMissing()
true if Variable has missing data values |
boolean |
hasMissingValue()
true if Variable has missing_value attribute |
boolean |
hasScaleOffset()
true if Variable data will be converted using scale and offet |
boolean |
isFillValue(double val)
return true if val equals the _FillValue |
boolean |
isInvalidData(double val)
return true if val is outside the valid range |
boolean |
isMissing(double val)
true if val is a missing data value. |
boolean |
isMissingFast(double val)
true if val is a missing data value. |
boolean |
isMissingValue(double val)
return true if val equals a missing_value |
void |
setFillValueIsMissing(boolean b)
set if _FillValue is considered isMissing(); better set in constructor if possible |
void |
setInvalidDataIsMissing(boolean b)
set if valid_range is considered isMissing(); better set in constructor if possible |
void |
setMissingDataIsMissing(boolean b)
set if missing_data is considered isMissing(); better set in constructor if possible |
void |
setUseNaNs(boolean useNaNs)
set whether to use NaNs for missing float/double values, for efficiency |
hasMissing
boolean hasMissing()
- true if Variable has missing data values
isMissing
boolean isMissing(double val)
- true if val is a missing data value.
- Returns:
- true if this value is a misssing value
isMissingFast
boolean isMissingFast(double val)
- true if val is a missing data value.
if useNaNs is true (default) and the variable is of type float or double,
then assume that missing values have been set to NaN, which speeds up the test considerably.
- Returns:
- true if this value is a misssing value
hasInvalidData
boolean hasInvalidData()
- true if Variable has valid_range, valid_min or valid_max attributes
getValidMin
double getValidMin()
- return the minimum value in the valid range
getValidMax
double getValidMax()
- return the maximum value in the valid range
isInvalidData
boolean isInvalidData(double val)
- return true if val is outside the valid range
hasFillValue
boolean hasFillValue()
- true if Variable has _FillValue attribute
isFillValue
boolean isFillValue(double val)
- return true if val equals the _FillValue
hasMissingValue
boolean hasMissingValue()
- true if Variable has missing_value attribute
isMissingValue
boolean isMissingValue(double val)
- return true if val equals a missing_value
setFillValueIsMissing
void setFillValueIsMissing(boolean b)
- set if _FillValue is considered isMissing(); better set in constructor if possible
setInvalidDataIsMissing
void setInvalidDataIsMissing(boolean b)
- set if valid_range is considered isMissing(); better set in constructor if possible
setMissingDataIsMissing
void setMissingDataIsMissing(boolean b)
- set if missing_data is considered isMissing(); better set in constructor if possible
setUseNaNs
void setUseNaNs(boolean useNaNs)
- set whether to use NaNs for missing float/double values, for efficiency
getUseNaNs
boolean getUseNaNs()
- get whether to use NaNs for missing float/double values (for efficiency)
hasScaleOffset
boolean hasScaleOffset()
- true if Variable data will be converted using scale and offet
convertScaleOffsetMissing
Array convertScaleOffsetMissing(Array data)
throws java.io.IOException
- Convert data if hasScaleOffset, using scale and offset.
Also if useNaNs = true, return NaN if value is missing data.
- Parameters:
data
- convert this
- Returns:
- converted data.
- Throws:
java.io.IOException
convertScaleOffsetMissing
double convertScaleOffsetMissing(byte value)
- Convert this byte value to a double, using scale/offset/missing value if applicable
convertScaleOffsetMissing
double convertScaleOffsetMissing(short value)
- Convert this short value to a double, using scale/offset/missing value if applicable
convertScaleOffsetMissing
double convertScaleOffsetMissing(int value)
- Convert this int value to a double, using scale/offset/missing value if applicable
convertScaleOffsetMissing
double convertScaleOffsetMissing(long value)
- Convert this long value to a double, using scale/offset/missing value if applicable
convertScaleOffsetMissing
double convertScaleOffsetMissing(double value)
- Convert this double value using scale/offset/missing value if applicable