Package Bio :: Module MaxEntropy
[hide private]
[frames] | no frames]

Module MaxEntropy

source code

Maximum Entropy code.

Uses Improved Iterative Scaling: XXX ref

# XXX need to define terminology

Classes [hide private]
  MaxEntropy
Holds information for a Maximum Entropy classifier.
Functions [hide private]
list of log probs
calculate(me, observation)
Calculate the log of the probability for each class.
source code
class
classify(me, observation)
Classify an observation into a class.
source code
dict of values
_eval_feature_fn(fn, xs, classes)
Evaluate a feature function on every instance of the training set and class.
source code
list of expectations
_calc_empirical_expects(xs, ys, classes, features)
Calculate the expectation of each function from the data.
source code
list of expectations
_calc_model_expects(xs, classes, features, alphas)
Calculate the expectation of each feature from the model.
source code
matrix
_calc_p_class_given_x(xs, classes, features, alphas)
Calculate P(y|x), where y is the class and x is an instance from the training set.
source code
matrix of f sharp values.
_calc_f_sharp(N, nclasses, features) source code
 
_iis_solve_delta(N, feature, f_sharp, empirical, prob_yx) source code
 
_train_iis(xs, classes, features, f_sharp, alphas, e_empirical) source code
MaxEntropy object
train(training_set, results, feature_fns, update_fn=...)
Train a maximum entropy classifier on a training set.
source code
Variables [hide private]
  MAX_IIS_ITERATIONS = 10000
  IIS_CONVERGE = 1e-05
  MAX_NEWTON_ITERATIONS = 100
  NEWTON_CONVERGE = 1e-10
  Complex0 = 'F'
  Complex16 = 'F'
  Complex32 = 'F'
  Complex64 = 'D'
  Complex8 = 'F'
  Float0 = 'f'
  Float16 = 'f'
  Float32 = 'f'
  Float64 = 'd'
  Float8 = 'f'
  Int0 = '1'
  Int16 = 's'
  Int32 = 'i'
  Int8 = '1'
  absolute = <ufunc 'absolute'>
  add = <ufunc 'add'>
  arccos = <ufunc 'arccos'>
  arccosh = <ufunc 'arccosh'>
  arcsin = <ufunc 'arcsin'>
  arcsinh = <ufunc 'arcsinh'>
  arctan = <ufunc 'arctan'>
  arctan2 = <ufunc 'arctan2'>
  arctanh = <ufunc 'arctanh'>
  bitwise_and = <ufunc 'bitwise_and'>
  bitwise_or = <ufunc 'bitwise_or'>
  bitwise_xor = <ufunc 'bitwise_xor'>
  ceil = <ufunc 'ceil'>
  conjugate = <ufunc 'conjugate'>
  cos = <ufunc 'cos'>
  cosh = <ufunc 'cosh'>
  divide = <ufunc 'divide'>
  divide_safe = <ufunc 'divide_safe'>
  e = 2.71828182846
  equal = <ufunc 'equal'>
  exp = <ufunc 'exp'>
  fabs = <ufunc 'fabs'>
  floor = <ufunc 'floor'>
  floor_divide = <ufunc 'floor_divide'>
  fmod = <ufunc 'fmod'>
  greater = <ufunc 'greater'>
  greater_equal = <ufunc 'greater_equal'>
  hypot = <ufunc 'hypot'>
  invert = <ufunc 'invert'>
  left_shift = <ufunc 'left_shift'>
  less = <ufunc 'less'>
  less_equal = <ufunc 'less_equal'>
  log = <ufunc 'log'>
  log10 = <ufunc 'log10'>
  logical_and = <ufunc 'logical_and'>
  logical_not = <ufunc 'logical_not'>
  logical_or = <ufunc 'logical_or'>
  logical_xor = <ufunc 'logical_xor'>
  maximum = <ufunc 'maximum'>
  minimum = <ufunc 'minimum'>
  multiply = <ufunc 'multiply'>
  negative = <ufunc 'negative'>
  not_equal = <ufunc 'not_equal'>
  pi = 3.14159265359
  power = <ufunc 'power'>
  remainder = <ufunc 'remainder'>
  right_shift = <ufunc 'right_shift'>
  sin = <ufunc 'sin'>
  sinh = <ufunc 'sinh'>
  sqrt = <ufunc 'sqrt'>
  subtract = <ufunc 'subtract'>
  tan = <ufunc 'tan'>
  tanh = <ufunc 'tanh'>
  true_divide = <ufunc 'true_divide'>
Function Details [hide private]

calculate(me, observation)

source code 

Calculate the log of the probability for each class. me is a MaxEntropy object that has been trained. observation is a vector representing the observed data. The return value is a list of unnormalized log probabilities for each class.

Returns: list of log probs

_eval_feature_fn(fn, xs, classes)

source code 

Evaluate a feature function on every instance of the training set and class. fn is a callback function that takes two parameters: a training instance and a class. Return a dictionary of (training set index, class index) -> non-zero value. Values of 0 are not stored in the dictionary.

Returns: dict of values

_calc_empirical_expects(xs, ys, classes, features)

source code 

Calculate the expectation of each function from the data. This is the constraint for the maximum entropy distribution. Return a list of expectations, parallel to the list of features.

Returns: list of expectations

_calc_model_expects(xs, classes, features, alphas)

source code 

Calculate the expectation of each feature from the model. This is not used in maximum entropy training, but provides a good function for debugging.

Returns: list of expectations

_calc_p_class_given_x(xs, classes, features, alphas)

source code 

Calculate P(y|x), where y is the class and x is an instance from the training set. Return a XSxCLASSES matrix of probabilities.

Returns: matrix

train(training_set, results, feature_fns, update_fn=...)

source code 

Train a maximum entropy classifier on a training set. training_set is a list of observations. results is a list of the class assignments for each observation. feature_fns is a list of the features. These are callback functions that take an observation and class and return a 1 or 0. update_fn is a callback function that's called at each training iteration. It is passed a MaxEntropy object that encapsulates the current state of the training.

Returns: MaxEntropy object