Skip to content

Feature params

Mongo Filter

filter is a query that is used for the dataset pre-filtering.

Faas uses mongo query syntax, so check out mongo query tutorial and query operators reference.

In this example, only documents that have field price less than 145000.0 will be aggregated:

config = {
    ...
    'filter': {'price': {'$lt': 145000.0}},
}

More complex filter example:

config = {
    ...
    'filter': {
        'price': {'$ne': null},
        'rooms': {'$gte': 10},
        'area': {'$nin': [null, NaN, '']},
        'address': {'$regex': ".*London.*"},
    }
}

You can set a runtime filter, i.e. filter that depends on the calculated point. This filter throws out the dataset points that have id == point.id:

config = {
    ...
    'filter': {'id': {'$ne': '{point.id}'}},
}

Aggregation

Supported aggregation functions:

  1. sum — sum of values;
  2. min — minimum value;
  3. max — maximum value;
  4. mean — average value;
  5. median - median value;
  6. count — number of values;
  7. nunique - number of unique values.
config = {
    ...
    'agg': 'sum',
}

Note: for count and nunique aggregations, the default features fillna is 0.

Metrics

Поддерживаемые метрики:

  1. dist — straight line distance, m
  2. foot — walking time, min
  3. foot_dist — walking distance, m
  4. car_in — driving time to the point, min
  5. car_out — driving time from the point, min
  6. car_in_dist — driving distance to the point, m
  7. car_out_dist — driving distance from the point, m
  8. bike — cycling time, min
feature = {
    ...
    'metric': 'car_in',
}

Regions

Region is set in config, default 'cis'.

Metric Functions

Metric functions are used to change the weight of aggregated values.

Example:

Let's say we are calculating the sum of apartments in a 10 minute isochrone, and there are three houses in the isochrone.

house id apartments metric
1 100 8
2 120 5
3 150 1

By default, the values are simply summed: \(100 + 120 + 150 = 370\)

Now let's imagine that we want to take into account the houses on the edge of the isochrone with a linearly lower weight. Then we should apply a linear metric function, which will set the weight for each house depending on the distance in minutes from the calculated point. When linear(10) is applied, the weights appear:

house id apartments metric weight
1 100 8 0.2
2 120 5 0.5
3 150 1 0.9

And values are summed after multiplication by weights: \(0.2 * 100 + 0.5 * 120 + 0.9 * 150 = 215\)

Usually only single metric function is applied:

feature = {
    ...
    'metric_funcs': [
        {'func': 'sigmoid', 'coef': 2.0, 'shift': 1.0},
    ],
}

However, it is possible to apply several metric functions sequentially. This example is equal to \(donut(sigmoid(metric, coef, shift), limit\_from, limit\_to)\):

feature = {
    ...
    'metric_funcs': [
        {'func': 'sigmoid', 'coef': 2.0, 'shift': 1.0},
        {'func': 'donut', 'limit_from': 5, 'limit_to': 10},
    ],
}

Available metric functions (\(mt\) — metric value):

function weight formula params and default values image
linear \(max(0, 1 - \frac{mt}{limit})\) limit linear
sigmoid \((1 - \frac{1}{1 + e^{-coef(mt - shift)}})\) coef=1
shift=0		 sigmoid
normal \(e^{-\frac{1}{2} (\frac{mt}{sigma})^{power}}\) sigma=1
power=2		 normal
exp \(e^{log(\frac{1}{2}) * \frac{mt}{half\_limit}}\)
\(1\) if \(mt=0\)
\(\frac{1}{2}\) if \(mt=half\_limit\)		 half_limit exp
sqrt \(max(0, 1 - \sqrt{\frac{mt}{limit}})\) limit sqrt
pow \(max(0, 1 - (\frac{mt}{limit})^{power})\) power
limit		 pow
circle \(1\) if \(0 < mt < limit\)
\(0\) otherwise		 limit circle
donut \(1\) if \(limit_{from} < mt < limit_{to}\)
\(0\) otherwise		 limit_from
limit_to=np.inf		 donut
huffnet \(y(x) = \frac{\sum_i y_i \cdot \exp\left( -\frac{(x - x_i)^2}{\text{scale}^2} \right)}{\sum_i \exp\left( -\frac{(x - x_i)^2}{\text{scale}^2} \right)}\) params
limit
scale=None
start_from=1
inverse=False		 parametrized monotonic function

Influence Functions

The influence is the value that is used to calculate probability in huff features.

By default linear influence function is used, but you can set another in feature config:

feature = {
    ...
    'infl_func': [
        {'func': 'pow', 'pow_coef': 2.0},
    ],
}

Available influence functions (\(mt\) — metric value, \(wt\) — weight value):

function influence formula params and default values
linear \(\frac{wt}{mt}\)
pow \(\frac{wt}{mt^{\frac{pow\_coef}{(1 + wt)^{weight\_pow}}}}\) pow_coef=1
weight_pow=0
sigmoid \(wt \cdot (1 - \frac{1}{1 + e^{-coef(mt - shift)}})\) coef=1
shift=0
normal \(wt \cdot e^{-\frac{1}{2} (\frac{mt}{sigma})^{power}}\) sigma=1
power=2
huffnet parametrized monotonic function \(y(x) = \frac{\sum_i y_i \cdot \exp\left( -\frac{(x - x_i)^2}{\text{scale}^2} \right)}{\sum_i \exp\left( -\frac{(x - x_i)^2}{\text{scale}^2} \right)}\) params
limit
scale=None
start_from=1
inverse=False

Geometry filters

Supported geometry filters:

  1. cross_right — take into account only those LineStrings where the point lies to the right relative to the direction of movement;
  2. cross_left — take into account only those LineStrings where the point lies to the left relative to the direction of movement;

Note: For both modifiers dataset's geometry type must be geojson LineString.

config = {
    ...
    'type': 'near',
    'geometry_filter': 'cross_right',
}

Formula instead of field

For all features instead of aggregating single field you can calculate an expression over several fields and then aggregate it.

Suppose we know area of the apartments and their price and we want to aggregate price per square meter. In that case we can use formula instead of single field:

feature = {
    ...
    'field': 'price / apartment_area',
}

The formula expression should use numexpr syntax.

Note: field names with spaces are not allowed.

Note: for Near feature dropna and dropzero will be applied for the evaluated formula field.

Virtual ones

Sometimes it is useful to perform aggregation without taking into account any specific field. For example, to calculate huff features for individual objects and aggregate the impact of each house. For such situations, you can use the virtual field of ones. It is denoted as _ones and can be used in formula field.

feature = {
    ...
    'field': '_ones',
}

Such a field is completely equivalent to a field of ones if it were in the dataset. You can perform similar formula operations with it without restrictions.