EvoTrees.jl

EvoTrees.jl is a library for fitting decision trees in Julia.

Gradient boosted tree regression

Here is an example:

julia> using JuMP, MathOptAI, EvoTrees
julia> truth(x::Vector) = x[1] <= 0.5 ? -2 : (x[2] <= 0.3 ? 3 : 4)truth (generic function with 1 method)
julia> x_train = abs.(sin.((1:10) .* (3:4)'));
julia> size(x_train)(10, 2)
julia> y_train = truth.(Vector.(eachrow(x_train)));
julia> config = EvoTrees.EvoTreeRegressor(; nrounds = 3);
julia> predictor = EvoTrees.fit(config; x_train, y_train);
julia> model = Model();
julia> @variable(model, 0 <= x[1:2] <= 1);
julia> y, formulation = MathOptAI.add_predictor(model, predictor, x);
julia> y1-element Vector{JuMP.VariableRef}:
 moai_AffineCombination[1]
julia> formulationAffineCombination
├ 1.0 * BinaryDecisionTree{Float64,Float64} [leaves=3, depth=2]
├ 1.0 * BinaryDecisionTree{Float64,Float64} [leaves=3, depth=2]
├ 1.0 * BinaryDecisionTree{Float64,Float64} [leaves=3, depth=2]
└ 1.0 * [2.0]
├ variables [1]
│ └ moai_AffineCombination[1]
└ constraints [1]
  └ moai_BinaryDecisionTree_value + moai_BinaryDecisionTree_value + moai_BinaryDecisionTree_value - moai_AffineCombination[1] = -2
BinaryDecisionTree{Float64,Float64} [leaves=3, depth=2]
├ variables [4]
│ ├ moai_BinaryDecisionTree_value
│ ├ moai_BinaryDecisionTree_z[1]
│ ├ moai_BinaryDecisionTree_z[2]
│ └ moai_BinaryDecisionTree_z[3]
└ constraints [7]
  ├ moai_BinaryDecisionTree_z[1] + moai_BinaryDecisionTree_z[2] + moai_BinaryDecisionTree_z[3] = 1
  ├ moai_BinaryDecisionTree_z[1] --> {x[1] ≤ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[2] --> {x[1] ≥ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[2] --> {x[2] ≤ 0.3189870168319865}
  ├ moai_BinaryDecisionTree_z[3] --> {x[1] ≥ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[3] --> {x[2] ≥ 0.3189870168319865}
  └ 0.34285715222358704 moai_BinaryDecisionTree_z[1] - 0.07999999821186066 moai_BinaryDecisionTree_z[2] - 0.1818181872367859 moai_BinaryDecisionTree_z[3] + moai_BinaryDecisionTree_value = 0
BinaryDecisionTree{Float64,Float64} [leaves=3, depth=2]
├ variables [4]
│ ├ moai_BinaryDecisionTree_value
│ ├ moai_BinaryDecisionTree_z[1]
│ ├ moai_BinaryDecisionTree_z[2]
│ └ moai_BinaryDecisionTree_z[3]
└ constraints [7]
  ├ moai_BinaryDecisionTree_z[1] + moai_BinaryDecisionTree_z[2] + moai_BinaryDecisionTree_z[3] = 1
  ├ moai_BinaryDecisionTree_z[1] --> {x[1] ≤ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[2] --> {x[1] ≥ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[2] --> {x[2] ≤ 0.3189870168319865}
  ├ moai_BinaryDecisionTree_z[3] --> {x[1] ≥ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[3] --> {x[2] ≥ 0.3189870168319865}
  └ 0.31346938014030457 moai_BinaryDecisionTree_z[1] - 0.07360000908374786 moai_BinaryDecisionTree_z[2] - 0.1652892529964447 moai_BinaryDecisionTree_z[3] + moai_BinaryDecisionTree_value = 0
BinaryDecisionTree{Float64,Float64} [leaves=3, depth=2]
├ variables [4]
│ ├ moai_BinaryDecisionTree_value
│ ├ moai_BinaryDecisionTree_z[1]
│ ├ moai_BinaryDecisionTree_z[2]
│ └ moai_BinaryDecisionTree_z[3]
└ constraints [7]
  ├ moai_BinaryDecisionTree_z[1] + moai_BinaryDecisionTree_z[2] + moai_BinaryDecisionTree_z[3] = 1
  ├ moai_BinaryDecisionTree_z[1] --> {x[1] ≤ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[2] --> {x[1] ≥ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[2] --> {x[2] ≤ 0.3189870168319865}
  ├ moai_BinaryDecisionTree_z[3] --> {x[1] ≥ 0.425730688824737}
  ├ moai_BinaryDecisionTree_z[3] --> {x[2] ≥ 0.3189870168319865}
  └ 0.28660058975219727 moai_BinaryDecisionTree_z[1] - 0.06771200150251389 moai_BinaryDecisionTree_z[2] - 0.1502629667520523 moai_BinaryDecisionTree_z[3] + moai_BinaryDecisionTree_value = 0