OptaPlanner

Benchmark report

Summary

Result summary

Best score summary

Useful for visualizing the best solver configuration.

Solver cb-0100comp-0300proc cb-0200comp-0600proc cb-0400comp-1200proc cb-0800comp-2400proc Average Standard Deviation Ranking
First Fit  4 0hard/-137440soft  4 0hard/-253960soft  4 0hard/-544680soft  4 0hard/-1115430soft  4 0hard/-512878soft 0.0/378177.37 4
First Fit Decreasing  3 0hard/-132440soft  3 0hard/-244360soft  3 0hard/-529680soft  3 0hard/-1065950soft  3 0hard/-493108soft 0.0/361054.85 3
Tabu Search  2 0hard/-113800soft  2 0hard/-200620soft  2 0hard/-455030soft  2 0hard/-933480soft  2 0hard/-425733soft 0.0/318842.98 2
Simulated Annealing  1 0hard/-112400soft  0 0hard/-196350soft  0 0hard/-443210soft  1 0hard/-917390soft  1 0hard/-417338soft 0.0/313266.75 1
Late acceptance  0 0hard/-112470soft  1 0hard/-197320soft  1 0hard/-441870soft  0 0hard/-915260soft  0 0hard/-416730soft 0.0/312201.48 0

Best score scalability summary

Useful for visualizing the scalability of each solver configuration.

Winning score difference summary

Useful for zooming in on the results of the best score summary.

Solver cb-0100comp-0300proc cb-0200comp-0600proc cb-0400comp-1200proc cb-0800comp-2400proc Average
First Fit  4 0hard/-25040soft  4 0hard/-57610soft  4 0hard/-102810soft  4 0hard/-200170soft  4 0hard/-96408soft
First Fit Decreasing  3 0hard/-20040soft  3 0hard/-48010soft  3 0hard/-87810soft  3 0hard/-150690soft  3 0hard/-76638soft
Tabu Search  2 0hard/-1400soft  2 0hard/-4270soft  2 0hard/-13160soft  2 0hard/-18220soft  2 0hard/-9263soft
Simulated Annealing  1 0hard/0soft  0 0hard/0soft  0 0hard/-1340soft  1 0hard/-2130soft  1 0hard/-868soft
Late acceptance  0 0hard/-70soft  1 0hard/-970soft  1 0hard/0soft  0 0hard/0soft  0 0hard/-260soft

Worst score difference percentage summary (ROI)

Useful for visualizing the return on investment (ROI) to decision makers.

Solver cb-0100comp-0300proc cb-0200comp-0600proc cb-0400comp-1200proc cb-0800comp-2400proc Average
First Fit  4 0.00%/0.00%  4 0.00%/0.00%  4 0.00%/0.00%  4 0.00%/0.00%  4 0.00%/0.00%
First Fit Decreasing  3 0.00%/3.64%  3 0.00%/3.78%  3 0.00%/2.75%  3 0.00%/4.44%  3 0.00%/3.65%
Tabu Search  2 0.00%/17.20%  2 0.00%/21.00%  2 0.00%/16.46%  2 0.00%/16.31%  2 0.00%/17.74%
Simulated Annealing  1 0.00%/18.22%  0 0.00%/22.68%  0 0.00%/18.63%  1 0.00%/17.75%  1 0.00%/19.32%
Late acceptance  0 0.00%/18.17%  1 0.00%/22.30%  1 0.00%/18.88%  0 0.00%/17.95%  0 0.00%/19.32%

Performance summary

Average calculate count summary

Useful for comparing different score calculators and/or score rule implementations (presuming that the solver configurations do not differ otherwise). Also useful to measure the scalability cost of an extra constraint.

Solver cb-0100comp-0300proc cb-0200comp-0600proc cb-0400comp-1200proc cb-0800comp-2400proc Average
Problem scale 30,000 120,000 480,000 1,920,000 637,500
First Fit  4 47,763/s 72,287/s 73,090/s 80,020/s 68,290/s
First Fit Decreasing  3 45,684/s 66,905/s 69,538/s 78,475/s 65,150/s
Tabu Search  2 54,099/s 55,263/s 51,267/s 48,160/s 52,197/s
Simulated Annealing  1 50,398/s 48,896/s 44,108/s 40,227/s 45,907/s
Late acceptance  0 53,204/s 49,738/s 45,562/s 41,562/s 47,516/s

Time spend summary

Useful for visualizing the performance of construction heuristics (presuming that no other solver phases are configured).

Solver cb-0100comp-0300proc cb-0200comp-0600proc cb-0400comp-1200proc cb-0800comp-2400proc
Problem scale 30,000 120,000 480,000 1,920,000
First Fit  4 410 1,097 4,144 15,187
First Fit Decreasing  3 447 1,207 4,410 15,287
Tabu Search  2 300,000 300,000 300,000 300,000
Simulated Annealing  1 300,000 300,000 300,000 300,000
Late acceptance  0 300,000 300,000 300,000 300,000

Time spend scalability summary

Useful for extrapolating the scalability of construction heuristics (presuming that no other solver phases are configured).

Best score per time spend summary

Useful for visualizing trade-off between the best score versus the time spend for construction heuristics (presuming that no other solver phases are configured).

Problem benchmarks

cb-0100comp-0300proc

cb-0200comp-0600proc

cb-0400comp-1200proc

cb-0800comp-2400proc

Solver benchmarks

First Fit  4 

<solver>
  <solutionClass>org.optaplanner.examples.cloudbalancing.domain.CloudBalance</solutionClass>
  <planningEntityClass>org.optaplanner.examples.cloudbalancing.domain.CloudProcess</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/cloudbalancing/solver/cloudBalancingScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumMinutesSpend>5</maximumMinutesSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT</constructionHeuristicType>
    <forager>
      <pickEarlyType>FIRST_NON_DETERIORATING_SCORE</pickEarlyType>
    </forager>
  </constructionHeuristic>
</solver>

First Fit Decreasing  3 

<solver>
  <solutionClass>org.optaplanner.examples.cloudbalancing.domain.CloudBalance</solutionClass>
  <planningEntityClass>org.optaplanner.examples.cloudbalancing.domain.CloudProcess</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/cloudbalancing/solver/cloudBalancingScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumMinutesSpend>5</maximumMinutesSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
    <forager>
      <pickEarlyType>FIRST_NON_DETERIORATING_SCORE</pickEarlyType>
    </forager>
  </constructionHeuristic>
</solver>

Simulated Annealing  1 

<solver>
  <solutionClass>org.optaplanner.examples.cloudbalancing.domain.CloudBalance</solutionClass>
  <planningEntityClass>org.optaplanner.examples.cloudbalancing.domain.CloudProcess</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/cloudbalancing/solver/cloudBalancingScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumMinutesSpend>5</maximumMinutesSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
    <forager>
      <pickEarlyType>FIRST_NON_DETERIORATING_SCORE</pickEarlyType>
    </forager>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector/>
      <swapMoveSelector/>
    </unionMoveSelector>
    <acceptor>
      <simulatedAnnealingStartingTemperature>0hard/400soft</simulatedAnnealingStartingTemperature>
    </acceptor>
    <forager>
      <acceptedCountLimit>4</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Late acceptance  0 

<solver>
  <solutionClass>org.optaplanner.examples.cloudbalancing.domain.CloudBalance</solutionClass>
  <planningEntityClass>org.optaplanner.examples.cloudbalancing.domain.CloudProcess</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/cloudbalancing/solver/cloudBalancingScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumMinutesSpend>5</maximumMinutesSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
    <forager>
      <pickEarlyType>FIRST_NON_DETERIORATING_SCORE</pickEarlyType>
    </forager>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector/>
      <swapMoveSelector/>
    </unionMoveSelector>
    <acceptor>
      <lateAcceptanceSize>400</lateAcceptanceSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>4</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Benchmark information

name 2013-08-19_141655
startingTimestamp Aug 19, 2013 2:16:55 PM
warmUpTimeMillisSpend 30,000 ms
parallelBenchmarkCount / availableProcessors 2 / 4
benchmarkTimeMillisSpend 1,869,238 ms
failureCount 0
VM max memory (as in -Xmx but lower) 1,431,699,456 bytes
Operating system Linux i386 3.2.0-51-generic-pae
Java version Java 1.7.0_25 (Oracle Corporation)
Java VM Java OpenJDK Server VM 23.7-b01 (Oracle Corporation)
Planner version Unjarred development snapshot
Report locale en_US