OptaPlanner

Benchmark report

Summary

Result summary

Best score summary

Useful for visualizing the best solver configuration.

Solver Total Average Standard Deviation Problem
medium01 medium02 medium_hint01 medium_hint02
Simulated Annealing (Phreak (Drools 6.0))  1 0hard/-656soft 0hard/-164soft 0.0/91.78 0hard/-259soft  3 0hard/-249soft  0 0hard/-49soft  0 0hard/-99soft  1
Simulated Annealing (ReteOO (Drools 6.0))  0 0hard/-645soft 0hard/-162soft 0.0/90.71 0hard/-249soft  0 0hard/-253soft  3 0hard/-53soft  1 0hard/-90soft  0
Tabu Search (Phreak (Drools 6.0))  2 0hard/-695soft 0hard/-174soft 0.0/81.70 0hard/-251soft  1 0hard/-249soft  0 0hard/-56soft  2 0hard/-139soft  2
Tabu Search (ReteOO (Drools 6.0))  3 0hard/-700soft 0hard/-175soft 0.0/81.68 0hard/-252soft  2 0hard/-251soft  2 0hard/-58soft  3 0hard/-139soft  2

Best score scalability summary

Useful for visualizing the scalability of each solver configuration.

Winning score difference summary

Useful for zooming in on the results of the best score summary.

Solver Total Average Problem
medium01 medium02 medium_hint01 medium_hint02
Simulated Annealing (Phreak (Drools 6.0))  1 0hard/-19soft 0hard/-5soft 0hard/-10soft  3 0hard/0soft  0 0hard/0soft  0 0hard/-9soft  1
Simulated Annealing (ReteOO (Drools 6.0))  0 0hard/-8soft 0hard/-2soft 0hard/0soft  0 0hard/-4soft  3 0hard/-4soft  1 0hard/0soft  0
Tabu Search (Phreak (Drools 6.0))  2 0hard/-58soft 0hard/-15soft 0hard/-2soft  1 0hard/0soft  0 0hard/-7soft  2 0hard/-49soft  2
Tabu Search (ReteOO (Drools 6.0))  3 0hard/-63soft 0hard/-16soft 0hard/-3soft  2 0hard/-2soft  2 0hard/-9soft  3 0hard/-49soft  2

Worst score difference percentage summary (ROI)

Useful for visualizing the return on investment (ROI) to decision makers.

Solver Average Problem
medium01 medium02 medium_hint01 medium_hint02
Simulated Annealing (Phreak (Drools 6.0))  1 0.00%/11.47% 0.00%/0.00%  3 0.00%/1.58%  0 0.00%/15.52%  0 0.00%/28.78%  1
Simulated Annealing (ReteOO (Drools 6.0))  0 0.00%/11.93% 0.00%/3.86%  0 0.00%/0.00%  3 0.00%/8.62%  1 0.00%/35.25%  0
Tabu Search (Phreak (Drools 6.0))  2 0.00%/2.03% 0.00%/3.09%  1 0.00%/1.58%  0 0.00%/3.45%  2 0.00%/0.00%  2
Tabu Search (ReteOO (Drools 6.0))  3 0.00%/0.87% 0.00%/2.70%  2 0.00%/0.79%  2 0.00%/0.00%  3 0.00%/0.00%  2

Performance summary

Average calculate count summary

Useful for comparing different score calculators and/or score rule implementations (presuming that the solver configurations do not differ otherwise). Also useful to measure the scalability cost of an extra constraint.

Solver Average Problem
medium01 medium02 medium_hint01 medium_hint02
Problem scale 15,844 18,848 18,848 12,840 12,840
Simulated Annealing (Phreak (Drools 6.0))  1 3,454/s 3,525/s 3,466/s 3,296/s 3,532/s
Simulated Annealing (ReteOO (Drools 6.0))  0 2,830/s 2,618/s 2,690/s 2,866/s 3,148/s
Tabu Search (Phreak (Drools 6.0))  2 3,620/s 3,780/s 3,695/s 3,415/s 3,593/s
Tabu Search (ReteOO (Drools 6.0))  3 3,029/s 2,946/s 2,895/s 3,020/s 3,256/s

Time spend summary

Useful for visualizing the performance of construction heuristics (presuming that no other solver phases are configured).

Solver Average Problem
medium01 medium02 medium_hint01 medium_hint02
Problem scale 15,844 18,848 18,848 12,840 12,840
Simulated Annealing (Phreak (Drools 6.0))  1 700,000 700,000 700,000 700,000 700,000
Simulated Annealing (ReteOO (Drools 6.0))  0 700,000 700,000 700,000 700,000 700,000
Tabu Search (Phreak (Drools 6.0))  2 700,000 700,000 700,000 700,000 700,000
Tabu Search (ReteOO (Drools 6.0))  3 700,000 700,000 700,000 700,000 700,000

Time spend scalability summary

Useful for extrapolating the scalability of construction heuristics (presuming that no other solver phases are configured).

Best score per time spend summary

Useful for visualizing trade-off between the best score versus the time spend for construction heuristics (presuming that no other solver phases are configured).

Problem benchmarks

medium01

CSV files per solver:

medium02

CSV files per solver:

medium_hint01

CSV files per solver:

medium_hint02

CSV files per solver:

Solver benchmarks

Simulated Annealing (Phreak (Drools 6.0))  1 

<solver>
  <solutionClass>org.optaplanner.examples.nurserostering.domain.NurseRoster</solutionClass>
  <planningEntityClass>org.optaplanner.examples.nurserostering.domain.ShiftAssignment</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/nurserostering/solver/nurseRosteringScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>700</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.EmployeeChangeMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>STEP</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentPillarPartSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>5</entityTabuSize>
      <simulatedAnnealingStartingTemperature>0hard/8soft</simulatedAnnealingStartingTemperature>
    </acceptor>
    <forager>
      <acceptedCountLimit>4</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Simulated Annealing (ReteOO (Drools 6.0))  0 

<solver>
  <solutionClass>org.optaplanner.examples.nurserostering.domain.NurseRoster</solutionClass>
  <planningEntityClass>org.optaplanner.examples.nurserostering.domain.ShiftAssignment</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/nurserostering/solver/nurseRosteringScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>700</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.EmployeeChangeMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>STEP</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentPillarPartSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>5</entityTabuSize>
      <simulatedAnnealingStartingTemperature>0hard/8soft</simulatedAnnealingStartingTemperature>
    </acceptor>
    <forager>
      <acceptedCountLimit>4</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Tabu Search (Phreak (Drools 6.0))  2 

<solver>
  <solutionClass>org.optaplanner.examples.nurserostering.domain.NurseRoster</solutionClass>
  <planningEntityClass>org.optaplanner.examples.nurserostering.domain.ShiftAssignment</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/nurserostering/solver/nurseRosteringScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>700</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT</constructionHeuristicType>
  </constructionHeuristic>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.EmployeeChangeMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>STEP</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentPillarPartSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>7</entityTabuSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>800</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Tabu Search (ReteOO (Drools 6.0))  3 

<solver>
  <solutionClass>org.optaplanner.examples.nurserostering.domain.NurseRoster</solutionClass>
  <planningEntityClass>org.optaplanner.examples.nurserostering.domain.ShiftAssignment</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/nurserostering/solver/nurseRosteringScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>700</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT</constructionHeuristicType>
  </constructionHeuristic>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.EmployeeChangeMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>PHASE</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
      <moveListFactory>
        <cacheType>STEP</cacheType>
        <moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentPillarPartSwapMoveFactory</moveListFactoryClass>
      </moveListFactory>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>7</entityTabuSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>800</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Benchmark information

Name 2014-01-23_113523
Aggregation true
startingTimestamp Jan 23, 2014 11:35:23 AM
warmUpTimeMillisSpend 30,000 ms
parallelBenchmarkCount / availableProcessors 2 / 4
benchmarkTimeMillisSpend ms
failureCount 0
VM max memory (as in -Xmx but lower) 1,431,699,456 bytes
Operating system Linux i386 3.2.0-57-generic-pae
Java version Java 1.7.0_25 (Oracle Corporation)
Java VM Java OpenJDK Server VM 23.7-b01 (Oracle Corporation)
OptaPlanner version Unjarred development snapshot
Report locale en_US