OptaPlanner

Benchmark report

Summary

Result summary

Best score summary

Useful for visualizing the best solver configuration.

Solver Total Average Standard Deviation Problem
comp01 comp02 comp03 comp04 comp05 comp06 comp07 comp08 comp09 comp10 comp11 comp12 comp13 comp14
Late Acceptance (Phreak (Drools 6.0))  0 0hard/-2941soft 0hard/-211soft 0.0/173.04 0hard/-9soft  2 0hard/-211soft  0 0hard/-197soft  0 0hard/-114soft  2 0hard/-639soft  0 0hard/-181soft  0 0hard/-201soft  2 0hard/-145soft  2 0hard/-215soft  0 0hard/-160soft  1 0hard/0soft  0 0hard/-566soft  0 0hard/-154soft  0 0hard/-149soft  1
Late Acceptance (ReteOO (Drools 6.0))  1 0hard/-3051soft 0hard/-218soft 0.0/174.59 0hard/-9soft  2 0hard/-223soft  1 0hard/-200soft  1 0hard/-117soft  3 0hard/-645soft  1 0hard/-199soft  1 0hard/-232soft  3 0hard/-151soft  3 0hard/-222soft  1 0hard/-169soft  3 0hard/-1soft  1 0hard/-575soft  1 0hard/-156soft  1 0hard/-152soft  3
Tabu Search (Phreak (Drools 6.0))  2 -9hard/-3143soft -1hard/-225soft 1.58/195.11 0hard/-7soft  0 -1hard/-235soft  2 ! 0hard/-242soft  2 0hard/-109soft  0 -6hard/-748soft  2 ! -1hard/-223soft  2 ! 0hard/-157soft  0 0hard/-130soft  0 0hard/-230soft  2 0hard/-154soft  0 0hard/-16soft  2 -1hard/-580soft  2 ! 0hard/-164soft  2 0hard/-148soft  0
Tabu Search (ReteOO (Drools 6.0))  3 -9hard/-3192soft -1hard/-228soft 1.58/195.02 0hard/-7soft  0 -1hard/-238soft  3 ! 0hard/-243soft  3 0hard/-113soft  1 -6hard/-748soft  2 ! -1hard/-227soft  3 ! 0hard/-166soft  1 0hard/-133soft  1 0hard/-232soft  3 0hard/-168soft  2 0hard/-16soft  2 -1hard/-586soft  3 ! 0hard/-166soft  3 0hard/-149soft  1

Best score scalability summary

Useful for visualizing the scalability of each solver configuration.

Winning score difference summary

Useful for zooming in on the results of the best score summary.

Solver Total Average Problem
comp01 comp02 comp03 comp04 comp05 comp06 comp07 comp08 comp09 comp10 comp11 comp12 comp13 comp14
Late Acceptance (Phreak (Drools 6.0))  0 0hard/-73soft 0hard/-6soft 0hard/-2soft  2 0hard/0soft  0 0hard/0soft  0 0hard/-5soft  2 0hard/0soft  0 0hard/0soft  0 0hard/-44soft  2 0hard/-15soft  2 0hard/0soft  0 0hard/-6soft  1 0hard/0soft  0 0hard/0soft  0 0hard/0soft  0 0hard/-1soft  1
Late Acceptance (ReteOO (Drools 6.0))  1 0hard/-183soft 0hard/-14soft 0hard/-2soft  2 0hard/-12soft  1 0hard/-3soft  1 0hard/-8soft  3 0hard/-6soft  1 0hard/-18soft  1 0hard/-75soft  3 0hard/-21soft  3 0hard/-7soft  1 0hard/-15soft  3 0hard/-1soft  1 0hard/-9soft  1 0hard/-2soft  1 0hard/-4soft  3
Tabu Search (Phreak (Drools 6.0))  2 -9hard/-275soft -1hard/-20soft 0hard/0soft  0 -1hard/-24soft  2 ! 0hard/-45soft  2 0hard/0soft  0 -6hard/-109soft  2 ! -1hard/-42soft  2 ! 0hard/0soft  0 0hard/0soft  0 0hard/-15soft  2 0hard/0soft  0 0hard/-16soft  2 -1hard/-14soft  2 ! 0hard/-10soft  2 0hard/0soft  0
Tabu Search (ReteOO (Drools 6.0))  3 -9hard/-324soft -1hard/-24soft 0hard/0soft  0 -1hard/-27soft  3 ! 0hard/-46soft  3 0hard/-4soft  1 -6hard/-109soft  2 ! -1hard/-46soft  3 ! 0hard/-9soft  1 0hard/-3soft  1 0hard/-17soft  3 0hard/-14soft  2 0hard/-16soft  2 -1hard/-20soft  3 ! 0hard/-12soft  3 0hard/-1soft  1

Worst score difference percentage summary (ROI)

Useful for visualizing the return on investment (ROI) to decision makers.

Solver Average Problem
comp01 comp02 comp03 comp04 comp05 comp06 comp07 comp08 comp09 comp10 comp11 comp12 comp13 comp14
Late Acceptance (Phreak (Drools 6.0))  0 28.57%/15.02% 0.00%/0.00%  2 100.00%/11.34%  0 0.00%/18.93%  0 0.00%/2.56%  2 100.00%/14.57%  0 100.00%/20.26%  0 0.00%/13.36%  2 0.00%/3.97%  2 0.00%/7.33%  0 0.00%/5.33%  1 0.00%/100.00%  0 100.00%/3.41%  0 0.00%/7.23%  0 0.00%/1.97%  1
Late Acceptance (ReteOO (Drools 6.0))  1 28.57%/11.15% 0.00%/0.00%  2 100.00%/6.30%  1 0.00%/17.70%  1 0.00%/0.00%  3 100.00%/13.77%  1 100.00%/12.33%  1 0.00%/0.00%  3 0.00%/0.00%  3 0.00%/4.31%  1 0.00%/0.00%  3 0.00%/93.75%  1 100.00%/1.88%  1 0.00%/6.02%  1 0.00%/0.00%  3
Tabu Search (Phreak (Drools 6.0))  2 0.00%/6.67% 0.00%/22.22%  0 0.00%/1.26%  2 ! 0.00%/0.41%  2 0.00%/6.84%  0 0.00%/0.00%  2 ! 0.00%/1.76%  2 ! 0.00%/32.33%  0 0.00%/13.91%  0 0.00%/0.86%  2 0.00%/8.88%  0 0.00%/0.00%  2 0.00%/1.02%  2 ! 0.00%/1.20%  2 0.00%/2.63%  0
Tabu Search (ReteOO (Drools 6.0))  3 0.00%/4.90% 0.00%/22.22%  0 0.00%/0.00%  3 ! 0.00%/0.00%  3 0.00%/3.42%  1 0.00%/0.00%  2 ! 0.00%/0.00%  3 ! 0.00%/28.45%  1 0.00%/11.92%  1 0.00%/0.00%  3 0.00%/0.59%  2 0.00%/0.00%  2 0.00%/0.00%  3 ! 0.00%/0.00%  3 0.00%/1.97%  1

Performance summary

Average calculate count summary

Useful for comparing different score calculators and/or score rule implementations (presuming that the solver configurations do not differ otherwise). Also useful to measure the scalability cost of an extra constraint.

Solver Average Problem
comp01 comp02 comp03 comp04 comp05 comp06 comp07 comp08 comp09 comp10 comp11 comp12 comp13 comp14
Problem scale 115,971 28,800 113,200 100,400 128,700 49,248 162,450 217,000 145,800 125,550 166,500 36,450 86,328 146,300 116,875
Late Acceptance (Phreak (Drools 6.0))  0 7,236/s 11,852/s 6,420/s 6,746/s 7,937/s 3,927/s 6,466/s 6,225/s 7,822/s 6,920/s 6,597/s 11,697/s 4,138/s 7,427/s 7,136/s
Late Acceptance (ReteOO (Drools 6.0))  1 6,024/s 10,917/s 5,165/s 5,523/s 6,588/s 3,070/s 5,116/s 4,821/s 6,402/s 5,597/s 5,324/s 10,692/s 3,202/s 6,111/s 5,816/s
Tabu Search (Phreak (Drools 6.0))  2 7,347/s 11,841/s 6,513/s 6,833/s 8,105/s 4,098/s 6,661/s 6,361/s 7,832/s 7,075/s 6,759/s 11,712/s 4,214/s 7,675/s 7,190/s
Tabu Search (ReteOO (Drools 6.0))  3 6,148/s 10,838/s 5,328/s 5,579/s 6,795/s 3,221/s 5,379/s 5,086/s 6,514/s 5,709/s 5,372/s 10,800/s 3,237/s 6,396/s 5,820/s

Time spend summary

Useful for visualizing the performance of construction heuristics (presuming that no other solver phases are configured).

Solver Average Problem
comp01 comp02 comp03 comp04 comp05 comp06 comp07 comp08 comp09 comp10 comp11 comp12 comp13 comp14
Problem scale 115,971 28,800 113,200 100,400 128,700 49,248 162,450 217,000 145,800 125,550 166,500 36,450 86,328 146,300 116,875
Late Acceptance (Phreak (Drools 6.0))  0 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000
Late Acceptance (ReteOO (Drools 6.0))  1 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,003 429,000 429,000 429,000 429,000 429,000 429,000 429,000
Tabu Search (Phreak (Drools 6.0))  2 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000
Tabu Search (ReteOO (Drools 6.0))  3 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000 429,000

Time spend scalability summary

Useful for extrapolating the scalability of construction heuristics (presuming that no other solver phases are configured).

Best score per time spend summary

Useful for visualizing trade-off between the best score versus the time spend for construction heuristics (presuming that no other solver phases are configured).

Problem benchmarks

comp01

CSV files per solver:

comp02

CSV files per solver:

comp03

CSV files per solver:

comp04

CSV files per solver:

comp05

CSV files per solver:

comp06

CSV files per solver:

comp07

CSV files per solver:

comp08

CSV files per solver:

comp09

CSV files per solver:

comp10

CSV files per solver:

comp11

CSV files per solver:

comp12

CSV files per solver:

comp13

CSV files per solver:

comp14

CSV files per solver:

Solver benchmarks

Late Acceptance (Phreak (Drools 6.0))  0 

<solver>
  <solutionClass>org.optaplanner.examples.curriculumcourse.domain.CourseSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.curriculumcourse.domain.Lecture</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/curriculumcourse/solver/curriculumCourseScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>429</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>period</variableName>
        </valueSelector>
      </changeMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>room</variableName>
        </valueSelector>
      </changeMoveSelector>
      <swapMoveSelector>
        <filterClass>org.optaplanner.examples.curriculumcourse.solver.move.DifferentCourseSwapMoveFilter</filterClass>
      </swapMoveSelector>
    </unionMoveSelector>
    <acceptor>
      <lateAcceptanceSize>600</lateAcceptanceSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>4</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Late Acceptance (ReteOO (Drools 6.0))  1 

<solver>
  <solutionClass>org.optaplanner.examples.curriculumcourse.domain.CourseSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.curriculumcourse.domain.Lecture</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/curriculumcourse/solver/curriculumCourseScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>429</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>period</variableName>
        </valueSelector>
      </changeMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>room</variableName>
        </valueSelector>
      </changeMoveSelector>
      <swapMoveSelector>
        <filterClass>org.optaplanner.examples.curriculumcourse.solver.move.DifferentCourseSwapMoveFilter</filterClass>
      </swapMoveSelector>
    </unionMoveSelector>
    <acceptor>
      <lateAcceptanceSize>600</lateAcceptanceSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>4</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Tabu Search (Phreak (Drools 6.0))  2 

<solver>
  <solutionClass>org.optaplanner.examples.curriculumcourse.domain.CourseSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.curriculumcourse.domain.Lecture</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/curriculumcourse/solver/curriculumCourseScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>429</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>period</variableName>
        </valueSelector>
      </changeMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>room</variableName>
        </valueSelector>
      </changeMoveSelector>
      <swapMoveSelector>
        <filterClass>org.optaplanner.examples.curriculumcourse.solver.move.DifferentCourseSwapMoveFilter</filterClass>
      </swapMoveSelector>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>9</entityTabuSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>900</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Tabu Search (ReteOO (Drools 6.0))  3 

<solver>
  <solutionClass>org.optaplanner.examples.curriculumcourse.domain.CourseSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.curriculumcourse.domain.Lecture</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/curriculumcourse/solver/curriculumCourseScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>429</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>period</variableName>
        </valueSelector>
      </changeMoveSelector>
      <changeMoveSelector>
        <valueSelector>
          <variableName>room</variableName>
        </valueSelector>
      </changeMoveSelector>
      <swapMoveSelector>
        <filterClass>org.optaplanner.examples.curriculumcourse.solver.move.DifferentCourseSwapMoveFilter</filterClass>
      </swapMoveSelector>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>9</entityTabuSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>900</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Benchmark information

Name 2014-01-23_115754
Aggregation true
startingTimestamp Jan 23, 2014 11:57:54 AM
warmUpTimeMillisSpend 30,000 ms
parallelBenchmarkCount / availableProcessors 2 / 4
benchmarkTimeMillisSpend ms
failureCount 0
VM max memory (as in -Xmx but lower) 1,431,699,456 bytes
Operating system Linux i386 3.2.0-57-generic-pae
Java version Java 1.7.0_25 (Oracle Corporation)
Java VM Java OpenJDK Server VM 23.7-b01 (Oracle Corporation)
OptaPlanner version Unjarred development snapshot
Report locale en_US