OptaPlanner

Benchmark report

Summary

Result summary

Best score summary

Useful for visualizing the best solver configuration.

Solver Total Average Standard Deviation Problem
testdata01 testdata02 testdata03 testdata04 testdata05 testdata06 testdata07 testdata08 testdata09 testdata10 testdata11 testdata12 testdata13
Late Acceptance (Phreak (Drools 6.0))  0 0hard/-1338medium/-1077460soft 0hard/-103medium/-82882soft 0.0/256.47/91373.17 0hard/0medium/-7284soft  3 0hard/0medium/-13112soft  3 0hard/0medium/-9734soft  3 0hard/0medium/-13450soft  3 0hard/0medium/-6660soft  1 0hard/0medium/-9158soft  3 0hard/0medium/-16912soft  3 0hard/0medium/-55392soft  3 0hard/-930medium/-186786soft  0 0hard/-2medium/-114918soft  2 0hard/-1medium/-175338soft  2 0hard/-354medium/-285202soft  2 0hard/-51medium/-183514soft  1
Late Acceptance (ReteOO (Drools 6.0))  1 0hard/-1343medium/-1072978soft 0hard/-104medium/-82537soft 0.0/258.24/91053.26 0hard/0medium/-7276soft  2 0hard/0medium/-13036soft  2 0hard/0medium/-9676soft  2 0hard/0medium/-13410soft  2 0hard/0medium/-6656soft  0 0hard/0medium/-9138soft  2 0hard/0medium/-16744soft  2 0hard/0medium/-55288soft  2 0hard/-937medium/-181770soft  1 0hard/-2medium/-114938soft  3 0hard/-1medium/-175338soft  2 0hard/-354medium/-285206soft  3 0hard/-49medium/-184502soft  0
Tabu Search (Phreak (Drools 6.0))  2 0hard/-1420medium/-932376soft 0hard/-110medium/-71722soft 0.0/259.96/82751.56 0hard/0medium/-7216soft  0 0hard/0medium/-12990soft  1 0hard/0medium/-8906soft  1 0hard/0medium/-13200soft  1 0hard/0medium/-6704soft  2 0hard/0medium/-8776soft  0 0hard/0medium/-15328soft  1 0hard/0medium/-52040soft  1 0hard/-946medium/-135058soft  2 0hard/-1medium/-112184soft  0 0hard/-1medium/-174244soft  0 0hard/-353medium/-284442soft  0 0hard/-119medium/-101288soft  2
Tabu Search (ReteOO (Drools 6.0))  3 0hard/-1425medium/-932666soft 0hard/-110medium/-71744soft 0.0/260.92/82837.10 0hard/0medium/-7216soft  0 0hard/0medium/-12978soft  0 0hard/0medium/-8876soft  0 0hard/0medium/-13168soft  0 0hard/0medium/-6704soft  2 0hard/0medium/-8776soft  0 0hard/0medium/-15256soft  0 0hard/0medium/-51910soft  0 0hard/-950medium/-135160soft  3 0hard/-2medium/-112138soft  1 0hard/-1medium/-174536soft  1 0hard/-353medium/-284660soft  1 0hard/-119medium/-101288soft  2

Best score scalability summary

Useful for visualizing the scalability of each solver configuration.

Winning score difference summary

Useful for zooming in on the results of the best score summary.

Solver Total Average Problem
testdata01 testdata02 testdata03 testdata04 testdata05 testdata06 testdata07 testdata08 testdata09 testdata10 testdata11 testdata12 testdata13
Late Acceptance (Phreak (Drools 6.0))  0 0hard/-4medium/-10466soft 0hard/-1medium/-806soft 0hard/0medium/-68soft  3 0hard/0medium/-134soft  3 0hard/0medium/-858soft  3 0hard/0medium/-282soft  3 0hard/0medium/-4soft  1 0hard/0medium/-382soft  3 0hard/0medium/-1656soft  3 0hard/0medium/-3482soft  3 0hard/0medium/0soft  0 0hard/-1medium/-2734soft  2 0hard/0medium/-1094soft  2 0hard/-1medium/-760soft  2 0hard/-2medium/988soft  1
Late Acceptance (ReteOO (Drools 6.0))  1 0hard/-9medium/-5984soft 0hard/-1medium/-461soft 0hard/0medium/-60soft  2 0hard/0medium/-58soft  2 0hard/0medium/-800soft  2 0hard/0medium/-242soft  2 0hard/0medium/0soft  0 0hard/0medium/-362soft  2 0hard/0medium/-1488soft  2 0hard/0medium/-3378soft  2 0hard/-7medium/5016soft  1 0hard/-1medium/-2754soft  3 0hard/0medium/-1094soft  2 0hard/-1medium/-764soft  3 0hard/0medium/0soft  0
Tabu Search (Phreak (Drools 6.0))  2 0hard/-86medium/134618soft 0hard/-7medium/10355soft 0hard/0medium/0soft  0 0hard/0medium/-12soft  1 0hard/0medium/-30soft  1 0hard/0medium/-32soft  1 0hard/0medium/-48soft  2 0hard/0medium/0soft  0 0hard/0medium/-72soft  1 0hard/0medium/-130soft  1 0hard/-16medium/51728soft  2 0hard/0medium/0soft  0 0hard/0medium/0soft  0 0hard/0medium/0soft  0 0hard/-70medium/83214soft  2
Tabu Search (ReteOO (Drools 6.0))  3 0hard/-91medium/134328soft 0hard/-7medium/10332soft 0hard/0medium/0soft  0 0hard/0medium/0soft  0 0hard/0medium/0soft  0 0hard/0medium/0soft  0 0hard/0medium/-48soft  2 0hard/0medium/0soft  0 0hard/0medium/0soft  0 0hard/0medium/0soft  0 0hard/-20medium/51626soft  3 0hard/-1medium/46soft  1 0hard/0medium/-292soft  1 0hard/0medium/-218soft  1 0hard/-70medium/83214soft  2

Worst score difference percentage summary (ROI)

Useful for visualizing the return on investment (ROI) to decision makers.

Solver Average Problem
testdata01 testdata02 testdata03 testdata04 testdata05 testdata06 testdata07 testdata08 testdata09 testdata10 testdata11 testdata12 testdata13
Late Acceptance (Phreak (Drools 6.0))  0 0.00%/4.56%/-9.13% 0.00%/0.00%/0.00%  3 0.00%/0.00%/0.00%  3 0.00%/0.00%/0.00%  3 0.00%/0.00%/0.00%  3 0.00%/0.00%/0.66%  1 0.00%/0.00%/0.00%  3 0.00%/0.00%/0.00%  3 0.00%/0.00%/0.00%  3 0.00%/2.11%/-38.20%  0 0.00%/0.00%/0.02%  2 0.00%/0.00%/0.00%  2 0.00%/0.00%/0.00%  2 0.00%/57.14%/-81.18%  1
Late Acceptance (ReteOO (Drools 6.0))  1 0.00%/4.63%/-8.69% 0.00%/0.00%/0.11%  2 0.00%/0.00%/0.58%  2 0.00%/0.00%/0.60%  2 0.00%/0.00%/0.30%  2 0.00%/0.00%/0.72%  0 0.00%/0.00%/0.22%  2 0.00%/0.00%/0.99%  2 0.00%/0.00%/0.19%  2 0.00%/1.37%/-34.49%  1 0.00%/0.00%/0.00%  3 0.00%/0.00%/0.00%  2 0.00%/0.00%/0.00%  3 0.00%/58.82%/-82.16%  0
Tabu Search (Phreak (Drools 6.0))  2 0.00%/3.90%/2.71% 0.00%/0.00%/0.93%  0 0.00%/0.00%/0.93%  1 0.00%/0.00%/8.51%  1 0.00%/0.00%/1.86%  1 0.00%/0.00%/0.00%  2 0.00%/0.00%/4.17%  0 0.00%/0.00%/9.37%  1 0.00%/0.00%/6.05%  1 0.00%/0.42%/0.08%  2 0.00%/50.00%/2.40%  0 0.00%/0.00%/0.62%  0 0.00%/0.28%/0.27%  0 0.00%/0.00%/0.00%  2
Tabu Search (ReteOO (Drools 6.0))  3 0.00%/0.02%/2.78% 0.00%/0.00%/0.93%  0 0.00%/0.00%/1.02%  0 0.00%/0.00%/8.81%  0 0.00%/0.00%/2.10%  0 0.00%/0.00%/0.00%  2 0.00%/0.00%/4.17%  0 0.00%/0.00%/9.79%  0 0.00%/0.00%/6.29%  0 0.00%/0.00%/0.00%  3 0.00%/0.00%/2.44%  1 0.00%/0.00%/0.46%  1 0.00%/0.28%/0.19%  1 0.00%/0.00%/0.00%  2

Performance summary

Average calculate count summary

Useful for comparing different score calculators and/or score rule implementations (presuming that the solver configurations do not differ otherwise). Also useful to measure the scalability cost of an extra constraint.

Solver Average Problem
testdata01 testdata02 testdata03 testdata04 testdata05 testdata06 testdata07 testdata08 testdata09 testdata10 testdata11 testdata12 testdata13
Problem scale 400,575 187,124 351,830 280,368 352,112 191,362 215,090 245,487 395,590 435,400 486,675 801,966 855,250 409,221
Late Acceptance (Phreak (Drools 6.0))  0 11,228/s 13,100/s 14,948/s 14,344/s 14,929/s 14,019/s 13,635/s 12,941/s 10,940/s 6,784/s 7,831/s 6,424/s 6,496/s 9,575/s
Late Acceptance (ReteOO (Drools 6.0))  1 11,584/s 13,567/s 16,240/s 15,182/s 15,856/s 15,152/s 14,054/s 14,011/s 11,348/s 6,352/s 7,205/s 5,906/s 5,885/s 9,834/s
Tabu Search (Phreak (Drools 6.0))  2 11,602/s 13,859/s 15,535/s 15,161/s 15,509/s 15,420/s 14,347/s 13,193/s 11,042/s 6,903/s 7,793/s 6,224/s 6,371/s 9,477/s
Tabu Search (ReteOO (Drools 6.0))  3 12,105/s 14,401/s 16,667/s 16,340/s 16,993/s 16,376/s 15,281/s 14,920/s 11,791/s 6,381/s 7,256/s 5,719/s 5,859/s 9,383/s

Time spend summary

Useful for visualizing the performance of construction heuristics (presuming that no other solver phases are configured).

Solver Average Problem
testdata01 testdata02 testdata03 testdata04 testdata05 testdata06 testdata07 testdata08 testdata09 testdata10 testdata11 testdata12 testdata13
Problem scale 400,575 187,124 351,830 280,368 352,112 191,362 215,090 245,487 395,590 435,400 486,675 801,966 855,250 409,221
Late Acceptance (Phreak (Drools 6.0))  0 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,006 300,000 300,000 300,000 300,000
Late Acceptance (ReteOO (Drools 6.0))  1 300,001 300,000 300,000 300,000 300,000 300,000 300,003 300,000 300,000 300,011 300,000 300,000 300,000 300,000
Tabu Search (Phreak (Drools 6.0))  2 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,008 300,000 300,000 300,000 300,000
Tabu Search (ReteOO (Drools 6.0))  3 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,000 300,004 300,000 300,000 300,002 300,000

Time spend scalability summary

Useful for extrapolating the scalability of construction heuristics (presuming that no other solver phases are configured).

Best score per time spend summary

Useful for visualizing trade-off between the best score versus the time spend for construction heuristics (presuming that no other solver phases are configured).

Problem benchmarks

testdata01

CSV files per solver:

testdata02

CSV files per solver:

testdata03

CSV files per solver:

testdata04

CSV files per solver:

testdata05

CSV files per solver:

testdata06

CSV files per solver:

testdata07

CSV files per solver:

testdata08

CSV files per solver:

testdata09

CSV files per solver:

testdata10

CSV files per solver:

testdata11

CSV files per solver:

testdata12

CSV files per solver:

testdata13

CSV files per solver:

Solver benchmarks

Late Acceptance (Phreak (Drools 6.0))  0 

<solver>
  <solutionClass>org.optaplanner.examples.pas.domain.PatientAdmissionSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.pas.domain.BedDesignation</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_MEDIUM_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/pas/solver/patientAdmissionScheduleScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>300</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector/>
      <swapMoveSelector/>
      <pillarSwapMoveSelector/>
    </unionMoveSelector>
    <acceptor>
      <lateAcceptanceSize>200</lateAcceptanceSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>1</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Late Acceptance (ReteOO (Drools 6.0))  1 

<solver>
  <solutionClass>org.optaplanner.examples.pas.domain.PatientAdmissionSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.pas.domain.BedDesignation</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_MEDIUM_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/pas/solver/patientAdmissionScheduleScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>300</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector/>
      <swapMoveSelector/>
      <pillarSwapMoveSelector/>
    </unionMoveSelector>
    <acceptor>
      <lateAcceptanceSize>200</lateAcceptanceSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>1</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Tabu Search (Phreak (Drools 6.0))  2 

<solver>
  <solutionClass>org.optaplanner.examples.pas.domain.PatientAdmissionSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.pas.domain.BedDesignation</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_MEDIUM_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/pas/solver/patientAdmissionScheduleScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>300</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector/>
      <swapMoveSelector/>
      <pillarSwapMoveSelector/>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>7</entityTabuSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>1000</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Tabu Search (ReteOO (Drools 6.0))  3 

<solver>
  <solutionClass>org.optaplanner.examples.pas.domain.PatientAdmissionSchedule</solutionClass>
  <planningEntityClass>org.optaplanner.examples.pas.domain.BedDesignation</planningEntityClass>
  <scoreDirectorFactory>
    <scoreDefinitionType>HARD_MEDIUM_SOFT</scoreDefinitionType>
    <scoreDrl>/org/optaplanner/examples/pas/solver/patientAdmissionScheduleScoreRules.drl</scoreDrl>
  </scoreDirectorFactory>
  <termination>
    <maximumSecondsSpend>300</maximumSecondsSpend>
  </termination>
  <constructionHeuristic>
    <constructionHeuristicType>BEST_FIT_DECREASING</constructionHeuristicType>
  </constructionHeuristic>
  <localSearch>
    <unionMoveSelector>
      <changeMoveSelector/>
      <swapMoveSelector/>
      <pillarSwapMoveSelector/>
    </unionMoveSelector>
    <acceptor>
      <entityTabuSize>7</entityTabuSize>
    </acceptor>
    <forager>
      <acceptedCountLimit>1000</acceptedCountLimit>
    </forager>
  </localSearch>
</solver>

Benchmark information

Name 2014-01-23_120147
Aggregation true
startingTimestamp Jan 23, 2014 12:01:47 PM
warmUpTimeMillisSpend 30,000 ms
parallelBenchmarkCount / availableProcessors 2 / 4
benchmarkTimeMillisSpend ms
failureCount 0
VM max memory (as in -Xmx but lower) 1,431,699,456 bytes
Operating system Linux i386 3.2.0-57-generic-pae
Java version Java 1.7.0_25 (Oracle Corporation)
Java VM Java OpenJDK Server VM 23.7-b01 (Oracle Corporation)
OptaPlanner version Unjarred development snapshot
Report locale en_US