Benchmark report
Summary
Result summary
Best score summary
Useful for visualizing the best solver configuration.
| Solver | Total | Average | Standard Deviation | Problem | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 200computers-600processes | 800computers-2400processes | model_b_1 | model_b_10 | comp07 | comp08 | exam_comp_set2 | exam_comp_set3 | medium01 | medium_hint01 | 1-nl14 | ||||
| Cloud Balancing Late Acceptance 4 | 0hard/-1105795soft | 0hard/-552898soft | 0.0/356429.50 | |||||||||||
| Machine Reassignment Tabu Search 5 | 0hard/-23164681621soft | 0hard/-11582340811soft | 0.0/7.81 | |||||||||||
| Course Scheduling Late Acceptance 1 | 0hard/-359soft | 0hard/-180soft | 0.0/43.50 | |||||||||||
| Examination Tabu Search 2 | 0hard/-12841soft | 0hard/-6421soft | 0.0/5825.50 | |||||||||||
| Nurse Rostering Tabu Search 0 | 0hard/-313soft | 0hard/-157soft | 0.0/97.50 | |||||||||||
| TravelingTournament Tabu Search 3 | 0hard/-232367soft | 0hard/-232367soft | 0.0/0.0 | |||||||||||
Best score scalability summary
Useful for visualizing the scalability of each solver configuration.
Winning score difference summary
Useful for zooming in on the results of the best score summary.
| Solver | Total | Average | Problem | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 200computers-600processes | 800computers-2400processes | model_b_1 | model_b_10 | comp07 | comp08 | exam_comp_set2 | exam_comp_set3 | medium01 | medium_hint01 | 1-nl14 | |||
| Cloud Balancing Late Acceptance 4 | 0hard/0soft | 0hard/0soft | 0hard/0soft 0 | 0hard/0soft 0 | |||||||||
| Machine Reassignment Tabu Search 5 | 0hard/0soft | 0hard/0soft | 0hard/0soft 0 | 0hard/0soft 0 | |||||||||
| Course Scheduling Late Acceptance 1 | 0hard/0soft | 0hard/0soft | 0hard/0soft 0 | 0hard/0soft 0 | |||||||||
| Examination Tabu Search 2 | 0hard/0soft | 0hard/0soft | 0hard/0soft 0 | 0hard/0soft 0 | |||||||||
| Nurse Rostering Tabu Search 0 | 0hard/0soft | 0hard/0soft | 0hard/0soft 0 | 0hard/0soft 0 | |||||||||
| TravelingTournament Tabu Search 3 | 0hard/0soft | 0hard/0soft | 0hard/0soft 0 | ||||||||||
Worst score difference percentage summary (ROI)
Useful for visualizing the return on investment (ROI) to decision makers.
| Solver | Average | Problem | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 200computers-600processes | 800computers-2400processes | model_b_1 | model_b_10 | comp07 | comp08 | exam_comp_set2 | exam_comp_set3 | medium01 | medium_hint01 | 1-nl14 | ||
| Cloud Balancing Late Acceptance 4 | 0.00%/0.00% | 0.00%/0.00% 0 | 0.00%/0.00% 0 | |||||||||
| Machine Reassignment Tabu Search 5 | 0.00%/0.00% | 0.00%/0.00% 0 | 0.00%/0.00% 0 | |||||||||
| Course Scheduling Late Acceptance 1 | 0.00%/0.00% | 0.00%/0.00% 0 | 0.00%/0.00% 0 | |||||||||
| Examination Tabu Search 2 | 0.00%/0.00% | 0.00%/0.00% 0 | 0.00%/0.00% 0 | |||||||||
| Nurse Rostering Tabu Search 0 | 0.00%/0.00% | 0.00%/0.00% 0 | 0.00%/0.00% 0 | |||||||||
| TravelingTournament Tabu Search 3 | 0.00%/0.00% | 0.00%/0.00% 0 | ||||||||||
Performance summary
Average calculate count summary
Useful for comparing different score calculators and/or score rule implementations (presuming that the solver configurations do not differ otherwise). Also useful to measure the scalability cost of an extra constraint.
| Solver | Average | Problem | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 200computers-600processes | 800computers-2400processes | model_b_1 | model_b_10 | comp07 | comp08 | exam_comp_set2 | exam_comp_set3 | medium01 | medium_hint01 | 1-nl14 | ||
| Problem scale | 23,283,913 | 120,000 | 1,920,000 | 500,000 | 250,000,000 | 217,000 | 145,800 | 1,697,556 | 1,486,272 | 18,848 | 12,840 | 4,732 |
| Cloud Balancing Late Acceptance 4 | 75,768/s | |||||||||||
| Machine Reassignment Tabu Search 5 | 115,316/s | |||||||||||
| Course Scheduling Late Acceptance 1 | 9,983/s | |||||||||||
| Examination Tabu Search 2 | 12,409/s | |||||||||||
| Nurse Rostering Tabu Search 0 | 5,964/s | |||||||||||
| TravelingTournament Tabu Search 3 | 1,946/s | |||||||||||
Time spent summary
Useful for visualizing the performance of construction heuristics (presuming that no other solver phases are configured).
| Solver | Average | Problem | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 200computers-600processes | 800computers-2400processes | model_b_1 | model_b_10 | comp07 | comp08 | exam_comp_set2 | exam_comp_set3 | medium01 | medium_hint01 | 1-nl14 | ||
| Problem scale | 23,283,913 | 120,000 | 1,920,000 | 500,000 | 250,000,000 | 217,000 | 145,800 | 1,697,556 | 1,486,272 | 18,848 | 12,840 | 4,732 |
| Cloud Balancing Late Acceptance 4 | 300,000 | |||||||||||
| Machine Reassignment Tabu Search 5 | 300,010 | |||||||||||
| Course Scheduling Late Acceptance 1 | 300,000 | |||||||||||
| Examination Tabu Search 2 | 300,000 | |||||||||||
| Nurse Rostering Tabu Search 0 | 300,000 | |||||||||||
| TravelingTournament Tabu Search 3 | 300,000 | |||||||||||
Time spent scalability summary
Useful for extrapolating the scalability of construction heuristics (presuming that no other solver phases are configured).
Best score per time spent summary
Useful for visualizing trade-off between the best score versus the time spent for construction heuristics (presuming that no other solver phases are configured).
Problem benchmarks
200computers-600processes
Entity count: 600
Variable count: 600
Problem scale: 120,000
Memory usage after loading the inputSolution (before creating the Solver): 14,974,040 bytes on average.
800computers-2400processes
Entity count: 2,400
Variable count: 2,400
Problem scale: 1,920,000
Memory usage after loading the inputSolution (before creating the Solver): 15,153,440 bytes on average.
model_b_1
Entity count: 5,000
Variable count: 5,000
Problem scale: 500,000
Memory usage after loading the inputSolution (before creating the Solver): 20,331,888 bytes on average.
model_b_10
Entity count: 50,000
Variable count: 50,000
Problem scale: 250,000,000
Memory usage after loading the inputSolution (before creating the Solver): 998,161,736 bytes on average.
comp07
Entity count: 434
Variable count: 868
Problem scale: 217,000
Memory usage after loading the inputSolution (before creating the Solver): 14,402,800 bytes on average.
comp08
Entity count: 324
Variable count: 648
Problem scale: 145,800
Memory usage after loading the inputSolution (before creating the Solver): 14,370,528 bytes on average.
exam_comp_set2
Entity count: 870
Variable count: 1,736
Problem scale: 1,697,556
Memory usage after loading the inputSolution (before creating the Solver): 15,608,024 bytes on average.
exam_comp_set3
Entity count: 934
Variable count: 1,792
Problem scale: 1,486,272
Memory usage after loading the inputSolution (before creating the Solver): 15,902,120 bytes on average.
medium01
Entity count: 608
Variable count: 608
Problem scale: 18,848
Memory usage after loading the inputSolution (before creating the Solver): 15,829,512 bytes on average.
medium_hint01
Entity count: 428
Variable count: 428
Problem scale: 12,840
Memory usage after loading the inputSolution (before creating the Solver): 15,883,680 bytes on average.
1-nl14
Entity count: 182
Variable count: 182
Problem scale: 4,732
Memory usage after loading the inputSolution (before creating the Solver): 14,254,872 bytes on average.
Solver benchmarks
Cloud Balancing Late Acceptance 4
<solver>
<environmentMode>PRODUCTION</environmentMode>
<solutionClass>org.optaplanner.examples.cloudbalancing.domain.CloudBalance</solutionClass>
<entityClass>org.optaplanner.examples.cloudbalancing.domain.CloudProcess</entityClass>
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
<scoreDrl>org/optaplanner/examples/cloudbalancing/solver/cloudBalancingScoreRules.drl</scoreDrl>
<initializingScoreTrend>ONLY_DOWN</initializingScoreTrend>
</scoreDirectorFactory>
<termination>
<minutesSpentLimit>5</minutesSpentLimit>
</termination>
<constructionHeuristic>
<constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
</constructionHeuristic>
<localSearch>
<unionMoveSelector>
<changeMoveSelector/>
<swapMoveSelector/>
</unionMoveSelector>
<acceptor>
<lateAcceptanceSize>400</lateAcceptanceSize>
</acceptor>
<forager>
<acceptedCountLimit>4</acceptedCountLimit>
</forager>
</localSearch>
</solver>
Machine Reassignment Tabu Search 5
<solver>
<environmentMode>PRODUCTION</environmentMode>
<solutionClass>org.optaplanner.examples.machinereassignment.domain.MachineReassignment</solutionClass>
<entityClass>org.optaplanner.examples.machinereassignment.domain.MrProcessAssignment</entityClass>
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT_LONG</scoreDefinitionType>
<incrementalScoreCalculatorClass>org.optaplanner.examples.machinereassignment.solver.score.MachineReassignmentIncrementalScoreCalculator</incrementalScoreCalculatorClass>
</scoreDirectorFactory>
<termination>
<minutesSpentLimit>5</minutesSpentLimit>
</termination>
<customPhase>
<customPhaseCommandClass>org.optaplanner.examples.machinereassignment.solver.solution.initializer.ToOriginalMachineSolutionInitializer</customPhaseCommandClass>
</customPhase>
<localSearch>
<unionMoveSelector>
<changeMoveSelector/>
<swapMoveSelector/>
</unionMoveSelector>
<acceptor>
<entityTabuSize>7</entityTabuSize>
</acceptor>
<forager>
<acceptedCountLimit>2000</acceptedCountLimit>
</forager>
</localSearch>
</solver>
Course Scheduling Late Acceptance 1
<solver>
<environmentMode>PRODUCTION</environmentMode>
<solutionClass>org.optaplanner.examples.curriculumcourse.domain.CourseSchedule</solutionClass>
<entityClass>org.optaplanner.examples.curriculumcourse.domain.Lecture</entityClass>
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
<scoreDrl>org/optaplanner/examples/curriculumcourse/solver/curriculumCourseScoreRules.drl</scoreDrl>
</scoreDirectorFactory>
<termination>
<minutesSpentLimit>5</minutesSpentLimit>
</termination>
<constructionHeuristic>
<constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
</constructionHeuristic>
<localSearch>
<unionMoveSelector>
<changeMoveSelector/>
<swapMoveSelector>
<filterClass>org.optaplanner.examples.curriculumcourse.solver.move.DifferentCourseSwapMoveFilter</filterClass>
</swapMoveSelector>
</unionMoveSelector>
<acceptor>
<lateAcceptanceSize>600</lateAcceptanceSize>
</acceptor>
<forager>
<acceptedCountLimit>4</acceptedCountLimit>
</forager>
</localSearch>
</solver>
Examination Tabu Search 2
<solver>
<environmentMode>PRODUCTION</environmentMode>
<solutionClass>org.optaplanner.examples.examination.domain.Examination</solutionClass>
<entityClass>org.optaplanner.examples.examination.domain.Exam</entityClass>
<entityClass>org.optaplanner.examples.examination.domain.LeadingExam</entityClass>
<entityClass>org.optaplanner.examples.examination.domain.FollowingExam</entityClass>
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
<scoreDrl>org/optaplanner/examples/examination/solver/examinationScoreRules.drl</scoreDrl>
</scoreDirectorFactory>
<termination>
<minutesSpentLimit>5</minutesSpentLimit>
</termination>
<constructionHeuristic>
<queuedEntityPlacer>
<entitySelector id="placerEntitySelector">
<entityClass>org.optaplanner.examples.examination.domain.Exam</entityClass>
<cacheType>PHASE</cacheType>
<selectionOrder>SORTED</selectionOrder>
<sorterManner>DECREASING_DIFFICULTY</sorterManner>
</entitySelector>
<cartesianProductMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector>
<downcastEntityClass>org.optaplanner.examples.examination.domain.LeadingExam</downcastEntityClass>
<variableName>period</variableName>
<cacheType>PHASE</cacheType>
</valueSelector>
</changeMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector>
<variableName>room</variableName>
<cacheType>PHASE</cacheType>
<selectionOrder>SORTED</selectionOrder>
<sorterManner>INCREASING_STRENGTH</sorterManner>
</valueSelector>
</changeMoveSelector>
</cartesianProductMoveSelector>
</queuedEntityPlacer>
</constructionHeuristic>
<localSearch>
<unionMoveSelector>
<cartesianProductMoveSelector>
<changeMoveSelector>
<entitySelector id="cartesianProductEntitySelector">
<entityClass>org.optaplanner.examples.examination.domain.Exam</entityClass>
</entitySelector>
<valueSelector>
<variableName>room</variableName>
</valueSelector>
</changeMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="cartesianProductEntitySelector"/>
<valueSelector>
<downcastEntityClass>org.optaplanner.examples.examination.domain.LeadingExam</downcastEntityClass>
<variableName>period</variableName>
</valueSelector>
</changeMoveSelector>
</cartesianProductMoveSelector>
<swapMoveSelector>
<entitySelector>
<entityClass>org.optaplanner.examples.examination.domain.LeadingExam</entityClass>
</entitySelector>
</swapMoveSelector>
</unionMoveSelector>
<acceptor>
<entityTabuSize>10</entityTabuSize>
</acceptor>
<forager>
<acceptedCountLimit>2000</acceptedCountLimit>
</forager>
</localSearch>
</solver>
Nurse Rostering Tabu Search 0
<solver>
<environmentMode>PRODUCTION</environmentMode>
<solutionClass>org.optaplanner.examples.nurserostering.domain.NurseRoster</solutionClass>
<entityClass>org.optaplanner.examples.nurserostering.domain.ShiftAssignment</entityClass>
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
<scoreDrl>org/optaplanner/examples/nurserostering/solver/nurseRosteringScoreRules.drl</scoreDrl>
</scoreDirectorFactory>
<termination>
<minutesSpentLimit>5</minutesSpentLimit>
</termination>
<constructionHeuristic>
<constructionHeuristicType>WEAKEST_FIT</constructionHeuristicType>
</constructionHeuristic>
<localSearch>
<unionMoveSelector>
<moveListFactory>
<cacheType>PHASE</cacheType>
<moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.EmployeeChangeMoveFactory</moveListFactoryClass>
</moveListFactory>
<moveListFactory>
<cacheType>PHASE</cacheType>
<moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentSwapMoveFactory</moveListFactoryClass>
</moveListFactory>
<moveListFactory>
<cacheType>STEP</cacheType>
<moveListFactoryClass>org.optaplanner.examples.nurserostering.solver.move.factory.ShiftAssignmentPillarPartSwapMoveFactory</moveListFactoryClass>
</moveListFactory>
</unionMoveSelector>
<acceptor>
<entityTabuSize>7</entityTabuSize>
</acceptor>
<forager>
<acceptedCountLimit>800</acceptedCountLimit>
</forager>
</localSearch>
</solver>
TravelingTournament Tabu Search 3
<solver>
<environmentMode>PRODUCTION</environmentMode>
<solutionClass>org.optaplanner.examples.travelingtournament.domain.TravelingTournament</solutionClass>
<entityClass>org.optaplanner.examples.travelingtournament.domain.Match</entityClass>
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
<scoreDrl>org/optaplanner/examples/travelingtournament/solver/travelingTournamentScoreRules.drl</scoreDrl>
</scoreDirectorFactory>
<termination>
<minutesSpentLimit>5</minutesSpentLimit>
</termination>
<localSearch>
<unionMoveSelector>
<swapMoveSelector>
<cacheType>PHASE</cacheType>
<selectionOrder>SHUFFLED</selectionOrder>
<filterClass>org.optaplanner.examples.travelingtournament.solver.move.factory.InverseMatchSwapMoveFilter</filterClass>
</swapMoveSelector>
<moveListFactory>
<cacheType>STEP</cacheType>
<selectionOrder>SHUFFLED</selectionOrder>
<moveListFactoryClass>org.optaplanner.examples.travelingtournament.solver.move.factory.MatchChainRotationsMoveFactory</moveListFactoryClass>
</moveListFactory>
</unionMoveSelector>
<acceptor>
<moveTabuSize>7</moveTabuSize>
<solutionTabuSize>1500</solutionTabuSize>
</acceptor>
<forager/>
</localSearch>
</solver>
Benchmark information
| Name | 2015-09-24_230752 |
|---|---|
| Aggregation | false |
| Failure count | 0 |
| Starting timestamp | Sep 24, 2015 11:07:52 PM |
| Warm up time spent | 30,000 ms |
| Parallel benchmark count / available processors | 1 / 8 |
| Benchmark time spent | 36,426,146 ms |
| Environment mode | PRODUCTION |
| Logging level | debug |
| Solver ranking class | TotalScoreSolverRankingComparator |
| VM max memory (as in -Xmx but lower) | 7,635,730,432 bytes |
| OptaPlanner version | Unjarred development snapshot |
| Java version | Java 1.7.0_79 (Oracle Corporation) |
| Java VM | Java Java HotSpot(TM) 64-Bit Server VM 24.79-b02 (Oracle Corporation) |
| Operating system | Linux amd64 4.1.6-201.fc22.x86_64 |
| Report locale | en_US |