The rest of this section presents the implementation of
@C { KheIntervalGrouping } in detail.  Its aim is to group tasks
into sequences of suitable duration, that is, total duration.  In
the example, the suitable durations are 4 and 5, although shorter
durations are also accepted (with a suitable penalty), to ensure
that every instance of the problem has a solution.
@PP
@BI { Constraint classes. }
@C { KheIntervalGrouping } begins by using the constraint classes
module (Section {@NumberOf resource_structural.constraint_classes})
to group the limit active intervals constraints (plus offsets) into
classes.  The constraints of one class @M { C } have the same time
groups in the same order with the same polarities, but they may differ
in other respects.  Notably, they may have different minimum
and maximum limits, and may apply to different resources.  We have
to consider the effect of this variability.  For example, suppose
some of the resources can have sequences of maximum duration 4, and
others can have sequences of maximum duration 5.  To build groups of
maximum duration 4 would be to throw away some potentially valuable
groups of duration 5.  But to build groups of maximum duration 5
could be dangerous, because not all resources can be assigned to
those groups.
@PP
In practice, however, the limits of the kinds of constraints we are
dealing with here do not seem to vary between resources.  They express
ideas about good working practice that are the same for all workers.
So we say that a class @M { C } has @I { uniform limits } if, within
the class, the same minimum limit applies to all resources, and the
same maximum limit applies to all resources.  We let @M { C sub "min" }
and @M { C sub "max" } be these uniform limits, when they exist.
@PP
After @C { KheIntervalGrouping } groups the limit active intervals
constraints, it selects those classes @M { C } which have the
following properties:
@NumberedList

@LI @OneRow {
@M { C } has the same number of time groups as there are days
in the days frame, and the @M { i }th time group of @M { C }
is a subset of the @M { i }th time group of the days frame,
for all @M { i }.
}

@LI @OneRow {
All @M { C }'s time groups are singletons.
}

@LI @OneRow {
All @M { C }'s time groups are positive.
}

@LI @OneRow {
@M { C } applies to every resource of type @C { rt }.
}

@LI @OneRow {
@M { C } has uniform limits @M { C sub "min" } and @M { C sub "max" },
as just defined.
}

@LI @OneRow {
@M { C sub "min" } and @M { C sub "max" } satisfy the conditions above
involving the @C { rs_interval_grouping_min } and
@C { rs_interval_grouping_range } options.
}

@EndList
It then applies the following algorithm to each selected class
@M { C } in turn.
# @PP
# Throughout this section we'll refer to times monitored by the
# constraints of @M { C } as night times, tasks running at those
# times as night tasks, and so on.  A day task is any task that
# is not a night task.  This is simpler than perpetually referring
# back to @M { C }.  Also, by a @I day we mean one time group of the
# common frame, and by a @I weekend we mean two adjacent days for which
# every resource is subject to a complete weekends constraint.
@PP
@BI { Admissible and included tasks. }
For one class @M { C }, the first step is to decide which tasks are
@I { admissible }:  able to be included in the solve.  Task
@M { s } is admissible if it satisfies these conditions:
@NumberedList

@LI @OneRow {
Task @M { s } has type @C { rt }.
}

@LI @OneRow {
Task @M { s } is returned by @C { KheMTaskTask } from
Section {@NumberOf resource_structural.mtask_finding.ops}
(so it is a proper root task).
}

@LI @OneRow {
The busy times of @M { s }, taken chronologically, lie
on consecutive days.
}

@LI @OneRow {
Task @M { s } is running at at least one time.
}

@LI @OneRow {
The busy times of @M { s }, taken chronologically, form a sequence
of times.  Within this sequence, if any of the times are monitored by
@M { C }, then those times must be adjacent, and must appear either
at the start of the sequence or at the end.  This point is discussed
further below.
}

@EndList
Our algorithm builds groups from a subset of the admissible tasks
called the @I { included tasks }.
@PP
Before we specify exactly which admissible tasks are included, we
need some definitions.  A @I { night time } is a time monitored
by the constraints of @M { C }.  Of course, @M { C } does not
have to monitor night shifts, but it often does.  Calling its
times night times is simpler than perpetually referring back to
@M { C }.  Also, by a @I day we mean one time group of the common
frame, and by a @I weekend we mean two adjacent days for which
every resource is subject to a complete weekends constraint.
@PP
There is nothing to prevent an admissible task @M { s } from
being a multi-day task, or a previously constructed group of
several tasks, some running during night times and others not.
We define @M { s }'s @I { (full) duration } to be its number of
busy times, and its @I { night duration } to be its number of
night times.  We define a @I { night task } to be an admissible
task whose night duration is one or more, and a @I { day task }
to be an admissible task whose night duration is 0.
@PP
For any admissible task @M { s } we can call @C { KheTaskNonAsstAndAsstCost }
(Section {@NumberOf resource_structural.mtask_finding.ops}).  This
returns @M { s }'s @I { non-assignment cost } (the cost of leaving
@M { s } unassigned) and its @I { assignment cost } (the cost of
assigning @M { s }).  If @M { s }'s non-assignment cost is larger
than its assignment cost, then we can expect to assign a resource
to @M { s } eventually, so we call @M { s } a @I { must-assign task }.
Otherwise we call @M { s } a @I { non-must-assign task }, since
usually it will not matter whether we assign a resource to @M { s }
or not, although it is possible that assigning a resource to @M { s }
might incur a positive net cost.
@PP
Although we allow our algorithm to include day tasks,
our actual aim is to group night tasks.  And although we allow
our algorithm to include non-must-assign tasks, we
group tasks in the expectation that the group will be assigned a
resource, so our actual aim is to group must-assign tasks.  So
admissible tasks which are both night tasks and must-assign tasks
are called @I { compulsory tasks } and are always included in the
solve; all other admissible tasks are called @I { optional tasks }.
They are optional in two senses:  they do not have to be included
in the solve at all, and if they are included, they do not have to
be grouped.  Actually we insist that every included task must go
into exactly one group, so we should rather say that a group
containing only optional tasks can have any duration and is always
assigned cost 0.
@PP
Night and day times can be intermingled arbitrarily within one task
@M { s }.  Our algorithm is limited in what it can do in such cases,
partly inherently.  For example, if @M { s } contains three tasks,
in the order day-night-day, the night task can't be made part of a
longer sequence of night tasks, because the day tasks prevent it.
We avoid such cases by means of admissibility condition (5) above.
For example, it makes day-day, night-day, day-night. and night-night
admissible, but excludes day-night-day and night-day-night.  The
day-night case can arise in practice, when weekend grouping runs
first and groups a Saturday day task with a Sunday night task.
@PP
Our algorithm is limited to building groups that contain at most
one sequence of night times.  Thus, a day-night task can only
occur at the start of a group (it could occur later, but only if
the preceding tasks are all day tasks, and an examination of the
tasks that are actually included, to be given next, will show that
that cannot happen), and a night-day task can only occur at the
end of a group.  This leads us to define a property of each
task, called its @I { placement }, whose type is an enumerated
type with three values:  @I { first_only }, meaning that the
task must be chronologically first in any group it is added to;
@I { last_only }, meaning that the task must be chronologically
last in any group it is added to; and @I { any }, meaning that
the task may appear anywhere in its group.
@PP
# The algorithm does not deal with tasks in their full complexity.
# Instead, for each task @M { s } it utilizes only the interval of
# days that @M { s } is running; whether @M { s } is optional; its
# placement; and its @I { night duration }, which is the number of
# night times that it is busy (possibly 0).  The algorithm also
# calls on the task grouper
# (Section {@NumberOf resource_structural.task_grouping.task_grouper})
# to find the resource constraint cost of each potential group and to
# carry out the actual grouping.  The task grouper utilizes the tasks'
# domains and the exact times when they are running.
# @PP
We can now say which admissible tasks are included.  There are
three kinds of included tasks.
(1) @I { Compulsory tasks. }
All compulsory tasks are included.  Most included tasks are of this
kind.
(2) @I { Weekend tasks. }
When the numbers of compulsory tasks on the two days
of a weekend differ, we include some must-assign day tasks from
the less busy of the two days, to allow the algorithm to
group every weekend must-assign night task with another must-assign
task, to satisfy complete weekends constraints.  An extra Saturday
day task has placement @I { first_only }; an extra Sunday day task
has placement @I { last_only }.
(3) @I { Lengthener tasks. }
As explained below, if the
algorithm fails to give every group a suitable duration on its
first attempt, then we add a few non-must-assign night tasks that
might solve this problem, and run the algorithm again.
@PP
@BI { Groups, solutions, and cost. }
Let @M { T sub 1 ,..., T sub n } be the time groups representing the days.
A @I group is a set @M { g } of one or more included tasks such that the
multiset of all times that the tasks of @M { g } are running contains one
time from each element of a set of consecutive days.  The placement property
of each task in the group must be respected.  The @I { (full) duration }
and @I { night duration } of a group are the sums of the corresponding
quantities for its individual tasks.  A group is @I { compulsory } if it
contains at least one compulsory task; otherwise it is @I { optional }.
@PP
A @I solution for a set of included tasks @M { S } is a pair
@M { (S, G) }, where @M { S } holds the included tasks and
@M { G } is a set of groups, such that each @M { s in S } lies
in exactly one group @M { g in G }.  For any @M { S } there is
always at least one solution, because one way to make one
is to place each @M { s in S } into its own group.
@PP
For each day @M { T sub i }, define two sets of tasks:
@BulletList

@LI @OneRow {
@M { X sub i }, the set of included tasks @M { s } such that
the first time @M { t } that @M { s } is running satisfies
@M { t in T sub i }.
}

@LI @OneRow {
@M { Y sub i }, the set of included tasks @M { s } such that
some time @M { t } that @M { s } is running satisfies
@M { t in T sub i }.
}

@EndList
# As just remarked, this is ignoring times outside the time groups of
# @M { C }.
We have @M { X sub i subset Y sub i }, and also the curious result
@M { X sub 1 cup cdots cup X sub i = Y sub 1 cup cdots cup Y sub i }.
The @M { X sub i } are pairwise disjoint; the @M { Y sub i } may not be,
since each @M { s } appears in one @M { Y sub i } for each time
it is running.  We will be particularly interested in the sets
@M { S sub i = X sub 1 cup cdots cup X sub i }.
# All included tasks are treated uniformly here; their
# placement and other properties do not matter.
@PP
Let @M { ( S sub i , G sub i ) } be a solution for @M { S sub i }.
For example, @M { ( S sub 7 , G sub 7 ) } might be this:
@CD @Diag paint { lightgrey } margin { 0c } { @VContract {
1.5c @Wide @M { T sub 1 } |
1.5c @Wide @M { T sub 2 } |
1.5c @Wide @M { T sub 3 } |
1.5c @Wide @M { T sub 4 } |
1.5c @Wide @M { T sub 5 } |
1.5c @Wide @M { T sub 6 } |
1.5c @Wide @M { T sub 7 } |
1.5c @Wide @M { T sub 8 } |
1.5c @Wide @M { T sub 9 } |
//0.2f
@Box { 6c @Wide 0.5c @High } |
@Box { 4.5c @Wide 0.5c @High } |
//
@Box paint { white } outlinestyle { noline } { 3c @Wide 0.5c @High } |
@Box { 7.5c @Wide 0.5c @High }
//
@Box { 7.5c @Wide 0.5c @High } |
@Box paint { white } outlinestyle { noline } { 1.5c @Wide 0.5c @High } |
@Box { 3c @Wide 0.5c @High }
} }
where each grey rectangle represents one group.  Every included task which
begins at or before @M { T sub 7 } is present in a group of @M { G sub 7 }.
# Groups that
# end before @M { T sub 7 } are finished, and will usually have
# a suitable length (4 or 5 in the example); other groups may
# still be forming and are likely to have smaller length.
@PP
The next step is to define a cost @M { c( S sub i , G sub i ) }
for each solution @M { ( S sub i , G sub i ) }.  This cost needs
to be a very good estimate of the cost to a complete solution
of grouping tasks in this way.  Our aim is to find a solution
of minimum cost for the whole set of included tasks.
@PP
We are free to assign different resources to different groups, so
each group will make an independent contribution.  Letting
@M { c(g) } be the as-yet-undefined cost of group @M { g }, we have
@ID @Math {
c( S sub i , G sub i ) = big sum from { g in G sub i } c(g)
}
Also, @M { c(g) = 0 } when @M { g } is optional.  So it remains to
define the cost of each compulsory group.
@PP
Within a given @M { ( S sub i , G sub i ) }, a @I { finished group }
is a group @M { g } that cannot be extended to the right.  If
@M { g } is not finished it is @I { unfinished }.  There are four
ways in which @M { g } can come to be finished:
@ParenAlphaList

@LI @OneRow {
If @M { g } does not include a task running at @M { T sub i },
it is finished because it is now too late to add such a task,
since every task running during @M { T sub i } is already in
a group; and adding a task from a later time group would create
a gap in @M { g } at @M { T sub i }.
}

@LI @OneRow { 
If @M { g }'s night duration is equal to or larger than
@M { C sub "max" }, then @M { g } is finished because adding any
task would make @M { g }'s night duration too large, which is not
permitted.  (Actually we could add a task whose night duration is
0.  So we should apply this rule only when @M { X sub {i+1} } does
not contain any tasks whose night duration is 0 and whose placement
is not @I { first_only }.  This is not currently implemented.)
}

@LI @OneRow {
If @M { g } contains a task whose placement is @I { last_only },
then @M { g } is finished because extending @M { g } to the right
would contradict this requirement.
}

@LI @OneRow {
If @M { g } is running during @M { T sub n }, the last time
group, then @M { g } is finished because there are no tasks
to the right of @M { T sub n } to add to @M { g }.
}

@EndList
# In the last three cases (but not the first), whether @M { g } is
# finished or not is independent of @M { G sub i }.
We can only determine @M { c(g) } for finished groups.  For example,
the cost of an unfinished group whose tasks are all optional will
depend on whether any compulsory tasks are added to it.  Even if
an unfinished group is compulsory, some constraints that affect
its cost will depend on what is still to come---the constraints
of @M { C }, for example.  It is true that this problem is solved
by time sweep resource assignment
(Section {@NumberOf resource_solvers.matching.time.sweep}), but
here it seems better to wait until a group is finished, and only
then assign a cost to it.  So we'll define @M { c(g) = 0 } for
unfinished groups as well as for optional groups.  It remains to
find @M { c(g) } for compulsory finished groups.
@PP
We list the costs that add up to @M { c(g) } in no particular order.
It is easy to invent many unlikely costs.  For example, if every
resource is unavailable for either all of day 9 or all of day 10,
then any group which covers both of those days should have high
cost.  We don't claim to handle such unlikely cases.  The reader
is going to have to trust that our list of costs is sufficiently
complete to be useful in practice.  We start with costs that can
be attributed to individual tasks.
# , then move on to costs attributable to the group as a whole.
@PP
If @M { t } is a must-assign task (night or day), we can ignore the
non-assignment and assignment costs of @M { t }.  This is because,
however @M { t } is grouped, we expect to assign a resource to
@M { t } in every solution, producing non-assignment and assignment
costs which are the same in every solution.
@PP
If @M { t } is a non-must-assign task (night or day), it may lie in
a compulsory group in some solutions and in an optional group in
others.  When it lies in a compulsory group @M { g }, we need to
add its assignment cost minus its non-assignment cost to @M { c(g) },
to reflect the fact that we have chosen a grouping which (we expect)
will cause it to be assigned. @B { (i) }
@PP
There is another, quite different kind of cost that can be
attributed to an individual task @M { t }.  Suppose @M { t }
is a non-must-assign task (night or day) lying in a compulsory
group.  This means that we expect to assign @M { t } even though,
when @M { t } is considered alone, we do not have to.  Now suppose
that the supply of resources falls short of or equals the demand
for resources made by must-assign tasks.  In that case, assigning
@M { t } adds to the amount by which resources are overloaded, so
for each time of @M { t }, we have to add the usual cost of
overloading a resource by one time.  @B { (ii) }
@PP
One way to handle supply and demand imbalances is to simply not assign
some must-assign tasks.  Reflecting this in our algorithm would involve
allowing a must-assign task @M { t } to appear alone in a group whose
cost is the cost of not assigning @M { t }.  This is not currently done.
# @PP
# Coincidentally, the last two sources of cost are associated with
# the same kinds of tasks:  non-must-assign tasks (night or day).
# The implementation can calculate them in advance of the solve
# and store them as a single cost associated with the task.
@PP
We move on now to costs associated with group @M { g } as a whole.
These all come from resource constraints, which can be divided into
@I { local constraints }, affected by what happens on a few adjacent
days, and @I { global constraints }, affected by what happens on many
(usually all) days.
@PP
Local constraints include unwanted pattern
constraints, complete weekends constraints, and so on.  They are
evaluated for @M { g } by function @C { KheTaskGrouperEntryCost }
(Section {@NumberOf resource_structural.task_grouping.task_grouper}).
@B { (iii) }
@PP
Global constraints include limits on total workload, limits on
consecutive busy shifts of a particular kind, limits on consecutive
busy days, and limits on consecutive free days.  Most of them are
irrelevant here (we assume that any limits on total or consecutive
busy days are not more restrictive that the limits on consecutive
night shifts).  The only relevant ones are those in our class @M { C },
and we include their costs.  @B { (iv) }
@PP
To summarize, the cost of a compulsory finished group @M { g } is
(i) the sum of the assignment minus non-assignment costs of its
non-must-assign tasks, plus (ii) the cost of overloading one resource
by one time for each time of each non-must-assign task (if needed),
plus (iii) the cost returned by @C { KheTaskGrouperEntryCost }
given @M { g }, plus (iv) any cost of violating @M { C }'s limits.
@PP
To find @M { c(g) } efficiently, for each included task we
calculate (i) plus (ii) at the start of the solve.  We can only
include these costs when @M { g } is finished and compulsory, but at
that point a quick scan of the tasks of @M { g } finds them.
Items (iii) and (iv) depend only on the set of times that
@M { g }'s tasks are running.  For each such set of times
that we come across, we calculate (iii) plus (iv) and store
it in a cache indexed by the set of times.  So after these
costs are calculated once, again a quick scan of the tasks
can be used to index the cache and retrieve the cost.
@PP
@BI { The algorithm, and dominance testing. }
The key building block of our algorithm is a move from a solution
@M { ( S sub {i-1} , G sub {i-1} ) } for the set of tasks
@M { S sub {i-1} = X sub 1 cup cdots cup X sub {i-1} }
to a solution @M { ( S sub i , G sub i ) } for
@M { S sub i = X sub 1 cup cdots cup X sub i }.  To make this move,
we need to assign each task of @M { X sub i } to a group:  either
to an existing unfinished group of @M { G sub {i-1} } containing
a task running at @M { T sub {i-1} } but not at @M { T sub i },
or to a new group.  Only the second option is open to tasks whose
placement is @I { start_only }.  After this, any unfinished groups
from @M { G sub {i-1} } that are not running during @M { T sub i } are
declared finished, as are all groups running during @M { T sub i }
that satisfy (b), (c), or (d) above.  To find all solutions, we do
this in all ways and for all @M { i }, starting from
@M { ( S sub 0 , G sub 0 ) }, the empty set of tasks and groups
which is the sole solution for no time groups.  Then a solution
for @M { ( S sub n , G sub n ) } of minimum cost is the result
we are seeking.
@PP
As described, this process is hopelessly exponential.  However,
in many cases, one solution for @M { S sub i } can be shown to
@I dominate another, meaning that for each complete solution
derived from the second solution, there is a complete solution
derived from the first solution whose cost is equal or less.  We can
drop the dominated solutions, except that when two solutions dominate
each other, we can only drop one of the two.  The algorithm, then,
is to build all solutions for @M { S sub 0 }, then all solutions for
@M { S sub 1 }, and so on, dropping dominated solutions at each stage.
We proceed in this breadth-first manner because dominance testing is
only practicable between pairs of solutions for the same set of tasks.
@PP
For example, after the last day all groups are finished and each
solution's cost is its final cost.  So after the last day
@M { T sub n }, solution @M { ( S sub n , G sub n ) } dominates
solution @M { ( S sub n , G prime tsub n ) } if
@M { c( S sub n , G sub n ) <= c( S sub n , G prime tsub n ) }.
(The only complete solution derived from @M { ( S sub n , G sub n ) } is
@M { ( S sub n , G sub n ) } itself.)  So dropping dominated solutions on
the last day leaves just one survivor, which is the result of the algorithm.
@PP
Testing whether solution @M { ( S sub i , G sub i ) } dominates
solution @M { ( S sub i , G prime tsub i ) } has two steps.  First,
for each solution we calculate a @I { signature }:  a summary of the
solution containing everything needed for testing dominance and nothing
else.  For solutions for @M { S sub n }, the signature is just the
cost, but in general it is more complicated, as we'll see; it must
include everything that determines how the solve can proceed from
here onwards.  Second, we compare signatures, giving an outcome
that indicates either that @M { ( S sub i , G sub i ) } dominates
@M { ( S sub i , G prime tsub i ) } or that it does not.  If
desired, a separate test can be used to decide whether
@M { ( S sub i , G prime tsub i ) } dominates @M { ( S sub i , G sub i ) }
or not.  So to complete the algorithm we need to define the signature
of each solution and say how signatures are compared.
@PP
Our dominance test does not have to be perfect.  If it declares that
one solution dominates another, that must be true, otherwise the
algorithm is in danger of missing the optimal solution.  But if it
declares that one solution does not dominate another, that need not
be true; if it is false, it just means that the algorithm keeps more
solutions than is strictly necessary, making its running time and
memory consumption larger than it needs to be.
@PP
The cost of a solution is an important part of its signature:  we
have seen that it is the entire signature on the last day.  It will be
part of the signature of every solution on every day.  Finished groups
have no effect on what happens on later days, so they can be and will
be left out of the signature, except that their cost is included in
the solution cost.  To a first approximation, then, the signature of a
solution @M { ( S sub i , G sub i ) } is @M { c( S sub i , G sub i ) }
plus the unfinished groups of @M { G sub i }.
@PP
For example, if two solutions have identical unfinished groups, it
is clear that they can only develop in identical ways, and so only
one needs to be kept---one whose cost is less than or equal to the
other's.  But to require unfinished groups to be identical is to
give up too many cases of dominance.  Instead, for solution
@M { ( S sub i , G sub i ) } to dominate solution
@M { ( S sub i , G prime tsub i ) }, we require:
@BulletList

@LI {
@M { c( S sub i , G sub i ) <= c( S sub i , G prime tsub i ) };
}

@LI {
@M { G sub i } and @M { G prime tsub i } have the same
number of unfinished groups;
}

@LI {
The unfinished groups of @M { G prime tsub i } can be permuted so
that for each pair of corresponding unfinished groups @M { g } from
@M { G sub i } and @M { g prime } from @M { G prime tsub i },
@M { g } dominates @M { g prime }.
}

@EndList
We discuss what it means for one group to dominate another below.
Notice the `imperfection' in this definition:  we have given away
any prospect of declaring dominance when the number of unfinished
groups differs.  This is basically because anything more is
impractical.  There is often a one-to-one correspondence between
unfinished groups in @M { G sub i } and tasks in @M { X sub i },
however, so fewer cases of dominance will be missed for this reason
than might at first seem likely.  Similarly, we will be trying just
one permutation of the unfinished groups of @M { G prime tsub i },
and if we try the wrong one, we will miss a case of dominance.
This source of imperfection will turn out to be quite likely.
@PP
It remains to determine the signature of an unfinished group
@M { g }, and how two group signatures can be compared for
dominance.  The idea is that the tasks of the two groups, @M { g }
and @M { g prime }, do not have to be identical, but nevertheless
@M { g } and @M { g prime } have to be similar enough to ensure
that any way of extending @M { g prime } into a finished group
@M { g prime cup Q } makes sense for @M { g } as well, and that the
resulting costs satisfy @M { c(g cup Q) <= c( g prime cup Q) }.
@PP
Here are the conditions that make up the dominance test between
groups, with reasons for each.  By examining these conditions we
can determine which properties of each group @M { g } need to be
included in the signature.  The first three conditions derive from
the conditions that permit a task to be added to a group
(Section {@NumberOf resource_structural.task_grouping.task_grouper});
they are needed so that we can assert that, for any @M { Q }, if
@M { g prime cup Q } is a legal group, then so is @M { g cup Q }.
The last three relate to cost.
@NumberedList

@LI {
Let @M { d(g) } be the intersection of the domains of the tasks
of @M { g }.  Then we must have @M { d(g) supseteq d( g prime ) },
so that for any additional tasks @M { Q }, if @M { g prime cup Q }
is a legal group, then so is @M { g cup Q }.
}

@LI {
If any of the tasks of @M { g } are assigned a resource @M { r },
let @M { a(g) } be that resource, otherwise let @M { a(g) } be
@C { NULL }.  Then we must have @M { a(g) = a( g prime ) },
since obviously an assignment has a strong effect on how
a group can grow.
}

@LI {
Let @M { f(g) } be true when @M { g } has a fixed non-assignment.
Then we must have @M { f(g) = f( g prime ) }.
}

@LI {
The tasks of @M { g } must be running at exactly the same times
as the tasks of @M { g prime }.  Without this, we would have no
way of knowing what @C { KheTaskGrouperEntryCost } returns for
the two groups.  With it, we know that it returns the same cost
for both, because that cost is derived entirely from cluster
busy times and limit busy times constraints, and so it depends
only on the times that assigning a resource to the group causes
the resource to be running.  This also handles the constraints of
@M { C }, which also depend entirely on those times.
}

@LI {
Let @M { o(g) } be true when @M { g } is an optional group.
Then we must have @M { o( g prime ) ==> o(g) }.  Otherwise
@M { g prime cup Q } could be optional with cost 0,
when @M { g cup Q } is compulsory with non-zero cost.
}

@LI {
Let @M { e(g) } be the sum of costs (i) and (ii) documented above
for non-must-assign tasks in @M { g }.  These will become part of
the solution cost if @M { g } turns out to be a compulsory group
when it is extended into @M { g cup Q }, so we must have
@M { e(g) <= e( g prime ) }.
}

@EndList
Clearly all these properties must be included, for the reasons
given.  We have not proven that these properties are sufficient
to ensure dominance, but an audit of the conditions which
determine whether a set of tasks can form a group, plus
the costs that go to make up the cost of a finished group,
will show that nothing has been left out.
@PP
In practice, most groups are unassigned compulsory night groups
with no non-must-assign or fixed tasks.  For them, only the first
and fourth conditions can fail to hold, the latter only when the
two groups have different durations.  This is important for the
time complexity analysis.
@PP
We still need to explain how the finished groups are permuted.  We
do this by sorting them lexicographically by the properties just
given.  As a first priority they are sorted by increasing value
of the sequence of busy times; then when those are equal, they are
sorted by increasing domain size; then when those are equal, they
are sorted by increasing @M { e(g) }; and so on.  This is not
perfect, but as we have explained above, we do not need perfection.
@PP
Finally, we need to consider how the dominance test can be
implemented efficiently.  The information needed by points (1),
(2), (3), and (5) is held in the object that defines the group, so
there are no problems there.  Points (4) and (6) can be handled
by scanning the unfinished group in the same ways as described
above for finding the cost of a finished compulsory group.
# is quite fast given that unfinished groups contain at most
# 4 elements in our example.  Function @C { KheTaskGrouperEntryCost }
# is slow, and so the author has installed a cache indexed by
# a set of times which returns a cost.  This can be used when
# calculating the cost of a finished group; it holds the whole
# cost except for the part called @M { e(g) } above.  That part
# has one component for each task; that component can be and is
# calculated just once for each task and stored with the task.
# @PP
# a set of 4-tuples
# @M { langle c(g), o(g), a(g), d(g) rangle }, one for each unfinished
# group @M { g }:
# @NumberedList
# 
# @LI @OneRow {
# The night duration of @M { g }, denoted @M { c(g) }.
# # If @M { g }
# # is assigned a resource @M { r } and covers the first time group,
# # then @M { c (g) } is increased by the history value of @M { r }.
# # See below for more on this.
# }
# 
# @LI @OneRow {
# The @I { overhang } of @M { g }, denoted @M { o(g) }, which is
# that part of the full duration which lies strictly to the right
# of @M { T sub i }.
# # It satisfies @M { 0 <= o(g) < l(g) }.
# }
# 
# @LI @OneRow {
# The @I { assignment } of @M { g }, denoted @M { a(g) }, a
# resource.  If at least one task of @M { g } is assigned a
# resource, then @M { a(g) } is that resource.  If no tasks are
# assigned a resource, then @M { a(g) } is @C { NULL }.
# }
# 
# @LI @OneRow {
# The @I { domain } of @M { g }, denoted @M { d(g) }, which is
# the set of resources that could be assigned to @M { g }.  It
# is the intersection of the domains of the tasks of @M { g },
# as computed by the task grouper.
# }
# 
# @EndList
# The order of the 4-tuples does not matter.  The set is a multiset:
# if there are two unfinished groups with the same 4-tuple, that
# 4-tuple appears twice.  Placements do not matter:  if @M { g }
# contains a @I { first_only } task, that is already taken care
# of and does not affect the future; while if @M { g } contains
# a @I { last_only } task, then @M { g } is finished and is not
# in the signature.
# @PP
# @M { G sub i } dominates @M { G prime tsub i } when
# @M { c( G sub i ) <= c( G prime tsub i ) } and the 4-tuples in the two
# sets are equal in number and can be permuted so that for corresponding
# groups @M { g } and @M { g prime } we have @M { l(g) = l( g prime ) },
# @M { o(g) = o( g prime ) }, @M { a(g) = a( g prime ) }, and
# @M { d(g) supseteq d( g prime ) }.  The permuting is done by sorting
# the 4-tuples by decreasing overhang, then increasing assigned
# resource index (or @M { minus 1 } if unassigned), then increasing
# night duration, then increasing domain size, and then comparing
# corresponding 4-tuples.  This is not perfect (it may miss some
# cases of dominance), but it is good enough.
# @PP
# At the end, in @M { G sub n }, every group is finished, as we
# explained above, so there are no 4-tuples and dominance depends
# only on cost.  There will therefore be just one undominated
# solution, and that is the solution of minimum cost that we
# are seeking.
@PP
@BI { Grouping tasks with similar domains. }
At first sight, the algorithm does not seem to include anything
which favours grouping tasks with similar domains.  However, the
dominance test does favour such groups, as we now show.
@PP
Suppose that there are two groups, @M { g sub 1 } and
@M { g sub 2 }, in some solution, and that their domains satisfy
@M { d( g sub 1 ) subseteq d( g sub 2 ) }.  Suppose that there
are two tasks @M { t sub 1 } and @M { t sub 2 } running on the
next day, and that their domains happen to be
@M { d( t sub 1 ) = d( g sub 1 ) } and 
@M { d( t sub 2 ) = d( g sub 2 ) }.  If we group @M { t sub 1 }
with @M { g sub 1 } and @M { t sub 2 } with @M { g sub 2 }, the
domains of the new groups will be @M { d( g sub 1 ) } and
@M { d( g sub 2 ) }.  If we group @M { t sub 1 } with @M { g sub 2 }
and @M { t sub 2 } with @M { g sub 1 }, their domains will both
be @M { d( g sub 1 ) cap d( g sub 2 ) = d( g sub 1 ) }.  But
@M { d( g sub 1 ) subseteq d( g sub 2 ) }, so this second solution
is dominated by the first.
@PP
Nevertheless there are occasional points in the optimal solution where
swapping two tasks which run at the same times, that is, moving each
to the other's group, enlarges the domain of one of the two groups
without shrinking the other.  The author has not fully analysed this
situation.  However, when two solutions dominate each other (when
they have the same cost and the same signature), he has added a
further test.  Each solution keeps track of the number of its
complete groups whose tasks do not all have the same domain.  The
further test favours retaining solutions for which this number
is smaller.  This seems to have fixed the immediate problem.
# Avoiding these cases does not seem to
# be feasible.  So after finding the optimal solution, the algorithm
# checks all pairs of unassigned tasks running at the same times, and
# tries swapping them, keeping the swap if it enlarges domains.
@PP
@BI { Adding randomness. }
When two solutions dominate each other (when they have the same
signature, including the same number of complete groups whose
tasks do not all have the same domain), there is an opportunity
to add randomness.  We do this by finding a random integer for
each solution based on the total number of solutions that have
been generated so far, plus the KHE solution's diversifier.  If
all else is equal, a solution with a smaller (or equal) random
integer is considered to dominate.  In practice this seems to
produce fairly different solutions.
@PP
@BI { Assigned tasks and history. }
Some included tasks may be assigned a resource initially.  Such
tasks are handled correctly, without changing their assignments.
This is needed in practice to preserve the results of assign by
history.
@PP
A task @M { t } with an assigned resource @M { r } may be grouped
with a task assigned @M { r }, or with a task assigned no resource.
(The resulting group will then be assigned @M { r }.)  But @M { t }
may not be grouped with a task assigned some other resource, nor
with an unassigned task running on the same day as some other task
assigned @M { r }.  Because these rules are enforced by the task
grouper (the last is the no-interference rule) and interval grouping
uses the task grouper to build its groups, interval grouping does
not have to worry about them.
@PP
Any group @M { g } with an assigned resource which covers the first
time group of @M { C } has its night duration increased
by the history value of the resource.  For example, suppose that we
are grouping night shifts with minimum limit 4 and maximum limit 5.
Suppose that resource @M { r } with history value 2 has been assigned
two night shifts at the start of the cycle by assign by history.
These two shifts will be grouped by interval grouping (because any
other grouping fails the no-interference rule), but furthermore,
this group will be assigned night duration 4, to count in the 2 from
history.  This means that the dynamic programming algorithm will
be free to either leave it as is or to add one task, making
duration 5, but not more.  Which is just what is needed.
@PP
@BI { Taking advantage of symmetries among task groups and tasks. }
Consider the basic @I expansion step of the algorithm, in which
we are given a solution @M { ( S sub {i-1} , G sub {i-1} ) } and a
set of tasks @M { X sub i } starting on day @M { i } that we need to
add to @M { ( S sub {i-1} , G sub {i-1} ) } in all possible ways.
Let @M { U sub {i-1} } be the unfinished groups of @M { G sub {i-1} }.
Some of the tasks of @M { X sub i } will be added to some of the
groups of @M { U sub {i-1} }; the other tasks of @M { X sub i } will
start new groups.
@PP
Even when dominance testing is working well, there may still be many
solutions requiring expansion.  Expanding each of them in all
possible ways, as just described, may produce thousands of solutions
needing to be tested for dominance, leading to slow run times.
@PP
Time can be saved by doing as much work as possible outside of the
recursive generation of these solutions.  For example, for
each unfinished group @M { g in U sub {i-1} } and each task
@M { x in X sub i }, we can decide whether @M { x } can be added
to @M { g } before the recursion starts, and record it so that
we don't need to decide it over and over again during the
recursion.  But this does not address the basic problem of
thousands of solutions being generated, only to then be deleted
by dominance testing.
@PP
A more radical way to save time is to detect equivalent task
groups and tasks (we'll define equivalence later, but informally
we mean interchangeable), and avoid generating solutions which
just permute these interchangeable objects among themselves.  Let
@M { g sub 1 } and @M { g sub 2 } be two distinct unfinished
groups from @M { G sub {i-1} }, and let @M { x sub 1 } and
@M { x sub 2 } be two distinct tasks from @M { X sub i }.
If @M { g sub 1 } and @M { g sub 2 } are equivalent, or if
@M { x sub 1 } and @M { x sub 2 } are equivalent (note this
is `or', not `and'), then any solution which includes
@M { g sub 1 + x sub 1 } and @M { g sub 2 + x sub 2 } is
equivalent to the same solution with these two groups replaced
by @M { g sub 1 + x sub 2 } and @M { g sub 2 + x sub 1 }.  So
only one of these two solutions needs to be generated.  Here we
write @M { g + x } for the task group @M { g cup lbrace x rbrace }.
We realize these savings by grouping the task groups of
@M { U sub {i-1} } into equivalence classes before expansion
begins, and grouping the tasks of @M { X sub i } into equivalence
classes before solving begins, and recursing over task group
classes and task classes, rather than over task groups and tasks.
@PP
It remains to define what it means for two task groups @M { g sub 1 }
and @M { g sub 2 } to be equivalent.  (We can re-use this definition
for tasks, by considering each task to be a task group with a single
element.)  Clearly, the two groups are equivalent if they are equal with
respect to every property that influences cost or dominance testing.
Here is the full list of these properties for task group @M { g }:
@NumberedList

@LI @OneRow {
The domain @M { d(g) }.
}

@LI @OneRow {
The set of times @M { T(g) } that the tasks of @M { g } are running,
including history times (in fact just an integer) when relevant.
}

@LI @OneRow {
The resource assignment @M { a(g) }, which will be @C { NULL }
if @C { g } is not assigned.
}

@LI @OneRow {
The fixed non-assignment @M { f(g) }, a Boolean value which is
true when @M { g } contains an unassigned task whose assignment
is fixed.
}

@LI @OneRow {
The optionality @M { o(g) }, a Boolean value which is true
when each task of @M { g } is optional (that is, either a
non-must-assign task or not a night task).
}

@LI @OneRow {
The non-must-assign cost @M { e(g) } as defined above.
}

@EndList
We classify these properties into three groups:  the @I { domain },
the @I { times }, and the @I { other properties }.
@PP
@BI { Optimizing the algorithm. }
Although the algorithm just given will work as is, it can be slow.
We'll now identify some cases where we can save time.
@PP
We are given a solution @M { ( S sub {i-1} , G sub {i-1} ) } and a
set of tasks @M { X sub i } starting on day @M { i } that we need to
add in in all possible ways, to obtain each @M { ( S sub i , G sub i ) }.
Let @M { U sub {i-1} } be the unfinished groups of @M { G sub {i-1} }.
Some of the tasks of @M { X sub i } will be added to some of the
groups of @M { U sub {i-1} }; the other tasks of @M { X sub i } will
start new groups.  This matching up of the groups @M { U sub {i-1} }
and tasks @M { X sub i } can be split into four cases:
@NumberedList

@LI {
@I { Overhangs case. }
For each group @M { u in U sub {i-1} } with an overhang, we
must include @M { u } in @M { ( S sub i , G sub i ) }.
}

@LI {
@I { Continuing assignments case. }
For each group @M { u in U sub {i-1} } not handled by (1)
and @M { x in X sub i } which are assigned the same resource,
we must add @M { x } to @M { u }.
}

@LI {
@I { Undersized groups case. }
For each undersized @M { u in U sub {i-1} } not handled
by cases (1) and (2), we can insist that @M { u } receive some
@M { x in X sub i } whenever a suitable @M { x } can be found.
This is quite different from the main algorithm, which insists
that each @M { x } be included.  It implies that avoiding
undersized groups is the highest priority, which may not be
strictly true (its priority depends on the cost of undersized
groups), but which we take to be true in practice.
}

@LI {
@I { Default case. }
This covers all @M { u } and @M { x } which are not
covered by cases (1), (2), and (3).
}

@EndList
For a given @M { ( S sub {i-1} , G sub {i-1} ) } and @M { X sub i },
we first handle cases (1) and (2), producing a single partial
solution for @M {  S sub i }.  We then recursively assign a task to
each undersized group not handled by (1) and (2) in each possible way.
For each partial solution thus obtained, we then recursively assign
each task not handled by cases (1), (2), and (3) to either an existing
unhandled group or to a new group.  This saves time by not generating
solutions in which undersized groups are not assigned tasks.
@PP
@BI { Early dominance testing. }  Let @M { d(g) } be the domain of
task group @M { g }, and let @M { d(x) } be the domain of task
@M { x }.  Suppose that we have two distinct tasks, @M { x sub 1 }
and @M { x sub 2 }, which have the same properties except that
their domains are different (this case will be very common).  Let
@M { g sub 1 } and @M { g sub 2 } be any two distinct task groups.
Write @M { g + x } for the task group @M { g cup lbrace x rbrace }.
Suppose that
@ID @Math {
d( g sub 1 + x sub 1 ) supseteq d( g sub 1 + x sub 2 )
@B "  and  "
d( g sub 2 + x sub 2 ) supseteq d( g sub 2 + x sub 1 )
}
where at least one of the superset relations is proper.  Then any
solution in which @M { x sub 2 } is grouped with @M { g sub 1 } and
@M { x sub 1 } is grouped with @M { g sub 2 } will be dominated
by the solution which is the same except that @M { x sub 2 } is
grouped with @M { g sub 2 } and @M { x sub 1 } is grouped with
@M { g sub 1 }.
@PP
The dominated solutions will be eliminated by dominance testing as
usual, but we now wish to avoid generating them at all.  For each
case like the one just given, we can halve the number of solutions
generated.
@PP
Another common case is when @M { x sub 1 } and @M { x sub 2 } have
the same properties except for their domains and non-assignment
costs.  We can draw the same conclusion, provided that the
four task groups (@M { g sub 1 + x sub 1 }, @M { g sub 1 + x sub 2 },
@M { g sub 2 + x sub 1 }, and @M { g sub 2 + x sub 2 }) are not
optional, since in that case the non-assigment costs will not be used.
Writing @M { n(g) } for `@M { g } is not optional' and
@M { n(x) } for `@M { x } is not optional', we find
@IndentedList

@LI @Math {
  n( g sub 1 + x sub 1 ) wedge n( g sub 1 + x sub 2 ) wedge
  n( g sub 2 + x sub 1 ) wedge n( g sub 2 + x sub 2 )
}

@LI @Math { &4s =
  [ n( g sub 1 ) vee n( x sub 1 ) ] wedge
  [ n( g sub 1 ) vee n( x sub 2 ) ] wedge
  [ n( g sub 2 ) vee n( x sub 1 ) ] wedge
  [ n( g sub 2 ) vee n( x sub 2 ) ]
}

@LI @Math { &4s =
  [ n( g sub 1 ) vee [ n( x sub 1 ) wedge n( x sub 2 ) ] ] wedge
  [ n( g sub 2 ) vee [ n( x sub 1 ) wedge n( x sub 2 ) ] ]
}

@LI @Math { &4s =
  [ n( g sub 1 ) wedge n( g sub 2 ) ] vee [ n( x sub 1 ) wedge n( x sub 2 ) ]
}

@EndList
That is, either both groups are not optional, or both tasks are not
optional.  In summary, we can avoid generating solutions for each
@M { g sub 1 }, @M { g sub 2 }, @M { x sub 1 }, and @M { x sub 2 }
such that the superset condition above holds, and @M { x sub 1 }
and @M { x sub 2 } have the same properties except for domains and
non-assignment costs; although when their non-assignment costs differ
we also need
@M { [n( g sub 1 ) wedge n( g sub 2 )] vee [n( x sub 1 ) wedge n( x sub 2 )] }
to hold.
@PP
We can uncover these cases before beginning to add the tasks of
@M { X sub i } to the unfinished groups of @M { G sub {i-1} } in
all possible ways, and we can implement the avoidance by marking
the link between @M { g sub 2 } and @M { x sub 1 } while a new
group @M { g sub 1 + x sub 2 } is being tried, and marking the
link between @M { g sub 1 } and @M { x sub 2 } while a new group
@M { g sub 2 + x sub 1 } is being tried, and declining
to build groups whose links are marked.
@PP
We can allow @M { g sub 1 } or @M { g sub 2 } to be the empty group,
corresponding to the case where @M { x sub 1 } or @M { x sub 2 }
begins a new group.  If @M { g sub 1 } is the empty group, in effect
its domain is the set of all resources, and the superset condition
becomes
@ID @Math {
d( x sub 1 ) supseteq d( x sub 2 )
@B "  and  "
d( g sub 2 + x sub 2 ) supseteq d( g sub 2 + x sub 1 )
}
where at least one of the superset relations is proper.  Since the
optionality of the empty group is indeterminate, when non-assignment
costs differ we must have @M { n( x sub 1 ) wedge n( x sub 2 ) }.
# @PP
# @BI { Early dominance testing. }  Suppose we have a group @M { g sub 1 }
# whose domain @M { d( g sub 1 ) } is the set @M { D sub 1 } of senior
# nurses, and another group @M { g sub 2 } whose domain @M { d ( g sub 2 ) }
# is the set @M { D sub 2 } of ordinary nurses.  Some nurses may lie in
# both domains.  Suppose we have two tasks that could be added to these
# two groups:  @M { x sub 1 } with domain @M { d( x sub 1 ) = D sub 1 },
# and @M { x sub 2 } with domain @M { d( x sub 2 ) = D sub 2 }.  Suppose
# the @I { other properties } that affect dominance (assignments, fixed
# non-assignments, times, and optionality) are the same for both groups
# and for both tasks (this will usually be the case).  Then solutions in
# which @M { x sub 1 } is added to @M { g sub 1 } and @M { x sub 2 } is
# added to @M { g sub 2 } must dominate solutions in which @M { x sub 1 }
# is added to @M { g sub 2 } and @M { x sub 2 } is added to @M { g sub 1 }.
# This is because @M { d( g sub 1 + x sub 1 ) = D sub 1 },
# @M { d( g sub 2 + x sub 2 ) = D sub 2 }, and
# @M {d( g sub 2 + x sub 1 ) = d( g sub 1 + x sub 2 ) = D sub 1 cap D sub 2}.
# We write @M { g sub i + x sub j } as shorthand for
# @M { g sub i cup lbrace x sub j rbrace }, and we use the fact
# that @M { d( g sub i + x sub j ) = d( g sub i ) cap d( x sub j ) }.  We
# don't include non-must-assign costs in the other properties, despite the
# fact that they affect dominance, because they will be the same either way.
# @PP
# The dominated solutions will be eliminated by dominance testing as
# usual, but we now wish to avoid generating them at all.  For each
# case like the one just given, we can halve the number of solutions
# generated.
# @PP
# Starting from a given @M { ( S sub {i-1} , G sub {i-1} ) } and
# @M { X sub i }, we look for pairs
# @M { ( lbrace g sub 1 , g sub 2 rbrace , lbrace x sub 1 , x sub 2 rbrace ) }
# such that @M { g sub 1 } and @M { g sub 2 } are distinct unfinished
# groups from @M { G sub {i-1} } with equal other properties, @M { x sub 1 }
# and @M { x sub 2 } are distinct tasks from @M { X sub i } with equal
# other properties, and the following condition holds:
# @ID @Math {
# d( g sub 1 + x sub 1 ) supseteq d( g sub 1 + x sub 2 )
# @B "  and  "
# d( g sub 2 + x sub 2 ) supseteq d( g sub 2 + x sub 1 )
# }
# where at least one of the superset relations is proper.  Then we
# can safely avoid generating solutions which contain both of the
# new groups @M { g sub 1 + x sub 2 } and @M { g sub 2 + x sub 1 }.
# We can uncover these cases before beginning to add the tasks of
# @M { X sub i } to the unfinished groups of @M { G sub {i-1} } in
# all possible ways, and we can implement the avoidance by marking
# the link between @M { g sub 2 } and @M { x sub 1 } while a new
# group @M { g sub 1 + x sub 2 } is being tried, and marking the
# link between @M { g sub 1 } and @M { x sub 2 } while a new group
# @M { g sub 2 + x sub 1 } is being tried, and declining
# to build groups whose links are marked.
# @PP
# Some thought will show that we can allow @M { g sub 1 } or
# @M { g sub 2 } to be the empty group, corresponding to
# the case where @M { x sub 1 } or @M { x sub 2 } begins a
# new group.  If @M { g sub 1 } is the empty group, in effect
# its domain is the set of all resources, and our condition becomes
# @ID @Math {
# d( x sub 1 ) supseteq d( x sub 2 )
# @B "  and  "
# d( g sub 2 + x sub 2 ) supseteq d( g sub 2 + x sub 1 )
# }
# where at least one of the superset relations is proper.
######################
# our plan is to handle cases (1), (2), and (3), producing a single
# partial solution for @M {  S sub i }, and then, omitting the groups
# and tasks handled by these cases, to run the main algorithm to
# handle case (4) and complete @M { ( S sub i , G sub i ) } in all
# possible ways.  Cases (1) and (2) are simple fixed assignments, so we
# are left with case (3).
# @PP
# In case (3), each undersized @M { u in U sub {i-1} } not handled by
# cases (1) and (2) must be matched with an @M { x in X sub i }.
# We use weighted bipartite matching to do this.
# We draw an edge from a @M { u } to an @M { x } whenever @M { x }
# can be added to @M { u } without violating the usual conditions, and
# we assign a weight to encourage good matches to be made (see below).
# Then we use a weighted bipartite matching to assign tasks to groups.
# Any unmatched groups and tasks go back into the pot for case (4).
# @PP
# For each edge @M { (u, x) }, the weight is the negative of the size
# of the intersection of the domains of @M { u } and @M { x }.  This
# encourages matches between groups and tasks with similar domains.
# @PP
# In practice it is fair to assume that each undersized @M { u }
# must be assigned an @M { x }.  However, there are many
# alternative matchings and it is possible that using just
# one of them, as we do here, could cause the optimal solution to
# be missed.  We choose to wear the risk of this for the sake
# of the considerable time saving we can make by using the matching.
# @PP
# One thing that we can do about this optimality issue is to avoid
# including unusual groups and tasks in the matching.  Leaving them
# out means that they then get handled with full generality by case
# (4).  So we define a group or task to be @I ordinary when it is
# not optional, not assigned a resource, and not fixed.  Only ordinary
# groups and tasks are included in case (3).
@PP
# @BI { A restriction. }
# For efficiency, the author has imposed a rule which restricts the
# search space.  This could cause the optimal solution to be missed,
# although that is unlikely.  Here is the rule:
# @ID @OneRow @I {
# Suppose that solution @M { S } contains two groups, @M { g sub 1 }
# and @M { g sub 2 }, such that the night duration of @M { g sub 1 }
# is less than the minimum limit, @M { g sub 2 } starts immediately
# after @M { g sub 1 } ends, and moving the first task @M { t } of
# @M { g sub 2 ` } from the start of @M { g sub 2 } to the end of
# @M { g sub 1 } is legal.  (That is, the move would not leave
# any task with a wrong placement, @M { t }'s domain is compatible
# with @M { g sub 1 }'s domain, and the move would not cause the
# night duration of @M { g sub 1 } to exceed the maximum
# limit.)  Then @M { S } is excluded from the search space.
# }
# Expressed less formally, we give preference to growing undersized
# groups over starting new groups.  For example, suppose groups of
# night duration 4 or 5 are wanted, and there is one task (of class
# duration 1) available to be grouped at each of six consecutive
# time groups.  Then one possible solution, grouping the first task
# with itself and the other five with each other, is ruled out, as
# is grouping the first two and the last four, and the first three
# and the last three.  In these cases the first task of the second
# group could have been included in the undersized first group.  The
# point of this restriction is that it greatly reduces the size of
# the search space, but is unlikely to cause the algorithm to miss
# the optimal solution, given that undersized groups have a cost.
# (Yes, there is a problem here when the cost function of the limit
# active intervals constraint is quadratic.)
# # @PP
# # There are cases where using a task to extend an undersized group
# # turns out to be impossible owing to incompatible domains.  As a
# # patch for this problem, whenever a task is not able to be added
# # to any existing group, it is always allowable to use it to start
# # a new group.
# @PP
@BI { Including lengthener tasks. }
The algorithm never produces a group whose night duration exceeds
@M { C sub "max" }, except for groups that are unavoidable because
they contain just one task, or because all their tasks are assigned
the same resource.  But it does produce groups whose night duration
falls short of @M { C sub "min" }, when it cannot avoid it.
@PP
In good solutions, these short sequences are often extended by
non-must-assign night tasks.  But we can't include all such tasks
in the solve, because the algorithm might become swamped by these
tasks and run too slowly.
@PP
So we do this.  First, we run the algorithm without including
non-must-assign night tasks.  Then, for each undersized group
in the solution we identify up to two non-must-assign night tasks
that could be used to lengthen that group, one just before it
and one just after it.  We then run the algorithm a second time,
adding these @I { lengthener tasks } to the previously included tasks.
# @PP
# There has to be a cost associated with choosing an optional task,
# otherwise the best solution will have more of them than it
# needs to.  At present the algorithm is using the cost returned
# by @C { KheBalanceSolverMarginalCost }
# (Section {@NumberOf resource_structural.supply_and_demand.balance})
# multiplied by the duration of the task.
# @PP
# As mentioned earlier, a group (of any duration) containing only
# optional tasks is assigned cost 0, since although it must be
# considered to be a group (because of our rule that every admissible
# task ends up in exactly one group), there is no need for any resource
# to be assigned to it, and so its existence does not foreshadow any
# cost in actual solutions.
@PP
On one run, including lengthener tasks reduced the total duration
of undersized groups from 15 to 4, while introducing 10
non-must-assign night tasks grouped with compulsory tasks.
@PP
@BI { Time complexity. }
The key to finding the time complexity of this algorithm is to
estimate the number of undominated @M { G sub i } for each @M { i }.
We ignore optional tasks, multi-day tasks, domains, and assignments,
because apart from domains, few tasks differ in these properties,
and there are likely to be only a few distinct domains.  Suppose
@M { G sub i } has @M { n } unfinished groups, each of which has
night duration in the range @M { [1, K] }.
Then the signature of each undominated @M { G sub i } is nothing
other than a distinct multiset of night durations, since given
two solutions with equal night durations, one always dominates
the other.  So the number of undominated solutions is at most
@M { p(n, K) }, the number of distinct multisets of cardinality
@M { n } whose elements are integers in the range @M { [1, K] }.
For example, if @M { n = 4 } and @M { K = 3 } there are 15 of
these multisets:
@CD @OneRow @F lines @Break {
3 3 3 3  &2c  3 3 1 1  &2c  2 2 2 2
3 3 3 2  &2c  3 2 2 2  &2c  2 2 2 1
3 3 3 1  &2c  3 2 2 1  &2c  2 2 1 1
3 3 2 2  &2c  3 2 1 1  &2c  2 1 1 1
3 3 2 1  &2c  3 1 1 1  &2c  1 1 1 1
}
In general, we can argue as follows.  Divide these multisets into
two parts.  In the first part place those multisets that contain
at least one @M { K }.  This fixes one of the values in the multiset
but places no new constraints on the others.  So the number of such
multisets is @M { p(n-1, K) }.  In the second part place those
multisets that do not contain at least one @M { K }.  There are
@M { p(n, K-1) } of those.  So
@ID @M { p(n, K) ``=`` p(n-1, K) ``+`` p(n, K-1) }
We can have @M { n = 0 }, but the smallest valid @M { K } is @M { 1 },
so the bases of this recurrence are @M { p(n, 1) = 1 }, a multiset
containing just one element (a sequence of @M { n } ones), and
@M { p(0, K) = 1 }, a multiset containing just one element (the empty
sequence).  Although the base is not the usual one, the recurrence
is familiar and tells us that @M { p(n, K) } is a combination:
@ID @M {
p(n, K) `` = `` pmatrix { row col n + K-1 row col {K-1} }
}
For example, @M { p(4, 3) = "-2p" @Font pmatrix { row col 6 row col 2 } = 15 }.
For a fixed @M { K } this is polynomial in @M { n }, of order @M { n sup K }.
# @PP
# If we elected to not sort the sets, each of the @M { n } elements
# would have a value in the range @M { [1, K] } independently of the
# others, making @M { K sup n } distinct sequences altogether.  This
# is exponential in @M { n }, and larger in practice.  For example,
# @M { 3 sup 4 = 81 }.
@PP
In instance INRC2-4-100-0-1108, the maximum night duration of an
unfinished group is @M { K = 4 }, and there are at most about 25
nurses on the night shift.  So the value that interests us is
@M { p(25, 4) = "-2p" @Font pmatrix { row col @R "28" row col 3 } = 3276 }.
This is a manageable number.  In practice, our preference for
extending undersized groups rather than starting new ones should
reduce it considerably.
